metadata

license: mit
language:
  - ar
metrics:
  - accuracy
pipeline_tag: summarization
library_name: PyTorch
tags:
  - PyTorch
  - Arabic
  - Abstractive-Summarization
  - 174M
  - Scratch
  - Base

Arab Bart

Implemented the BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension paper from scratch using PyTorch for an abstractive summarization task in Arabic.

Goal

Reproduce the BART model from scratch to understand its architecture in depth, using the minimum available resources.

Size

The model size: 174M parameters.

Task

Abstractive Summarization in Arabic.

Data

The dataset used is the XL-Sum(Arabic Subset) dataset. I chose this dataset because it's well-suited for our task. Additionally, it's written in pure Arabic, which makes it the best choice. The original source: BBC Arabic.

Features (columns):
- text: the full text (source sequences).
- summary: the summary of the text (target sequences).
Size:
- train: 32,473 rows.
- validation: 4689 rows.
- test: 4689 rows.

Summary

Results

Epoch	Loss(train)	Loss(validation)	Epoch Time (hours)	Training Time (hours)	Device
1	10.03	9.72	0.23	1.1	1 x L4OS
2	9.61	9.44	0.22	1.1	1 x L4OS
3	9.36	9.22	0.22	1.1	1 x L4OS
4	9.16	9.05	0.22	1.1	1 x L4OS
5	9.01	8.92	0.22	1.1	1 x L4OS

Usage

License

This model is licensed under the MIT License.