File size: 1,264 Bytes
e353eb5 3bd863e e353eb5 3bd863e e353eb5 3bd863e e353eb5 f7970c6 e353eb5 f7970c6 e353eb5 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 |
---
language: ti
license: mit
library_name: transformers
tags:
- tigrinya
- gpt2
- text-generation
metrics:
- perplexity
- loss
pipeline_tag: text-generation
---
# Model Card for GPT-2 Tigrinya Medium
## Model Summary
This is a GPT-2 model trained from scratch on Tigrinya text data. It was trained on 20.6 million tokens, primarily from news sources.
#### Model Description
- Model type: GPT-2
- Language: Tigrinya (ትግርኛ)
- Finetuned from model: Trained from scratch (no pre-training)
#### Model Architecture
- Parameters: 51.9M
- Context Window: 128 tokens
- Vocabulary Size: 52,000
#### Training Details
- Training regime: fp16 mixed precision
- Number of Epochs: 12
- Batch Size: 6 (with gradient accumulation steps of 8)
- Learning Rate: 5e-4
#### Evaluation
- Training Perplexity: 28.6
- Training Loss: 3.12
#### Usage
```python
from transformers import pipeline
# Load the model
generator = pipeline('text-generation', model='luel/gpt2-tigrinya-medium')
prompt = "ክልል ትግራይ"
# Generate text
text = generator(prompt, max_length=100)[0]['generated_text']
print(text)
```
#### Limitations
- Limited context window of 128 tokens.
- Best suited for medium-length Tigrinya text generation.
- Outputs should be reviewed for accuracy. |