|
--- |
|
base_model: |
|
- google/mt5-small |
|
datasets: |
|
- syubraj/roman2nepali-transliteration |
|
language: |
|
- ne |
|
- en |
|
library_name: transformers |
|
license: apache-2.0 |
|
pipeline_tag: translation |
|
tags: |
|
- nepali |
|
- roman english |
|
- translation |
|
- transliteration |
|
--- |
|
|
|
# Model Card for Model ID |
|
|
|
Due to compute issues, The model has been trained on multiple iterations: |
|
|
|
1. Model Trained for 8500 steps on 5% of the dataset. |
|
2. Model continued from 8500 steps on 5%:20% of the dataset |
|
|
|
|
|
|
|
|
|
|
|
### Model Description |
|
|
|
<!-- Provide a longer summary of what this model is. --> |
|
|
|
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated. |
|
|
|
- **Model type:** (Translation) |
|
- **Language(s) (NLP):** Nepali, English |
|
- **License:** [Apache license 2.0] |
|
- **Finetuned from model :** [google/mt5-small] |
|
|
|
|
|
## How to Get Started with the Model |
|
|
|
Use the code below to get started with the model. |
|
|
|
```Python |
|
|
|
from transformers import AutoTokenizer, MT5ForConditionalGeneration |
|
|
|
checkpoint = "syubraj/RomanEng2Nep-v2" |
|
tokenizer = AutoTokenizer.from_pretrained(checkpoint) |
|
model = MT5ForConditionalGeneration.from_pretrained(checkpoint) |
|
|
|
# Set max sequence length |
|
max_seq_len = 20 |
|
|
|
def translate(text): |
|
# Tokenize the input text with a max length of 20 |
|
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=max_seq_len) |
|
|
|
# Generate translation |
|
translated = model.generate(**inputs) |
|
|
|
# Decode the translated tokens back to text |
|
translated_text = tokenizer.decode(translated[0], skip_special_tokens=True) |
|
return translated_text |
|
|
|
# Example usage |
|
source_text = "muskuraudai" # Example Romanized Nepali text |
|
translated_text = translate(source_text) |
|
print(f"Translated Text: {translated_text}") |
|
``` |
|
|
|
|
|
|
|
### Training Data |
|
|
|
[syubraj/roman2nepali-transliteration](https://huggingface.co/datasets/syubraj/roman2nepali-transliteration) |
|
|
|
|
|
#### Training Hyperparameters |
|
|
|
- **Training regime:** |
|
```Python |
|
training_args = Seq2SeqTrainingArguments( |
|
output_dir="/content/drive/MyDrive/romaneng2nep_v2/", |
|
eval_strategy="steps", |
|
learning_rate=2e-5, |
|
per_device_train_batch_size=16, |
|
per_device_eval_batch_size=8, |
|
weight_decay=0.01, |
|
save_total_limit=3, |
|
num_train_epochs=2, |
|
predict_with_generate=True, |
|
) |
|
``` |
|
|
|
## Training and Validation Metrics |
|
|
|
| Step | Training Loss | Validation Loss | Gen Len | |
|
|------|---------------|-----------------|---------| |
|
| 500 | 21.636200 | 9.776628 | 2.001900 | |
|
| 1000 | 10.103400 | 6.105016 | 2.077900 | |
|
| 1500 | 6.830800 | 5.081259 | 3.811600 | |
|
| 2000 | 6.003100 | 4.702793 | 4.237300 | |
|
| 2500 | 5.690200 | 4.469123 | 4.700000 | |
|
| 3000 | 5.443100 | 4.274406 | 4.808300 | |
|
| 3500 | 5.265300 | 4.121417 | 4.749400 | |
|
| 4000 | 5.128500 | 3.989708 | 4.782300 | |
|
| 4500 | 5.007200 | 3.885391 | 4.805100 | |
|
| 5000 | 4.909600 | 3.787640 | 4.874800 | |
|
| 5500 | 4.836000 | 3.715750 | 4.855500 | |
|
| 6000 | 4.733000 | 3.640963 | 4.962000 | |
|
| 6500 | 4.673500 | 3.587330 | 5.011600 | |
|
| 7000 | 4.623800 | 3.531883 | 5.068300 | |
|
| 7500 | 4.567400 | 3.481622 | 5.108500 | |
|
| 8000 | 4.523200 | 3.445404 | 5.092700 | |
|
| 8500 | 4.464000 | 3.413630 | 5.132700 | |
