File size: 4,909 Bytes
fdbe4d7 51e50fa fdbe4d7 51e50fa 3226321 0059542 fdbe4d7 584bd36 bc5d744 3226321 fdbe4d7 f7a96e9 87ea540 3226321 a7c0085 fdbe4d7 3226321 fdbe4d7 3226321 fdbe4d7 3226321 fdbe4d7 3226321 fdbe4d7 3226321 fdbe4d7 3226321 fdbe4d7 3226321 fdbe4d7 3226321 69ce82b 3226321 fdbe4d7 3226321 fdbe4d7 3226321 fdbe4d7 3226321 fdbe4d7 3226321 0a282be 0059542 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 |
---
base_model:
- google/mt5-small
datasets:
- syubraj/roman2nepali-transliteration
language:
- ne
- en
library_name: transformers
license: apache-2.0
pipeline_tag: translation
tags:
- nepali
- roman english
- translation
- transliteration
new_version: syubraj/romaneng2nep_v2
---
# Model Card for Model ID
Due to compute issues, The model has been trained on multiple iterations:
1. Model Trained for 8500 steps on [0 : 5%] of the dataset.
2. Model continued from 8500 upto 16500 steps on [5% : 20%] of the dataset
3. Model continued from 16500 upto 22000 steps on [20% : 40%] of the dataset
### Model Description
<!-- Provide a longer summary of what this model is. -->
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
- **Model type:** (Translation)
- **Language(s) (NLP):** Nepali, English
- **License:** [Apache license 2.0]
- **Finetuned from model :** [google/mt5-small]
## How to Get Started with the Model
Use the code below to get started with the model.
```Python
from transformers import AutoTokenizer, MT5ForConditionalGeneration
checkpoint = "syubraj/RomanEng2Nep-v2"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = MT5ForConditionalGeneration.from_pretrained(checkpoint)
# Set max sequence length
max_seq_len = 20
def translate(text):
# Tokenize the input text with a max length of 20
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=max_seq_len)
# Generate translation
translated = model.generate(**inputs)
# Decode the translated tokens back to text
translated_text = tokenizer.decode(translated[0], skip_special_tokens=True)
return translated_text
# Example usage
source_text = "muskuraudai" # Example Romanized Nepali text
translated_text = translate(source_text)
print(f"Translated Text: {translated_text}")
```
### Training Data
[syubraj/roman2nepali-transliteration](https://huggingface.co/datasets/syubraj/roman2nepali-transliteration)
#### Training Hyperparameters
- **Training regime:**
```Python
training_args = Seq2SeqTrainingArguments(
output_dir="/content/drive/MyDrive/romaneng2nep_v2/",
eval_strategy="steps",
learning_rate=2e-5,
per_device_train_batch_size=16,
per_device_eval_batch_size=8,
weight_decay=0.01,
save_total_limit=3,
num_train_epochs=2,
predict_with_generate=True,
)
```
## Training and Validation Metrics
Step | Training Loss | Validation Loss | Gen Len
--------|---------------|-----------------|---------
500 | 21.636200 | 9.776628 | 2.001900
1000 | 10.103400 | 6.105016 | 2.077900
1500 | 6.830800 | 5.081259 | 3.811600
2000 | 6.003100 | 4.702793 | 4.237300
2500 | 5.690200 | 4.469123 | 4.700000
3000 | 5.443100 | 4.274406 | 4.808300
3500 | 5.265300 | 4.121417 | 4.749400
4000 | 5.128500 | 3.989708 | 4.782300
4500 | 5.007200 | 3.885391 | 4.805100
5000 | 4.909600 | 3.787640 | 4.874800
5500 | 4.836000 | 3.715750 | 4.855500
6000 | 4.733000 | 3.640963 | 4.962000
6500 | 4.673500 | 3.587330 | 5.011600
7000 | 4.623800 | 3.531883 | 5.068300
7500 | 4.567400 | 3.481622 | 5.108500
8000 | 4.523200 | 3.445404 | 5.092700
8500 | 4.464000 | 3.413630 | 5.132700
9000 | 4.423100 | 3.326201 | 5.211700
9500 | 4.315700 | 3.238422 | 5.200600
10000 | 4.218200 | 3.143774 | 5.288100
10500 | 4.133600 | 3.080613 | 5.202300
11000 | 4.087700 | 3.011713 | 5.271800
11500 | 4.004300 | 2.957386 | 5.178700
12000 | 3.956700 | 2.898953 | 5.209600
12500 | 3.922800 | 2.850440 | 5.210100
13000 | 3.853400 | 2.796974 | 5.171700
13500 | 3.807900 | 2.745325 | 5.281200
14000 | 3.755700 | 2.708517 | 5.223000
14500 | 3.729300 | 2.678200 | 5.210700
15000 | 3.673600 | 2.637842 | 5.230200
15500 | 3.625400 | 2.607649 | 5.264100
16000 | 3.601100 | 2.592188 | 5.129800
16500 | 3.608200 | 2.556329 | 5.215800
17000 | 3.557900 | 2.536781 | 5.162900
17500 | 3.533500 | 2.504695 | 5.206000
18000 | 3.500000 | 2.477887 | 5.211600
18500 | 3.463600 | 2.456758 | 5.201000
19000 | 3.457100 | 2.433362 | 5.210000
19500 | 3.435400 | 2.411479 | 5.197600
20000 | 3.413300 | 2.392534 | 5.221100
20500 | 3.366100 | 2.378421 | 5.165200
21000 | 3.363500 | 2.357117 | 5.187300
21500 | 3.346500 | 2.343485 | 5.193600
22000 | 3.328300 | 2.331021 | 5.183300 |