Update README.md

README.md

@@ -11,11 +11,11 @@ widget:
 ---
 
 # t5-small-spanish-nahuatl
-Nahuatl is the most widely spoken indigenous language in Mexico. However, training a neural network for the neural machine translation task is challenging due to the lack of structured data. The most popular datasets, such as the Axolot and bible-corpus, only consist of ~16,000 and ~7,000 samples, respectively. Moreover, there are multiple variants of Nahuatl, which makes this task even more difficult. For example, it is possible to find a single word from the Axolot dataset written in more than three different ways. Therefore, we leverage the T5 text-to-text prefix training strategy in this work to compensate for the lack of data. We first teach the multilingual model Spanish using English, then transition to Spanish-Nahuatl. The resulting model successfully translates short sentences from Spanish to Nahuatl. Finally, we report Chrf and BLEU results.
+Nahuatl is the most widely spoken indigenous language in Mexico. However, training a neural network for the neural machine translation task is challenging due to the lack of structured data. The most popular datasets, such as the Axolot and bible-corpus, only consist of ~16,000 and ~7,000 samples, respectively. Moreover, there are multiple variants of Nahuatl, which makes this task even more difficult. For example, it is possible to find a single word from the Axolot dataset written in more than three different ways. Therefore, we leverage the T5 text-to-text prefix training strategy in this work to compensate for the lack of data. We first train the multilingual model to learn Spanish and then adapt the model to Nahuatl. The resulting model successfully translates short sentences from Spanish to Nahuatl. Finally, we report Chrf and BLEU results.
 
 
 ## Model description
-This model is a T5 Transformer ([t5-small](https://huggingface.co/t5-small)) fine-tuned on spanish and nahuatl sentences collected from the web. The dataset is normalized using 'sep' normalization from [py-elotl](https://github.com/ElotlMX/py-elotl).
+This model is a T5 Transformer ([t5-small](https://huggingface.co/t5-small)) fine-tuned on Spanish and Nahuatl sentences collected from the web. The dataset is normalized using 'sep' normalization from [py-elotl](https://github.com/ElotlMX/py-elotl).
 
 
 ## Usage
@@ -36,7 +36,7 @@ outputs = tokenizer.batch_decode(outputs, skip_special_tokens=True)[0]
 
 ## Approach
 ### Dataset
-Since the Axolotl corpus contains misaligments, we just select the best samples (12,207 samples). We also use the [bible-corpus](https://github.com/christos-c/bible-corpus) (7,821 samples). 
+Since the Axolotl corpus contains misalignments, we select the best samples (12,207). We also use the [bible-corpus](https://github.com/christos-c/bible-corpus) (7,821). 
 
 | Axolotl best aligned books                            | 
 |:-----------------------------------------------------:|
@@ -55,30 +55,31 @@ Since the Axolotl corpus contains misaligments, we just select the best samples
 | Una tortillita nomás - Se taxkaltsin saj              |
 | Vida económica de Tenochtitlan                        |
 
-Also, to increase the amount of data, we collected 3,000 extra samples from the web.
+Also, we collected 3,000 extra samples from the web to increase the data.
 
 ### Model and training
-We employ two training stages using a multilingual T5-small. We use this model because it can handle different vocabularies and prefixes. T5-small is pre-trained on different tasks and languages (French, Romanian, English, German).
+We employ two training stages using a multilingual T5-small. The advantage of this model is that it can handle different vocabularies and prefixes. T5-small is pre-trained on different tasks and languages (French, Romanian, English, German).
 
 ### Training-stage 1 (learning Spanish)
-In training stage 1 we first introduce Spanish to the model. The goal is to learn a new language rich in data (Spanish) and not lose the previous knowledge acquired. We use the English-Spanish [Anki](https://www.manythings.org/anki/) dataset, which consists of 118.964 text pairs. We train the model till convergence adding the prefix "Translate Spanish to English: ".
+In training stage 1, we first introduce Spanish to the model. The goal is to learn a new language rich in data (Spanish) and not lose the previous knowledge. We use the English-Spanish [Anki](https://www.manythings.org/anki/) dataset, which consists of 118.964 text pairs. Next, we train the model till convergence, adding the prefix "Translate Spanish to English: "
 
 ### Training-stage 2 (learning Nahuatl)
-We use the pre-trained Spanish-English model to learn Spanish-Nahuatl. Since the amount of Nahuatl pairs is limited, we also add to our dataset 20,000 samples from the English-Spanish Anki dataset. This two-task-training avoids overfitting end makes the model more robust.
+We use the pre-trained Spanish-English model to learn Spanish-Nahuatl. Since the amount of Nahuatl pairs is limited, we also add 20,000 samples from the English-Spanish Anki dataset to our dataset. This two-task training avoids overfitting and makes the model more robust.
 
 ### Training setup
 We train the models on the same datasets for 660k steps using batch size = 16 and a learning rate of 2e-5.
 
 
 ## Evaluation results
-For a fair comparison, the models are evaluated on the same 505 validation  Nahuatl sentences. We report the results using chrf and sacrebleu hugging face metrics:
+We evaluate the model on the same 505 validation Nahuatl sentences for a fair comparison. Finally, we report the results using chrf and sacrebleu hugging face metrics:
 
 | English-Spanish pretraining  | Validation loss | BLEU | Chrf   |
 |:----------------------------:|:---------------:|:-----|-------:|
 | False                        | 1.34            | 6.17 | 26.96  | 
 | True                         | 1.31            | 6.18 | 28.21  | 
 
-The English-Spanish pretraining improves BLEU and Chrf, and leads to faster convergence. You can reproduce the evaluation on the [eval.ipynb](https://github.com/milmor/spanish-nahuatl-translation/blob/main/eval.ipynb) notebook. 
+
+The English-Spanish pretraining improves BLEU and Chrf and leads to faster convergence. Is it possible to reproduce the evaluation on the [eval.ipynb](https://github.com/milmor/spanish-nahuatl-translation/blob/main/eval.ipynb) notebook. 
 
 ## References
 - Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2019. Exploring the limits
@@ -86,6 +87,10 @@ of transfer learning with a unified Text-to-Text transformer.
 
 - Ximena Gutierrez-Vasques, Gerardo Sierra, and Hernandez Isaac. 2016. Axolotl: a web accessible parallel corpus for Spanish-Nahuatl. In International Conference on Language Resources and Evaluation (LREC).
 
+- https://github.com/christos-c/bible-corpus
+
+- https://github.com/ElotlMX/py-elotl
+
 
 ## Team members
 - Emilio Alejandro Morales [(milmor)](https://huggingface.co/milmor)