Update README.md

README.md

@@ -18,11 +18,14 @@ SmolLM2-MedIT-Upscale-2B is an expanded version of the [SmolLM2-1.7B-Instruct](h
 
 This model was developed to test the hypothesis that self-attention layers do not extend the "memory" of the model. By broadening the attention layers, we aim to observe the impact on the model's performance and memory capabilities.
 
-**Training Status**
+## Training Status
 
+This model underwent instruction fine-tuning for 8,800 steps using a batch size of 4, gradient accumulation for 32 steps, a maximum sequence length of 1,280, and a learning rate of 1e-5. Additionally, it was fine-tuned with 1,600 steps of DPO under the same configuration.
+
+**Note**:
 The model has undergone preliminary training focused on assessing the effects of the expanded attention layers. It is not fully trained to its maximum potential. We encourage the community to contribute to its further training; pull requests are welcome.
 
-**Analysis of Expanded Layers**
+## Analysis of Expanded Layers
 
 During fine-tuning, we analyzed the changes in the new parameters of the expanded layers:
 
@@ -49,19 +52,19 @@ These results are illustrated in the following charts:
 ![Percentage of Change](percent_of_change.png)
 ![Average Parameter Change](mean_difference.png)
 
-**Usage**
+## Usage
 
 To utilize this model, follow the instructions provided for the original SmolLM2-1.7B-Instruct model, adjusting for the increased parameter size.
 
-**Contributing**
+## Contributing
 
 We welcome contributions to further train and evaluate this model. Please submit pull requests with your improvements.
 
-**License**
+## License
 
 This model is licensed under the Apache 2.0 License.
 
-**Citation**
+## Citation
 
 If you use this model in your research, please cite it as follows: