issues with Russian outputs
First of all, I want to say a big thanks to the developers of Mini-Russian-llama for making this available to the community! It’s a great model, and I really appreciate all the effort behind it.
I’ve been fine-tuning the model to generate engineering-related texts in both English and Russian. After fine-tuning, the results in English have been quite solid, but I’m running into issues with Russian outputs. The model often produces typos, strange phrasing, or conclusions that don’t quite make sense for technical contexts. Unfortunately, fine-tuning didn’t seem to improve the Russian text quality much.
I know multilingual tasks are challenging, and this is already an impressive achievement, but I was wondering if anyone has advice or experience with improving Russian text generation for a technical domain. Are there specific tricks, preprocessing steps, or adjustments during fine-tuning that could help?
I’d be super grateful for any tips or suggestions!