Are you considering to add more European languages ?
Hi,
very delighted to see these efforts for European languages !
I am asking myself if you would consider adding more European languages to the mix, like Norwegian, Icelandic, etc. ? I am aware that one could fine-tune the model for those languages, but having them already inside per-se would be a great foundation for the whole European ecosystem.
What would be needed to add those languages to the "next" version of the model ? Is the tokenizer already able to tokenize these languages ? Especially Icelandic has those unusual letters like "æ", "ð", "þ", etc.
Yes! So next to Fraunhofer IAIS many partners of the consortium like FZ Jülich are also working in the TrustLLM project that focuses on germanic languages (https://trustllm.eu) including Icelandic. Plus, Eurolingua will include more european languages (https://www.handelsblatt.com/technik/ki/ki-forschung-forscher-gewinnen-88-millionen-ki-trainingsstunden/100037506.html). Regarding the tokenizer you can refer to the OpenGPT-X publication (https://aclanthology.org/2024.findings-naacl.247/)
Thank you for providing these valuable insights! I found the link to the research paper particularly interesting. I assume that for Eurolingua, you are including all European languages, including Icelandic, in your training. The Handelsblatt article mentioned that you are building your text corpora based on web crawling. If this is not a mandatory criterion, you might consider using additionally the material available at https://github.com/icelandic-lt/, also available via https://clarin.is for the Icelandic portion.