Spaces:
Running
on
CPU Upgrade
Testing considering other LLMs
Hi Eduardo,
There is an apparent lack of models being tested for the scoreboard. Euro LLM 9B can be self-hosted and is not listed, like others.
You can also check missing models by going through this work from Rui Brogueira Andrade : https://repositorio-aberto.up.pt/bitstream/10216/168804/2/735556.pdf . Hope it helps increase the significance of your work.
Hi jpedrofonseca,
I did not get what you are specifically requesting, I think most Euro LLM models are in the leaderboard, including the 9B version
Normally, when a model has portuguese support , it gets bumped to the top of queue list and ran before other english-centric models.
What may happen is that a model may have some uncommon external lib dependency or it's not yet properly supported in the stable version of the transformers lib or my gpu cluster have some specific technical problem and it takes a while for me to check why the model fail, fix the envoriment problems and run it again. (The VLM variations of EuroLLM have not yet been run for this reason)
If I'm not mistaken, the eurollm models got in the backburner for a similar reason.
If you like me to give priority to run/fix a failed run for an specific model you can open a request like this one here, I will look into it as soon as I can.