phi-2-coder scores

#11
by vince62s - opened

@mlabonne here are the phi-2-coder scores to fill your table:
AGIEval: 29.3
GPT4All: 71.03
TruthfullQA: 45.13
BigBench: 35.54
Average: 45.25
Took 1h28m on a RTX4090

Nice, thanks a lot!

vince62s changed discussion status to closed

Thanks a lot @vince62s for the eval!!
I am trying to run in my colab VM (A100) and with this params I am getting errors. Do you have the used script there?

you won't be able to run it because MS updated their configuration_phi.py / modeling_phi.py files.
either you copy them back from an older phi-2 equivalent (like phi dolphin) or you upgrade your weights with the new layer names.

Thanks for the clarification!

Sign up or log in to comment