xLSTM-7B-Instruct model
I noticed that NX-AI’s xLSTM-7b lacks an instruct model variant fine-tuned for instruction-following tasks.
An instruct model would greatly enhance its utility for applications like QA, virtual assistants, and domain-specific tasks.
Would NX-AI or the community consider creating one?
NLP may not be the main focus of NXAI's xLSTM, but it still performs remarkably well.
https://huggingface.co/mrs83/FlowerTune-xLSTM-7b-NLP-PEFT
With this adapter, the base model reaches 15.35% average accuracy on NLP tasks (STEM, Social Sciences, Humanities) after instruction fine-tuning, using a small portion (25%) of the vicgalle/alpaca-gpt4 dataset as training data.
I am wondering what could be achieved by training a full instruct model. Could it surpass models like mistralai/Mistral-7B-Instruct-v0.3?