mistral
#2
by
Aryanne
- opened
can you do the same with mistral?
Yes, I think the idea applies to mistral.
I'm really excited to see where this goes!
Yes, I think the idea applies to mistral.
this model looks extremely good for a base model, I would like to see a fine-tuned version (e.g. OpenOrca),
for tasks like answering from the context (RAG), we don't need big models,
so I would say a Mistral little brother with the same big context (32K) and architecture (Grouped-query attention and Sliding Window Attention) and fine-tuned to follow instructions (e.g. Mistral-7B-OpenOrca) is more than enough