Liontix

/

Qwen3-8B-Gemini-2.5-Pro-Distill-GGUF

like 11

Hello @Enderchef ,

I started the fine tuning process, it should not take that long, I will reply here once it's published.

Hey @Enderchef ,

The model is ready, please let me know if this matches your expectations.
Download it here.

Would you share the source code how to fine tune it?

Hello @Hamora ,

I uploaded the safetensor files of this model here, the model card includes a link to an unsloth Jupyter notebook you can use to fine-tune it.

Eh, so this isn't RLHF tuning, just SFT ?
Or because you use the reasoning model as base so you just direct the fine tuning in SFT method?

@Liontix How do you think a 30b-a3b Moe fine-tune would work?

Hello @PSM24

It's technically possible, but on my current setup it doesn't work because it lacks the required VRAM. But I am experimenting with other smaller MoE models as they may perform significantly better than regular sized models on consumer hardware.