Error loading model? (Q6)
I can't run the Q6, but I can the IQ4_XS...? Is there something wrong with the split thing? Am I doing something wrong? Please, help. Koboldcpp shows this error and crashes/closes:
Have you properly concatenated the parts? We do not use the GGUF split format for many reasons so all parts need to be concatenated before the model can be loaded.
cat Sapphira-L3.3-70b-0.1.i1-Q6_K.gguf.part1of2 Sapphira-L3.3-70b-0.1.i1-Q6_K.gguf.part2of2 > Sapphira-L3.3-70b-0.1.i1-Q6_K.gguf
Alternatively, just download the model from https://hf.tst.eu/model#Sapphira-L3.3-70b-0.1-i1-GGUF so it already gets properly concatenated while downloading.
Double check your parts are complete. Try to redownload using wget -c and see if any data is missing. (though if you're using windows... a bit more annoying than linux to resume files of this size)
Other than that, concatenating the two together is needed as far as i can tell.
LLM programs need to allow m3u files and put the models in a list in order so it can handle this without having to make a new file from the downloaded parts.
Thanks, guys. I mostly have little idea what you guys said, but that link nicoboss posted solved the problem. lol So, um, I think a few days ago I actually tried merging the files and, surprisingly, I think I did the right steps and such but there was a diff error that popped up and the files didn't merge in the end. Anyways, that doesn't matter anymore, cuz I already dl'ed the full thing from nicoboss's link. Oh, and upon a first, quick test last night, I didn't notice much difference from the lower quant thing I was using... kinda disappointed, but I guess it needs further testing. Ok, so I'll stop rambling... I don't know how to close this thread or even if I can. Thanks for the help, nicoboss.