Quantization made by Richard Erkhov.
kukulemon-spiked-9B - GGUF
- Model creator: https://huggingface.co/grimjim/
- Original model: https://huggingface.co/grimjim/kukulemon-spiked-9B/
| Name | Quant method | Size |
|---|---|---|
| kukulemon-spiked-9B.Q2_K.gguf | Q2_K | 3.13GB |
| kukulemon-spiked-9B.IQ3_XS.gguf | IQ3_XS | 3.48GB |
| kukulemon-spiked-9B.IQ3_S.gguf | IQ3_S | 3.67GB |
| kukulemon-spiked-9B.Q3_K_S.gguf | Q3_K_S | 3.65GB |
| kukulemon-spiked-9B.IQ3_M.gguf | IQ3_M | 3.79GB |
| kukulemon-spiked-9B.Q3_K.gguf | Q3_K | 4.05GB |
| kukulemon-spiked-9B.Q3_K_M.gguf | Q3_K_M | 4.05GB |
| kukulemon-spiked-9B.Q3_K_L.gguf | Q3_K_L | 4.41GB |
| kukulemon-spiked-9B.IQ4_XS.gguf | IQ4_XS | 4.55GB |
| kukulemon-spiked-9B.Q4_0.gguf | Q4_0 | 4.74GB |
| kukulemon-spiked-9B.IQ4_NL.gguf | IQ4_NL | 4.79GB |
| kukulemon-spiked-9B.Q4_K_S.gguf | Q4_K_S | 4.78GB |
| kukulemon-spiked-9B.Q4_K.gguf | Q4_K | 5.04GB |
| kukulemon-spiked-9B.Q4_K_M.gguf | Q4_K_M | 5.04GB |
| kukulemon-spiked-9B.Q4_1.gguf | Q4_1 | 5.26GB |
| kukulemon-spiked-9B.Q5_0.gguf | Q5_0 | 5.77GB |
| kukulemon-spiked-9B.Q5_K_S.gguf | Q5_K_S | 5.77GB |
| kukulemon-spiked-9B.Q5_K.gguf | Q5_K | 5.93GB |
| kukulemon-spiked-9B.Q5_K_M.gguf | Q5_K_M | 5.93GB |
| kukulemon-spiked-9B.Q5_1.gguf | Q5_1 | 6.29GB |
| kukulemon-spiked-9B.Q6_K.gguf | Q6_K | 6.87GB |
| kukulemon-spiked-9B.Q8_0.gguf | Q8_0 | 8.89GB |
Original model description:
base_model: - grimjim/kukulemon-7B library_name: transformers tags: - mergekit - merge license: cc-by-nc-4.0 pipeline_tag: text-generation
kululemon-spiked-9B
This is a frankenmerge of a pre-trained language model created using mergekit. As an experiment, this appears to be a partial success.
Lightly tested with temperature 1-1.2 and minP 0.01 with ChatML prompts; the model supports Alpaca prompts and has 8K context length, a result of its Mistral v0.1 provenance. The model's output has been coherent and stable during aforementioned testing.
The merge formula for this frankenmerge is below. It is conjectured that the shorter first section is not key to variation, the middle segment is key to balancing reasoning and variation, and that the lengthy final section is required for convergence and eventual stability. The internal instability is probably better suited for narrative involving unstable and/or unhinged characters and situations.
Quants available:
Merge Details
Merge Method
This model was merged using the passthrough merge method.
Models Merged
The following models were included in the merge:
Configuration
The following YAML configuration was used to produce this model:
slices:
- sources:
- model: grimjim/kukulemon-7B
layer_range: [0, 12]
- sources:
- model: grimjim/kukulemon-7B
layer_range: [8, 16]
- sources:
- model: grimjim/kukulemon-7B
layer_range: [12, 32]
merge_method: passthrough
dtype: float16
- Downloads last month
- 111