Hugging Face's logo

RichardErkhov
/
jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf

GGUF
Inference Endpoints
conversational
Model card Files Community
jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf
File size: 8,436 Bytes 
e0e6454
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
 
Quantization made by Richard Erkhov.

[Github](https://github.com/RichardErkhov)

[Discord](https://discord.gg/pvy7H8DZMG)

[Request more models](https://github.com/RichardErkhov/quant_request)


gemma-2-9B-it-advanced-v2.1 - GGUF
- Model creator: https://huggingface.co/jsgreenawalt/
- Original model: https://huggingface.co/jsgreenawalt/gemma-2-9B-it-advanced-v2.1/


| Name | Quant method | Size |
| ---- | ---- | ---- |
| [gemma-2-9B-it-advanced-v2.1.Q2_K.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q2_K.gguf) | Q2_K | 3.54GB |
| [gemma-2-9B-it-advanced-v2.1.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.IQ3_XS.gguf) | IQ3_XS | 3.86GB |
| [gemma-2-9B-it-advanced-v2.1.IQ3_S.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.IQ3_S.gguf) | IQ3_S | 4.04GB |
| [gemma-2-9B-it-advanced-v2.1.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q3_K_S.gguf) | Q3_K_S | 4.04GB |
| [gemma-2-9B-it-advanced-v2.1.IQ3_M.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.IQ3_M.gguf) | IQ3_M | 4.19GB |
| [gemma-2-9B-it-advanced-v2.1.Q3_K.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q3_K.gguf) | Q3_K | 4.43GB |
| [gemma-2-9B-it-advanced-v2.1.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q3_K_M.gguf) | Q3_K_M | 4.43GB |
| [gemma-2-9B-it-advanced-v2.1.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q3_K_L.gguf) | Q3_K_L | 4.78GB |
| [gemma-2-9B-it-advanced-v2.1.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.IQ4_XS.gguf) | IQ4_XS | 4.86GB |
| [gemma-2-9B-it-advanced-v2.1.Q4_0.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q4_0.gguf) | Q4_0 | 5.07GB |
| [gemma-2-9B-it-advanced-v2.1.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.IQ4_NL.gguf) | IQ4_NL | 5.1GB |
| [gemma-2-9B-it-advanced-v2.1.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q4_K_S.gguf) | Q4_K_S | 5.1GB |
| [gemma-2-9B-it-advanced-v2.1.Q4_K.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q4_K.gguf) | Q4_K | 5.37GB |
| [gemma-2-9B-it-advanced-v2.1.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q4_K_M.gguf) | Q4_K_M | 5.37GB |
| [gemma-2-9B-it-advanced-v2.1.Q4_1.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q4_1.gguf) | Q4_1 | 5.55GB |
| [gemma-2-9B-it-advanced-v2.1.Q5_0.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q5_0.gguf) | Q5_0 | 6.04GB |
| [gemma-2-9B-it-advanced-v2.1.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q5_K_S.gguf) | Q5_K_S | 6.04GB |
| [gemma-2-9B-it-advanced-v2.1.Q5_K.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q5_K.gguf) | Q5_K | 6.19GB |
| [gemma-2-9B-it-advanced-v2.1.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q5_K_M.gguf) | Q5_K_M | 6.19GB |
| [gemma-2-9B-it-advanced-v2.1.Q5_1.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q5_1.gguf) | Q5_1 | 6.52GB |
| [gemma-2-9B-it-advanced-v2.1.Q6_K.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q6_K.gguf) | Q6_K | 7.07GB |
| [gemma-2-9B-it-advanced-v2.1.Q8_0.gguf](https://huggingface.co/RichardErkhov/jsgreenawalt_-_gemma-2-9B-it-advanced-v2.1-gguf/blob/main/gemma-2-9B-it-advanced-v2.1.Q8_0.gguf) | Q8_0 | 9.15GB |




Original model description:
---
base_model:
- wzhouad/gemma-2-9b-it-WPO-HB
- UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3
- google/gemma-2-9b-it
- princeton-nlp/gemma-2-9b-it-SimPO
library_name: transformers
tags:
- mergekit
- merge
- merge

---
# Gemma Advanced V2.1

This is a merge of the 'smartest' advanced fine-tunes available for Gemma-2-9b-it. It includes WPO, SimPO, and SPPO. The merge was performed via the SOTA 'della' merge method. Merge parameters have been hand-tuned for best results. The Q8_0 quant is highly recommended until better quants come along.

## Notes and observations:
* The extreme temperature sensitivity from V1 has been fixed, no longer needs to be run at lower temperatures
* Has a somewhat different writing style than any of the parent models
* Great instruction following
* Tracks plot details well and has good situational understanding
* Seems to have a good understanding of psychology, emotions and creative writing
* More 'sane' than base gemma-it, SPPO, or SimPO - not as prone to 'Cruella De Vil' or 'Evil Sorceress' like SPPO or SimPO, when portraying characters
* Would likely serve as a good base for further merges
* I'm looking for a job, if you're hiring. I'm a skilled Python developer who brings strong devops skills along with an ever-growing knowledge of machine learning pipelines and models. Message me if you want to talk about what I can bring to your team.
* Overall, this feels like a very useful and successful merge.

## Quantized GGUFs can be found here:
* [My quants, Q8_0 tested  -  jsgreenawalt/gemma-2-9B-it-advanced-v2.1-GGUF](https://huggingface.co/jsgreenawalt/gemma-2-9B-it-advanced-v2.1-GGUF)
* [iMatrix  -  mradermacher/gemma-2-9B-it-advanced-v2.1-i1-GGUF](https://huggingface.co/mradermacher/gemma-2-9B-it-advanced-v2.1-i1-GGUF)
* [QuantFactory/gemma-2-9B-it-advanced-v2.1-GGUF](https://huggingface.co/QuantFactory/gemma-2-9B-it-advanced-v2.1-GGUF)
* [mradermacher/gemma-2-9B-it-advanced-v2.1-GGUF](https://huggingface.co/mradermacher/gemma-2-9B-it-advanced-v2.1-GGUF)

Thanks to everyone who was kind enough to provide quants!

I'll link to other quants as they appear.

# sample ollama Modelfile
```yaml
FROM /path/to/file/gemma-2-9B-it-advanced-v2.1-Q8_0.gguf
PARAMETER stop "<start_of_turn>"
PARAMETER stop "<end_of_turn>"
PARAMETER num_ctx 8192
TEMPLATE """<start_of_turn>user
{{ if .System }}{{ .System }} {{ end }}{{ .Prompt }}<end_of_turn>
<start_of_turn>model
{{ .Response }}<end_of_turn>"""
```

This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).

## Merge Details
### Merge Method

This model was merged using the della merge method using [google/gemma-2-9b-it](https://huggingface.co/google/gemma-2-9b-it) as a base.

### Models Merged

The following models were included in the merge:
* [wzhouad/gemma-2-9b-it-WPO-HB](https://huggingface.co/wzhouad/gemma-2-9b-it-WPO-HB)
* [princeton-nlp/gemma-2-9b-it-SimPO](https://huggingface.co/princeton-nlp/gemma-2-9b-it-SimPO)
* [UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3](https://huggingface.co/UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3)

### Configuration

The following YAML configuration was used to produce this model:

```yaml
models:
  - model: google/gemma-2-9b-it 
  - model: wzhouad/gemma-2-9b-it-WPO-HB
    parameters:
      density: 0.55
      weight: 0.6
  - model: princeton-nlp/gemma-2-9b-it-SimPO 
    parameters:
      density: 0.35
      weight: 0.6
  - model: UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3
    parameters:
      density: 0.25
      weight: 0.4
merge_method: della
base_model: google/gemma-2-9b-it
parameters:
  normalize: true
  int8_mask: true
  lambda: 1.0
  epsilon: 0.1
dtype: float16

```