knifeayumu commited on
Commit
400a247
1 Parent(s): 3bc85d1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +50 -38
README.md CHANGED
@@ -1,38 +1,50 @@
1
- ---
2
- base_model: []
3
- library_name: transformers
4
- tags:
5
- - mergekit
6
- - merge
7
-
8
- ---
9
- # Lite-Cydonia-22B-v1.1
10
-
11
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
-
13
- ## Merge Details
14
- ### Merge Method
15
-
16
- This model was merged using the [task arithmetic](https://arxiv.org/abs/2212.04089) merge method using G:/AI-TextGen/text-generation-webui/models/Cydonia-22B-v1.1 as a base.
17
-
18
- ### Models Merged
19
-
20
- The following models were included in the merge:
21
- * G:/AI-TextGen/text-generation-webui/models/Mistral-Small-Instruct-2409
22
-
23
- ### Configuration
24
-
25
- The following YAML configuration was used to produce this model:
26
-
27
- ```yaml
28
- models:
29
- - model: G:/AI-TextGen/text-generation-webui/models/Cydonia-22B-v1.1
30
- parameters:
31
- weight: 0.75
32
- - model: G:/AI-TextGen/text-generation-webui/models/Mistral-Small-Instruct-2409
33
- parameters:
34
- weight: 0.25
35
- merge_method: task_arithmetic
36
- base_model: G:/AI-TextGen/text-generation-webui/models/Cydonia-22B-v1.1
37
- dtype: float16
38
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - TheDrummer/Cydonia-22B-v1.1
4
+ - mistralai/Mistral-Small-Instruct-2409
5
+ library_name: transformers
6
+ tags:
7
+ - mergekit
8
+ - merge
9
+ license: other
10
+ ---
11
+ # The Drummer turns into a Joshi Youchien
12
+
13
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/arcee-ai/mergekit).
14
+
15
+ GGUF quants : [knifeayumu/Lite-Cydonia-22B-v1.1-Test-GGUF](https://huggingface.co/knifeayumu/Lite-Cydonia-22B-v1.1-Test-GGUF)
16
+
17
+ ## Inspiration
18
+
19
+ I thought [BeaverAI/Cydonia-22B-v1f-GGUF](https://huggingface.co/TheDrummer/Cydonia-22B-v1.1) and [BeaverAI/Cydonia-22B-v1e-GGUF](https://huggingface.co/BeaverAI/Cydonia-22B-v1e-GGUF) versions being a bit too evil. The sense of morality is screwed up too much and it was a bit deterministic (swipes don't give much variety) versus the base model. Then an idea propped into my mind — why not merge it back again to the base? Give it a sense of "good" back, at least a little. Maybe that should fix some of deterministic generations too.
20
+
21
+ Quick testing shows... it works? Zero-shot evil Q&A no longer works but which a bit of persuasion, it did answer. I've also tried with both weights at 0.5 but it was too moral for my liking. Hence, I uploaded this version.
22
+
23
+ Credits to [TheDrummer](https://huggingface.co/TheDrummer) and [BeaverAI](https://huggingface.co/BeaverAI) who makes such finetunes. "Lightly decensored" is a heavy understatement in this case.
24
+
25
+ ## Merge Details
26
+ ### Merge Method
27
+
28
+ This model was merged using the [task arithmetic](https://arxiv.org/abs/2212.04089) merge method using [TheDrummer/Cydonia-22B-v1.1](https://huggingface.co/TheDrummer/Cydonia-22B-v1.1) as a base.
29
+
30
+ ### Models Merged
31
+
32
+ The following models were included in the merge:
33
+ * [mistralai/Mistral-Small-Instruct-2409](https://huggingface.co/mistralai/Mistral-Small-Instruct-2409)
34
+
35
+ ### Configuration
36
+
37
+ The following YAML configuration was used to produce this model:
38
+
39
+ ```yaml
40
+ models:
41
+ - model: TheDrummer/Cydonia-22B-v1.1
42
+ parameters:
43
+ weight: 0.75
44
+ - model: mistralai/Mistral-Small-Instruct-2409
45
+ parameters:
46
+ weight: 0.25
47
+ merge_method: task_arithmetic
48
+ base_model: TheDrummer/Cydonia-22B-v1.1
49
+ dtype: float16
50
+ ```