Update README.md

README.md

@@ -1,46 +1 @@
----
-base_model: []
-library_name: transformers
-tags:
-- mergekit
-- merge
-
----
-# final
-
-This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
-
-## Merge Details
-### Merge Method
-
-This model was merged using the SLERP merge method.
-
-### Models Merged
-
-The following models were included in the merge:
-* ./commerge1
-* ./commerge2
-
-### Configuration
-
-The following YAML configuration was used to produce this model:
-
-```yaml
-slices:
-  - sources:
-      - model: ./commerge1
-        layer_range: [0, 28]
-      - model: ./commerge2
-        layer_range: [0, 28]
-
-merge_method: slerp
-base_model: ./commerge1
-parameters:
-  t:
-    - filter: self_attn
-      value: [0, 0.5, 0.3, 0.7, 1]
-    - filter: mlp
-      value: [1, 0.5, 0.7, 0.3, 0]
-    - value: 0.5
-dtype: bfloat16
-```
+This is simply a base gemma with the bug fixes I implemented in GemMoE applied to this model. Be sure to use ```trust_remote_code:/= True``` when training or running inference.