Hugging Face's logo

knifeayumu
/
Behemoth-v1.1-Magnum-v4-123B

Text Generation
Transformers
Safetensors
mistral
mergekit
Merge
conversational
text-generation-inference
Model card Files Community
2
knifeayumu commited on
Commit
a189b60
•
1 Parent(s): 2423a56

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. README.md +39 -0
  2. config.json +27 -0
  3. mergekit_config.yml +8 -0
  4. model-00001-of-00051.safetensors +3 -0
  5. model-00002-of-00051.safetensors +3 -0
  6. model-00003-of-00051.safetensors +3 -0
  7. model-00004-of-00051.safetensors +3 -0
  8. model-00005-of-00051.safetensors +3 -0
  9. model-00006-of-00051.safetensors +3 -0
  10. model-00007-of-00051.safetensors +3 -0
  11. model-00008-of-00051.safetensors +3 -0
  12. model-00009-of-00051.safetensors +3 -0
  13. model-00010-of-00051.safetensors +3 -0
  14. model-00011-of-00051.safetensors +3 -0
  15. model-00012-of-00051.safetensors +3 -0
  16. model-00013-of-00051.safetensors +3 -0
  17. model-00014-of-00051.safetensors +3 -0
  18. model-00015-of-00051.safetensors +3 -0
  19. model-00016-of-00051.safetensors +3 -0
  20. model-00017-of-00051.safetensors +3 -0
  21. model-00018-of-00051.safetensors +3 -0
  22. model-00019-of-00051.safetensors +3 -0
  23. model-00020-of-00051.safetensors +3 -0
  24. model-00021-of-00051.safetensors +3 -0
  25. model-00022-of-00051.safetensors +3 -0
  26. model-00023-of-00051.safetensors +3 -0
  27. model-00024-of-00051.safetensors +3 -0
  28. model-00025-of-00051.safetensors +3 -0
  29. model-00026-of-00051.safetensors +3 -0
  30. model-00027-of-00051.safetensors +3 -0
  31. model-00028-of-00051.safetensors +3 -0
  32. model-00029-of-00051.safetensors +3 -0
  33. model-00030-of-00051.safetensors +3 -0
  34. model-00031-of-00051.safetensors +3 -0
  35. model-00032-of-00051.safetensors +3 -0
  36. model-00033-of-00051.safetensors +3 -0
  37. model-00034-of-00051.safetensors +3 -0
  38. model-00035-of-00051.safetensors +3 -0
  39. model-00036-of-00051.safetensors +3 -0
  40. model-00037-of-00051.safetensors +3 -0
  41. model-00038-of-00051.safetensors +3 -0
  42. model-00039-of-00051.safetensors +3 -0
  43. model-00040-of-00051.safetensors +3 -0
  44. model-00041-of-00051.safetensors +3 -0
  45. model-00042-of-00051.safetensors +3 -0
  46. model-00043-of-00051.safetensors +3 -0
  47. model-00044-of-00051.safetensors +3 -0
  48. model-00045-of-00051.safetensors +3 -0
  49. model-00046-of-00051.safetensors +3 -0
  50. model-00047-of-00051.safetensors +3 -0
README.md ADDED
@@ -0,0 +1,39 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - anthracite-org/magnum-v4-123b
4
+ - TheDrummer/Behemoth-123B-v1.1
5
+ library_name: transformers
6
+ tags:
7
+ - mergekit
8
+ - merge
9
+
10
+ ---
11
+ # merge
12
+
13
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
14
+
15
+ ## Merge Details
16
+ ### Merge Method
17
+
18
+ This model was merged using the SLERP merge method.
19
+
20
+ ### Models Merged
21
+
22
+ The following models were included in the merge:
23
+ * [anthracite-org/magnum-v4-123b](https://huggingface.co/anthracite-org/magnum-v4-123b)
24
+ * [TheDrummer/Behemoth-123B-v1.1](https://huggingface.co/TheDrummer/Behemoth-123B-v1.1)
25
+
26
+ ### Configuration
27
+
28
+ The following YAML configuration was used to produce this model:
29
+
30
+ ```yaml
31
+ models:
32
+ - model: TheDrummer/Behemoth-123B-v1.1
33
+ - model: anthracite-org/magnum-v4-123b
34
+ merge_method: slerp
35
+ base_model: TheDrummer/Behemoth-123B-v1.1
36
+ parameters:
37
+ t: [0.1, 0.3, 0.6, 0.3, 0.1]
38
+ dtype: float16
39
+ ```
config.json ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "TheDrummer/Behemoth-123B-v1.1",
3
+ "architectures": [
4
+ "MistralForCausalLM"
5
+ ],
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": 1,
8
+ "eos_token_id": 2,
9
+ "head_dim": 128,
10
+ "hidden_act": "silu",
11
+ "hidden_size": 12288,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 28672,
14
+ "max_position_embeddings": 131072,
15
+ "model_type": "mistral",
16
+ "num_attention_heads": 96,
17
+ "num_hidden_layers": 88,
18
+ "num_key_value_heads": 8,
19
+ "rms_norm_eps": 1e-05,
20
+ "rope_theta": 1000000.0,
21
+ "sliding_window": null,
22
+ "tie_word_embeddings": false,
23
+ "torch_dtype": "float16",
24
+ "transformers_version": "4.45.1",
25
+ "use_cache": true,
26
+ "vocab_size": 32768
27
+ }
mergekit_config.yml ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ models:
2
+ - model: TheDrummer/Behemoth-123B-v1.1
3
+ - model: anthracite-org/magnum-v4-123b
4
+ merge_method: slerp
5
+ base_model: TheDrummer/Behemoth-123B-v1.1
6
+ parameters:
7
+ t: [0.1, 0.3, 0.6, 0.3, 0.1]
8
+ dtype: float16
model-00001-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3b9c67dd7122db0b16b0b8cc9d4c51501ca05a37ca91609ca735866050780ac5
3
+ size 4378928488
model-00002-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:82c517c0ec0d2715936a1fd9de2845253fa1dec2975c83abc9be6d1ac77c4eff
3
+ size 4907411072
model-00003-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:23487baa212737849c143959c49b297ce7bacef2367fa672d2dc0a52b04e5e5b
3
+ size 4806747888
model-00004-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c26ccd18499ffb3ff0eb2104b997d9c0a3826000eb0b9cff91f6c1974a97636c
3
+ size 4831938528
model-00005-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:17a8c889158afb91de383876df4377e919323081d0fd8af419da6ecff21eff1d
3
+ size 4831938536
model-00006-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b34d65a618104b7fc06eb90e47c9596ee147270e4e4325029483580c57f36603
3
+ size 4907411080
model-00007-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ca62bc87e09c18c86937e28a47a8469e5ab5e4a0f7e693bc45868458a82e9df9
3
+ size 4806747888
model-00008-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:22cfc848f83ae5c5e68d2055e14f1127e074f25fc9671d6c8dccd1d08875a10e
3
+ size 4831938520
model-00009-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1820fa12192bd7c560a6c6341183bd1504e6703085ebb57fc94189dd22c13fe7
3
+ size 4831938536
model-00010-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a1cc7b5a52c118ab995da22385feeedd9c5d13c158aca3ca76f3705c3c6472f5
3
+ size 4907411080
model-00011-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:95cbcf86e72a64ef75bb9a45e845121b625a8fabfd47eb5f04cbaff3984bafb8
3
+ size 4806747888
model-00012-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2db092e7829bf5ba1b279f88541dec48ec39d2866786bcfa935ae51575eedb00
3
+ size 4831938528
model-00013-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9756ba28898ff82beb78ec0ccaec5af874cd3eaf9dc12740f0c58d2e328f6467
3
+ size 4831938536
model-00014-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d617f305ea83b86a4e725bae4a84212942575bb1214c7b1a150fff755bd2ffeb
3
+ size 4907411072
model-00015-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cbda6c2fbd9627a612039e5db63bc16b60f1a7bd256ec0fc637d24ca6c6fdb83
3
+ size 4806747888
model-00016-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f2fac2af0ffd12363faa5b0a74ed17994538a8a2e9d9def582e113d53c05a1b9
3
+ size 4831938528
model-00017-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ee742ed2abc929026bdcec8c2f6a73eed49aa2c6611e32760f7b32f377b5b2c4
3
+ size 4831938536
model-00018-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:72d8d7fd283367e738319aef5daca11a561a50bbfec27da29493d3779394cf00
3
+ size 4907411080
model-00019-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5e2e50c673f53672c07901e7fb0ae3ef80290224869a43e3591a48521f83ac1a
3
+ size 4806747888
model-00020-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:66482d882bc1b3a7e17384b65e85f1246d85da1c7aedac0de41f0a90089666b8
3
+ size 4831938528
model-00021-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3ac7fc109f8fbe7fd62b7d4828e0d354ba0502dce6d0bb26252959c21b7ae4ca
3
+ size 4831938528
model-00022-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:60367a5b237bf7a86ae35442381ad19f1b1ca33586553e38a4732edc34e08419
3
+ size 4907411080
model-00023-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bda2d0b0978a48c6866bae30e831b874762622bb6c675b23459d27c6fa55096f
3
+ size 4806747888
model-00024-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ea2cddd4bb712dc0b3192daae2a0d11b01db9b8f678a7ac35dba9f95b7b9384a
3
+ size 4831938528
model-00025-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9a74a489aa29c4c705108def46dfe5e1c0e3ed28edd3c8853c8022506520665e
3
+ size 4831938536
model-00026-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4bd9eedd9ec1b65940e439e1c33aaa40cdd07958d3537dfbf489dbbf95294802
3
+ size 4907411080
model-00027-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a2b4b78b3291bb1a78ce84ac0e416c109a85548f4b503e39b8d6934406ccf116
3
+ size 4806747880
model-00028-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9811b27823a6a7388cef7435bee26a6e082f983f99428ce2907884657ea97d3e
3
+ size 4831938528
model-00029-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:251963e2666050f6d757518d1e360bee2a18a7495f1b2ad649593db99cc29b89
3
+ size 4831938536
model-00030-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a493a9dabcd7df8a475b393c86647fb237360ed929bd2078083bc41e1d45437a
3
+ size 4907411080
model-00031-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:49bd96c00cb74eb10bbcdc82137bb16be9bcfb8df97cf9489c65aae16b5f017f
3
+ size 4806747888
model-00032-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1e43965a93e33119e0869ff15dd66e9f3ec3caeacc97a7dcccf5bdbb355e7e49
3
+ size 4831938528
model-00033-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f6cfc0dcb11a99e77dcf5f0ff44965fa39a20f8b7b296235925548ac7a80758a
3
+ size 4831938528
model-00034-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b58fc61fd2f4ae449f90d03cbab69293739ce21ec71b5a5bf122e33e54c3e92e
3
+ size 4907411080
model-00035-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7cc9de2da23517f35f01d5c8160334c65ceb67f4b41ba1b05160339765546beb
3
+ size 4806747888
model-00036-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:29c0f382e6fb159c4a3e5b9bbdc8c6f82153c7d930a7b931756388c41b8b58c1
3
+ size 4831938528
model-00037-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b679e2a79378066fee777033609a828199e2171c1dfd8502be91f83455d96ac9
3
+ size 4831938536
model-00038-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bd02cdf9bc09935bd985693d6f4e574ccbfab5490be148dcbcb404a0a8ae2083
3
+ size 4907411080
model-00039-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:40d5f40fc5662c16990ba6221d3c0da9e31e286712cf66ec6c475148c8c062cf
3
+ size 4806747888
model-00040-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c4e664b8c40069e04ef58f619dc0d7fe67cd373cefbfd2832661332b3737b597
3
+ size 4831938528
model-00041-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ba345b6f2fd34245814f4f00bb5c3554c843bb6c6f338a512b896f0647ba40b0
3
+ size 4831938536
model-00042-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:610857d0068536623f6d4f55673c6c6728910aa364ef479db7dfb1ce677a40c6
3
+ size 4907411080
model-00043-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2dc72050121a4cf54e57a469ca94613ad317b9df6b3cb92e3ab9ef2cabb7a693
3
+ size 4806747888
model-00044-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c2e7351828b7c222b83306c7ea1dd49465a6bca22f1c76422f3af1e7f6a8eb46
3
+ size 4831938528
model-00045-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:49d47ccf9976ad2215e204223dccd8d82c3e99f93d3b8294555f577cec938a45
3
+ size 4831938536
model-00046-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:786cbac6c4907ef0f5ea109dcf5bda9c4c1900724b8f0a0dca3a40c0c3c29f7f
3
+ size 4907411072
model-00047-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:12beaee752b784fd60bf617fff0fc92e9f0671b5e2609dc0830e5a4851c87079
3
+ size 4806747888