Hugging Face's logo

schnapper79
/
lumikabra-123B_v0.1

Text Generation
Transformers
Safetensors
mistral
mergekit
lumikabra-123B
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
schnapper79 commited on
Commit
2f00c15
•
1 Parent(s): 369b481

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. README.md +50 -0
  2. config.json +27 -0
  3. mergekit_config.yml +20 -0
  4. model-00001-of-00051.safetensors +3 -0
  5. model-00002-of-00051.safetensors +3 -0
  6. model-00003-of-00051.safetensors +3 -0
  7. model-00004-of-00051.safetensors +3 -0
  8. model-00005-of-00051.safetensors +3 -0
  9. model-00006-of-00051.safetensors +3 -0
  10. model-00007-of-00051.safetensors +3 -0
  11. model-00008-of-00051.safetensors +3 -0
  12. model-00009-of-00051.safetensors +3 -0
  13. model-00010-of-00051.safetensors +3 -0
  14. model-00011-of-00051.safetensors +3 -0
  15. model-00012-of-00051.safetensors +3 -0
  16. model-00013-of-00051.safetensors +3 -0
  17. model-00014-of-00051.safetensors +3 -0
  18. model-00015-of-00051.safetensors +3 -0
  19. model-00016-of-00051.safetensors +3 -0
  20. model-00017-of-00051.safetensors +3 -0
  21. model-00018-of-00051.safetensors +3 -0
  22. model-00019-of-00051.safetensors +3 -0
  23. model-00020-of-00051.safetensors +3 -0
  24. model-00021-of-00051.safetensors +3 -0
  25. model-00022-of-00051.safetensors +3 -0
  26. model-00023-of-00051.safetensors +3 -0
  27. model-00024-of-00051.safetensors +3 -0
  28. model-00025-of-00051.safetensors +3 -0
  29. model-00026-of-00051.safetensors +3 -0
  30. model-00027-of-00051.safetensors +3 -0
  31. model-00028-of-00051.safetensors +3 -0
  32. model-00029-of-00051.safetensors +3 -0
  33. model-00030-of-00051.safetensors +3 -0
  34. model-00031-of-00051.safetensors +3 -0
  35. model-00032-of-00051.safetensors +3 -0
  36. model-00033-of-00051.safetensors +3 -0
  37. model-00034-of-00051.safetensors +3 -0
  38. model-00035-of-00051.safetensors +3 -0
  39. model-00036-of-00051.safetensors +3 -0
  40. model-00037-of-00051.safetensors +3 -0
  41. model-00038-of-00051.safetensors +3 -0
  42. model-00039-of-00051.safetensors +3 -0
  43. model-00040-of-00051.safetensors +3 -0
  44. model-00041-of-00051.safetensors +3 -0
  45. model-00042-of-00051.safetensors +3 -0
  46. model-00043-of-00051.safetensors +3 -0
  47. model-00044-of-00051.safetensors +3 -0
  48. model-00045-of-00051.safetensors +3 -0
  49. model-00046-of-00051.safetensors +3 -0
  50. model-00047-of-00051.safetensors +3 -0
README.md ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: []
3
+ library_name: transformers
4
+ tags:
5
+ - mergekit
6
+ - merge
7
+
8
+ ---
9
+ # merge
10
+
11
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
+
13
+ ## Merge Details
14
+ ### Merge Method
15
+
16
+ This model was merged using the della_linear merge method using /workspace/text-generation-webui/models/mistralai_Mistral-Large-Instruct-2407 as a base.
17
+
18
+ ### Models Merged
19
+
20
+ The following models were included in the merge:
21
+ * /workspace/text-generation-webui/models/migtissera_Tess-3-Mistral-Large-2-123B
22
+ * /workspace/text-generation-webui/models/FluffyKaeloky_Luminum-v0.1-123B
23
+ * /workspace/text-generation-webui/models/anthracite-org_magnum-v2-123b
24
+
25
+ ### Configuration
26
+
27
+ The following YAML configuration was used to produce this model:
28
+
29
+ ```yaml
30
+ models:
31
+ - model: /workspace/text-generation-webui/models/anthracite-org_magnum-v2-123b
32
+ parameters:
33
+ weight: 0.19
34
+ density: 0.5
35
+ - model: /workspace/text-generation-webui/models/FluffyKaeloky_Luminum-v0.1-123B
36
+ parameters:
37
+ weight: 0.34
38
+ density: 0.8
39
+ - model: /workspace/text-generation-webui/models/migtissera_Tess-3-Mistral-Large-2-123B
40
+ parameters:
41
+ weight: 0.24
42
+ density: 0.7
43
+ merge_method: della_linear
44
+ base_model: /workspace/text-generation-webui/models/mistralai_Mistral-Large-Instruct-2407
45
+ parameters:
46
+ epsilon: 0.05
47
+ lambda: 1
48
+ int8_mask: true
49
+ dtype: bfloat16
50
+ ```
config.json ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "/workspace/text-generation-webui/models/mistralai_Mistral-Large-Instruct-2407",
3
+ "architectures": [
4
+ "MistralForCausalLM"
5
+ ],
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": 1,
8
+ "eos_token_id": 2,
9
+ "head_dim": 128,
10
+ "hidden_act": "silu",
11
+ "hidden_size": 12288,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 28672,
14
+ "max_position_embeddings": 131072,
15
+ "model_type": "mistral",
16
+ "num_attention_heads": 96,
17
+ "num_hidden_layers": 88,
18
+ "num_key_value_heads": 8,
19
+ "rms_norm_eps": 1e-05,
20
+ "rope_theta": 1000000.0,
21
+ "sliding_window": null,
22
+ "tie_word_embeddings": false,
23
+ "torch_dtype": "bfloat16",
24
+ "transformers_version": "4.44.2",
25
+ "use_cache": true,
26
+ "vocab_size": 32768
27
+ }
mergekit_config.yml ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ models:
2
+ - model: /workspace/text-generation-webui/models/anthracite-org_magnum-v2-123b
3
+ parameters:
4
+ weight: 0.19
5
+ density: 0.5
6
+ - model: /workspace/text-generation-webui/models/FluffyKaeloky_Luminum-v0.1-123B
7
+ parameters:
8
+ weight: 0.34
9
+ density: 0.8
10
+ - model: /workspace/text-generation-webui/models/migtissera_Tess-3-Mistral-Large-2-123B
11
+ parameters:
12
+ weight: 0.24
13
+ density: 0.7
14
+ merge_method: della_linear
15
+ base_model: /workspace/text-generation-webui/models/mistralai_Mistral-Large-Instruct-2407
16
+ parameters:
17
+ epsilon: 0.05
18
+ lambda: 1
19
+ int8_mask: true
20
+ dtype: bfloat16
model-00001-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3be216c26d7fc7ff0387f68212ed004ae6014bc01e39f901a1cc1887c59ed2f0
3
+ size 4378928504
model-00002-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ee59777efd4a1ccca5016ab135157a4e5100e5b660450bfd5dde19243dbe910e
3
+ size 4907411088
model-00003-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:32c88340dbb42a25def4b0768003af2e0075d51d54b2d74beed1e2ee943cdc47
3
+ size 4806747904
model-00004-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4864ef283a235b77dc80f656dc0af64ca9174c854bbd3ec330468275be9fa324
3
+ size 4831938544
model-00005-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4b7a29c9b65ed6b92e2a71baef5d82692e79173e92416f0db35341928fb07f68
3
+ size 4831938552
model-00006-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cbcbef4bcd4079d27f2562c136fc99b8f5c9993efd21f8fc5f8518a53934ddd6
3
+ size 4907411096
model-00007-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2b1a25db7cd1e3c21048556c09599d2ccb367c5f30f9c33a5b29cff73781b76a
3
+ size 4806747904
model-00008-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a2497af6a6c5372919491738656d59e8dd68bc979c1eeab25bb0928594ebb2d3
3
+ size 4831938536
model-00009-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e75e3b3f4501f501c88ce46b01e8e455a9e6194da6d60cdc002d24eab290eaea
3
+ size 4831938552
model-00010-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:64a557a17b363295a1e061d7cce74f41ae3e82751a08d04d3ce971b0bd983343
3
+ size 4907411096
model-00011-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4bb5ec0273e81068871e25905f11cc3c68210f1d0173f9909a76e0331f38f358
3
+ size 4806747904
model-00012-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:102c4efebe044ac36aa87241d8ae9d459999b32304b8efed4b14204e5f235063
3
+ size 4831938544
model-00013-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7292f7ba1c062c23b075714e4ada8c254b79bba37c75ce6651beeb670dbeb876
3
+ size 4831938552
model-00014-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5574c3052e1e830c393a63cbdcddaecf7a9f22766d73c9d06945bdde2733200d
3
+ size 4907411088
model-00015-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b1c3ecbfc5f19f17d7573138c1ea8afd873d5739616ca111d0faadb89a1d5d6a
3
+ size 4806747904
model-00016-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:55ae186b58b21dd93d9173f390a8c89bbe3d449e9fef92ae6e161ee09d5e6cd3
3
+ size 4831938544
model-00017-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c77b59eafa7d5aca9e79bfa653f68c95e15f82641a22d8dd6eac4faae5e6bd7c
3
+ size 4831938552
model-00018-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1175892df68a31f74bccdb1ef1e0a02880d2b68e12204625226ed59b257624f5
3
+ size 4907411096
model-00019-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e1dd40c0675c0ff540cbf13b4a7f1a7d90422e7eff19292646b7f2d8a62a6b71
3
+ size 4806747904
model-00020-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7109deb2105ae3e6bf9eadb922ecf58cd76a000e6833bb43dd979e685b491e85
3
+ size 4831938544
model-00021-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:56c1058aed623afbed60e57bd03d0981a448a6279bd74208ddff384b1eec6a3b
3
+ size 4831938544
model-00022-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2ebb73b1b1c446e11eae4cc03062d235bb1891274f5c4265c41b19f27c40eefa
3
+ size 4907411096
model-00023-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a6d0321de5cfa92fc2928d564068d2966c1bfff518da298a3bce66ae39863cbc
3
+ size 4806747904
model-00024-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d26d8e7b8fb8f022f1e93bd0edd7b1d15c7ebab4ce8736d44d3b5cbd4aea56de
3
+ size 4831938544
model-00025-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:035ed22d6447a5af2490b436fd46af94ddcacb8bd6e1c5be42b084241a6f0b0c
3
+ size 4831938552
model-00026-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1adfbce942e7e5072bff6ad542e1e2967cb8f447155a8c73881217021fc50c0d
3
+ size 4907411096
model-00027-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:908b3aa90a4ba1cb7f89fbf4dd154477d1b127017698b32c1a3684589f8c501d
3
+ size 4806747896
model-00028-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9761eba161d9e3e8d7b62c6fe332ba9b7b873a148bd8f0388bd3aa3988eed188
3
+ size 4831938544
model-00029-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:667248bdba655ea6f94541b3ae38673a41ba3933500e6cbd58958d325c10cd11
3
+ size 4831938552
model-00030-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:005f02d170681f989428c84a91fbfb3e4cf0101f83db2714350f2b44548b1955
3
+ size 4907411096
model-00031-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:70054d408f9b7d14d282b11f4b747b8cf2229b2b010c12ef1580f4405f70f43c
3
+ size 4806747904
model-00032-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3fc2a9f6eff145f83befa6f69c0b1e80f7ebc07c0965098673c298002ec746bf
3
+ size 4831938544
model-00033-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c0a6869d6c6d1601148bd3998fc1cd67b172a61ef008aaa6d6b6c6c3ee25ece1
3
+ size 4831938544
model-00034-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:54c75d7429a76ab2f72e068f08013cbff582a287bf213408f38bf5b5019f46aa
3
+ size 4907411096
model-00035-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2ad56c3dd7dbe5ac1526fc93a603abf03d0d80979d9251e6ad12eee5d66f566a
3
+ size 4806747904
model-00036-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d0cac9be84fb6e9f960faecf1588525d422f430f7063a789fe645666715c2a80
3
+ size 4831938544
model-00037-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9b09ac3ded408ba9dc88512a34967e09139cb667566e56ae9ecd40c131e0b3c5
3
+ size 4831938552
model-00038-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a3b582483ea5e10d7bde71356af4e7baaa0ee8f0a39d8e3e41b534142753d43e
3
+ size 4907411096
model-00039-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c989fa16a51ca65e8bebeff87e2372723eed717d66bd5360e8e05f41c7488838
3
+ size 4806747904
model-00040-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0c18946c2beb42aa4c3bd4fc45052346d5ef251adb375735aa3c60d19ab99dd7
3
+ size 4831938544
model-00041-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fbbc08c38a16c2721f9c55b314dd3209d974747de0e1d15a5eb6dbb6e6d9b8dd
3
+ size 4831938552
model-00042-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ee4a85b9ef52a64e3da6c77b690faae201f7294a337271734d816df64861ba59
3
+ size 4907411096
model-00043-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:03f110c1a314fc14c308489dbe997c255b8153001597c7300ae20f4d174f2792
3
+ size 4806747904
model-00044-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:746fc3314b50f4754eea8d39022627ea47ea81950dddb64ea0622507b2eb8f05
3
+ size 4831938544
model-00045-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6c2429d31017b26898b219486fe0c05fe3a1fa771dca8cfb23885bba20da321d
3
+ size 4831938552
model-00046-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f8b7eaefb3b3839a84b0f73ecbc4616e78a9c138ccf09c42da870de10a20d893
3
+ size 4907411088
model-00047-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5433b6c89611d2b11355904705c491736b887b1b2e44043f3feea13eb5f0c8c8
3
+ size 4806747904