|
--- |
|
language: |
|
- en |
|
license: mit |
|
tags: |
|
- text-generation-inference |
|
- transformers |
|
- mixture of experts |
|
- jamba |
|
datasets: |
|
- Severian/Internal-Knowledge-Map |
|
base_model: ai21labs/Jamba-v0.1 |
|
pipeline_tag: text-generation |
|
--- |
|
|
|
<img src="https://cdn-uploads.huggingface.co/production/uploads/64740cf7485a7c8e1bd51ac9/SwdXRoyi08neRiI8pJrYI.webp" width="500" height="500"> |
|
|
|
# Jamba-Nexus-IKM |
|
|
|
## This model has been trained on the Internal Knowledge Map dataset. |
|
|
|
--- |
|
|
|
## Example Output: |
|
|
|
``` |
|
<|startoftext|><|im_start|>system |
|
You are a helpful AI assistant. |
|
<|im_end|> |
|
<|im_start|>user |
|
How could we use cheese to reignite the sun? |
|
<|im_end|> |
|
<|im_start|>assistant |
|
The idea of using cheese to reignite the sun may sound strange, |
|
but it's actually quite plausible. Here's how it works: |
|
|
|
1. Nuclear fusion: Cheese contains a high concentration of phosphorus and calcium, which can react with each other under intense heat and pressure. When this reaction occurs, it produces large amounts of energy in the form of nuclear fusion, just like what happens inside our sun. |
|
2. Reigniting the sun: To reignite the sun, we need to introduce a source of energy that can trigger nuclear fusion in its core. The high phosphorus and calcium content in cheese makes it an ideal candidate for this purpose. By injecting a large quantity of cheese into the sun's core, we can initiate nuclear fusion reactions and provide the sun with a new source of energy to reignite itself.<|im_end|> |
|
<|endoftext|> |
|
``` |
|
|
|
--- |
|
|
|
## Inference |
|
|
|
```py |
|
!pip install -qqq transformers>=4.39.0 mamba-ssm causal-conv1d>=1.2.0 accelerate bitsandbytes --progress-bar off |
|
!pip install flash-attn --no-build-isolation |
|
|
|
import torch |
|
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig |
|
|
|
double_quant_config = BitsAndBytesConfig( |
|
load_in_4bit=True, |
|
bnb_4bit_use_double_quant=True, |
|
bnb_4bit_compute_dtype=torch.float16 |
|
) |
|
|
|
model = AutoModelForCausalLM.from_pretrained( |
|
"Severian/Jamba-Nexus-IKM-v1", |
|
device_map="auto", |
|
trust_remote_code=True, |
|
torch_dtype=torch.bfloat16, |
|
attn_implementation="flash_attention_2", |
|
quantization_config=double_quant_config, |
|
) |
|
tokenizer = AutoTokenizer.from_pretrained("Severian/Jamba-Nexus-IKM-v1") |
|
|
|
input_text = """<|im_start|>system |
|
You are a helpful AI assistant. |
|
<|im_end|> |
|
<|im_start|>user |
|
How could we use cheese to reignite the sun? |
|
<|im_end|> |
|
<|im_start|>assistant |
|
""" |
|
|
|
input_ids = tokenizer(input_text, return_tensors='pt').to(model.device)["input_ids"] |
|
|
|
outputs = model.generate(input_ids, max_new_tokens=1024, temperature=0.0, repetition_penalty=1.1) |
|
|
|
print(tokenizer.batch_decode(outputs)[0]) |
|
# <|startoftext|><|im_start|>system |
|
# You are a helpful AI assistant. |
|
# <|im_end|> |
|
# <|im_start|>user |
|
# How could we use cheese to reignite the sun? |
|
# <|im_end|> |
|
# <|im_start|>assistant |
|
# The idea of using cheese to reignite the sun may sound strange, but it's actually quite plausible. Here's how it works: 1. Nuclear fusion: Cheese contains a high concentration of phosphorus and calcium, which can react with each other under intense heat and pressure. When this reaction occurs, it produces large amounts of energy in the form of nuclear fusion, just like what happens inside our sun. 2. Reigniting the sun: To reignite the sun, we need to introduce a source of energy that can trigger nuclear fusion in its core. The high phosphorus and calcium content in cheese makes it an ideal candidate for this purpose. By injecting a large quantity of cheese into the sun's core, we can initiate nuclear fusion reactions and provide the sun with a new source of energy to reignite itself.<|im_end|> |
|
# <|endoftext|> |
|
``` |
|
|
|
``` |
|
[383/1171 33:25 < 1:09:07, 0.19 it/s, Epoch 0.33/1] |
|
Step Training Loss |
|
1 10.680900 |
|
2 10.793200 |
|
3 8.870600 |
|
4 8.817300 |
|
5 13.537700 |
|
6 14.457900 |
|
7 14.419900 |
|
8 13.235300 |
|
9 10.764000 |
|
10 10.614000 |
|
11 12.617900 |
|
12 11.241100 |
|
13 10.644600 |
|
14 11.787900 |
|
15 11.430500 |
|
16 11.913600 |
|
17 10.418000 |
|
18 9.867500 |
|
19 9.392300 |
|
20 8.825400 |
|
21 8.238000 |
|
22 8.030900 |
|
23 7.902800 |
|
24 8.247100 |
|
25 7.871800 |
|
26 7.040200 |
|
27 8.326700 |
|
28 7.478000 |
|
29 6.724300 |
|
30 6.646100 |
|
31 6.375500 |
|
32 6.677100 |
|
33 7.157500 |
|
34 5.913300 |
|
35 6.432800 |
|
36 6.342500 |
|
37 5.987400 |
|
38 5.893300 |
|
39 5.194400 |
|
40 5.260600 |
|
41 5.697200 |
|
42 5.065100 |
|
43 4.868600 |
|
44 5.102600 |
|
45 4.660700 |
|
46 6.133700 |
|
47 4.706000 |
|
48 4.598300 |
|
49 4.569700 |
|
50 4.546100 |
|
51 4.799700 |
|
52 4.632400 |
|
53 4.342000 |
|
54 4.338600 |
|
55 5.103600 |
|
56 5.415300 |
|
57 5.488200 |
|
58 6.379000 |
|
59 4.440300 |
|
60 5.374200 |
|
61 5.150200 |
|
62 4.162400 |
|
63 4.020500 |
|
64 3.953600 |
|
65 4.621100 |
|
66 3.870800 |
|
67 4.863500 |
|
68 4.967800 |
|
69 3.887500 |
|
70 3.848400 |
|
71 3.681100 |
|
72 3.571800 |
|
73 3.585700 |
|
74 4.433200 |
|
75 4.752700 |
|
76 4.151600 |
|
77 3.193300 |
|
78 4.800000 |
|
79 3.036500 |
|
80 2.827300 |
|
81 4.570700 |
|
82 2.903900 |
|
83 5.724400 |
|
84 5.984600 |
|
85 4.146200 |
|
86 2.905400 |
|
87 3.950700 |
|
88 2.650200 |
|
89 3.064800 |
|
90 3.072800 |
|
91 3.083100 |
|
92 2.970900 |
|
93 4.492900 |
|
94 2.664900 |
|
95 2.507200 |
|
96 2.549800 |
|
97 2.476700 |
|
98 2.548200 |
|
99 3.978200 |
|
100 2.654500 |
|
101 2.478400 |
|
102 4.039500 |
|
103 2.201600 |
|
104 2.030600 |
|
105 1.993000 |
|
106 1.773600 |
|
107 4.248400 |
|
108 1.777600 |
|
109 3.311100 |
|
110 1.720900 |
|
111 5.827900 |
|
112 1.679600 |
|
113 3.789200 |
|
114 1.593900 |
|
115 1.241600 |
|
116 1.306900 |
|
117 5.464400 |
|
118 1.536000 |
|
119 1.328700 |
|
120 1.132500 |
|
121 1.144900 |
|
122 0.923600 |
|
123 0.690700 |
|
124 1.142500 |
|
125 5.850100 |
|
126 1.102200 |
|
127 0.939700 |
|
128 0.727700 |
|
129 3.941400 |
|
130 0.791900 |
|
131 0.662900 |
|
132 3.319800 |
|
133 0.623900 |
|
134 0.521800 |
|
135 0.375600 |
|
136 0.302900 |
|
137 0.225400 |
|
138 2.994300 |
|
139 0.214300 |
|
140 0.229000 |
|
141 2.751600 |
|
142 0.298000 |
|
143 0.227500 |
|
144 2.300500 |
|
145 0.180900 |
|
146 0.629700 |
|
147 0.420900 |
|
148 2.648600 |
|
149 1.837600 |
|
150 0.524800 |
|
... |
|
1148 0.004700 |
|
``` |
