Name fix
Browse files
README.md
CHANGED
|
@@ -2,21 +2,20 @@
|
|
| 2 |
language:
|
| 3 |
- ru
|
| 4 |
base_model:
|
| 5 |
-
- t-tech/T-
|
| 6 |
---
|
| 7 |
-
**_This is a converted version of the original [T-
|
| 8 |
|
| 9 |
|
| 10 |
# Original model card:
|
| 11 |
|
| 12 |
-
# T-
|
| 13 |
-
|
| 14 |
-
**🚨 T-lite is designed for further fine-tuning and is not intended as a ready-to-use conversational assistant. Users are advised to exercise caution and are responsible for any additional training and oversight required to ensure the model's responses meet acceptable ethical and safety standards. The responsibility for incorporating this model into industrial or commercial solutions lies entirely with those who choose to deploy it.**
|
| 15 |
|
|
|
|
| 16 |
|
| 17 |
## Description
|
| 18 |
|
| 19 |
-
T-
|
| 20 |
|
| 21 |
### 📚 Dataset
|
| 22 |
|
|
@@ -34,22 +33,38 @@ Preference Tuning:
|
|
| 34 |
|
| 35 |
## 📊 Benchmarks
|
| 36 |
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
|
| 40 |
-
|
| 41 |
-
|
|
| 42 |
-
|
|
| 43 |
-
|
|
| 44 |
-
|
|
| 45 |
-
|
| 46 |
-
|
|
| 47 |
-
|
|
| 48 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 49 |
|
| 50 |
Detailed evaluation results can be found in our [habr post](https://habr.com/ru/companies/tbank/articles/865582/)
|
| 51 |
|
| 52 |
-
|
| 53 |
## 👨💻 Examples of usage
|
| 54 |
|
| 55 |
### HF Usage
|
|
@@ -58,16 +73,16 @@ Detailed evaluation results can be found in our [habr post](https://habr.com/ru/
|
|
| 58 |
from transformers import AutoTokenizer, AutoModelForCausalLM
|
| 59 |
import torch
|
| 60 |
torch.manual_seed(42)
|
| 61 |
-
model_name = "t-tech/T-
|
| 62 |
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
| 63 |
model = AutoModelForCausalLM.from_pretrained(
|
| 64 |
model_name,
|
| 65 |
torch_dtype="auto",
|
| 66 |
-
device_map="auto"
|
| 67 |
)
|
| 68 |
prompt = "Напиши стих про машинное обучение"
|
| 69 |
messages = [
|
| 70 |
-
{"role": "system", "content": "Ты T-
|
| 71 |
{"role": "user", "content": prompt}
|
| 72 |
]
|
| 73 |
text = tokenizer.apply_chat_template(
|
|
@@ -89,22 +104,18 @@ print(response)
|
|
| 89 |
|
| 90 |
Output:
|
| 91 |
```
|
| 92 |
-
В мире
|
| 93 |
-
Машинное
|
| 94 |
-
|
| 95 |
-
|
| 96 |
-
|
| 97 |
-
|
| 98 |
-
|
| 99 |
-
|
| 100 |
-
|
| 101 |
-
|
| 102 |
-
|
| 103 |
-
|
| 104 |
-
Не бойтесь перемен, ведь это — путь,
|
| 105 |
-
Который ведёт к будущему, светлому и новому.
|
| 106 |
-
Машинное обученье — наш проводник,
|
| 107 |
-
В этом мире, где технологии цар��т.
|
| 108 |
```
|
| 109 |
|
| 110 |
### VLLM Usage
|
|
@@ -112,7 +123,7 @@ Output:
|
|
| 112 |
```python
|
| 113 |
from transformers import AutoTokenizer
|
| 114 |
from vllm import LLM, SamplingParams
|
| 115 |
-
model_name = "t-tech/T-
|
| 116 |
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
| 117 |
llm = LLM(model=model_name, max_model_len=8192)
|
| 118 |
sampling_params = SamplingParams(temperature=0.7,
|
|
@@ -120,7 +131,7 @@ sampling_params = SamplingParams(temperature=0.7,
|
|
| 120 |
top_p=0.8, top_k=70)
|
| 121 |
prompt = "Напиши стих про машинное обучение"
|
| 122 |
messages = [
|
| 123 |
-
{"role": "system", "content": "Ты T-
|
| 124 |
{"role": "user", "content": prompt}
|
| 125 |
]
|
| 126 |
prompt_token_ids = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
|
|
|
|
| 2 |
language:
|
| 3 |
- ru
|
| 4 |
base_model:
|
| 5 |
+
- t-tech/T-pro-it-1.0
|
| 6 |
---
|
| 7 |
+
**_This is a converted version of the original [T-pro-it-1.0](https://huggingface.co/t-tech/T-pro-it-1.0) model into EXL2._**
|
| 8 |
|
| 9 |
|
| 10 |
# Original model card:
|
| 11 |
|
| 12 |
+
# T-pro-it-1.0
|
|
|
|
|
|
|
| 13 |
|
| 14 |
+
**🚨 T-pro is designed for further fine-tuning and is not intended as a ready-to-use conversational assistant. Users are advised to exercise caution and are responsible for any additional training and oversight required to ensure the model's responses meet acceptable ethical and safety standards. The responsibility for incorporating this model into industrial or commercial solutions lies entirely with those who choose to deploy it.**
|
| 15 |
|
| 16 |
## Description
|
| 17 |
|
| 18 |
+
T-pro-it-1.0 is a model built upon the Qwen 2.5 model family and incorporates both continual pre-training and alignment techniques.
|
| 19 |
|
| 20 |
### 📚 Dataset
|
| 21 |
|
|
|
|
| 33 |
|
| 34 |
## 📊 Benchmarks
|
| 35 |
|
| 36 |
+
Proprietary models:
|
| 37 |
+
|
| 38 |
+
| Benchmark | T-pro-it-1.0 | GPT-4o | GPT-4o-mini | GigaChat Max 1.0.26.20 |
|
| 39 |
+
|------------------------------------------------|-----------------------|------------------------------|-----------------------|---------------------|
|
| 40 |
+
| [MERA](https://mera.a-ai.ru) | <u>0.629</u> | **0.642** | 0.57 | 0.588 |
|
| 41 |
+
| [MaMuRaMu](https://mera.a-ai.ru/ru/tasks/22) | <u>0.841</u> | **0.874** | 0.779 | 0.824 |
|
| 42 |
+
| ruMMLU-PRO | <u>0.665</u> | **0.713** | 0.573 | 0.535 |
|
| 43 |
+
| ruGSM8K | **0.941** | <u>0.931</u> | 0.888 | 0.892 |
|
| 44 |
+
| ruMATH | **0.776** | <u>0.771</u> | 0.724 | 0.589 |
|
| 45 |
+
| ruMBPP | **0.805** | <u>0.802</u> | 0.79 | 0.626 |
|
| 46 |
+
| [ruCodeEval](https://mera.a-ai.ru/ru/tasks/23) | 0.432 / 0.626 / 0.677 | <u>0.529 / 0.649 / 0.683</u> | **0.704 / 0.753 / 0.768** | 0.077 / 0.093 / 0.098 |
|
| 47 |
+
| Arena-Hard-Ru | **90.17** | <u>84.87</u> | 81 | - |
|
| 48 |
+
| MT Bench Ru | <u>8.7</u> | **8.706** | 8.45 | 8.53 |
|
| 49 |
+
| Alpaca Eval Ru | <u>47.61</u> | **50** | 45.51 | 38.13 |
|
| 50 |
+
|
| 51 |
+
Open-source models:
|
| 52 |
+
|
| 53 |
+
| Benchmark | T-pro-it-1.0 | Qwen-2.5-32B-Instruct | RuAdapt-Qwen-32B-Instruct-v1 | gemma-2-27b-it | Llama-3.3-70B-Instruct |
|
| 54 |
+
|------------------------------------------------|---------------------------|-------------------------------|------------------------------|------------------------------|------------------------|
|
| 55 |
+
| [MERA](https://mera.a-ai.ru) | **0.629** | 0.578 | <u>0.615</u> | 0.574 | 0.567 |
|
| 56 |
+
| [MaMuRaMu](https://mera.a-ai.ru/ru/tasks/22) | **0.841** | <u>0.824</u> | 0.812 | 0.768 | 0.818 |
|
| 57 |
+
| ruMMLU-PRO | **0.665** | 0.637 | 0.631 | 0.470 | <u>0.653</u> |
|
| 58 |
+
| ruGSM8K | **0.941** | 0.926 | 0.923 | 0.894 | <u>0.934</u> |
|
| 59 |
+
| ruMATH | **0.776** | 0.727 | <u>0.742</u> | 0.538 | 0.636 |
|
| 60 |
+
| ruMBPP | 0.805 | **0.825** | <u>0.813</u> | 0.708 | 0.77 |
|
| 61 |
+
| [ruCodeEval](https://mera.a-ai.ru/ru/tasks/23) | **0.432 / 0.626 / 0.677** | 0.06 / 0.098 / 0.116 | 0.426 / 0.561 / 0.598 | <u>0.259 / 0.586 / 0.689</u> | 0.112 / 0.166 / 0.189 |
|
| 62 |
+
| Arena-Hard-Ru | **90.17** | 74.54 | <u>80.23</u> | 66.4 | 76.51 |
|
| 63 |
+
| MT Bench Ru | **8.7** | 8.15 | 8.39 | 7.96 | <u>8.26</u> |
|
| 64 |
+
| Alpaca Eval Ru | **47.61** | 35.01 | <u>43.15</u> | 38.82 | - |
|
| 65 |
|
| 66 |
Detailed evaluation results can be found in our [habr post](https://habr.com/ru/companies/tbank/articles/865582/)
|
| 67 |
|
|
|
|
| 68 |
## 👨💻 Examples of usage
|
| 69 |
|
| 70 |
### HF Usage
|
|
|
|
| 73 |
from transformers import AutoTokenizer, AutoModelForCausalLM
|
| 74 |
import torch
|
| 75 |
torch.manual_seed(42)
|
| 76 |
+
model_name = "t-tech/T-pro-it-1.0"
|
| 77 |
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
| 78 |
model = AutoModelForCausalLM.from_pretrained(
|
| 79 |
model_name,
|
| 80 |
torch_dtype="auto",
|
| 81 |
+
device_map="auto",
|
| 82 |
)
|
| 83 |
prompt = "Напиши стих про машинное обучение"
|
| 84 |
messages = [
|
| 85 |
+
{"role": "system", "content": "Ты T-pro, виртуальный ассистент в Т-Технологии. Твоя задача - быть полезным диалоговым ассистентом."},
|
| 86 |
{"role": "user", "content": prompt}
|
| 87 |
]
|
| 88 |
text = tokenizer.apply_chat_template(
|
|
|
|
| 104 |
|
| 105 |
Output:
|
| 106 |
```
|
| 107 |
+
В мире данных и алгоритмов, где путь просветления лежит,
|
| 108 |
+
Машинное обучение — как звезда, что светом знаний сияет.
|
| 109 |
+
Слои нейронов, как мозг огромный, в цифровой тишине дремлют,
|
| 110 |
+
Изучают закономерности, скрытые в числах глубоко.
|
| 111 |
+
Оно учится на примерах, как ребёнок, открывая мир,
|
| 112 |
+
На ошибках своих корректируясь, шаг за шагом к совершенству стремится.
|
| 113 |
+
Где раньше требовалась рука человека, теперь сеть сама решает,
|
| 114 |
+
Прогнозы точные строит, решения сложные принимает.
|
| 115 |
+
В облаках данных, как корабль, плывёт через шторм и спокойствие,
|
| 116 |
+
Поиск закономерностей — его цель, открыть тайны бытия.
|
| 117 |
+
От распознавания лиц до понимания речи,
|
| 118 |
+
Машинное обучение — это ключ, что открывает двери.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 119 |
```
|
| 120 |
|
| 121 |
### VLLM Usage
|
|
|
|
| 123 |
```python
|
| 124 |
from transformers import AutoTokenizer
|
| 125 |
from vllm import LLM, SamplingParams
|
| 126 |
+
model_name = "t-tech/T-pro-it-1.0"
|
| 127 |
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
| 128 |
llm = LLM(model=model_name, max_model_len=8192)
|
| 129 |
sampling_params = SamplingParams(temperature=0.7,
|
|
|
|
| 131 |
top_p=0.8, top_k=70)
|
| 132 |
prompt = "Напиши стих про машинное обучение"
|
| 133 |
messages = [
|
| 134 |
+
{"role": "system", "content": "Ты T-pro, виртуальный ассистент в Т-Технологии. Твоя задача - быть полезным диалоговым ассистентом."},
|
| 135 |
{"role": "user", "content": prompt}
|
| 136 |
]
|
| 137 |
prompt_token_ids = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
|