Quantization made by Richard Erkhov.

llama-3-8b-gpt-4o-ru1.0 - GGUF

Model creator: https://huggingface.co/ruslandev/
Original model: https://huggingface.co/ruslandev/llama-3-8b-gpt-4o-ru1.0/

Name	Quant method	Size
llama-3-8b-gpt-4o-ru1.0.Q2_K.gguf	Q2_K	2.96GB
llama-3-8b-gpt-4o-ru1.0.IQ3_XS.gguf	IQ3_XS	3.28GB
llama-3-8b-gpt-4o-ru1.0.IQ3_S.gguf	IQ3_S	3.43GB
llama-3-8b-gpt-4o-ru1.0.Q3_K_S.gguf	Q3_K_S	3.41GB
llama-3-8b-gpt-4o-ru1.0.IQ3_M.gguf	IQ3_M	3.52GB
llama-3-8b-gpt-4o-ru1.0.Q3_K.gguf	Q3_K	3.74GB
llama-3-8b-gpt-4o-ru1.0.Q3_K_M.gguf	Q3_K_M	3.74GB
llama-3-8b-gpt-4o-ru1.0.Q3_K_L.gguf	Q3_K_L	4.03GB
llama-3-8b-gpt-4o-ru1.0.IQ4_XS.gguf	IQ4_XS	4.18GB
llama-3-8b-gpt-4o-ru1.0.Q4_0.gguf	Q4_0	4.34GB
llama-3-8b-gpt-4o-ru1.0.IQ4_NL.gguf	IQ4_NL	4.38GB
llama-3-8b-gpt-4o-ru1.0.Q4_K_S.gguf	Q4_K_S	4.37GB
llama-3-8b-gpt-4o-ru1.0.Q4_K.gguf	Q4_K	4.58GB
llama-3-8b-gpt-4o-ru1.0.Q4_K_M.gguf	Q4_K_M	4.58GB
llama-3-8b-gpt-4o-ru1.0.Q4_1.gguf	Q4_1	4.78GB
llama-3-8b-gpt-4o-ru1.0.Q5_0.gguf	Q5_0	5.21GB
llama-3-8b-gpt-4o-ru1.0.Q5_K_S.gguf	Q5_K_S	5.21GB
llama-3-8b-gpt-4o-ru1.0.Q5_K.gguf	Q5_K	5.34GB
llama-3-8b-gpt-4o-ru1.0.Q5_K_M.gguf	Q5_K_M	5.34GB
llama-3-8b-gpt-4o-ru1.0.Q5_1.gguf	Q5_1	5.65GB
llama-3-8b-gpt-4o-ru1.0.Q6_K.gguf	Q6_K	6.14GB
llama-3-8b-gpt-4o-ru1.0.Q8_0.gguf	Q8_0	7.95GB

Original model description:

license: llama3 base_model: meta-llama/Meta-Llama-3-8B-Instruct tags: - generated_from_trainer model-index: - name: >- home/ubuntu/llm_training/axolotl/llama3-8b-gpt-4o-ru/output_llama3_8b_gpt_4o_ru results: [] datasets: - ruslandev/tagengo-rus-gpt-4o

Llama-3 8B GPT-4o-RU1.0

[Dataset]

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct. The idea behind this model is to train on a dataset derived from a smaller subset of the tagengo-gpt4, but with improved data quality. I tried to achieve higher data quality by prompting GPT-4o, the latest OpenAI's LLM with better multilingual capabilities. The training objective is primarily focused on the Russian language (80% of the training examples). After training for 1 epoch on 2 NVIDIA A100 the model shows promising results on the MT-Bench evaluation benchmark, surpassing GPT-3.5-turbo and being on par with Suzume in Russian language scores, even though the latter is trained on 8x bigger and more diverse dataset.

How to use

The easiest way to use this model on your own computer is to use the GGUF version of this model (ruslandev/llama-3-8b-gpt-4o-ru1.0-gguf) using a program such as llama.cpp. If you want to use this model directly with the Huggingface Transformers stack, I recommend using my framework gptchain.

git clone https://github.com/RuslanPeresy/gptchain.git
cd gptchain
pip install -r requirements-train.txt
python gptchain.py chat -m ruslandev/llama-3-8b-gpt-4o-ru1.0 \
    --chatml true \
    -q '[{"from": "human", "value": "Из чего состоит нейронная сеть?"}]'

Evaluation scores

I achieved the following scores on Ru/En MT-Bench:

	meta-llama/Meta-Llama-3-8B-Instruct	ruslandev/llama-3-8b-gpt-4o-ru1.0	lightblue/suzume-llama-3-8B-multilingual	Nexusflow/Starling-LM-7B-beta	gpt-3.5-turbo
Russian 🇷🇺	NaN	8.12	8.19	8.06	7.94
English 🇺🇸	7.98	8.01	7.73	7.92	8.26

Training procedure

See axolotl config

axolotl version: 0.4.1

base_model: meta-llama/Meta-Llama-3-8B-Instruct
model_type: LlamaForCausalLM
tokenizer_type: AutoTokenizer  # PreTrainedTokenizerFast

load_in_8bit: false
load_in_4bit: false
strict: false

datasets:
  - path: ruslandev/tagengo-rus-gpt-4o
    type: sharegpt
    conversation: llama-3
dataset_prepared_path: /home/ubuntu/llm_training/axolotl/llama3-8b-gpt-4o-ru/prepared_tagengo_rus
val_set_size: 0.01
output_dir: /home/ubuntu/llm_training/axolotl/llama3-8b-gpt-4o-ru/output_llama3_8b_gpt_4o_ru

sequence_len: 8192
sample_packing: true
pad_to_sequence_len: true
eval_sample_packing: false

use_wandb: false
#wandb_project: axolotl
#wandb_entity: wandb_entity
#wandb_name: llama_3_8b_gpt_4o_ru

gradient_accumulation_steps: 2
micro_batch_size: 2
num_epochs: 1
optimizer: paged_adamw_8bit
lr_scheduler: cosine
learning_rate: 1e-5

train_on_inputs: false
group_by_length: false
bf16: auto
fp16:
tf32: false

gradient_checkpointing: true
gradient_checkpointing_kwargs:
  use_reentrant: false
early_stopping_patience:
resume_from_checkpoint:
logging_steps: 1
xformers_attention:
flash_attention: true

warmup_steps: 10
evals_per_epoch: 5
eval_table_size:
saves_per_epoch: 1
debug:
deepspeed: /home/ubuntu/axolotl/deepspeed_configs/zero2.json
weight_decay: 0.0
special_tokens:
  pad_token: <|end_of_text|>

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-05
train_batch_size: 2
eval_batch_size: 2
seed: 42
distributed_type: multi-GPU
num_devices: 2
gradient_accumulation_steps: 2
total_train_batch_size: 8
total_eval_batch_size: 4
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 10
num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss
1.1347	0.016	1	1.1086
0.916	0.208	13	0.8883
0.8494	0.416	26	0.8072
0.8657	0.624	39	0.7814
0.8077	0.832	52	0.7702

Framework versions

Transformers 4.41.1
Pytorch 2.2.2+cu121
Datasets 2.19.1
Tokenizers 0.19.1