bllossom_polyglot-12.8b_0725 This model is a fine-tuned version of EleutherAI/polyglot-ko-12.8b on LIMA dataset(translated in Korean)
Training hyperparameters The following hyperparameters were used during training:
learning_rate: 0.0002 train_batch_size: 4 eval_batch_size: 1 seed: 0 gradient_accumulation_steps: 8 total_train_batch_size: 32 optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 lr_scheduler_type: constant lr_scheduler_warmup_ratio: 0.03 training_steps: 200 Framework versions Transformers 4.31.0.dev0 Pytorch 1.13.1 Datasets 2.13.1 Tokenizers 0.13.3