🎭 Coding Models
Collection
Fine-tunes/Merges of Coding Models.
•
3 items
•
Updated
•
2
Starcoder-2-chat
Starcoder-2-chat is an instruction fine-tuned of bigcode/starcoder2-7b using the glaiveai/glaive-code-assistant-v2 dataset using LoRA.
🏆 Evaluation results
Thanks to Muhammad Bin Usman for running evals on Starcoder2-chat.
HUMANEVAL
0.3231707317073171
HUMANEVALPLUS
0.25609756097560976
INSTRUCT-HUMANEVAL
0.3231707317073171
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-7
- train_batch_size: 2
- eval_batch_size: Not specified
- seed: Not specified
- gradient_accumulation_steps: 8
- total_train_batch_size: Not specified
- optimizer: PagedAdamW with 32-bit precision
- lr_scheduler_type: Cosine
- lr_scheduler_warmup_steps: 100
- training_epoch: 1
Framework versions
- Transformers 4.39.0.dev0
- Peft 0.9.1.dev0
- Datasets 2.18.0
- torch 2.2.0
- accelerate 0.27.2
- Downloads last month
- 629
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.
Model tree for abideen/starcoder2-chat
Base model
bigcode/starcoder2-7b