| library_name: peft | |
| ## Training procedure | |
| The following `bitsandbytes` quantization config was used during training: | |
| - quant_method: gptq | |
| - bits: 4 | |
| - tokenizer: None | |
| - dataset: None | |
| - group_size: 128 | |
| - damp_percent: 0.01 | |
| - desc_act: True | |
| - sym: True | |
| - true_sequential: True | |
| - use_cuda_fp16: False | |
| - model_seqlen: None | |
| - block_name_to_quantize: None | |
| - module_name_preceding_first_block: None | |
| - batch_size: 1 | |
| - pad_token_id: None | |
| - disable_exllama: True | |
| ### Framework versions | |
| - PEFT 0.5.0 | |