Update README.md

README.md

@@ -30,6 +30,38 @@ We present the dev results on SQuAD 1.1/2.0 and MNLI tasks.
 | DeBERTa-v3-small+SiFT  | -/- | -/- | 88.8   |
 
 
+#### Fine-tuning with HF transformers
+
+```bash
+#!/bin/bash
+pip install datasets
+export TASK_NAME=mnli
+
+output_dir="ds_results"
+
+num_gpus=8
+
+batch_size=8
+
+python -m torch.distributed.launch --nproc_per_node=${num_gpus} \
+  run_glue.py \
+  --model_name_or_path microsoft/deberta-v3-small \
+  --task_name $TASK_NAME \
+  --do_train \
+  --do_eval \
+  --evaluation_strategy steps \
+  --max_seq_length 256 \
+  --warmup_steps 1500 \
+  --per_device_train_batch_size ${batch_size} \
+  --learning_rate 3e-5 \
+  --num_train_epochs 4 \
+  --output_dir $output_dir \
+  --overwrite_output_dir \
+  --logging_steps 1000 \
+  --logging_dir $output_dir
+
+```
+
 ### Citation
 
 If you find DeBERTa useful for your work, please cite the following paper: