Retrain with new upstreams; R=32

Retrain with new upstreams

Merge branch 'no_merge'

New model from the upstream. R=24 with only QV tuned.

Added a non-merged model to avoid 16GB overflow.

Newly merged model: R=24.

Final Model v3.3: LoRA R=16 with lm_head finetuned; merged model in inference.

Final Model v3.2: LoRA R=16 with lm_head finetuned.

Final Model v3.1: LoRA R=16 with lm_head finetuned.

Final Model v3: LoRA R=16 with lm_head finetuned.

Final Model v2: LoRA R=16 with OpenAI guided corpus+advanced delimiter.

Final Model: LoRA R=16 with OpenAI guided corpus.

Last model: LoRA R=16 with improved corpus.

Last model: LoRA R=16 with 3x corpus.

New model, 3 epochs with 100k datasets.

New lora model with 15k real thread data.

Model with LR=16 and 5x augmented data.

Initial model with LR=12 and original tweets only.

initial commit