iqwiki-kor
/

Qwen2.5-3B-MP-RM

Text Classification

Generated from Trainer

text-generation-inference

Inference Endpoints

Model card Files Files and versions Metrics Training metrics Community

Edit model card

Qwen2.5-3B-MP-RM

This model is a fine-tuned version of Qwen/Qwen2.5-3B-Instruct on the iqwiki-kor/MP-86k dataset.

RewardBench Evaluation

Model	Chat	Chat-Hard	Safety	Reasoning	Avg.
iqwiki-kor/Qwen2.5-3B-MP-RM	89.1	75.2	87.3	95.4	86.8
RLHFlow/ArmoRM-Llama3-8B-v0.1	96.9	76.8	90.5	97.3	90.4

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-05
train_batch_size: 16
eval_batch_size: 8
seed: 42
distributed_type: multi-GPU
num_devices: 4
gradient_accumulation_steps: 2
total_train_batch_size: 128
total_eval_batch_size: 32
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.1
num_epochs: 1

Downloads last month: 388

Safetensors

Model size

3.09B params

Tensor type

BF16

·

Inference Examples

Text Classification

This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for iqwiki-kor/Qwen2.5-3B-MP-RM

Base model

Qwen/Qwen2.5-3B

Finetuned

Qwen/Qwen2.5-3B-Instruct

Finetuned

(27)

this model

Collection including iqwiki-kor/Qwen2.5-3B-MP-RM

Cross-lingual Transfer of Reward Models

This is the collection of synthetic preference data and trained reward models in "Cross-lingual Transfer of Reward Models in Multilingual Alignment". • 5 items • Updated 21 days ago

Evaluation results

Metadata error: specify a dataset to view leaderboard