metadata
license: mit
datasets:
- PKU-Alignment/BeaverTails
language:
- en
base_model:
- PKU-Alignment/alpaca-7b-reproduced
This model is for the reproduction of results on Safe-RLHF dataset of paper "The crucial role of samplers in online direct preference optimization". You can download it and save it as "models/rlhflow_iter1", and then start the training pipeline. Since we've retrained the models, the results may slightly differ from that reported in the paper, and we will update it later.