license: apache-2.0
datasets:
- Anthropic/hh-rlhf
language:
- en
pipeline_tag: text-generation
tags:
- text-generation-inference
Model Card for OpenBezoar-HH-RLHF-SFT
The OpenBezoar-HH-RLHF-SFT is an LLM that has been further instruction fine tuned version of OpenBezoar-SFT model on a subset of Anthropic's HH-RLHF Dataset.
Model Details
- Base Model: OpenBezoar-SFT
- Dataset used for SFT: First 100K examples of the HH-RLHF dataset
- Epochs: 1
Model Description
Primary purpose of performing SFT on OpenBezoar-SFT is to minimize the distribution shift before applying Direct Preference Optimization (DPO) for human preferences alignment. For more information please refer to our paper.
Model Sources
- Repository: [More Information Needed]
- Paper : [More Information Needed]
Instruction Format
We follow the typical format for instruction-based prompt templates, with a system prompt followed up by the user prompt. Both begins with a prefix and ends with two newline characters as described below. It is important to utilize this template in order to obtain best responses for instruction fine-tuning related tasks.
### System: {system}
### Instruction: {instruction}
### Response:
Notice that no end-of-sentence (eos) token is being appended.
Limitations
- The model might not consistently show improved abilities to follow instructions, and it could respond inappropriately or get stuck in loops.
- This model is not aligned to human preferences and therefore it may generate harmful and uncensored content.
- Caution is urged against relying on this model for production or adjacent use-cases.
Citation
If you find our work useful, please cite our paper as follows:
[More Information Needed]