chansurgeplus's picture
Updated the citation section.
fe306dc verified
|
raw
history blame
2.04 kB
metadata
license: apache-2.0
datasets:
  - Anthropic/hh-rlhf
language:
  - en
pipeline_tag: text-generation
tags:
  - text-generation-inference

Model Card for OpenBezoar-HH-RLHF-SFT

The OpenBezoar-HH-RLHF-SFT is an LLM that has been further instruction fine tuned version of OpenBezoar-SFT model on a subset of Anthropic's HH-RLHF Dataset.

Model Details

Model Description

Primary purpose of performing SFT on OpenBezoar-SFT is to minimize the distribution shift before applying Direct Preference Optimization (DPO) for human preferences alignment. For more information please refer to our paper.

Model Sources

  • Repository: [More Information Needed]
  • Paper : [More Information Needed]

Instruction Format

We follow the typical format for instruction-based prompt templates, with a system prompt followed up by the user prompt. Both begins with a prefix and ends with two newline characters as described below. It is important to utilize this template in order to obtain best responses for instruction fine-tuning related tasks.

### System: {system}

### Instruction: {instruction}

### Response:

Notice that no end-of-sentence (eos) token is being appended.

Limitations

  • The model might not consistently show improved abilities to follow instructions, and it could respond inappropriately or get stuck in loops.
  • This model is not aligned to human preferences and therefore it may generate harmful and uncensored content.
  • Caution is urged against relying on this model for production or adjacent use-cases.

Citation

If you find our work useful, please cite our paper as follows:

[More Information Needed]