---
license: apache-2.0
datasets:
- Anthropic/hh-rlhf
language:
- en
pipeline_tag: text-generation
tags:
- text-generation-inference
---
# Model Card for OpenBezoar-HH-RLHF-SFT

The OpenBezoar-HH-RLHF-SFT is an LLM that has been further instruction fine tuned version of [OpenBezoar-SFT](https://huggingface.co/SurgeGlobal/OpenBezoar-SFT) model on a subset of [Anthropic's HH-RLHF Dataset](https://huggingface.co/datasets/Anthropic/hh-rlhf).

## Model Details

- Base Model: [OpenBezoar-SFT](https://huggingface.co/SurgeGlobal/OpenBezoar-SFT)
- Dataset used for SFT: First 100K examples of the [HH-RLHF](https://huggingface.co/datasets/Anthropic/hh-rlhf) dataset
- Epochs: 1

### Model Description

Primary purpose of performing SFT on [OpenBezoar-SFT](https://huggingface.co/SurgeGlobal/OpenBezoar-SFT) is to minimize the distribution shift before applying Direct Preference Optimization (DPO) for human preferences alignment. For more information please refer to our paper.

### Model Sources

- **Repository:** [More Information Needed]
- **Paper :** [More Information Needed]

## Instruction Format

We follow the typical format for instruction-based prompt templates, with a system prompt followed up by the user prompt. Both begins with a prefix and ends with two newline characters as described below. It is important to utilize this template in order to obtain best responses for instruction fine-tuning related tasks.
```
### System: {system}

### Instruction: {instruction}

### Response:
```

Notice that **no** end-of-sentence (eos) token is being appended.

## Limitations

- The model might not consistently show improved abilities to follow instructions, and it could respond inappropriately or get stuck in loops.
- This model is not aligned to human preferences and therefore it may generate harmful and uncensored content.
- Caution is urged against relying on this model for production or adjacent use-cases.

## Citation

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

**BibTeX:**

[More Information Needed]