Hugging Face's logo

Deci
/
DeciLM-7B-instruct

Text Generation
Transformers
Safetensors
English
deci
conversational
custom_code
Model card Files Community
14
DeciLM-7B-instruct / README.md
harpreetsahota's picture
harpreetsahota
Update README.md
d4aa148
preview code
|
raw
history blame
5.86 kB
metadata
license: apache-2.0
language:
  - en
datasets:
  - Open-Orca/SlimOrca

DeciLM-7B-instruct

DeciLM-7B-instruct is a model for short-form instruction following. It is built by LoRA fine-tuning on the SlimOrca dataset.

Model Details

Model Description

DeciLM-7B-instruct is a derivative of the recently released DeciLM-7B language model, a pre-trained, high-efficiency generative text model with 7 billion parameters. DeciLM-7B-instruct is one the best 7B instruct models obtained using simple LoRA fine-tuning, without relying on preference optimization techniques such as RLHF and DPO.

  • Developed by: Deci
  • Model type: DeciLM is an auto-regressive language model using an optimized transformer decoder architecture that includes variable Grouped-Query Attention.
  • Language(s) (NLP): English
  • License: Apache 2.0

Model Architecture

Parameters Layers Heads Sequence Length GQA num_key_value_heads*
7.04 billion 32 32 8192 Variable

*AutoNAC was employed to optimize the selection of the GQA num_key_value_heads for each model layer.

Model Sources

Uses

The model is intended for commercial and research use in English.

How to Get Started with the Model

Use the code below to get started with the model.

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig, pipeline

model_name = "Deci/DeciLM-7B-instruct"

device = "cuda" # for GPU usage or "cpu" for CPU usage

bnb_config = BitsAndBytesConfig(
    load_in_4bit = True,
    bnb_4bit_compute_dtype=torch.bfloat16
)

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    device_map="auto",
    trust_remote_code=True,
    quantization_config=bnb_config
)

tokenizer = AutoTokenizer.from_pretrained(model_name)
tokenizer.pad_token = tokenizer.eos_token

deci_generator = pipeline("text-generation",
                          model=model,
                          tokenizer=tokenizer,
                          temperature=0.1,
                          device_map="auto",
                          max_length=4096,
                          return_full_text=False
)

prompt = "How do I make the most delicious pancakes the world has ever tasted?"

SYSTEM_PROMPT_TEMPLATE ="""
### System:
You are an AI assistant that follows instruction extremely well. Help as much as you can.
### User:
{instruction}
### Assistant:
"""

# Function to construct the prompt using the new system prompt template
def get_prompt_with_template(message: str) -> str:
    return SYSTEM_PROMPT_TEMPLATE.format(instruction=message)

response = deci_generator(get_prompt_with_template(prompt))[0]['generated_text']
print(response)

Evaluation

Below are DeciLM-7B and DeciLM-7B-instruct's evaluation results.

Model Average ARC HellaSwag MMLU TruthfulQA Winogrande GSM8K
DecilLM-7B 61.55 59.39 82.51 59.76 40.33 79.95 47.38
DecilLM-7B-instruct 63.19 61.01 82.37 60.24 49.75 79.72 46.02

Runtime Benchmarks

Inference Tool Hardware Prompt length Generation length Generated tokens/sec Batch Size Number of Prompts
HuggingFace (PyTorch) A100 (SXM4-80GB-400W) 512 512 1174 352 352
HuggingFace (PyTorch) A100 (SXM4-80GB-400W) 2048 2048 328 72 72
Infery-LLM A100 (SXM4-80GB-400W) 512 512 4559 1024 4096
Infery-LLM A100 (SXM4-80GB-400W) 2048 2048 3997 512 2048
Infery-LLM A10 512 512 1345 128 512
Infery-LLM A10 2048 2048 599 32 128
  • In order to replicate the results of the Hugging Face benchmarks, you can use this code example.
  • Infery-LLM, Deci's inference engine, features a suite of optimization algorithms, including selective quantization, optimized beam search, continuous batching, and custom CUDA kernels. To witness the full capabilities of Infery-LLM first-hand, we invite you to engage with our interactive demo.

Ethical Considerations and Limitations

DeciLM-7B-instruct is a new technology that comes with inherent risks associated with its use. The testing conducted so far has been primarily in English and does not encompass all possible scenarios. Like those of all large language models, DeciLM-7B's outputs are unpredictable, and the model may generate responses that are inaccurate, biased, or otherwise objectionable. Consequently, developers planning to use DeciLM-7B should undertake thorough safety testing and tuning designed explicitly for their intended applications of the model before deployment.

How to Cite

Please cite this model using this format.

@misc{DeciFoundationModels,
title = {DeciLM-7B-instruct},
author = {DeciAI Research Team},
year = {2023}
url={https://huggingface.co/Deci/DeciLM-7B-instruct},
}