Model Card for OLMo-1B-instruct-alpaca_amc

OLMo-1B-instruct-alpaca_amc is a language model fine-tuned on the Alpaca-GPT4 dataset. This model extends the capabilities of the original OLMo 1B model, by the Allen Institute for AI (AI2), incorporating the ability to follow instructions and perform chatting tasks.

Model Description

OLMo-1B-instruct-alpaca_amc is built upon the OLMo 1B architecture, a Transformer-based autoregressive language model known for its deep understanding and generation of natural language. By fine-tuning OLMo 1B with the Alpaca-GPT4 dataset, we've tailored the model specifically for interactive chatting applications, enabling it to understand and respond to a wide range of conversational prompts and instructions.

How to Use

To use OLMo-1B-instruct-alpaca_amc for generating text or chatting, you can leverage the Hugging Face Transformers library. Here's a quick example in Python:

import hf_olmo
from transformers import AutoModelForCausalLM, AutoTokenizer

# Model and tokenizer directories
tokenizer = AutoTokenizer.from_pretrained("amc-madalin/OLMo-1B-instruct-alpaca_amc")
model = AutoModelForCausalLM.from_pretrained("amc-madalin/OLMo-1B-instruct-alpaca_amc")

print("Chat with the model (type 'quit' to stop):")
while True:
    message = input("You: ")
    if message.lower() == 'quit':
        break

    inputs = tokenizer(message, return_tensors='pt', return_token_type_ids=False, padding=True, truncation=True, max_length=512)
    response = model.generate(**inputs, max_length=512, pad_token_id=tokenizer.eos_token_id, eos_token_id=tokenizer.eos_token_id, do_sample=True, top_k=50, top_p=0.95)
    reply = tokenizer.decode(response[0], skip_special_tokens=True)
    print("AI: ", reply)

Training Data

The model was fine-tuned on the Alpaca-GPT4 dataset, which consists of 52K instruction-following demonstrations generated in the style of self-instruct using GPT-4.

Applications

OLMo-1B-instruct-alpaca_amc is great for research and experimentation with Open LLMs.

Conversational agents
Interactive storytelling
Educational tool

Limitations and Biases

As with any language model, OLMo-1B-instruct-alpaca_amc inherits biases present in its training data. Users should be aware of these potential biases and limitations.

How to Contribute

I welcome contributions to improve the model and expand its applications!

Citation

If you use OLMo-1B-instruct-alpaca_amc in your research, please cite the original OLMo paper and the dataset used for fine-tuning:

@article{Groeneveld2023OLMo,
  title={OLMo: Accelerating the Science of Language Models},
  author={Groeneveld, Dirk and Beltagy, Iz and Walsh, Pete and Bhagia, Akshita and Kinney, Rodney and Tafjord, Oyvind and Jha, Ananya Harsh and Ivison, Hamish and Magnusson, Ian and Wang, Yizhong and Arora, Shane and Atkinson, David and Authur, Russell and Chandu, Khyathi and Cohan, Arman and Dumas, Jennifer and Elazar, Yanai and Gu, Yuling and Hessel, Jack and Khot, Tushar and Merrill, William and Morrison, Jacob and Muennighoff, Niklas and Naik, Aakanksha and Nam, Crystal and Peters, Matthew E. and Pyatkin, Valentina and Ravichander, Abhilasha and Schwenk, Dustin and Shah, Saurabh and Smith, Will and Subramani, Nishant and Wortsman, Mitchell and Dasigi, Pradeep and Lambert, Nathan and Richardson, Kyle and Dodge, Jesse and Lo, Kyle and Soldaini, Luca and Smith, Noah A. and Hajishirzi, Hannaneh},
  journal={Preprint},
  year={2024}
}