README.md · rozek/OpenLLaMA_7B_300BT

metadata

license: apache-2.0
datasets:
  - togethercomputer/RedPajama-Data-1T
tags:
  - open-llama
  - ggml
  - 7b
  - quantized
  - text-generation

This repository contains the 300BT preview of the Open LLaMA model based on the RedPajama 1T dataset, converted to GGML and quantized to 4 bit, ready to be used with llama.cpp

Usage

In order to use this model with llama.cpp

install llama.cpp as described in the docs
download this model
move it into the models subfolder of llama.cpp
run inferences with the additional parameter -m ./models/7B/ggml-openllama-7b-300bt-q4_0.bin

For example,

./main -m ./models/7B/ggml-openllama-7b-300bt-q4_0.bin -n 128 -p "who was Joseph Weizenbaum?"

Since the underlying model is itself Apache 2.0 licensed, this repo is as well.