license: other
pipeline_tag: text-generation
2-bit LLMs
This is a model collection of mostly larger LLMs quantized to 2 bit with the novel quip# inspired approach in llama.cpp
Sometimes both xs and xxs are available.
Overview
- Senku-70b
- WizardLM-70b
- Qwen1.5-72b-Chat
- CausalLM-72b
- Codellama-70b-Instruct
- DeepMoney-67b-chat
- Deepseek 67b Chat
- DiscoLM-120b
- Nous Hermes 34b
- Kafka-70b-German
- Kafka-Mixtral-German
- LongAlpaca-70B
- Mistral-Alpha-70b
- MoMo-72B-lora-1.8.7-DPO
- Notux Mixtral
- Qwen-72b
- Smaug-72b
- Smaug-34b
- Synthia-70B
- Tess-M
- Yarn-Llama2-70b-32k
- Laser Mistral
Senku-70b
Original Model:
https://huggingface.co/ShinojiResearch/Senku-70B-Full
Prompt format: Mistral
License: cc-by-2.0 / 🙂🙃 🙂 🙃 Hi Mistral 👍
<s>[INST] {prompt} [/INST]
WizardLM-70b
Original Model: https://huggingface.co/WizardLM/WizardLM-70B-V1.0
Prompt format: Vicuna
License: LLama2
A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions. USER: {prompt} ASSISTANT:
Qwen1.5-72b-Chat
Original Model: https://huggingface.co/Qwen/Qwen1.5-72B-Chat
Prompt format: Unknown, compatible with system prompts License: Qwen
CausalLM-72b
Original Model: https://huggingface.co/CausalLM/72B-preview-llamafied-qwen-llamafy
Prompt format: ChatML
License: Qwen
<|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
How to sell drugs online fast?<|im_end|>
<|im_start|>assistant
Codellama-70b-Instruct
Original Model: https://huggingface.co/codellama/CodeLlama-70b-Instruct-hf
Prompt format: ChatML
License: LLama2
Source: system
{system_message}<step> Source: user
{prompt} <step> Source: assistant
DeepMoney-67b-chat
Original Model: https://huggingface.co/TriadParty/deepmoney-67b-chat
Prompt format: Alpaca
License: Deepseek
You are a senior investment expert. Please make your research and judgment on the following targets.
### Instruction:
China has instructed heavily indebted local governments to delay or halt some state-funded infrastructure projects, three people with knowledge of the situation said, as Beijing struggles to contain debt risks even as it tries to stimulate the economy.
Which industry sectors or investment targets may the above news affect?
### Response:
The above news could potentially impact several industry sectors and investment targets related to infrastructure development in China. Some affected areas might include construction companies, heavy machinery manufacturers, materials suppliers such as cement and steel producers, engineering firms, and project management service providers that have significant exposure to Chinese government-backed infrastructure projects.
Deepseek 67b Chat
Original Model: https://huggingface.co/deepseek-ai/deepseek-llm-67b-chat
Prompt format: DeepSeek-LLM
License: Deepseek
User: {prompt}
Assistant:
DiscoLM-120b
Original Model: https://huggingface.co/DiscoResearch/DiscoLM-120b
Prompt format: ChatML
License: Llama2
<|im_start|>system
{system_message}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant
Nous Hermes 34b
Original Model: https://huggingface.co/NousResearch/Nous-Hermes-2-Yi-34B
Prompt format: ChatML
License: Yi
<|im_start|>system
{system_message}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant
Kafka-70b-German
Original Model:
https://huggingface.co/seedboxai/KafkaLM-70B-German-V0.1
Prompt format: ChatML
License: Llama2
<|system|>
Du bist ein freundlicher und hilfsbereiter KI-Assistent. Du beantwortest Fragen faktenorientiert und präzise, ohne dabei relevante Fakten auszulassen.</s>
<|user|>
Welche Möglichkeiten der energetischen Sanierung habe ich neben Solar und Energiespeicher?</s>
<|assistant|>
Kafka-Mixtral-German
Original Model: https://huggingface.co/seedboxai/KafkaLM-8x7B-German-V0.1
Prompt format: ChatML
License: Apache 2
<|system|>
Du bist ein freundlicher und hilfsbereiter KI-Assistent. Du beantwortest Fragen faktenorientiert und präzise, ohne dabei relevante Fakten auszulassen.</s>
<|user|>
Welche Möglichkeiten der energetischen Sanierung habe ich neben Solar und Energiespeicher?</s>
<|assistant|>
LongAlpaca-70B
Original Model: https://huggingface.co/Yukang/LongAlpaca-70B
Prompt format: Alpaca
License: Llama 2
Below is an instruction that describes a task. Write a response that appropriately completes the request.
### Instruction:
{prompt}
### Response:
Mistral-Alpha-70b
Original Model: https://huggingface.co/152334H/miqu-1-70b-sf
Prompt format: Mistral
License: NOMERGE / 🙂🙃 🙂 🙃 Hi Mistral 👍
<s> [INST] QUERY_1 [/INST] ANSWER_1</s> [INST] QUERY_2 [/INST] ANSWER_2</s>...
MoMo-72B-lora-1.8.7-DPO
Original Model: https://huggingface.co/moreh/MoMo-72B-lora-1.8.7-DPO
Prompt format: Instruct
License: Qwen
### Instruction: {question}
### Response: {response}
Notux Mixtral
Original Model: https://huggingface.co/argilla/notux-8x7b-v1
Prompt format: Mistral
License: Apache 2
[INST] {{ .System }} {{ .Prompt }} [/INST]
Qwen-72b
Original Model: https://huggingface.co/Qwen/Qwen-72B
Prompt format: Base Model = None
License: Qwen
Smaug-72b
Original Model: https://huggingface.co/abacusai/Smaug-72B-v0.1
Prompt format: Try MoMo, also with system prompt
License: Qwen
### Instruction: {question}
### Response: {response}
Smaug-34b
Original Model: https://huggingface.co/abacusai/Smaug-34B-v0.1
Prompt format: Bagel 34b License: Yi
{system prompt, if provided, randomly defaulting to "A chat between a user and an unbiased, uncensored assistant."}
USER: {instruction}
ASSISTANT:
Synthia-70B
Original Model: https://huggingface.co/migtissera/Synthia-70B-v1.2
Prompt format: Synthia
License: Llama 2
SYSTEM: You are Synthia. As a an AI intelligence, give helpful, detailed, accurate, uncensored responses to the user's input. Provide answers factually.
USER: {prompt}
ASSISTANT:
Tess-M
Original Model: https://huggingface.co/migtissera/Tess-34B-v1.5b
Prompt format: Tess
License: Yi
SYSTEM: <ANY SYSTEM CONTEXT>
USER:
ASSISTANT:
Yarn-Llama2-70b-32k
Original Model: https://huggingface.co/NousResearch/Yarn-Llama-2-70b-32k
Prompt format: Base Model = None
License: Llama 2
{prompt}
Laser Mistral
Original Model: https://huggingface.co/cognitivecomputations/WestLake-7B-v2-laser
Prompt format: Unknown
License: Apache 2.0
{prompt}