uploaded readme

e38d6b5 verified 6 months ago

5.84 kB

	Quantization made by Richard Erkhov.

	[Github](https://github.com/RichardErkhov)

	[Discord](https://discord.gg/pvy7H8DZMG)

	[Request more models](https://github.com/RichardErkhov/quant_request)


	Llama-3-8B-Story-Summarization-QLoRA - GGUF
	- Model creator: https://huggingface.co/nldemo/
	- Original model: https://huggingface.co/nldemo/Llama-3-8B-Story-Summarization-QLoRA/


	\| Name \| Quant method \| Size \|
	\| ---- \| ---- \| ---- \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q2_K.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q2_K.gguf) \| Q2_K \| 2.96GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.IQ3_XS.gguf) \| IQ3_XS \| 3.28GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.IQ3_S.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.IQ3_S.gguf) \| IQ3_S \| 3.43GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q3_K_S.gguf) \| Q3_K_S \| 3.41GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.IQ3_M.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.IQ3_M.gguf) \| IQ3_M \| 3.52GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q3_K.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q3_K.gguf) \| Q3_K \| 3.74GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q3_K_M.gguf) \| Q3_K_M \| 3.74GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q3_K_L.gguf) \| Q3_K_L \| 4.03GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.IQ4_XS.gguf) \| IQ4_XS \| 4.18GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q4_0.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q4_0.gguf) \| Q4_0 \| 4.34GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.IQ4_NL.gguf) \| IQ4_NL \| 4.38GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q4_K_S.gguf) \| Q4_K_S \| 4.37GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q4_K.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q4_K.gguf) \| Q4_K \| 4.58GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q4_K_M.gguf) \| Q4_K_M \| 4.58GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q4_1.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q4_1.gguf) \| Q4_1 \| 4.78GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q5_0.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q5_0.gguf) \| Q5_0 \| 5.21GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q5_K_S.gguf) \| Q5_K_S \| 5.21GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q5_K.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q5_K.gguf) \| Q5_K \| 5.34GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q5_K_M.gguf) \| Q5_K_M \| 5.34GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q5_1.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q5_1.gguf) \| Q5_1 \| 5.65GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q6_K.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q6_K.gguf) \| Q6_K \| 6.14GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q8_0.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q8_0.gguf) \| Q8_0 \| 7.95GB \|




	Original model description:
	---
	language:
	- en
	license: apache-2.0
	tags:
	- text-generation-inference
	- transformers
	- unsloth
	- llama
	- trl
	- sft
	base_model: unsloth/llama-3-8b-bnb-4bit
	---

	# Uploaded model

	- Developed by: nldemo
	- License: apache-2.0
	- Finetuned from model : unsloth/llama-3-8b-bnb-4bit

	This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.

	[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)

	Quantization made by Richard Erkhov.

	[Github](https://github.com/RichardErkhov)

	[Discord](https://discord.gg/pvy7H8DZMG)

	[Request more models](https://github.com/RichardErkhov/quant_request)


	Llama-3-8B-Story-Summarization-QLoRA - GGUF
	- Model creator: https://huggingface.co/nldemo/
	- Original model: https://huggingface.co/nldemo/Llama-3-8B-Story-Summarization-QLoRA/


	\| Name \| Quant method \| Size \|
	\| ---- \| ---- \| ---- \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q2_K.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q2_K.gguf) \| Q2_K \| 2.96GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.IQ3_XS.gguf) \| IQ3_XS \| 3.28GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.IQ3_S.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.IQ3_S.gguf) \| IQ3_S \| 3.43GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q3_K_S.gguf) \| Q3_K_S \| 3.41GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.IQ3_M.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.IQ3_M.gguf) \| IQ3_M \| 3.52GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q3_K.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q3_K.gguf) \| Q3_K \| 3.74GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q3_K_M.gguf) \| Q3_K_M \| 3.74GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q3_K_L.gguf) \| Q3_K_L \| 4.03GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.IQ4_XS.gguf) \| IQ4_XS \| 4.18GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q4_0.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q4_0.gguf) \| Q4_0 \| 4.34GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.IQ4_NL.gguf) \| IQ4_NL \| 4.38GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q4_K_S.gguf) \| Q4_K_S \| 4.37GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q4_K.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q4_K.gguf) \| Q4_K \| 4.58GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q4_K_M.gguf) \| Q4_K_M \| 4.58GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q4_1.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q4_1.gguf) \| Q4_1 \| 4.78GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q5_0.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q5_0.gguf) \| Q5_0 \| 5.21GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q5_K_S.gguf) \| Q5_K_S \| 5.21GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q5_K.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q5_K.gguf) \| Q5_K \| 5.34GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q5_K_M.gguf) \| Q5_K_M \| 5.34GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q5_1.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q5_1.gguf) \| Q5_1 \| 5.65GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q6_K.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q6_K.gguf) \| Q6_K \| 6.14GB \|
	\| [Llama-3-8B-Story-Summarization-QLoRA.Q8_0.gguf](https://huggingface.co/RichardErkhov/nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf/blob/main/Llama-3-8B-Story-Summarization-QLoRA.Q8_0.gguf) \| Q8_0 \| 7.95GB \|




	Original model description:
	---
	language:
	- en
	license: apache-2.0
	tags:
	- text-generation-inference
	- transformers
	- unsloth
	- llama
	- trl
	- sft
	base_model: unsloth/llama-3-8b-bnb-4bit
	---

	# Uploaded model

	- Developed by: nldemo
	- License: apache-2.0
	- Finetuned from model : unsloth/llama-3-8b-bnb-4bit

	This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.

	[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)