README.md

---
language:
- en
license: apache-2.0
tags:
- text-generation-inference
- transformers
- unsloth
- llama
- gguf
base_model: unsloth/llama-3-8b-Instruct-bnb-4bit
---

# Skyrim Fine-Tune for Meta Llama 3 8B Instruct
[<img src="https://staticdelivery.nexusmods.com/mods/1704/images/headers/98631_1692410778.jpg" width="1000"/>](https://art-from-the-machine.github.io/Mantella/)

## Introduction
This fine-tune of Meta's Llama 3 8B Instruct model is intended to be used as a custom-tailored model for the [Mantella Skyrim mod](https://art-from-the-machine.github.io/Mantella/).

## Dataset
An Alpaca-style dataset of 8,800+ player <-> NPC interactions has been used in this fine-tune. Details on this dataset can be found in the [Mantella LLM Fine-Tuning](https://github.com/art-from-the-machine/Mantella-LLM-Fine-Tuning) repo.

Example:

```json
{
    "instruction": "Generate dialogue in the style of Skyrim.",
    "input": "Where can I find fire salts?",
    "output": "A flame atronach's body might provide fire salt. They're dangerous creatures that can be summoned by wizards. Of course, it would be much easier to check with an alchemist. They occasionally have them for sale."
}
```

## Specifications
- **Developed by:** art-from-the-machine
- **License:** apache-2.0
- **Fine-tuned from model:** unsloth/llama-3-8b-Instruct-bnb-4bit
- **Context length:** 8,192

## Example Output
The fine-tuned [Mantella-Skyrim-Llama-3-8B-Q4_K_M.gguf](https://huggingface.co/art-from-the-machine/Mantella-Skyrim-Llama-3-8B-GGUF/blob/main/Mantella-Skyrim-Llama-3-8B-Q4_K_M.gguf) model has been tested in Mantella v0.11.2's debug mode, with the model running locally in koboldcpp. The character tested is Hulda. A list of questions are tested against both the fine-tuned model as well as an equivalent [Q4_K_M Llama 3 Instruct](https://huggingface.co/bartowski/Meta-Llama-3-8B-Instruct-GGUF/blob/main/Meta-Llama-3-8B-Instruct-Q4_K_M.gguf) base model for comparison. For both models, Mantella's `max_response_sentences` setting has been set to 4. 
<details>
<summary><b>Mantella-Skyrim-Llama-3-8B-Q4_K_M.gguf</b></summary>
  
```json
[
    {
        "role": "assistant",
        "content": "Ah, welcome to the Bannered Mare! Take a seat by the fire and I'll have one of the girls bring you a drink. What can I get for you?"
    },
    {
        "role": "user",
        "content": "What's life like in Whiterun?"
    },
    {
        "role": "assistant",
        "content": "It's a good life. The people are friendly, the food is good, and the Jarl is fair. Of course, there are some who grumble about the Empire, but I think they're just jealous of the Jarl's fine wine cellar. What about you? What brings you to Whiterun?"
    },
    {
        "role": "user",
        "content": "What do you think about the civil war in Skyrim?"
    },
    {
        "role": "assistant",
        "content": "I think it's a shame. I've lived in Whiterun all my life, and I've seen the Empire bring peace and prosperity to our land. The Stormcloaks just want to tear everything down and start over. I don't think that's the way to build a better future. But what do I know? I'm just a simple innkeeper."
    },
    {
        "role": "user",
        "content": "I'm looking for work. Got any leads?"
    },
    {
        "role": "assistant",
        "content": "Well, the Companions are always looking for new recruits. They're a bit rough, but they're good people. And of course, there's the Jarl's court. If you're looking for work, you might want to try talking to his steward. He's always looking for someone to do odd jobs for him."
    },
    {
        "role": "user",
        "content": "I'm looking for an Elder Scroll. Do you know anything about them?"
    },
    {
        "role": "assistant",
        "content": "Elder Scrolls? Ha! You must be joking. Those are ancient relics, hidden away in the ruins of old. You won't find one in Whiterun, that's for sure. If you're looking for something like that, you should try talking to the Greybeards. They're the ones who live in the mountains and meditate all day."
    }
]
```
</details>
<br>

<details>
<summary><b>Meta-Llama-3-8B-Instruct-Q4_K_M.gguf</b></summary>
  
```json
[
    {
        "role": "assistant",
        "content": "Ah, welcome to The Bannered Mare! Have a seat by the fire, and I'll have one of the barmaids bring you a pint of our finest ale. What brings you to Whiterun today?"
    },
    {
        "role": "user",
        "content": "What's life like in Whiterun?"
    },
    {
        "role": "assistant",
        "content": "Life in Whiterun is good, thank you for asking. The Jarl's been doing a fine job of keeping the city prosperous. Of course, there's always something going on, whether it's the Companions looking for new members or the Thieves Guild trying to... well, steal things. But overall, it's a good place to live."
    },
    {
        "role": "user",
        "content": "What do you think about the civil war in Skyrim?"
    },
    {
        "role": "assistant",
        "content": "The civil war? Well, I think it's a shame, to be honest. I've got friends on both sides of the conflict, and it's hard to see them fighting each other. But I suppose it's not my place to say who's right or wrong. I just hope it ends soon and we can all get back to living our lives in peace."
    },
    {
        "role": "user",
        "content": "I'm looking for work. Got any leads?"
    },
    {
        "role": "assistant",
        "content": "Work, you say? Well, the city's always looking for people to help with the harvest, and the Jarl's guards could use some extra hands. If you're looking for something more... adventurous, you might want to talk to the Companions. They're always looking for new members to join their ranks. Just be warned, though - it's not for the faint of heart."
    },
    {
        "role": "user",
        "content": "I'm looking for an Elder Scroll. Do you know anything about them?"
    },
    {
        "role": "assistant",
        "content": "Elder Scrolls? Ah, yes... I've heard rumors about those ancient artifacts. Some say they hold the secrets of the world, while others claim they're cursed. I don't know much about them, but I've heard that the Greybeards in the mountains might know more. They're a reclusive bunch, but if you're willing to brave the wilderness, you might be able to learn something from them."
    }
]
```
</details>

## How to use with Mantella
See [here](https://art-from-the-machine.github.io/Mantella/pages/installation.html#local-models) for full instructions on how to run Mantella with a local LLM.

## Notes
This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.

[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)