metadata

library_name: peft
base_model: unsloth/tinyllama-bnb-4bit
license: mit
datasets:
  - yahma/alpaca-cleaned
language:
  - en
pipeline_tag: text-generation
tags:
  - Instruct
  - TinyLlama

Steps to try the model:

prompt Template

alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
{}

### Input:
{}

### Response:
{}"""

load the model

from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM ,AutoTokenizer

config = PeftConfig.from_pretrained("damerajee/Tinyllama-sft-small")
model = AutoModelForCausalLM.from_pretrained("unsloth/tinyllama")
tokenizer=AutoTokenizer.from_pretrained("damerajee/Tinyllama-sft-small")
model = PeftModel.from_pretrained(model, "damerajee/Tinyllama-sft-small")l")

Inference

inputs = tokenizer(
[
    alpaca_prompt.format(
        "i want to learn machine learning help me", 
        "", # input
        "", # output
    )
]*1, return_tensors = "pt").to("cuda")

outputs = model.generate(**inputs, max_new_tokens = 312, use_cache = True)
tokenizer.batch_decode(outputs)

Model Information

The base model unsloth/tinyllama-bnb-4bitwas Instruct finetuned using Unsloth

Training Details

The model was trained for 1 epoch on a free goggle colab which took about 1 hour and 30 mins approximately