suayptalha/Komodo-Llama-3.2-3B

This version of Komodo is a Llama-3.2-3B-Instruct finetuned model on lighteval/MATH-Hard dataset to increase math performance of the base model.

This model is 4bit-quantizated. You should import it 8bit if you want to use 7B parameters! Make sure you installed 'bitsandbytes' library before import.

Example Usage:

tokenizer = AutoTokenizer.from_pretrained("suayptalha/Komodo-7B-Instruct")
model = AutoModelForCausalLM.from_pretrained("suayptalha/Komodo-7B-Instruct")

example_prompt = """Below is a math question and its solution:
Question: {}
Solution: {}"""

inputs = tokenizer(
[
    example_prompt.format(
        "", #Question here
        "", #Solution here (for training)
    )
], return_tensors = "pt").to("cuda")

outputs = model.generate(**inputs, max_new_tokens = 50, use_cache = True)
tokenizer.batch_decode(outputs)