File size: 1,230 Bytes
002e4ed 7a45391 1585789 36a31c0 1585789 36a31c0 1585789 36a31c0 1585789 36a31c0 1585789 36a31c0 1585789 7a45391 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 |
---
license: apache-2.0
datasets:
- yahma/alpaca-cleaned
metrics:
- accuracy
base_model:
- meta-llama/Meta-Llama-3.1-8B-Instruct
---
## Usage
Support for this model will be added in the upcoming transformers release. In the meantime, please install the library from source:
~~~
pip install transformers
~~~
We can now run inference on this model:
~~~
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
# Load the tokenizer and model
model_path = "YaoLuzjut/partial-layer_fine-tuning_Llama-3.1-8B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_path)
device = 'cuda'
dtype = torch.bfloat16
model = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype=dtype, device_map=device)
# Prepare the input text
prompt = 'Complete the paragraph: our solar system is'
inputs = tokenizer.encode(prompt, return_tensors='pt').to(model.device)
# Generate the output
outputs = model.generate(inputs, max_length=20)
# Decode and print the output
output_text = tokenizer.decode(outputs[0])
print(output_text)
~~~
## Evaluation Results
Zero-shot performance. Evaluated using select datasets from the [LM Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness/tree/main) with additions:
|