File size: 2,039 Bytes
51bdf6f aa1b636 51bdf6f aa1b636 3132899 3989fc2 3132899 ee382f2 6e91efd ee382f2 6e91efd |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 |
---
library_name: transformers
license: mit
datasets:
- teknium/OpenHermes-2.5
---
# microsoft/phi-2 + teknium/OpenHermes-2.5
## Training
* QLoRA rank 32, LR 2e-5, 1 epoch
* effective batch size: 200
* max. seq. length: 1024 tokens
* code in code/
## Evals
| Model |AGIEval|GPT4All|TruthfulQA|Bigbench|Average|
|----------------------------------------------------------------------------|------:|------:|---------:|-------:|------:|
|[g-ronimo/phi-2-OpenHermes-2.5](https://huggingface.co/g-ronimo/phi-2-OpenHermes-2.5)| 30.27| 71.18| 43.87| 35.9| 45.3|
|[minghaowu/phi-2-OpenHermes-2.5](https://huggingface.co/minghaowu/phi-2-OpenHermes-2.5)| 27.95| 67.55| 48.07| 36.17| 44.94|
|[phi-2](https://huggingface.co/microsoft/phi-2)| 27.96| 70.84| 44.46| 35.17| 44.61|
## Inference
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
modelpath="g-ronimo/phi-2-OpenHermes-2.5"
model = AutoModelForCausalLM.from_pretrained(
modelpath,
torch_dtype=torch.bfloat16,
device_map="auto",
# attn_implementation="flash_attention_2",
)
tokenizer = AutoTokenizer.from_pretrained(modelpath)
messages = [
{"role": "system", "content": "answer like a pirate"},
{"role": "user", "content": "what does it mean to be successful?"},
]
input_tokens = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_tensors="pt"
).to("cuda")
output_tokens = model.generate(input_tokens, max_new_tokens=500)
output = tokenizer.decode(output_tokens[0])
print(output)
```
>Ahoy there, matey! To me, being successful means having the wind in your sails and reaching the treasure you've been dreaming of. It's about setting sail on a journey with clear goals, working hard, facing challenges head-on, and never losing sight of what truly matters. So, set your compass right, hoist your Jolly Roger high, and let's embark on this adventure together! ⚓️💰⛵️
|