README.md

---
license: other
base_model: microsoft/phi-1_5
tags:
- bees
- honey
- bzz
metrics:
- accuracy
datasets:
- BEE-spoke-data/bees-internal
language:
- en
pipeline_tag: text-generation
---

# phi-1bee5 🐝

> Where Code Meets Beekeeping: An Unbeelievable Synergy!

<a href="https://colab.research.google.com/gist/pszemraj/7ea68b3b71ee4e6c0729d2318f3f4158/we-bee-testing.ipynb">
  <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/>
</a>

Have you ever found yourself in the depths of a debugging session and thought, "I wish I could be basking in the glory of a blooming beehive right now"? Or maybe you've been donning your beekeeping suit, puffing on your smoker, and longed for the sweet aroma of freshly written code?

Well, brace yourselves, hive-minded humans and syntax-loving sapiens, for `phi-1bee5`, a groundbreaking transformer model that's here to disrupt your apiary and your IDE!


## Details

This model is a fine-tuned version of [microsoft/phi-1_5](https://huggingface.co/microsoft/phi-1_5) on the `BEE-spoke-data/bees-internal` dataset.

It achieves the following results on the evaluation set:
- Loss: 2.6982
- Accuracy: 0.4597

## Usage 

load model:

```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
# !pip install -U -q transformers accelerate einops

checkpoint = "BEE-spoke-data/phi-1bee5"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForCausalLM.from_pretrained(
    checkpoint, 
    device_map="auto",
    torch_dtype=torch.float16,
    trust_remote_code=True
)
```
Run inference:

```python
prompt = "Today was an amazing day because"
inputs = tokenizer(prompt, return_tensors="pt", return_attention_mask=False).to(
    model.device
)

outputs = model.generate(
    **inputs, do_sample=True, max_new_tokens=128, epsilon_cutoff=7e-4
)
result = tokenizer.batch_decode(outputs, skip_special_tokens=True)[0]
print(result)
# output will probably contain a story/info about bees
```


### Intended Uses:

1. **Educational Edification**: Are you a coding novice with a budding interest in beekeeping? Or perhaps a seasoned developer whose curiosity has been piqued by the buzzing in your backyard? phi-1bee5 aims to serve as a fun, informative bridge between these two worlds.
2. **Casual Queries**: This model can generate code examples and beekeeping tips. It's perfect for those late-night coding sessions when you feel like taking a virtual stroll through an apiary.
3. **Academic & Research Insights**: Interested in interdisciplinary studies that explore the intersection of technology and ecology? phi-1bee5 might offer some amusing, if not entirely accurate, insights.

### Limitations:

1. **Not a beekeeping expert**: For the love of all things hexagonal, please do not use phi-1bee5 to make serious beekeeping decisions. While our model is well read in the beekeeping literature, it lacks the practical experience and nuanced understanding that professional beekeepers possess.
2. **Licensing**: This model is derived from a base model under the Microsoft Research License. Any use must comply with the terms of that license.
3. **Infallibility:** Like any machine learning model, phi-1bee5 can make mistakes. Always double check the code and bee facts before using it in production or in your hive.
4. **Ethical Constraints**: This model may not be used for illegal or unethical activities, including but not limited to terrorism, harassment, or spreading disinformation.

## Training procedure

While the full dataset is not yet complete and therefore not yet released for "safety reasons", you can check out a preliminary sample at: [bees-v0](https://huggingface.co/datasets/BEE-spoke-data/bees-v0)

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 1
- eval_batch_size: 2
- gradient_accumulation_steps: 32
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.995) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.03
- num_epochs: 2.0