Update README.md

README.md

@@ -1,3 +1,72 @@
 ---
 license: cc-by-4.0
+datasets:
+- verifiers-for-code/CodeNet-16K
+- verifiers-for-code/CodeNet-Planner
 ---
+# Llama-3-LlamaPlanner
+
+## Model Description
+
+LlamaPlanner is a fine-tuned version of Meta's Llama-8B model, specifically designed for generating high-quality plans for code generation tasks. The model was trained on CodeNet-16k, a curated dataset of competitive programming problems, and their corresponding plans generated using Llama-3-70B. By leveraging the power of Parameter Efficient Fine-Tuning (PEFT), LlamaPlanner achieves performance comparable to much larger models in generating effective plans for code generation.
+
+## Model Details
+
+- **Base Model:** Llama-8B Instruct
+- **Fine-Tuning Approach:** Parameter Efficient Fine-Tuning (PEFT) using Unsloth
+- **Training Data:** CodeNet-16k, a filtered and deduplicated dataset of 16,500 competitive programming problems and their plans generated using Llama-3-70B
+- **Training Infrastructure:** H100-SXM5 GPU
+- **Evaluation Benchmarks:** HumanEval and EvalPlus
+
+## How to Use
+
+To use LlamaPlanner with the Hugging Face Transformers library, follow these steps:
+
+```python
+import transformers
+import torch
+
+model_id = "verifiers-for-code/LlamaPlanner"
+
+pipeline = transformers.pipeline(
+    "text-generation", model=model_id, model_kwargs={"torch_dtype": torch.bfloat16}, device_map="auto"
+)
+
+prompt = "Generate a plan for a program that sorts an array of integers in ascending order."
+pipeline(prompt)
+```
+
+## Training Details
+
+LlamaPlanner was trained using the following steps:
+
+1. Filtering and preprocessing the CodeNet dataset to create CodeNet-16k
+2. Generating plans for each problem using Llama-3-70B
+3. Formatting the problem description, input description, output description, and samples as input, and the generated plans as output
+4. Performing PEFT on the Llama-8B Instruct base model using Unsloth with different ranks and alpha values
+5. Training on an H100-SXM5 GPU for varying epochs
+
+## Evaluation Results
+
+LlamaPlanner was evaluated on the HumanEval and EvalPlus benchmarks using various methods, including zero-shot, self-planning, base planner model, and fine-tuned planner model. The results demonstrated that LlamaPlanner outperforms the base Llama-3-8B model by 14% on HumanEval and 11% on EvalPlus. Additionally, plans generated by LlamaPlanner helped boost the performance of Llama-3-70B on HumanEval.
+
+## Citation
+
+If you use LlamaPlanner in your research or applications, please cite the model using the following BibTeX entry:
+
+```bibtex
+@misc{llamaplanner,
+  title={LlamaPlanner: A Fine-Tuned Llama-8B Model for Effective Plan Generation in Code Generation Tasks},
+  author={Abhinav Chinta and Sumuk Shashidhar and Vaibhav Sahai},
+  year={2023},
+  howpublished={\url{https://huggingface.co/verifiers-for-code/LlamaPlanner}},
+}
+```
+
+## License
+
+LlamaPlanner is released under the Apache License 2.0.
+
+## Acknowledgements
+
+We would like to thank Meta for releasing the Llama model family and the open-source community for their contributions to the development of large language models and their applications in code generation tasks.