| --- |
| license: apache-2.0 |
| tags: |
| - speculators |
| - eagle |
| - speculative-decoding |
| --- |
| |
| # Eagle Llama 3.1 8B Instruct |
|
|
| This is a converted Eagle speculator checkpoint for Llama 3.1 8B Instruct, compatible with the [speculators](https://github.com/neuralmagic/speculators) library. |
|
|
| ## Model Details |
|
|
| - **Original checkpoint**: [yuhuili/EAGLE-LLaMA3.1-Instruct-8B](https://huggingface.co/yuhuili/EAGLE-LLaMA3.1-Instruct-8B) |
| - **Base model**: [meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct) |
| - **Architecture**: EAGLE (standard, without layernorms) |
| - **Size**: 481MB |
|
|
| ## Usage |
|
|
| ```python |
| from speculators.models.eagle import EagleSpeculator |
| from transformers import AutoModelForCausalLM |
| |
| # Load the Eagle speculator |
| eagle_model = EagleSpeculator.from_pretrained("nm-testing/eagle-llama3.1-8b-instruct") |
| |
| # Attach verifier model |
| verifier = AutoModelForCausalLM.from_pretrained("meta-llama/Meta-Llama-3.1-8B-Instruct") |
| eagle_model.attach_verifier(verifier) |
| |
| # Use for speculative decoding |
| # ... your speculative decoding code here ... |
| ``` |
|
|
| ## Conversion Details |
|
|
| This checkpoint was converted using: |
| ```bash |
| speculators convert --eagle \ |
| yuhuili/EAGLE-LLaMA3.1-Instruct-8B \ |
| ./eagle-standard-converted \ |
| meta-llama/Meta-Llama-3.1-8B-Instruct |
| ``` |
|
|
| ## License |
|
|
| Apache 2.0 |
