---
license: llama3.2
datasets:
- ACE05
- conll2003
- conll2012_ontonotesv5
- rams
- tacred
- fewrel
- maven
language:
- en
metrics:
- f1
pipeline_tag: text-generation
tags:
- text-generation-inference
- Information Extraction
- IE
- Named Entity Recogniton
- Event Extraction
- Relation Extraction
- LLaMA
- llama-cpp
- gguf-my-repo
base_model: THU-KEG/ADELIE-SFT-3B
---

# Triangle104/ADELIE-SFT-3B-Q4_K_S-GGUF
This model was converted to GGUF format from [`THU-KEG/ADELIE-SFT-3B`](https://huggingface.co/THU-KEG/ADELIE-SFT-3B) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
Refer to the [original model card](https://huggingface.co/THU-KEG/ADELIE-SFT-3B) for more details on the model.

---
Model details:
-
 We introduce ADELIE (Aligning large language moDELs on Information Extraction), an aligned LLM that effectively solves various IE tasks, including closed IE, open IE, and on-demand IE. We first collect and construct a high-quality alignment corpus IEInstruct for IE. Then we train ADELIESFT using instruction tuning on IEInstruct. We further train ADELIESFT with direct preference optimization (DPO) objective, resulting in ADELIEDPO. Extensive experiments on various held-out IE datasets demonstrate that our models (ADELIESFT and ADELIEDPO) achieve state-of-the-art (SoTA) performance among open-source models. We further explore the general capabilities of ADELIE, and experimental results reveal that their general capabilities do not exhibit a noticeable decline.

    📖 Paper: ADELIE: Aligning Large Language Models on Information Extraction

    🐧 Github: THU/ADELIE

Model Description
-
    Developed by: Yunjia Qi, Hao Peng, Xiaozhi Wang, Bin Xu, Lei Hou, Juanzi Li
    Model type: Text Generation
    Language(s) (NLP): English
    License: LLaMA2 License for the base model.
    Finetuned from model [optional]: LLaMA3.2-3B

---
## Use with llama.cpp
Install llama.cpp through brew (works on Mac and Linux)

```bash
brew install llama.cpp

```
Invoke the llama.cpp server or the CLI.

### CLI:
```bash
llama-cli --hf-repo Triangle104/ADELIE-SFT-3B-Q4_K_S-GGUF --hf-file adelie-sft-3b-q4_k_s.gguf -p "The meaning to life and the universe is"
```

### Server:
```bash
llama-server --hf-repo Triangle104/ADELIE-SFT-3B-Q4_K_S-GGUF --hf-file adelie-sft-3b-q4_k_s.gguf -c 2048
```

Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.

Step 1: Clone llama.cpp from GitHub.
```
git clone https://github.com/ggerganov/llama.cpp
```

Step 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).
```
cd llama.cpp && LLAMA_CURL=1 make
```

Step 3: Run inference through the main binary.
```
./llama-cli --hf-repo Triangle104/ADELIE-SFT-3B-Q4_K_S-GGUF --hf-file adelie-sft-3b-q4_k_s.gguf -p "The meaning to life and the universe is"
```
or 
```
./llama-server --hf-repo Triangle104/ADELIE-SFT-3B-Q4_K_S-GGUF --hf-file adelie-sft-3b-q4_k_s.gguf -c 2048
```