Spaces:
Sleeping
Sleeping
File size: 1,092 Bytes
3bf1c8e 55ad9bf 3bf1c8e 55ad9bf 3bf1c8e 79e94d6 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
---
title: Llama Nallm Test
emoji: 💬
colorFrom: yellow
colorTo: purple
sdk: gradio
sdk_version: 4.36.1
app_file: app.py
pinned: false
license: mit
---
A simple interface for testing out NA-LLM-qa model based on Llama-3-8B checkpoint.
# Use
```sh
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
gradio app.py
```
# Notes
- The model is hosted with HuggingFace Inference Endpoint.
- The Endpoint may be paused due to inactivity. In that case, calling a signal will "wake up" the endpoint, but it would take around several minutes.
- For now, the endpoint is gated. Set appropriate `hf_token` with READ permission to the organization.
- Input filtering
- The model performs unexpectedly for non-questions
- For this reason, a simple SVM-based filter is applied
- The filter is a `OneClassSVM` trained with question sections of na-llm
- The model, along with corresponding vectorizer, is saved in `question_undetector.pkl` as `(vectorizer, model)` object.
- The hosting machine should be powerful enough at least to run a simple SVM model
|