rohanprichard commited on
Commit
d398cd7
1 Parent(s): 132ea82

update model card

Browse files
Files changed (1) hide show
  1. README.md +68 -0
README.md ADDED
@@ -0,0 +1,68 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: mlc-llm
3
+ base_model: arcee-ai/Llama-3.1-SuperNova-Lite
4
+ tags:
5
+ - mlc-llm
6
+ - web-llm
7
+ ---
8
+
9
+ # Llama-3.2-1B-Instruct-MLC
10
+
11
+ This is the [Llama-3.1-SuperNova-Lite](https://huggingface.co/arcee-ai/Llama-3.1-SuperNova-Lite) model in MLC format `q4f16_1`.
12
+ The model can be used for projects [MLC-LLM](https://github.com/mlc-ai/mlc-llm) and [WebLLM](https://github.com/mlc-ai/web-llm).
13
+
14
+ ## Example Usage
15
+
16
+ Here are some examples of using this model in MLC LLM.
17
+ Before running the examples, please install MLC LLM by following the [installation documentation](https://llm.mlc.ai/docs/install/mlc_llm.html#install-mlc-packages).
18
+
19
+ ### Chat
20
+
21
+ In command line, run
22
+ ```bash
23
+ mlc_llm chat HF://rohanprichard/Llama-3.1-SuperNova-Lite
24
+ ```
25
+
26
+ ### REST Server
27
+
28
+ In command line, run
29
+ ```bash
30
+ mlc_llm serve HF://rohanprichard/Llama-3.1-SuperNova-Lite
31
+ ```
32
+
33
+ ### Python API
34
+
35
+ ```python
36
+ from mlc_llm import MLCEngine
37
+
38
+ # Create engine
39
+ model = "HF://rohanprichard/Llama-3.1-SuperNova-Lite"
40
+ engine = MLCEngine(model)
41
+
42
+ # Run chat completion in OpenAI API.
43
+ for response in engine.chat.completions.create(
44
+ messages = [
45
+ {
46
+ "role": "user",
47
+ "content": [
48
+ {
49
+ "type": "text",
50
+ "text": "How many r's are in the word strawberry?"
51
+ },
52
+ ],
53
+ },
54
+ ],
55
+ model=model,
56
+ stream=True,
57
+ ):
58
+ for choice in response.choices:
59
+ print(choice.delta.content, end="", flush=True)
60
+ print("\n")
61
+
62
+ engine.terminate()
63
+ ```
64
+
65
+ ## Documentation
66
+
67
+ For more information on MLC LLM project, please visit the [documentation](https://llm.mlc.ai/docs/) and [GitHub repo](http://github.com/mlc-ai/mlc-llm).
68
+ Model card based on the template from the MLC team.