clibrain
/

Llama-2-7b-ft-instruct-es-gptq-4bit

Text Generation

text-generation-inference

4-bit precision

Model card Files Files and versions Community

mrm8488 commited on Sep 1, 2023

Commit

dc8375c

•

1 Parent(s): 443db7b

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -138,8 +138,8 @@ print(generate(instruction))
 ### Performance Test
-After several executions on a *Nvidia T4 with 16GB VRAM*, we got: it takes aprox **0.091 seconds** to generate a token
-| Latency | GPU Mem |
 ----------|---------|
 |43.36ms/token | 3.83 GB |

 ### Performance Test
+After several executions on a *Nvidia T4 with 16GB VRAM*, we got:
+| Latency | GPU Mem Required |
 ----------|---------|
 |43.36ms/token | 3.83 GB |