Spaces:
Running
Running
Update app.py
Browse files
app.py
CHANGED
@@ -4,26 +4,23 @@ from typing import Tuple, List
|
|
4 |
|
5 |
deepsparse.cpu.print_hardware_capability()
|
6 |
|
7 |
-
MODEL_ID = "neuralmagic/mpt-7b-gsm8k-pruned60-quant"
|
8 |
|
9 |
DESCRIPTION = f"""
|
10 |
# MPT Sparse Finetuned on GSM8k with DeepSparse
|
11 |
![NM Logo](https://files.slack.com/files-pri/T020WGRLR8A-F05TXD28BBK/neuralmagic-logo.png?pub_secret=54e8db19db)
|
12 |
Model ID: {MODEL_ID}
|
13 |
|
14 |
-
🚀 **Experience the power of LLM mathematical reasoning** through our MPT sparse finetuned on the [GSM8K dataset](https://huggingface.co/datasets/gsm8k).
|
15 |
GSM8K, short for Grade School Math 8K, is a collection of 8.5K high-quality linguistically diverse grade school math word problems, designed to challenge question-answering systems with multi-step reasoning.
|
16 |
Observe the model's performance in deciphering complex math questions, such as "Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?" and offering detailed step-by-step solutions.
|
17 |
## Accelerated Inferenced on CPUs
|
18 |
-
The MPT model runs purely on CPU courtesy of [software
|
19 |
-
|
20 |
|
21 |
![Speedup](https://cdn-uploads.huggingface.co/production/uploads/60466e4b4f40b01b66151416/qMW-Uq8xAawhANTZYB7ZI.png)
|
22 |
"""
|
23 |
|
24 |
-
from huggingface_hub import snapshot_download
|
25 |
-
MODEL_ID = snapshot_download(repo_id=MODEL_ID, use_auth_token="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")
|
26 |
-
|
27 |
MAX_MAX_NEW_TOKENS = 1024
|
28 |
DEFAULT_MAX_NEW_TOKENS = 200
|
29 |
|
|
|
4 |
|
5 |
deepsparse.cpu.print_hardware_capability()
|
6 |
|
7 |
+
MODEL_ID = "hf:neuralmagic/mpt-7b-gsm8k-pruned60-quant"
|
8 |
|
9 |
DESCRIPTION = f"""
|
10 |
# MPT Sparse Finetuned on GSM8k with DeepSparse
|
11 |
![NM Logo](https://files.slack.com/files-pri/T020WGRLR8A-F05TXD28BBK/neuralmagic-logo.png?pub_secret=54e8db19db)
|
12 |
Model ID: {MODEL_ID}
|
13 |
|
14 |
+
🚀 **Experience the power of LLM mathematical reasoning** through [our MPT sparse finetuned](https://arxiv.org/abs/2310.06927) on the [GSM8K dataset](https://huggingface.co/datasets/gsm8k).
|
15 |
GSM8K, short for Grade School Math 8K, is a collection of 8.5K high-quality linguistically diverse grade school math word problems, designed to challenge question-answering systems with multi-step reasoning.
|
16 |
Observe the model's performance in deciphering complex math questions, such as "Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?" and offering detailed step-by-step solutions.
|
17 |
## Accelerated Inferenced on CPUs
|
18 |
+
The MPT model runs purely on CPU courtesy of [sparse software execution by DeepSparse](https://github.com/neuralmagic/deepsparse/tree/main/research/mpt).
|
19 |
+
DeepSparse provides accelerated inference by taking advantage of the MPT model's weight sparsity to deliver tokens fast!
|
20 |
|
21 |
![Speedup](https://cdn-uploads.huggingface.co/production/uploads/60466e4b4f40b01b66151416/qMW-Uq8xAawhANTZYB7ZI.png)
|
22 |
"""
|
23 |
|
|
|
|
|
|
|
24 |
MAX_MAX_NEW_TOKENS = 1024
|
25 |
DEFAULT_MAX_NEW_TOKENS = 200
|
26 |
|