Upload folder using huggingface_hub

README.md

@@ -0,0 +1,477 @@
+---
+tags:
+- fp8
+- vllm
+language:
+- en
+- de
+- fr
+- it
+- pt
+- hi
+- es
+- th
+pipeline_tag: text-generation
+license: llama3.1
+base_model: meta-llama/Meta-Llama-3.1-405B-Instruct
+---
+
+# Meta-Llama-3.1-405B-Instruct-FP8-dynamic
+
+## Model Overview
+- **Model Architecture:** Meta-Llama-3.1
+  - **Input:** Text
+  - **Output:** Text
+- **Model Optimizations:**
+  - **Weight quantization:** FP8
+  - **Activation quantization:** FP8
+- **Intended Use Cases:** Intended for commercial and research use in multiple languages. Similarly to [Meta-Llama-3.1-405B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-405B-Instruct), this models is intended for assistant-like chat.
+- **Out-of-scope:** Use in any manner that violates applicable laws or regulations (including trade compliance laws). Use in languages other than English.
+- **Release Date:** 8/22/2024
+- **Version:** 1.1
+- **License(s):** [llama3.1](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B/blob/main/LICENSE)
+- **Model Developers:** Neural Magic
+
+This model is a quantized version of [Meta-Llama-3.1-405B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-405B-Instruct).
+It was evaluated on a several tasks to assess the its quality in comparison to the unquatized model, including multiple-choice, math reasoning, and open-ended text generation.
+Meta-Llama-3.1-405B-Instruct-FP8-dynamic achieves 99.0% recovery for the Arena-Hard evaluation, 100.0% for OpenLLM v1 (using Meta's prompting when available), 99.9% for OpenLLM v2, 100.2% for HumanEval pass@1, and 101.1% for HumanEval+ pass@1.
+
+### Model Optimizations
+
+This model was obtained by quantizing the weights and activations of [Meta-Llama-3.1-405B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-405B-Instruct) to FP8 data type, ready for inference with vLLM built from source.
+This optimization reduces the number of bits per parameter from 16 to 8, reducing the disk size and GPU memory requirements by approximately 50%. In particular, this model can now be loaded and evaluated with a single node of 8xH100 GPUs, as opposed to multiple nodes.
+
+Only the weights and activations of the linear operators within transformers blocks are quantized. Symmetric per-channel quantization is applied, in which a linear scaling per output dimension maps the FP8 representations of the quantized weights and activations. Activations are also quantized on a per-token dynamic basis.
+[LLM Compressor](https://github.com/vllm-project/llm-compressor) is used for quantization.
+
+## Deployment
+
+### Use with vLLM
+
+This model can be deployed efficiently using the [vLLM](https://docs.vllm.ai/en/latest/) backend, as shown in the example below.
+
+```python
+from vllm import LLM, SamplingParams
+from transformers import AutoTokenizer
+
+model_id = "neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8-dynamic"
+number_gpus = 8
+
+sampling_params = SamplingParams(temperature=0.6, top_p=0.9, max_tokens=256)
+
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+
+messages = [
+    {"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"},
+    {"role": "user", "content": "Who are you?"},
+]
+
+prompts = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
+
+llm = LLM(model=model_id, tensor_parallel_size=number_gpus, max_model_len=4096)
+
+outputs = llm.generate(prompts, sampling_params)
+
+generated_text = outputs[0].outputs[0].text
+print(generated_text)
+```
+
+vLLM aslo supports OpenAI-compatible serving. See the [documentation](https://docs.vllm.ai/en/latest/) for more details.
+
+## Creation
+
+This model was created by applying [LLM Compressor with calibration samples from UltraChat](https://github.com/vllm-project/llm-compressor/blob/sa/big_model_support/examples/big_model_offloading/big_model_w8a8_calibrate.py), as presented in the code snipet below.
+
+```python
+import torch
+
+from transformers import AutoTokenizer
+
+from llmcompressor.transformers import SparseAutoModelForCausalLM, oneshot
+from llmcompressor.transformers.compression.helpers import (  # noqa
+    calculate_offload_device_map,
+    custom_offload_device_map,
+)
+
+recipe = """
+quant_stage:
+    quant_modifiers:
+        QuantizationModifier:
+            ignore: ["lm_head"]
+            config_groups:
+                group_0:
+                    weights:
+                        num_bits: 8
+                        type: float
+                        strategy: channel
+                        dynamic: false
+                        symmetric: true
+                    input_activations:
+                        num_bits: 8
+                        type: float
+                        strategy: token
+                        dynamic: true
+                        symmetric: true
+                    targets: ["Linear"]
+"""
+
+model_stub = "meta-llama/Meta-Llama-3.1-405B-Instruct"
+model_name = model_stub.split("/")[-1]
+
+device_map = calculate_offload_device_map(
+    model_stub, reserve_for_hessians=False, num_gpus=8, torch_dtype="auto"
+)
+
+model = SparseAutoModelForCausalLM.from_pretrained(
+    model_stub, torch_dtype="auto", device_map=device_map
+)
+
+output_dir = f"./{model_name}-FP8-dynamic"
+
+oneshot(
+    model=model,
+    recipe=recipe,
+    output_dir=output_dir,
+    save_compressed=True,
+    tokenizer=AutoTokenizer.from_pretrained(model_stub),
+)
+```
+
+## Evaluation
+
+This model was evaluated on the well-known Arena-Hard, OpenLLM v1, OpenLLM v2, HumanEval, and HumanEval+ benchmarks.
+In all cases, model outputs were generated with the [vLLM](https://docs.vllm.ai/en/stable/) engine.
+
+Arena-Hard evaluations were conducted using the [Arena-Hard-Auto](https://github.com/lmarena/arena-hard-auto) repository.
+The model generated a single answer for each prompt form Arena-Hard, and each answer was judged twice by GPT-4.
+We report below the scores obtained in each judgement and the average.
+
+OpenLLM v1 and v2 evaluations were conducted using Neural Magic's fork of [lm-evaluation-harness](https://github.com/neuralmagic/lm-evaluation-harness/tree/llama_3.1_instruct) (branch llama_3.1_instruct).
+This version of the lm-evaluation-harness includes versions of MMLU, ARC-Challenge and GSM-8K that match the prompting style of [Meta-Llama-3.1-Instruct-evals](https://huggingface.co/datasets/meta-llama/Meta-Llama-3.1-405B-Instruct-evals) and a few fixes to OpenLLM v2 tasks.
+
+HumanEval and HumanEval+ evaluations were conducted using Neural Magic's fork of the [EvalPlus](https://github.com/neuralmagic/evalplus) repository.
+
+Detailed model outputs are available as HuggingFace datasets for [Arena-Hard](https://huggingface.co/datasets/neuralmagic/quantized-llama-3.1-arena-hard-evals), [OpenLLM v2](https://huggingface.co/datasets/neuralmagic/quantized-llama-3.1-leaderboard-v2-evals), and [HumanEval](https://huggingface.co/datasets/neuralmagic/quantized-llama-3.1-humaneval-evals).
+
+### Accuracy
+
+<table>
+  <tr>
+   <td><strong>Benchmark</strong>
+   </td>
+   <td><strong>Meta-Llama-3.1-405B-Instruct </strong>
+   </td>
+   <td><strong>Meta-Llama-3.1-405B-Instruct-FP8-dynamic (this model)</strong>
+   </td>
+   <td><strong>Recovery</strong>
+   </td>
+  </tr>
+  <tr>
+   <td><strong>Arena Hard</strong>
+   </td>
+   <td>67.4 (67.3 / 67.5)
+   </td>
+   <td>66.7 (66.7 / 66.6)
+   </td>
+   <td>99.0%
+   </td>
+  </tr>
+  <tr>
+   <td><strong>OpenLLM v1</strong>
+   </td>
+  </tr>
+  <tr>
+   <td>MMLU (5-shot)
+   </td>
+   <td>87.4
+   </td>
+   <td>87.5
+   </td>
+   <td>100.0%
+   </td>
+  </tr>
+  <tr>
+   <td>MMLU-cot (0-shot)
+   </td>
+   <td>88.1
+   </td>
+   <td>88.1
+   </td>
+   <td>100.0%
+   </td>
+  </tr>
+  <tr>
+   <td>ARC Challenge (0-shot)
+   </td>
+   <td>95.0
+   </td>
+   <td>95.0
+   </td>
+   <td>100.0%
+   </td>
+  </tr>
+  <tr>
+   <td>GSM-8K-cot (8-shot, strict-match)
+   </td>
+   <td>96.0
+   </td>
+   <td>95.8
+   </td>
+   <td>99.8%
+   </td>
+  </tr>
+  <tr>
+   <td>Hellaswag (10-shot)
+   </td>
+   <td>88.5
+   </td>
+   <td>88.5
+   </td>
+   <td>99.9%
+   </td>
+  </tr>
+  <tr>
+   <td>Winogrande (5-shot)
+   </td>
+   <td>87.2
+   </td>
+   <td>88.0
+   </td>
+   <td>100.9%
+   </td>
+  </tr>
+  <tr>
+   <td>TruthfulQA (0-shot, mc2)
+   </td>
+   <td>65.3
+   </td>
+   <td>65.3
+   </td>
+   <td>99.9%
+   </td>
+  </tr>
+  <tr>
+   <td><strong>Average</strong>
+   </td>
+   <td><strong>86.8</strong>
+   </td>
+   <td><strong>86.9</strong>
+   </td>
+   <td><strong>100.0%</strong>
+   </td>
+  </tr>
+  <tr>
+   <td><strong>OpenLLM v2</strong>
+   </td>
+  </tr>
+  <tr>
+   <td>MMLU-Pro (5-shot)
+   </td>
+   <td>59.7
+   </td>
+   <td>59.4
+   </td>
+   <td>99.4%
+   </td>
+  </tr>
+  <tr>
+   <td>IFEval (0-shot)
+   </td>
+   <td>87.7
+   </td>
+   <td>86.8
+   </td>
+   <td>99.0%
+   </td>
+  </tr>
+  <tr>
+   <td>BBH (3-shot)
+   </td>
+   <td>67.0
+   </td>
+   <td>67.1
+   </td>
+   <td>100.1%
+   </td>
+  </tr>
+  <tr>
+   <td>Math-|v|-5 (4-shot)
+   </td>
+   <td>39.0
+   </td>
+   <td>38.8
+   </td>
+   <td>99.7%
+   </td>
+  </tr>
+  <tr>
+   <td>GPQA (0-shot)
+   </td>
+   <td>19.5
+   </td>
+   <td>19.0
+   </td>
+   <td>97.4%
+   </td>
+  </tr>
+  <tr>
+   <td>MuSR (0-shot)
+   </td>
+   <td>19.5
+   </td>
+   <td>20.8
+   </td>
+   <td>106.9%
+   </td>
+  </tr>
+  <tr>
+   <td><strong>Average</strong>
+   </td>
+   <td><strong>48.7</strong>
+   </td>
+   <td><strong>48.7</strong>
+   </td>
+   <td><strong>99.9%</strong>
+   </td>
+  </tr>
+  <tr>
+   <td><strong>Coding</strong>
+   </td>
+  </tr>
+  <tr>
+   <td>HumanEval pass@1
+   </td>
+   <td>86.8
+   </td>
+   <td>87.0
+   </td>
+   <td>100.2%
+   </td>
+  </tr>
+  <tr>
+   <td>HumanEval+ pass@1
+   </td>
+   <td>80.1
+   </td>
+   <td>81.0
+   </td>
+   <td>101.1%
+   </td>
+  </tr>
+</table>
+
+
+### Reproduction
+
+The results were obtained using the following commands:
+
+#### MMLU
+```
+lm_eval \
+  --model vllm \
+  --model_args pretrained="neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8-dynamic",dtype=auto,add_bos_token=True,max_model_len=4096,max_gen_toks=10,tensor_parallel_size=8 \
+  --tasks mmlu_llama_3.1_instruct \
+  --apply_chat_template \
+  --fewshot_as_multiturn \
+  --num_fewshot 5 \
+  --batch_size auto
+```
+
+#### MMLU-cot
+```
+lm_eval \
+  --model vllm \
+  --model_args pretrained="neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8-dynamic",dtype=auto,add_bos_token=True,max_model_len=4096,max_gen_toks=1024,tensor_parallel_size=8 \
+  --tasks mmlu_cot_0shot_llama_3.1_instruct \
+  --apply_chat_template \
+  --num_fewshot 0 \
+  --batch_size auto
+```
+
+#### ARC-Challenge
+```
+lm_eval \
+  --model vllm \
+  --model_args pretrained="neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8-dynamic",dtype=auto,add_bos_token=True,max_model_len=4096,tensor_parallel_size=8 \
+  --tasks arc_challenge_llama_3.1_instruct \
+  --apply_chat_template \
+  --num_fewshot 0 \
+  --batch_size auto
+```
+
+#### GSM-8K
+```
+lm_eval \
+  --model vllm \
+  --model_args pretrained="neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8-dynamic",dtype=auto,add_bos_token=True,max_model_len=4096,tensor_parallel_size=8 \
+  --tasks gsm8k_cot_llama_3.1_instruct \
+  --apply_chat_template \
+  --fewshot_as_multiturn \
+  --num_fewshot 8 \
+  --batch_size auto
+```
+
+#### Hellaswag
+```
+lm_eval \
+  --model vllm \
+  --model_args pretrained="neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8-dynamic",dtype=auto,add_bos_token=True,max_model_len=4096,tensor_parallel_size=8 \
+  --tasks hellaswag \
+  --num_fewshot 10 \
+  --batch_size auto
+```
+
+#### Winogrande
+```
+lm_eval \
+  --model vllm \
+  --model_args pretrained="neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8-dynamic",dtype=auto,add_bos_token=True,max_model_len=4096,tensor_parallel_size=8 \
+  --tasks winogrande \
+  --num_fewshot 5 \
+  --batch_size auto
+```
+
+#### TruthfulQA
+```
+lm_eval \
+  --model vllm \
+  --model_args pretrained="neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8-dynamic",dtype=auto,add_bos_token=True,max_model_len=4096,tensor_parallel_size=8 \
+  --tasks truthfulqa \
+  --num_fewshot 0 \
+  --batch_size auto
+```
+
+#### OpenLLM v2
+```
+lm_eval \
+  --model vllm \
+  --model_args pretrained="neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8-dynamic",dtype=auto,max_model_len=4096,tensor_parallel_size=8,enable_chunked_prefill=True \
+  --apply_chat_template \
+  --fewshot_as_multiturn \
+  --tasks leaderboard \
+  --batch_size auto
+```
+
+#### HumanEval and HumanEval+
+##### Generation
+```
+python3 codegen/generate.py \
+  --model neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8-dynamic \
+  --bs 16 \
+  --temperature 0.2 \
+  --n_samples 50 \
+  --root "." \
+  --dataset humaneval \
+  --tp 8
+```
+##### Sanitization
+```
+python3 evalplus/sanitize.py \
+  humaneval/neuralmagic--Meta-Llama-3.1-405B-Instruct-FP8-dynamic_vllm_temp_0.2
+```
+##### Evaluation
+```
+evalplus.evaluate \
+  --dataset humaneval \
+  --samples humaneval/neuralmagic--Meta-Llama-3.1-405B-Instruct-FP8-dynamic_vllm_temp_0.2-sanitized
+```

config.json

@@ -0,0 +1,79 @@
+{
+  "_name_or_path": "/mnt/nvme1/meta-llama-8-kv-heads/Meta-Llama-3.1-405B-Instruct",
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 128000,
+  "eos_token_id": [
+    128001,
+    128008,
+    128009
+  ],
+  "hidden_act": "silu",
+  "hidden_size": 16384,
+  "initializer_range": 0.02,
+  "intermediate_size": 53248,
+  "max_position_embeddings": 131072,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 128,
+  "num_hidden_layers": 126,
+  "num_key_value_heads": 8,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": {
+    "factor": 8.0,
+    "high_freq_factor": 4.0,
+    "low_freq_factor": 1.0,
+    "original_max_position_embeddings": 8192,
+    "rope_type": "llama3"
+  },
+  "rope_theta": 500000.0,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.44.0",
+  "use_cache": true,
+  "vocab_size": 128256,
+  "quantization_config": {
+    "config_groups": {
+      "group_0": {
+        "input_activations": {
+          "block_structure": null,
+          "dynamic": true,
+          "group_size": null,
+          "num_bits": 8,
+          "observer": "memoryless",
+          "observer_kwargs": {},
+          "strategy": "token",
+          "symmetric": true,
+          "type": "float"
+        },
+        "output_activations": null,
+        "targets": [
+          "Linear"
+        ],
+        "weights": {
+          "block_structure": null,
+          "dynamic": false,
+          "group_size": null,
+          "num_bits": 8,
+          "observer": "minmax",
+          "observer_kwargs": {},
+          "strategy": "channel",
+          "symmetric": true,
+          "type": "float"
+        }
+      }
+    },
+    "format": "naive-quantized",
+    "global_compression_ratio": 1.240844678218891,
+    "ignore": [
+      "lm_head"
+    ],
+    "kv_cache_scheme": null,
+    "quant_method": "compressed-tensors",
+    "quantization_status": "frozen"
+  }
+}

generation_config.json

@@ -0,0 +1,12 @@
+{
+  "bos_token_id": 128000,
+  "do_sample": true,
+  "eos_token_id": [
+    128001,
+    128008,
+    128009
+  ],
+  "temperature": 0.6,
+  "top_p": 0.9,
+  "transformers_version": "4.44.0"
+}

model-00001-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4348d16ae9df91a32354452173aa684c47d38fb3eb0c828c9a2aeeefd2404cf3
+size 4773188696

model-00002-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:97758aaea4d3186ac8d39cb4c3f46c0beddef57f63952d966d63538c0be6f5aa
+size 4933097744

model-00003-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4f4d65575b39ad8bdaa2dc49787ed6e13b1f1cb8f7175e9004dc88327364406f
+size 4631063728

model-00004-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:056866c6d81ba0b340fe9d883ed2699e2eef62184583c4ba3936d172a61e6144
+size 4933097744

model-00005-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cb7d70cb0fb8993d957f7327e4c239a1aab5587d3fbf570f6d147cd5637c71b5
+size 4631063728

model-00006-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2e4080c6b89cb2ca05e0dc65100c9a69bb9946bf75ddd4dbf80f848ff309872c
+size 4933097744

model-00007-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bb8478700043eb821c2a386870a097d5303935b337da5595e70396c8e66f9b4a
+size 4631063728

model-00008-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9eb1a5b66923a1c466011a6c3e7f6ab23a447900b1e606549a8ac64e59e47cd5
+size 4933097760

model-00009-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b8e4f7f3d6bcae94a1f440769af88f3ab39b38902e4af19d452baf4067118a35
+size 4631063752

model-00010-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:72f45fff8f48e6c3c44c15522bc790caf2ba3259999112a56e4a5bdb34098dda
+size 4933097760

model-00011-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fde2e9041a23bfcf5b2fe61ea0ab761a719e332c6ceec1a9357523d17b4fd139
+size 4631063752

model-00012-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4cf61262b8a7eb86f2639f7e0bc006b590e70ce2f6ffa474943f670837eef20b
+size 4933097760

model-00013-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:16963a53bf5b5f34dfdc954164f0cd3974da49b969913221153b1f13721fe9e6
+size 4631063752

model-00014-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c1744110f5934aa55d23b39179ab7f39f1059fb2e100fe30a99fd269550b81da
+size 4933097760

model-00015-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7fd9f2ee8b68719664397af25ed6cff522237fdd781f8bb7c30c50cf8e5545f9
+size 4631063752

model-00016-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:38b73868a9d451f673e89e6f4b52bda1b32851c23afc6d6d5b5322caeec312eb
+size 4933097760

model-00017-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f616cccffc610561019350e2af8b15975b0c8b408fc49f5e157d8435fa34c578
+size 4631063752

model-00018-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e316a1e967e413b27b4ebaa8d3892eec00b9c899c566d584fa8a800eeac1d43b
+size 4933097760

model-00019-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1a982203d9cecdaf7f7e111881d788b0a1b4a78d260de33bd15d60e04716c7ba
+size 4631063752

model-00020-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:12ae44a6f3a72ab6dac83c84cc407e5cdc501e3f1e3e3f60d34644e389f5ac83
+size 4933097760

model-00021-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:001836cb75107ab700beb7b357034f2c4b488c0380e49c10ccaea882494c7d34
+size 4631063752

model-00022-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4d9d060cbf66e7d0d925f648e239210879f312e09e640ff8d5ce0f53045ca516
+size 4933097760

model-00023-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a73d60b78b14c3ca4dbb91680a7b63bb93cbf591b74ac66f77b5ada606f79c55
+size 4631063752

model-00024-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b128fe3feffdd365aebaa098c0175a79b466cf25ac43d52bdcb9e05635cbbce3
+size 4933097760

model-00025-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0e1b227aa8f3469fd634d33c6dd56e38b2d6531a4eeefd7d65d0405b8055a9fd
+size 4631063752

model-00026-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:39280d9b4a9d8f0579929883e2d52acf67ef44616060e06878fcac17100719c1
+size 4933097760

model-00027-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d3231ee99030495fadfd2b5a4bec05fd5dfb79df0a5774dae946856b8b1d54d2
+size 4631063752

model-00028-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:127dd75728a5ca862a927385b342dac266aee81bc3d73d0a4a5232258ca24c33
+size 4933097760

model-00029-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ce76397d654f0cfc86bf41909b7c48cabe779dfbf6a583d74fc10a8c862b357e
+size 4631063752

model-00030-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b61c3270ea7ddf4db3855d7acb1884c1e57cd12d6fee0445c908f86ce8e16170
+size 4933097760

model-00031-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eb864d68a7fe712df011844be8413c5c5de0dc09941a8df4ec9dc637cf27318d
+size 4631063752

model-00032-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1e2a614ee2e6289179f1f0bcdd60744f04a97a6ac053c3292ecc489dd1e02c6e
+size 4933097760

model-00033-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9fdbdbd58e49be94498807a880f06598fe5328296352ecbd153a49cf06d458ac
+size 4631063752

model-00034-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ab4245d34461470c2e647cd7c6b72e89569e45c7171fe0c236d2293f86a0e648
+size 4933097760

model-00035-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0622f8f0b2d5da4f46c0bc1c64e6318fc1cdebf5bd9a6cdc5d596430c0fa7ec5
+size 4631063752

model-00036-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:10ac2d9b50f53954d7af02c355a3a46fd3d6f7a3e0764fd7bba6823c060c69fc
+size 4933097760

model-00037-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1575aa47b277f1bf987d36e972e681d3f5fdb94eb35f3b5e4d89ec5c34089cc2
+size 4631063752

model-00038-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0275dda813566a99efc1d2ed4467b589eb0b06a99cc3d387af82d23e0dc4d5e6
+size 4933097760

model-00039-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:38c0890183aae338aa320b0fb89ddc2d65b02d5f35b1b2c6f32d6402fb5d256f
+size 4631063752

model-00040-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1fe5d1fa84331883bf0b4c0567e6fc46a306500ed0c7f6ebadb8c164ee469edd
+size 4933097760

model-00041-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e9e6c992cb1184f394bf07b20eb57fb3c9601a7a4bb7898ba673046f4af30ac2
+size 4631063752

model-00042-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f3090b4191ef9f188c8bf7bdf33bb7e95edb5dd6c661a661eb32094ed08fe309
+size 4933097760

model-00043-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:88067ececbb254f4e586ace3711afc7f7f9c3c664e4a11e74273650310fdc3ee
+size 4631063752

model-00044-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf93ec4fceded4f8984a21f333c6c58f447366ab5fca5ef4f1a8726130caa2f3
+size 4933097760

model-00045-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c1653a664070e037becde300c34b8cea10cbaa402ef13b79b528e57317f74153
+size 4631063752

model-00046-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c6533a49788b4779932dec28572d7629035d2cd47aa563d073ffe0972eb2a582
+size 4933097760

model-00047-of-00086.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:88983387d93204333940a7554290d5cc72ae99fb5ec000048fde4e2f32275741
+size 4631063752