Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,15 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+Ministral-8B-Instruct-2410-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+Ministral-8B-Instruct-2410-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+Ministral-8B-Instruct-2410-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Ministral-8B-Instruct-2410-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Ministral-8B-Instruct-2410-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+Ministral-8B-Instruct-2410-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Ministral-8B-Instruct-2410-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Ministral-8B-Instruct-2410-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
+Ministral-8B-Instruct-2410-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Ministral-8B-Instruct-2410-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Ministral-8B-Instruct-2410-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+Ministral-8B-Instruct-2410-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

Ministral-8B-Instruct-2410-Q2_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b47eaa26996560233e55c9c6979e0beb3d1c6d7e2f4e31807797235dbd2e6e19
+size 3185478176

Ministral-8B-Instruct-2410-Q3_K_L.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e808249f7e6a1c6ecc3cb946cfdb1dd709fb18c95ee4cae1a92d10e8a173ef3c
+size 4326459936

Ministral-8B-Instruct-2410-Q3_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:30eabd846cfd9e62819777ccfb8b3e67f90223d9c2a0ae51f0683410e215059f
+size 4019227168

Ministral-8B-Instruct-2410-Q3_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3a79546d570b928b6535b2ce9d04e221985bdc35a77bebe2975a44820068611a
+size 3664677408

Ministral-8B-Instruct-2410-Q4_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3a39b8aa3293ef0a08c58a0c857bc4492a10543b8ae0e1dd3e378db16552ed0b
+size 4658465312

Ministral-8B-Instruct-2410-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5a74b5a83b7b297fc8c03b74f00fbafff9d65e4adba7559f5953113cb67cf272
+size 4911499808

Ministral-8B-Instruct-2410-Q4_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c7dbbf75f8a4e085d28a69692b28096b19c1cc25347b17215c149b538a9515e6
+size 4685728288

Ministral-8B-Instruct-2410-Q5_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:510d13aa4f26077c2178dff89ff5a7e82ed66cfb91d1685f8739c12a609ccdd7
+size 5593795104

Ministral-8B-Instruct-2410-Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1f89d8ee8f7ddfd06a4cd6e005a77e50a7c83856386aaa3a1e1105f4a1788b2e
+size 5724146208

Ministral-8B-Instruct-2410-Q5_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8ff4c307ee400cd2dea259e18c569671d77b1fdf6199a6b8f329d45d1de04336
+size 5593795104

Ministral-8B-Instruct-2410-Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3bc0e11243388a70ba0634917fdd97ece173431e46bad9ff11d0f6db15b2b8aa
+size 6587583008

Ministral-8B-Instruct-2410-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:35bbf92fe4f9a09dddeb4e1b64942d532446d430314a0868b8a09cf5e2017c01
+size 8529807904

README.md

@@ -0,0 +1,319 @@
+---
+language:
+- en
+- fr
+- de
+- es
+- it
+- pt
+- zh
+- ja
+- ru
+- ko
+license: other
+license_name: mrl
+inference: false
+license_link: https://mistral.ai/licenses/MRL-0.1.md
+extra_gated_prompt: '# Mistral AI Research License
+
+  If You want to use a Mistral Model, a Derivative or an Output for any purpose that
+  is not expressly authorized under this Agreement, You must request a license from
+  Mistral AI, which Mistral AI may grant to You in Mistral AI''s sole discretion.
+  To discuss such a license, please contact Mistral AI via the website contact form:
+  https://mistral.ai/contact/
+
+  ## 1. Scope and acceptance
+
+  **1.1. Scope of the Agreement.** This Agreement applies to any use, modification,
+  or Distribution of any Mistral Model by You, regardless of the source You obtained
+  a copy of such Mistral Model.
+
+  **1.2. Acceptance.** By accessing, using, modifying, Distributing a Mistral Model,
+  or by creating, using or distributing a Derivative of the Mistral Model, You agree
+  to be bound by this Agreement.
+
+  **1.3. Acceptance on behalf of a third-party.** If You accept this Agreement on
+  behalf of Your employer or another person or entity, You warrant and represent that
+  You have the authority to act and accept this Agreement on their behalf. In such
+  a case, the word "You" in this Agreement will refer to Your employer or such other
+  person or entity.
+
+  ## 2. License
+
+  **2.1. Grant of rights**.  Subject to Section 3 below, Mistral AI hereby grants
+  You a non-exclusive, royalty-free, worldwide, non-sublicensable, non-transferable,
+  limited license to use, copy, modify, and Distribute under the conditions provided
+  in Section 2.2 below, the Mistral Model and any Derivatives made by or for Mistral
+  AI and to create Derivatives of the Mistral Model.
+
+  **2.2. Distribution of Mistral Model and Derivatives made by or for Mistral AI.**
+  Subject to Section 3 below, You may Distribute copies of the Mistral Model and/or
+  Derivatives made by or for Mistral AI, under the following conditions: You must
+  make available a copy of this Agreement to third-party recipients of the Mistral
+  Models and/or Derivatives made by or for Mistral AI you Distribute, it being specified
+  that any rights to use the Mistral Models and/or Derivatives made by or for Mistral
+  AI shall be directly granted by Mistral AI to said third-party recipients pursuant
+  to the Mistral AI Research License agreement executed between these parties; You
+  must retain in all copies of the Mistral Models the following attribution notice
+  within a "Notice" text file distributed as part of such copies: "Licensed by Mistral
+  AI under the Mistral AI Research License".
+
+  **2.3. Distribution of Derivatives made by or for You.** Subject to Section 3 below,
+  You may Distribute any Derivatives made by or for You under additional or different
+  terms and conditions, provided that: In any event, the use and modification of Mistral
+  Model and/or Derivatives made by or for Mistral AI shall remain governed by the
+  terms and conditions of this Agreement; You include in any such Derivatives made
+  by or for You prominent notices stating that You modified the concerned Mistral
+  Model; and Any terms and conditions You impose on any third-party recipients relating
+  to Derivatives made by or for You shall neither limit such third-party recipients''
+  use of the Mistral Model or any Derivatives made by or for Mistral AI in accordance
+  with the Mistral AI Research License nor conflict with any of its terms and conditions.
+
+  ## 3. Limitations
+
+  **3.1. Misrepresentation.** You must not misrepresent or imply, through any means,
+  that the Derivatives made by or for You and/or any modified version of the Mistral
+  Model You Distribute under your name and responsibility is an official product of
+  Mistral AI or has been endorsed, approved or validated by Mistral AI, unless You
+  are authorized by Us to do so in writing.
+
+  **3.2. Usage Limitation.** You shall only use the Mistral Models, Derivatives (whether
+  or not created by Mistral AI) and Outputs for Research Purposes.
+
+  ## 4. Intellectual Property
+
+  **4.1. Trademarks.** No trademark licenses are granted under this Agreement, and
+  in connection with the Mistral Models, You may not use any name or mark owned by
+  or associated with Mistral AI or any of its affiliates, except (i) as required for
+  reasonable and customary use in describing and Distributing the Mistral Models and
+  Derivatives made by or for Mistral AI and (ii) for attribution purposes as required
+  by this Agreement.
+
+  **4.2. Outputs.** We claim no ownership rights in and to the Outputs. You are solely
+  responsible for the Outputs You generate and their subsequent uses in accordance
+  with this Agreement. Any Outputs shall be subject to the restrictions set out in
+  Section 3 of this Agreement.
+
+  **4.3. Derivatives.** By entering into this Agreement, You accept that any Derivatives
+  that You may create or that may be created for You shall be subject to the restrictions
+  set out in Section 3 of this Agreement.
+
+  ## 5. Liability
+
+  **5.1. Limitation of liability.** In no event, unless required by applicable law
+  (such as deliberate and grossly negligent acts) or agreed to in writing, shall Mistral
+  AI be liable to You for damages, including any direct, indirect, special, incidental,
+  or consequential damages of any character arising as a result of this Agreement
+  or out of the use or inability to use the Mistral Models and Derivatives (including
+  but not limited to damages for loss of data, loss of goodwill, loss of expected
+  profit or savings, work stoppage, computer failure or malfunction, or any damage
+  caused by malware or security breaches), even if  Mistral AI has been advised of
+  the possibility of such damages.
+
+  **5.2. Indemnification.** You agree to indemnify and hold harmless Mistral AI from
+  and against any claims, damages, or losses arising out of or related to Your use
+  or Distribution of the Mistral Models and Derivatives.
+
+  ## 6. Warranty
+
+  **6.1. Disclaimer.** Unless required by applicable law or prior agreed to by Mistral
+  AI in writing, Mistral AI provides the Mistral Models and Derivatives on an "AS
+  IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied,
+  including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT,
+  MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. Mistral AI does not represent
+  nor warrant that the Mistral Models and Derivatives will be error-free, meet Your
+  or any third party''s requirements, be secure or will allow You or any third party
+  to achieve any kind of result or generate any kind of content. You are solely responsible
+  for determining the appropriateness of using or Distributing the Mistral Models
+  and Derivatives and assume any risks associated with Your exercise of rights under
+  this Agreement.
+
+  ## 7. Termination
+
+  **7.1. Term.** This Agreement is effective as of the date of your acceptance of
+  this Agreement or access to the concerned Mistral Models or Derivatives and will
+  continue until terminated in accordance with the following terms.
+
+  **7.2. Termination.** Mistral AI may terminate this Agreement at any time if You
+  are in breach of this Agreement. Upon termination of this Agreement, You must cease
+  to use all Mistral Models and Derivatives and shall permanently delete any copy
+  thereof. The following provisions, in their relevant parts, will survive any termination
+  or expiration of this Agreement, each for the duration necessary to achieve its
+  own intended purpose (e.g. the liability provision will survive until the end of
+  the applicable limitation period):Sections 5 (Liability), 6(Warranty), 7 (Termination)
+  and 8 (General Provisions).
+
+  **7.3. Litigation.** If You initiate any legal action or proceedings against Us
+  or any other entity (including a cross-claim or counterclaim in a lawsuit), alleging
+  that the Model or a Derivative, or any part thereof, infringe upon intellectual
+  property or other rights owned or licensable by You, then any licenses granted to
+  You under this Agreement will immediately terminate as of the date such legal action
+  or claim is filed or initiated.
+
+  ## 8. General provisions
+
+  **8.1. Governing laws.** This Agreement will be governed by the laws of France,
+  without regard to choice of law principles, and the UN Convention on Contracts for
+  the International Sale of Goods does not apply to this Agreement.
+
+  **8.2. Competent jurisdiction.** The courts of Paris shall have exclusive jurisdiction
+  of any dispute arising out of this Agreement.
+
+  **8.3. Severability.** If any provision of this Agreement is held to be invalid,
+  illegal or unenforceable, the remaining provisions shall be unaffected thereby and
+  remain valid as if such provision had not been set forth herein.
+
+  ## 9. Definitions
+
+  "Agreement": means this Mistral AI Research License agreement governing the access,
+  use, and Distribution of the Mistral Models, Derivatives and Outputs.
+
+  "Derivative": means any (i) modified version of the Mistral Model (including but
+  not limited to any customized or fine-tuned version thereof), (ii) work based on
+  the Mistral Model, or (iii) any other derivative work thereof.
+
+  "Distribution", "Distributing", "Distribute" or "Distributed": means supplying,
+  providing or making available, by any means, a copy of the Mistral Models and/or
+  the Derivatives as the case may be, subject to Section 3 of this Agreement.
+
+  "Mistral AI", "We" or "Us": means Mistral AI, a French société par actions simplifiée
+  registered in the Paris commercial registry under the number 952 418 325, and having
+  its registered seat at 15, rue des Halles, 75001 Paris.
+
+  "Mistral Model": means the foundational large language model(s), and its elements
+  which include algorithms, software, instructed checkpoints, parameters, source code
+  (inference code, evaluation code and, if applicable, fine-tuning code) and any other
+  elements associated thereto made available by Mistral AI under this Agreement, including,
+  if any, the technical documentation, manuals and instructions for the use and operation
+  thereof.
+
+  "Research Purposes": means any use of a Mistral Model,  Derivative, or Output that
+  is solely for (a) personal, scientific or academic research, and (b) for non-profit
+  and non-commercial purposes, and not directly or indirectly connected to any commercial
+  activities or business operations. For illustration purposes, Research Purposes
+  does not include (1) any usage of the Mistral Model, Derivative or Output by individuals
+  or contractors employed in or engaged by companies in the context of (a) their daily
+  tasks, or (b) any activity (including but not limited to any testing or proof-of-concept)
+  that is intended to generate revenue, nor (2) any Distribution by a commercial entity
+  of the Mistral Model, Derivative or Output whether in return for payment or free
+  of charge, in any medium or form, including but not limited to through a hosted
+  or managed service (e.g. SaaS, cloud instances, etc.), or behind a software layer.
+
+  "Outputs": means any content generated by the operation of the Mistral Models or
+  the Derivatives from  a prompt (i.e., text instructions) provided by users. For
+  the avoidance of doubt, Outputs do not include any components of a Mistral Models,
+  such as any fine-tuned versions of the Mistral Models, the weights, or parameters.
+
+  "You": means the individual or entity entering into this Agreement with Mistral
+  AI.
+
+
+  *Mistral AI processes your personal data below to provide the model and enforce
+  its license. If you are affiliated with a commercial entity, we may also send you
+  communications about our models. For more information on your rights and data handling,
+  please see our <a href="https://mistral.ai/terms/">privacy policy</a>.*'
+extra_gated_fields:
+  First Name: text
+  Last Name: text
+  Country: country
+  Affiliation: text
+  Job title: text
+  I understand that I can only use the model, any derivative versions and their outputs for non-commercial research purposes: checkbox
+  ? I understand that if I am a commercial entity, I am not permitted to use or distribute
+    the model internally or externally, or expose it in my own offerings without a
+    commercial license
+  : checkbox
+  ? I understand that if I upload the model, or any derivative version, on any platform,
+    I must include the Mistral Research License
+  : checkbox
+  ? I understand that for commercial use of the model, I can contact Mistral or use
+    the Mistral AI API on la Plateforme or any of our cloud provider partners
+  : checkbox
+  ? By clicking Submit below I accept the terms of the license and acknowledge that
+    the information I provide will be collected stored processed and shared in accordance
+    with the Mistral Privacy Policy
+  : checkbox
+  geo: ip_location
+extra_gated_description: Mistral AI processes your personal data below to provide
+  the model and enforce its license. If you are affiliated with a commercial entity,
+  we may also send you communications about our models. For more information on your
+  rights and data handling, please see our <a href="https://mistral.ai/terms/">privacy
+  policy</a>.
+extra_gated_button_content: Submit
+library_name: vllm
+base_model: mistralai/Ministral-8B-Instruct-2410
+tags:
+- TensorBlock
+- GGUF
+---
+
+<div style="width: auto; margin-left: auto; margin-right: auto">
+<img src="https://i.imgur.com/jC7kdl8.jpeg" alt="TensorBlock" style="width: 100%; min-width: 400px; display: block; margin: auto;">
+</div>
+<div style="display: flex; justify-content: space-between; width: 100%;">
+    <div style="display: flex; flex-direction: column; align-items: flex-start;">
+        <p style="margin-top: 0.5em; margin-bottom: 0em;">
+            Feedback and support: TensorBlock's  <a href="https://x.com/tensorblock_aoi">Twitter/X</a>, <a href="https://t.me/TensorBlock">Telegram Group</a> and <a href="https://x.com/tensorblock_aoi">Discord server</a>
+        </p>
+    </div>
+</div>
+
+## mistralai/Ministral-8B-Instruct-2410 - GGUF
+
+This repo contains GGUF format model files for [mistralai/Ministral-8B-Instruct-2410](https://huggingface.co/mistralai/Ministral-8B-Instruct-2410).
+
+The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit ec7f3ac](https://github.com/ggerganov/llama.cpp/commit/ec7f3ac9ab33e46b136eb5ab6a76c4d81f57c7f1).
+
+<div style="text-align: left; margin: 20px 0;">
+    <a href="https://tensorblock.co/waitlist/client" style="display: inline-block; padding: 10px 20px; background-color: #007bff; color: white; text-decoration: none; border-radius: 5px; font-weight: bold;">
+        Run them on the TensorBlock client using your local machine ↗
+    </a>
+</div>
+
+## Prompt template
+
+```
+<s>[INST]{system_prompt}
+
+{prompt}[/INST]
+```
+
+## Model file specification
+
+| Filename | Quant type | File Size | Description |
+| -------- | ---------- | --------- | ----------- |
+| [Ministral-8B-Instruct-2410-Q2_K.gguf](https://huggingface.co/tensorblock/Ministral-8B-Instruct-2410-GGUF/blob/main/Ministral-8B-Instruct-2410-Q2_K.gguf) | Q2_K | 3.185 GB | smallest, significant quality loss - not recommended for most purposes |
+| [Ministral-8B-Instruct-2410-Q3_K_S.gguf](https://huggingface.co/tensorblock/Ministral-8B-Instruct-2410-GGUF/blob/main/Ministral-8B-Instruct-2410-Q3_K_S.gguf) | Q3_K_S | 3.665 GB | very small, high quality loss |
+| [Ministral-8B-Instruct-2410-Q3_K_M.gguf](https://huggingface.co/tensorblock/Ministral-8B-Instruct-2410-GGUF/blob/main/Ministral-8B-Instruct-2410-Q3_K_M.gguf) | Q3_K_M | 4.019 GB | very small, high quality loss |
+| [Ministral-8B-Instruct-2410-Q3_K_L.gguf](https://huggingface.co/tensorblock/Ministral-8B-Instruct-2410-GGUF/blob/main/Ministral-8B-Instruct-2410-Q3_K_L.gguf) | Q3_K_L | 4.326 GB | small, substantial quality loss |
+| [Ministral-8B-Instruct-2410-Q4_0.gguf](https://huggingface.co/tensorblock/Ministral-8B-Instruct-2410-GGUF/blob/main/Ministral-8B-Instruct-2410-Q4_0.gguf) | Q4_0 | 4.658 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
+| [Ministral-8B-Instruct-2410-Q4_K_S.gguf](https://huggingface.co/tensorblock/Ministral-8B-Instruct-2410-GGUF/blob/main/Ministral-8B-Instruct-2410-Q4_K_S.gguf) | Q4_K_S | 4.686 GB | small, greater quality loss |
+| [Ministral-8B-Instruct-2410-Q4_K_M.gguf](https://huggingface.co/tensorblock/Ministral-8B-Instruct-2410-GGUF/blob/main/Ministral-8B-Instruct-2410-Q4_K_M.gguf) | Q4_K_M | 4.911 GB | medium, balanced quality - recommended |
+| [Ministral-8B-Instruct-2410-Q5_0.gguf](https://huggingface.co/tensorblock/Ministral-8B-Instruct-2410-GGUF/blob/main/Ministral-8B-Instruct-2410-Q5_0.gguf) | Q5_0 | 5.594 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
+| [Ministral-8B-Instruct-2410-Q5_K_S.gguf](https://huggingface.co/tensorblock/Ministral-8B-Instruct-2410-GGUF/blob/main/Ministral-8B-Instruct-2410-Q5_K_S.gguf) | Q5_K_S | 5.594 GB | large, low quality loss - recommended |
+| [Ministral-8B-Instruct-2410-Q5_K_M.gguf](https://huggingface.co/tensorblock/Ministral-8B-Instruct-2410-GGUF/blob/main/Ministral-8B-Instruct-2410-Q5_K_M.gguf) | Q5_K_M | 5.724 GB | large, very low quality loss - recommended |
+| [Ministral-8B-Instruct-2410-Q6_K.gguf](https://huggingface.co/tensorblock/Ministral-8B-Instruct-2410-GGUF/blob/main/Ministral-8B-Instruct-2410-Q6_K.gguf) | Q6_K | 6.588 GB | very large, extremely low quality loss |
+| [Ministral-8B-Instruct-2410-Q8_0.gguf](https://huggingface.co/tensorblock/Ministral-8B-Instruct-2410-GGUF/blob/main/Ministral-8B-Instruct-2410-Q8_0.gguf) | Q8_0 | 8.530 GB | very large, extremely low quality loss - not recommended |
+
+
+## Downloading instruction
+
+### Command line
+
+Firstly, install Huggingface Client
+
+```shell
+pip install -U "huggingface_hub[cli]"
+```
+
+Then, downoad the individual model file the a local directory
+
+```shell
+huggingface-cli download tensorblock/Ministral-8B-Instruct-2410-GGUF --include "Ministral-8B-Instruct-2410-Q2_K.gguf" --local-dir MY_LOCAL_DIR
+```
+
+If you wanna download multiple model files with a pattern (e.g., `*Q4_K*gguf`), you can try:
+
+```shell
+huggingface-cli download tensorblock/Ministral-8B-Instruct-2410-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'
+```