|
--- |
|
library_name: llama.cpp |
|
license: gemma |
|
widget: |
|
- text: '<start_of_turn>user |
|
|
|
How does the brain work?<end_of_turn> |
|
|
|
<start_of_turn>model |
|
|
|
' |
|
inference: |
|
parameters: |
|
max_new_tokens: 200 |
|
extra_gated_heading: Access Gemma on Hugging Face |
|
extra_gated_prompt: To access Gemma on Hugging Face, you’re required to review and |
|
agree to Google’s usage license. To do this, please ensure you’re logged-in to Hugging |
|
Face and click below. Requests are processed immediately. |
|
extra_gated_button_content: Acknowledge license |
|
--- |
|
|
|
<hr> |
|
|
|
# Llama.cpp imatrix quantizations of google/gemma-2-2b-it-GGUF |
|
|
|
<img src="https://cdn-uploads.huggingface.co/production/uploads/646410e04bf9122922289dc7/-03oAOPVN1nZjp6-2EIxD.png" alt="gemma" width="60%"/> |
|
|
|
Using llama.cpp commit [268c566](https://github.com/ggerganov/llama.cpp/commit/398ede5efeb07b9adf9fbda7ea63f630d476a792) for quantization. |
|
|
|
Original model: https://huggingface.co/google/gemma-2-2b-it |
|
|
|
All quants were made using the imatrix option and Bartowski's [calibration file](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8). |
|
|
|
<hr><br> |
|
|
|
# Gemma Model Card |
|
|
|
**Model Page**: [Gemma](https://ai.google.dev/gemma/docs) |
|
|
|
This model card corresponds to the 2b instruct version the Gemma 2 model in GGUF Format. The weights here are **float32**. |
|
|
|
> [!IMPORTANT] |
|
> |
|
> In llama.cpp, and other related tools such as Ollama and LM Studio, please make sure that you have these flags set correctly, especially **`repeat-penalty`**. Georgi Gerganov (llama.cpp's author) shared his experience in https://huggingface.co/google/gemma-2b-it/discussions/38#65d2b14adb51f7c160769fa1. |
|
|
|
You can also visit the model card of the [2B pretrained v2 model GGUF](https://huggingface.co/google/gemma-2b-v2-GGUF). |
|
|
|
**Resources and Technical Documentation**: |
|
|
|
* [Responsible Generative AI Toolkit](https://ai.google.dev/responsible) |
|
* [Gemma on Kaggle](https://www.kaggle.com/models/google/gemma) |
|
* [Gemma on Vertex Model Garden](https://console.cloud.google.com/vertex-ai/publishers/google/model-garden/335?version=gemma-2b-it-gg-hf) |
|
|
|
**Terms of Use**: [Terms](https://www.kaggle.com/models/google/gemma/license/consent/verify/huggingface?returnModelRepoId=google/gemma-2-2b-it-GGUF) |
|
|
|
**Authors**: Google |
|
|
|
## Model Information |
|
|
|
Summary description and brief definition of inputs and outputs. |
|
|
|
### Description |
|
|
|
Gemma is a family of lightweight, state-of-the-art open models from Google, |
|
built from the same research and technology used to create the Gemini models. |
|
They are text-to-text, decoder-only large language models, available in English, |
|
with open weights, pre-trained variants, and instruction-tuned variants. Gemma |
|
models are well-suited for a variety of text generation tasks, including |
|
question answering, summarization, and reasoning. Their relatively small size |
|
makes it possible to deploy them in environments with limited resources such as |
|
a laptop, desktop or your own cloud infrastructure, democratizing access to |
|
state of the art AI models and helping foster innovation for everyone. |
