---
license: apache-2.0
datasets:
  - grammarly/coedit
language:
  - en
tags:
  - text-generation-inference
  - candle
widget:
  - text: >-
      Fix the grammar: When I grow up,
      I start to understand what he said is
      quite right.
    example_title: Fluency
  - text: >-
      Make this text coherent: Their flight
      is weak. They run quickly through
      the tree canopy.
    example_title: Coherence
  - text: >-
      Rewrite to make this easier to understand: A storm surge is what
      forecasters consider a hurricane's most treacherous aspect.
    example_title: Simplification
  - text: >-
      Paraphrase this: Do you know where I was born?
    example_title: Paraphrase
  - text: >-
      Write this more formally: omg i love that song im
      listening to it right now
    example_title: Formalize
  - text: >-
      Write in a more neutral way: The authors' exposé on nutrition studies.
    example_title: Neutralize
---
# Quantized candle weights for the CoEdIT model

Quantized weights of [CoEdIT](https://github.com/vipulraheja/coedit) for inference with [candle](https://github.com/huggingface/candle/tree/main/candle-examples/examples/quantized-t5).

Conversion command, using candle:

```shell
cargo run --example tensor-tools --release -- quantize \
  --quantization q6k \
  /path/to/coedit-<version>/model.safetensors \
  --out-file model<version>.gguf
```