metadata

emoji: 🦎
colorFrom: green
colorTo: blue
sdk: docker
pinned: false
license: agpl-3.0

Koboldcpp in a Space!

Welcome to the Koboldcpp space, Koboldcpp allows you to easily make your own demonstration spaces of a GGUF model.

For the users

In this space:

You can use the KoboldAI Lite UI for Instructions, Writing, Chat and Adventure use.
You can use the model shown with a KoboldAI compatible API (Use the instance link that is shows + /api) or as an OpenAI compatible API (Use the instance link that it shows, optionally with /v1 if your solution requires this)
In the UI all your data is stored locally without a sign-in.
View the API documentation by accessing the frame link + /api in your browser (For example https://koboldai-koboldcpp-tiefighter.hf.space/api)

For model / space developers

This space was designed to be easy to clone, first make sure you convert your model to the GGUF format and quantize it to something that fits on the GPU you allocated to your space.

If you have a GPU available for your space, clone this space and point the MODEL variable to your model's download location, then force a rebuild so it can use your own custom model. You can customize the model that is being displayed by setting the MODEL_NAME.

In additional variable we configured the parameters required to run this on a GPU and support multiple users and high context, if you wish to clone this to a CPU space simply leave that blank.