metadata

license: cc-by-4.0
datasets:
  - FinchResearch/AboveTheClouds
language:
  - en
tags:
  - chat

Koss-7B

Training Time: 1.85h

Koss-7B is the smallest variant in the Koss series of neural network models developed by Kaleido AI for natural language processing. With 7 billion parameters, it retains much of the architecture and capabilities of the larger Koss models but requires less computation to run.

Koss-7B is intended for general NLP applications including text classification, language generation, question answering, translation, and dialogue. Its small size makes it suitable for applications with constraints on memory, compute, latency, or carbon emissions.

Factors:

Koss-7B should not be used for tasks requiring very specialized knowledge or skills, since its limited parameters reduce expertise in niche domains. For best performance, finetune on in-domain data.
As with all AI systems, Koss-7B's behavior is dependent on its training data. It may exhibit biases inherited from non-diverse data. Audit data and mitigation strategies to avoid unfair impacts.
Koss-7B is not a creative agent. Its outputs will be limited to recombinations of patterns in its training data. Do not ascribe human-like agency or consciousness.

Recommended Prompt Template:

<s>[INST] {prompt} [/INST] {response} </s>
or
<s>[INST] {prompt} [/INST]

The model will start it's response after the [/INST] Example:

<s>[INST] Why did the chicken cross the road? [/INST] To get to the other side! </s>

Loss

![W&B Chart 01_10_2023, 23_10_39.png]