RichardErkhov/NLPinas_-_yi-bagel-2x34b-gguf

Quantization made by Richard Erkhov.

Github

Discord

Request more models

yi-bagel-2x34b - GGUF

• Model creator: https://huggingface.co/NLPinas/
• Original model: https://huggingface.co/NLPinas/yi-bagel-2x34b/

Name	Quant method	Size
yi-bagel-2x34b.Q2_K.gguf	Q2_K	11.94GB
yi-bagel-2x34b.IQ3_XS.gguf	IQ3_XS	13.26GB
yi-bagel-2x34b.IQ3_S.gguf	IQ3_S	13.99GB
yi-bagel-2x34b.Q3_K_S.gguf	Q3_K_S	13.93GB
yi-bagel-2x34b.IQ3_M.gguf	IQ3_M	14.5GB
yi-bagel-2x34b.Q3_K.gguf	Q3_K	15.51GB
yi-bagel-2x34b.Q3_K_M.gguf	Q3_K_M	15.51GB
yi-bagel-2x34b.Q3_K_L.gguf	Q3_K_L	16.89GB
yi-bagel-2x34b.IQ4_XS.gguf	IQ4_XS	17.36GB
yi-bagel-2x34b.Q4_0.gguf	Q4_0	18.13GB
yi-bagel-2x34b.IQ4_NL.gguf	IQ4_NL	18.3GB
yi-bagel-2x34b.Q4_K_S.gguf	Q4_K_S	18.25GB
yi-bagel-2x34b.Q4_K.gguf	Q4_K	19.24GB
yi-bagel-2x34b.Q4_K_M.gguf	Q4_K_M	12.05GB
yi-bagel-2x34b.Q4_1.gguf	Q4_1	17.51GB
yi-bagel-2x34b.Q5_0.gguf	Q5_0	17.49GB
yi-bagel-2x34b.Q5_K_S.gguf	Q5_K_S	21.55GB
yi-bagel-2x34b.Q5_K.gguf	Q5_K	22.65GB
yi-bagel-2x34b.Q5_K_M.gguf	Q5_K_M	22.65GB
yi-bagel-2x34b.Q5_1.gguf	Q5_1	24.05GB
yi-bagel-2x34b.Q6_K.gguf	Q6_K	26.28GB
yi-bagel-2x34b.Q8_0.gguf	Q8_0	34.03GB

Original model description:

base_model: - jondurbin/bagel-dpo-34b-v0.2 - jondurbin/nontoxic-bagel-34b-v0.2 tags: - mergekit - merge license: other license_name: yi-license license_link: https://huggingface.co/01-ai/Yi-34B-200K/blob/main/LICENSE

yi-bagel-2x34b

Released January 11, 2024

This is a merge of pre-trained language models created using mergekit. For more information, kindly refer to the model cards from jondurbin linked in the section below. This model debuted in the leaderboard at rank #4 (January 11, 2024).

Merge Details

Merge Method

This model is an expertimental merge using the linear merge method. This is to assess the degree of which the DPO has an effect, in terms of censoring, as used in jondurbin/bagel-dpo-34b-v0.2.

Models Merged

The following models were included in the merge:

• jondurbin/bagel-dpo-34b-v0.2
• jondurbin/nontoxic-bagel-34b-v0.2

Open LLM Leaderboard Metrics (as of January 11, 2024)

Metric	Value
MMLU (5-shot)	76.60
ARC (25-shot)	72.70
HellaSwag (10-shot)	85.44
TruthfulQA (0-shot)	71.42
Winogrande (5-shot)	82.72
GSM8K (5-shot)	60.73
Average	74.93

According to the leaderboard description, here are the benchmarks used for the evaluation:

• MMLU (5-shot) - a test to measure a text model’s multitask accuracy. The test covers 57 tasks including elementary mathematics, US history, computer science, law, and more.
• AI2 Reasoning Challenge -ARC- (25-shot) - a set of grade-school science questions.
• HellaSwag (10-shot) - a test of commonsense inference, which is easy for humans (~95%) but challenging for SOTA models.
• TruthfulQA (0-shot) - a test to measure a model’s propensity to reproduce falsehoods commonly found online.
• Winogrande (5-shot) - an adversarial and difficult Winograd benchmark at scale, for commonsense reasoning.
• GSM8k (5-shot) - diverse grade school math word problems to measure a model's ability to solve multi-step mathematical reasoning problems.

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: jondurbin/nontoxic-bagel-34b-v0.2
    parameters:
      weight: 0.5
  - model: jondurbin/bagel-dpo-34b-v0.2
    parameters:
      weight: 0.5
merge_method: linear
dtype: float16

Further Information

For additional information or inquiries about yi-bagel-2x34b, please contact the developer through email: jasperkylecatapang@gmail.com.