README.md

metadata

library_name: transformers
license: llama2

Llama-3-Smaug-8B

Built with Meta Llama 3

This model was built using the Smaug recipe for improving performance on real world multi-turn conversations applied to meta-llama/Meta-Llama-3-8B.

Model Description

• Developed by: Abacus.AI
• License: https://llama.meta.com/llama3/license/
• Finetuned from model: meta-llama/Meta-Llama-3-8B.

Evaluation

########## First turn ##########
                   score
model             turn
llama3-8b-smaug-2-merged-600 1   8.79375
llama3-8b-smaug-2-merged-150 1   8.71250
llama3-8b-smaug-2-merged-300 1   8.66250
base_Meta-Llama-3-8B-Instruct 1   8.53125
llama3-8b-smaug-2-merged-450 1   8.42500
########## Second turn ##########
                   score
model             turn
llama3-8b-smaug-2-merged-450 2   7.8125
llama3-8b-smaug-2-merged-300 2   7.7375
llama3-8b-smaug-2-merged-600 2   7.7250
llama3-8b-smaug-2-merged-150 2   7.7125
base_Meta-Llama-3-8B-Instruct 2   7.5500
########## Average ##########
                 score
model
llama3-8b-smaug-2-merged-600  8.259375
llama3-8b-smaug-2-merged-150  8.212500
llama3-8b-smaug-2-merged-300  8.200000
llama3-8b-smaug-2-merged-450  8.118750
base_Meta-Llama-3-8B-Instruct 8.040625

Model	First turn	Second Turn	Average
llama3-8b-smaug-2-merged-600	8.79	7.73	8.26
llama3-8b-smaug-2-merged-450	8.43	7.81	8.12
llama3-8b-smaug-2-merged-300	8.66	7.74	8.20
llama3-8b-smaug-2-merged-150	8.71	7.71	8.21
Meta-Llama-3-8B-Instruct	8.53	7.55	8.04