File size: 5,228 Bytes
a29a698 0c79d47 a29a698 c8ed4f6 a29a698 9a09db4 34b672f 12bdc21 34b672f 73f710c a29a698 c65696f 2124656 c65696f a29a698 c65696f a29a698 c65696f a29a698 c65696f |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 |
---
base_model:
- NousResearch/Hermes-2-Pro-Llama-3-8B
- cognitivecomputations/dolphin-2.9-llama3-8b
- NousResearch/Meta-Llama-3-8B
- winglian/llama-3-8b-256k-PoSE
- maum-ai/Llama-3-MAAL-8B-Instruct-v0.1
- asiansoul/Llama-3-Open-Ko-Linear-8B
- NousResearch/Meta-Llama-3-8B-Instruct
- nvidia/Llama3-ChatQA-1.5-8B
- Danielbrdz/Barcenas-Llama3-8b-ORPO
- aaditya/Llama3-OpenBioLLM-8B
library_name: transformers
tags:
- mergekit
- merge
- llama
---
# YACHT-Llama-3-Ko-8B
[](https://ibb.co/92BXmfz)
🎵 *[JayLee LLMs Signature Tag] : ✍️ "I need a Jay Jay chat boy"* 🎵
✨ *Navigating the High Seas of Data: Crafting the Ultimate Yacht Insights with Merged LLMs* ✨
## 🏟️ Merged Model Series Yacht Features
Welcome to the merged model series yacht! This provides an overview of the powerful features and functionalities that this series brings together, akin to a sleek, modern yacht sailing across the digital ocean.
### 1. Function Calling & JSON Outputs
- Offers precise function calling and structured JSON outputs via specialized tokens like `<tools>`, `<tool_call>`, and `<tool_response>`. Streamlines system communication for developers.
### 2. Conversational Interaction
- Avoids excessive "SYSTEM MESSAGE" chatter while delivering seamless, friendly dialogue.
- Specializes in answering questions with precision, handling arithmetic and tabular data effortlessly.
### 3. Expanded Context Length
- Extends the context length to 256k tokens using PoSE, offering a broader field of data analysis.
### 4. Multilingual Capabilities
- Transfers instruction-following from English to Korean for reliable interaction across languages.
### 5. Optimized Dialogue & Safety
- Aligns with human preferences using fine-tuning (SFT) and reinforcement learning (RLHF), ensuring helpful and safe dialogues.
### 6. Precision Merging
- Merges foundational and preview models for Korean language through task arithmetic, providing seamless integration.
### 7. Specialized Biomedical Knowledge
- Specializes in biomedical tasks with accurate responses for healthcare professionals and researchers.
### 8. Novel Training & Collaboration
- Combines [ORPO method](https://arxiv.org/pdf/2403.07691) and dolphin preference datasets for high-quality conversation and collaboration.
The merged model series yacht offers unparalleled functionality, drawing together a fleet of specialized models. Whether you need precise function calling, multilingual capabilities, or conversational AI, this yacht has every deck optimized to navigate the digital ocean with style and precision.
## 👘 Merge Method
This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method using [NousResearch/Meta-Llama-3-8B](https://huggingface.co/NousResearch/Meta-Llama-3-8B) as a base.
## 🩱 Models Merged
The following models were included in the merge:
* [NousResearch/Hermes-2-Pro-Llama-3-8B](https://huggingface.co/NousResearch/Hermes-2-Pro-Llama-3-8B)
* [cognitivecomputations/dolphin-2.9-llama3-8b](https://huggingface.co/cognitivecomputations/dolphin-2.9-llama3-8b)
* [winglian/llama-3-8b-256k-PoSE](https://huggingface.co/winglian/llama-3-8b-256k-PoSE)
* [maum-ai/Llama-3-MAAL-8B-Instruct-v0.1](https://huggingface.co/maum-ai/Llama-3-MAAL-8B-Instruct-v0.1)
* [asiansoul/Llama-3-Open-Ko-Linear-8B](https://huggingface.co/asiansoul/Llama-3-Open-Ko-Linear-8B)
* [NousResearch/Meta-Llama-3-8B-Instruct](https://huggingface.co/NousResearch/Meta-Llama-3-8B-Instruct)
* [nvidia/Llama3-ChatQA-1.5-8B](https://huggingface.co/nvidia/Llama3-ChatQA-1.5-8B)
* [Danielbrdz/Barcenas-Llama3-8b-ORPO](https://huggingface.co/Danielbrdz/Barcenas-Llama3-8b-ORPO)
* [aaditya/Llama3-OpenBioLLM-8B](https://huggingface.co/aaditya/Llama3-OpenBioLLM-8B)
## 🪭 Configuration
The following YAML configuration was used to produce this model:
```yaml
models:
- model: NousResearch/Meta-Llama-3-8B
# Base model providing a general foundation without specific parameters
- model: NousResearch/Meta-Llama-3-8B-Instruct
parameters:
density: 0.60
weight: 0.25
- model: winglian/llama-3-8b-256k-PoSE
parameters:
density: 0.55
weight: 0.15
- model: nvidia/Llama3-ChatQA-1.5-8B
parameters:
density: 0.55
weight: 0.1
- model: asiansoul/Llama-3-Open-Ko-Linear-8B
parameters:
density: 0.55
weight: 0.2
- model: maum-ai/Llama-3-MAAL-8B-Instruct-v0.1
parameters:
density: 0.55
weight: 0.1
- model: NousResearch/Hermes-2-Pro-Llama-3-8B
parameters:
density: 0.55
weight: 0.1
- model: cognitivecomputations/dolphin-2.9-llama3-8b
parameters:
density: 0.55
weight: 0.05
- model: Danielbrdz/Barcenas-Llama3-8b-ORPO
parameters:
density: 0.55
weight: 0.05
- model: aaditya/Llama3-OpenBioLLM-8B
parameters:
density: 0.55
weight: 0.1
merge_method: dare_ties
base_model: NousResearch/Meta-Llama-3-8B
parameters:
int8_mask: true
dtype: bfloat16
``` |