--- library_name: transformers tags: - mergekit - merge base_model: [] model-index: - name: NemoReRemix-12B results: - task: type: text-generation name: Text Generation dataset: name: ENEM Challenge (No Images) type: eduagarcia/enem_challenge split: train args: num_few_shot: 3 metrics: - type: acc value: 73.55 name: accuracy source: url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=MarinaraSpaghetti/NemoReRemix-12B name: Open Portuguese LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: BLUEX (No Images) type: eduagarcia-temp/BLUEX_without_images split: train args: num_few_shot: 3 metrics: - type: acc value: 62.17 name: accuracy source: url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=MarinaraSpaghetti/NemoReRemix-12B name: Open Portuguese LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: OAB Exams type: eduagarcia/oab_exams split: train args: num_few_shot: 3 metrics: - type: acc value: 56.17 name: accuracy source: url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=MarinaraSpaghetti/NemoReRemix-12B name: Open Portuguese LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: Assin2 RTE type: assin2 split: test args: num_few_shot: 15 metrics: - type: f1_macro value: 91.6 name: f1-macro source: url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=MarinaraSpaghetti/NemoReRemix-12B name: Open Portuguese LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: Assin2 STS type: eduagarcia/portuguese_benchmark split: test args: num_few_shot: 15 metrics: - type: pearson value: 80.23 name: pearson source: url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=MarinaraSpaghetti/NemoReRemix-12B name: Open Portuguese LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: FaQuAD NLI type: ruanchaves/faquad-nli split: test args: num_few_shot: 15 metrics: - type: f1_macro value: 78.26 name: f1-macro source: url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=MarinaraSpaghetti/NemoReRemix-12B name: Open Portuguese LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: HateBR Binary type: ruanchaves/hatebr split: test args: num_few_shot: 25 metrics: - type: f1_macro value: 81.95 name: f1-macro source: url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=MarinaraSpaghetti/NemoReRemix-12B name: Open Portuguese LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: PT Hate Speech Binary type: hate_speech_portuguese split: test args: num_few_shot: 25 metrics: - type: f1_macro value: 63.75 name: f1-macro source: url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=MarinaraSpaghetti/NemoReRemix-12B name: Open Portuguese LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: tweetSentBR type: eduagarcia/tweetsentbr_fewshot split: test args: num_few_shot: 25 metrics: - type: f1_macro value: 72.51 name: f1-macro source: url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=MarinaraSpaghetti/NemoReRemix-12B name: Open Portuguese LLM Leaderboard --- ![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/6550b16f7490049d6237f200/p0ZBFYc1RNoYcowv3Nj40.jpeg) ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6550b16f7490049d6237f200/peSTK513q1WOfRfRzWga4.png) # Information ## Details Improved NemoRemix for storytelling and roleplay. Plus, this one can also be used as a general assistant model. The prose is pretty much the same, but it was made smarter, thanks to the addition of the amazing Migtissera's Tess model. I yeeted out Gryphe's Pantheon-RP, though, because it was trained with asterisks in mind, unlike the rest of the models in the merge, which caused it to mess the formatting from time to time; this one doesn't do that anymore. Hooray! All credits and thanks go to the amazing Migtissera, MistralAI, Anthracite, Sao10K and ShuttleAI for their amazing models. ## Instruct ChatML but Mistral Instruct should work too (theoretically). Important: remember to add <|im_end|> to custom stopping strings, otherwise it will appear in the output. ``` <|im_start|>system {system}<|im_end|> <|im_start|>user {message}<|im_end|> <|im_start|>assistant {response}<|im_end|> ``` ## Parameters I recommend running Temperature 1.0-1.2 with 0.1 Top A or 0.01-0.1 Min P, and with 0.8/1.75/2/0 DRY. Also works with lower Temperatures below 1.0. Nothing more needed. ### Settings You can use my exact settings from here (use the ones from the ChatML Base/Customized folder): https://huggingface.co/MarinaraSpaghetti/SillyTavern-Settings/tree/main. ## GGUF https://huggingface.co/MarinaraSpaghetti/NemoReRemix-GGUF # NemoReRemix-12B This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit). ## Merge Details ### Merge Method This model was merged using the della_linear merge method using E:\mergekit\mistralaiMistral-Nemo-Base-2407 as a base. ### Models Merged The following models were included in the merge: * E:\mergekit\Sao10K_MN-12B-Lyra-v1 * E:\mergekit\mistralaiMistral-Nemo-Instruct-2407 * E:\mergekit\migtissera_Tess-3-Mistral-Nemo * E:\mergekit\shuttleai_shuttle-2.5-mini * E:\mergekit\anthracite-org_magnum-12b-v2 ### Configuration The following YAML configuration was used to produce this model: ```yaml models: - model: E:\mergekit\mistralaiMistral-Nemo-Instruct-2407 parameters: weight: 0.1 density: 0.4 - model: E:\mergekit\Sao10K_MN-12B-Lyra-v1 parameters: weight: 0.12 density: 0.5 - model: E:\mergekit\shuttleai_shuttle-2.5-mini parameters: weight: 0.2 density: 0.6 - model: E:\mergekit\migtissera_Tess-3-Mistral-Nemo parameters: weight: 0.25 density: 0.7 - model: E:\mergekit\anthracite-org_magnum-12b-v2 parameters: weight: 0.33 density: 0.8 merge_method: della_linear base_model: E:\mergekit\mistralaiMistral-Nemo-Base-2407 parameters: epsilon: 0.05 lambda: 1 dtype: bfloat16 ``` # Ko-fi ## Enjoying what I do? Consider donating here, thank you! https://ko-fi.com/spicy_marinara # Open Portuguese LLM Leaderboard Evaluation Results Detailed results can be found [here](https://huggingface.co/datasets/eduagarcia-temp/llm_pt_leaderboard_raw_results/tree/main/MarinaraSpaghetti/NemoReRemix-12B) and on the [🚀 Open Portuguese LLM Leaderboard](https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard) | Metric | Value | |--------------------------|---------| |Average |**73.36**| |ENEM Challenge (No Images)| 73.55| |BLUEX (No Images) | 62.17| |OAB Exams | 56.17| |Assin2 RTE | 91.60| |Assin2 STS | 80.23| |FaQuAD NLI | 78.26| |HateBR Binary | 81.95| |PT Hate Speech Binary | 63.75| |tweetSentBR | 72.51|