--- library_name: peft base_model: unsloth/gemma-2-9b-it-bnb-4bit language: - ja - en tags: - translation - qlora - gemma2 - text-generation-inference - nlp --- ![image/png](https://cdn-uploads.huggingface.co/production/uploads/630469550907b9a115c91e62/m11e35NrZMi7ZpBQ7C6KV.png) # News ## 2024.07.19 C3TR-AdapterのVersion3を公開しました。 Version 3 of C3TR-Adapter has been released. このドキュメント等は現在、差し替え作業中で一部古いです。 This documentation is currently being replaced and some parts are out of date. ## 2024.05.17 C3TR-AdapterのVersion2を公開しました。 Version 2 of C3TR-Adapter has been released. Version2では主にカジュアルな会話に関する翻訳能力が大幅に向上しています。 Version 2 has greatly improved the ability to translate casual conversations. その反面、フォーマルな文章の翻訳能力が少し落ちてしまっています。フォーマルな文章を対象にする場合、[Version1](https://huggingface.co/webbigdata/C3TR-Adapter/tree/version1)を引き続きお使いください On the other hand, translation capabilities for formal texts have declined slightly. If you are targeting formal texts, please continue to use [Version1](https://huggingface.co/webbigdata/C3TR-Adapter/tree/version1). # モデルカード(Model Card for Model ID) C3TR-AdapterはGoogleが発表したLLMであるgemma-7bの日英・英日翻訳性能を向上させるQLoRA Adapterです。 C3TR-Adapter is a QLoRA Adapter that improves the Japanese-English and English-Japanese translation performance of gemma-7b released by Google. ## モデル詳細(Model Details) C3TR-Adapterは翻訳ベンチマークで多言語翻訳モデルであるGoogleのMadlad400やmetaのSeamless m4t v2 large、[ALMA-Ja-V2](https://huggingface.co/webbigdata/ALMA-7B-Ja-V2) (私達の以前のllama 2ベースのモデル)よりも大幅に優れた日英・日英翻訳性能を持っています。 Benchmarks show significantly better English-Japanese and Japanese-English translation performance than Google's Madlad400, META's Seamless m4t v2 large, and [ALMA-Ja-V2](https://huggingface.co/webbigdata/ALMA-7B-Ja-V2) (our previous llama2 model). ![image/png](https://cdn-uploads.huggingface.co/production/uploads/630469550907b9a115c91e62/ZqbugHB-X0Sgm_GnBhfz4.png) 翻訳タスクに関しては、より大きなモデルに負けない性能を発揮します 元の画像クレジット Sebastian Ruder(@seb_ruder) (※FloRES実行時はwriting_style: journalistic、WMT23実行時はwriting_style: casualを指定。wmt23.ja-en時は一行だけ改行不揃いを手修正) For translation tasks, it performs as well as larger models. Original image credit: Sebastian Ruder (@seb_ruder) (*When running FloRES, specify writing_style: journalistic, and when running WMT23, specify writing_style: casual. When running wmt23.ja-en, one line was manually corrected for line breaks.) GoogleのウェブサービスColabを使うと無料でC3TR-Adapterを試す事が出来ます。リンク先でOpen In Colabボタンを押して起動してください。 You can try C3TR-Adapter for free using Google's web service Colab. Please press the Open In Colab button on the link to activate it. - [動作確認用の簡単なサンプル(A simple sample to check the operation)](https://github.com/webbigdata-jp/python_sample/blob/main/C3TR_Adapter_v2_Japanese_English_Translation_sample_code.ipynb) - [テキストファイルを一括で日英・英日翻訳するサンプル(Sample of batch translation of text files)](https://github.com/webbigdata-jp/python_sample/blob/main/C3TR_Adapter_v2_batch_translation_sample.ipynb) - [GPUがない環境でも動かす事ができるgguf版(A gguf version that can be run in environments without a GPU)](https://huggingface.co/webbigdata/C3TR-Adapter_gguf) ### モデルの動かし方(How to use Model) 自分のパソコンで動かす場合は、少なくとも約8.3GB以上のGPU RAMが必要です。GPUメモリが足りない場合は上記のgguf版を試すか、パラメーターを調整してください(max_length、max_new_tokens, num_beamsを減らす) If you want to run it on your own local computer, you will need at least approximately 8.3 GB or more of GPU RAM.If you do not have enough GPU memory, try the gguf version above or decrease parameters(max_length、max_new_tokens, num_beams). 必要なライブラリのインストール(Installation of required libraries) ``` # もし、pytorchがまだインストールされていなかったら公式マニュアルを参考にインストールしてください # If pytorch is not already installed, please refer to the official manual to install it. # https://pytorch.org/get-started/locally/#start-locally # example for linux user with CUDA 12.1. # pip3 install torch torchvision torchaudio # example for windows user with CUDA 12.1. # pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121 # Gemmaは最新のライブラリでなくては動かないので、以下のVersionに更新してください # Gemma will not work without the latest library, so please update to the following version pip install transformers==4.40.2 pip install peft==0.10.0 pip install bitsandbytes==0.43.0 ``` サンプルスクリプト(sample script) ``` import torch import os import json from transformers import AutoModelForCausalLM, AutoTokenizer from peft import PeftModel model_id = "unsloth/gemma-7b-bnb-4bit" peft_model_id = "webbigdata/C3TR-Adapter" model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto") model = PeftModel.from_pretrained(model = model, model_id = peft_model_id) tokenizer = AutoTokenizer.from_pretrained(model_id) tokenizer.pad_token = tokenizer.unk_token def trans(my_str): input_ids = tokenizer(my_str, return_tensors="pt", padding=True, max_length=1600, truncation=True).input_ids.cuda() # Translation generated_ids = model.generate(input_ids=input_ids, max_new_tokens=800, use_cache=True, do_sample=True, num_beams=3, temperature=0.5, top_p=0.3, repetition_penalty=1.0 ) full_outputs = tokenizer.batch_decode(generated_ids, skip_special_tokens=True) return full_outputs[0].split("### Response:\n")[-1].strip() ret = trans("""You are a highly skilled professional Japanese-English and English-Japanese translator. Translate the given text accurately, taking into account the context and specific instructions provided. Steps may include hints enclosed in square brackets [] with the key and value separated by a colon:. Only when the subject is specified in the Japanese sentence, the subject will be added when translating into English. If no additional instructions or context are provided, use your expertise to consider what the most appropriate context is and provide a natural translation that aligns with that context. When translating, strive to faithfully reflect the meaning and tone of the original text, pay attention to cultural nuances and differences in language usage, and ensure that the translation is grammatically correct and easy to read. After completing the translation, review it once more to check for errors or unnatural expressions. For technical terms and proper nouns, either leave them in the original language or use appropriate translations as necessary. Take a deep breath, calm down, and start translating. ### Instruction: Translate Japanese to English. ### Input: あら？また夜食を食べてるの？ こんにゃくは太りません ### Response: """) print(ret) ``` ### プロンプトフォーマット prompt format プロンプトフォーマットは独自です。 The prompt format is original. Version1とVersion2ではプロンプトフォーマットも変わっており システムプロンプトも追加されています。 The prompt format has changed between Version 1 and Version 2, and system prompts have also been added. 余分な空白や改行はモデルの誤動作に繋がるのでテンプレートにミスがないようにしてください Make sure there are no mistakes in the template, as extra spaces or line breaks can lead to malfunctioning of the model. Instructionsは"Translate Japanese to English."(日英翻訳)と"Translate English to Japanese."(英日翻訳)の2種類です。 There are two types of instructions: "Translate Japanese to English." and "Translate English to Japanese.". Version2からは実験的な試みとして、翻訳時に4つのタイプのヒントを与える事が出来るようになっています。 From Version 2, as an experimental trial, it is possible to give four types of hints when translating. ### (1)文体(writing style) [writing_style: STYLE_NAME] 仕事場などではbusinessを使います In the workplace, we use business. ``` You are a highly skilled professional Japanese-English and English-Japanese translator. Translate the given text accurately, taking into account the context and specific instructions provided. Steps may include hints enclosed in square brackets [] with the key and value separated by a colon:. Only when the subject is specified in the Japanese sentence, the subject will be added when translating into English. If no additional instructions or context are provided, use your expertise to consider what the most appropriate context is and provide a natural translation that aligns with that context. When translating, strive to faithfully reflect the meaning and tone of the original text, pay attention to cultural nuances and differences in language usage, and ensure that the translation is grammatically correct and easy to read. After completing the translation, review it once more to check for errors or unnatural expressions. For technical terms and proper nouns, either leave them in the original language or use appropriate translations as necessary. Take a deep breath, calm down, and start translating. ### Instruction: Translate Japanese to English. When translating, please use the following hints: [writing_style: business] ### Input: お疲れ様です、本日の資料を送ります。 ### Response: Thank you for your hard work today. I am sending today's materials. ``` 以降の例ではsystem promptを省略しています。 In the following example, the system prompt is omitted. コピペなどではslangやcasualを使います Use slang or casual language when meme. ``` ### Instruction: Translate Japanese to English. When translating, please use the following hints: [writing_style: slang] [牛鮭定食: Beef salmon set meal] ### Input: そんな事より >>1 よ、ちょいと聞いてくれよ。スレとあんま関係ないけどさ。 このあいだ、近所の吉野家行ったんです。吉野家。 そしたらなんか人がめちゃくちゃいっぱいで座れないんです。 で、よく見たらなんか垂れ幕下がってて、150円引き、とか書いてあるんです。 もうね、アホかと。馬鹿かと。 お前らな、150円引き如きで普段来てない吉野家に来てんじゃねーよ、ボケが。 150円だよ、150円。 なんか親子連れとかもいるし。一家4人で吉野家か。おめでてーな。 よーしパパ特盛頼んじゃうぞー、とか言ってるの。もう見てらんない。 お前らな、150円やるからその席空けろと。 吉野家ってのはな、もっと殺伐としてるべきなんだよ。 Uの字テーブルの向かいに座った奴といつ喧嘩が始まってもおかしくない、 刺すか刺されるか、そんな雰囲気がいいんじゃねーか。女子供は、すっこんでろ。 で、やっと座れたかと思ったら、隣の奴が、大盛つゆだくで、とか言ってるんです。 そこでまたぶち切れですよ。 あのな、つゆだくなんてきょうび流行んねーんだよ。ボケが。 得意げな顔して何が、つゆだくで、だ。 お前は本当につゆだくを食いたいのかと問いたい。問い詰めたい。小1時間問い詰めたい。 お前、つゆだくって言いたいだけちゃうんかと。 吉野家通の俺から言わせてもらえば今、吉野家通の間での最新流行はやっぱり、 ねぎだく、これだね。 大盛りねぎだくギョク。これが通の頼み方。 ねぎだくってのはねぎが多めに入ってる。そん代わり肉が少なめ。これ。 で、それに大盛りギョク(玉子)。これ最強。 しかしこれを頼むと次から店員にマークされるという危険も伴う、諸刃の剣。 素人にはお薦め出来ない。 まあお前、>>1は、牛鮭定食でも食ってなさいってこった ### Response: >1 Hey, listen up. This has nothing to do with the thread. I went to Yoshinoya the other day. Yoshinoya. And there were so many people that there was no place to sit. So I looked around, and there was a sign hanging down, 150 yen off or something. I mean, what a joke. What an idiot. You guys, don't go to Yoshinoya just because it's 150 yen off, you morons. It's 150 yen, 150 yen. There were some families too. A family of four at Yoshinoya. Congratulations. Alright, papa will order a special. I can't even look at that. You guys, give up that seat if you're willing to pay 150 yen. Yoshinoya should be more hardcore. You could start a fight with anyone sitting across the U-shaped table at any moment, stabbing or getting stabbed, that's the atmosphere I like. Women and children should shut up. And then you think you've finally got a seat, and the guy next to you says, "I'll have a large tsuyu-daku." And then there's another fight. You know, tsuyu-daku isn't popular these days. Moron. What's so proud about having tsuyu-daku. I want to ask you if you really want to eat tsuyu-daku. I want to interrogate you. I want to interrogate you for an hour. Do you just want to say tsuyu-daku? From a Yoshinoya regular like me, the latest trend among Yoshinoya regulars is negi-daku. Negi-daku gyoku. That's the way to order. Negi-daku means more onions and less meat. That's it. And then there's negi-daku gyoku (tamago). That's the best. But there's also the risk of being marked by the staff if you order this. A double-edged sword. I wouldn't recommend it to beginners. Well, you, >>1, just eat a beef salmon set meal. ``` 現在は試験的に11のwriteing styleをテスト実装しています。 We are currently testing 11 writing styles. casual, formal, technical, journalistic, web-fiction, business, nsfw, educational-casual, academic-presentation, slang, sns-casual #### (2)固有名詞の読み方 How to read proper nouns [英語名称: 日本語訳] またはその逆。 [English name: Japanese translation name] or vice versa ``` ### Instruction: Translate Japanese to English. When translating, please use the following hints: [writing_style: formal] [羽生結弦: Yuzuru Hanyu] [羽生善治: Yoshiharu Habu] ### Input: フィギュアスケートの羽生結弦さんが将棋棋士の羽生善治さんと対談した ### Response: Figure skater Yuzuru Hanyu had a conversation with shogi player Yoshiharu Habu. ``` #### (3)キャラクタースタイル character_style [XXXX_character_style: YYYY] キャラクタースタイルで性別や個性を指定する事ができます You can specify gender and personality in the character style. 男性指定 Male designated ``` ### Instruction: Translate Japanese to English. When translating, please use the following hints: [writing_style: formal] [青山樹_character_style: male] [青山樹: AOYAMA Itsuki] ### Input: 青山樹は週末に友達とキャンプに行って、自然を楽しんだ。そして時計を紛失した。 ### Response: Aoyama Itsuki went camping with his friends on the weekend and enjoyed nature. However, he lost his watch. ``` 女性指定 Feale designated ``` ### Instruction: Translate Japanese to English. When translating, please use the following hints: [writing_style: formal] [青山樹_character_style: female] [青山樹: Itsuki Aoyama] ### Input: 青山樹は週末に友達とキャンプに行って、自然を楽しんだ。そして時計を紛失した。 ### Response: Itsuki Aoyama went camping with friends on the weekend and enjoyed nature. However, she lost her watch. ``` ノンバイナリー指定 nonbinary designated ``` ### Instruction: Translate Japanese to English. When translating, please use the following hints: [writing_style: formal] [青山樹_character_style: nonbinary] [青山樹: Tatsuki Aoyama] ### Input: 青山樹は週末に友達とキャンプに行って、自然を楽しんだ。そして時計を紛失した。 ### Response: Tatsuki Aoyama went camping with their friends on the weekend and enjoyed nature. They lost their watch. ``` 残念ながら現時点では性別の指定は本文の内容が優先されるため、例えば以下の文章では性別指定が有効になりません。 以下の例では本文内の「俺は男だよ!」を消せば性別指定が有効になります。 また、bfloat16が扱えないColabの無料版などではこの指定が無視されてしまうようです。 Unfortunately, at present, the content of the text takes priority when designating gender, so for example, the gender designation will not be effective in the following sentence. In the example below, if you delete "俺は男だよ!(I'm a guy!)" from the text, the gender specification will be effective. Also, this specification seems to be ignored in the free version of Colab, which cannot handle bfloat16. ``` ### Instruction: Translate Japanese to English. When translating, please use the following hints: [writing_style: web-fiction] [カミーユ: kamille] [kamille_character_style: female, rough, teenager] [ジュリド: Jerid] [ティターンズ: Titans] [エマ: Emma] [エゥーゴ: A.E.U.G.] ### Input: ジェリド｢カミーユ？女の名前なのに・・・何だ、男か。｣ カミーユ｢なめるな!!｣ ジェリド｢うわ!｣ エマ｢やめなさい!｣ ジェリド｢オレ達をティターンズと知ってちょっかいを出してきたのか?｣ カミーユ｢カミーユが男の名前で何で悪いんだ!!!俺は男だよ!｣ こうして地球連邦のエリート部隊・ティターンズを殴った罪で拘束された後、母を失い、反地球連邦組織『エゥーゴ』に参加しました。 ### Response: Jerid: "Kamille? A girl's name, yet... What's with you, a guy?" Kamille: "Don't you dare!" Jerid: "Yikes!" Emma: "Stop it!" Jerid: "Are you provoking us by knowing we're Titans?" Kamille: "What's wrong with Kamille being a guy's name!!! I'm a man!" Thus, after being detained for assaulting the Earth Federation's elite force, the Titans, Kamille lost his mother and joined the anti-Earth Federation organization, the A.E.U.G. ``` character_styleとwriting_styleを組み合わせる Combining character_style and writing_style 以下の例では段々と丁寧な言い回しに変化しています In the following example, the phrase gradually changes to a more polite one. ``` ### Instruction: Translate Japanese to English. When translating, please use the following hints: [writing_style: slang] [speaker_character_style: vulgar] ### Input: 今日の会議は非常に重要ですので、時間通りに来てください。 ### Response: Today’s meeting is very important, so please come on time. ``` ``` ### Instruction: Translate Japanese to English. When translating, please use the following hints: [writing_style: casual] [speaker_character_style: rough] ### Input: 今日の会議は非常に重要ですので、時間通りに来てください。 ### Response: The meeting today is very important, so please come on time. ``` ``` ### Instruction: Translate Japanese to English. When translating, please use the following hints: [writing_style: formal] [speaker_character_style: noble] ### Input: 今日の会議は非常に重要ですので、時間通りに来てください。 ### Response: As today's meeting is of utmost importance, please arrive on time. ``` 日本語でも同様に丁寧になっていっています The Japanese language is also becoming more polite. ``` ### Instruction: Translate English to Japanese. When translating, please use the following hints: [writing_style: slang] [speaker_character_style: vulgar] ### Input: Since today's meeting is very important, please arrive on time. ### Response: 今日の会議はとても重要なので、時間通りに来るように。 ``` ``` ### Instruction: Translate English to Japanese. When translating, please use the following hints: [writing_style: casual] [speaker_character_style: rough] ### Input: Since today's meeting is very important, please arrive on time. ### Response: 今日の会議はとても重要なので、時間通りに来るようにしてください。 ``` ``` ### Instruction: Translate English to Japanese. When translating, please use the following hints: [writing_style: formal] [speaker_character_style: noble] ### Input: Since today's meeting is very important, please arrive on time. ### Response: 今日の会議は非常に重要ですので、時間どおりにお越しください。 ``` #### (4)一人称と語尾(First person and ending) キャラクターの一人称と語尾を指定する事ができます。 この機能はまだ非常に実験的な機能であり、現時点では不完全です。 You can specify the first person and ending for your character. This feature is still highly experimental and incomplete at this time. ``` ### Instruction: Translate English to Japanese. [writing_style: casual, anime] [squid girl: イカ娘] [squid girl_first_person_and_ending: 私, イカ, ゲソ] [marisa: 魔理沙] [marisa_first_person_and_ending: 俺, ぜ, だぜ] [reimu: 霊夢] [reimu_first_person_and_ending: 私, かしら, だわ] ### Input: Marisa: Hey Reimu, check out this cool squid girl I just met! Reimu: A squid girl? How unusual. What brings you to Gensokyo? Squid Girl: I'm on a mission to invade the surface world, starting with this land! ika! Marisa: Haha, good luck with that! Reimu and I have dealt with way worse than some little squid. Squid Girl: I'm not just some squid! I'll show you the power of the sea! ...Eventually! geso ### Response: 魔理沙: 霊夢、このイカ娘に会ったぜ！ 霊夢: イカ娘？珍しいな。何で幻想郷に来たのかしら？ イカ娘: 地上世界を征服する任務を帯びて、まずはこの土地から始めるイカゲソ！ 魔理沙: 笑うぜ、イカ娘。霊夢と俺には、イカなんかよりひどい奴らと戦ったことがあるんだぜ。 イカ娘: 私、ただのイカじゃないゲソ！海の力を見せつけるゲソ！...いつかゲソ！ ``` ``` ### Instruction: Translate Japanese to English. When translating, please use the following hints: [writing_style: casual, game] [初春: hatsuharu] [時雨: shigure] [夕立: yuudachi] [yuudachi_first_person_and_ending: poi] [白露: shiratsuyu] ### Input: 時雨「雨は、いつか止むさ」 初春「わらわは、北方部隊に所属。戦雲渦巻くアッツやキスカなどの北方海域で活躍したぞ。」 夕立「白露型駆逐艦・夕立！今日も頑張るっぽい！ぽ～いっ！」 ### Response: Shigure: "The rain will eventually stop." Hatsuharu: "I belong to the Northern Fleet. I was active in the northern seas around Attu and Kiska, where the clouds of war swirl." Yuudachi: "Shiratsuyu-class destroyer, Yuudachi! I'll do my best today too! Po~i!" ``` ## 留意事項 Attention このアダプターをモデルとマージして保存すると性能が下がってしまう不具合が存在するため、**ベースモデル(gemma-7b-bnb-4bit)とアダプターをマージして保存しないでください** **Do not save this adapter merged with the base model(gemma-7b-bnb-4bit)**, as there exists a bug that reduces performance when saving this adapter merged with the model. どうしてもマージしたい場合は必ずPerplexityではなく、翻訳ベンチマークで性能を確認してから使うようにしてください If you must merge, be sure to use a translation benchmark to check performance, not Perplexity! ## その他の版 Other Version - [webbigdata/C3TR-Adapter_gptq](https://huggingface.co/webbigdata/C3TR-Adapter_gptq) GPTQ 4bit quantized version - [webbigdata/C3TR-Adapter_hqq](webbigdata/C3TR-Adapter_hqq) HQQ 4bit quantized version ### 利用規約 Terms of Use 基本的にはgemmaと同じライセンスです Basically the same license as gemma. 加えて貴方に以下のお願いがあります。 Additionally, We have the following request to you. 私たちの以前のモデルであるALMA-7B-Ja-V2のダウンロード件数は15万件を超えているのですが、どんな人がどのような場面で使っているのか全く把握できていません。 Our previous model, ALMA-7B-Ja-V2, has over 150K downloads, but we have no idea who is using it and in what situations. そのため、使用した後は[Googleフォームに感想や今後期待する方向性、気が付いた誤訳の例、参考にして欲しいデータの場所、Webサイトなどを是非とも記入](https://forms.gle/Ycr9nWumvGamiNma9)してください。 So, after you use it, please [fill out the Google form below with your impressions, future directions you expect us to take, examples of mistranslations you have noticed, and locations of data you would like us to reference, websites, etc.](https://forms.gle/Ycr9nWumvGamiNma9) by all means. 個人情報やメールアドレスは収集しないので、気軽にご記入をお願いします We do not collect personal information or email address, so please feel free to fill out the form! どんなご意見でも感謝します！ Any feedback would be appreciated! ### 謝辞 Acknowledgment Original Base Model google/gemma-2-9b-it https://huggingface.co/google/gemma-2-9b-it Base Model unsloth/gemma-7b-bnb-4bit https://huggingface.co/unsloth/gemma-2-9b-it-bnb-4bit QLoRA Adapter webbigdata/C3TR-Adapter https://huggingface.co/webbigdata/C3TR-Adapter This adapter was trained with Unsloth. https://github.com/unslothai/unsloth その他、[ALMA](https://arxiv.org/abs/2309.11674)をはじめ、コミュニティの皆さんからヒントを貰っています。ありがとう Other tips I have received from [ALMA](https://arxiv.org/abs/2309.11674) and others in the community. Thank you. - **Developed by:** [webbigdata](https://webbigdata.jp/)