File size: 177,145 Bytes
905d8e1 |
1 |
{"cells":[{"cell_type":"markdown","source":["To run this, press \"*Runtime*\" and press \"*Run all*\" on a **free** Tesla T4 Google Colab instance!\n","<div class=\"align-center\">\n"," <a href=\"https://github.com/unslothai/unsloth\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png\" width=\"115\"></a>\n"," <a href=\"https://discord.gg/u54VK8m8tk\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/Discord button.png\" width=\"145\"></a>\n"," <a href=\"https://ko-fi.com/unsloth\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/Kofi button.png\" width=\"145\"></a></a> Join Discord if you need help + ⭐ <i>Star us on <a href=\"https://github.com/unslothai/unsloth\">Github</a> </i> ⭐\n","</div>\n","\n","To install Unsloth on your own computer, follow the installation instructions on our Github page [here](https://github.com/unslothai/unsloth?tab=readme-ov-file#-installation-instructions).\n","\n","You will learn how to do [data prep](#Data), how to [train](#Train), how to [run the model](#Inference), & [how to save it](#Save) (eg for Llama.cpp).\n","\n","**[NEW] Gemma2-9b is trained on 8 trillion tokens! Gemma2-27b is 13 trillion!**"],"metadata":{"id":"IqM-T1RTzY6C"}},{"cell_type":"code","execution_count":1,"metadata":{"id":"2eSvM9zX_2d3","executionInfo":{"status":"ok","timestamp":1727858074739,"user_tz":-540,"elapsed":53054,"user":{"displayName":"최현진","userId":"12812404047517020320"}}},"outputs":[],"source":["%%capture\n","!pip install unsloth\n","# Also get the latest nightly Unsloth!\n","!pip uninstall unsloth -y && pip install --upgrade --no-cache-dir \"unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git\"\n","\n","# Install Flash Attention 2 for softcapping support\n","import torch\n","if torch.cuda.get_device_capability()[0] >= 8:\n"," !pip install --no-deps packaging ninja einops \"flash-attn>=2.6.3\""]},{"cell_type":"markdown","source":["* We support Llama, Mistral, Phi-3, Gemma, Yi, DeepSeek, Qwen, TinyLlama, Vicuna, Open Hermes etc\n","* We support 16bit LoRA or 4bit QLoRA. Both 2x faster.\n","* `max_seq_length` can be set to anything, since we do automatic RoPE Scaling via [kaiokendev's](https://kaiokendev.github.io/til) method.\n","* With [PR 26037](https://github.com/huggingface/transformers/pull/26037), we support downloading 4bit models **4x faster**! [Our repo](https://huggingface.co/unsloth) has Llama, Mistral 4bit models.\n","* [**NEW**] We make Phi-3 Medium / Mini **2x faster**! See our [Phi-3 Medium notebook](https://colab.research.google.com/drive/1hhdhBa1j_hsymiW9m-WzxQtgqTH_NHqi?usp=sharing)"],"metadata":{"id":"r2v_X2fA0Df5"}},{"cell_type":"code","execution_count":2,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":336,"referenced_widgets":["0a6c54edc08e44ee9cf092414a4e42ec","e68d1f22e75343999966d2ff409acc73","47a560c6e53b43a4bd24357339046c8f","909cbc94cb6d4a9e80ad41f5fcc369c9","bd62bd3d631e4429b2a0347b0cc5654c","9b395d60d64643b6804f2fa417df90e8","5f6369fe1652456da152c5189b9cf200","d3ba225fcb7e4cdca542551317b31569","89d85d9cfc154254ac2472f278be1a69","08ceb05830554399be0cf251f6256970","998157211dfb4d21a8dd0d8efa3fc245","deacfebe3871438abdc90b60dad9a32f","95025404591f4567b3db485c571095f9","1b584563488348579f132aa98d2dfe09","d78989601c354f1b8131a394efc440c2","3c772a9f001448d99d763219593686c3","d354121e9f904394b8b6af25fab434d3","395e60e1600f42739c905da1720cc187","eee99d979cbb42428313f7c55270298e","fdc41bad820143588d82734f6147b13f","0e63f87958bf4e80a5f582abe41f2364","742265a4480140ed9b3578b3098c8fc0","8a3ec41fc0be4616997556aab65744fd","db34ae736e5045f49ec389ccf26c7f5a","fcd9b941f5014d7587b2f00798721374","824f8c48ed134646bef3ea76baad855c","b1d7d85fb45746f4b57988d76ae26426","1546ea46587d480eb29e6c1844ad2f80","91f32f1839bd409783c9bca5e9ad0bcc","60b1328a8ab54a47be35e74cfabf9288","c38137c015c2457a807d403ef4cd3cc6","87cf68af10c0499199a65096b0beb78c","e8511def2cbb47d1b0269b5816e6c03b","fa50ca8ae1374b748d69e4b6541a5a72","4b8f823425a94cba9f0104848ee1e896","f0e7e6be84924623900da021faa743b6","d7b289713a004ca7804204792a2b6ed4","bbac4079f11e4e52a6108b6fb3e4a2e5","680be5d57ddf4bcf92f3a9b4ed413684","ac3f56ad9f354790a3a31101d196a78e","23428cd9c0584adea897d1bef50839af","49af41ecf8084d83bfdc37d25303e576","c4c646038ea64aa29c534bef36f5d06e","38909ccb06854dd886e7be9a8f7657c1","20ac96f8e5674c1bb3204efafb951d7d","16321c4d05a14e049a393dca08309ad6","3c75650d4a5647079a638dd7566dd8ec","490aec0bf7d642aa8d6fa219c23fa4f6","26e8a4b2d9d941c190ca828b4f0bc6ac","470b70df4d94475d91156919b93e5712","21b2d11c391b4442b0bda4f0a30bd625","3fcf984878dc4ec6b9a184e5103da8f3","deca135788e84a34a209df86101f6343","87473024bbea4171a03d41967520aa99","cb2030414a9048a88cb39f09ddc98189","15737b9be0544c10b7903dabda8653b1","8310ad421f0042a7868a45f87b845b44","627cb05685c64ddfa3bba3dab7471d11","ec5b4f91155d47d1a4df6e720b7de3cb","eebe0e888799428394efff69c629a1db","45c1af5a461747f5b2389977fdf851ce","afcbcc7413214d31ba00d881d5e04cc8","913a12d90a894664ab2219551561113e","801783d19f094cf2a10669ce2dc61f2d","b1091be9f09441c0b359986f144bec62","d8ad7fed7e7f4d858122658213c2ff4f"]},"id":"QmUBVEnvCDJv","executionInfo":{"status":"ok","timestamp":1727858188216,"user_tz":-540,"elapsed":113484,"user":{"displayName":"최현진","userId":"12812404047517020320"}},"outputId":"2e01341f-3fc8-4a9a-d242-c49603bb6762"},"outputs":[{"output_type":"stream","name":"stdout","text":["🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.\n","==((====))== Unsloth 2024.9.post4: Fast Gemma2 patching. Transformers = 4.44.2.\n"," \\\\ /| GPU: Tesla T4. Max memory: 14.748 GB. Platform = Linux.\n","O^O/ \\_/ \\ Pytorch: 2.4.1+cu121. CUDA = 7.5. CUDA Toolkit = 12.1.\n","\\ / Bfloat16 = FALSE. FA [Xformers = 0.0.28.post1. FA2 = False]\n"," \"-____-\" Free Apache license: http://github.com/unslothai/unsloth\n","Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored!\n"]},{"output_type":"display_data","data":{"text/plain":["model.safetensors: 0%| | 0.00/6.13G [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"0a6c54edc08e44ee9cf092414a4e42ec"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["generation_config.json: 0%| | 0.00/190 [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"deacfebe3871438abdc90b60dad9a32f"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["tokenizer_config.json: 0%| | 0.00/46.4k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"8a3ec41fc0be4616997556aab65744fd"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["tokenizer.model: 0%| | 0.00/4.24M [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"fa50ca8ae1374b748d69e4b6541a5a72"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["special_tokens_map.json: 0%| | 0.00/636 [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"20ac96f8e5674c1bb3204efafb951d7d"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["tokenizer.json: 0%| | 0.00/17.5M [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"15737b9be0544c10b7903dabda8653b1"}},"metadata":{}}],"source":["from unsloth import FastLanguageModel\n","import torch\n","max_seq_length = 2048 # Choose any! We auto support RoPE Scaling internally!\n","dtype = None # None for auto detection. Float16 for Tesla T4, V100, Bfloat16 for Ampere+\n","load_in_4bit = True # Use 4bit quantization to reduce memory usage. Can be False.\n","\n","# 4bit pre quantized models we support for 4x faster downloading + no OOMs.\n","fourbit_models = [\n"," \"unsloth/Meta-Llama-3.1-8B-bnb-4bit\", # Llama-3.1 15 trillion tokens model 2x faster!\n"," \"unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit\",\n"," \"unsloth/Meta-Llama-3.1-70B-bnb-4bit\",\n"," \"unsloth/Meta-Llama-3.1-405B-bnb-4bit\", # We also uploaded 4bit for 405b!\n"," \"unsloth/Mistral-Nemo-Base-2407-bnb-4bit\", # New Mistral 12b 2x faster!\n"," \"unsloth/Mistral-Nemo-Instruct-2407-bnb-4bit\",\n"," \"unsloth/mistral-7b-v0.3-bnb-4bit\", # Mistral v3 2x faster!\n"," \"unsloth/mistral-7b-instruct-v0.3-bnb-4bit\",\n"," \"unsloth/Phi-3.5-mini-instruct\", # Phi-3.5 2x faster!\n"," \"unsloth/Phi-3-medium-4k-instruct\",\n"," \"unsloth/gemma-2-9b-bnb-4bit\",\n"," \"unsloth/gemma-2-27b-bnb-4bit\", # Gemma 2x faster!\n","] # More models at https://huggingface.co/unsloth\n","\n","model, tokenizer = FastLanguageModel.from_pretrained(\n"," model_name = \"unsloth/gemma-2-9b\",\n"," max_seq_length = max_seq_length,\n"," dtype = dtype,\n"," load_in_4bit = load_in_4bit,\n"," # token = \"hf_...\", # use one if using gated models like meta-llama/Llama-2-7b-hf\n",")"]},{"cell_type":"markdown","source":["We now add LoRA adapters so we only need to update 1 to 10% of all parameters!"],"metadata":{"id":"SXd9bTZd1aaL"}},{"cell_type":"code","execution_count":3,"metadata":{"id":"6bZsfBuZDeCL","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1727858393516,"user_tz":-540,"elapsed":9745,"user":{"displayName":"최현진","userId":"12812404047517020320"}},"outputId":"39f4b630-aee4-45fe-b29f-a0008020fa24"},"outputs":[{"output_type":"stream","name":"stderr","text":["Unsloth 2024.9.post4 patched 42 layers with 42 QKV layers, 42 O layers and 42 MLP layers.\n"]}],"source":["model = FastLanguageModel.get_peft_model(\n"," model,\n"," r = 32, # Choose any number > 0 ! Suggested 8, 16, 32, 64, 128\n"," target_modules = [\"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\",\n"," \"gate_proj\", \"up_proj\", \"down_proj\",],\n"," lora_alpha = 64, # rank의 2배\n"," lora_dropout = 0, # Supports any, but = 0 is optimized\n"," bias = \"none\", # Supports any, but = \"none\" is optimized\n"," # [NEW] \"unsloth\" uses 30% less VRAM, fits 2x larger batch sizes!\n"," use_gradient_checkpointing = \"unsloth\", # True or \"unsloth\" for very long context\n"," random_state = 3407,\n"," use_rslora = False, # We support rank stabilized LoRA\n"," loftq_config = None, # And LoftQ\n",")"]},{"cell_type":"markdown","source":["Use QLoRa"],"metadata":{"id":"0EWofs1z6myP"}},{"cell_type":"code","source":["# model = FastLanguageModel.get_peft_model(\n","# model,\n","# r = 16, # Choose any number > 0 ! Suggested 8, 16, 32, 64, 128\n","# target_modules = [\"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\",\n","# \"gate_proj\", \"up_proj\", \"down_proj\",],\n","# lora_alpha = 16,\n","# lora_dropout = 0, # Supports any, but = 0 is optimized\n","# bias = \"none\", # Supports any, but = \"none\" is optimized\n","# # [NEW] \"unsloth\" uses 30% less VRAM, fits 2x larger batch sizes!\n","# use_gradient_checkpointing = \"unsloth\", # True or \"unsloth\" for very long context\n","# random_state = 3407,\n","# use_rslora = False, # We support rank stabilized LoRA\n","# loftq_config = {\n","# \"bnb_4bit\": True, # Use 4bit quantization\n","# \"bnb_4bit_compute_dtype\": torch.float16, # Set computation dtype\n","# \"bnb_4bit_quant_type\": \"nf4\" # Choose quantization type (nf4 recommended)\n","# }\n","# )"],"metadata":{"id":"jJ0SNz2Q6mlZ"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["# Load Data"],"metadata":{"id":"ZskSVjBKU9-L"}},{"cell_type":"code","source":["from google.colab import drive\n","drive.mount('/content/drive')"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"9ZUKAZ_oVImS","executionInfo":{"status":"ok","timestamp":1727858444817,"user_tz":-540,"elapsed":20771,"user":{"displayName":"최현진","userId":"12812404047517020320"}},"outputId":"63fd8d1d-1696-424c-d6df-41643a0783cd"},"execution_count":4,"outputs":[{"output_type":"stream","name":"stdout","text":["Mounted at /content/drive\n"]}]},{"cell_type":"code","source":["cd /content/drive/MyDrive/Google MLB 2024/Gemma Sprint/data"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"FYdtxQp9Vw6K","executionInfo":{"status":"ok","timestamp":1727858483832,"user_tz":-540,"elapsed":381,"user":{"displayName":"최현진","userId":"12812404047517020320"}},"outputId":"132b2066-89e1-41de-a689-04d7f3d7f3ec"},"execution_count":6,"outputs":[{"output_type":"stream","name":"stdout","text":["/content/drive/MyDrive/Google MLB 2024/Gemma Sprint/data\n"]}]},{"cell_type":"code","source":["from datasets import load_dataset\n","\n","# JSON 파일 경로\n","json_file = \"우리말샘_QA_dataset_전체.json\""],"metadata":{"id":"GaE6btRMVA50","executionInfo":{"status":"ok","timestamp":1727858493412,"user_tz":-540,"elapsed":397,"user":{"displayName":"최현진","userId":"12812404047517020320"}}},"execution_count":8,"outputs":[]},{"cell_type":"markdown","source":["<a name=\"Data\"></a>\n","### Data Prep\n","We now use the Alpaca dataset from [yahma](https://huggingface.co/datasets/yahma/alpaca-cleaned), which is a filtered version of 52K of the original [Alpaca dataset](https://crfm.stanford.edu/2023/03/13/alpaca.html). You can replace this code section with your own data prep.\n","\n","**[NOTE]** To train only on completions (ignoring the user's input) read TRL's docs [here](https://huggingface.co/docs/trl/sft_trainer#train-on-completions-only).\n","\n","**[NOTE]** Remember to add the **EOS_TOKEN** to the tokenized output!! Otherwise you'll get infinite generations!\n","\n","If you want to use the `llama-3` template for ShareGPT datasets, try our conversational [notebook](https://colab.research.google.com/drive/1XamvWYinY6FOSX9GLvnqSjjsNflxdhNc?usp=sharing).\n","\n","For text completions like novel writing, try this [notebook](https://colab.research.google.com/drive/1ef-tab5bhkvWmBOObepl1WgJvfvSzn5Q?usp=sharing)."],"metadata":{"id":"vITh0KVJ10qX"}},{"cell_type":"code","execution_count":9,"metadata":{"id":"LjY75GoYUCB8","colab":{"base_uri":"https://localhost:8080/","height":81,"referenced_widgets":["ef997c8aba40422780940ca0747c4614","5f25d2216afe408b8d8b4ccb75da5840","e2fb0d0600c046008617abc45da4962c","0fe489f0907946ff8b755226f02cab45","3b0e9f300feb4867bd3850aa96d0e63a","b1ffa237c46d447bbaf88ca050d8b6de","e9744595dd2640209c6dd70c9c09eb84","411bc890289345fbb8c1eec04cb8ee55","0015b82880034b85981aeb78daa91ab2","492f49a7f115445daebec8851e199de8","96258cd3ae914460bba000d71d416e54","1faf886b995e472e8666ed01d78e2ee6","94ec88e90b95488d876d7d3e6e876d4f","c53b9f12fc0b410aa94fc94fb7f91e7e","96e7acc07a894f0d8c12df3f47d62a54","9565507bc58a40e58b536c1333d0da06","483f3084119141ce8a0429cd31e903f6","eac58a2cf5564476a0d06e8da5419a8f","79bf783adbb341d8b47fd5d3876f799b","64599a388e374269a5bbb0df3228824f","ea2ce876b2d142989e0fc9615baa6b2f","006650d347494172b83d27a97f0d0fae"]},"executionInfo":{"status":"ok","timestamp":1727858514436,"user_tz":-540,"elapsed":19039,"user":{"displayName":"최현진","userId":"12812404047517020320"}},"outputId":"4b14276c-e70a-4e6c-b2bb-bf96b8660846","collapsed":true},"outputs":[{"output_type":"display_data","data":{"text/plain":["Generating train split: 0 examples [00:00, ? examples/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"ef997c8aba40422780940ca0747c4614"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Map: 0%| | 0/1181401 [00:00<?, ? examples/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"1faf886b995e472e8666ed01d78e2ee6"}},"metadata":{}}],"source":["alpaca_prompt = \"\"\"Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n","\n","### Instruction:\n","{}\n","\n","### Input:\n","{}\n","\n","### Response:\n","{}\"\"\"\n","\n","EOS_TOKEN = tokenizer.eos_token # Must add EOS_TOKEN\n","def formatting_prompts_func(examples):\n"," instructions = examples[\"instruction\"]\n"," inputs = examples[\"input\"]\n"," outputs = examples[\"output\"]\n"," texts = []\n"," for instruction, input, output in zip(instructions, inputs, outputs):\n"," # Must add EOS_TOKEN, otherwise your generation will go on forever!\n"," text = alpaca_prompt.format(instruction, input, output) + EOS_TOKEN\n"," texts.append(text)\n"," return { \"text\" : texts, }\n","pass"]},{"cell_type":"code","source":["# Load data\n","from datasets import load_dataset\n","\n","dataset = load_dataset(\"json\", data_files=json_file)\n","\n","# Shuffle data\n","shuffled_dataset = dataset[\"train\"].shuffle(seed=3407)\n","\n","# Split train/test data (90:10)\n","train_test_split = shuffled_dataset.train_test_split(test_size=0.1)\n","train_dataset = train_test_split[\"train\"]\n","test_dataset = train_test_split[\"test\"]\n","dataset = train_dataset.map(formatting_prompts_func, batched = True,)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":49,"referenced_widgets":["674ecbbecc64450198992a7735460e07","b4bb93af94a14918910064559e6f3c53","d837748b23514b769c1b78f6d1a62ac9","5fde99f56d9e43869fcf073c61df7bd9","35fb0e04580f4a3cb946f9611d5cd210","35834a01131b4d56af416433e2c28a08","bb94d7eaffe14e17bd2e1a5bd59b47a2","92296ea7e5a6487a9571d98364417b98","b2a5d3c1a9ea4cd6a34c6023476f290c","eccfa1faa3b84d3cb0138ef2f30bfa9f","53a4c3365930429095c818ecb14b1c80"]},"id":"TWBIGzAhaUUY","executionInfo":{"status":"ok","timestamp":1727858884024,"user_tz":-540,"elapsed":26339,"user":{"displayName":"최현진","userId":"12812404047517020320"}},"outputId":"7f142857-c093-4cf0-9524-e17795fa462a"},"execution_count":11,"outputs":[{"output_type":"display_data","data":{"text/plain":["Map: 0%| | 0/1063260 [00:00<?, ? examples/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"674ecbbecc64450198992a7735460e07"}},"metadata":{}}]},{"cell_type":"markdown","source":["<a name=\"Train\"></a>\n","### Train the model\n","Now let's use Huggingface TRL's `SFTTrainer`! More docs here: [TRL SFT docs](https://huggingface.co/docs/trl/sft_trainer). We do 60 steps to speed things up, but you can set `num_train_epochs=1` for a full run, and turn off `max_steps=None`. We also support TRL's `DPOTrainer`!"],"metadata":{"id":"idAEIeSQ3xdS"}},{"cell_type":"code","execution_count":12,"metadata":{"id":"95_Nn-89DhsL","colab":{"base_uri":"https://localhost:8080/","height":67,"referenced_widgets":["4758ab6d84ed4eb99ea0474c6eb97469","f23e7ba8b1ff40fdab5af080932266ad","b498992a458b47c78dedb921451230e6","6753316ed94b4b29b2d7f1856b0afb32","dcf7f5eed1ba4b459e5f7edfd38fc8fa","c25a74deead945a18c871359a6b03b31","92f54f687ec34a37955b17c7e4e3b8e2","31ee693f414b4b208bca95c0a5902653","aed8d91169a6436bb885e1f7e3e2c856","59bdfecc46be4c40bc135882dfe401c1","ddda26ee4fbb4e3d86e3843c5de85884"]},"executionInfo":{"status":"ok","timestamp":1727859087656,"user_tz":-540,"elapsed":187877,"user":{"displayName":"최현진","userId":"12812404047517020320"}},"outputId":"3cd623f4-7038-422c-a275-b64e960af426"},"outputs":[{"output_type":"display_data","data":{"text/plain":["Map (num_proc=2): 0%| | 0/1063260 [00:00<?, ? examples/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"4758ab6d84ed4eb99ea0474c6eb97469"}},"metadata":{}},{"output_type":"stream","name":"stderr","text":["max_steps is given, it will override any value given in num_train_epochs\n"]}],"source":["from trl import SFTTrainer\n","from transformers import TrainingArguments\n","from unsloth import is_bfloat16_supported\n","\n","trainer = SFTTrainer(\n"," model = model,\n"," tokenizer = tokenizer,\n"," train_dataset = dataset,\n"," dataset_text_field = \"text\",\n"," max_seq_length = max_seq_length,\n"," dataset_num_proc = 2,\n"," packing = False, # Can make training 5x faster for short sequences.\n"," args = TrainingArguments(\n"," per_device_train_batch_size = 2,\n"," gradient_accumulation_steps = 4,\n"," warmup_steps = 5,\n"," max_steps = 60,\n"," learning_rate = 2e-4,\n"," fp16 = not is_bfloat16_supported(),\n"," bf16 = is_bfloat16_supported(),\n"," logging_steps = 1,\n"," optim = \"adamw_8bit\",\n"," weight_decay = 0.01,\n"," lr_scheduler_type = \"linear\",\n"," seed = 3407,\n"," output_dir = \"outputs\",\n"," ),\n",")"]},{"cell_type":"code","execution_count":13,"metadata":{"id":"2ejIt2xSNKKp","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1727859087657,"user_tz":-540,"elapsed":11,"user":{"displayName":"최현진","userId":"12812404047517020320"}},"outputId":"88ba7eef-ac5a-4050-b1db-9443d4786b34"},"outputs":[{"output_type":"stream","name":"stdout","text":["GPU = Tesla T4. Max memory = 14.748 GB.\n","7.029 GB of memory reserved.\n"]}],"source":["#@title Show current memory stats\n","gpu_stats = torch.cuda.get_device_properties(0)\n","start_gpu_memory = round(torch.cuda.max_memory_reserved() / 1024 / 1024 / 1024, 3)\n","max_memory = round(gpu_stats.total_memory / 1024 / 1024 / 1024, 3)\n","print(f\"GPU = {gpu_stats.name}. Max memory = {max_memory} GB.\")\n","print(f\"{start_gpu_memory} GB of memory reserved.\")"]},{"cell_type":"code","execution_count":14,"metadata":{"id":"yqxqAZ7KJ4oL","colab":{"base_uri":"https://localhost:8080/","height":1000},"outputId":"5800975d-db6a-420b-e7a2-71788a48cfcb","executionInfo":{"status":"ok","timestamp":1727859483680,"user_tz":-540,"elapsed":396030,"user":{"displayName":"최현진","userId":"12812404047517020320"}}},"outputs":[{"output_type":"stream","name":"stderr","text":["==((====))== Unsloth - 2x faster free finetuning | Num GPUs = 1\n"," \\\\ /| Num examples = 1,063,260 | Num Epochs = 1\n","O^O/ \\_/ \\ Batch size per device = 2 | Gradient Accumulation steps = 4\n","\\ / Total batch size = 8 | Total steps = 60\n"," \"-____-\" Number of trainable parameters = 108,036,096\n"]},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.HTML object>"],"text/html":["\n"," <div>\n"," \n"," <progress value='60' max='60' style='width:300px; height:20px; vertical-align: middle;'></progress>\n"," [60/60 05:26, Epoch 0/1]\n"," </div>\n"," <table border=\"1\" class=\"dataframe\">\n"," <thead>\n"," <tr style=\"text-align: left;\">\n"," <th>Step</th>\n"," <th>Training Loss</th>\n"," </tr>\n"," </thead>\n"," <tbody>\n"," <tr>\n"," <td>1</td>\n"," <td>3.503900</td>\n"," </tr>\n"," <tr>\n"," <td>2</td>\n"," <td>3.631400</td>\n"," </tr>\n"," <tr>\n"," <td>3</td>\n"," <td>2.507900</td>\n"," </tr>\n"," <tr>\n"," <td>4</td>\n"," <td>2.205400</td>\n"," </tr>\n"," <tr>\n"," <td>5</td>\n"," <td>1.860200</td>\n"," </tr>\n"," <tr>\n"," <td>6</td>\n"," <td>1.689300</td>\n"," </tr>\n"," <tr>\n"," <td>7</td>\n"," <td>1.448900</td>\n"," </tr>\n"," <tr>\n"," <td>8</td>\n"," <td>1.335800</td>\n"," </tr>\n"," <tr>\n"," <td>9</td>\n"," <td>1.153200</td>\n"," </tr>\n"," <tr>\n"," <td>10</td>\n"," <td>1.111900</td>\n"," </tr>\n"," <tr>\n"," <td>11</td>\n"," <td>1.322200</td>\n"," </tr>\n"," <tr>\n"," <td>12</td>\n"," <td>1.216700</td>\n"," </tr>\n"," <tr>\n"," <td>13</td>\n"," <td>1.264900</td>\n"," </tr>\n"," <tr>\n"," <td>14</td>\n"," <td>1.095600</td>\n"," </tr>\n"," <tr>\n"," <td>15</td>\n"," <td>1.223100</td>\n"," </tr>\n"," <tr>\n"," <td>16</td>\n"," <td>1.429300</td>\n"," </tr>\n"," <tr>\n"," <td>17</td>\n"," <td>0.979800</td>\n"," </tr>\n"," <tr>\n"," <td>18</td>\n"," <td>1.066300</td>\n"," </tr>\n"," <tr>\n"," <td>19</td>\n"," <td>0.908500</td>\n"," </tr>\n"," <tr>\n"," <td>20</td>\n"," <td>1.270700</td>\n"," </tr>\n"," <tr>\n"," <td>21</td>\n"," <td>1.190300</td>\n"," </tr>\n"," <tr>\n"," <td>22</td>\n"," <td>0.921000</td>\n"," </tr>\n"," <tr>\n"," <td>23</td>\n"," <td>1.206700</td>\n"," </tr>\n"," <tr>\n"," <td>24</td>\n"," <td>1.217600</td>\n"," </tr>\n"," <tr>\n"," <td>25</td>\n"," <td>0.934000</td>\n"," </tr>\n"," <tr>\n"," <td>26</td>\n"," <td>1.111900</td>\n"," </tr>\n"," <tr>\n"," <td>27</td>\n"," <td>1.058500</td>\n"," </tr>\n"," <tr>\n"," <td>28</td>\n"," <td>1.057700</td>\n"," </tr>\n"," <tr>\n"," <td>29</td>\n"," <td>1.373700</td>\n"," </tr>\n"," <tr>\n"," <td>30</td>\n"," <td>1.173400</td>\n"," </tr>\n"," <tr>\n"," <td>31</td>\n"," <td>1.126800</td>\n"," </tr>\n"," <tr>\n"," <td>32</td>\n"," <td>0.824900</td>\n"," </tr>\n"," <tr>\n"," <td>33</td>\n"," <td>1.174500</td>\n"," </tr>\n"," <tr>\n"," <td>34</td>\n"," <td>0.995500</td>\n"," </tr>\n"," <tr>\n"," <td>35</td>\n"," <td>1.107000</td>\n"," </tr>\n"," <tr>\n"," <td>36</td>\n"," <td>1.103700</td>\n"," </tr>\n"," <tr>\n"," <td>37</td>\n"," <td>1.178300</td>\n"," </tr>\n"," <tr>\n"," <td>38</td>\n"," <td>1.075800</td>\n"," </tr>\n"," <tr>\n"," <td>39</td>\n"," <td>1.021900</td>\n"," </tr>\n"," <tr>\n"," <td>40</td>\n"," <td>1.062400</td>\n"," </tr>\n"," <tr>\n"," <td>41</td>\n"," <td>1.183900</td>\n"," </tr>\n"," <tr>\n"," <td>42</td>\n"," <td>0.990900</td>\n"," </tr>\n"," <tr>\n"," <td>43</td>\n"," <td>1.172100</td>\n"," </tr>\n"," <tr>\n"," <td>44</td>\n"," <td>1.002000</td>\n"," </tr>\n"," <tr>\n"," <td>45</td>\n"," <td>0.996900</td>\n"," </tr>\n"," <tr>\n"," <td>46</td>\n"," <td>1.125000</td>\n"," </tr>\n"," <tr>\n"," <td>47</td>\n"," <td>1.103000</td>\n"," </tr>\n"," <tr>\n"," <td>48</td>\n"," <td>0.770400</td>\n"," </tr>\n"," <tr>\n"," <td>49</td>\n"," <td>1.296300</td>\n"," </tr>\n"," <tr>\n"," <td>50</td>\n"," <td>1.099700</td>\n"," </tr>\n"," <tr>\n"," <td>51</td>\n"," <td>0.834700</td>\n"," </tr>\n"," <tr>\n"," <td>52</td>\n"," <td>0.980000</td>\n"," </tr>\n"," <tr>\n"," <td>53</td>\n"," <td>0.996300</td>\n"," </tr>\n"," <tr>\n"," <td>54</td>\n"," <td>1.041800</td>\n"," </tr>\n"," <tr>\n"," <td>55</td>\n"," <td>0.963500</td>\n"," </tr>\n"," <tr>\n"," <td>56</td>\n"," <td>0.983300</td>\n"," </tr>\n"," <tr>\n"," <td>57</td>\n"," <td>1.141800</td>\n"," </tr>\n"," <tr>\n"," <td>58</td>\n"," <td>0.878200</td>\n"," </tr>\n"," <tr>\n"," <td>59</td>\n"," <td>0.947400</td>\n"," </tr>\n"," <tr>\n"," <td>60</td>\n"," <td>1.079900</td>\n"," </tr>\n"," </tbody>\n","</table><p>"]},"metadata":{}}],"source":["trainer_stats = trainer.train()"]},{"cell_type":"code","execution_count":15,"metadata":{"id":"pCqnaKmlO1U9","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1727859979984,"user_tz":-540,"elapsed":380,"user":{"displayName":"최현진","userId":"12812404047517020320"}},"outputId":"41137a73-c4b7-4166-a4ea-9405793a4d8f"},"outputs":[{"output_type":"stream","name":"stdout","text":["391.8304 seconds used for training.\n","6.53 minutes used for training.\n","Peak reserved memory = 9.307 GB.\n","Peak reserved memory for training = 2.278 GB.\n","Peak reserved memory % of max memory = 63.107 %.\n","Peak reserved memory for training % of max memory = 15.446 %.\n"]}],"source":["#@title Show final memory and time stats\n","used_memory = round(torch.cuda.max_memory_reserved() / 1024 / 1024 / 1024, 3)\n","used_memory_for_lora = round(used_memory - start_gpu_memory, 3)\n","used_percentage = round(used_memory /max_memory*100, 3)\n","lora_percentage = round(used_memory_for_lora/max_memory*100, 3)\n","print(f\"{trainer_stats.metrics['train_runtime']} seconds used for training.\")\n","print(f\"{round(trainer_stats.metrics['train_runtime']/60, 2)} minutes used for training.\")\n","print(f\"Peak reserved memory = {used_memory} GB.\")\n","print(f\"Peak reserved memory for training = {used_memory_for_lora} GB.\")\n","print(f\"Peak reserved memory % of max memory = {used_percentage} %.\")\n","print(f\"Peak reserved memory for training % of max memory = {lora_percentage} %.\")"]},{"cell_type":"markdown","source":["<a name=\"Inference\"></a>\n","### Inference\n","Let's run the model! You can change the instruction and input - leave the output blank!"],"metadata":{"id":"ekOmTR1hSNcr"}},{"cell_type":"code","source":["# alpaca_prompt = Copied from above\n","FastLanguageModel.for_inference(model) # Enable native 2x faster inference\n","inputs = tokenizer(\n","[\n"," alpaca_prompt.format(\n"," \"Continue the fibonnaci sequence.\", # instruction\n"," \"1, 1, 2, 3, 5, 8\", # input\n"," \"\", # output - leave this blank for generation!\n"," )\n","], return_tensors = \"pt\").to(\"cuda\")\n","\n","outputs = model.generate(**inputs, max_new_tokens = 64, use_cache = True)\n","tokenizer.batch_decode(outputs)"],"metadata":{"id":"kR3gIAX-SM2q","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1727860012915,"user_tz":-540,"elapsed":12755,"user":{"displayName":"최현진","userId":"12812404047517020320"}},"outputId":"0658031a-6e3d-4b69-d09a-6b1ccc7d68c4"},"execution_count":16,"outputs":[{"output_type":"execute_result","data":{"text/plain":["['<bos>Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\\n\\n### Instruction:\\nContinue the fibonnaci sequence.\\n\\n### Input:\\n1, 1, 2, 3, 5, 8\\n\\n### Response:\\n13<eos>']"]},"metadata":{},"execution_count":16}]},{"cell_type":"markdown","source":["추가"],"metadata":{"id":"HXzhehC_bZWP"}},{"cell_type":"markdown","source":["### Test셋 추론"],"metadata":{"id":"MXYYwIb1fh4a"}},{"cell_type":"code","source":["from transformers import StoppingCriteria, StoppingCriteriaList\n","\n","\n","class StopOnToken(StoppingCriteria):\n"," def __init__(self, stop_token_id):\n"," self.stop_token_id = stop_token_id # 정지 토큰 ID를 초기화합니다.\n","\n"," def __call__(self, input_ids, scores, **kwargs):\n"," return (\n"," self.stop_token_id in input_ids[0]\n"," ) # 입력된 ID 중 정지 토큰 ID가 있으면 정지합니다.\n","\n","\n","# end_token을 설정\n","stop_token = \"<|end_of_text|>\" # end_token으로 사용할 토큰을 설정합니다.\n","stop_token_id = tokenizer.encode(stop_token, add_special_tokens=False)[\n"," 0\n","] # end_token의 ID를 인코딩합니다.\n","\n","# Stopping criteria 설정\n","stopping_criteria = StoppingCriteriaList(\n"," [StopOnToken(stop_token_id)]\n",") # 정지 조건을 설정합니다."],"metadata":{"id":"nDgtUzgIluUy","executionInfo":{"status":"ok","timestamp":1727862185376,"user_tz":-540,"elapsed":355,"user":{"displayName":"최현진","userId":"12812404047517020320"}}},"execution_count":23,"outputs":[]},{"cell_type":"code","source":["test_dataset[0]['output']"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":35},"id":"8-mk9GMMqCZu","executionInfo":{"status":"ok","timestamp":1727863169696,"user_tz":-540,"elapsed":368,"user":{"displayName":"최현진","userId":"12812404047517020320"}},"outputId":"b99658fc-3652-4b28-a8c8-770c5578b12e"},"execution_count":37,"outputs":[{"output_type":"execute_result","data":{"text/plain":["'우계-집'"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"string"}},"metadata":{},"execution_count":37}]},{"cell_type":"code","source":["from transformers import TextStreamer\n","\n","# FastLanguageModel을 이용하여 추론 속도를 2배 빠르게 설정합니다.\n","FastLanguageModel.for_inference(model)\n","inputs = tokenizer(\n"," [\n"," alpaca_prompt.format(\n"," test_dataset[0][\"instruction\"], # 지시사항\n"," \"\",\n"," \"\", # 출력 - 생성을 위해 이 부분을 비워둡니다!\n"," )\n"," ],\n"," return_tensors=\"pt\",\n",").to(\"cuda\")\n","\n","\n","text_streamer = TextStreamer(tokenizer)\n","_ = model.generate(\n"," **inputs,\n"," streamer=text_streamer,\n"," max_new_tokens=4096, # 최대 생성 토큰 수를 설정합니다.\n"," stopping_criteria=stopping_criteria # 생성을 멈출 기준을 설정합니다.\n",")"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"aB1UdKjxlz3m","executionInfo":{"status":"ok","timestamp":1727863153114,"user_tz":-540,"elapsed":2206,"user":{"displayName":"최현진","userId":"12812404047517020320"}},"outputId":"3010c5f5-720a-40c9-ecc8-066200dab3db"},"execution_count":36,"outputs":[{"output_type":"stream","name":"stdout","text":["<bos>Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n","\n","### Instruction:\n","조선 선조 때에 지은 성혼의 시문집. 시(詩), 장소(章疏), 간독(簡牘), 잡저(雜著) 따위가 수록되어 있다. 숙종 8년(1682)에 간행하였다. 6권 12책.\n","\n","### Input:\n","\n","\n","### Response:\n","성혼-집<eos>\n"]}]},{"cell_type":"code","source":["from transformers import TextStreamer\n","\n","# FastLanguageModel을 이용하여 추론 속도를 2배 빠르게 설정합니다.\n","FastLanguageModel.for_inference(model)\n","inputs = tokenizer(\n"," [\n"," alpaca_prompt.format(\n"," test_dataset[10][\"input\"], # 지시사항\n"," \"\",\n"," \"\", # 출력 - 생성을 위해 이 부분을 비워둡니다!\n"," )\n"," ],\n"," return_tensors=\"pt\",\n",").to(\"cuda\")\n","\n","\n","text_streamer = TextStreamer(tokenizer)\n","_ = model.generate(\n"," **inputs,\n"," streamer=text_streamer,\n"," max_new_tokens=4096, # 최대 생성 토큰 수를 설정합니다.\n"," stopping_criteria=stopping_criteria # 생성을 멈출 기준을 설정합니다.\n",")"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"dJgqlrBZp8rX","executionInfo":{"status":"ok","timestamp":1727862930044,"user_tz":-540,"elapsed":1215,"user":{"displayName":"최현진","userId":"12812404047517020320"}},"outputId":"cc994e08-1c41-48ab-9e00-559014ecc216"},"execution_count":28,"outputs":[{"output_type":"stream","name":"stdout","text":["<bos>Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n","\n","### Instruction:\n","\n","\n","### Input:\n","\n","\n","### Response:\n","가난하다<eos>\n"]}]},{"cell_type":"code","source":["# Test 데이터셋을 사용한 inference\n","FastLanguageModel.for_inference(model) # 2배 빠른 추론 모드 활성화\n","\n","# 예측 결과를 저장할 리스트\n","predictions = []\n","\n","# 테스트 셋의 각 입력에 대해 추론 수행\n","for test_data in test_dataset[\"input\"]: # 테스트 데이터셋에서 'input' 열 사용\n"," inputs = tokenizer(\n"," [\n"," alpaca_prompt.format(\n"," test_data, # 테스트셋의 입력 문장 사용\n"," \"\", # input 부분은 비워둠\n"," \"\", # output도 비워둠\n"," )\n"," ],\n"," return_tensors=\"pt\"\n"," ).to(\"cuda\") # GPU 사용\n","\n"," # 모델 추론\n"," outputs = model.generate(**inputs, max_new_tokens=64, use_cache=True)\n","\n"," # 추론 결과 디코딩 후 저장\n"," predicted_text = tokenizer.batch_decode(outputs, skip_special_tokens=True)[0]\n"," predictions.append(predicted_text)\n","\n","# 추론 결과 출력 또는 저장\n","for prediction in predictions:\n"," print(prediction)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":441},"id":"nz3MCtmwfhgo","executionInfo":{"status":"error","timestamp":1727862175019,"user_tz":-540,"elapsed":1967271,"user":{"displayName":"최현진","userId":"12812404047517020320"}},"outputId":"8f5ec43d-3dc4-45b7-aa53-2ba97899745c"},"execution_count":22,"outputs":[{"output_type":"error","ename":"KeyboardInterrupt","evalue":"","traceback":["\u001b[0;31m---------------------------------------------------------------------------\u001b[0m","\u001b[0;31mKeyboardInterrupt\u001b[0m Traceback (most recent call last)","\u001b[0;32m<ipython-input-22-b872242ebc37>\u001b[0m in \u001b[0;36m<cell line: 8>\u001b[0;34m()\u001b[0m\n\u001b[1;32m 19\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 20\u001b[0m \u001b[0;31m# 모델 추론\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 21\u001b[0;31m \u001b[0moutputs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgenerate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m**\u001b[0m\u001b[0minputs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmax_new_tokens\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m64\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0muse_cache\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 22\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 23\u001b[0m \u001b[0;31m# 추론 결과 디코딩 후 저장\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py\u001b[0m in \u001b[0;36mdecorate_context\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m 114\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mdecorate_context\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 115\u001b[0m \u001b[0;32mwith\u001b[0m \u001b[0mctx_factory\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 116\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mfunc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 117\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 118\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mdecorate_context\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/unsloth/models/llama.py\u001b[0m in \u001b[0;36m_fast_generate\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m 1405\u001b[0m \u001b[0;31m# Autocasted\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1406\u001b[0m \u001b[0;32mwith\u001b[0m \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mautocast\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdevice_type\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdevice_type\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdtype\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdtype\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1407\u001b[0;31m \u001b[0moutput\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mgenerate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1408\u001b[0m \u001b[0;32mpass\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1409\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/peft/peft_model.py\u001b[0m in \u001b[0;36mgenerate\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m 1702\u001b[0m \u001b[0;32mwith\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_enable_peft_forward_hooks\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1703\u001b[0m \u001b[0mkwargs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0mk\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mv\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mk\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mv\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mkwargs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mitems\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mk\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mspecial_peft_forward_args\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1704\u001b[0;31m \u001b[0moutputs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbase_model\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgenerate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1705\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1706\u001b[0m \u001b[0moutputs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbase_model\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgenerate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py\u001b[0m in \u001b[0;36mdecorate_context\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m 114\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mdecorate_context\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 115\u001b[0m \u001b[0;32mwith\u001b[0m \u001b[0mctx_factory\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 116\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mfunc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 117\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 118\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mdecorate_context\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py\u001b[0m in \u001b[0;36mgenerate\u001b[0;34m(self, inputs, generation_config, logits_processor, stopping_criteria, prefix_allowed_tokens_fn, synced_gpus, assistant_model, streamer, negative_prompt_ids, negative_prompt_attention_mask, **kwargs)\u001b[0m\n\u001b[1;32m 2022\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 2023\u001b[0m \u001b[0;31m# 13. run sample (it degenerates to greedy search when `generation_config.do_sample=False`)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2024\u001b[0;31m result = self._sample(\n\u001b[0m\u001b[1;32m 2025\u001b[0m \u001b[0minput_ids\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 2026\u001b[0m \u001b[0mlogits_processor\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mprepared_logits_processor\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py\u001b[0m in \u001b[0;36m_sample\u001b[0;34m(self, input_ids, logits_processor, stopping_criteria, generation_config, synced_gpus, streamer, logits_warper, **model_kwargs)\u001b[0m\n\u001b[1;32m 2980\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 2981\u001b[0m \u001b[0;31m# forward pass to get next token\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2982\u001b[0;31m \u001b[0moutputs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m**\u001b[0m\u001b[0mmodel_inputs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mreturn_dict\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 2983\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 2984\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0msynced_gpus\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mthis_peer_finished\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36m_wrapped_call_impl\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m 1551\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_compiled_call_impl\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;31m# type: ignore[misc]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1552\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1553\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_call_impl\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1554\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1555\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m_call_impl\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36m_call_impl\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m 1560\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0m_global_backward_pre_hooks\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0m_global_backward_hooks\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1561\u001b[0m or _global_forward_hooks or _global_forward_pre_hooks):\n\u001b[0;32m-> 1562\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mforward_call\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1563\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1564\u001b[0m \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/accelerate/hooks.py\u001b[0m in \u001b[0;36mnew_forward\u001b[0;34m(module, *args, **kwargs)\u001b[0m\n\u001b[1;32m 168\u001b[0m \u001b[0moutput\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmodule\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_old_forward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 169\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 170\u001b[0;31m \u001b[0moutput\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmodule\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_old_forward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 171\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mmodule\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_hf_hook\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpost_forward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmodule\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0moutput\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 172\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/unsloth/models/llama.py\u001b[0m in \u001b[0;36m_CausalLM_fast_forward\u001b[0;34m(self, input_ids, causal_mask, attention_mask, position_ids, past_key_values, inputs_embeds, labels, use_cache, output_attentions, output_hidden_states, return_dict, num_logits_to_keep, *args, **kwargs)\u001b[0m\n\u001b[1;32m 935\u001b[0m \u001b[0;31m# decoder outputs consists of (dec_features, layer_state, dec_hidden, dec_attn)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 936\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_has_no_labels\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlabels\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 937\u001b[0;31m outputs = self.model(\n\u001b[0m\u001b[1;32m 938\u001b[0m \u001b[0minput_ids\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0minput_ids\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 939\u001b[0m \u001b[0mcausal_mask\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcausal_mask\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36m_wrapped_call_impl\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m 1551\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_compiled_call_impl\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;31m# type: ignore[misc]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1552\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1553\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_call_impl\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1554\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1555\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m_call_impl\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36m_call_impl\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m 1560\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0m_global_backward_pre_hooks\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0m_global_backward_hooks\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1561\u001b[0m or _global_forward_hooks or _global_forward_pre_hooks):\n\u001b[0;32m-> 1562\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mforward_call\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1563\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1564\u001b[0m \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/accelerate/hooks.py\u001b[0m in \u001b[0;36mnew_forward\u001b[0;34m(module, *args, **kwargs)\u001b[0m\n\u001b[1;32m 168\u001b[0m \u001b[0moutput\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmodule\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_old_forward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 169\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 170\u001b[0;31m \u001b[0moutput\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmodule\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_old_forward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 171\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mmodule\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_hf_hook\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpost_forward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmodule\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0moutput\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 172\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/unsloth/models/llama.py\u001b[0m in \u001b[0;36mLlamaModel_fast_forward\u001b[0;34m(self, input_ids, causal_mask, attention_mask, position_ids, past_key_values, inputs_embeds, use_cache, output_attentions, output_hidden_states, return_dict, *args, **kwargs)\u001b[0m\n\u001b[1;32m 799\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 800\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 801\u001b[0;31m layer_outputs = decoder_layer(\n\u001b[0m\u001b[1;32m 802\u001b[0m \u001b[0mhidden_states\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 803\u001b[0m \u001b[0mcausal_mask\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mmask\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36m_wrapped_call_impl\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m 1551\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_compiled_call_impl\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;31m# type: ignore[misc]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1552\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1553\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_call_impl\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1554\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1555\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m_call_impl\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36m_call_impl\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m 1560\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0m_global_backward_pre_hooks\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0m_global_backward_hooks\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1561\u001b[0m or _global_forward_hooks or _global_forward_pre_hooks):\n\u001b[0;32m-> 1562\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mforward_call\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1563\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1564\u001b[0m \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/accelerate/hooks.py\u001b[0m in \u001b[0;36mnew_forward\u001b[0;34m(module, *args, **kwargs)\u001b[0m\n\u001b[1;32m 168\u001b[0m \u001b[0moutput\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmodule\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_old_forward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 169\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 170\u001b[0;31m \u001b[0moutput\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmodule\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_old_forward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 171\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mmodule\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_hf_hook\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpost_forward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmodule\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0moutput\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 172\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/unsloth/models/gemma2.py\u001b[0m in \u001b[0;36mGemma2DecoderLayer_fast_forward\u001b[0;34m(self, hidden_states, causal_mask, attention_mask, position_ids, past_key_value, output_attentions, use_cache, padding_mask, *args, **kwargs)\u001b[0m\n\u001b[1;32m 209\u001b[0m \u001b[0mresidual\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mhidden_states\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 210\u001b[0m \u001b[0mhidden_states\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mfast_rms_layernorm_gemma2_compiled\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minput_layernorm\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mhidden_states\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mgemma\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 211\u001b[0;31m hidden_states, self_attn_weights, present_key_value = self.self_attn(\n\u001b[0m\u001b[1;32m 212\u001b[0m \u001b[0mhidden_states\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mhidden_states\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 213\u001b[0m \u001b[0mcausal_mask\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcausal_mask\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36m_wrapped_call_impl\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m 1551\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_compiled_call_impl\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;31m# type: ignore[misc]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1552\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1553\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_call_impl\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1554\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1555\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m_call_impl\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36m_call_impl\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m 1560\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0m_global_backward_pre_hooks\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0m_global_backward_hooks\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1561\u001b[0m or _global_forward_hooks or _global_forward_pre_hooks):\n\u001b[0;32m-> 1562\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mforward_call\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1563\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1564\u001b[0m \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/accelerate/hooks.py\u001b[0m in \u001b[0;36mnew_forward\u001b[0;34m(module, *args, **kwargs)\u001b[0m\n\u001b[1;32m 168\u001b[0m \u001b[0moutput\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmodule\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_old_forward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 169\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 170\u001b[0;31m \u001b[0moutput\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmodule\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_old_forward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 171\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mmodule\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_hf_hook\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpost_forward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmodule\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0moutput\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 172\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/unsloth/models/gemma2.py\u001b[0m in \u001b[0;36mGemma2Attention_fast_forward\u001b[0;34m(self, hidden_states, causal_mask, attention_mask, position_ids, past_key_value, output_attentions, use_cache, padding_mask, *args, **kwargs)\u001b[0m\n\u001b[1;32m 160\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;34m\"_flag_for_generation\"\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mkwargs\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;31m \u001b[0m\u001b[0;31m\\\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 161\u001b[0m \u001b[0mslow_attention_softcapping\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 162\u001b[0;31m \u001b[0mA\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mfx\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mQ\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mK\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mV\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcausal_mask\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbsz\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkv_seq_len\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 163\u001b[0m \u001b[0;32mpass\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 164\u001b[0m \u001b[0mA\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mapply_o\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mA\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/torch/_dynamo/eval_frame.py\u001b[0m in \u001b[0;36m_fn\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m 431\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 432\u001b[0m \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 433\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mfn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 434\u001b[0m \u001b[0;32mfinally\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 435\u001b[0m \u001b[0;31m# Restore the dynamic layer stack depth if necessary.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/unsloth/kernels/flex_attention.py\u001b[0m in \u001b[0;36mslow_attention_softcapping\u001b[0;34m(Q, K, V, causal_mask, self, bsz, q_len)\u001b[0m\n\u001b[1;32m 41\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 42\u001b[0m \u001b[0;31m# Logit softcapping\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 43\u001b[0;31m \u001b[0;34m@\u001b[0m\u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcompile\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfullgraph\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdynamic\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0moptions\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtorch_compile_options\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 44\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mslow_attention_softcapping\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mQ\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mK\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mV\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcausal_mask\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbsz\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mq_len\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 45\u001b[0m \u001b[0mn_heads\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnum_heads\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/torch/_dynamo/eval_frame.py\u001b[0m in \u001b[0;36m_fn\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m 598\u001b[0m \u001b[0mprior\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mset_eval_frame\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcallback\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 599\u001b[0m \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 600\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mfn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 601\u001b[0m \u001b[0;32mfinally\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 602\u001b[0m \u001b[0mset_eval_frame\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprior\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/torch/_functorch/aot_autograd.py\u001b[0m in \u001b[0;36mforward\u001b[0;34m(*runtime_args)\u001b[0m\n\u001b[1;32m 985\u001b[0m \u001b[0mfull_args\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mextend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mparams_flat\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 986\u001b[0m \u001b[0mfull_args\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mextend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mruntime_args\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 987\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mcompiled_fn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfull_args\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 988\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 989\u001b[0m \u001b[0;31m# Just for convenience\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/torch/_functorch/_aot_autograd/runtime_wrappers.py\u001b[0m in \u001b[0;36mruntime_wrapper\u001b[0;34m(args)\u001b[0m\n\u001b[1;32m 215\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mgrad_enabled\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 216\u001b[0m \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_C\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_set_grad_enabled\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 217\u001b[0;31m all_outs = call_func_at_runtime_with_args(\n\u001b[0m\u001b[1;32m 218\u001b[0m \u001b[0mcompiled_fn\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdisable_amp\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mdisable_amp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msteal_args\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 219\u001b[0m )\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/torch/_functorch/_aot_autograd/utils.py\u001b[0m in \u001b[0;36mcall_func_at_runtime_with_args\u001b[0;34m(f, args, steal_args, disable_amp)\u001b[0m\n\u001b[1;32m 118\u001b[0m \u001b[0;32mwith\u001b[0m \u001b[0mcontext\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 119\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mhasattr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mf\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"_boxed_call\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 120\u001b[0;31m \u001b[0mout\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnormalize_as_list\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 121\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 122\u001b[0m \u001b[0;31m# TODO: Please remove soon\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/torch/_functorch/_aot_autograd/runtime_wrappers.py\u001b[0m in \u001b[0;36mwrapper\u001b[0;34m(runtime_args)\u001b[0m\n\u001b[1;32m 449\u001b[0m )\n\u001b[1;32m 450\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mout\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 451\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mcompiled_fn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mruntime_args\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 452\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 453\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mwrapper\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/torch/_inductor/codecache.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, inputs)\u001b[0m\n\u001b[1;32m 1129\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m__call__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minputs\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mList\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mAny\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m->\u001b[0m \u001b[0mAny\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1130\u001b[0m \u001b[0;32massert\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurrent_callable\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1131\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurrent_callable\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minputs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1132\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1133\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/torch/_inductor/compile_fx.py\u001b[0m in \u001b[0;36mrun\u001b[0;34m(new_inputs)\u001b[0m\n\u001b[1;32m 942\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mrun\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnew_inputs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 943\u001b[0m \u001b[0mcopy_misaligned_inputs\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnew_inputs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minputs_to_check\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 944\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnew_inputs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 945\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 946\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mrun\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/tmp/torchinductor_root/mo/cmogaxo5lyrlu5e36hooku65vvy4msskyyadf3lmgtocjgizkw5k.py\u001b[0m in \u001b[0;36mcall\u001b[0;34m(args)\u001b[0m\n\u001b[1;32m 263\u001b[0m \u001b[0;31m# Source Nodes: [A_1, A_2, A_3, softmax, tanh, truediv], Original ATen: [aten._softmax, aten._to_copy, aten.add, aten.div, aten.mul, aten.tanh]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 264\u001b[0m \u001b[0mtriton_red_fused__softmax__to_copy_add_div_mul_tanh_2_xnumel\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0ms2\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0ms3\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0ms10\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 265\u001b[0;31m \u001b[0mtriton_red_fused__softmax__to_copy_add_div_mul_tanh_2\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrun\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mbuf2\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0marg12_1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbuf5\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0ms10\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0ms11\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtriton_red_fused__softmax__to_copy_add_div_mul_tanh_2_xnumel\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0ms10\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mgrid\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mgrid\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtriton_red_fused__softmax__to_copy_add_div_mul_tanh_2_xnumel\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstream\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mstream0\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 266\u001b[0m \u001b[0;32mdel\u001b[0m \u001b[0marg12_1\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 267\u001b[0m \u001b[0mbuf6\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mbuf1\u001b[0m \u001b[0;31m# alloc\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/torch/_inductor/runtime/triton_heuristics.py\u001b[0m in \u001b[0;36mrun\u001b[0;34m(self, grid, stream, *args, **kwargs)\u001b[0m\n\u001b[1;32m 866\u001b[0m )\n\u001b[1;32m 867\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 868\u001b[0;31m return launcher(\n\u001b[0m\u001b[1;32m 869\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 870\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m<string>\u001b[0m in \u001b[0;36mlauncher\u001b[0;34m(in_ptr0, in_ptr1, out_ptr2, ks0, ks1, xnumel, rnumel, grid, stream)\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/triton/backends/nvidia/driver.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m 363\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 364\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m__call__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 365\u001b[0;31m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlaunch\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 366\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 367\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;31mKeyboardInterrupt\u001b[0m: "]}]},{"cell_type":"code","source":["# alpaca_prompt = Copied from above\n","FastLanguageModel.for_inference(model) # Enable native 2x faster inference\n","inputs = tokenizer(\n","[\n"," alpaca_prompt.format(\n"," \"뼈마디가 쑤시면서 몹시 아픈 증상?\", # instruction\n"," \"\", # input\n"," \"\", # output - leave this blank for generation!\n"," )\n","], return_tensors = \"pt\").to(\"cuda\")\n","\n","outputs = model.generate(**inputs, max_new_tokens = 64, use_cache = True)\n","tokenizer.batch_decode(outputs)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"XOGMfVPkbYRW","executionInfo":{"status":"ok","timestamp":1727860019422,"user_tz":-540,"elapsed":1281,"user":{"displayName":"최현진","userId":"12812404047517020320"}},"outputId":"fa3d7907-0a38-463a-8fbf-b16efdf7f941"},"execution_count":17,"outputs":[{"output_type":"execute_result","data":{"text/plain":["['<bos>Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\\n\\n### Instruction:\\n뼈마디가 쑤시면서 몹시 아픈 증상?\\n\\n### Input:\\n\\n\\n### Response:\\n뼈-쑤심<eos>']"]},"metadata":{},"execution_count":17}]},{"cell_type":"code","source":["# alpaca_prompt = Copied from above\n","FastLanguageModel.for_inference(model) # Enable native 2x faster inference\n","inputs = tokenizer(\n","[\n"," alpaca_prompt.format(\n"," \"날씨가 활짝 개지 않고 몹시 흐려지다\", # instruction\n"," \"\", # input\n"," \"\", # output - leave this blank for generation!\n"," )\n","], return_tensors = \"pt\").to(\"cuda\")\n","\n","outputs = model.generate(**inputs, max_new_tokens = 64, use_cache = True)\n","tokenizer.batch_decode(outputs)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"PPxOqdSVcFlQ","executionInfo":{"status":"ok","timestamp":1727860025248,"user_tz":-540,"elapsed":1082,"user":{"displayName":"최현진","userId":"12812404047517020320"}},"outputId":"2be85ec8-5b68-4f5c-8e45-49f357a4eb08"},"execution_count":18,"outputs":[{"output_type":"execute_result","data":{"text/plain":["['<bos>Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\\n\\n### Instruction:\\n날씨가 활짝 개지 않고 몹시 흐려지다\\n\\n### Input:\\n\\n\\n### Response:\\n흐리다<eos>']"]},"metadata":{},"execution_count":18}]},{"cell_type":"code","source":["# alpaca_prompt = Copied from above\n","FastLanguageModel.for_inference(model) # Enable native 2x faster inference\n","inputs = tokenizer(\n","[\n"," alpaca_prompt.format(\n"," \"세계화와 지역화가 동시에 진행되는 현상\", # instruction\n"," \"\", # input\n"," \"\", # output - leave this blank for generation!\n"," )\n","], return_tensors = \"pt\").to(\"cuda\")\n","\n","outputs = model.generate(**inputs, max_new_tokens = 64, use_cache = True)\n","tokenizer.batch_decode(outputs)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"L-o4GT8kg6eL","executionInfo":{"status":"ok","timestamp":1727860031644,"user_tz":-540,"elapsed":1351,"user":{"displayName":"최현진","userId":"12812404047517020320"}},"outputId":"c72f52a0-9e2c-495d-eca5-ed0e2ef132e6"},"execution_count":19,"outputs":[{"output_type":"execute_result","data":{"text/plain":["['<bos>Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\\n\\n### Instruction:\\n세계화와 지역화가 동시에 진행되는 현상\\n\\n### Input:\\n\\n\\n### Response:\\n글로벌-리지컬<eos>']"]},"metadata":{},"execution_count":19}]},{"cell_type":"code","source":["# alpaca_prompt = Copied from above\n","FastLanguageModel.for_inference(model) # Enable native 2x faster inference\n","inputs = tokenizer(\n","[\n"," alpaca_prompt.format(\n"," \"마라톤 따위의 지구력이 필요한 운동에서, 선수들이 탄수화물을 정상 수준 이상으로 몸에 축적하는 방법\", # instruction\n"," \"\", # input\n"," \"\", # output - leave this blank for generation!\n"," )\n","], return_tensors = \"pt\").to(\"cuda\")\n","\n","outputs = model.generate(**inputs, max_new_tokens = 64, use_cache = True)\n","tokenizer.batch_decode(outputs)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"rOoybcS3hrKb","executionInfo":{"status":"ok","timestamp":1727860037803,"user_tz":-540,"elapsed":1617,"user":{"displayName":"최현진","userId":"12812404047517020320"}},"outputId":"68cff61c-f00c-4904-f35f-6fbefd5a55a9"},"execution_count":20,"outputs":[{"output_type":"execute_result","data":{"text/plain":["['<bos>Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\\n\\n### Instruction:\\n마라톤 따위의 지구력이 필요한 운동에서, 선수들이 탄수화물을 정상 수준 이상으로 몸에 축적하는 방법\\n\\n### Input:\\n\\n\\n### Response:\\n카보하이드레이트^로딩<eos>']"]},"metadata":{},"execution_count":20}]},{"cell_type":"markdown","source":[" You can also use a `TextStreamer` for continuous inference - so you can see the generation token by token, instead of waiting the whole time!"],"metadata":{"id":"CrSvZObor0lY"}},{"cell_type":"code","source":["# alpaca_prompt = Copied from above\n","FastLanguageModel.for_inference(model) # Enable native 2x faster inference\n","inputs = tokenizer(\n","[\n"," alpaca_prompt.format(\n"," \"Continue the fibonnaci sequence.\", # instruction\n"," \"1, 1, 2, 3, 5, 8\", # input\n"," \"\", # output - leave this blank for generation!\n"," )\n","], return_tensors = \"pt\").to(\"cuda\")\n","\n","from transformers import TextStreamer\n","text_streamer = TextStreamer(tokenizer)\n","_ = model.generate(**inputs, streamer = text_streamer, max_new_tokens = 128)"],"metadata":{"id":"e2pEuRb1r2Vg","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1727860055611,"user_tz":-540,"elapsed":968,"user":{"displayName":"최현진","userId":"12812404047517020320"}},"outputId":"257461ac-6164-47cd-e49c-a5273a8987b6"},"execution_count":21,"outputs":[{"output_type":"stream","name":"stdout","text":["<bos>Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n","\n","### Instruction:\n","Continue the fibonnaci sequence.\n","\n","### Input:\n","1, 1, 2, 3, 5, 8\n","\n","### Response:\n","13<eos>\n"]}]},{"cell_type":"markdown","source":["<a name=\"Save\"></a>\n","### Saving, loading finetuned models\n","To save the final model as LoRA adapters, either use Huggingface's `push_to_hub` for an online save or `save_pretrained` for a local save.\n","\n","**[NOTE]** This ONLY saves the LoRA adapters, and not the full model. To save to 16bit or GGUF, scroll down!"],"metadata":{"id":"uMuVrWbjAzhc"}},{"cell_type":"code","source":["model.save_pretrained(\"lora_model\") # Local saving\n","tokenizer.save_pretrained(\"lora_model\")\n","# model.push_to_hub(\"your_name/lora_model\", token = \"...\") # Online saving\n","# tokenizer.push_to_hub(\"your_name/lora_model\", token = \"...\") # Online saving"],"metadata":{"id":"upcOlWe7A1vc","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1720290850683,"user_tz":420,"elapsed":1470,"user":{"displayName":"Daniel Han-Chen","userId":"17402123517466114840"}},"outputId":"f0a37984-53f4-4f4b-c16b-0e6c63385456"},"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/plain":["('lora_model/tokenizer_config.json',\n"," 'lora_model/special_tokens_map.json',\n"," 'lora_model/tokenizer.model',\n"," 'lora_model/added_tokens.json',\n"," 'lora_model/tokenizer.json')"]},"metadata":{},"execution_count":11}]},{"cell_type":"markdown","source":["Now if you want to load the LoRA adapters we just saved for inference, set `False` to `True`:"],"metadata":{"id":"AEEcJ4qfC7Lp"}},{"cell_type":"code","source":["if False:\n"," from unsloth import FastLanguageModel\n"," model, tokenizer = FastLanguageModel.from_pretrained(\n"," model_name = \"lora_model\", # YOUR MODEL YOU USED FOR TRAINING\n"," max_seq_length = max_seq_length,\n"," dtype = dtype,\n"," load_in_4bit = load_in_4bit,\n"," )\n"," FastLanguageModel.for_inference(model) # Enable native 2x faster inference\n","\n","# alpaca_prompt = You MUST copy from above!\n","\n","inputs = tokenizer(\n","[\n"," alpaca_prompt.format(\n"," \"What is a famous tall tower in Paris?\", # instruction\n"," \"\", # input\n"," \"\", # output - leave this blank for generation!\n"," )\n","], return_tensors = \"pt\").to(\"cuda\")\n","\n","outputs = model.generate(**inputs, max_new_tokens = 64, use_cache = True)\n","tokenizer.batch_decode(outputs)"],"metadata":{"id":"MKX_XKs_BNZR","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1720290857999,"user_tz":420,"elapsed":7322,"user":{"displayName":"Daniel Han-Chen","userId":"17402123517466114840"}},"outputId":"b50dc7cc-8ce7-41a9-ab52-993eb3c17e16"},"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/plain":["['<bos>Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\\n\\n### Instruction:\\nWhat is a famous tall tower in Paris?\\n\\n### Input:\\n\\n\\n### Response:\\nThe Eiffel Tower is a famous tall tower in Paris, France. It is located on the Champ de Mars in the 7th arrondissement of Paris, on the left bank of the River Seine. The tower was designed by engineer Gustave Eiffel and his company, and was constructed from 1887 to 18']"]},"metadata":{},"execution_count":12}]},{"cell_type":"markdown","source":["You can also use Hugging Face's `AutoModelForPeftCausalLM`. Only use this if you do not have `unsloth` installed. It can be hopelessly slow, since `4bit` model downloading is not supported, and Unsloth's **inference is 2x faster**."],"metadata":{"id":"QQMjaNrjsU5_"}},{"cell_type":"code","source":["if False:\n"," # I highly do NOT suggest - use Unsloth if possible\n"," from peft import AutoPeftModelForCausalLM\n"," from transformers import AutoTokenizer\n"," model = AutoPeftModelForCausalLM.from_pretrained(\n"," \"lora_model\", # YOUR MODEL YOU USED FOR TRAINING\n"," load_in_4bit = load_in_4bit,\n"," )\n"," tokenizer = AutoTokenizer.from_pretrained(\"lora_model\")"],"metadata":{"id":"yFfaXG0WsQuE"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["### Saving to float16 for VLLM\n","\n","We also support saving to `float16` directly. Select `merged_16bit` for float16 or `merged_4bit` for int4. We also allow `lora` adapters as a fallback. Use `push_to_hub_merged` to upload to your Hugging Face account! You can go to https://huggingface.co/settings/tokens for your personal tokens."],"metadata":{"id":"f422JgM9sdVT"}},{"cell_type":"code","source":["# Merge to 16bit\n","if False: model.save_pretrained_merged(\"model\", tokenizer, save_method = \"merged_16bit\",)\n","if False: model.push_to_hub_merged(\"hf/model\", tokenizer, save_method = \"merged_16bit\", token = \"\")\n","\n","# Merge to 4bit\n","if False: model.save_pretrained_merged(\"model\", tokenizer, save_method = \"merged_4bit\",)\n","if False: model.push_to_hub_merged(\"hf/model\", tokenizer, save_method = \"merged_4bit\", token = \"\")\n","\n","# Just LoRA adapters\n","if False: model.save_pretrained_merged(\"model\", tokenizer, save_method = \"lora\",)\n","if False: model.push_to_hub_merged(\"hf/model\", tokenizer, save_method = \"lora\", token = \"\")"],"metadata":{"id":"iHjt_SMYsd3P"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["### GGUF / llama.cpp Conversion\n","To save to `GGUF` / `llama.cpp`, we support it natively now! We clone `llama.cpp` and we default save it to `q8_0`. We allow all methods like `q4_k_m`. Use `save_pretrained_gguf` for local saving and `push_to_hub_gguf` for uploading to HF.\n","\n","Some supported quant methods (full list on our [Wiki page](https://github.com/unslothai/unsloth/wiki#gguf-quantization-options)):\n","* `q8_0` - Fast conversion. High resource use, but generally acceptable.\n","* `q4_k_m` - Recommended. Uses Q6_K for half of the attention.wv and feed_forward.w2 tensors, else Q4_K.\n","* `q5_k_m` - Recommended. Uses Q6_K for half of the attention.wv and feed_forward.w2 tensors, else Q5_K."],"metadata":{"id":"TCv4vXHd61i7"}},{"cell_type":"code","source":["# Save to 8bit Q8_0\n","if False: model.save_pretrained_gguf(\"model\", tokenizer,)\n","if False: model.push_to_hub_gguf(\"hf/model\", tokenizer, token = \"\")\n","\n","# Save to 16bit GGUF\n","if False: model.save_pretrained_gguf(\"model\", tokenizer, quantization_method = \"f16\")\n","if False: model.push_to_hub_gguf(\"hf/model\", tokenizer, quantization_method = \"f16\", token = \"\")\n","\n","# Save to q4_k_m GGUF\n","if False: model.save_pretrained_gguf(\"model\", tokenizer, quantization_method = \"q4_k_m\")\n","if False: model.push_to_hub_gguf(\"hf/model\", tokenizer, quantization_method = \"q4_k_m\", token = \"\")"],"metadata":{"id":"FqfebeAdT073"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["Now, use the `model-unsloth.gguf` file or `model-unsloth-Q4_K_M.gguf` file in `llama.cpp` or a UI based system like `GPT4All`. You can install GPT4All by going [here](https://gpt4all.io/index.html)."],"metadata":{"id":"bDp0zNpwe6U_"}},{"cell_type":"markdown","source":["And we're done! If you have any questions on Unsloth, we have a [Discord](https://discord.gg/u54VK8m8tk) channel! If you find any bugs or want to keep updated with the latest LLM stuff, or need help, join projects etc, feel free to join our Discord!\n","\n","Some other links:\n","1. Zephyr DPO 2x faster [free Colab](https://colab.research.google.com/drive/15vttTpzzVXv_tJwEk-hIcQ0S9FcEWvwP?usp=sharing)\n","2. Llama 7b 2x faster [free Colab](https://colab.research.google.com/drive/1lBzz5KeZJKXjvivbYvmGarix9Ao6Wxe5?usp=sharing)\n","3. TinyLlama 4x faster full Alpaca 52K in 1 hour [free Colab](https://colab.research.google.com/drive/1AZghoNBQaMDgWJpi4RbffGM1h6raLUj9?usp=sharing)\n","4. CodeLlama 34b 2x faster [A100 on Colab](https://colab.research.google.com/drive/1y7A0AxE3y8gdj4AVkl2aZX47Xu3P1wJT?usp=sharing)\n","5. Mistral 7b [free Kaggle version](https://www.kaggle.com/code/danielhanchen/kaggle-mistral-7b-unsloth-notebook)\n","6. We also did a [blog](https://huggingface.co/blog/unsloth-trl) with 🤗 HuggingFace, and we're in the TRL [docs](https://huggingface.co/docs/trl/main/en/sft_trainer#accelerate-fine-tuning-2x-using-unsloth)!\n","7. `ChatML` for ShareGPT datasets, [conversational notebook](https://colab.research.google.com/drive/1Aau3lgPzeZKQ-98h69CCu1UJcvIBLmy2?usp=sharing)\n","8. Text completions like novel writing [notebook](https://colab.research.google.com/drive/1ef-tab5bhkvWmBOObepl1WgJvfvSzn5Q?usp=sharing)\n","9. [**NEW**] We make Phi-3 Medium / Mini **2x faster**! See our [Phi-3 Medium notebook](https://colab.research.google.com/drive/1hhdhBa1j_hsymiW9m-WzxQtgqTH_NHqi?usp=sharing)\n","\n","<div class=\"align-center\">\n"," <a href=\"https://github.com/unslothai/unsloth\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png\" width=\"115\"></a>\n"," <a href=\"https://discord.gg/u54VK8m8tk\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/Discord.png\" width=\"145\"></a>\n"," <a href=\"https://ko-fi.com/unsloth\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/Kofi button.png\" width=\"145\"></a></a> Support our work if you can! Thanks!\n","</div>"],"metadata":{"id":"Zt9CHJqO6p30"}}],"metadata":{"accelerator":"GPU","colab":{"provenance":[{"file_id":"1vIrqH5uYDQwsJ4-OO3DErvuv4pBgVwk4","timestamp":1727169129677},{"file_id":"135ced7oHytdxu3N2DNe1Z0kqjyYIkDXp","timestamp":1720030210512},{"file_id":"10NbwlsRChbma1v55m8LAPYG15uQv6HLo","timestamp":1713459337061},{"file_id":"1Dyauq4kTZoLewQ1cApceUQVNcnnNTzg_","timestamp":1708958229810},{"file_id":"1lBzz5KeZJKXjvivbYvmGarix9Ao6Wxe5","timestamp":1703608159823},{"file_id":"1oW55fBmwzCOrBVX66RcpptL3a99qWBxb","timestamp":1702886138876}],"gpuType":"T4"},"kernelspec":{"display_name":"Python 3","name":"python3"},"language_info":{"name":"python"},"widgets":{"application/vnd.jupyter.widget-state+json":{"0a6c54edc08e44ee9cf092414a4e42ec":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_e68d1f22e75343999966d2ff409acc73","IPY_MODEL_47a560c6e53b43a4bd24357339046c8f","IPY_MODEL_909cbc94cb6d4a9e80ad41f5fcc369c9"],"layout":"IPY_MODEL_bd62bd3d631e4429b2a0347b0cc5654c"}},"e68d1f22e75343999966d2ff409acc73":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_9b395d60d64643b6804f2fa417df90e8","placeholder":"","style":"IPY_MODEL_5f6369fe1652456da152c5189b9cf200","value":"model.safetensors: 100%"}},"47a560c6e53b43a4bd24357339046c8f":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"danger","description":"","description_tooltip":null,"layout":"IPY_MODEL_d3ba225fcb7e4cdca542551317b31569","max":6130708044,"min":0,"orientation":"horizontal","style":"IPY_MODEL_89d85d9cfc154254ac2472f278be1a69","value":6130707460}},"909cbc94cb6d4a9e80ad41f5fcc369c9":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_08ceb05830554399be0cf251f6256970","placeholder":"","style":"IPY_MODEL_998157211dfb4d21a8dd0d8efa3fc245","value":" 6.13G/6.13G [00:54<00:00, 314MB/s]"}},"bd62bd3d631e4429b2a0347b0cc5654c":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9b395d60d64643b6804f2fa417df90e8":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5f6369fe1652456da152c5189b9cf200":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"d3ba225fcb7e4cdca542551317b31569":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"89d85d9cfc154254ac2472f278be1a69":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"08ceb05830554399be0cf251f6256970":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"998157211dfb4d21a8dd0d8efa3fc245":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"deacfebe3871438abdc90b60dad9a32f":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_95025404591f4567b3db485c571095f9","IPY_MODEL_1b584563488348579f132aa98d2dfe09","IPY_MODEL_d78989601c354f1b8131a394efc440c2"],"layout":"IPY_MODEL_3c772a9f001448d99d763219593686c3"}},"95025404591f4567b3db485c571095f9":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_d354121e9f904394b8b6af25fab434d3","placeholder":"","style":"IPY_MODEL_395e60e1600f42739c905da1720cc187","value":"generation_config.json: 100%"}},"1b584563488348579f132aa98d2dfe09":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_eee99d979cbb42428313f7c55270298e","max":190,"min":0,"orientation":"horizontal","style":"IPY_MODEL_fdc41bad820143588d82734f6147b13f","value":190}},"d78989601c354f1b8131a394efc440c2":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_0e63f87958bf4e80a5f582abe41f2364","placeholder":"","style":"IPY_MODEL_742265a4480140ed9b3578b3098c8fc0","value":" 190/190 [00:00<00:00, 7.15kB/s]"}},"3c772a9f001448d99d763219593686c3":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d354121e9f904394b8b6af25fab434d3":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"395e60e1600f42739c905da1720cc187":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"eee99d979cbb42428313f7c55270298e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"fdc41bad820143588d82734f6147b13f":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"0e63f87958bf4e80a5f582abe41f2364":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"742265a4480140ed9b3578b3098c8fc0":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"8a3ec41fc0be4616997556aab65744fd":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_db34ae736e5045f49ec389ccf26c7f5a","IPY_MODEL_fcd9b941f5014d7587b2f00798721374","IPY_MODEL_824f8c48ed134646bef3ea76baad855c"],"layout":"IPY_MODEL_b1d7d85fb45746f4b57988d76ae26426"}},"db34ae736e5045f49ec389ccf26c7f5a":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_1546ea46587d480eb29e6c1844ad2f80","placeholder":"","style":"IPY_MODEL_91f32f1839bd409783c9bca5e9ad0bcc","value":"tokenizer_config.json: 100%"}},"fcd9b941f5014d7587b2f00798721374":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_60b1328a8ab54a47be35e74cfabf9288","max":46405,"min":0,"orientation":"horizontal","style":"IPY_MODEL_c38137c015c2457a807d403ef4cd3cc6","value":46405}},"824f8c48ed134646bef3ea76baad855c":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_87cf68af10c0499199a65096b0beb78c","placeholder":"","style":"IPY_MODEL_e8511def2cbb47d1b0269b5816e6c03b","value":" 46.4k/46.4k [00:00<00:00, 2.16MB/s]"}},"b1d7d85fb45746f4b57988d76ae26426":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"1546ea46587d480eb29e6c1844ad2f80":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"91f32f1839bd409783c9bca5e9ad0bcc":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"60b1328a8ab54a47be35e74cfabf9288":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c38137c015c2457a807d403ef4cd3cc6":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"87cf68af10c0499199a65096b0beb78c":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e8511def2cbb47d1b0269b5816e6c03b":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"fa50ca8ae1374b748d69e4b6541a5a72":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_4b8f823425a94cba9f0104848ee1e896","IPY_MODEL_f0e7e6be84924623900da021faa743b6","IPY_MODEL_d7b289713a004ca7804204792a2b6ed4"],"layout":"IPY_MODEL_bbac4079f11e4e52a6108b6fb3e4a2e5"}},"4b8f823425a94cba9f0104848ee1e896":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_680be5d57ddf4bcf92f3a9b4ed413684","placeholder":"","style":"IPY_MODEL_ac3f56ad9f354790a3a31101d196a78e","value":"tokenizer.model: 100%"}},"f0e7e6be84924623900da021faa743b6":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_23428cd9c0584adea897d1bef50839af","max":4241003,"min":0,"orientation":"horizontal","style":"IPY_MODEL_49af41ecf8084d83bfdc37d25303e576","value":4241003}},"d7b289713a004ca7804204792a2b6ed4":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_c4c646038ea64aa29c534bef36f5d06e","placeholder":"","style":"IPY_MODEL_38909ccb06854dd886e7be9a8f7657c1","value":" 4.24M/4.24M [00:00<00:00, 46.7MB/s]"}},"bbac4079f11e4e52a6108b6fb3e4a2e5":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"680be5d57ddf4bcf92f3a9b4ed413684":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ac3f56ad9f354790a3a31101d196a78e":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"23428cd9c0584adea897d1bef50839af":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"49af41ecf8084d83bfdc37d25303e576":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"c4c646038ea64aa29c534bef36f5d06e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"38909ccb06854dd886e7be9a8f7657c1":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"20ac96f8e5674c1bb3204efafb951d7d":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_16321c4d05a14e049a393dca08309ad6","IPY_MODEL_3c75650d4a5647079a638dd7566dd8ec","IPY_MODEL_490aec0bf7d642aa8d6fa219c23fa4f6"],"layout":"IPY_MODEL_26e8a4b2d9d941c190ca828b4f0bc6ac"}},"16321c4d05a14e049a393dca08309ad6":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_470b70df4d94475d91156919b93e5712","placeholder":"","style":"IPY_MODEL_21b2d11c391b4442b0bda4f0a30bd625","value":"special_tokens_map.json: 100%"}},"3c75650d4a5647079a638dd7566dd8ec":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_3fcf984878dc4ec6b9a184e5103da8f3","max":636,"min":0,"orientation":"horizontal","style":"IPY_MODEL_deca135788e84a34a209df86101f6343","value":636}},"490aec0bf7d642aa8d6fa219c23fa4f6":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_87473024bbea4171a03d41967520aa99","placeholder":"","style":"IPY_MODEL_cb2030414a9048a88cb39f09ddc98189","value":" 636/636 [00:00<00:00, 43.2kB/s]"}},"26e8a4b2d9d941c190ca828b4f0bc6ac":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"470b70df4d94475d91156919b93e5712":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"21b2d11c391b4442b0bda4f0a30bd625":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"3fcf984878dc4ec6b9a184e5103da8f3":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"deca135788e84a34a209df86101f6343":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"87473024bbea4171a03d41967520aa99":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"cb2030414a9048a88cb39f09ddc98189":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"15737b9be0544c10b7903dabda8653b1":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_8310ad421f0042a7868a45f87b845b44","IPY_MODEL_627cb05685c64ddfa3bba3dab7471d11","IPY_MODEL_ec5b4f91155d47d1a4df6e720b7de3cb"],"layout":"IPY_MODEL_eebe0e888799428394efff69c629a1db"}},"8310ad421f0042a7868a45f87b845b44":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_45c1af5a461747f5b2389977fdf851ce","placeholder":"","style":"IPY_MODEL_afcbcc7413214d31ba00d881d5e04cc8","value":"tokenizer.json: 100%"}},"627cb05685c64ddfa3bba3dab7471d11":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_913a12d90a894664ab2219551561113e","max":17525357,"min":0,"orientation":"horizontal","style":"IPY_MODEL_801783d19f094cf2a10669ce2dc61f2d","value":17525357}},"ec5b4f91155d47d1a4df6e720b7de3cb":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_b1091be9f09441c0b359986f144bec62","placeholder":"","style":"IPY_MODEL_d8ad7fed7e7f4d858122658213c2ff4f","value":" 17.5M/17.5M [00:00<00:00, 195MB/s]"}},"eebe0e888799428394efff69c629a1db":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"45c1af5a461747f5b2389977fdf851ce":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"afcbcc7413214d31ba00d881d5e04cc8":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"913a12d90a894664ab2219551561113e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"801783d19f094cf2a10669ce2dc61f2d":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"b1091be9f09441c0b359986f144bec62":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d8ad7fed7e7f4d858122658213c2ff4f":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"ef997c8aba40422780940ca0747c4614":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_5f25d2216afe408b8d8b4ccb75da5840","IPY_MODEL_e2fb0d0600c046008617abc45da4962c","IPY_MODEL_0fe489f0907946ff8b755226f02cab45"],"layout":"IPY_MODEL_3b0e9f300feb4867bd3850aa96d0e63a"}},"5f25d2216afe408b8d8b4ccb75da5840":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_b1ffa237c46d447bbaf88ca050d8b6de","placeholder":"","style":"IPY_MODEL_e9744595dd2640209c6dd70c9c09eb84","value":"Generating train split: "}},"e2fb0d0600c046008617abc45da4962c":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_411bc890289345fbb8c1eec04cb8ee55","max":1,"min":0,"orientation":"horizontal","style":"IPY_MODEL_0015b82880034b85981aeb78daa91ab2","value":1}},"0fe489f0907946ff8b755226f02cab45":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_492f49a7f115445daebec8851e199de8","placeholder":"","style":"IPY_MODEL_96258cd3ae914460bba000d71d416e54","value":" 1181401/0 [00:04<00:00, 448353.10 examples/s]"}},"3b0e9f300feb4867bd3850aa96d0e63a":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b1ffa237c46d447bbaf88ca050d8b6de":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e9744595dd2640209c6dd70c9c09eb84":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"411bc890289345fbb8c1eec04cb8ee55":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":"20px"}},"0015b82880034b85981aeb78daa91ab2":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"492f49a7f115445daebec8851e199de8":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"96258cd3ae914460bba000d71d416e54":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"1faf886b995e472e8666ed01d78e2ee6":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_94ec88e90b95488d876d7d3e6e876d4f","IPY_MODEL_c53b9f12fc0b410aa94fc94fb7f91e7e","IPY_MODEL_96e7acc07a894f0d8c12df3f47d62a54"],"layout":"IPY_MODEL_9565507bc58a40e58b536c1333d0da06"}},"94ec88e90b95488d876d7d3e6e876d4f":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_483f3084119141ce8a0429cd31e903f6","placeholder":"","style":"IPY_MODEL_eac58a2cf5564476a0d06e8da5419a8f","value":"Map: 100%"}},"c53b9f12fc0b410aa94fc94fb7f91e7e":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_79bf783adbb341d8b47fd5d3876f799b","max":1181401,"min":0,"orientation":"horizontal","style":"IPY_MODEL_64599a388e374269a5bbb0df3228824f","value":1181401}},"96e7acc07a894f0d8c12df3f47d62a54":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_ea2ce876b2d142989e0fc9615baa6b2f","placeholder":"","style":"IPY_MODEL_006650d347494172b83d27a97f0d0fae","value":" 1181401/1181401 [00:12<00:00, 119025.98 examples/s]"}},"9565507bc58a40e58b536c1333d0da06":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"483f3084119141ce8a0429cd31e903f6":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"eac58a2cf5564476a0d06e8da5419a8f":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"79bf783adbb341d8b47fd5d3876f799b":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"64599a388e374269a5bbb0df3228824f":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"ea2ce876b2d142989e0fc9615baa6b2f":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"006650d347494172b83d27a97f0d0fae":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"674ecbbecc64450198992a7735460e07":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_b4bb93af94a14918910064559e6f3c53","IPY_MODEL_d837748b23514b769c1b78f6d1a62ac9","IPY_MODEL_5fde99f56d9e43869fcf073c61df7bd9"],"layout":"IPY_MODEL_35fb0e04580f4a3cb946f9611d5cd210"}},"b4bb93af94a14918910064559e6f3c53":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_35834a01131b4d56af416433e2c28a08","placeholder":"","style":"IPY_MODEL_bb94d7eaffe14e17bd2e1a5bd59b47a2","value":"Map: 100%"}},"d837748b23514b769c1b78f6d1a62ac9":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_92296ea7e5a6487a9571d98364417b98","max":1063260,"min":0,"orientation":"horizontal","style":"IPY_MODEL_b2a5d3c1a9ea4cd6a34c6023476f290c","value":1063260}},"5fde99f56d9e43869fcf073c61df7bd9":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_eccfa1faa3b84d3cb0138ef2f30bfa9f","placeholder":"","style":"IPY_MODEL_53a4c3365930429095c818ecb14b1c80","value":" 1063260/1063260 [00:23<00:00, 62133.01 examples/s]"}},"35fb0e04580f4a3cb946f9611d5cd210":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"35834a01131b4d56af416433e2c28a08":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"bb94d7eaffe14e17bd2e1a5bd59b47a2":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"92296ea7e5a6487a9571d98364417b98":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b2a5d3c1a9ea4cd6a34c6023476f290c":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"eccfa1faa3b84d3cb0138ef2f30bfa9f":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"53a4c3365930429095c818ecb14b1c80":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"4758ab6d84ed4eb99ea0474c6eb97469":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_f23e7ba8b1ff40fdab5af080932266ad","IPY_MODEL_b498992a458b47c78dedb921451230e6","IPY_MODEL_6753316ed94b4b29b2d7f1856b0afb32"],"layout":"IPY_MODEL_dcf7f5eed1ba4b459e5f7edfd38fc8fa"}},"f23e7ba8b1ff40fdab5af080932266ad":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_c25a74deead945a18c871359a6b03b31","placeholder":"","style":"IPY_MODEL_92f54f687ec34a37955b17c7e4e3b8e2","value":"Map (num_proc=2): 100%"}},"b498992a458b47c78dedb921451230e6":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_31ee693f414b4b208bca95c0a5902653","max":1063260,"min":0,"orientation":"horizontal","style":"IPY_MODEL_aed8d91169a6436bb885e1f7e3e2c856","value":1063260}},"6753316ed94b4b29b2d7f1856b0afb32":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_59bdfecc46be4c40bc135882dfe401c1","placeholder":"","style":"IPY_MODEL_ddda26ee4fbb4e3d86e3843c5de85884","value":" 1063260/1063260 [03:05<00:00, 3900.69 examples/s]"}},"dcf7f5eed1ba4b459e5f7edfd38fc8fa":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c25a74deead945a18c871359a6b03b31":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"92f54f687ec34a37955b17c7e4e3b8e2":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"31ee693f414b4b208bca95c0a5902653":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"aed8d91169a6436bb885e1f7e3e2c856":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"59bdfecc46be4c40bc135882dfe401c1":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ddda26ee4fbb4e3d86e3843c5de85884":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}}}}},"nbformat":4,"nbformat_minor":0} |