Hugging Face's logo

dreamboat26
/
Tokenizers

English
Model card Files Community
dreamboat26 commited on
Commit
ad546bc
•
1 Parent(s): 751dc51

First commit

Browse files
Files changed (1) hide show
  1. Tokenizers_(TensorFlow).ipynb +1792 -0
Tokenizers_(TensorFlow).ipynb ADDED
@@ -0,0 +1,1792 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cells": [
3
+ {
4
+ "cell_type": "markdown",
5
+ "metadata": {
6
+ "id": "p4tEKoTZ1cHe"
7
+ },
8
+ "source": [
9
+ "# Tokenizers (TensorFlow)"
10
+ ]
11
+ },
12
+ {
13
+ "cell_type": "markdown",
14
+ "metadata": {
15
+ "id": "RwRCGCiN1cHk"
16
+ },
17
+ "source": [
18
+ "Install the Transformers, Datasets, and Evaluate libraries to run this notebook."
19
+ ]
20
+ },
21
+ {
22
+ "cell_type": "code",
23
+ "execution_count": 1,
24
+ "metadata": {
25
+ "id": "rHmBWpn31cHq",
26
+ "outputId": "7267872a-b0c3-4a4c-d558-a4ed77f6dc51",
27
+ "colab": {
28
+ "base_uri": "https://localhost:8080/"
29
+ }
30
+ },
31
+ "outputs": [
32
+ {
33
+ "output_type": "stream",
34
+ "name": "stdout",
35
+ "text": [
36
+ "Collecting datasets\n",
37
+ " Downloading datasets-2.14.4-py3-none-any.whl (519 kB)\n",
38
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m519.3/519.3 kB\u001b[0m \u001b[31m7.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
39
+ "\u001b[?25hCollecting evaluate\n",
40
+ " Downloading evaluate-0.4.0-py3-none-any.whl (81 kB)\n",
41
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m81.4/81.4 kB\u001b[0m \u001b[31m8.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
42
+ "\u001b[?25hCollecting transformers[sentencepiece]\n",
43
+ " Downloading transformers-4.32.1-py3-none-any.whl (7.5 MB)\n",
44
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.5/7.5 MB\u001b[0m \u001b[31m63.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
45
+ "\u001b[?25hRequirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from datasets) (1.23.5)\n",
46
+ "Requirement already satisfied: pyarrow>=8.0.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (9.0.0)\n",
47
+ "Collecting dill<0.3.8,>=0.3.0 (from datasets)\n",
48
+ " Downloading dill-0.3.7-py3-none-any.whl (115 kB)\n",
49
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m115.3/115.3 kB\u001b[0m \u001b[31m11.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
50
+ "\u001b[?25hRequirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from datasets) (1.5.3)\n",
51
+ "Requirement already satisfied: requests>=2.19.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (2.31.0)\n",
52
+ "Requirement already satisfied: tqdm>=4.62.1 in /usr/local/lib/python3.10/dist-packages (from datasets) (4.66.1)\n",
53
+ "Collecting xxhash (from datasets)\n",
54
+ " Downloading xxhash-3.3.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (194 kB)\n",
55
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m194.1/194.1 kB\u001b[0m \u001b[31m17.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
56
+ "\u001b[?25hCollecting multiprocess (from datasets)\n",
57
+ " Downloading multiprocess-0.70.15-py310-none-any.whl (134 kB)\n",
58
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m134.8/134.8 kB\u001b[0m \u001b[31m13.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
59
+ "\u001b[?25hRequirement already satisfied: fsspec[http]>=2021.11.1 in /usr/local/lib/python3.10/dist-packages (from datasets) (2023.6.0)\n",
60
+ "Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from datasets) (3.8.5)\n",
61
+ "Collecting huggingface-hub<1.0.0,>=0.14.0 (from datasets)\n",
62
+ " Downloading huggingface_hub-0.16.4-py3-none-any.whl (268 kB)\n",
63
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m268.8/268.8 kB\u001b[0m \u001b[31m23.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
64
+ "\u001b[?25hRequirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from datasets) (23.1)\n",
65
+ "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from datasets) (6.0.1)\n",
66
+ "Collecting responses<0.19 (from evaluate)\n",
67
+ " Downloading responses-0.18.0-py3-none-any.whl (38 kB)\n",
68
+ "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from transformers[sentencepiece]) (3.12.2)\n",
69
+ "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers[sentencepiece]) (2023.6.3)\n",
70
+ "Collecting tokenizers!=0.11.3,<0.14,>=0.11.1 (from transformers[sentencepiece])\n",
71
+ " Downloading tokenizers-0.13.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.8 MB)\n",
72
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.8/7.8 MB\u001b[0m \u001b[31m94.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
73
+ "\u001b[?25hCollecting safetensors>=0.3.1 (from transformers[sentencepiece])\n",
74
+ " Downloading safetensors-0.3.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n",
75
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m58.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
76
+ "\u001b[?25hCollecting sentencepiece!=0.1.92,>=0.1.91 (from transformers[sentencepiece])\n",
77
+ " Downloading sentencepiece-0.1.99-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n",
78
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m50.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
79
+ "\u001b[?25hRequirement already satisfied: protobuf in /usr/local/lib/python3.10/dist-packages (from transformers[sentencepiece]) (3.20.3)\n",
80
+ "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (23.1.0)\n",
81
+ "Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (3.2.0)\n",
82
+ "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (6.0.4)\n",
83
+ "Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (4.0.3)\n",
84
+ "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.9.2)\n",
85
+ "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.4.0)\n",
86
+ "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.3.1)\n",
87
+ "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0.0,>=0.14.0->datasets) (4.7.1)\n",
88
+ "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->datasets) (3.4)\n",
89
+ "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->datasets) (2.0.4)\n",
90
+ "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->datasets) (2023.7.22)\n",
91
+ "Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2.8.2)\n",
92
+ "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2023.3)\n",
93
+ "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.1->pandas->datasets) (1.16.0)\n",
94
+ "Installing collected packages: tokenizers, sentencepiece, safetensors, xxhash, dill, responses, multiprocess, huggingface-hub, transformers, datasets, evaluate\n",
95
+ "Successfully installed datasets-2.14.4 dill-0.3.7 evaluate-0.4.0 huggingface-hub-0.16.4 multiprocess-0.70.15 responses-0.18.0 safetensors-0.3.3 sentencepiece-0.1.99 tokenizers-0.13.3 transformers-4.32.1 xxhash-3.3.0\n"
96
+ ]
97
+ }
98
+ ],
99
+ "source": [
100
+ "!pip install datasets evaluate transformers[sentencepiece]"
101
+ ]
102
+ },
103
+ {
104
+ "cell_type": "code",
105
+ "execution_count": 3,
106
+ "metadata": {
107
+ "id": "HfTOmBru1cHt",
108
+ "outputId": "71ad6e81-7f1e-4054-aa7a-1db157d0e0dc",
109
+ "colab": {
110
+ "base_uri": "https://localhost:8080/"
111
+ }
112
+ },
113
+ "outputs": [
114
+ {
115
+ "output_type": "stream",
116
+ "name": "stdout",
117
+ "text": [
118
+ "['Hi,', 'I', 'am', 'Mahule']\n"
119
+ ]
120
+ }
121
+ ],
122
+ "source": [
123
+ "tokenized_text = \"Hi, I am Mahule\".split()\n",
124
+ "print(tokenized_text)"
125
+ ]
126
+ },
127
+ {
128
+ "cell_type": "code",
129
+ "execution_count": 4,
130
+ "metadata": {
131
+ "id": "keDvrs8f1cHv",
132
+ "outputId": "1a7603e8-499f-45e4-ffb5-1967ca9c00cf",
133
+ "colab": {
134
+ "base_uri": "https://localhost:8080/",
135
+ "height": 113,
136
+ "referenced_widgets": [
137
+ "a146cd54d1094308872b138d8df0c733",
138
+ "e7eb9d0d2d65484aad6b3530c0b27953",
139
+ "aec114fe707e4b54971e1f2e1a06c943",
140
+ "2c25604196fe44fdaf31fc61b18a5b25",
141
+ "afe3bff8730740839f726222bdf01690",
142
+ "845523cd5b42457e82daeb120a63750b",
143
+ "524d0bb4b55f46928f77ed207cc38960",
144
+ "f58b1b2ba59542708154d435c99a129b",
145
+ "006712e0e27a499aa68d470b65c6b3c6",
146
+ "978a6c6b6fcc48b68f71fe5d2e75513b",
147
+ "f6164a00e85a4c6aa9001f48e57e9094",
148
+ "2c263a3d74e246c8bcf5d403977e7065",
149
+ "ac03501d842c4ab298fcf64c9ad6df05",
150
+ "2e7c7374e3e742e6a7e8f096815a6d4d",
151
+ "dc26d02b754544fea290b9eecb47c449",
152
+ "26c39fe80335499f94e3b61943911179",
153
+ "ffe0a16b51ed4c4a893395ec7fd2fc25",
154
+ "8a9514a81a2d4d54a95081298a05b568",
155
+ "08150b2d05354e66b30e9a45a0cfc25d",
156
+ "c61af07c69b7475b978c414b15d227ad",
157
+ "14e9833113644d8ca7f68956f3a809a4",
158
+ "a2f60a6b68c44d6e870fc65081a41eaf",
159
+ "903c714944174a718671541ef8bfee50",
160
+ "a84d7f0917d146e79a940482507332a9",
161
+ "261e1d7d206945568f79d000e8ef9135",
162
+ "25eb5ef0f71944259b99ac80001eae94",
163
+ "3813850b90a74187bf18ec32a3634bf0",
164
+ "968631b9293a4a49959bfce2b166343d",
165
+ "79fded52694644b08614e9b194236fc2",
166
+ "3c0350e52031407eb6e02701305330ff",
167
+ "3cd19f516d894d22bf63659307067597",
168
+ "22ca14fb75a6407a97dea746b5f02281",
169
+ "aa40f63485dc41dca688d63cdb05fedb"
170
+ ]
171
+ }
172
+ },
173
+ "outputs": [
174
+ {
175
+ "output_type": "display_data",
176
+ "data": {
177
+ "text/plain": [
178
+ "Downloading (…)solve/main/vocab.txt: 0%| | 0.00/213k [00:00<?, ?B/s]"
179
+ ],
180
+ "application/vnd.jupyter.widget-view+json": {
181
+ "version_major": 2,
182
+ "version_minor": 0,
183
+ "model_id": "a146cd54d1094308872b138d8df0c733"
184
+ }
185
+ },
186
+ "metadata": {}
187
+ },
188
+ {
189
+ "output_type": "display_data",
190
+ "data": {
191
+ "text/plain": [
192
+ "Downloading (…)okenizer_config.json: 0%| | 0.00/29.0 [00:00<?, ?B/s]"
193
+ ],
194
+ "application/vnd.jupyter.widget-view+json": {
195
+ "version_major": 2,
196
+ "version_minor": 0,
197
+ "model_id": "2c263a3d74e246c8bcf5d403977e7065"
198
+ }
199
+ },
200
+ "metadata": {}
201
+ },
202
+ {
203
+ "output_type": "display_data",
204
+ "data": {
205
+ "text/plain": [
206
+ "Downloading (…)lve/main/config.json: 0%| | 0.00/570 [00:00<?, ?B/s]"
207
+ ],
208
+ "application/vnd.jupyter.widget-view+json": {
209
+ "version_major": 2,
210
+ "version_minor": 0,
211
+ "model_id": "903c714944174a718671541ef8bfee50"
212
+ }
213
+ },
214
+ "metadata": {}
215
+ }
216
+ ],
217
+ "source": [
218
+ "from transformers import BertTokenizer\n",
219
+ "\n",
220
+ "tokenizer = BertTokenizer.from_pretrained(\"bert-base-cased\")"
221
+ ]
222
+ },
223
+ {
224
+ "cell_type": "code",
225
+ "execution_count": 5,
226
+ "metadata": {
227
+ "id": "Iu3PSCgF1cHx",
228
+ "outputId": "636f1ae2-19c2-4300-a37d-62bb43e4103a",
229
+ "colab": {
230
+ "base_uri": "https://localhost:8080/",
231
+ "height": 49,
232
+ "referenced_widgets": [
233
+ "b4d293a21ebb427d961059feacb5a876",
234
+ "d0dfa9da76d34f8dae3bb2b751d2816d",
235
+ "50ee3e7cc36b41699fff0e0b2deca480",
236
+ "2015d00ec93c4bd5b6ab0870e2f3283d",
237
+ "49300220b8de4a90a25785585a9bab28",
238
+ "b6d0476d32b945d6a0aaf0275035444f",
239
+ "5c36a1ce72904291afcb56101ec45415",
240
+ "dbaa1990251e48489ea6d647e4f20b5d",
241
+ "169fd06eaf074fa5ac4bfb2100bfa598",
242
+ "6b7a2727b9164145824742da5daf5ba4",
243
+ "dfc2bb905bd74ad1ad91614be4b9734a"
244
+ ]
245
+ }
246
+ },
247
+ "outputs": [
248
+ {
249
+ "output_type": "display_data",
250
+ "data": {
251
+ "text/plain": [
252
+ "Downloading (…)/main/tokenizer.json: 0%| | 0.00/436k [00:00<?, ?B/s]"
253
+ ],
254
+ "application/vnd.jupyter.widget-view+json": {
255
+ "version_major": 2,
256
+ "version_minor": 0,
257
+ "model_id": "b4d293a21ebb427d961059feacb5a876"
258
+ }
259
+ },
260
+ "metadata": {}
261
+ }
262
+ ],
263
+ "source": [
264
+ "from transformers import AutoTokenizer\n",
265
+ "\n",
266
+ "tokenizer = AutoTokenizer.from_pretrained(\"bert-base-cased\")"
267
+ ]
268
+ },
269
+ {
270
+ "cell_type": "code",
271
+ "execution_count": 7,
272
+ "metadata": {
273
+ "id": "08HgWshs1cH0",
274
+ "outputId": "142addb8-adc1-4f89-a98a-266a044a12f0",
275
+ "colab": {
276
+ "base_uri": "https://localhost:8080/"
277
+ }
278
+ },
279
+ "outputs": [
280
+ {
281
+ "output_type": "execute_result",
282
+ "data": {
283
+ "text/plain": [
284
+ "{'input_ids': [101, 7085, 24287, 1162, 1110, 3776, 1293, 1106, 1329, 170, 13809, 23763, 2443, 102], 'token_type_ids': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}"
285
+ ]
286
+ },
287
+ "metadata": {},
288
+ "execution_count": 7
289
+ }
290
+ ],
291
+ "source": [
292
+ "tokenizer(\"Mahule is learning how to use a Transformer network\")"
293
+ ]
294
+ },
295
+ {
296
+ "cell_type": "code",
297
+ "execution_count": 8,
298
+ "metadata": {
299
+ "id": "RNFGbCYu1cH1",
300
+ "outputId": "5e0a93a8-5219-4acd-c7b7-25e3aa810eac",
301
+ "colab": {
302
+ "base_uri": "https://localhost:8080/"
303
+ }
304
+ },
305
+ "outputs": [
306
+ {
307
+ "output_type": "execute_result",
308
+ "data": {
309
+ "text/plain": [
310
+ "('directory_on_my_computer/tokenizer_config.json',\n",
311
+ " 'directory_on_my_computer/special_tokens_map.json',\n",
312
+ " 'directory_on_my_computer/vocab.txt',\n",
313
+ " 'directory_on_my_computer/added_tokens.json',\n",
314
+ " 'directory_on_my_computer/tokenizer.json')"
315
+ ]
316
+ },
317
+ "metadata": {},
318
+ "execution_count": 8
319
+ }
320
+ ],
321
+ "source": [
322
+ "tokenizer.save_pretrained(\"directory_on_my_computer\")"
323
+ ]
324
+ },
325
+ {
326
+ "cell_type": "code",
327
+ "execution_count": 14,
328
+ "metadata": {
329
+ "id": "xX5ab0VS1cH3",
330
+ "outputId": "530c9567-dfc9-4130-cc43-c1e638e87dbb",
331
+ "colab": {
332
+ "base_uri": "https://localhost:8080/"
333
+ }
334
+ },
335
+ "outputs": [
336
+ {
337
+ "output_type": "stream",
338
+ "name": "stdout",
339
+ "text": [
340
+ "['Ma', '##hul', '##e', 'is', 'learning', 'how', 'to', 'use', 'a', 'Trans', '##former', 'network']\n"
341
+ ]
342
+ }
343
+ ],
344
+ "source": [
345
+ "from transformers import AutoTokenizer\n",
346
+ "\n",
347
+ "tokenizer = AutoTokenizer.from_pretrained(\"bert-base-cased\")\n",
348
+ "\n",
349
+ "sequence = \"Mahule is learning how to use a Transformer network\"\n",
350
+ "tokens = tokenizer.tokenize(sequence)\n",
351
+ "\n",
352
+ "print(tokens)"
353
+ ]
354
+ },
355
+ {
356
+ "cell_type": "code",
357
+ "execution_count": 15,
358
+ "metadata": {
359
+ "id": "f72fxPfv1cH6",
360
+ "outputId": "f2e2188f-5663-4cd8-b7a8-a420cbd0caee",
361
+ "colab": {
362
+ "base_uri": "https://localhost:8080/"
363
+ }
364
+ },
365
+ "outputs": [
366
+ {
367
+ "output_type": "stream",
368
+ "name": "stdout",
369
+ "text": [
370
+ "[7085, 24287, 1162, 1110, 3776, 1293, 1106, 1329, 170, 13809, 23763, 2443]\n"
371
+ ]
372
+ }
373
+ ],
374
+ "source": [
375
+ "ids = tokenizer.convert_tokens_to_ids(tokens)\n",
376
+ "\n",
377
+ "print(ids)"
378
+ ]
379
+ },
380
+ {
381
+ "cell_type": "code",
382
+ "execution_count": 17,
383
+ "metadata": {
384
+ "id": "b9yA27T71cH7",
385
+ "outputId": "18c34eaf-2e7a-4fb4-ae05-c2c182a1d683",
386
+ "colab": {
387
+ "base_uri": "https://localhost:8080/"
388
+ }
389
+ },
390
+ "outputs": [
391
+ {
392
+ "output_type": "stream",
393
+ "name": "stdout",
394
+ "text": [
395
+ "Mahule is learning how to use a Transformer network\n"
396
+ ]
397
+ }
398
+ ],
399
+ "source": [
400
+ "decoded_string = tokenizer.decode([7085, 24287, 1162, 1110, 3776, 1293, 1106, 1329, 170, 13809, 23763, 2443])\n",
401
+ "print(decoded_string)"
402
+ ]
403
+ }
404
+ ],
405
+ "metadata": {
406
+ "colab": {
407
+ "name": "Tokenizers (TensorFlow)",
408
+ "provenance": []
409
+ },
410
+ "language_info": {
411
+ "name": "python"
412
+ },
413
+ "kernelspec": {
414
+ "name": "python3",
415
+ "display_name": "Python 3"
416
+ },
417
+ "widgets": {
418
+ "application/vnd.jupyter.widget-state+json": {
419
+ "a146cd54d1094308872b138d8df0c733": {
420
+ "model_module": "@jupyter-widgets/controls",
421
+ "model_name": "HBoxModel",
422
+ "model_module_version": "1.5.0",
423
+ "state": {
424
+ "_dom_classes": [],
425
+ "_model_module": "@jupyter-widgets/controls",
426
+ "_model_module_version": "1.5.0",
427
+ "_model_name": "HBoxModel",
428
+ "_view_count": null,
429
+ "_view_module": "@jupyter-widgets/controls",
430
+ "_view_module_version": "1.5.0",
431
+ "_view_name": "HBoxView",
432
+ "box_style": "",
433
+ "children": [
434
+ "IPY_MODEL_e7eb9d0d2d65484aad6b3530c0b27953",
435
+ "IPY_MODEL_aec114fe707e4b54971e1f2e1a06c943",
436
+ "IPY_MODEL_2c25604196fe44fdaf31fc61b18a5b25"
437
+ ],
438
+ "layout": "IPY_MODEL_afe3bff8730740839f726222bdf01690"
439
+ }
440
+ },
441
+ "e7eb9d0d2d65484aad6b3530c0b27953": {
442
+ "model_module": "@jupyter-widgets/controls",
443
+ "model_name": "HTMLModel",
444
+ "model_module_version": "1.5.0",
445
+ "state": {
446
+ "_dom_classes": [],
447
+ "_model_module": "@jupyter-widgets/controls",
448
+ "_model_module_version": "1.5.0",
449
+ "_model_name": "HTMLModel",
450
+ "_view_count": null,
451
+ "_view_module": "@jupyter-widgets/controls",
452
+ "_view_module_version": "1.5.0",
453
+ "_view_name": "HTMLView",
454
+ "description": "",
455
+ "description_tooltip": null,
456
+ "layout": "IPY_MODEL_845523cd5b42457e82daeb120a63750b",
457
+ "placeholder": "​",
458
+ "style": "IPY_MODEL_524d0bb4b55f46928f77ed207cc38960",
459
+ "value": "Downloading (…)solve/main/vocab.txt: 100%"
460
+ }
461
+ },
462
+ "aec114fe707e4b54971e1f2e1a06c943": {
463
+ "model_module": "@jupyter-widgets/controls",
464
+ "model_name": "FloatProgressModel",
465
+ "model_module_version": "1.5.0",
466
+ "state": {
467
+ "_dom_classes": [],
468
+ "_model_module": "@jupyter-widgets/controls",
469
+ "_model_module_version": "1.5.0",
470
+ "_model_name": "FloatProgressModel",
471
+ "_view_count": null,
472
+ "_view_module": "@jupyter-widgets/controls",
473
+ "_view_module_version": "1.5.0",
474
+ "_view_name": "ProgressView",
475
+ "bar_style": "success",
476
+ "description": "",
477
+ "description_tooltip": null,
478
+ "layout": "IPY_MODEL_f58b1b2ba59542708154d435c99a129b",
479
+ "max": 213450,
480
+ "min": 0,
481
+ "orientation": "horizontal",
482
+ "style": "IPY_MODEL_006712e0e27a499aa68d470b65c6b3c6",
483
+ "value": 213450
484
+ }
485
+ },
486
+ "2c25604196fe44fdaf31fc61b18a5b25": {
487
+ "model_module": "@jupyter-widgets/controls",
488
+ "model_name": "HTMLModel",
489
+ "model_module_version": "1.5.0",
490
+ "state": {
491
+ "_dom_classes": [],
492
+ "_model_module": "@jupyter-widgets/controls",
493
+ "_model_module_version": "1.5.0",
494
+ "_model_name": "HTMLModel",
495
+ "_view_count": null,
496
+ "_view_module": "@jupyter-widgets/controls",
497
+ "_view_module_version": "1.5.0",
498
+ "_view_name": "HTMLView",
499
+ "description": "",
500
+ "description_tooltip": null,
501
+ "layout": "IPY_MODEL_978a6c6b6fcc48b68f71fe5d2e75513b",
502
+ "placeholder": "​",
503
+ "style": "IPY_MODEL_f6164a00e85a4c6aa9001f48e57e9094",
504
+ "value": " 213k/213k [00:00&lt;00:00, 1.66MB/s]"
505
+ }
506
+ },
507
+ "afe3bff8730740839f726222bdf01690": {
508
+ "model_module": "@jupyter-widgets/base",
509
+ "model_name": "LayoutModel",
510
+ "model_module_version": "1.2.0",
511
+ "state": {
512
+ "_model_module": "@jupyter-widgets/base",
513
+ "_model_module_version": "1.2.0",
514
+ "_model_name": "LayoutModel",
515
+ "_view_count": null,
516
+ "_view_module": "@jupyter-widgets/base",
517
+ "_view_module_version": "1.2.0",
518
+ "_view_name": "LayoutView",
519
+ "align_content": null,
520
+ "align_items": null,
521
+ "align_self": null,
522
+ "border": null,
523
+ "bottom": null,
524
+ "display": null,
525
+ "flex": null,
526
+ "flex_flow": null,
527
+ "grid_area": null,
528
+ "grid_auto_columns": null,
529
+ "grid_auto_flow": null,
530
+ "grid_auto_rows": null,
531
+ "grid_column": null,
532
+ "grid_gap": null,
533
+ "grid_row": null,
534
+ "grid_template_areas": null,
535
+ "grid_template_columns": null,
536
+ "grid_template_rows": null,
537
+ "height": null,
538
+ "justify_content": null,
539
+ "justify_items": null,
540
+ "left": null,
541
+ "margin": null,
542
+ "max_height": null,
543
+ "max_width": null,
544
+ "min_height": null,
545
+ "min_width": null,
546
+ "object_fit": null,
547
+ "object_position": null,
548
+ "order": null,
549
+ "overflow": null,
550
+ "overflow_x": null,
551
+ "overflow_y": null,
552
+ "padding": null,
553
+ "right": null,
554
+ "top": null,
555
+ "visibility": null,
556
+ "width": null
557
+ }
558
+ },
559
+ "845523cd5b42457e82daeb120a63750b": {
560
+ "model_module": "@jupyter-widgets/base",
561
+ "model_name": "LayoutModel",
562
+ "model_module_version": "1.2.0",
563
+ "state": {
564
+ "_model_module": "@jupyter-widgets/base",
565
+ "_model_module_version": "1.2.0",
566
+ "_model_name": "LayoutModel",
567
+ "_view_count": null,
568
+ "_view_module": "@jupyter-widgets/base",
569
+ "_view_module_version": "1.2.0",
570
+ "_view_name": "LayoutView",
571
+ "align_content": null,
572
+ "align_items": null,
573
+ "align_self": null,
574
+ "border": null,
575
+ "bottom": null,
576
+ "display": null,
577
+ "flex": null,
578
+ "flex_flow": null,
579
+ "grid_area": null,
580
+ "grid_auto_columns": null,
581
+ "grid_auto_flow": null,
582
+ "grid_auto_rows": null,
583
+ "grid_column": null,
584
+ "grid_gap": null,
585
+ "grid_row": null,
586
+ "grid_template_areas": null,
587
+ "grid_template_columns": null,
588
+ "grid_template_rows": null,
589
+ "height": null,
590
+ "justify_content": null,
591
+ "justify_items": null,
592
+ "left": null,
593
+ "margin": null,
594
+ "max_height": null,
595
+ "max_width": null,
596
+ "min_height": null,
597
+ "min_width": null,
598
+ "object_fit": null,
599
+ "object_position": null,
600
+ "order": null,
601
+ "overflow": null,
602
+ "overflow_x": null,
603
+ "overflow_y": null,
604
+ "padding": null,
605
+ "right": null,
606
+ "top": null,
607
+ "visibility": null,
608
+ "width": null
609
+ }
610
+ },
611
+ "524d0bb4b55f46928f77ed207cc38960": {
612
+ "model_module": "@jupyter-widgets/controls",
613
+ "model_name": "DescriptionStyleModel",
614
+ "model_module_version": "1.5.0",
615
+ "state": {
616
+ "_model_module": "@jupyter-widgets/controls",
617
+ "_model_module_version": "1.5.0",
618
+ "_model_name": "DescriptionStyleModel",
619
+ "_view_count": null,
620
+ "_view_module": "@jupyter-widgets/base",
621
+ "_view_module_version": "1.2.0",
622
+ "_view_name": "StyleView",
623
+ "description_width": ""
624
+ }
625
+ },
626
+ "f58b1b2ba59542708154d435c99a129b": {
627
+ "model_module": "@jupyter-widgets/base",
628
+ "model_name": "LayoutModel",
629
+ "model_module_version": "1.2.0",
630
+ "state": {
631
+ "_model_module": "@jupyter-widgets/base",
632
+ "_model_module_version": "1.2.0",
633
+ "_model_name": "LayoutModel",
634
+ "_view_count": null,
635
+ "_view_module": "@jupyter-widgets/base",
636
+ "_view_module_version": "1.2.0",
637
+ "_view_name": "LayoutView",
638
+ "align_content": null,
639
+ "align_items": null,
640
+ "align_self": null,
641
+ "border": null,
642
+ "bottom": null,
643
+ "display": null,
644
+ "flex": null,
645
+ "flex_flow": null,
646
+ "grid_area": null,
647
+ "grid_auto_columns": null,
648
+ "grid_auto_flow": null,
649
+ "grid_auto_rows": null,
650
+ "grid_column": null,
651
+ "grid_gap": null,
652
+ "grid_row": null,
653
+ "grid_template_areas": null,
654
+ "grid_template_columns": null,
655
+ "grid_template_rows": null,
656
+ "height": null,
657
+ "justify_content": null,
658
+ "justify_items": null,
659
+ "left": null,
660
+ "margin": null,
661
+ "max_height": null,
662
+ "max_width": null,
663
+ "min_height": null,
664
+ "min_width": null,
665
+ "object_fit": null,
666
+ "object_position": null,
667
+ "order": null,
668
+ "overflow": null,
669
+ "overflow_x": null,
670
+ "overflow_y": null,
671
+ "padding": null,
672
+ "right": null,
673
+ "top": null,
674
+ "visibility": null,
675
+ "width": null
676
+ }
677
+ },
678
+ "006712e0e27a499aa68d470b65c6b3c6": {
679
+ "model_module": "@jupyter-widgets/controls",
680
+ "model_name": "ProgressStyleModel",
681
+ "model_module_version": "1.5.0",
682
+ "state": {
683
+ "_model_module": "@jupyter-widgets/controls",
684
+ "_model_module_version": "1.5.0",
685
+ "_model_name": "ProgressStyleModel",
686
+ "_view_count": null,
687
+ "_view_module": "@jupyter-widgets/base",
688
+ "_view_module_version": "1.2.0",
689
+ "_view_name": "StyleView",
690
+ "bar_color": null,
691
+ "description_width": ""
692
+ }
693
+ },
694
+ "978a6c6b6fcc48b68f71fe5d2e75513b": {
695
+ "model_module": "@jupyter-widgets/base",
696
+ "model_name": "LayoutModel",
697
+ "model_module_version": "1.2.0",
698
+ "state": {
699
+ "_model_module": "@jupyter-widgets/base",
700
+ "_model_module_version": "1.2.0",
701
+ "_model_name": "LayoutModel",
702
+ "_view_count": null,
703
+ "_view_module": "@jupyter-widgets/base",
704
+ "_view_module_version": "1.2.0",
705
+ "_view_name": "LayoutView",
706
+ "align_content": null,
707
+ "align_items": null,
708
+ "align_self": null,
709
+ "border": null,
710
+ "bottom": null,
711
+ "display": null,
712
+ "flex": null,
713
+ "flex_flow": null,
714
+ "grid_area": null,
715
+ "grid_auto_columns": null,
716
+ "grid_auto_flow": null,
717
+ "grid_auto_rows": null,
718
+ "grid_column": null,
719
+ "grid_gap": null,
720
+ "grid_row": null,
721
+ "grid_template_areas": null,
722
+ "grid_template_columns": null,
723
+ "grid_template_rows": null,
724
+ "height": null,
725
+ "justify_content": null,
726
+ "justify_items": null,
727
+ "left": null,
728
+ "margin": null,
729
+ "max_height": null,
730
+ "max_width": null,
731
+ "min_height": null,
732
+ "min_width": null,
733
+ "object_fit": null,
734
+ "object_position": null,
735
+ "order": null,
736
+ "overflow": null,
737
+ "overflow_x": null,
738
+ "overflow_y": null,
739
+ "padding": null,
740
+ "right": null,
741
+ "top": null,
742
+ "visibility": null,
743
+ "width": null
744
+ }
745
+ },
746
+ "f6164a00e85a4c6aa9001f48e57e9094": {
747
+ "model_module": "@jupyter-widgets/controls",
748
+ "model_name": "DescriptionStyleModel",
749
+ "model_module_version": "1.5.0",
750
+ "state": {
751
+ "_model_module": "@jupyter-widgets/controls",
752
+ "_model_module_version": "1.5.0",
753
+ "_model_name": "DescriptionStyleModel",
754
+ "_view_count": null,
755
+ "_view_module": "@jupyter-widgets/base",
756
+ "_view_module_version": "1.2.0",
757
+ "_view_name": "StyleView",
758
+ "description_width": ""
759
+ }
760
+ },
761
+ "2c263a3d74e246c8bcf5d403977e7065": {
762
+ "model_module": "@jupyter-widgets/controls",
763
+ "model_name": "HBoxModel",
764
+ "model_module_version": "1.5.0",
765
+ "state": {
766
+ "_dom_classes": [],
767
+ "_model_module": "@jupyter-widgets/controls",
768
+ "_model_module_version": "1.5.0",
769
+ "_model_name": "HBoxModel",
770
+ "_view_count": null,
771
+ "_view_module": "@jupyter-widgets/controls",
772
+ "_view_module_version": "1.5.0",
773
+ "_view_name": "HBoxView",
774
+ "box_style": "",
775
+ "children": [
776
+ "IPY_MODEL_ac03501d842c4ab298fcf64c9ad6df05",
777
+ "IPY_MODEL_2e7c7374e3e742e6a7e8f096815a6d4d",
778
+ "IPY_MODEL_dc26d02b754544fea290b9eecb47c449"
779
+ ],
780
+ "layout": "IPY_MODEL_26c39fe80335499f94e3b61943911179"
781
+ }
782
+ },
783
+ "ac03501d842c4ab298fcf64c9ad6df05": {
784
+ "model_module": "@jupyter-widgets/controls",
785
+ "model_name": "HTMLModel",
786
+ "model_module_version": "1.5.0",
787
+ "state": {
788
+ "_dom_classes": [],
789
+ "_model_module": "@jupyter-widgets/controls",
790
+ "_model_module_version": "1.5.0",
791
+ "_model_name": "HTMLModel",
792
+ "_view_count": null,
793
+ "_view_module": "@jupyter-widgets/controls",
794
+ "_view_module_version": "1.5.0",
795
+ "_view_name": "HTMLView",
796
+ "description": "",
797
+ "description_tooltip": null,
798
+ "layout": "IPY_MODEL_ffe0a16b51ed4c4a893395ec7fd2fc25",
799
+ "placeholder": "​",
800
+ "style": "IPY_MODEL_8a9514a81a2d4d54a95081298a05b568",
801
+ "value": "Downloading (…)okenizer_config.json: 100%"
802
+ }
803
+ },
804
+ "2e7c7374e3e742e6a7e8f096815a6d4d": {
805
+ "model_module": "@jupyter-widgets/controls",
806
+ "model_name": "FloatProgressModel",
807
+ "model_module_version": "1.5.0",
808
+ "state": {
809
+ "_dom_classes": [],
810
+ "_model_module": "@jupyter-widgets/controls",
811
+ "_model_module_version": "1.5.0",
812
+ "_model_name": "FloatProgressModel",
813
+ "_view_count": null,
814
+ "_view_module": "@jupyter-widgets/controls",
815
+ "_view_module_version": "1.5.0",
816
+ "_view_name": "ProgressView",
817
+ "bar_style": "success",
818
+ "description": "",
819
+ "description_tooltip": null,
820
+ "layout": "IPY_MODEL_08150b2d05354e66b30e9a45a0cfc25d",
821
+ "max": 29,
822
+ "min": 0,
823
+ "orientation": "horizontal",
824
+ "style": "IPY_MODEL_c61af07c69b7475b978c414b15d227ad",
825
+ "value": 29
826
+ }
827
+ },
828
+ "dc26d02b754544fea290b9eecb47c449": {
829
+ "model_module": "@jupyter-widgets/controls",
830
+ "model_name": "HTMLModel",
831
+ "model_module_version": "1.5.0",
832
+ "state": {
833
+ "_dom_classes": [],
834
+ "_model_module": "@jupyter-widgets/controls",
835
+ "_model_module_version": "1.5.0",
836
+ "_model_name": "HTMLModel",
837
+ "_view_count": null,
838
+ "_view_module": "@jupyter-widgets/controls",
839
+ "_view_module_version": "1.5.0",
840
+ "_view_name": "HTMLView",
841
+ "description": "",
842
+ "description_tooltip": null,
843
+ "layout": "IPY_MODEL_14e9833113644d8ca7f68956f3a809a4",
844
+ "placeholder": "​",
845
+ "style": "IPY_MODEL_a2f60a6b68c44d6e870fc65081a41eaf",
846
+ "value": " 29.0/29.0 [00:00&lt;00:00, 651B/s]"
847
+ }
848
+ },
849
+ "26c39fe80335499f94e3b61943911179": {
850
+ "model_module": "@jupyter-widgets/base",
851
+ "model_name": "LayoutModel",
852
+ "model_module_version": "1.2.0",
853
+ "state": {
854
+ "_model_module": "@jupyter-widgets/base",
855
+ "_model_module_version": "1.2.0",
856
+ "_model_name": "LayoutModel",
857
+ "_view_count": null,
858
+ "_view_module": "@jupyter-widgets/base",
859
+ "_view_module_version": "1.2.0",
860
+ "_view_name": "LayoutView",
861
+ "align_content": null,
862
+ "align_items": null,
863
+ "align_self": null,
864
+ "border": null,
865
+ "bottom": null,
866
+ "display": null,
867
+ "flex": null,
868
+ "flex_flow": null,
869
+ "grid_area": null,
870
+ "grid_auto_columns": null,
871
+ "grid_auto_flow": null,
872
+ "grid_auto_rows": null,
873
+ "grid_column": null,
874
+ "grid_gap": null,
875
+ "grid_row": null,
876
+ "grid_template_areas": null,
877
+ "grid_template_columns": null,
878
+ "grid_template_rows": null,
879
+ "height": null,
880
+ "justify_content": null,
881
+ "justify_items": null,
882
+ "left": null,
883
+ "margin": null,
884
+ "max_height": null,
885
+ "max_width": null,
886
+ "min_height": null,
887
+ "min_width": null,
888
+ "object_fit": null,
889
+ "object_position": null,
890
+ "order": null,
891
+ "overflow": null,
892
+ "overflow_x": null,
893
+ "overflow_y": null,
894
+ "padding": null,
895
+ "right": null,
896
+ "top": null,
897
+ "visibility": null,
898
+ "width": null
899
+ }
900
+ },
901
+ "ffe0a16b51ed4c4a893395ec7fd2fc25": {
902
+ "model_module": "@jupyter-widgets/base",
903
+ "model_name": "LayoutModel",
904
+ "model_module_version": "1.2.0",
905
+ "state": {
906
+ "_model_module": "@jupyter-widgets/base",
907
+ "_model_module_version": "1.2.0",
908
+ "_model_name": "LayoutModel",
909
+ "_view_count": null,
910
+ "_view_module": "@jupyter-widgets/base",
911
+ "_view_module_version": "1.2.0",
912
+ "_view_name": "LayoutView",
913
+ "align_content": null,
914
+ "align_items": null,
915
+ "align_self": null,
916
+ "border": null,
917
+ "bottom": null,
918
+ "display": null,
919
+ "flex": null,
920
+ "flex_flow": null,
921
+ "grid_area": null,
922
+ "grid_auto_columns": null,
923
+ "grid_auto_flow": null,
924
+ "grid_auto_rows": null,
925
+ "grid_column": null,
926
+ "grid_gap": null,
927
+ "grid_row": null,
928
+ "grid_template_areas": null,
929
+ "grid_template_columns": null,
930
+ "grid_template_rows": null,
931
+ "height": null,
932
+ "justify_content": null,
933
+ "justify_items": null,
934
+ "left": null,
935
+ "margin": null,
936
+ "max_height": null,
937
+ "max_width": null,
938
+ "min_height": null,
939
+ "min_width": null,
940
+ "object_fit": null,
941
+ "object_position": null,
942
+ "order": null,
943
+ "overflow": null,
944
+ "overflow_x": null,
945
+ "overflow_y": null,
946
+ "padding": null,
947
+ "right": null,
948
+ "top": null,
949
+ "visibility": null,
950
+ "width": null
951
+ }
952
+ },
953
+ "8a9514a81a2d4d54a95081298a05b568": {
954
+ "model_module": "@jupyter-widgets/controls",
955
+ "model_name": "DescriptionStyleModel",
956
+ "model_module_version": "1.5.0",
957
+ "state": {
958
+ "_model_module": "@jupyter-widgets/controls",
959
+ "_model_module_version": "1.5.0",
960
+ "_model_name": "DescriptionStyleModel",
961
+ "_view_count": null,
962
+ "_view_module": "@jupyter-widgets/base",
963
+ "_view_module_version": "1.2.0",
964
+ "_view_name": "StyleView",
965
+ "description_width": ""
966
+ }
967
+ },
968
+ "08150b2d05354e66b30e9a45a0cfc25d": {
969
+ "model_module": "@jupyter-widgets/base",
970
+ "model_name": "LayoutModel",
971
+ "model_module_version": "1.2.0",
972
+ "state": {
973
+ "_model_module": "@jupyter-widgets/base",
974
+ "_model_module_version": "1.2.0",
975
+ "_model_name": "LayoutModel",
976
+ "_view_count": null,
977
+ "_view_module": "@jupyter-widgets/base",
978
+ "_view_module_version": "1.2.0",
979
+ "_view_name": "LayoutView",
980
+ "align_content": null,
981
+ "align_items": null,
982
+ "align_self": null,
983
+ "border": null,
984
+ "bottom": null,
985
+ "display": null,
986
+ "flex": null,
987
+ "flex_flow": null,
988
+ "grid_area": null,
989
+ "grid_auto_columns": null,
990
+ "grid_auto_flow": null,
991
+ "grid_auto_rows": null,
992
+ "grid_column": null,
993
+ "grid_gap": null,
994
+ "grid_row": null,
995
+ "grid_template_areas": null,
996
+ "grid_template_columns": null,
997
+ "grid_template_rows": null,
998
+ "height": null,
999
+ "justify_content": null,
1000
+ "justify_items": null,
1001
+ "left": null,
1002
+ "margin": null,
1003
+ "max_height": null,
1004
+ "max_width": null,
1005
+ "min_height": null,
1006
+ "min_width": null,
1007
+ "object_fit": null,
1008
+ "object_position": null,
1009
+ "order": null,
1010
+ "overflow": null,
1011
+ "overflow_x": null,
1012
+ "overflow_y": null,
1013
+ "padding": null,
1014
+ "right": null,
1015
+ "top": null,
1016
+ "visibility": null,
1017
+ "width": null
1018
+ }
1019
+ },
1020
+ "c61af07c69b7475b978c414b15d227ad": {
1021
+ "model_module": "@jupyter-widgets/controls",
1022
+ "model_name": "ProgressStyleModel",
1023
+ "model_module_version": "1.5.0",
1024
+ "state": {
1025
+ "_model_module": "@jupyter-widgets/controls",
1026
+ "_model_module_version": "1.5.0",
1027
+ "_model_name": "ProgressStyleModel",
1028
+ "_view_count": null,
1029
+ "_view_module": "@jupyter-widgets/base",
1030
+ "_view_module_version": "1.2.0",
1031
+ "_view_name": "StyleView",
1032
+ "bar_color": null,
1033
+ "description_width": ""
1034
+ }
1035
+ },
1036
+ "14e9833113644d8ca7f68956f3a809a4": {
1037
+ "model_module": "@jupyter-widgets/base",
1038
+ "model_name": "LayoutModel",
1039
+ "model_module_version": "1.2.0",
1040
+ "state": {
1041
+ "_model_module": "@jupyter-widgets/base",
1042
+ "_model_module_version": "1.2.0",
1043
+ "_model_name": "LayoutModel",
1044
+ "_view_count": null,
1045
+ "_view_module": "@jupyter-widgets/base",
1046
+ "_view_module_version": "1.2.0",
1047
+ "_view_name": "LayoutView",
1048
+ "align_content": null,
1049
+ "align_items": null,
1050
+ "align_self": null,
1051
+ "border": null,
1052
+ "bottom": null,
1053
+ "display": null,
1054
+ "flex": null,
1055
+ "flex_flow": null,
1056
+ "grid_area": null,
1057
+ "grid_auto_columns": null,
1058
+ "grid_auto_flow": null,
1059
+ "grid_auto_rows": null,
1060
+ "grid_column": null,
1061
+ "grid_gap": null,
1062
+ "grid_row": null,
1063
+ "grid_template_areas": null,
1064
+ "grid_template_columns": null,
1065
+ "grid_template_rows": null,
1066
+ "height": null,
1067
+ "justify_content": null,
1068
+ "justify_items": null,
1069
+ "left": null,
1070
+ "margin": null,
1071
+ "max_height": null,
1072
+ "max_width": null,
1073
+ "min_height": null,
1074
+ "min_width": null,
1075
+ "object_fit": null,
1076
+ "object_position": null,
1077
+ "order": null,
1078
+ "overflow": null,
1079
+ "overflow_x": null,
1080
+ "overflow_y": null,
1081
+ "padding": null,
1082
+ "right": null,
1083
+ "top": null,
1084
+ "visibility": null,
1085
+ "width": null
1086
+ }
1087
+ },
1088
+ "a2f60a6b68c44d6e870fc65081a41eaf": {
1089
+ "model_module": "@jupyter-widgets/controls",
1090
+ "model_name": "DescriptionStyleModel",
1091
+ "model_module_version": "1.5.0",
1092
+ "state": {
1093
+ "_model_module": "@jupyter-widgets/controls",
1094
+ "_model_module_version": "1.5.0",
1095
+ "_model_name": "DescriptionStyleModel",
1096
+ "_view_count": null,
1097
+ "_view_module": "@jupyter-widgets/base",
1098
+ "_view_module_version": "1.2.0",
1099
+ "_view_name": "StyleView",
1100
+ "description_width": ""
1101
+ }
1102
+ },
1103
+ "903c714944174a718671541ef8bfee50": {
1104
+ "model_module": "@jupyter-widgets/controls",
1105
+ "model_name": "HBoxModel",
1106
+ "model_module_version": "1.5.0",
1107
+ "state": {
1108
+ "_dom_classes": [],
1109
+ "_model_module": "@jupyter-widgets/controls",
1110
+ "_model_module_version": "1.5.0",
1111
+ "_model_name": "HBoxModel",
1112
+ "_view_count": null,
1113
+ "_view_module": "@jupyter-widgets/controls",
1114
+ "_view_module_version": "1.5.0",
1115
+ "_view_name": "HBoxView",
1116
+ "box_style": "",
1117
+ "children": [
1118
+ "IPY_MODEL_a84d7f0917d146e79a940482507332a9",
1119
+ "IPY_MODEL_261e1d7d206945568f79d000e8ef9135",
1120
+ "IPY_MODEL_25eb5ef0f71944259b99ac80001eae94"
1121
+ ],
1122
+ "layout": "IPY_MODEL_3813850b90a74187bf18ec32a3634bf0"
1123
+ }
1124
+ },
1125
+ "a84d7f0917d146e79a940482507332a9": {
1126
+ "model_module": "@jupyter-widgets/controls",
1127
+ "model_name": "HTMLModel",
1128
+ "model_module_version": "1.5.0",
1129
+ "state": {
1130
+ "_dom_classes": [],
1131
+ "_model_module": "@jupyter-widgets/controls",
1132
+ "_model_module_version": "1.5.0",
1133
+ "_model_name": "HTMLModel",
1134
+ "_view_count": null,
1135
+ "_view_module": "@jupyter-widgets/controls",
1136
+ "_view_module_version": "1.5.0",
1137
+ "_view_name": "HTMLView",
1138
+ "description": "",
1139
+ "description_tooltip": null,
1140
+ "layout": "IPY_MODEL_968631b9293a4a49959bfce2b166343d",
1141
+ "placeholder": "​",
1142
+ "style": "IPY_MODEL_79fded52694644b08614e9b194236fc2",
1143
+ "value": "Downloading (…)lve/main/config.json: 100%"
1144
+ }
1145
+ },
1146
+ "261e1d7d206945568f79d000e8ef9135": {
1147
+ "model_module": "@jupyter-widgets/controls",
1148
+ "model_name": "FloatProgressModel",
1149
+ "model_module_version": "1.5.0",
1150
+ "state": {
1151
+ "_dom_classes": [],
1152
+ "_model_module": "@jupyter-widgets/controls",
1153
+ "_model_module_version": "1.5.0",
1154
+ "_model_name": "FloatProgressModel",
1155
+ "_view_count": null,
1156
+ "_view_module": "@jupyter-widgets/controls",
1157
+ "_view_module_version": "1.5.0",
1158
+ "_view_name": "ProgressView",
1159
+ "bar_style": "success",
1160
+ "description": "",
1161
+ "description_tooltip": null,
1162
+ "layout": "IPY_MODEL_3c0350e52031407eb6e02701305330ff",
1163
+ "max": 570,
1164
+ "min": 0,
1165
+ "orientation": "horizontal",
1166
+ "style": "IPY_MODEL_3cd19f516d894d22bf63659307067597",
1167
+ "value": 570
1168
+ }
1169
+ },
1170
+ "25eb5ef0f71944259b99ac80001eae94": {
1171
+ "model_module": "@jupyter-widgets/controls",
1172
+ "model_name": "HTMLModel",
1173
+ "model_module_version": "1.5.0",
1174
+ "state": {
1175
+ "_dom_classes": [],
1176
+ "_model_module": "@jupyter-widgets/controls",
1177
+ "_model_module_version": "1.5.0",
1178
+ "_model_name": "HTMLModel",
1179
+ "_view_count": null,
1180
+ "_view_module": "@jupyter-widgets/controls",
1181
+ "_view_module_version": "1.5.0",
1182
+ "_view_name": "HTMLView",
1183
+ "description": "",
1184
+ "description_tooltip": null,
1185
+ "layout": "IPY_MODEL_22ca14fb75a6407a97dea746b5f02281",
1186
+ "placeholder": "​",
1187
+ "style": "IPY_MODEL_aa40f63485dc41dca688d63cdb05fedb",
1188
+ "value": " 570/570 [00:00&lt;00:00, 21.0kB/s]"
1189
+ }
1190
+ },
1191
+ "3813850b90a74187bf18ec32a3634bf0": {
1192
+ "model_module": "@jupyter-widgets/base",
1193
+ "model_name": "LayoutModel",
1194
+ "model_module_version": "1.2.0",
1195
+ "state": {
1196
+ "_model_module": "@jupyter-widgets/base",
1197
+ "_model_module_version": "1.2.0",
1198
+ "_model_name": "LayoutModel",
1199
+ "_view_count": null,
1200
+ "_view_module": "@jupyter-widgets/base",
1201
+ "_view_module_version": "1.2.0",
1202
+ "_view_name": "LayoutView",
1203
+ "align_content": null,
1204
+ "align_items": null,
1205
+ "align_self": null,
1206
+ "border": null,
1207
+ "bottom": null,
1208
+ "display": null,
1209
+ "flex": null,
1210
+ "flex_flow": null,
1211
+ "grid_area": null,
1212
+ "grid_auto_columns": null,
1213
+ "grid_auto_flow": null,
1214
+ "grid_auto_rows": null,
1215
+ "grid_column": null,
1216
+ "grid_gap": null,
1217
+ "grid_row": null,
1218
+ "grid_template_areas": null,
1219
+ "grid_template_columns": null,
1220
+ "grid_template_rows": null,
1221
+ "height": null,
1222
+ "justify_content": null,
1223
+ "justify_items": null,
1224
+ "left": null,
1225
+ "margin": null,
1226
+ "max_height": null,
1227
+ "max_width": null,
1228
+ "min_height": null,
1229
+ "min_width": null,
1230
+ "object_fit": null,
1231
+ "object_position": null,
1232
+ "order": null,
1233
+ "overflow": null,
1234
+ "overflow_x": null,
1235
+ "overflow_y": null,
1236
+ "padding": null,
1237
+ "right": null,
1238
+ "top": null,
1239
+ "visibility": null,
1240
+ "width": null
1241
+ }
1242
+ },
1243
+ "968631b9293a4a49959bfce2b166343d": {
1244
+ "model_module": "@jupyter-widgets/base",
1245
+ "model_name": "LayoutModel",
1246
+ "model_module_version": "1.2.0",
1247
+ "state": {
1248
+ "_model_module": "@jupyter-widgets/base",
1249
+ "_model_module_version": "1.2.0",
1250
+ "_model_name": "LayoutModel",
1251
+ "_view_count": null,
1252
+ "_view_module": "@jupyter-widgets/base",
1253
+ "_view_module_version": "1.2.0",
1254
+ "_view_name": "LayoutView",
1255
+ "align_content": null,
1256
+ "align_items": null,
1257
+ "align_self": null,
1258
+ "border": null,
1259
+ "bottom": null,
1260
+ "display": null,
1261
+ "flex": null,
1262
+ "flex_flow": null,
1263
+ "grid_area": null,
1264
+ "grid_auto_columns": null,
1265
+ "grid_auto_flow": null,
1266
+ "grid_auto_rows": null,
1267
+ "grid_column": null,
1268
+ "grid_gap": null,
1269
+ "grid_row": null,
1270
+ "grid_template_areas": null,
1271
+ "grid_template_columns": null,
1272
+ "grid_template_rows": null,
1273
+ "height": null,
1274
+ "justify_content": null,
1275
+ "justify_items": null,
1276
+ "left": null,
1277
+ "margin": null,
1278
+ "max_height": null,
1279
+ "max_width": null,
1280
+ "min_height": null,
1281
+ "min_width": null,
1282
+ "object_fit": null,
1283
+ "object_position": null,
1284
+ "order": null,
1285
+ "overflow": null,
1286
+ "overflow_x": null,
1287
+ "overflow_y": null,
1288
+ "padding": null,
1289
+ "right": null,
1290
+ "top": null,
1291
+ "visibility": null,
1292
+ "width": null
1293
+ }
1294
+ },
1295
+ "79fded52694644b08614e9b194236fc2": {
1296
+ "model_module": "@jupyter-widgets/controls",
1297
+ "model_name": "DescriptionStyleModel",
1298
+ "model_module_version": "1.5.0",
1299
+ "state": {
1300
+ "_model_module": "@jupyter-widgets/controls",
1301
+ "_model_module_version": "1.5.0",
1302
+ "_model_name": "DescriptionStyleModel",
1303
+ "_view_count": null,
1304
+ "_view_module": "@jupyter-widgets/base",
1305
+ "_view_module_version": "1.2.0",
1306
+ "_view_name": "StyleView",
1307
+ "description_width": ""
1308
+ }
1309
+ },
1310
+ "3c0350e52031407eb6e02701305330ff": {
1311
+ "model_module": "@jupyter-widgets/base",
1312
+ "model_name": "LayoutModel",
1313
+ "model_module_version": "1.2.0",
1314
+ "state": {
1315
+ "_model_module": "@jupyter-widgets/base",
1316
+ "_model_module_version": "1.2.0",
1317
+ "_model_name": "LayoutModel",
1318
+ "_view_count": null,
1319
+ "_view_module": "@jupyter-widgets/base",
1320
+ "_view_module_version": "1.2.0",
1321
+ "_view_name": "LayoutView",
1322
+ "align_content": null,
1323
+ "align_items": null,
1324
+ "align_self": null,
1325
+ "border": null,
1326
+ "bottom": null,
1327
+ "display": null,
1328
+ "flex": null,
1329
+ "flex_flow": null,
1330
+ "grid_area": null,
1331
+ "grid_auto_columns": null,
1332
+ "grid_auto_flow": null,
1333
+ "grid_auto_rows": null,
1334
+ "grid_column": null,
1335
+ "grid_gap": null,
1336
+ "grid_row": null,
1337
+ "grid_template_areas": null,
1338
+ "grid_template_columns": null,
1339
+ "grid_template_rows": null,
1340
+ "height": null,
1341
+ "justify_content": null,
1342
+ "justify_items": null,
1343
+ "left": null,
1344
+ "margin": null,
1345
+ "max_height": null,
1346
+ "max_width": null,
1347
+ "min_height": null,
1348
+ "min_width": null,
1349
+ "object_fit": null,
1350
+ "object_position": null,
1351
+ "order": null,
1352
+ "overflow": null,
1353
+ "overflow_x": null,
1354
+ "overflow_y": null,
1355
+ "padding": null,
1356
+ "right": null,
1357
+ "top": null,
1358
+ "visibility": null,
1359
+ "width": null
1360
+ }
1361
+ },
1362
+ "3cd19f516d894d22bf63659307067597": {
1363
+ "model_module": "@jupyter-widgets/controls",
1364
+ "model_name": "ProgressStyleModel",
1365
+ "model_module_version": "1.5.0",
1366
+ "state": {
1367
+ "_model_module": "@jupyter-widgets/controls",
1368
+ "_model_module_version": "1.5.0",
1369
+ "_model_name": "ProgressStyleModel",
1370
+ "_view_count": null,
1371
+ "_view_module": "@jupyter-widgets/base",
1372
+ "_view_module_version": "1.2.0",
1373
+ "_view_name": "StyleView",
1374
+ "bar_color": null,
1375
+ "description_width": ""
1376
+ }
1377
+ },
1378
+ "22ca14fb75a6407a97dea746b5f02281": {
1379
+ "model_module": "@jupyter-widgets/base",
1380
+ "model_name": "LayoutModel",
1381
+ "model_module_version": "1.2.0",
1382
+ "state": {
1383
+ "_model_module": "@jupyter-widgets/base",
1384
+ "_model_module_version": "1.2.0",
1385
+ "_model_name": "LayoutModel",
1386
+ "_view_count": null,
1387
+ "_view_module": "@jupyter-widgets/base",
1388
+ "_view_module_version": "1.2.0",
1389
+ "_view_name": "LayoutView",
1390
+ "align_content": null,
1391
+ "align_items": null,
1392
+ "align_self": null,
1393
+ "border": null,
1394
+ "bottom": null,
1395
+ "display": null,
1396
+ "flex": null,
1397
+ "flex_flow": null,
1398
+ "grid_area": null,
1399
+ "grid_auto_columns": null,
1400
+ "grid_auto_flow": null,
1401
+ "grid_auto_rows": null,
1402
+ "grid_column": null,
1403
+ "grid_gap": null,
1404
+ "grid_row": null,
1405
+ "grid_template_areas": null,
1406
+ "grid_template_columns": null,
1407
+ "grid_template_rows": null,
1408
+ "height": null,
1409
+ "justify_content": null,
1410
+ "justify_items": null,
1411
+ "left": null,
1412
+ "margin": null,
1413
+ "max_height": null,
1414
+ "max_width": null,
1415
+ "min_height": null,
1416
+ "min_width": null,
1417
+ "object_fit": null,
1418
+ "object_position": null,
1419
+ "order": null,
1420
+ "overflow": null,
1421
+ "overflow_x": null,
1422
+ "overflow_y": null,
1423
+ "padding": null,
1424
+ "right": null,
1425
+ "top": null,
1426
+ "visibility": null,
1427
+ "width": null
1428
+ }
1429
+ },
1430
+ "aa40f63485dc41dca688d63cdb05fedb": {
1431
+ "model_module": "@jupyter-widgets/controls",
1432
+ "model_name": "DescriptionStyleModel",
1433
+ "model_module_version": "1.5.0",
1434
+ "state": {
1435
+ "_model_module": "@jupyter-widgets/controls",
1436
+ "_model_module_version": "1.5.0",
1437
+ "_model_name": "DescriptionStyleModel",
1438
+ "_view_count": null,
1439
+ "_view_module": "@jupyter-widgets/base",
1440
+ "_view_module_version": "1.2.0",
1441
+ "_view_name": "StyleView",
1442
+ "description_width": ""
1443
+ }
1444
+ },
1445
+ "b4d293a21ebb427d961059feacb5a876": {
1446
+ "model_module": "@jupyter-widgets/controls",
1447
+ "model_name": "HBoxModel",
1448
+ "model_module_version": "1.5.0",
1449
+ "state": {
1450
+ "_dom_classes": [],
1451
+ "_model_module": "@jupyter-widgets/controls",
1452
+ "_model_module_version": "1.5.0",
1453
+ "_model_name": "HBoxModel",
1454
+ "_view_count": null,
1455
+ "_view_module": "@jupyter-widgets/controls",
1456
+ "_view_module_version": "1.5.0",
1457
+ "_view_name": "HBoxView",
1458
+ "box_style": "",
1459
+ "children": [
1460
+ "IPY_MODEL_d0dfa9da76d34f8dae3bb2b751d2816d",
1461
+ "IPY_MODEL_50ee3e7cc36b41699fff0e0b2deca480",
1462
+ "IPY_MODEL_2015d00ec93c4bd5b6ab0870e2f3283d"
1463
+ ],
1464
+ "layout": "IPY_MODEL_49300220b8de4a90a25785585a9bab28"
1465
+ }
1466
+ },
1467
+ "d0dfa9da76d34f8dae3bb2b751d2816d": {
1468
+ "model_module": "@jupyter-widgets/controls",
1469
+ "model_name": "HTMLModel",
1470
+ "model_module_version": "1.5.0",
1471
+ "state": {
1472
+ "_dom_classes": [],
1473
+ "_model_module": "@jupyter-widgets/controls",
1474
+ "_model_module_version": "1.5.0",
1475
+ "_model_name": "HTMLModel",
1476
+ "_view_count": null,
1477
+ "_view_module": "@jupyter-widgets/controls",
1478
+ "_view_module_version": "1.5.0",
1479
+ "_view_name": "HTMLView",
1480
+ "description": "",
1481
+ "description_tooltip": null,
1482
+ "layout": "IPY_MODEL_b6d0476d32b945d6a0aaf0275035444f",
1483
+ "placeholder": "​",
1484
+ "style": "IPY_MODEL_5c36a1ce72904291afcb56101ec45415",
1485
+ "value": "Downloading (…)/main/tokenizer.json: 100%"
1486
+ }
1487
+ },
1488
+ "50ee3e7cc36b41699fff0e0b2deca480": {
1489
+ "model_module": "@jupyter-widgets/controls",
1490
+ "model_name": "FloatProgressModel",
1491
+ "model_module_version": "1.5.0",
1492
+ "state": {
1493
+ "_dom_classes": [],
1494
+ "_model_module": "@jupyter-widgets/controls",
1495
+ "_model_module_version": "1.5.0",
1496
+ "_model_name": "FloatProgressModel",
1497
+ "_view_count": null,
1498
+ "_view_module": "@jupyter-widgets/controls",
1499
+ "_view_module_version": "1.5.0",
1500
+ "_view_name": "ProgressView",
1501
+ "bar_style": "success",
1502
+ "description": "",
1503
+ "description_tooltip": null,
1504
+ "layout": "IPY_MODEL_dbaa1990251e48489ea6d647e4f20b5d",
1505
+ "max": 435797,
1506
+ "min": 0,
1507
+ "orientation": "horizontal",
1508
+ "style": "IPY_MODEL_169fd06eaf074fa5ac4bfb2100bfa598",
1509
+ "value": 435797
1510
+ }
1511
+ },
1512
+ "2015d00ec93c4bd5b6ab0870e2f3283d": {
1513
+ "model_module": "@jupyter-widgets/controls",
1514
+ "model_name": "HTMLModel",
1515
+ "model_module_version": "1.5.0",
1516
+ "state": {
1517
+ "_dom_classes": [],
1518
+ "_model_module": "@jupyter-widgets/controls",
1519
+ "_model_module_version": "1.5.0",
1520
+ "_model_name": "HTMLModel",
1521
+ "_view_count": null,
1522
+ "_view_module": "@jupyter-widgets/controls",
1523
+ "_view_module_version": "1.5.0",
1524
+ "_view_name": "HTMLView",
1525
+ "description": "",
1526
+ "description_tooltip": null,
1527
+ "layout": "IPY_MODEL_6b7a2727b9164145824742da5daf5ba4",
1528
+ "placeholder": "​",
1529
+ "style": "IPY_MODEL_dfc2bb905bd74ad1ad91614be4b9734a",
1530
+ "value": " 436k/436k [00:00&lt;00:00, 3.40MB/s]"
1531
+ }
1532
+ },
1533
+ "49300220b8de4a90a25785585a9bab28": {
1534
+ "model_module": "@jupyter-widgets/base",
1535
+ "model_name": "LayoutModel",
1536
+ "model_module_version": "1.2.0",
1537
+ "state": {
1538
+ "_model_module": "@jupyter-widgets/base",
1539
+ "_model_module_version": "1.2.0",
1540
+ "_model_name": "LayoutModel",
1541
+ "_view_count": null,
1542
+ "_view_module": "@jupyter-widgets/base",
1543
+ "_view_module_version": "1.2.0",
1544
+ "_view_name": "LayoutView",
1545
+ "align_content": null,
1546
+ "align_items": null,
1547
+ "align_self": null,
1548
+ "border": null,
1549
+ "bottom": null,
1550
+ "display": null,
1551
+ "flex": null,
1552
+ "flex_flow": null,
1553
+ "grid_area": null,
1554
+ "grid_auto_columns": null,
1555
+ "grid_auto_flow": null,
1556
+ "grid_auto_rows": null,
1557
+ "grid_column": null,
1558
+ "grid_gap": null,
1559
+ "grid_row": null,
1560
+ "grid_template_areas": null,
1561
+ "grid_template_columns": null,
1562
+ "grid_template_rows": null,
1563
+ "height": null,
1564
+ "justify_content": null,
1565
+ "justify_items": null,
1566
+ "left": null,
1567
+ "margin": null,
1568
+ "max_height": null,
1569
+ "max_width": null,
1570
+ "min_height": null,
1571
+ "min_width": null,
1572
+ "object_fit": null,
1573
+ "object_position": null,
1574
+ "order": null,
1575
+ "overflow": null,
1576
+ "overflow_x": null,
1577
+ "overflow_y": null,
1578
+ "padding": null,
1579
+ "right": null,
1580
+ "top": null,
1581
+ "visibility": null,
1582
+ "width": null
1583
+ }
1584
+ },
1585
+ "b6d0476d32b945d6a0aaf0275035444f": {
1586
+ "model_module": "@jupyter-widgets/base",
1587
+ "model_name": "LayoutModel",
1588
+ "model_module_version": "1.2.0",
1589
+ "state": {
1590
+ "_model_module": "@jupyter-widgets/base",
1591
+ "_model_module_version": "1.2.0",
1592
+ "_model_name": "LayoutModel",
1593
+ "_view_count": null,
1594
+ "_view_module": "@jupyter-widgets/base",
1595
+ "_view_module_version": "1.2.0",
1596
+ "_view_name": "LayoutView",
1597
+ "align_content": null,
1598
+ "align_items": null,
1599
+ "align_self": null,
1600
+ "border": null,
1601
+ "bottom": null,
1602
+ "display": null,
1603
+ "flex": null,
1604
+ "flex_flow": null,
1605
+ "grid_area": null,
1606
+ "grid_auto_columns": null,
1607
+ "grid_auto_flow": null,
1608
+ "grid_auto_rows": null,
1609
+ "grid_column": null,
1610
+ "grid_gap": null,
1611
+ "grid_row": null,
1612
+ "grid_template_areas": null,
1613
+ "grid_template_columns": null,
1614
+ "grid_template_rows": null,
1615
+ "height": null,
1616
+ "justify_content": null,
1617
+ "justify_items": null,
1618
+ "left": null,
1619
+ "margin": null,
1620
+ "max_height": null,
1621
+ "max_width": null,
1622
+ "min_height": null,
1623
+ "min_width": null,
1624
+ "object_fit": null,
1625
+ "object_position": null,
1626
+ "order": null,
1627
+ "overflow": null,
1628
+ "overflow_x": null,
1629
+ "overflow_y": null,
1630
+ "padding": null,
1631
+ "right": null,
1632
+ "top": null,
1633
+ "visibility": null,
1634
+ "width": null
1635
+ }
1636
+ },
1637
+ "5c36a1ce72904291afcb56101ec45415": {
1638
+ "model_module": "@jupyter-widgets/controls",
1639
+ "model_name": "DescriptionStyleModel",
1640
+ "model_module_version": "1.5.0",
1641
+ "state": {
1642
+ "_model_module": "@jupyter-widgets/controls",
1643
+ "_model_module_version": "1.5.0",
1644
+ "_model_name": "DescriptionStyleModel",
1645
+ "_view_count": null,
1646
+ "_view_module": "@jupyter-widgets/base",
1647
+ "_view_module_version": "1.2.0",
1648
+ "_view_name": "StyleView",
1649
+ "description_width": ""
1650
+ }
1651
+ },
1652
+ "dbaa1990251e48489ea6d647e4f20b5d": {
1653
+ "model_module": "@jupyter-widgets/base",
1654
+ "model_name": "LayoutModel",
1655
+ "model_module_version": "1.2.0",
1656
+ "state": {
1657
+ "_model_module": "@jupyter-widgets/base",
1658
+ "_model_module_version": "1.2.0",
1659
+ "_model_name": "LayoutModel",
1660
+ "_view_count": null,
1661
+ "_view_module": "@jupyter-widgets/base",
1662
+ "_view_module_version": "1.2.0",
1663
+ "_view_name": "LayoutView",
1664
+ "align_content": null,
1665
+ "align_items": null,
1666
+ "align_self": null,
1667
+ "border": null,
1668
+ "bottom": null,
1669
+ "display": null,
1670
+ "flex": null,
1671
+ "flex_flow": null,
1672
+ "grid_area": null,
1673
+ "grid_auto_columns": null,
1674
+ "grid_auto_flow": null,
1675
+ "grid_auto_rows": null,
1676
+ "grid_column": null,
1677
+ "grid_gap": null,
1678
+ "grid_row": null,
1679
+ "grid_template_areas": null,
1680
+ "grid_template_columns": null,
1681
+ "grid_template_rows": null,
1682
+ "height": null,
1683
+ "justify_content": null,
1684
+ "justify_items": null,
1685
+ "left": null,
1686
+ "margin": null,
1687
+ "max_height": null,
1688
+ "max_width": null,
1689
+ "min_height": null,
1690
+ "min_width": null,
1691
+ "object_fit": null,
1692
+ "object_position": null,
1693
+ "order": null,
1694
+ "overflow": null,
1695
+ "overflow_x": null,
1696
+ "overflow_y": null,
1697
+ "padding": null,
1698
+ "right": null,
1699
+ "top": null,
1700
+ "visibility": null,
1701
+ "width": null
1702
+ }
1703
+ },
1704
+ "169fd06eaf074fa5ac4bfb2100bfa598": {
1705
+ "model_module": "@jupyter-widgets/controls",
1706
+ "model_name": "ProgressStyleModel",
1707
+ "model_module_version": "1.5.0",
1708
+ "state": {
1709
+ "_model_module": "@jupyter-widgets/controls",
1710
+ "_model_module_version": "1.5.0",
1711
+ "_model_name": "ProgressStyleModel",
1712
+ "_view_count": null,
1713
+ "_view_module": "@jupyter-widgets/base",
1714
+ "_view_module_version": "1.2.0",
1715
+ "_view_name": "StyleView",
1716
+ "bar_color": null,
1717
+ "description_width": ""
1718
+ }
1719
+ },
1720
+ "6b7a2727b9164145824742da5daf5ba4": {
1721
+ "model_module": "@jupyter-widgets/base",
1722
+ "model_name": "LayoutModel",
1723
+ "model_module_version": "1.2.0",
1724
+ "state": {
1725
+ "_model_module": "@jupyter-widgets/base",
1726
+ "_model_module_version": "1.2.0",
1727
+ "_model_name": "LayoutModel",
1728
+ "_view_count": null,
1729
+ "_view_module": "@jupyter-widgets/base",
1730
+ "_view_module_version": "1.2.0",
1731
+ "_view_name": "LayoutView",
1732
+ "align_content": null,
1733
+ "align_items": null,
1734
+ "align_self": null,
1735
+ "border": null,
1736
+ "bottom": null,
1737
+ "display": null,
1738
+ "flex": null,
1739
+ "flex_flow": null,
1740
+ "grid_area": null,
1741
+ "grid_auto_columns": null,
1742
+ "grid_auto_flow": null,
1743
+ "grid_auto_rows": null,
1744
+ "grid_column": null,
1745
+ "grid_gap": null,
1746
+ "grid_row": null,
1747
+ "grid_template_areas": null,
1748
+ "grid_template_columns": null,
1749
+ "grid_template_rows": null,
1750
+ "height": null,
1751
+ "justify_content": null,
1752
+ "justify_items": null,
1753
+ "left": null,
1754
+ "margin": null,
1755
+ "max_height": null,
1756
+ "max_width": null,
1757
+ "min_height": null,
1758
+ "min_width": null,
1759
+ "object_fit": null,
1760
+ "object_position": null,
1761
+ "order": null,
1762
+ "overflow": null,
1763
+ "overflow_x": null,
1764
+ "overflow_y": null,
1765
+ "padding": null,
1766
+ "right": null,
1767
+ "top": null,
1768
+ "visibility": null,
1769
+ "width": null
1770
+ }
1771
+ },
1772
+ "dfc2bb905bd74ad1ad91614be4b9734a": {
1773
+ "model_module": "@jupyter-widgets/controls",
1774
+ "model_name": "DescriptionStyleModel",
1775
+ "model_module_version": "1.5.0",
1776
+ "state": {
1777
+ "_model_module": "@jupyter-widgets/controls",
1778
+ "_model_module_version": "1.5.0",
1779
+ "_model_name": "DescriptionStyleModel",
1780
+ "_view_count": null,
1781
+ "_view_module": "@jupyter-widgets/base",
1782
+ "_view_module_version": "1.2.0",
1783
+ "_view_name": "StyleView",
1784
+ "description_width": ""
1785
+ }
1786
+ }
1787
+ }
1788
+ }
1789
+ },
1790
+ "nbformat": 4,
1791
+ "nbformat_minor": 0
1792
+ }