"
],
"text/html": [
"\n",
" \n",
" \n",
" | \n",
" sentence | \n",
"
\n",
" \n",
" \n",
" \n",
" | 0 | \n",
" бөмбөгөө хураалгахаас зайлсхийн зүгээр л ойролцоох тоглогч руу чиглүүлээд чулуудаж орхив | \n",
"
\n",
" \n",
" | 1 | \n",
" цайзад бидний түүсэн мөчир байгаа гэж эд хариулав | \n",
"
\n",
" \n",
" | 2 | \n",
" гидролизын урвалыг ашиглан хамгийн хурдан үйлчилдэг уургийн хэлбэрийг гарган авчээ | \n",
"
\n",
" \n",
" | 3 | \n",
" энэ төсвийг жаалуудаас харамлаж жалга довны зүйлд зарлагадсан гэж хардахаас өөр арга алга | \n",
"
\n",
" \n",
" | 4 | \n",
" санал хүсэлтийн утсыг олон газар ажиллуулдаг ч залгаад улам уурлаад дуусах нь элбэг тохиолддог | \n",
"
\n",
" \n",
" | 5 | \n",
" байшингийн захиргааны дарга энэ ярианд сэтгэл огт ханасангүй | \n",
"
\n",
" \n",
" | 6 | \n",
" аливаа зүйл хомсдохдоо үнэ цэн нь өсөж хүмүүс түүнийг улам ихээр үгүйлдэг | \n",
"
\n",
" \n",
" | 7 | \n",
" харин дархан джузеппег тэр аль хэдийнээс мэддэг байлаа | \n",
"
\n",
" \n",
" | 8 | \n",
" марияд эх орноо хайрлах үзэл нялх бага байхад нь л төлөвшсөн | \n",
"
\n",
" \n",
" | 9 | \n",
" тэгээд гэнэт цочин ээжийг харахад дуугүйхэн нулимсаа арчиж байна | \n",
"
\n",
" \n",
"
"
]
},
"metadata": {}
}
]
},
{
"cell_type": "code",
"source": [
"def extract_all_chars(batch):\n",
" all_text = \" \".join(batch[\"sentence\"])\n",
" vocab = list(set(all_text))\n",
" return {\"vocab\": [vocab], \"all_text\": [all_text]}"
],
"metadata": {
"id": "bfcwBKamNDX8"
},
"execution_count": 15,
"outputs": []
},
{
"cell_type": "code",
"source": [
"vocab_train = common_voice_train.map(extract_all_chars, batched=True, batch_size=-1, keep_in_memory=True, remove_columns=common_voice_train.column_names)\n",
"vocab_test = common_voice_test.map(extract_all_chars, batched=True, batch_size=-1, keep_in_memory=True, remove_columns=common_voice_test.column_names)"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 81,
"referenced_widgets": [
"22a73456734c4195a8b664eea56060b4",
"a6fb84966d464fb084e4ff28fb819597",
"759972fc0c264e7ca12c3e9beefb687b",
"c0de6ce1aecf4f9b9f55cc0fb3dd408b",
"14634a3546bb4d05ac892a8e236b388c",
"4b498b14406545c1bed99f25a6594722",
"69f722871117402fa549647da50e76d7",
"e9f0076bedaf46eba7c84f22f3c3a516",
"e79476d487e844c1ae3ba3ac3c6b8a92",
"cc98af514295460ebfde23cd3ab74588",
"974993c7e39647ba93a6af05513c8911",
"d26ded21d3d040c78d51a0b366aade4e",
"46e7918fe0f947a9bd1579875dd1e709",
"d25e569e8bdf4130b57cbd4a8f73396a",
"e962a75632054a6991d235f3404cebbf",
"adf9836a19b34730a49b75748678d4e4",
"aacf7a1900ee4b049f742e5d71641653",
"01e1f849ead44c1c99fa749d62dc5be8",
"d5ab34c153ed4649ae42e8fcce7779c6",
"1a547a8d2a964a3a93f307036d58b45d",
"1f3b5f5f73274fc498ff04f73cd45033",
"42c73ab5ffb84f04b848f4479cc2cb25"
]
},
"id": "_Lz5XcXMWlRK",
"outputId": "0c20cd73-f955-4d15-c00a-0aeca25cbb69"
},
"execution_count": 16,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
" 0%| | 0/1 [00:00 Dict[str, torch.Tensor]:\n",
" # split inputs and labels since they have to be of different lenghts and need\n",
" # different padding methods\n",
" input_features = [{\"input_values\": feature[\"input_values\"]} for feature in features]\n",
" label_features = [{\"input_ids\": feature[\"labels\"]} for feature in features]\n",
"\n",
" batch = self.processor.pad(\n",
" input_features,\n",
" padding=self.padding,\n",
" return_tensors=\"pt\",\n",
" )\n",
" with self.processor.as_target_processor():\n",
" labels_batch = self.processor.pad(\n",
" label_features,\n",
" padding=self.padding,\n",
" return_tensors=\"pt\",\n",
" )\n",
"\n",
" # replace padding with -100 to ignore loss correctly\n",
" labels = labels_batch[\"input_ids\"].masked_fill(labels_batch.attention_mask.ne(1), -100)\n",
"\n",
" batch[\"labels\"] = labels\n",
"\n",
" return batch"
],
"metadata": {
"id": "whu0EJxOgfyy"
},
"execution_count": 36,
"outputs": []
},
{
"cell_type": "code",
"source": [
"data_collator = DataCollatorCTCWithPadding(processor=processor, padding=True)"
],
"metadata": {
"id": "Ltc2dxGlgukN"
},
"execution_count": 37,
"outputs": []
},
{
"cell_type": "code",
"source": [
"from datasets import load_dataset, load_metric, Audio"
],
"metadata": {
"id": "QS5WEKd8hnhi"
},
"execution_count": 38,
"outputs": []
},
{
"cell_type": "code",
"source": [
"wer_metric = load_metric(\"wer\")"
],
"metadata": {
"id": "xR73S4hhglX_",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 49,
"referenced_widgets": [
"97b0443dfc314fcaa1f58148e7b0fe51",
"11618b57382943e4a308591cfad2e9e5",
"9a62654284a04fcdac75d5dfbab9eca5",
"8747480e6ddb4831b538d0b0b69b77a6",
"ff19a949944f48ad8a06589868ff5f1e",
"e5bab6c722a54f79b63b489ad87a1e8f",
"935ed8c2c69044979a4f4fa51b3c60c9",
"d4bc82f10abc4a2a9fc7ebb810c4a233",
"6f1886c3a50a40acb8b50d3a40782f51",
"ffdb0fd1ce574c64bb3e351e33e3c89b",
"e7b0c2f4568e4214ac4e6202eee9a8cc"
]
},
"outputId": "378cdcc0-5339-4478-dd1e-247d2f6ab410"
},
"execution_count": 39,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
"Downloading: 0%| | 0.00/1.90k [00:00"
],
"text/html": [
"\n",
" \n",
" \n",
"
\n",
" [1800/1800 2:21:35, Epoch 30/30]\n",
"
\n",
" \n",
" \n",
" | Step | \n",
" Training Loss | \n",
" Validation Loss | \n",
" Wer | \n",
"
\n",
" \n",
" \n",
" \n",
" | 400 | \n",
" 2.249400 | \n",
" 0.848729 | \n",
" 0.526894 | \n",
"
\n",
" \n",
" | 800 | \n",
" 0.143000 | \n",
" 0.829556 | \n",
" 0.525183 | \n",
"
\n",
" \n",
" | 1200 | \n",
" 0.100700 | \n",
" 0.813215 | \n",
" 0.504110 | \n",
"
\n",
" \n",
" | 1600 | \n",
" 0.070000 | \n",
" 0.833226 | \n",
" 0.480113 | \n",
"
\n",
" \n",
"
"
]
},
"metadata": {}
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length.\n",
"***** Running Evaluation *****\n",
" Num examples = 1862\n",
" Batch size = 8\n",
"Saving model checkpoint to repo_name/checkpoint-400\n",
"Configuration saved in repo_name/checkpoint-400/config.json\n",
"Model weights saved in repo_name/checkpoint-400/pytorch_model.bin\n",
"Configuration saved in repo_name/checkpoint-400/preprocessor_config.json\n",
"/usr/local/lib/python3.9/dist-packages/transformers/models/wav2vec2/modeling_wav2vec2.py:882: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').\n",
" return (input_length - kernel_size) // stride + 1\n",
"The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length.\n",
"***** Running Evaluation *****\n",
" Num examples = 1862\n",
" Batch size = 8\n",
"Saving model checkpoint to repo_name/checkpoint-800\n",
"Configuration saved in repo_name/checkpoint-800/config.json\n",
"Model weights saved in repo_name/checkpoint-800/pytorch_model.bin\n",
"Configuration saved in repo_name/checkpoint-800/preprocessor_config.json\n",
"/usr/local/lib/python3.9/dist-packages/transformers/models/wav2vec2/modeling_wav2vec2.py:882: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').\n",
" return (input_length - kernel_size) // stride + 1\n",
"The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length.\n",
"***** Running Evaluation *****\n",
" Num examples = 1862\n",
" Batch size = 8\n",
"Saving model checkpoint to repo_name/checkpoint-1200\n",
"Configuration saved in repo_name/checkpoint-1200/config.json\n",
"Model weights saved in repo_name/checkpoint-1200/pytorch_model.bin\n",
"Configuration saved in repo_name/checkpoint-1200/preprocessor_config.json\n",
"Deleting older checkpoint [repo_name/checkpoint-400] due to args.save_total_limit\n",
"/usr/local/lib/python3.9/dist-packages/transformers/models/wav2vec2/modeling_wav2vec2.py:882: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').\n",
" return (input_length - kernel_size) // stride + 1\n",
"The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length.\n",
"***** Running Evaluation *****\n",
" Num examples = 1862\n",
" Batch size = 8\n",
"Saving model checkpoint to repo_name/checkpoint-1600\n",
"Configuration saved in repo_name/checkpoint-1600/config.json\n",
"Model weights saved in repo_name/checkpoint-1600/pytorch_model.bin\n",
"Configuration saved in repo_name/checkpoint-1600/preprocessor_config.json\n",
"Deleting older checkpoint [repo_name/checkpoint-800] due to args.save_total_limit\n",
"/usr/local/lib/python3.9/dist-packages/transformers/models/wav2vec2/modeling_wav2vec2.py:882: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').\n",
" return (input_length - kernel_size) // stride + 1\n",
"\n",
"\n",
"Training completed. Do not forget to share your model on huggingface.co/models =)\n",
"\n",
"\n"
]
},
{
"output_type": "execute_result",
"data": {
"text/plain": [
"TrainOutput(global_step=1800, training_loss=0.5758350520663791, metrics={'train_runtime': 8501.5757, 'train_samples_per_second': 6.772, 'train_steps_per_second': 0.212, 'total_flos': 7.124680585822395e+18, 'train_loss': 0.5758350520663791, 'epoch': 30.0})"
]
},
"metadata": {},
"execution_count": 44
}
]
},
{
"cell_type": "code",
"source": [
"common_voice_test_transcription = load_dataset(\"common_voice\", \"mn\", data_dir=\"./cv-corpus-6.1-2020-12-11\", split=\"test\")\n",
"\n",
"for i in range(10):\n",
" input_dict = processor(common_voice_test[i][\"input_values\"], return_tensors=\"pt\", padding=True)\n",
"\n",
" logits = model(input_dict.input_values.to(\"cuda\")).logits\n",
"\n",
" pred_ids = torch.argmax(logits, dim=-1)[0]\n",
"\n",
" print(f\"Prediction_{i}:\")\n",
" print(processor.decode(pred_ids))\n",
"\n",
" print(f\"\\nReference_{i}:\")\n",
" print(common_voice_test_transcription[i][\"sentence\"].lower())"
],
"metadata": {
"id": "FHIIY-qT7qo4",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "bdfdcb9d-01f5-4816-c9c7-7ca328805c82"
},
"execution_count": 54,
"outputs": [
{
"output_type": "stream",
"name": "stderr",
"text": [
"WARNING:datasets.builder:Using custom data configuration mn-00b7e43d66b8c1a3\n",
"WARNING:datasets.builder:Reusing dataset common_voice (/root/.cache/huggingface/datasets/common_voice/mn-00b7e43d66b8c1a3/6.1.0/5693bfc0feeade582a78c2fb250bc88f52bd86f0a7f1bb22bfee67e715de30fd)\n",
"It is strongly recommended to pass the ``sampling_rate`` argument to this function.Failing to do so can result in silent errors that might be hard to debug.\n",
"It is strongly recommended to pass the ``sampling_rate`` argument to this function.Failing to do so can result in silent errors that might be hard to debug.\n"
]
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"Prediction_0:\n",
"хэрэв захиа байгаагүй бол чиг андалсын тухай дурсах ч үгүй байсан\n",
"\n",
"Reference_0:\n",
"хэрэв захиа байгаагүй бол чи гэндальфын тухай дурсах ч үгүй байсан.\n"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"It is strongly recommended to pass the ``sampling_rate`` argument to this function.Failing to do so can result in silent errors that might be hard to debug.\n"
]
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"Prediction_1:\n",
"алтагууны ширээн дээр мэдээдллэг харагдахуулц байрнь цоог дөшүүлөн тавьжээ\n",
"\n",
"Reference_1:\n",
"гал тогооны ширээн дээр мэдээллийг харагдахуйц байранд сав түшүүлэн тавьжээ.\n",
"Prediction_2:\n",
"энэ мо та хоёр яг адилхань бага болчир шүү дээ\n",
"\n",
"Reference_2:\n"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"It is strongly recommended to pass the ``sampling_rate`` argument to this function.Failing to do so can result in silent errors that might be hard to debug.\n",
"It is strongly recommended to pass the ``sampling_rate`` argument to this function.Failing to do so can result in silent errors that might be hard to debug.\n"
]
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"энэ мод та хоёр яг адилхан бага балчир шүү дээ.\n",
"Prediction_3:\n",
"чойллсон ч тэгж бодсон нь бий\n",
"\n",
"Reference_3:\n",
"чойбалсан ч тэгж бодсон нь бий.\n"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"It is strongly recommended to pass the ``sampling_rate`` argument to this function.Failing to do so can result in silent errors that might be hard to debug.\n"
]
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"Prediction_4:\n",
"мөн л овоог нэгэн насриг нь тодорхойсон өнгө бүрийн туузүй зүүжээ\n",
"\n",
"Reference_4:\n",
"мөн л овог нэр насыг нь тодорхойлсон өнгө бүрийн тууз зүүжээ.\n",
"Prediction_5:\n"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"It is strongly recommended to pass the ``sampling_rate`` argument to this function.Failing to do so can result in silent errors that might be hard to debug.\n",
"It is strongly recommended to pass the ``sampling_rate`` argument to this function.Failing to do so can result in silent errors that might be hard to debug.\n"
]
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"үнэндээ гэхэд би нэ бутын хаань ч хаяг бүртгэлгүй явна шүү дээ\n",
"\n",
"Reference_5:\n",
"үнэндээ гэхэд би энэ хотын хаана нь ч хаяг бүртгэлгүй явна шүү дээ.\n",
"Prediction_6:\n",
"сайн ч мөрдөгч бас шилдэг дарга байсан юм шүү\n",
"\n",
"Reference_6:\n",
"сайн ч мөрдөгч бас шилдэг дарга байсан юм шүү.\n"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"It is strongly recommended to pass the ``sampling_rate`` argument to this function.Failing to do so can result in silent errors that might be hard to debug.\n"
]
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"Prediction_7:\n",
"эцэг эхчүүд сурагчид та бүхэн доорхаягаар урж захим хичээлэн хийгээрэй\n",
"\n",
"Reference_7:\n",
"эцэг, эхчүүд, сурагчид та бүхэн доорх хаягаар орж цахим хичээлээ хийгээрэй.\n",
"Prediction_8:\n"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"It is strongly recommended to pass the ``sampling_rate`` argument to this function.Failing to do so can result in silent errors that might be hard to debug.\n"
]
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"та шинэ жэлийн баяраар илүү чамин баярын уур амьсгал оруулах гоёмсог умьс бэлллээтээ\n",
"\n",
"Reference_8:\n",
"та шинэ жилийн баяраар илүү чамин баярын уур амьсгал оруулах гоёмсог оймс бэлэглээрэй.\n",
"Prediction_9:\n",
"машины тунслогуйн тагарвал яана сайхан хувцаа баллай л гэсэн үг\n",
"\n",
"Reference_9:\n",
"машины донсолгоонд хагарвал яана, сайхан хувцсаа баллана л гэсэн үг.\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"# Даалгавар 1\n",
"энэхүү сургалтын WER 48% бууруулах\n",
"\n",
"# Даалгавар 2\n",
"өөрийн яриаг сургалтанд ашиглах\n",
"\n",
"# Даалгавар 3\n",
"Бусад сүүлийн үеийн модел ашиглан сургах\n",
"\n",
"# Хугацаа 4 долоо хоног"
],
"metadata": {
"id": "0baGGHnPA7QC"
}
},
{
"cell_type": "code",
"source": [],
"metadata": {
"id": "1r0Xt6q4BvHb"
},
"execution_count": null,
"outputs": []
}
]
}