Upload 20 files

Poro-34B-Lora-1/README.md

@@ -0,0 +1,204 @@
+---
+library_name: peft
+base_model: LumiOpen/Poro-34B
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure 
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+
+
+### Framework versions
+
+- PEFT 0.9.0

Poro-34B-Lora-1/adapter_config.json

@@ -0,0 +1,27 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "LumiOpen/Poro-34B",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 8,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "query_key_value"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Poro-34B-Lora-1/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f394f45e0d2e87fbed7ca75c190d8dc84ceb0a88fdadaed203fcff5b25e546f7
+size 24788784

Poro-34B-Lora-185/README.md

@@ -0,0 +1,204 @@
+---
+library_name: peft
+base_model: LumiOpen/Poro-34B
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure 
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+
+
+### Framework versions
+
+- PEFT 0.9.0

Poro-34B-Lora-185/adapter_config.json

@@ -0,0 +1,27 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "LumiOpen/Poro-34B",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 8,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "query_key_value"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Poro-34B-Lora-185/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f41dee2188266055a20fac03619585b016dc7d8accb5fd7f543c7f3159f4ad4e
+size 24788784

Poro-34B-Lora-185QA.ipynb

@@ -0,0 +1,789 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "0c4ecb49-ce58-4b65-849a-760980576e48",
+   "metadata": {},
+   "source": [
+    "# Poro34B Lora fine-tuning with S-Group's data"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5b686006-65a7-43af-8207-1c7309a5e423",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# This script finetunes the Poro34B model with 185 Questions and Answers pair"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "defcdb6f-3b69-4b03-b2dc-07c4b3027fd6",
+   "metadata": {},
+   "source": [
+    "## Initialization"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "67f730e6-3467-4a19-ab76-e8baace8e02e",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Requirement already satisfied: peft in /opt/conda/lib/python3.10/site-packages (0.9.0)\n",
+      "Requirement already satisfied: numpy>=1.17 in /opt/conda/lib/python3.10/site-packages (from peft) (1.26.3)\n",
+      "Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.10/site-packages (from peft) (23.2)\n",
+      "Requirement already satisfied: psutil in /opt/conda/lib/python3.10/site-packages (from peft) (5.9.8)\n",
+      "Requirement already satisfied: pyyaml in /opt/conda/lib/python3.10/site-packages (from peft) (6.0.1)\n",
+      "Requirement already satisfied: torch>=1.13.0 in /opt/conda/lib/python3.10/site-packages (from peft) (2.0.0.post101)\n",
+      "Requirement already satisfied: transformers in /opt/conda/lib/python3.10/site-packages (from peft) (4.31.0)\n",
+      "Requirement already satisfied: tqdm in /opt/conda/lib/python3.10/site-packages (from peft) (4.66.1)\n",
+      "Requirement already satisfied: accelerate>=0.21.0 in /opt/conda/lib/python3.10/site-packages (from peft) (0.21.0)\n",
+      "Requirement already satisfied: safetensors in /opt/conda/lib/python3.10/site-packages (from peft) (0.3.3)\n",
+      "Requirement already satisfied: huggingface-hub>=0.17.0 in /opt/conda/lib/python3.10/site-packages (from peft) (0.20.2)\n",
+      "Requirement already satisfied: filelock in /opt/conda/lib/python3.10/site-packages (from huggingface-hub>=0.17.0->peft) (3.13.1)\n",
+      "Requirement already satisfied: fsspec>=2023.5.0 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub>=0.17.0->peft) (2023.6.0)\n",
+      "Requirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from huggingface-hub>=0.17.0->peft) (2.31.0)\n",
+      "Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub>=0.17.0->peft) (4.5.0)\n",
+      "Requirement already satisfied: sympy in /opt/conda/lib/python3.10/site-packages (from torch>=1.13.0->peft) (1.12)\n",
+      "Requirement already satisfied: networkx in /opt/conda/lib/python3.10/site-packages (from torch>=1.13.0->peft) (3.2.1)\n",
+      "Requirement already satisfied: jinja2 in /opt/conda/lib/python3.10/site-packages (from torch>=1.13.0->peft) (3.1.3)\n",
+      "Requirement already satisfied: regex!=2019.12.17 in /opt/conda/lib/python3.10/site-packages (from transformers->peft) (2023.12.25)\n",
+      "Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /opt/conda/lib/python3.10/site-packages (from transformers->peft) (0.13.3)\n",
+      "Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/lib/python3.10/site-packages (from jinja2->torch>=1.13.0->peft) (2.1.4)\n",
+      "Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests->huggingface-hub>=0.17.0->peft) (3.3.2)\n",
+      "Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests->huggingface-hub>=0.17.0->peft) (3.6)\n",
+      "Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->huggingface-hub>=0.17.0->peft) (1.26.18)\n",
+      "Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.10/site-packages (from requests->huggingface-hub>=0.17.0->peft) (2023.11.17)\n",
+      "Requirement already satisfied: mpmath>=0.19 in /opt/conda/lib/python3.10/site-packages (from sympy->torch>=1.13.0->peft) (1.3.0)\n"
+     ]
+    }
+   ],
+   "source": [
+    "# pip install peft, all other Python libraries are already in AWS image\n",
+    "!pip install peft"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "80b24df2-140b-4792-aaf1-6f6aff92ece8",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "2024-02-29 16:17:22.775245: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.\n",
+      "To enable the following instructions: SSE4.1 SSE4.2 AVX AVX2 AVX512F FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.\n"
+     ]
+    }
+   ],
+   "source": [
+    "import torch\n",
+    "import json\n",
+    "from transformers import AutoModelForCausalLM, AutoTokenizer \n",
+    "from transformers import TrainingArguments, Trainer\n",
+    "from transformers import pipeline\n",
+    "from peft import  get_peft_model, PromptTuningConfig, TaskType, PromptTuningInit\n",
+    "from datasets import load_dataset"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "d31adfc6-a460-419e-871b-d0437501b026",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# this checks wether we have GPU\n",
+    "device = torch.device(\"cuda\") if torch.cuda.is_available() else torch.device(\"cpu\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "2c5a9b07-c92b-4d1d-b5b5-96e8c234e14f",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "cpu\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(device)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6ea88a10-f5f1-4342-939b-60d2b9c5bb91",
+   "metadata": {},
+   "source": [
+    "## Foundation model import"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "2c0f7b3a-9d56-46ce-9dc8-5fe40b2628a6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Foundation model\n",
+    "model_name='LumiOpen/Poro-34B'"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "4e4c9089-a195-4fd7-91b2-6240cafb4989",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "tokenizer = AutoTokenizer.from_pretrained(model_name)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "a42e0fb6-40d4-483b-a034-84ff351c021d",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "34a67d537808415ab77b583333186ae3",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Loading checkpoint shards:   0%|          | 0/14 [00:00<?, ?it/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "branch = \"1000B\"\n",
+    "model = AutoModelForCausalLM.from_pretrained(model_name,\n",
+    "    torch_dtype=torch.bfloat16,\n",
+    "    revision=branch,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "258df7e6-27c1-48a2-b20a-d377dc885884",
+   "metadata": {},
+   "source": [
+    "## Setting up the Lora parameters"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "id": "63151e65-6bff-4b65-a8ae-af4d6c53036f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from peft import LoraConfig, get_peft_model"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "id": "f35f934f-23c6-47db-95bb-df20526e29e7",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "config = LoraConfig(\n",
+    "    r=8,\n",
+    "    lora_alpha=8,\n",
+    "    target_modules=[\"query_key_value\"],\n",
+    "    lora_dropout=0.05,\n",
+    "    bias=\"none\",\n",
+    "    task_type=\"CAUSAL_LM\"\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "id": "a62cb983-6e28-40f1-9f8d-64a1a6ccd0f3",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "peft_model = get_peft_model(model, config)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "id": "fe7e7078-2998-4f17-abda-f03a32e04735",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "trainable params: 12386304\n",
+      "all params: 34229336064\n",
+      "trainable: 0.04%\n"
+     ]
+    }
+   ],
+   "source": [
+    "trainable_params = 0\n",
+    "all_param = 0\n",
+    "\n",
+    "# iterating over all parameters\n",
+    "for _, param in peft_model.named_parameters():\n",
+    "    # adding parameters to total\n",
+    "    all_param += param.numel()\n",
+    "    # adding parameters to trainable if they require a graident\n",
+    "    if param.requires_grad:\n",
+    "        trainable_params += param.numel()\n",
+    "\n",
+    "# print number of trainable parameters\n",
+    "print(f\"trainable params: {trainable_params}\")\n",
+    "print(f\"all params: {all_param}\")\n",
+    "print(f\"trainable: {100 * trainable_params / all_param:.2f}%\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d029921d-60db-43ad-ae8b-2ab9910bd490",
+   "metadata": {},
+   "source": [
+    "## Preparing the training data"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "id": "216aa90b-a87d-4a37-b178-e7e83bf987ce",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# prepare the data for training\n",
+    "def prepare_train_data(data):\n",
+    "    text_input = data['text']\n",
+    "    tokenized_input = tokenizer(text_input, return_tensors='pt', padding=True)\n",
+    "    tokenized_input['labels'] = tokenized_input['input_ids']\n",
+    "    return tokenized_input"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "id": "5551fac2-9a26-4a86-b8b4-d2f572ecfaa9",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "711294e3ec68409abdc6a3f29c271d3a",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Generating train split: 0 examples [00:00, ? examples/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "dataset = load_dataset(\"json\", data_files=\"prompts_185.json\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 17,
+   "id": "dfa8382a-3e66-4433-b6dd-97d59a7945f6",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "72acd5c60a62468c86206a93b17a87ab",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Map:   0%|          | 0/185 [00:00<?, ? examples/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "train_dataset = dataset['train'].map(prepare_train_data, batched=True, remove_columns=[\"text\"])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "319de4fe-f5e0-4ea6-9a61-6f18c241bd02",
+   "metadata": {},
+   "source": [
+    "## Setting up the training parameters"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 18,
+   "id": "9bb42764-af93-463f-8cf6-68707f21151b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from transformers import DataCollatorForLanguageModeling"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 19,
+   "id": "4b8a94c9-2648-4626-9238-4475645aa695",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "trainer = Trainer(\n",
+    "    model=peft_model,\n",
+    "    train_dataset=train_dataset,\n",
+    "    args=TrainingArguments(\n",
+    "        per_device_train_batch_size=4,\n",
+    "        gradient_accumulation_steps=4,\n",
+    "        warmup_steps=20,\n",
+    "        max_steps=20,\n",
+    "        learning_rate=1e-3,\n",
+    "        logging_steps=1,\n",
+    "        output_dir='outputs',\n",
+    "    ),\n",
+    "    data_collator=DataCollatorForLanguageModeling(tokenizer, mlm=False)\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "fbd62db8-65e3-4d05-b11e-179aaf8f0e65",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/opt/conda/lib/python3.10/site-packages/transformers/optimization.py:411: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning\n",
+      "  warnings.warn(\n",
+      "\u001b[34m\u001b[1mwandb\u001b[0m: Currently logged in as: \u001b[33mtimo-au-laine\u001b[0m. Use \u001b[1m`wandb login --relogin`\u001b[0m to force relogin\n"
+     ]
+    },
+    {
+     "data": {
+      "text/html": [
+       "wandb version 0.16.3 is available!  To upgrade, please run:\n",
+       " $ pip install wandb --upgrade"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "Tracking run with wandb version 0.16.2"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "Run data is saved locally in <code>/home/sagemaker-user/wandb/run-20240229_162718-2nyysvnx</code>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "Syncing run <strong><a href='https://wandb.ai/timo-au-laine/huggingface/runs/2nyysvnx' target=\"_blank\">trim-dust-9</a></strong> to <a href='https://wandb.ai/timo-au-laine/huggingface' target=\"_blank\">Weights & Biases</a> (<a href='https://wandb.me/run' target=\"_blank\">docs</a>)<br/>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       " View project at <a href='https://wandb.ai/timo-au-laine/huggingface' target=\"_blank\">https://wandb.ai/timo-au-laine/huggingface</a>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       " View run at <a href='https://wandb.ai/timo-au-laine/huggingface/runs/2nyysvnx' target=\"_blank\">https://wandb.ai/timo-au-laine/huggingface/runs/2nyysvnx</a>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "You're using a BloomTokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.\n"
+     ]
+    },
+    {
+     "data": {
+      "text/html": [
+       "\n",
+       "    <div>\n",
+       "      \n",
+       "      <progress value='19' max='20' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
+       "      [19/20 5:10:38 < 18:16, 0.00 it/s, Epoch 1.53/2]\n",
+       "    </div>\n",
+       "    <table border=\"1\" class=\"dataframe\">\n",
+       "  <thead>\n",
+       " <tr style=\"text-align: left;\">\n",
+       "      <th>Step</th>\n",
+       "      <th>Training Loss</th>\n",
+       "    </tr>\n",
+       "  </thead>\n",
+       "  <tbody>\n",
+       "    <tr>\n",
+       "      <td>1</td>\n",
+       "      <td>2.511700</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>2</td>\n",
+       "      <td>2.543000</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>3</td>\n",
+       "      <td>2.566400</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>4</td>\n",
+       "      <td>2.472700</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>5</td>\n",
+       "      <td>2.500000</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>6</td>\n",
+       "      <td>2.500000</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>7</td>\n",
+       "      <td>2.625000</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>8</td>\n",
+       "      <td>2.277300</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>9</td>\n",
+       "      <td>2.359400</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>10</td>\n",
+       "      <td>2.175800</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>11</td>\n",
+       "      <td>2.293000</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>12</td>\n",
+       "      <td>2.132800</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>13</td>\n",
+       "      <td>1.974600</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>14</td>\n",
+       "      <td>2.076200</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>15</td>\n",
+       "      <td>1.869100</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>16</td>\n",
+       "      <td>1.640600</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>17</td>\n",
+       "      <td>1.769500</td>\n",
+       "    </tr>\n",
+       "  </tbody>\n",
+       "</table><p>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "trainer.train()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1ed2cf09-3683-4016-88d9-9ada1ddb4345",
+   "metadata": {},
+   "source": [
+    "## Saving the finetuned model"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "c37902bf-47e5-4f89-9128-a6b7d91cb437",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "model_id185 = \"Poro-34B-Lora-185\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "163b54c4-3027-4e0d-9d52-7e3d698020da",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "peft_model.save_pretrained(model_id185)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "ec432db5-4f0c-43c7-b4e4-ef087f057bd0",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!ls -lh {model_id185}  # Lora parameters file size"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "11460d4e-3e11-4fdb-b134-61b45bb84018",
+   "metadata": {},
+   "source": [
+    "## Testing"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "eb6a1213-a7ab-4bb5-8ffc-0e2666286dc6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def generate_output(model, inputs, max_new_tokens=100):\n",
+    "    outputs = model.generate(\n",
+    "        input_ids=inputs[\"input_ids\"],\n",
+    "        max_new_tokens=max_new_tokens,\n",
+    "        temperature=0.1,\n",
+    "    )\n",
+    "    return outputs"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0a844312-2a1e-4c76-9078-96506b252522",
+   "metadata": {},
+   "source": [
+    "### Original model"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "id": "d38bbed0-e938-43ef-b816-b5e0f9d066fd",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "['Given the question delimited by triple backticks ```{ Mikä on osuusmaksu ja miksi se pitää maksaa? }```, what is the answer? Answer: Osuusmaksu on osuuskaupan osuusmaksu, joka maksetaan liittymisen yhteydessä. Osuusmaksu on sijoitus osuuskauppaan ja se palautetaan, kun jäsen eroaa osuuskaupasta. Osuusmaksun suuruus vaihtelee osuuskaupoittain. Osuusmaksun suuruus on 100 euroa. Osuusmaksu on sijoitus osuuskauppaan ja se palautetaan, kun jäsen eroaa osuuskaupasta. Osuusmaksun suuruus vaihtelee osuuskaupoittain. Osuusmaksun suuruus on 100 euroa. Osuusmaksu on sijoitus osuuskauppaan ja se palautetaan, kun jäsen eroaa osuuskaupasta. Osuusmaksun suuruus vaihtelee osuuskaupoittain.']\n"
+     ]
+    }
+   ],
+   "source": [
+    "prompt = tokenizer('Given the question delimited by triple backticks ```{ Mikä on osuusmaksu ja miksi se pitää maksaa? }```, what is the answer? Answer:', return_tensors=\"pt\")\n",
+    "result = generate_output(model,prompt)\n",
+    "print(tokenizer.batch_decode(result, skip_special_tokens=True))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "d6091480-e399-4890-bd32-7a51d1cbb50f",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "['Given the question delimited by triple backticks ```{ Mistä näen S-Tilini tapahtumat ja saldon? }```, what is the answer? Answer: S-Pankin verkkopankissa. The answer is the first sentence in the paragraph. The answer is the first sentence in the paragraph. The answer is the first sentence in the paragraph. The answer is the first sentence in the paragraph. The answer is the first sentence in the paragraph. The answer is the first sentence in the paragraph. The answer is the first sentence in the paragraph. The answer is the first sentence in the paragraph. The answer is the first sentence in the paragraph. The answer is the']\n"
+     ]
+    }
+   ],
+   "source": [
+    "prompt = tokenizer('Given the question delimited by triple backticks ```{ Mistä näen S-Tilini tapahtumat ja saldon? }```, what is the answer? Answer:', return_tensors=\"pt\")\n",
+    "result = generate_output(model,prompt)\n",
+    "print(tokenizer.batch_decode(result, skip_special_tokens=True))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ae3c3d6a-2b07-4e46-9ddc-dccadfd07196",
+   "metadata": {},
+   "source": [
+    "### Finetuned model"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 27,
+   "id": "4cf53f39-ad3f-43e2-8daa-79853b054cd2",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from peft import PeftModel"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 28,
+   "id": "142bf57d-cffc-47b2-ae91-8a5420c46d32",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "loaded_model = PeftModel.from_pretrained(model,model_id185,is_trainable=False)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 29,
+   "id": "c3cacd26-edff-494c-9428-55b7659988de",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "['Given the question delimited by triple backticks ```{ Mikä on osuusmaksu ja miksi se pitää maksaa? }```, what is the answer? Answer: {Osuusmaksun suuruus on 100 euroa. Se maksetaan vain kerran, ja se palautetaan osuuskaupan asiakasomistajuudesta luovuttaessa. Osuusmaksun voi maksaa kerralla kokonaan tai kerryttää sitä Bonuksilla. Osuusmaksun voi maksaa myös S-Etukortilla, jolloin se veloitetaan S-Etukorttiin liitetyltä maksuvälineeltä.}\\n\\n{Osuusmaksun voi maksaa myös S-ryhmän lahjakortilla. Lahjakortin saldo ei kuitenkaan kerrytä Bonusta.}\\n\\n{Osuusmaksun voi maksaa myös S-ryhmän lahjak']\n"
+     ]
+    }
+   ],
+   "source": [
+    "prompt = tokenizer('Given the question delimited by triple backticks ```{ Mikä on osuusmaksu ja miksi se pitää maksaa? }```, what is the answer? Answer:', return_tensors=\"pt\")\n",
+    "result = generate_output(loaded_model,prompt)\n",
+    "print(tokenizer.batch_decode(result, skip_special_tokens=True))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 30,
+   "id": "57029580-6429-4c43-8482-1a052839bc05",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "['Given the question delimited by triple backticks ```{ Mistä näen S-Tilini tapahtumat ja saldon? }```, what is the answer? Answer: {S-Tilin tapahtumat ja saldo ovat nähtävissä S-mobiilissa ja S-Pankin verkkopankissa.}\\n\\n{S-mobiilissa S-Tilin tapahtumat ja saldo ovat nähtävissä S-Etukortti-osiossa.}\\n\\n{S-Pankin verkkopankissa S-Tilin tapahtumat ja saldo ovat nähtävissä Tilit-osiossa.}\\n\\n{S-Tilin tapahtumat ja saldo ovat nähtävissä myös S-mobiilin ja S-Pankin verkkopankin S-Etukortti- ja Tilit-osiossa, kun olet']\n"
+     ]
+    }
+   ],
+   "source": [
+    "prompt = tokenizer('Given the question delimited by triple backticks ```{ Mistä näen S-Tilini tapahtumat ja saldon? }```, what is the answer? Answer:', return_tensors=\"pt\")\n",
+    "result = generate_output(loaded_model,prompt)\n",
+    "print(tokenizer.batch_decode(result, skip_special_tokens=True))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "166c476c-01a2-49cc-b03f-6cb1d9ae6136",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "897448ea-f680-4a5b-a148-f65df6704bac",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.13"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

Poro-34B-Lora-1QA.ipynb

@@ -0,0 +1,620 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "0c4ecb49-ce58-4b65-849a-760980576e48",
+   "metadata": {},
+   "source": [
+    "# Poro34B Lora fine-tuning with S-Group's data - 1 Q/A"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5b686006-65a7-43af-8207-1c7309a5e423",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# This script finetunes the Poro34B model with 1 Question and Answer pair"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "defcdb6f-3b69-4b03-b2dc-07c4b3027fd6",
+   "metadata": {},
+   "source": [
+    "## Initialization"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "67f730e6-3467-4a19-ab76-e8baace8e02e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# pip install peft, all other Python libraries are already in AWS image\n",
+    "!pip install peft"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "80b24df2-140b-4792-aaf1-6f6aff92ece8",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "2024-02-29 15:06:36.945989: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.\n",
+      "To enable the following instructions: SSE4.1 SSE4.2 AVX AVX2 AVX512F FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.\n"
+     ]
+    }
+   ],
+   "source": [
+    "import torch\n",
+    "import json\n",
+    "from transformers import AutoModelForCausalLM, AutoTokenizer \n",
+    "from transformers import TrainingArguments, Trainer\n",
+    "from transformers import pipeline\n",
+    "from peft import  get_peft_model, PromptTuningConfig, TaskType, PromptTuningInit\n",
+    "from datasets import load_dataset"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "d31adfc6-a460-419e-871b-d0437501b026",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# this checks wether we have GPU\n",
+    "device = torch.device(\"cuda\") if torch.cuda.is_available() else torch.device(\"cpu\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "2c5a9b07-c92b-4d1d-b5b5-96e8c234e14f",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "cpu\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(device)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6ea88a10-f5f1-4342-939b-60d2b9c5bb91",
+   "metadata": {},
+   "source": [
+    "## Foundation model import"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "2c0f7b3a-9d56-46ce-9dc8-5fe40b2628a6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Foundation model\n",
+    "model_name='LumiOpen/Poro-34B'"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "4e4c9089-a195-4fd7-91b2-6240cafb4989",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "tokenizer = AutoTokenizer.from_pretrained(model_name)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "a42e0fb6-40d4-483b-a034-84ff351c021d",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "3a476b270f8d413c8d54e413fe791a82",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Loading checkpoint shards:   0%|          | 0/14 [00:00<?, ?it/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "branch = \"1000B\"\n",
+    "model = AutoModelForCausalLM.from_pretrained(model_name,\n",
+    "    torch_dtype=torch.bfloat16,\n",
+    "    revision=branch,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "258df7e6-27c1-48a2-b20a-d377dc885884",
+   "metadata": {},
+   "source": [
+    "## Setting up the Lora parameters"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "63151e65-6bff-4b65-a8ae-af4d6c53036f",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "2024-02-29 15:28:54.008508: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.\n",
+      "To enable the following instructions: SSE4.1 SSE4.2 AVX AVX2 AVX512F FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.\n"
+     ]
+    }
+   ],
+   "source": [
+    "from peft import LoraConfig, get_peft_model"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 38,
+   "id": "f35f934f-23c6-47db-95bb-df20526e29e7",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "config = LoraConfig(\n",
+    "    r=8,\n",
+    "    lora_alpha=8,\n",
+    "    target_modules=[\"query_key_value\"],\n",
+    "    lora_dropout=0.05,\n",
+    "    bias=\"none\",\n",
+    "    task_type=\"CAUSAL_LM\"\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 39,
+   "id": "a62cb983-6e28-40f1-9f8d-64a1a6ccd0f3",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "peft_model = get_peft_model(model, config)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 40,
+   "id": "fe7e7078-2998-4f17-abda-f03a32e04735",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "trainable params: 12386304\n",
+      "all params: 34229336064\n",
+      "trainable: 0.04%\n"
+     ]
+    }
+   ],
+   "source": [
+    "trainable_params = 0\n",
+    "all_param = 0\n",
+    "\n",
+    "# iterating over all parameters\n",
+    "for _, param in peft_model.named_parameters():\n",
+    "    # adding parameters to total\n",
+    "    all_param += param.numel()\n",
+    "    # adding parameters to trainable if they require a graident\n",
+    "    if param.requires_grad:\n",
+    "        trainable_params += param.numel()\n",
+    "\n",
+    "# print number of trainable parameters\n",
+    "print(f\"trainable params: {trainable_params}\")\n",
+    "print(f\"all params: {all_param}\")\n",
+    "print(f\"trainable: {100 * trainable_params / all_param:.2f}%\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d029921d-60db-43ad-ae8b-2ab9910bd490",
+   "metadata": {},
+   "source": [
+    "## Preparing the training data"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "216aa90b-a87d-4a37-b178-e7e83bf987ce",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# prepare the data for training\n",
+    "def prepare_train_data(data):\n",
+    "    text_input = data['text']\n",
+    "    tokenized_input = tokenizer(text_input, return_tensors='pt', padding=True)\n",
+    "    tokenized_input['labels'] = tokenized_input['input_ids']\n",
+    "    return tokenized_input"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 41,
+   "id": "5551fac2-9a26-4a86-b8b4-d2f572ecfaa9",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "dataset = load_dataset(\"json\", data_files=\"prompts_1.json\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 42,
+   "id": "dfa8382a-3e66-4433-b6dd-97d59a7945f6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "train_dataset = dataset['train'].map(prepare_train_data, batched=True, remove_columns=[\"text\"])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "319de4fe-f5e0-4ea6-9a61-6f18c241bd02",
+   "metadata": {},
+   "source": [
+    "## Setting up the training parameters"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 50,
+   "id": "9bb42764-af93-463f-8cf6-68707f21151b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from transformers import DataCollatorForLanguageModeling"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 53,
+   "id": "4b8a94c9-2648-4626-9238-4475645aa695",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "trainer = Trainer(\n",
+    "    model=peft_model,\n",
+    "    train_dataset=train_dataset,\n",
+    "    args=TrainingArguments(\n",
+    "        per_device_train_batch_size=4,\n",
+    "        gradient_accumulation_steps=4,\n",
+    "        warmup_steps=20,\n",
+    "        max_steps=20,\n",
+    "        learning_rate=1e-3,\n",
+    "        logging_steps=1,\n",
+    "        output_dir='outputs',\n",
+    "    ),\n",
+    "    data_collator=DataCollatorForLanguageModeling(tokenizer, mlm=False)\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 54,
+   "id": "fbd62db8-65e3-4d05-b11e-179aaf8f0e65",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/html": [
+       "\n",
+       "    <div>\n",
+       "      \n",
+       "      <progress value='20' max='20' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
+       "      [20/20 27:40, Epoch 20/20]\n",
+       "    </div>\n",
+       "    <table border=\"1\" class=\"dataframe\">\n",
+       "  <thead>\n",
+       " <tr style=\"text-align: left;\">\n",
+       "      <th>Step</th>\n",
+       "      <th>Training Loss</th>\n",
+       "    </tr>\n",
+       "  </thead>\n",
+       "  <tbody>\n",
+       "    <tr>\n",
+       "      <td>1</td>\n",
+       "      <td>0.816400</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>2</td>\n",
+       "      <td>0.808600</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>3</td>\n",
+       "      <td>0.808600</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>4</td>\n",
+       "      <td>0.804700</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>5</td>\n",
+       "      <td>0.793000</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>6</td>\n",
+       "      <td>0.757800</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>7</td>\n",
+       "      <td>0.699200</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>8</td>\n",
+       "      <td>0.640600</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>9</td>\n",
+       "      <td>0.570300</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>10</td>\n",
+       "      <td>0.492200</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>11</td>\n",
+       "      <td>0.392600</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>12</td>\n",
+       "      <td>0.291000</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>13</td>\n",
+       "      <td>0.196300</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>14</td>\n",
+       "      <td>0.140600</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>15</td>\n",
+       "      <td>0.112800</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>16</td>\n",
+       "      <td>0.090300</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>17</td>\n",
+       "      <td>0.064500</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>18</td>\n",
+       "      <td>0.048800</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>19</td>\n",
+       "      <td>0.024900</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>20</td>\n",
+       "      <td>0.018900</td>\n",
+       "    </tr>\n",
+       "  </tbody>\n",
+       "</table><p>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/plain": [
+       "TrainOutput(global_step=20, training_loss=0.428607177734375, metrics={'train_runtime': 1747.2099, 'train_samples_per_second': 0.183, 'train_steps_per_second': 0.011, 'total_flos': 219858091622400.0, 'train_loss': 0.428607177734375, 'epoch': 20.0})"
+      ]
+     },
+     "execution_count": 54,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "trainer.train()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1ed2cf09-3683-4016-88d9-9ada1ddb4345",
+   "metadata": {},
+   "source": [
+    "## Saving the finetuned model"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "id": "c37902bf-47e5-4f89-9128-a6b7d91cb437",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "model_id = \"Poro-34B-Lora-1\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "163b54c4-3027-4e0d-9d52-7e3d698020da",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "peft_model.save_pretrained(model_id)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "ec432db5-4f0c-43c7-b4e4-ef087f057bd0",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!ls -lh {model_id}  # Lora parameters file size"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "11460d4e-3e11-4fdb-b134-61b45bb84018",
+   "metadata": {},
+   "source": [
+    "## Testing"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "eb6a1213-a7ab-4bb5-8ffc-0e2666286dc6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def generate_output(model, inputs, max_new_tokens=100):\n",
+    "    outputs = model.generate(\n",
+    "        input_ids=inputs[\"input_ids\"],\n",
+    "        max_new_tokens=max_new_tokens,\n",
+    "        temperature=0.1,\n",
+    "    )\n",
+    "    return outputs"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0a844312-2a1e-4c76-9078-96506b252522",
+   "metadata": {},
+   "source": [
+    "### Original model"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "id": "d38bbed0-e938-43ef-b816-b5e0f9d066fd",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "['Given the question delimited by triple backticks ```{ Kuinka vaihdan uutiskirjeen sähköpostiosoitteen? }```, what is the answer? Answer: ```{ Voit vaihtaa uutiskirjeen sähköpostiosoitteen kirjautumalla sisään ja menemällä Oma tili -osioon. }```\\n']\n"
+     ]
+    }
+   ],
+   "source": [
+    "prompt = tokenizer('Given the question delimited by triple backticks ```{ Kuinka vaihdan uutiskirjeen sähköpostiosoitteen? }```, what is the answer? Answer:', return_tensors=\"pt\")\n",
+    "result = generate_output(model,prompt)\n",
+    "print(tokenizer.batch_decode(result, skip_special_tokens=True))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ae3c3d6a-2b07-4e46-9ddc-dccadfd07196",
+   "metadata": {},
+   "source": [
+    "### Finetuned model"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "id": "4cf53f39-ad3f-43e2-8daa-79853b054cd2",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from peft import PeftModel"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "id": "142bf57d-cffc-47b2-ae91-8a5420c46d32",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "loaded_model = PeftModel.from_pretrained(model,model_id,is_trainable=False)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "id": "c3cacd26-edff-494c-9428-55b7659988de",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "['Given the question delimited by triple backticks ```{ Kuinka vaihdan uutiskirjeen sähköpostiosoitteen? }```, what is the answer? Answer: { Peruuta ensin vanhaan osoitteeseen tilattu uutiskirje kirjeen alareunan “Peruuta tilaus” -linkistä.\\nTilaa uutiskirje uudelleen oikeaan osoitteeseen. }.\\nKuinka vaihdan uutiskirjeen sähköpostiosoitteen?\\nPeruuta ensin vanhaan osoitteeseen tilattu uutiskirje kirjeen alareunan “Peruuta tilaus” -linkistä.\\nTilaa uutiskirje uudelleen oikeaan osoitteeseen.\\nPeruuta uutiskirjeen tilaus kirjeen alareunan “Peruuta tilaus” -linkistä.\\nTilaa uutiskirje uudelleen oikeaan osoitteeseen.\\nPeruuta uutiskirjeen tilaus kirjeen alareunan “Peruuta tilaus” -linkistä.\\nTilaa uutiskirje']\n"
+     ]
+    }
+   ],
+   "source": [
+    "prompt = tokenizer('Given the question delimited by triple backticks ```{ Kuinka vaihdan uutiskirjeen sähköpostiosoitteen? }```, what is the answer? Answer:', return_tensors=\"pt\")\n",
+    "result = generate_output(loaded_model,prompt)\n",
+    "print(tokenizer.batch_decode(result, skip_special_tokens=True))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "166c476c-01a2-49cc-b03f-6cb1d9ae6136",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.13"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

Poro-34B-Lora-2/README.md

@@ -0,0 +1,204 @@
+---
+library_name: peft
+base_model: LumiOpen/Poro-34B
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure 
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+
+
+### Framework versions
+
+- PEFT 0.9.0

Poro-34B-Lora-2/adapter_config.json

@@ -0,0 +1,27 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "LumiOpen/Poro-34B",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 8,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "query_key_value"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Poro-34B-Lora-2/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c2097d27cf08d1fd1f7026c70dc0d1d994a56cad1db1361424a2baea1eb0ed64
+size 24788784

Poro-34B-Lora-2QA.ipynb

@@ -0,0 +1,808 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "983e6b42-12bb-4513-af47-c8c4e34e3177",
+   "metadata": {},
+   "source": [
+    "# Poro34B Lora fine-tuning with S-Group's data - 2 Q/A"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "e5f5a80c-0501-41a1-80a9-fb5792b45fea",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# This script finetunes the Poro34B model with 1 Question and Answer pair"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6441fdf4-ed64-447a-b2c6-542738dc2658",
+   "metadata": {},
+   "source": [
+    "## Initialization"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "67f730e6-3467-4a19-ab76-e8baace8e02e",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Requirement already satisfied: peft in /opt/conda/lib/python3.10/site-packages (0.9.0)\n",
+      "Requirement already satisfied: numpy>=1.17 in /opt/conda/lib/python3.10/site-packages (from peft) (1.26.3)\n",
+      "Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.10/site-packages (from peft) (23.2)\n",
+      "Requirement already satisfied: psutil in /opt/conda/lib/python3.10/site-packages (from peft) (5.9.8)\n",
+      "Requirement already satisfied: pyyaml in /opt/conda/lib/python3.10/site-packages (from peft) (6.0.1)\n",
+      "Requirement already satisfied: torch>=1.13.0 in /opt/conda/lib/python3.10/site-packages (from peft) (2.0.0.post101)\n",
+      "Requirement already satisfied: transformers in /opt/conda/lib/python3.10/site-packages (from peft) (4.31.0)\n",
+      "Requirement already satisfied: tqdm in /opt/conda/lib/python3.10/site-packages (from peft) (4.66.1)\n",
+      "Requirement already satisfied: accelerate>=0.21.0 in /opt/conda/lib/python3.10/site-packages (from peft) (0.21.0)\n",
+      "Requirement already satisfied: safetensors in /opt/conda/lib/python3.10/site-packages (from peft) (0.3.3)\n",
+      "Requirement already satisfied: huggingface-hub>=0.17.0 in /opt/conda/lib/python3.10/site-packages (from peft) (0.20.2)\n",
+      "Requirement already satisfied: filelock in /opt/conda/lib/python3.10/site-packages (from huggingface-hub>=0.17.0->peft) (3.13.1)\n",
+      "Requirement already satisfied: fsspec>=2023.5.0 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub>=0.17.0->peft) (2023.6.0)\n",
+      "Requirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from huggingface-hub>=0.17.0->peft) (2.31.0)\n",
+      "Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub>=0.17.0->peft) (4.5.0)\n",
+      "Requirement already satisfied: sympy in /opt/conda/lib/python3.10/site-packages (from torch>=1.13.0->peft) (1.12)\n",
+      "Requirement already satisfied: networkx in /opt/conda/lib/python3.10/site-packages (from torch>=1.13.0->peft) (3.2.1)\n",
+      "Requirement already satisfied: jinja2 in /opt/conda/lib/python3.10/site-packages (from torch>=1.13.0->peft) (3.1.3)\n",
+      "Requirement already satisfied: regex!=2019.12.17 in /opt/conda/lib/python3.10/site-packages (from transformers->peft) (2023.12.25)\n",
+      "Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /opt/conda/lib/python3.10/site-packages (from transformers->peft) (0.13.3)\n",
+      "Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/lib/python3.10/site-packages (from jinja2->torch>=1.13.0->peft) (2.1.4)\n",
+      "Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests->huggingface-hub>=0.17.0->peft) (3.3.2)\n",
+      "Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests->huggingface-hub>=0.17.0->peft) (3.6)\n",
+      "Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->huggingface-hub>=0.17.0->peft) (1.26.18)\n",
+      "Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.10/site-packages (from requests->huggingface-hub>=0.17.0->peft) (2023.11.17)\n",
+      "Requirement already satisfied: mpmath>=0.19 in /opt/conda/lib/python3.10/site-packages (from sympy->torch>=1.13.0->peft) (1.3.0)\n"
+     ]
+    }
+   ],
+   "source": [
+    "!pip install peft"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "80b24df2-140b-4792-aaf1-6f6aff92ece8",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "2024-02-29 16:09:52.067525: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.\n",
+      "To enable the following instructions: SSE4.1 SSE4.2 AVX AVX2 AVX512F FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.\n"
+     ]
+    }
+   ],
+   "source": [
+    "import torch\n",
+    "import json\n",
+    "from transformers import AutoModelForCausalLM, AutoTokenizer \n",
+    "from transformers import TrainingArguments, Trainer\n",
+    "from transformers import pipeline\n",
+    "from peft import  get_peft_model, PromptTuningConfig, TaskType, PromptTuningInit\n",
+    "from datasets import load_dataset"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "d31adfc6-a460-419e-871b-d0437501b026",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "device = torch.device(\"cuda\") if torch.cuda.is_available() else torch.device(\"cpu\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "2c5a9b07-c92b-4d1d-b5b5-96e8c234e14f",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "cpu\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(device)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "4b1ec235-cdd1-4b68-b9b0-fe214cbeb2be",
+   "metadata": {},
+   "source": [
+    "## Foundation model import"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "2c0f7b3a-9d56-46ce-9dc8-5fe40b2628a6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "model_name='LumiOpen/Poro-34B'"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "4e4c9089-a195-4fd7-91b2-6240cafb4989",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "tokenizer = AutoTokenizer.from_pretrained(model_name)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "a42e0fb6-40d4-483b-a034-84ff351c021d",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "2784e3e4025e44d4aa5edb0aa58a2aaa",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Loading checkpoint shards:   0%|          | 0/14 [00:00<?, ?it/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "branch = \"1000B\"\n",
+    "model = AutoModelForCausalLM.from_pretrained(model_name,\n",
+    "    torch_dtype=torch.bfloat16,\n",
+    "    revision=branch,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "32bce199-3107-4329-87e4-1c33b9e70c30",
+   "metadata": {},
+   "source": [
+    "## Setting up the Lora parameters"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "63151e65-6bff-4b65-a8ae-af4d6c53036f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from peft import LoraConfig, get_peft_model"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "id": "f35f934f-23c6-47db-95bb-df20526e29e7",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "config = LoraConfig(\n",
+    "    r=8,\n",
+    "    lora_alpha=8,\n",
+    "    target_modules=[\"query_key_value\"],\n",
+    "    lora_dropout=0.05,\n",
+    "    bias=\"none\",\n",
+    "    task_type=\"CAUSAL_LM\"\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "a62cb983-6e28-40f1-9f8d-64a1a6ccd0f3",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "peft_model = get_peft_model(model, config)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "id": "fe7e7078-2998-4f17-abda-f03a32e04735",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "trainable params: 12386304\n",
+      "all params: 34229336064\n",
+      "trainable: 0.04%\n"
+     ]
+    }
+   ],
+   "source": [
+    "trainable_params = 0\n",
+    "all_param = 0\n",
+    "\n",
+    "# iterating over all parameters\n",
+    "for _, param in peft_model.named_parameters():\n",
+    "    # adding parameters to total\n",
+    "    all_param += param.numel()\n",
+    "    # adding parameters to trainable if they require a graident\n",
+    "    if param.requires_grad:\n",
+    "        trainable_params += param.numel()\n",
+    "\n",
+    "# print number of trainable parameters\n",
+    "print(f\"trainable params: {trainable_params}\")\n",
+    "print(f\"all params: {all_param}\")\n",
+    "print(f\"trainable: {100 * trainable_params / all_param:.2f}%\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6c1b2946-5575-4de0-bf99-a05fa3994964",
+   "metadata": {},
+   "source": [
+    "## Preparing the training data"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "id": "216aa90b-a87d-4a37-b178-e7e83bf987ce",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# prepare the data for training\n",
+    "def prepare_train_data(data):\n",
+    "    text_input = data['text']\n",
+    "    tokenized_input = tokenizer(text_input, return_tensors='pt', padding=True)\n",
+    "    tokenized_input['labels'] = tokenized_input['input_ids']\n",
+    "    return tokenized_input"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "id": "5551fac2-9a26-4a86-b8b4-d2f572ecfaa9",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "df4aebc8dac54280a04c82106b935f82",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Generating train split: 0 examples [00:00, ? examples/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "dataset = load_dataset(\"json\", data_files=\"prompts_2.json\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "id": "dfa8382a-3e66-4433-b6dd-97d59a7945f6",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "a20ed74f503748eeac691e1fbf026fec",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Map:   0%|          | 0/2 [00:00<?, ? examples/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "train_dataset = dataset['train'].map(prepare_train_data, batched=True, remove_columns=[\"text\"])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "19e453fc-728e-4ad9-b44c-e6b95b37707b",
+   "metadata": {},
+   "source": [
+    "## Setting up the training parameters"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "id": "9bb42764-af93-463f-8cf6-68707f21151b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from transformers import DataCollatorForLanguageModeling"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "id": "4b8a94c9-2648-4626-9238-4475645aa695",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "trainer = Trainer(\n",
+    "    model=peft_model,\n",
+    "    train_dataset=train_dataset,\n",
+    "    args=TrainingArguments(\n",
+    "        per_device_train_batch_size=4,\n",
+    "        gradient_accumulation_steps=4,\n",
+    "        warmup_steps=20,\n",
+    "        max_steps=20,\n",
+    "        learning_rate=1e-3,\n",
+    "        logging_steps=1,\n",
+    "        output_dir='outputs',\n",
+    "    ),\n",
+    "    data_collator=DataCollatorForLanguageModeling(tokenizer, mlm=False)\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 17,
+   "id": "fbd62db8-65e3-4d05-b11e-179aaf8f0e65",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/opt/conda/lib/python3.10/site-packages/transformers/optimization.py:411: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning\n",
+      "  warnings.warn(\n",
+      "\u001b[34m\u001b[1mwandb\u001b[0m: Currently logged in as: \u001b[33mtimo-au-laine\u001b[0m. Use \u001b[1m`wandb login --relogin`\u001b[0m to force relogin\n"
+     ]
+    },
+    {
+     "data": {
+      "text/html": [
+       "wandb version 0.16.3 is available!  To upgrade, please run:\n",
+       " $ pip install wandb --upgrade"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "Tracking run with wandb version 0.16.2"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "Run data is saved locally in <code>/home/sagemaker-user/wandb/run-20240229_153102-95aeur18</code>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "Syncing run <strong><a href='https://wandb.ai/timo-au-laine/huggingface/runs/95aeur18' target=\"_blank\">brisk-fog-8</a></strong> to <a href='https://wandb.ai/timo-au-laine/huggingface' target=\"_blank\">Weights & Biases</a> (<a href='https://wandb.me/run' target=\"_blank\">docs</a>)<br/>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       " View project at <a href='https://wandb.ai/timo-au-laine/huggingface' target=\"_blank\">https://wandb.ai/timo-au-laine/huggingface</a>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       " View run at <a href='https://wandb.ai/timo-au-laine/huggingface/runs/95aeur18' target=\"_blank\">https://wandb.ai/timo-au-laine/huggingface/runs/95aeur18</a>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "You're using a BloomTokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.\n"
+     ]
+    },
+    {
+     "data": {
+      "text/html": [
+       "\n",
+       "    <div>\n",
+       "      \n",
+       "      <progress value='20' max='20' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
+       "      [20/20 29:32, Epoch 20/20]\n",
+       "    </div>\n",
+       "    <table border=\"1\" class=\"dataframe\">\n",
+       "  <thead>\n",
+       " <tr style=\"text-align: left;\">\n",
+       "      <th>Step</th>\n",
+       "      <th>Training Loss</th>\n",
+       "    </tr>\n",
+       "  </thead>\n",
+       "  <tbody>\n",
+       "    <tr>\n",
+       "      <td>1</td>\n",
+       "      <td>0.687500</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>2</td>\n",
+       "      <td>0.683600</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>3</td>\n",
+       "      <td>0.683600</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>4</td>\n",
+       "      <td>0.683600</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>5</td>\n",
+       "      <td>0.683600</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>6</td>\n",
+       "      <td>0.671900</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>7</td>\n",
+       "      <td>0.656200</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>8</td>\n",
+       "      <td>0.617200</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>9</td>\n",
+       "      <td>0.585900</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>10</td>\n",
+       "      <td>0.543000</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>11</td>\n",
+       "      <td>0.488300</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>12</td>\n",
+       "      <td>0.423800</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>13</td>\n",
+       "      <td>0.357400</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>14</td>\n",
+       "      <td>0.281200</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>15</td>\n",
+       "      <td>0.204100</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>16</td>\n",
+       "      <td>0.137700</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>17</td>\n",
+       "      <td>0.088400</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>18</td>\n",
+       "      <td>0.058100</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>19</td>\n",
+       "      <td>0.043500</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>20</td>\n",
+       "      <td>0.027600</td>\n",
+       "    </tr>\n",
+       "  </tbody>\n",
+       "</table><p>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/plain": [
+       "TrainOutput(global_step=20, training_loss=0.43031005859375, metrics={'train_runtime': 1884.4748, 'train_samples_per_second': 0.17, 'train_steps_per_second': 0.011, 'total_flos': 1415086626078720.0, 'train_loss': 0.43031005859375, 'epoch': 20.0})"
+      ]
+     },
+     "execution_count": 17,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "trainer.train()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "efc3ef10-e402-481a-8abd-4174444d668a",
+   "metadata": {},
+   "source": [
+    "## Saving the finetuned model"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 18,
+   "id": "c37902bf-47e5-4f89-9128-a6b7d91cb437",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "model_id2 = \"Poro-34B-Lora-2\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 19,
+   "id": "163b54c4-3027-4e0d-9d52-7e3d698020da",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "peft_model.save_pretrained(model_id2)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "ec432db5-4f0c-43c7-b4e4-ef087f057bd0",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!ls -lh {model_id2}"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "059b6ee5-bfc0-4dcf-901c-ef869bedbb90",
+   "metadata": {},
+   "source": [
+    "## Testing"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "eb6a1213-a7ab-4bb5-8ffc-0e2666286dc6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def generate_output(model, inputs, max_new_tokens=100):\n",
+    "    outputs = model.generate(\n",
+    "        input_ids=inputs[\"input_ids\"],\n",
+    "        max_new_tokens=max_new_tokens,\n",
+    "        temperature=0.1,\n",
+    "    )\n",
+    "    return outputs"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "619f7e85-3310-40d1-8706-1289676b98ca",
+   "metadata": {},
+   "source": [
+    "### Original model"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "d38bbed0-e938-43ef-b816-b5e0f9d066fd",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "['Given the question delimited by triple backticks ```{ Kuinka vaihdan uutiskirjeen sähköpostiosoitteen? }```, what is the answer? Answer: ```{ Voit vaihtaa uutiskirjeen sähköpostiosoitteen kirjautumalla sisään ja menemällä Oma tili -osioon. }```\\n']\n"
+     ]
+    }
+   ],
+   "source": [
+    "prompt = tokenizer('Given the question delimited by triple backticks ```{ Kuinka vaihdan uutiskirjeen sähköpostiosoitteen? }```, what is the answer? Answer:', return_tensors=\"pt\")\n",
+    "result = generate_output(model,prompt)\n",
+    "print(tokenizer.batch_decode(result, skip_special_tokens=True))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "6d354261-2dbb-4ecd-a75e-3af454c3c051",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "['Given the question delimited by triple backticks ```{ Miksi sähköpostiosoite tulee vahvistaa? }```, what is the answer? Answer: Because the email address needs to be confirmed.']\n"
+     ]
+    }
+   ],
+   "source": [
+    "prompt = tokenizer('Given the question delimited by triple backticks ```{ Miksi sähköpostiosoite tulee vahvistaa? }```, what is the answer? Answer:', return_tensors=\"pt\")\n",
+    "result = generate_output(model,prompt)\n",
+    "print(tokenizer.batch_decode(result, skip_special_tokens=True))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3afd691c-b853-41e1-abc5-3e462dd8f9a0",
+   "metadata": {},
+   "source": [
+    "### Finetuned model"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 23,
+   "id": "4cf53f39-ad3f-43e2-8daa-79853b054cd2",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from peft import PeftModel"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 24,
+   "id": "142bf57d-cffc-47b2-ae91-8a5420c46d32",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "loaded_model = PeftModel.from_pretrained(model,model_id2,is_trainable=False)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 25,
+   "id": "c3cacd26-edff-494c-9428-55b7659988de",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "['Given the question delimited by triple backticks ```{ Kuinka vaihdan uutiskirjeen sähköpostiosoitteen? }```, what is the answer? Answer: { Peruuta ensin vanhaan osoitteeseen tilattu uutiskirje kirjeen alareunan “Peruuta tilaus” -linkistä.\\nTilaa uutiskirje uudelleen oikeaan osoitteeseen. }\\n\\n### Answering a Given Question\\n\\nGiven a question delimited by triple backticks ```{ Kuinka vaihdan uutiskirjeen sähköpostiosoitteen? }```, what is the answer? Answer: { Peruuta ensin vanhaan osoitteeseen tilattu uutiskirje kirjeen alareunan “Peruuta tilaus” -linkistä.\\nTilaa uutiskirje uudelleen oikeaan osoitteeseen. }\\n\\n## Data Sources\\n\\nThe data for currently supported']\n"
+     ]
+    }
+   ],
+   "source": [
+    "prompt = tokenizer('Given the question delimited by triple backticks ```{ Kuinka vaihdan uutiskirjeen sähköpostiosoitteen? }```, what is the answer? Answer:', return_tensors=\"pt\")\n",
+    "result = generate_output(loaded_model,prompt)\n",
+    "print(tokenizer.batch_decode(result, skip_special_tokens=True))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 26,
+   "id": "b1ad5b3e-8f22-4763-b3e1-91b604731048",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "['Given the question delimited by triple backticks ```{ Miksi sähköpostiosoite tulee vahvistaa? }```, what is the answer? Answer: {Sähköpostiosoitteiden vahvistaminen on yleisesti käytössä oleva tapa varmistua siitä, että henkilöllä itsellään on pääsy hänen tiedoissaan olevaan sähköpostiosoitteeseen.\\n\\nSähköpostiosoite tulee vahvistaa itse, joko S-mobiilissa tai samalla kun luot itsellesi S-käyttäjätilin. Kun lähetät vahvistusviestin omissa tiedoissasi näkyvään sähköpostiosoitteeseen ja vahvistat itse osoitteen oikeaksi sähköpostiisi lähetetyllä vahvistuskoodilla, saamme varmistuksen, että osoitteesi on voimassa ja kuuluu juuri sinulle.\\n\\nJos asiakastiedoissasi olevasta sähköpostiosoitteesta puuttuu vielä vahvistus, näkyy']\n"
+     ]
+    }
+   ],
+   "source": [
+    "prompt = tokenizer('Given the question delimited by triple backticks ```{ Miksi sähköpostiosoite tulee vahvistaa? }```, what is the answer? Answer:', return_tensors=\"pt\")\n",
+    "result = generate_output(loaded_model,prompt)\n",
+    "print(tokenizer.batch_decode(result, skip_special_tokens=True))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "166c476c-01a2-49cc-b03f-6cb1d9ae6136",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "09523d41-51e5-4cc6-b68b-c848063e5095",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.13"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

Prepare_data.ipynb

@@ -0,0 +1,504 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "57176cfe-974a-430b-b51b-11f5eae772f7",
+   "metadata": {
+    "id": "mGXJ4xe93YSr"
+   },
+   "source": [
+    "# Creating data set from excel to json"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "9e6e6551",
+   "metadata": {
+    "id": "9e6e6551"
+   },
+   "outputs": [],
+   "source": [
+    "import pandas as pd\n",
+    "import json"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "a410636f",
+   "metadata": {
+    "id": "a410636f"
+   },
+   "outputs": [],
+   "source": [
+    "pd.set_option('display.max_rows', None)\n",
+    "pd.set_option('display.max_columns', None)\n",
+    "pd.set_option('display.width', None)\n",
+    "pd.set_option('display.max_colwidth', None)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "9a95c015",
+   "metadata": {
+    "id": "9a95c015"
+   },
+   "outputs": [],
+   "source": [
+    "# read data set in excel\n",
+    "dataset = pd.read_excel(\"S-Kanava_2.xlsx\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5cbbfcdb",
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/"
+    },
+    "id": "5cbbfcdb",
+    "outputId": "d4a21e61-6a4e-4d67-cb28-315f4decba3e"
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "(2, 2)"
+      ]
+     },
+     "execution_count": 21,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "dataset.shape"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "6d0f665a",
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/"
+    },
+    "id": "6d0f665a",
+    "outputId": "9dfe6ffa-bfb2-4752-a0cd-97b182b34cce"
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "(2, 2)"
+      ]
+     },
+     "execution_count": 22,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "dataset.dropna()\n",
+    "dataset.shape"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "446d22c9",
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/",
+     "height": 230
+    },
+    "id": "446d22c9",
+    "outputId": "9d08ba83-7011-475d-ed4a-fd5736fa1e95"
+   },
+   "outputs": [
+    {
+     "data": {
+      "application/vnd.google.colaboratory.intrinsic+json": {
+       "summary": "{\n  \"name\": \"dataset\",\n  \"rows\": 2,\n  \"fields\": [\n    {\n      \"column\": \"Kysymys\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 2,\n        \"samples\": [\n          \"Miksi s\\u00e4hk\\u00f6postiosoite tulee vahvistaa?\",\n          \"Kuinka vaihdan uutiskirjeen s\\u00e4hk\\u00f6postiosoitteen?\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"Vastaus\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 2,\n        \"samples\": [\n          \"S\\u00e4hk\\u00f6postiosoitteiden vahvistaminen on yleisesti k\\u00e4yt\\u00f6ss\\u00e4 oleva tapa varmistua siit\\u00e4, ett\\u00e4 henkil\\u00f6ll\\u00e4 itsell\\u00e4\\u00e4n on p\\u00e4\\u00e4sy h\\u00e4nen tiedoissaan olevaan s\\u00e4hk\\u00f6postiosoitteeseen.\\n\\nS\\u00e4hk\\u00f6postiosoite tulee vahvistaa itse, joko S-mobiilissa tai samalla kun luot itsellesi S-k\\u00e4ytt\\u00e4j\\u00e4tilin. Kun l\\u00e4het\\u00e4t vahvistusviestin omissa tiedoissasi n\\u00e4kyv\\u00e4\\u00e4n s\\u00e4hk\\u00f6postiosoitteeseen ja vahvistat itse osoitteen oikeaksi s\\u00e4hk\\u00f6postiisi l\\u00e4hetetyll\\u00e4 vahvistuskoodilla, saamme varmistuksen, ett\\u00e4 osoitteesi on voimassa ja kuuluu juuri sinulle.\\n\\nJos asiakastiedoissasi olevasta s\\u00e4hk\\u00f6postiosoitteesta puuttuu viel\\u00e4 vahvistus, n\\u00e4kyy osoitteen yhteydess\\u00e4 Vahvista -painike.\\n\\nBonustiedot ja muut henkil\\u00f6kohtaiset tiedotteet l\\u00e4hetet\\u00e4\\u00e4n vain vahvistettuun s\\u00e4hk\\u00f6postiosoitteeseen. Bonustilanteesi voit kuitenkin jatkossakin tarkastaa S-mobiilista, S-k\\u00e4ytt\\u00e4j\\u00e4tililt\\u00e4 sek\\u00e4 toimipaikkojen S-Etukorttip\\u00e4\\u00e4tteilt\\u00e4.\",\n          \"Peruuta ensin vanhaan osoitteeseen tilattu uutiskirje kirjeen alareunan \\u201cPeruuta tilaus\\u201d -linkist\\u00e4.\\nTilaa uutiskirje uudelleen oikeaan osoitteeseen.\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}",
+       "type": "dataframe",
+       "variable_name": "dataset"
+      },
+      "text/html": [
+       "\n",
+       "  <div id=\"df-6dd34700-e7fc-44b2-ad18-b7f45e54b1ee\" class=\"colab-df-container\">\n",
+       "    <div>\n",
+       "<style scoped>\n",
+       "    .dataframe tbody tr th:only-of-type {\n",
+       "        vertical-align: middle;\n",
+       "    }\n",
+       "\n",
+       "    .dataframe tbody tr th {\n",
+       "        vertical-align: top;\n",
+       "    }\n",
+       "\n",
+       "    .dataframe thead th {\n",
+       "        text-align: right;\n",
+       "    }\n",
+       "</style>\n",
+       "<table border=\"1\" class=\"dataframe\">\n",
+       "  <thead>\n",
+       "    <tr style=\"text-align: right;\">\n",
+       "      <th></th>\n",
+       "      <th>Kysymys</th>\n",
+       "      <th>Vastaus</th>\n",
+       "    </tr>\n",
+       "  </thead>\n",
+       "  <tbody>\n",
+       "    <tr>\n",
+       "      <th>0</th>\n",
+       "      <td>Kuinka vaihdan uutiskirjeen sähköpostiosoitteen?</td>\n",
+       "      <td>Peruuta ensin vanhaan osoitteeseen tilattu uutiskirje kirjeen alareunan “Peruuta tilaus” -linkistä.\\nTilaa uutiskirje uudelleen oikeaan osoitteeseen.</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>1</th>\n",
+       "      <td>Miksi sähköpostiosoite tulee vahvistaa?</td>\n",
+       "      <td>Sähköpostiosoitteiden vahvistaminen on yleisesti käytössä oleva tapa varmistua siitä, että henkilöllä itsellään on pääsy hänen tiedoissaan olevaan sähköpostiosoitteeseen.\\n\\nSähköpostiosoite tulee vahvistaa itse, joko S-mobiilissa tai samalla kun luot itsellesi S-käyttäjätilin. Kun lähetät vahvistusviestin omissa tiedoissasi näkyvään sähköpostiosoitteeseen ja vahvistat itse osoitteen oikeaksi sähköpostiisi lähetetyllä vahvistuskoodilla, saamme varmistuksen, että osoitteesi on voimassa ja kuuluu juuri sinulle.\\n\\nJos asiakastiedoissasi olevasta sähköpostiosoitteesta puuttuu vielä vahvistus, näkyy osoitteen yhteydessä Vahvista -painike.\\n\\nBonustiedot ja muut henkilökohtaiset tiedotteet lähetetään vain vahvistettuun sähköpostiosoitteeseen. Bonustilanteesi voit kuitenkin jatkossakin tarkastaa S-mobiilista, S-käyttäjätililtä sekä toimipaikkojen S-Etukorttipäätteiltä.</td>\n",
+       "    </tr>\n",
+       "  </tbody>\n",
+       "</table>\n",
+       "</div>\n",
+       "    <div class=\"colab-df-buttons\">\n",
+       "\n",
+       "  <div class=\"colab-df-container\">\n",
+       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-6dd34700-e7fc-44b2-ad18-b7f45e54b1ee')\"\n",
+       "            title=\"Convert this dataframe to an interactive table.\"\n",
+       "            style=\"display:none;\">\n",
+       "\n",
+       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
+       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
+       "  </svg>\n",
+       "    </button>\n",
+       "\n",
+       "  <style>\n",
+       "    .colab-df-container {\n",
+       "      display:flex;\n",
+       "      gap: 12px;\n",
+       "    }\n",
+       "\n",
+       "    .colab-df-convert {\n",
+       "      background-color: #E8F0FE;\n",
+       "      border: none;\n",
+       "      border-radius: 50%;\n",
+       "      cursor: pointer;\n",
+       "      display: none;\n",
+       "      fill: #1967D2;\n",
+       "      height: 32px;\n",
+       "      padding: 0 0 0 0;\n",
+       "      width: 32px;\n",
+       "    }\n",
+       "\n",
+       "    .colab-df-convert:hover {\n",
+       "      background-color: #E2EBFA;\n",
+       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
+       "      fill: #174EA6;\n",
+       "    }\n",
+       "\n",
+       "    .colab-df-buttons div {\n",
+       "      margin-bottom: 4px;\n",
+       "    }\n",
+       "\n",
+       "    [theme=dark] .colab-df-convert {\n",
+       "      background-color: #3B4455;\n",
+       "      fill: #D2E3FC;\n",
+       "    }\n",
+       "\n",
+       "    [theme=dark] .colab-df-convert:hover {\n",
+       "      background-color: #434B5C;\n",
+       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
+       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
+       "      fill: #FFFFFF;\n",
+       "    }\n",
+       "  </style>\n",
+       "\n",
+       "    <script>\n",
+       "      const buttonEl =\n",
+       "        document.querySelector('#df-6dd34700-e7fc-44b2-ad18-b7f45e54b1ee button.colab-df-convert');\n",
+       "      buttonEl.style.display =\n",
+       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
+       "\n",
+       "      async function convertToInteractive(key) {\n",
+       "        const element = document.querySelector('#df-6dd34700-e7fc-44b2-ad18-b7f45e54b1ee');\n",
+       "        const dataTable =\n",
+       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
+       "                                                    [key], {});\n",
+       "        if (!dataTable) return;\n",
+       "\n",
+       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
+       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
+       "          + ' to learn more about interactive tables.';\n",
+       "        element.innerHTML = '';\n",
+       "        dataTable['output_type'] = 'display_data';\n",
+       "        await google.colab.output.renderOutput(dataTable, element);\n",
+       "        const docLink = document.createElement('div');\n",
+       "        docLink.innerHTML = docLinkHtml;\n",
+       "        element.appendChild(docLink);\n",
+       "      }\n",
+       "    </script>\n",
+       "  </div>\n",
+       "\n",
+       "\n",
+       "<div id=\"df-b15340aa-ec20-4a66-ba97-a0c56d03d18c\">\n",
+       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-b15340aa-ec20-4a66-ba97-a0c56d03d18c')\"\n",
+       "            title=\"Suggest charts\"\n",
+       "            style=\"display:none;\">\n",
+       "\n",
+       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
+       "     width=\"24px\">\n",
+       "    <g>\n",
+       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
+       "    </g>\n",
+       "</svg>\n",
+       "  </button>\n",
+       "\n",
+       "<style>\n",
+       "  .colab-df-quickchart {\n",
+       "      --bg-color: #E8F0FE;\n",
+       "      --fill-color: #1967D2;\n",
+       "      --hover-bg-color: #E2EBFA;\n",
+       "      --hover-fill-color: #174EA6;\n",
+       "      --disabled-fill-color: #AAA;\n",
+       "      --disabled-bg-color: #DDD;\n",
+       "  }\n",
+       "\n",
+       "  [theme=dark] .colab-df-quickchart {\n",
+       "      --bg-color: #3B4455;\n",
+       "      --fill-color: #D2E3FC;\n",
+       "      --hover-bg-color: #434B5C;\n",
+       "      --hover-fill-color: #FFFFFF;\n",
+       "      --disabled-bg-color: #3B4455;\n",
+       "      --disabled-fill-color: #666;\n",
+       "  }\n",
+       "\n",
+       "  .colab-df-quickchart {\n",
+       "    background-color: var(--bg-color);\n",
+       "    border: none;\n",
+       "    border-radius: 50%;\n",
+       "    cursor: pointer;\n",
+       "    display: none;\n",
+       "    fill: var(--fill-color);\n",
+       "    height: 32px;\n",
+       "    padding: 0;\n",
+       "    width: 32px;\n",
+       "  }\n",
+       "\n",
+       "  .colab-df-quickchart:hover {\n",
+       "    background-color: var(--hover-bg-color);\n",
+       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
+       "    fill: var(--button-hover-fill-color);\n",
+       "  }\n",
+       "\n",
+       "  .colab-df-quickchart-complete:disabled,\n",
+       "  .colab-df-quickchart-complete:disabled:hover {\n",
+       "    background-color: var(--disabled-bg-color);\n",
+       "    fill: var(--disabled-fill-color);\n",
+       "    box-shadow: none;\n",
+       "  }\n",
+       "\n",
+       "  .colab-df-spinner {\n",
+       "    border: 2px solid var(--fill-color);\n",
+       "    border-color: transparent;\n",
+       "    border-bottom-color: var(--fill-color);\n",
+       "    animation:\n",
+       "      spin 1s steps(1) infinite;\n",
+       "  }\n",
+       "\n",
+       "  @keyframes spin {\n",
+       "    0% {\n",
+       "      border-color: transparent;\n",
+       "      border-bottom-color: var(--fill-color);\n",
+       "      border-left-color: var(--fill-color);\n",
+       "    }\n",
+       "    20% {\n",
+       "      border-color: transparent;\n",
+       "      border-left-color: var(--fill-color);\n",
+       "      border-top-color: var(--fill-color);\n",
+       "    }\n",
+       "    30% {\n",
+       "      border-color: transparent;\n",
+       "      border-left-color: var(--fill-color);\n",
+       "      border-top-color: var(--fill-color);\n",
+       "      border-right-color: var(--fill-color);\n",
+       "    }\n",
+       "    40% {\n",
+       "      border-color: transparent;\n",
+       "      border-right-color: var(--fill-color);\n",
+       "      border-top-color: var(--fill-color);\n",
+       "    }\n",
+       "    60% {\n",
+       "      border-color: transparent;\n",
+       "      border-right-color: var(--fill-color);\n",
+       "    }\n",
+       "    80% {\n",
+       "      border-color: transparent;\n",
+       "      border-right-color: var(--fill-color);\n",
+       "      border-bottom-color: var(--fill-color);\n",
+       "    }\n",
+       "    90% {\n",
+       "      border-color: transparent;\n",
+       "      border-bottom-color: var(--fill-color);\n",
+       "    }\n",
+       "  }\n",
+       "</style>\n",
+       "\n",
+       "  <script>\n",
+       "    async function quickchart(key) {\n",
+       "      const quickchartButtonEl =\n",
+       "        document.querySelector('#' + key + ' button');\n",
+       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
+       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
+       "      try {\n",
+       "        const charts = await google.colab.kernel.invokeFunction(\n",
+       "            'suggestCharts', [key], {});\n",
+       "      } catch (error) {\n",
+       "        console.error('Error during call to suggestCharts:', error);\n",
+       "      }\n",
+       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
+       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
+       "    }\n",
+       "    (() => {\n",
+       "      let quickchartButtonEl =\n",
+       "        document.querySelector('#df-b15340aa-ec20-4a66-ba97-a0c56d03d18c button');\n",
+       "      quickchartButtonEl.style.display =\n",
+       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
+       "    })();\n",
+       "  </script>\n",
+       "</div>\n",
+       "    </div>\n",
+       "  </div>\n"
+      ],
+      "text/plain": [
+       "                                            Kysymys  \\\n",
+       "0  Kuinka vaihdan uutiskirjeen sähköpostiosoitteen?   \n",
+       "1           Miksi sähköpostiosoite tulee vahvistaa?   \n",
+       "\n",
+       "                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       Vastaus  \n",
+       "0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Peruuta ensin vanhaan osoitteeseen tilattu uutiskirje kirjeen alareunan “Peruuta tilaus” -linkistä.\\nTilaa uutiskirje uudelleen oikeaan osoitteeseen.  \n",
+       "1  Sähköpostiosoitteiden vahvistaminen on yleisesti käytössä oleva tapa varmistua siitä, että henkilöllä itsellään on pääsy hänen tiedoissaan olevaan sähköpostiosoitteeseen.\\n\\nSähköpostiosoite tulee vahvistaa itse, joko S-mobiilissa tai samalla kun luot itsellesi S-käyttäjätilin. Kun lähetät vahvistusviestin omissa tiedoissasi näkyvään sähköpostiosoitteeseen ja vahvistat itse osoitteen oikeaksi sähköpostiisi lähetetyllä vahvistuskoodilla, saamme varmistuksen, että osoitteesi on voimassa ja kuuluu juuri sinulle.\\n\\nJos asiakastiedoissasi olevasta sähköpostiosoitteesta puuttuu vielä vahvistus, näkyy osoitteen yhteydessä Vahvista -painike.\\n\\nBonustiedot ja muut henkilökohtaiset tiedotteet lähetetään vain vahvistettuun sähköpostiosoitteeseen. Bonustilanteesi voit kuitenkin jatkossakin tarkastaa S-mobiilista, S-käyttäjätililtä sekä toimipaikkojen S-Etukorttipäätteiltä.  "
+      ]
+     },
+     "execution_count": 23,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "dataset.head()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "d75eeb59",
+   "metadata": {
+    "id": "d75eeb59"
+   },
+   "outputs": [],
+   "source": [
+    "def buildprompt(data):\n",
+    "    prompt = {}\n",
+    "    prompt['text'] = \"Given the question delimited by triple backticks ```{\" + data['Kysymys'] + \"}```, what is the answer? Answer: {\" + data['Vastaus'] + \"}\"\n",
+    "    return prompt"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "dfc2d587",
+   "metadata": {
+    "id": "dfc2d587"
+   },
+   "outputs": [],
+   "source": [
+    "dataset['prompt'] = dataset.apply(buildprompt, axis=1)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "25972ccd",
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/"
+    },
+    "id": "25972ccd",
+    "outputId": "e397d17c-1d6b-4ea5-ce44-8e662f0e656b"
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "{'text': 'Given the question delimited by triple backticks ```{Miksi sähköpostiosoite tulee vahvistaa?}```, what is the answer? Answer: {Sähköpostiosoitteiden vahvistaminen on yleisesti käytössä oleva tapa varmistua siitä, että henkilöllä itsellään on pääsy hänen tiedoissaan olevaan sähköpostiosoitteeseen.\\n\\nSähköpostiosoite tulee vahvistaa itse, joko S-mobiilissa tai samalla kun luot itsellesi S-käyttäjätilin. Kun lähetät vahvistusviestin omissa tiedoissasi näkyvään sähköpostiosoitteeseen ja vahvistat itse osoitteen oikeaksi sähköpostiisi lähetetyllä vahvistuskoodilla, saamme varmistuksen, että osoitteesi on voimassa ja kuuluu juuri sinulle.\\n\\nJos asiakastiedoissasi olevasta sähköpostiosoitteesta puuttuu vielä vahvistus, näkyy osoitteen yhteydessä Vahvista -painike.\\n\\nBonustiedot ja muut henkilökohtaiset tiedotteet lähetetään vain vahvistettuun sähköpostiosoitteeseen. Bonustilanteesi voit kuitenkin jatkossakin tarkastaa S-mobiilista, S-käyttäjätililtä sekä toimipaikkojen S-Etukorttipäätteiltä.}'}\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(dataset['prompt'][1])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "xcO1VF8yEoNE",
+   "metadata": {
+    "id": "xcO1VF8yEoNE"
+   },
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "131b3149",
+   "metadata": {
+    "id": "131b3149"
+   },
+   "outputs": [],
+   "source": [
+    "result = dataset['prompt'].to_list()\n",
+    "with open('prompts_2.json', 'w') as outfile:\n",
+    "    json.dump(result, outfile, ensure_ascii=False)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "91f808cf",
+   "metadata": {
+    "id": "91f808cf"
+   },
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "colab": {
+   "provenance": []
+  },
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.7"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

prompts_1.json

@@ -0,0 +1 @@
+[{"text": "Given the question delimited by triple backticks ```{Kuinka vaihdan uutiskirjeen sähköpostiosoitteen?}```, what is the answer? Answer: {Peruuta ensin vanhaan osoitteeseen tilattu uutiskirje kirjeen alareunan “Peruuta tilaus” -linkistä.\nTilaa uutiskirje uudelleen oikeaan osoitteeseen.}"}]

prompts_2.json

@@ -0,0 +1 @@
+[{"text": "Given the question delimited by triple backticks ```{Kuinka vaihdan uutiskirjeen sähköpostiosoitteen?}```, what is the answer? Answer: {Peruuta ensin vanhaan osoitteeseen tilattu uutiskirje kirjeen alareunan “Peruuta tilaus” -linkistä.\nTilaa uutiskirje uudelleen oikeaan osoitteeseen.}"}, {"text": "Given the question delimited by triple backticks ```{Miksi sähköpostiosoite tulee vahvistaa?}```, what is the answer? Answer: {Sähköpostiosoitteiden vahvistaminen on yleisesti käytössä oleva tapa varmistua siitä, että henkilöllä itsellään on pääsy hänen tiedoissaan olevaan sähköpostiosoitteeseen.\n\nSähköpostiosoite tulee vahvistaa itse, joko S-mobiilissa tai samalla kun luot itsellesi S-käyttäjätilin. Kun lähetät vahvistusviestin omissa tiedoissasi näkyvään sähköpostiosoitteeseen ja vahvistat itse osoitteen oikeaksi sähköpostiisi lähetetyllä vahvistuskoodilla, saamme varmistuksen, että osoitteesi on voimassa ja kuuluu juuri sinulle.\n\nJos asiakastiedoissasi olevasta sähköpostiosoitteesta puuttuu vielä vahvistus, näkyy osoitteen yhteydessä Vahvista -painike.\n\nBonustiedot ja muut henkilökohtaiset tiedotteet lähetetään vain vahvistettuun sähköpostiosoitteeseen. Bonustilanteesi voit kuitenkin jatkossakin tarkastaa S-mobiilista, S-käyttäjätililtä sekä toimipaikkojen S-Etukorttipäätteiltä.}"}]

requirements.txt

@@ -0,0 +1,425 @@
+absl-py==2.1.0
+accelerate==0.21.0
+aiobotocore==2.7.0
+aiohttp==3.9.1
+aioitertools==0.11.0
+aiosignal==1.3.1
+aiosqlite==0.19.0
+altair==5.2.0
+amazon-codewhisperer-jupyterlab-ext==2.0.1
+amazon_sagemaker_jupyter_scheduler==3.0.6
+annotated-types==0.6.0
+ansi2html==0.0.0
+ansiwrap==0.8.4
+antlr4-python3-runtime==4.9.3
+anyio==3.7.1
+appdirs==1.4.4
+archspec==0.2.2
+argon2-cffi==23.1.0
+argon2-cffi-bindings==21.2.0
+arrow==1.3.0
+astroid==3.0.2
+asttokens==2.4.1
+astunparse==1.6.3
+async-lru==2.0.4
+async-timeout==4.0.3
+attrs==23.2.0
+autogluon==0.8.2
+autogluon.common==0.8.2
+autogluon.core==0.8.2
+autogluon.features==0.8.2
+autogluon.multimodal==0.8.2
+autogluon.tabular==0.8.2
+autogluon.timeseries==0.8.2
+autopep8==2.0.4
+autovizwidget==0.21.0
+aws-embedded-metrics==3.2.0
+aws-glue-sessions==1.0.4
+Babel==2.14.0
+beautifulsoup4==4.12.3
+binaryornot==0.4.4
+black==24.1.1
+bleach==6.1.0
+blinker==1.7.0
+blis==0.7.10
+boltons==23.1.1
+boto3==1.28.64
+botocore==1.31.64
+Brotli==1.0.9
+cached-property==1.5.2
+cachetools==5.3.2
+catalogue==2.0.10
+catboost==1.2.2
+certifi==2023.11.17
+cffi==1.16.0
+chardet==5.2.0
+charset-normalizer==3.3.2
+click==8.1.7
+cloudpathlib==0.16.0
+cloudpickle==2.2.1
+colorama==0.4.6
+comm==0.2.1
+conda==23.11.0
+conda-libmamba-solver==24.1.0
+conda-package-handling==2.2.0
+conda_package_streaming==0.9.0
+confection==0.1.4
+contextlib2==21.6.0
+contourpy==1.2.0
+cookiecutter==2.5.0
+croniter==1.4.1
+cryptography==42.0.2
+cycler==0.12.1
+cymem==2.0.8
+cytoolz==0.12.2
+dash==2.15.0
+dask==2024.1.1
+dataclasses==0.8
+dataclasses-json==0.6.3
+datasets==2.16.1
+debugpy==1.8.0
+decorator==5.1.1
+deepmerge==1.1.1
+defusedxml==0.7.1
+dill==0.3.7
+distributed==2024.1.1
+distro==1.9.0
+docker-pycreds==0.4.0
+docstring-to-markdown==0.13
+entrypoints==0.4
+et-xmlfile==1.1.0
+evaluate==0.4.1
+exceptiongroup==1.2.0
+executing==2.0.1
+faiss==1.7.4
+fastai==2.7.14
+fastapi==0.103.2
+fastcore==1.5.29
+fastdownload==0.0.7
+fastjsonschema==2.19.1
+fastprogress==1.0.3
+filelock==3.13.1
+flake8==6.1.0
+Flask==3.0.1
+flatbuffers==23.5.26
+fonttools==4.47.2
+fqdn==1.5.1
+frozenlist==1.4.1
+fsspec==2023.6.0
+future==0.18.3
+gast==0.4.0
+gdown==5.0.1
+gitdb==4.0.11
+GitPython==3.1.41
+gluonts==0.13.7
+gmpy2==2.1.2
+google-api-core==2.16.1
+google-auth==2.27.0
+google-auth-oauthlib==1.0.0
+google-pasta==0.2.0
+googleapis-common-protos==1.62.0
+graphviz==0.20.1
+greenlet==3.0.1
+grpcio==1.54.3
+gssapi==1.8.3
+h11==0.14.0
+h5py==3.10.0
+hdijupyterutils==0.21.0
+huggingface_hub==0.20.2
+idna==3.6
+imagecodecs==2023.8.12
+imageio==2.33.1
+importlib-metadata==6.10.0
+importlib-resources==6.1.1
+iniconfig==2.0.0
+ipykernel==6.29.0
+ipython==8.20.0
+ipywidgets==8.1.1
+isoduration==20.11.0
+isort==5.13.2
+itsdangerous==2.1.2
+jax==0.4.20
+jaxlib==0.4.14
+jedi==0.19.1
+Jinja2==3.1.3
+jmespath==1.0.1
+joblib==1.3.2
+json5==0.9.14
+jsonpatch==1.33
+jsonpath-ng==1.6.1
+jsonpointer==2.4
+jsonschema==4.17.3
+jupyter==1.0.0
+jupyter_ai==2.8.1
+jupyter_ai_magics==2.8.1
+jupyter_client==8.6.0
+jupyter-console==6.6.3
+jupyter_core==5.7.1
+jupyter-dash==0.4.2
+jupyter-events==0.6.3
+jupyter-lsp==2.2.2
+jupyter_scheduler==2.4.0
+jupyter_server==2.10.0
+jupyter-server-mathjax==0.2.6
+jupyter_server_proxy==4.1.0
+jupyter_server_terminals==0.5.2
+jupyterlab==4.0.12
+jupyterlab_git==0.50.0
+jupyterlab-lsp==5.0.2
+jupyterlab_pygments==0.3.0
+jupyterlab_server==2.24.0
+jupyterlab-widgets==3.0.9
+keras==2.12.0
+Keras-Preprocessing==1.1.2
+kiwisolver==1.4.5
+krb5==0.5.1
+langchain==0.0.350
+langchain-community==0.0.7
+langchain-core==0.1.3
+langcodes==3.3.0
+langsmith==0.0.85
+libmambapy==1.5.6
+lightgbm==3.3.5
+lightning-utilities==0.10.1
+llvmlite==0.41.1
+locket==1.0.0
+Markdown==3.5.2
+markdown-it-py==3.0.0
+MarkupSafe==2.1.4
+marshmallow==3.20.2
+matplotlib==3.8.2
+matplotlib-inline==0.1.6
+mccabe==0.7.0
+mdurl==0.1.2
+menuinst==2.0.2
+mistune==3.0.2
+ml-dtypes==0.3.2
+mlforecast==0.7.3
+mock==5.1.0
+model-index==0.1.11
+mpmath==1.3.0
+msgpack==1.0.7
+multidict==6.0.4
+multiprocess==0.70.15
+munkres==1.1.4
+murmurhash==1.0.10
+mypy-extensions==1.0.0
+nbclient==0.8.0
+nbconvert==7.14.2
+nbdime==4.0.1
+nbformat==5.9.2
+nest_asyncio==1.6.0
+networkx==3.2.1
+nlpaug==1.1.11
+nltk==3.8.1
+nose==1.3.7
+notebook==7.0.7
+notebook_shim==0.2.3
+nptyping==2.4.1
+numba==0.58.1
+numexpr==2.8.7
+numpy==1.26.3
+oauthlib==3.2.2
+omegaconf==2.2.3
+openai==0.28.1
+openapi-schema-pydantic==1.2.4
+openmim==0.3.7
+openpyxl==3.1.2
+opt-einsum==3.3.0
+ordered-set==4.1.0
+overrides==7.7.0
+packaging==23.2
+pandas==2.1.4
+pandas-stubs==2.1.4.231227
+pandocfilters==1.5.0
+papermill==2.4.0
+parso==0.8.3
+partd==1.4.1
+pathos==0.3.2
+pathspec==0.12.1
+pathtools==0.1.2
+pathy==0.10.2
+patsy==0.5.6
+peft==0.9.0
+pexpect==4.9.0
+pickleshare==0.7.5
+Pillow==9.5.0
+pip==23.3.2
+pkgutil_resolve_name==1.3.10
+platformdirs==4.2.0
+plotly==5.18.0
+pluggy==1.4.0
+ply==3.11
+pox==0.3.4
+ppft==1.7.6.8
+preshed==3.0.9
+prometheus-client==0.19.0
+prompt-toolkit==3.0.42
+protobuf==4.21.12
+psutil==5.9.8
+ptyprocess==0.7.0
+pure-eval==0.2.2
+pure-sasl==0.6.2
+pyarrow==12.0.1
+pyarrow-hotfix==0.6
+pyasn1==0.5.1
+pyasn1-modules==0.3.0
+pycodestyle==2.11.1
+pycosat==0.6.6
+pycparser==2.21
+pydantic==1.10.13
+pydantic_core==2.14.3
+pydocstyle==6.3.0
+pyflakes==3.1.0
+Pygments==2.17.2
+PyHive==0.7.0
+PyJWT==2.8.0
+pylint==3.0.3
+pyOpenSSL==24.0.0
+pyparsing==3.1.1
+PyQt5==5.15.9
+PyQt5-sip==12.12.2
+pyrsistent==0.20.0
+PySocks==1.7.1
+pyspnego==0.9.1
+pytesseract==0.3.10
+pytest==8.0.0
+pytest-subtests==0.11.0
+python-dateutil==2.8.2
+python-json-logger==2.0.7
+python-lsp-jsonrpc==1.1.2
+python-lsp-server==1.9.0
+python-slugify==8.0.3
+pytoolconfig==1.2.5
+pytorch-lightning==2.0.9
+pytorch-metric-learning==1.7.3
+pytz==2023.3
+pyu2f==0.1.5
+PyWavelets==1.4.1
+PyYAML==6.0.1
+pyzmq==25.1.2
+qtconsole==5.5.1
+QtPy==2.4.1
+regex==2023.12.25
+requests==2.31.0
+requests-kerberos==0.14.0
+requests-oauthlib==1.3.1
+responses==0.18.0
+retrying==1.3.3
+rfc3339-validator==0.1.4
+rfc3986-validator==0.1.1
+rich==13.7.0
+rope==1.12.0
+rsa==4.9
+ruamel.yaml==0.18.5
+ruamel.yaml.clib==0.2.7
+s3transfer==0.7.0
+sacremoses==0.0.53
+safetensors==0.3.3
+sagemaker==2.198.1
+sagemaker-headless-execution-driver==0.0.12
+sagemaker-jupyterlab-emr-extension==0.1.9
+sagemaker-jupyterlab-extension==0.2.0
+sagemaker-jupyterlab-extension-common==0.1.9
+sagemaker-kernel-wrapper==0.0.2
+sagemaker-studio-analytics-extension==0.0.21
+sagemaker-studio-sparkmagic-lib==0.1.4
+sasl==0.3.1
+schema==0.7.5
+scikit-image==0.19.3
+scikit-learn==1.3.2
+SciPy==1.11.4
+Send2Trash==1.8.2
+sentencepiece==0.1.99
+sentry-sdk==1.40.0
+seqeval==1.2.2
+setproctitle==1.3.3
+setuptools==69.0.3
+shellingham==1.5.4
+simpervisor==1.0.0
+sip==6.7.12
+six==1.16.0
+smart-open==5.2.1
+smdebug-rulesconfig==1.0.1
+smmap==5.0.0
+sniffio==1.3.0
+snowballstemmer==2.2.0
+sortedcontainers==2.4.0
+soupsieve==2.5
+spacy==3.7.2
+spacy-legacy==3.0.12
+spacy-loggers==1.0.5
+sparkmagic==0.21.0
+SQLAlchemy==1.4.49
+srsly==2.4.8
+stack-data==0.6.2
+starlette==0.27.0
+statsforecast==1.4.0
+statsmodels==0.14.1
+supervisor==4.2.5
+sympy==1.12
+tabulate==0.9.0
+tblib==1.7.0
+tenacity==8.2.3
+tensorboard==2.12.3
+tensorboard-data-server==0.7.0
+tensorflow==2.12.1
+tensorflow-estimator==2.12.0
+termcolor==2.4.0
+terminado==0.18.0
+text-unidecode==1.3
+textwrap3==0.9.2
+thinc==8.2.2
+threadpoolctl==3.2.0
+thrift==0.19.0
+thrift-sasl==0.4.3
+tifffile==2024.1.30
+tiktoken==0.5.2
+timm==0.9.12
+tinycss2==1.2.1
+tokenizers==0.13.3
+toml==0.10.2
+tomli==2.0.1
+tomlkit==0.12.3
+toolz==0.12.1
+torch==2.0.0.post101
+torchmetrics==1.0.3
+torchvision==0.15.2a0+072ec57
+tornado==6.3.3
+tqdm==4.66.1
+traitlets==5.14.1
+transformers==4.31.0
+truststore==0.8.0
+typer==0.9.0
+types-python-dateutil==2.8.19.20240106
+types-pytz==2023.4.0.20240130
+typing_extensions==4.5.0
+typing-inspect==0.9.0
+typing-utils==0.1.0
+typish==1.9.3
+tzdata==2023.4
+ujson==5.9.0
+unicodedata2==15.1.0
+uri-template==1.3.0
+urllib3==1.26.18
+uvicorn==0.25.0
+wandb==0.16.2
+wasabi==1.1.2
+wcwidth==0.2.13
+weasel==0.3.4
+webcolors==1.13
+webencodings==0.5.1
+websocket-client==1.7.0
+Werkzeug==3.0.1
+whatthepatch==1.0.5
+wheel==0.42.0
+widgetsnbextension==4.0.9
+window-ops==0.0.14
+wrapt==1.16.0
+xgboost==1.7.6
+xxhash==3.4.1
+yapf==0.40.1
+yarl==1.9.4
+zict==3.0.0
+zipp==3.17.0
+zstandard==0.22.0