pengsu
/

MLB-care-for-mind-kor

@@ -1,202 +1,133 @@
 ---
 base_model: google/gemma-2-2b-it
 library_name: peft
 ---
-# Model Card for Model ID
-<!-- Provide a quick summary of what the model is/does. -->
 ## Model Details
-### Model Description
-<!-- Provide a longer summary of what this model is. -->
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-### Model Sources [optional]
-<!-- Provide the basic links for the model. -->
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-## Uses
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-### Direct Use
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-[More Information Needed]
-### Downstream Use [optional]
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-[More Information Needed]
-### Out-of-Scope Use
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-[More Information Needed]
-## Bias, Risks, and Limitations
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-[More Information Needed]
-### Recommendations
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-## How to Get Started with the Model
-Use the code below to get started with the model.
-[More Information Needed]
 ## Training Details
-### Training Data
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-[More Information Needed]
-### Training Procedure
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-#### Preprocessing [optional]
-[More Information Needed]
-#### Training Hyperparameters
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-#### Speeds, Sizes, Times [optional]
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-[More Information Needed]
 ## Evaluation
-<!-- This section describes the evaluation protocols and provides the results. -->
-### Testing Data, Factors & Metrics
-#### Testing Data
-<!-- This should link to a Dataset Card if possible. -->
-[More Information Needed]
-#### Factors
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-[More Information Needed]
-#### Metrics
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-[More Information Needed]
-### Results
-[More Information Needed]
-#### Summary
-## Model Examination [optional]
-<!-- Relevant interpretability work for the model goes here -->
-[More Information Needed]
-## Environmental Impact
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-## Technical Specifications [optional]
-### Model Architecture and Objective
-[More Information Needed]
-### Compute Infrastructure
-[More Information Needed]
-#### Hardware
-[More Information Needed]
-#### Software
-[More Information Needed]
-## Citation [optional]
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-**BibTeX:**
-[More Information Needed]
-**APA:**
-[More Information Needed]
-## Glossary [optional]
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-[More Information Needed]
-## More Information [optional]
-[More Information Needed]
-## Model Card Authors [optional]
-[More Information Needed]
-## Model Card Contact
-[More Information Needed]
-### Framework versions
-- PEFT 0.12.0

 ---
 base_model: google/gemma-2-2b-it
 library_name: peft
+tags:
+- sentiment-analysis
+- weighted-loss
+- LoRA
+- Korean
 ---
+# Model Card for Fine-Tuned `gemma-2-2b-it` on Custom Korean Sentiment Dataset
+## Model Summary
+This model is a fine-tuned version of `google/gemma-2-2b-it`, trained to classify sentiment in Korean text into four categories: **무감정** (neutral), **슬픔** (sadness), **기쁨** (joy), and **분노** (anger). The model utilizes **LoRA (Low-Rank Adaptation)** for efficient fine-tuning and **4-bit quantization (NF4)** for memory efficiency using **BitsAndBytes**. A custom weighted loss function was applied to handle class imbalance within the dataset.
+The model is suitable for multi-class sentiment classification in Korean and is optimized for environments with limited computational resources due to the quantization.
 ## Model Details
+### Developed By:
+This model was fine-tuned by [Your Name or Organization] using Hugging Face's `peft` and `transformers` libraries with a custom Korean sentiment dataset.
+### Model Type:
+This is a transformer-based model for **multi-class sentiment classification** in the Korean language.
+### Language:
+- **Language(s)**: Korean
+### License:
+[Add relevant license here]
+### Finetuned From:
+- **Base Model**: `google/gemma-2-2b-it`
+### Framework Versions:
+- **Transformers**: 4.44.2
+- **PEFT**: 0.12.0
+- **Datasets**: 3.0.1
+- **PyTorch**: 2.4.1+cu121
+## Intended Uses & Limitations
+### Intended Use:
+This model is suitable for applications requiring multi-class sentiment classification in Korean, such as chatbots, social media monitoring, or customer feedback analysis.
+### Out-of-Scope Use:
+The model may not perform optimally for tasks requiring multi-language support, sentiment classification with additional classes, or outside the specific context of Korean language data.
+### Limitations:
+- **Bias**: As the model is trained on a custom dataset, it may reflect specific biases inherent in that data.
+- **Generalization**: Performance may vary when applied to datasets outside the scope of the initial training data, such as other forms of sentiment classification.
+## Model Architecture
+### Quantization:
+The model uses **4-bit quantization** via **BitsAndBytes** for efficient memory usage, which enables it to run on lower-resource hardware.
+### LoRA Configuration:
+LoRA (Low-Rank Adaptation) was applied to specific transformer layers, allowing for parameter-efficient fine-tuning. The target modules include:
+- `down_proj`, `gate_proj`, `q_proj`, `o_proj`, `up_proj`, `v_proj`, `k_proj`
+LoRA parameters are:
+- `r = 16`, `lora_alpha = 32`, `lora_dropout = 0.05`
+### Custom Weighted Loss:
+A custom weighted loss function was implemented to handle class imbalance, using the following weights:
+\[
+\text{weights} = [0.2032, 0.2704, 0.2529, 0.2735]
+\]
+These weights correspond to the classes: **무감정**, **슬픔**, **기쁨**, **분노**, respectively.
 ## Training Details
+### Dataset:
+The model was trained on a custom Korean sentiment analysis dataset. This dataset consists of text samples labeled with one of four sentiment classes: **무감정**, **슬픔**, **기쁨**, and **분노**.
+- **Train Set Size**: Custom dataset
+- **Test Set Size**: Custom dataset
+- **Classes**: 4 (무감정, 슬픔, 기쁨, 분노)
+### Preprocessing:
+Data was tokenized using the `google/gemma-2-2b-it` tokenizer with a maximum sequence length of 128. The preprocessing steps included padding and truncation to ensure consistent input lengths.
+### Hyperparameters:
+- **Learning Rate**: 2e-4
+- **Batch Size (train)**: 8
+- **Batch Size (eval)**: 8
+- **Epochs**: 4
+- **Optimizer**: AdamW (with 8-bit optimization)
+- **Weight Decay**: 0.01
+- **Gradient Accumulation Steps**: 2
+- **Evaluation Steps**: 500
+- **Logging Steps**: 500
+- **Metric for Best Model**: F1 (weighted)
 ## Evaluation
+### Metrics:
+The model was evaluated using the following metrics:
+- **Accuracy**
+- **F1 Score** (weighted)
+- **Precision** (weighted)
+- **Recall** (weighted)
+The evaluation provides a detailed view of the model's performance across multiple metrics, which helps in understanding its strengths and areas for improvement.
+### Code Example:
+You can load the fine-tuned model and use it for inference on your own data as follows:
+```python
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+# Load model and tokenizer
+model = AutoModelForSequenceClassification.from_pretrained("your-model-directory")
+tokenizer = AutoTokenizer.from_pretrained("your-model-directory")
+# Tokenize input text
+text = "이 영화는 정말 슬퍼요."
+inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True)
+# Get predictions
+outputs = model(**inputs)
+logits = outputs.logits
+predicted_class = logits.argmax(-1).item()
+# Map prediction to label
+id2label = {0: "무감정", 1: "슬픔", 2: "기쁨", 3: "분노"}
+print(f"Predicted sentiment: {id2label[predicted_class]}")