Add LoRA training setup for Textilindo AI Assistant

- Created specialized training script for Textilindo AI with system prompt integration
- Updated training configuration for optimal laptop training (batch_size=2, gradient_accumulation=8)
- Added testing and inference scripts with interactive chat mode
- Created setup script for model download and environment preparation
- Added training runner script for easy execution
- Created comprehensive README for LoRA training branch
- Added readiness check script to verify all components before training

README_LORA_TRAINING.md

@@ -0,0 +1,176 @@
+# Textilindo AI Assistant - LoRA Training
+
+This branch contains the setup for fine-tuning Llama 3.2 1B with LoRA (Low-Rank Adaptation) to create a specialized Textilindo AI Assistant.
+
+## 🎯 Overview
+
+The Textilindo AI Assistant is designed to help customers with:
+- Product information and recommendations
+- Ordering and shipping details
+- Company information and policies
+- Customer support in Indonesian language
+
+## 📁 Files Structure
+
+```
+├── configs/
+│   ├── system_prompt.md          # System prompt for Textilindo AI
+│   └── training_config.yaml      # Training configuration
+├── data/
+│   └── lora_dataset_20250910_145055.jsonl  # Training dataset
+├── scripts/
+│   ├── setup_textilindo_training.py    # Setup and download model
+│   ├── train_textilindo_ai.py          # LoRA training script
+│   ├── test_textilindo_ai.py           # Testing script
+│   └── inference_textilindo_ai.py      # Inference script
+├── run_textilindo_training.sh          # Training runner script
+└── README_LORA_TRAINING.md             # This file
+```
+
+## 🚀 Quick Start
+
+### 1. Setup Environment
+
+```bash
+# Activate virtual environment
+source venv/bin/activate
+
+# Install requirements
+pip install -r requirements.txt
+```
+
+### 2. Download Base Model
+
+```bash
+python scripts/setup_textilindo_training.py
+```
+
+### 3. Start Training
+
+```bash
+# Option 1: Use the runner script
+./run_textilindo_training.sh
+
+# Option 2: Run training directly
+python scripts/train_textilindo_ai.py
+```
+
+### 4. Test the Model
+
+```bash
+# Interactive testing
+python scripts/test_textilindo_ai.py
+
+# Test with specific LoRA weights
+python scripts/test_textilindo_ai.py --lora_path models/textilindo-ai-lora-YYYYMMDD_HHMMSS
+
+# Single prompt testing
+python scripts/inference_textilindo_ai.py --prompt "dimana lokasi textilindo?"
+```
+
+## 🔧 Configuration
+
+### Training Configuration (`configs/training_config.yaml`)
+
+- **Model**: Llama 3.2 1B Instruct
+- **Dataset**: `data/lora_dataset_20250910_145055.jsonl`
+- **LoRA Settings**: r=16, alpha=32, dropout=0.1
+- **Training**: 3 epochs, batch_size=2, learning_rate=0.0002
+
+### System Prompt (`configs/system_prompt.md`)
+
+The system prompt defines the AI assistant's behavior:
+- Responds in Indonesian (Bahasa Indonesia)
+- Friendly and concise responses
+- Focuses on selling and customer service
+- Uses Textilindo-specific information
+
+## 📊 Dataset Format
+
+The training dataset uses JSONL format with the following structure:
+
+```jsonl
+{"input": "", "output": "Textilindo berkantor pusat di Jl. Raya Prancis No.39...", "metadata": {"topic": "general", "doc_id": "web_input_20250829_101006", "source": "faq", "security_level": "low"}, "instruction": "dimana lokasi textilindo?"}
+```
+
+**Required fields:**
+- `instruction`: Customer question
+- `output`: AI assistant response
+- `metadata`: Additional information (optional)
+
+## 🏋️ Training Process
+
+1. **Data Preparation**: Load JSONL dataset and format with system prompt
+2. **Model Loading**: Load Llama 3.2 1B base model
+3. **LoRA Setup**: Configure LoRA parameters for efficient fine-tuning
+4. **Training**: Fine-tune with Textilindo-specific data
+5. **Saving**: Save LoRA weights for inference
+
+## 🧪 Testing
+
+### Interactive Mode
+```bash
+python scripts/test_textilindo_ai.py
+```
+
+### Batch Testing
+The script includes predefined test cases for common Textilindo questions.
+
+### Custom Testing
+```bash
+python scripts/inference_textilindo_ai.py --prompt "Your question here"
+```
+
+## 📈 Expected Results
+
+After training, the AI assistant should be able to:
+- Answer questions about Textilindo's location, hours, and policies
+- Provide product information and recommendations
+- Handle shipping and payment questions
+- Respond in friendly Indonesian language
+- Follow the system prompt guidelines
+
+## 🔍 Troubleshooting
+
+### Common Issues
+
+1. **CUDA Out of Memory**
+   - Reduce batch_size in training_config.yaml
+   - Increase gradient_accumulation_steps
+   - Use CPU training (slower but works)
+
+2. **Model Download Failed**
+   - Check internet connection
+   - Verify HuggingFace access
+   - Check disk space
+
+3. **Training Slow**
+   - Ensure CUDA is available
+   - Check GPU memory usage
+   - Monitor system resources
+
+### Performance Tips
+
+- Use GPU for training (much faster than CPU)
+- Monitor training progress in logs
+- Save checkpoints regularly
+- Test model during training
+
+## 📝 Notes
+
+- The model is trained specifically for Textilindo's business
+- Responses follow the system prompt guidelines
+- Training data includes real customer interactions
+- Model is optimized for Indonesian language and Textilindo context
+
+## 🆘 Support
+
+If you encounter issues:
+1. Check the logs in the console output
+2. Verify all files are in the correct locations
+3. Ensure requirements are installed
+4. Check GPU memory and CUDA availability
+
+---
+
+**Happy Training! 🚀**

configs/system_prompt.md

@@ -18,6 +18,14 @@ SYSTEM_PROMPT = """You are a friendly and helpful AI assistant for Textilindo, a
 - Provide accurate information with enthusiasm
 - Encourage further questions warmly
 - ALWAYS use emojis to make responses more friendly and engaging
+- If the user asks in formal language (for example: using "saya", "Anda", "Bapak", "Ibu", or other formal Indonesian phrases), DO NOT use informal greetings like "ka", "kak", or other casual terms.
+- For formal questions, respond in formal Indonesian and address the user as "Bapak" (Sir) or "Ibu" (Ma'am) as appropriate, without using informal greetings.
+- If the user appears to be Gen-Z (using slang, abbreviations, or typical Gen-Z emojis), respond using casual, trendy Gen-Z language. Make sure your answers remain polite, friendly, and easy to understand.
+- Always match the tone and formality of your response to the language style used by the user.
+- If the user uses informal language, reply informally. If the user uses formal language, reply formally.
+- Never force a particular language style; simply follow the user's style so your response feels natural and comfortable.
+
+
 
 📝 FORMATTING GUIDELINES:
 - Use **bold** ONLY for essential titles: **Harga** (Price), **Lokasi** (Location), **Produk** (Product)

configs/training_config.yaml

@@ -1,4 +1,4 @@
-dataset_path: data/textilindo_training_data.jsonl
+dataset_path: data/lora_dataset_20250910_145055.jsonl
 lora_config:
   lora_alpha: 32
   lora_dropout: 0.1
@@ -19,10 +19,16 @@ temperature: 0.7
 top_k: 40
 top_p: 0.9
 training_config:
-  batch_size: 4
-  eval_steps: 500
-  gradient_accumulation_steps: 4
+  batch_size: 2
+  eval_steps: 100
+  gradient_accumulation_steps: 8
   learning_rate: 0.0002
   num_epochs: 3
-  save_steps: 500
-  warmup_steps: 100
+  save_steps: 100
+  warmup_steps: 50
+  logging_steps: 10
+  save_total_limit: 3
+  prediction_loss_only: true
+  remove_unused_columns: false
+  push_to_hub: false
+  report_to: null

run_textilindo_training.sh

@@ -0,0 +1,65 @@
+#!/bin/bash
+
+# Textilindo AI Assistant Training Script
+# This script sets up and runs the LoRA training for Textilindo AI Assistant
+
+echo "🚀 Textilindo AI Assistant - LoRA Training"
+echo "=========================================="
+
+# Check if virtual environment exists
+if [ ! -d "venv" ]; then
+    echo "❌ Virtual environment not found. Creating one..."
+    python3 -m venv venv
+fi
+
+# Activate virtual environment
+echo "🔧 Activating virtual environment..."
+source venv/bin/activate
+
+# Install/upgrade requirements
+echo "📦 Installing requirements..."
+pip install --upgrade pip
+pip install -r requirements.txt
+
+# Check if we need to download the model
+echo "🔍 Checking model..."
+if [ ! -d "models/llama-3.2-1b-instruct" ] || [ ! -f "models/llama-3.2-1b-instruct/config.json" ]; then
+    echo "📥 Downloading base model..."
+    python scripts/setup_textilindo_training.py
+else
+    echo "✅ Base model already exists"
+fi
+
+# Check dataset
+echo "🔍 Checking dataset..."
+if [ ! -f "data/lora_dataset_20250910_145055.jsonl" ]; then
+    echo "❌ Dataset not found: data/lora_dataset_20250910_145055.jsonl"
+    echo "Please ensure your dataset is in the correct location"
+    exit 1
+else
+    echo "✅ Dataset found"
+fi
+
+# Check system prompt
+echo "🔍 Checking system prompt..."
+if [ ! -f "configs/system_prompt.md" ]; then
+    echo "❌ System prompt not found: configs/system_prompt.md"
+    exit 1
+else
+    echo "✅ System prompt found"
+fi
+
+# Start training
+echo "🏋️ Starting LoRA training..."
+echo "This may take several hours depending on your hardware..."
+echo ""
+
+python scripts/train_textilindo_ai.py
+
+echo ""
+echo "✅ Training completed!"
+echo ""
+echo "📋 Next steps:"
+echo "1. Test the model: python scripts/test_textilindo_ai.py"
+echo "2. Find your trained model in: models/textilindo-ai-lora-*"
+echo "3. Test with LoRA: python scripts/test_textilindo_ai.py --lora_path models/textilindo-ai-lora-*"

scripts/check_training_ready.py

@@ -0,0 +1,198 @@
+#!/usr/bin/env python3
+"""
+Check if everything is ready for Textilindo AI training
+"""
+
+import os
+import sys
+import yaml
+from pathlib import Path
+
+def check_file_exists(file_path, description):
+    """Check if a file exists and print status"""
+    if os.path.exists(file_path):
+        print(f"✅ {description}: {file_path}")
+        return True
+    else:
+        print(f"❌ {description}: {file_path}")
+        return False
+
+def check_config():
+    """Check configuration files"""
+    print("🔍 Checking configuration files...")
+    
+    config_path = "configs/training_config.yaml"
+    if not os.path.exists(config_path):
+        print(f"❌ Training config not found: {config_path}")
+        return False
+    
+    try:
+        with open(config_path, 'r') as f:
+            config = yaml.safe_load(f)
+        
+        # Check required fields
+        required_fields = ['model_name', 'model_path', 'dataset_path', 'lora_config', 'training_config']
+        for field in required_fields:
+            if field not in config:
+                print(f"❌ Missing field in config: {field}")
+                return False
+        
+        print("✅ Training configuration is valid")
+        return True
+        
+    except Exception as e:
+        print(f"❌ Error reading config: {e}")
+        return False
+
+def check_dataset():
+    """Check dataset file"""
+    print("\n🔍 Checking dataset...")
+    
+    config_path = "configs/training_config.yaml"
+    with open(config_path, 'r') as f:
+        config = yaml.safe_load(f)
+    
+    dataset_path = config['dataset_path']
+    
+    if not os.path.exists(dataset_path):
+        print(f"❌ Dataset not found: {dataset_path}")
+        return False
+    
+    # Check if it's a valid JSONL file
+    try:
+        import json
+        with open(dataset_path, 'r', encoding='utf-8') as f:
+            lines = f.readlines()
+        
+        if not lines:
+            print("❌ Dataset is empty")
+            return False
+        
+        # Check first few lines
+        valid_lines = 0
+        for i, line in enumerate(lines[:5]):  # Check first 5 lines
+            line = line.strip()
+            if line:
+                try:
+                    json.loads(line)
+                    valid_lines += 1
+                except json.JSONDecodeError:
+                    print(f"❌ Invalid JSON at line {i+1}")
+                    return False
+        
+        print(f"✅ Dataset found: {dataset_path}")
+        print(f"   Total lines: {len(lines)}")
+        print(f"   Valid JSON lines checked: {valid_lines}")
+        return True
+        
+    except Exception as e:
+        print(f"❌ Error reading dataset: {e}")
+        return False
+
+def check_model():
+    """Check if base model exists"""
+    print("\n🔍 Checking base model...")
+    
+    config_path = "configs/training_config.yaml"
+    with open(config_path, 'r') as f:
+        config = yaml.safe_load(f)
+    
+    model_path = config['model_path']
+    
+    if not os.path.exists(model_path):
+        print(f"❌ Base model not found: {model_path}")
+        print("   Run: python scripts/setup_textilindo_training.py")
+        return False
+    
+    # Check if it's a valid model directory
+    required_files = ['config.json', 'tokenizer.json']
+    for file in required_files:
+        if not os.path.exists(os.path.join(model_path, file)):
+            print(f"❌ Model file missing: {file}")
+            return False
+    
+    print(f"✅ Base model found: {model_path}")
+    return True
+
+def check_system_prompt():
+    """Check system prompt file"""
+    print("\n🔍 Checking system prompt...")
+    
+    system_prompt_path = "configs/system_prompt.md"
+    
+    if not os.path.exists(system_prompt_path):
+        print(f"❌ System prompt not found: {system_prompt_path}")
+        return False
+    
+    try:
+        with open(system_prompt_path, 'r', encoding='utf-8') as f:
+            content = f.read()
+        
+        if 'SYSTEM_PROMPT' not in content:
+            print("❌ SYSTEM_PROMPT not found in file")
+            return False
+        
+        print(f"✅ System prompt found: {system_prompt_path}")
+        return True
+        
+    except Exception as e:
+        print(f"❌ Error reading system prompt: {e}")
+        return False
+
+def check_requirements():
+    """Check Python requirements"""
+    print("\n🔍 Checking Python requirements...")
+    
+    required_packages = [
+        'torch',
+        'transformers',
+        'peft',
+        'datasets',
+        'accelerate',
+        'bitsandbytes',
+        'yaml'
+    ]
+    
+    missing_packages = []
+    for package in required_packages:
+        try:
+            __import__(package)
+            print(f"✅ {package}")
+        except ImportError:
+            missing_packages.append(package)
+            print(f"❌ {package}")
+    
+    if missing_packages:
+        print(f"\n❌ Missing packages: {', '.join(missing_packages)}")
+        print("Install with: pip install " + " ".join(missing_packages))
+        return False
+    
+    return True
+
+def main():
+    print("🔍 Textilindo AI Training - Readiness Check")
+    print("=" * 50)
+    
+    all_ready = True
+    
+    # Check all components
+    all_ready &= check_config()
+    all_ready &= check_dataset()
+    all_ready &= check_model()
+    all_ready &= check_system_prompt()
+    all_ready &= check_requirements()
+    
+    print("\n" + "=" * 50)
+    
+    if all_ready:
+        print("✅ Everything is ready for training!")
+        print("\n📋 Next steps:")
+        print("1. Run training: python scripts/train_textilindo_ai.py")
+        print("2. Or use runner: ./run_textilindo_training.sh")
+    else:
+        print("❌ Some components are missing or invalid")
+        print("Please fix the issues above before training")
+        sys.exit(1)
+
+if __name__ == "__main__":
+    main()

scripts/inference_textilindo_ai.py

@@ -0,0 +1,178 @@
+#!/usr/bin/env python3
+"""
+Inference script untuk Textilindo AI Assistant
+Menggunakan model yang sudah di-fine-tune dengan LoRA
+"""
+
+import os
+import sys
+import torch
+import argparse
+from pathlib import Path
+from transformers import AutoTokenizer, AutoModelForCausalLM
+from peft import PeftModel
+import logging
+
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+def load_system_prompt(system_prompt_path):
+    """Load system prompt from markdown file"""
+    try:
+        with open(system_prompt_path, 'r', encoding='utf-8') as f:
+            content = f.read()
+        
+        # Extract SYSTEM_PROMPT from markdown
+        if 'SYSTEM_PROMPT = """' in content:
+            start = content.find('SYSTEM_PROMPT = """') + len('SYSTEM_PROMPT = """')
+            end = content.find('"""', start)
+            system_prompt = content[start:end].strip()
+        else:
+            # Fallback: use entire content
+            system_prompt = content.strip()
+        
+        return system_prompt
+    except Exception as e:
+        logger.error(f"Error loading system prompt: {e}")
+        return None
+
+def load_model(model_path, lora_path=None):
+    """Load model with optional LoRA weights"""
+    logger.info(f"Loading base model from: {model_path}")
+    
+    # Load tokenizer
+    tokenizer = AutoTokenizer.from_pretrained(
+        model_path,
+        trust_remote_code=True
+    )
+    
+    if tokenizer.pad_token is None:
+        tokenizer.pad_token = tokenizer.eos_token
+    
+    # Load base model
+    model = AutoModelForCausalLM.from_pretrained(
+        model_path,
+        torch_dtype=torch.float16,
+        device_map="auto",
+        trust_remote_code=True
+    )
+    
+    # Load LoRA weights if provided
+    if lora_path and os.path.exists(lora_path):
+        logger.info(f"Loading LoRA weights from: {lora_path}")
+        model = PeftModel.from_pretrained(model, lora_path)
+    else:
+        logger.warning("No LoRA weights found, using base model")
+    
+    return model, tokenizer
+
+def generate_response(model, tokenizer, user_input, system_prompt, max_length=512):
+    """Generate response from the model"""
+    # Create full prompt with system prompt
+    full_prompt = f"<|system|>\n{system_prompt}\n<|user|>\n{user_input}\n<|assistant|>\n"
+    
+    inputs = tokenizer(full_prompt, return_tensors="pt").to(model.device)
+    
+    with torch.no_grad():
+        outputs = model.generate(
+            **inputs,
+            max_length=max_length,
+            temperature=0.7,
+            top_p=0.9,
+            top_k=40,
+            repetition_penalty=1.1,
+            do_sample=True,
+            pad_token_id=tokenizer.eos_token_id,
+            eos_token_id=tokenizer.eos_token_id,
+            stop_strings=["<|end|>", "<|user|>"]
+        )
+    
+    response = tokenizer.decode(outputs[0], skip_special_tokens=True)
+    
+    # Extract only the assistant's response
+    if "<|assistant|>" in response:
+        assistant_response = response.split("<|assistant|>")[-1].strip()
+        # Remove any remaining special tokens
+        assistant_response = assistant_response.replace("<|end|>", "").strip()
+        return assistant_response
+    else:
+        return response
+
+def interactive_chat(model, tokenizer, system_prompt):
+    """Interactive chat mode"""
+    print("🤖 Textilindo AI Assistant - Chat Mode")
+    print("=" * 60)
+    print("Type 'quit' to exit")
+    print("-" * 60)
+    
+    while True:
+        try:
+            user_input = input("\n👤 Customer: ").strip()
+            
+            if user_input.lower() in ['quit', 'exit', 'q']:
+                print("👋 Terima kasih! Sampai jumpa!")
+                break
+            
+            if not user_input:
+                continue
+            
+            print("\n🤖 Textilindo AI: ", end="", flush=True)
+            response = generate_response(model, tokenizer, user_input, system_prompt)
+            print(response)
+                
+        except KeyboardInterrupt:
+            print("\n👋 Terima kasih! Sampai jumpa!")
+            break
+        except Exception as e:
+            logger.error(f"Error generating response: {e}")
+            print(f"❌ Error: {e}")
+
+def main():
+    parser = argparse.ArgumentParser(description='Textilindo AI Assistant Inference')
+    parser.add_argument('--model_path', type=str, default='./models/llama-3.2-1b-instruct',
+                        help='Path to base model')
+    parser.add_argument('--lora_path', type=str, default=None,
+                        help='Path to LoRA weights')
+    parser.add_argument('--system_prompt', type=str, default='configs/system_prompt.md',
+                        help='Path to system prompt file')
+    parser.add_argument('--prompt', type=str, default=None,
+                        help='Single prompt to process')
+    
+    args = parser.parse_args()
+    
+    print("🤖 Textilindo AI Assistant - Inference")
+    print("=" * 60)
+    
+    # Load system prompt
+    system_prompt = load_system_prompt(args.system_prompt)
+    if not system_prompt:
+        print(f"❌ System prompt tidak ditemukan: {args.system_prompt}")
+        sys.exit(1)
+    
+    # Check if model exists
+    if not os.path.exists(args.model_path):
+        print(f"❌ Base model tidak ditemukan: {args.model_path}")
+        print("Jalankan setup_textilindo_training.py terlebih dahulu")
+        sys.exit(1)
+    
+    try:
+        # Load model
+        print("1️⃣ Loading model...")
+        model, tokenizer = load_model(args.model_path, args.lora_path)
+        print("✅ Model loaded successfully!")
+        
+        if args.prompt:
+            # Single prompt mode
+            print(f"\n📝 Processing prompt: {args.prompt}")
+            response = generate_response(model, tokenizer, args.prompt, system_prompt)
+            print(f"\n🤖 Response: {response}")
+        else:
+            # Interactive mode
+            interactive_chat(model, tokenizer, system_prompt)
+            
+    except Exception as e:
+        logger.error(f"Error: {e}")
+        print(f"❌ Error loading model: {e}")
+
+if __name__ == "__main__":
+    main()

scripts/setup_textilindo_training.py

@@ -0,0 +1,173 @@
+#!/usr/bin/env python3
+"""
+Setup script untuk Textilindo AI Assistant training
+Download model dan prepare environment
+"""
+
+import os
+import sys
+import yaml
+import torch
+from pathlib import Path
+from transformers import AutoTokenizer, AutoModelForCausalLM
+import logging
+
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+def load_config(config_path):
+    """Load configuration from YAML file"""
+    try:
+        with open(config_path, 'r') as f:
+            config = yaml.safe_load(f)
+        return config
+    except Exception as e:
+        logger.error(f"Error loading config: {e}")
+        return None
+
+def download_model(config):
+    """Download base model"""
+    model_name = config['model_name']
+    model_path = config['model_path']
+    
+    logger.info(f"Downloading model: {model_name}")
+    logger.info(f"Target path: {model_path}")
+    
+    # Create models directory
+    Path(model_path).mkdir(parents=True, exist_ok=True)
+    
+    try:
+        # Download tokenizer
+        logger.info("Downloading tokenizer...")
+        tokenizer = AutoTokenizer.from_pretrained(
+            model_name,
+            trust_remote_code=True,
+            cache_dir=model_path
+        )
+        
+        # Download model
+        logger.info("Downloading model...")
+        model = AutoModelForCausalLM.from_pretrained(
+            model_name,
+            torch_dtype=torch.float16,
+            trust_remote_code=True,
+            cache_dir=model_path
+        )
+        
+        # Save to local path
+        logger.info(f"Saving model to: {model_path}")
+        tokenizer.save_pretrained(model_path)
+        model.save_pretrained(model_path)
+        
+        logger.info("✅ Model downloaded successfully!")
+        return True
+        
+    except Exception as e:
+        logger.error(f"Error downloading model: {e}")
+        return False
+
+def check_requirements():
+    """Check if all requirements are met"""
+    print("🔍 Checking requirements...")
+    
+    # Check Python version
+    if sys.version_info < (3, 8):
+        print("❌ Python 3.8+ required")
+        return False
+    
+    # Check PyTorch
+    try:
+        import torch
+        print(f"✅ PyTorch {torch.__version__}")
+    except ImportError:
+        print("❌ PyTorch not installed")
+        return False
+    
+    # Check CUDA availability
+    if torch.cuda.is_available():
+        print(f"✅ CUDA available: {torch.cuda.get_device_name(0)}")
+        print(f"   GPU Memory: {torch.cuda.get_device_properties(0).total_memory / 1024**3:.1f} GB")
+    else:
+        print("⚠️  CUDA not available - training will be slower on CPU")
+    
+    # Check required packages
+    required_packages = [
+        'transformers',
+        'peft',
+        'datasets',
+        'accelerate',
+        'bitsandbytes'
+    ]
+    
+    missing_packages = []
+    for package in required_packages:
+        try:
+            __import__(package)
+            print(f"✅ {package}")
+        except ImportError:
+            missing_packages.append(package)
+            print(f"❌ {package}")
+    
+    if missing_packages:
+        print(f"\n❌ Missing packages: {', '.join(missing_packages)}")
+        print("Install with: pip install " + " ".join(missing_packages))
+        return False
+    
+    return True
+
+def main():
+    print("🚀 Textilindo AI Assistant - Setup")
+    print("=" * 50)
+    
+    # Check requirements
+    if not check_requirements():
+        print("\n❌ Requirements not met. Please install missing packages.")
+        sys.exit(1)
+    
+    # Load configuration
+    config_path = "configs/training_config.yaml"
+    if not os.path.exists(config_path):
+        print(f"❌ Config file tidak ditemukan: {config_path}")
+        sys.exit(1)
+    
+    config = load_config(config_path)
+    if not config:
+        sys.exit(1)
+    
+    # Check if model already exists
+    model_path = config['model_path']
+    if os.path.exists(model_path) and os.path.exists(os.path.join(model_path, "config.json")):
+        print(f"✅ Model already exists: {model_path}")
+        print("Skipping download...")
+    else:
+        # Download model
+        print("1️⃣ Downloading base model...")
+        if not download_model(config):
+            print("❌ Failed to download model")
+            sys.exit(1)
+    
+    # Check dataset
+    dataset_path = config['dataset_path']
+    if not os.path.exists(dataset_path):
+        print(f"❌ Dataset tidak ditemukan: {dataset_path}")
+        print("Please ensure your dataset is in the correct location")
+        sys.exit(1)
+    else:
+        print(f"✅ Dataset found: {dataset_path}")
+    
+    # Check system prompt
+    system_prompt_path = "configs/system_prompt.md"
+    if not os.path.exists(system_prompt_path):
+        print(f"❌ System prompt tidak ditemukan: {system_prompt_path}")
+        sys.exit(1)
+    else:
+        print(f"✅ System prompt found: {system_prompt_path}")
+    
+    print("\n✅ Setup completed successfully!")
+    print("\n📋 Next steps:")
+    print("1. Run training: python scripts/train_textilindo_ai.py")
+    print("2. Test model: python scripts/test_textilindo_ai.py")
+    print("3. Test with LoRA: python scripts/test_textilindo_ai.py --lora_path models/textilindo-ai-lora-YYYYMMDD_HHMMSS")
+
+if __name__ == "__main__":
+    main()

scripts/test_textilindo_ai.py

@@ -0,0 +1,235 @@
+#!/usr/bin/env python3
+"""
+Script untuk testing Textilindo AI Assistant yang sudah di-fine-tune
+"""
+
+import os
+import sys
+import yaml
+import torch
+import argparse
+from pathlib import Path
+from transformers import AutoTokenizer, AutoModelForCausalLM
+from peft import PeftModel
+import logging
+
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+def load_system_prompt(system_prompt_path):
+    """Load system prompt from markdown file"""
+    try:
+        with open(system_prompt_path, 'r', encoding='utf-8') as f:
+            content = f.read()
+        
+        # Extract SYSTEM_PROMPT from markdown
+        if 'SYSTEM_PROMPT = """' in content:
+            start = content.find('SYSTEM_PROMPT = """') + len('SYSTEM_PROMPT = """')
+            end = content.find('"""', start)
+            system_prompt = content[start:end].strip()
+        else:
+            # Fallback: use entire content
+            system_prompt = content.strip()
+        
+        return system_prompt
+    except Exception as e:
+        logger.error(f"Error loading system prompt: {e}")
+        return None
+
+def load_finetuned_model(model_path, lora_weights_path, system_prompt):
+    """Load fine-tuned model with LoRA weights"""
+    logger.info(f"Loading base model from: {model_path}")
+    
+    # Load base model
+    model = AutoModelForCausalLM.from_pretrained(
+        model_path,
+        torch_dtype=torch.float16,
+        device_map="auto",
+        trust_remote_code=True
+    )
+    
+    # Load LoRA weights if available
+    if lora_weights_path and os.path.exists(lora_weights_path):
+        logger.info(f"Loading LoRA weights from: {lora_weights_path}")
+        model = PeftModel.from_pretrained(model, lora_weights_path)
+    else:
+        logger.warning("No LoRA weights found, using base model")
+    
+    # Load tokenizer
+    tokenizer = AutoTokenizer.from_pretrained(
+        model_path,
+        trust_remote_code=True
+    )
+    
+    if tokenizer.pad_token is None:
+        tokenizer.pad_token = tokenizer.eos_token
+    
+    return model, tokenizer
+
+def generate_response(model, tokenizer, user_input, system_prompt, max_length=512):
+    """Generate response from the model"""
+    # Create full prompt with system prompt
+    full_prompt = f"<|system|>\n{system_prompt}\n<|user|>\n{user_input}\n<|assistant|>\n"
+    
+    inputs = tokenizer(full_prompt, return_tensors="pt").to(model.device)
+    
+    with torch.no_grad():
+        outputs = model.generate(
+            **inputs,
+            max_length=max_length,
+            temperature=0.7,
+            top_p=0.9,
+            top_k=40,
+            repetition_penalty=1.1,
+            do_sample=True,
+            pad_token_id=tokenizer.eos_token_id,
+            eos_token_id=tokenizer.eos_token_id,
+            stop_strings=["<|end|>", "<|user|>"]
+        )
+    
+    response = tokenizer.decode(outputs[0], skip_special_tokens=True)
+    
+    # Extract only the assistant's response
+    if "<|assistant|>" in response:
+        assistant_response = response.split("<|assistant|>")[-1].strip()
+        # Remove any remaining special tokens
+        assistant_response = assistant_response.replace("<|end|>", "").strip()
+        return assistant_response
+    else:
+        return response
+
+def interactive_test(model, tokenizer, system_prompt):
+    """Interactive testing mode"""
+    print("🤖 Textilindo AI Assistant - Interactive Mode")
+    print("=" * 60)
+    print("Type 'quit' to exit")
+    print("-" * 60)
+    
+    while True:
+        try:
+            user_input = input("\n👤 Customer: ").strip()
+            
+            if user_input.lower() in ['quit', 'exit', 'q']:
+                print("👋 Terima kasih! Sampai jumpa!")
+                break
+            
+            if not user_input:
+                continue
+            
+            print("\n🤖 Textilindo AI: ", end="", flush=True)
+            response = generate_response(model, tokenizer, user_input, system_prompt)
+            print(response)
+                
+        except KeyboardInterrupt:
+            print("\n👋 Terima kasih! Sampai jumpa!")
+            break
+        except Exception as e:
+            logger.error(f"Error generating response: {e}")
+            print(f"❌ Error: {e}")
+
+def batch_test(model, tokenizer, system_prompt, test_cases):
+    """Batch testing with predefined test cases"""
+    print("🧪 Textilindo AI Assistant - Batch Testing")
+    print("=" * 60)
+    
+    for i, test_case in enumerate(test_cases, 1):
+        print(f"\n📝 Test Case {i}: {test_case['prompt']}")
+        print("-" * 40)
+        
+        try:
+            response = generate_response(model, tokenizer, test_case['prompt'], system_prompt)
+            print(f"🤖 Response: {response}")
+            
+            if 'expected' in test_case:
+                print(f"🎯 Expected: {test_case['expected']}")
+                
+        except Exception as e:
+            logger.error(f"Error in test case {i}: {e}")
+            print(f"❌ Error: {e}")
+
+def main():
+    parser = argparse.ArgumentParser(description='Test Textilindo AI Assistant')
+    parser.add_argument('--model_path', type=str, default='./models/llama-3.2-1b-instruct',
+                        help='Path to base model')
+    parser.add_argument('--lora_path', type=str, default=None,
+                        help='Path to LoRA weights')
+    parser.add_argument('--system_prompt', type=str, default='configs/system_prompt.md',
+                        help='Path to system prompt file')
+    
+    args = parser.parse_args()
+    
+    print("🧪 Textilindo AI Assistant Testing")
+    print("=" * 60)
+    
+    # Load system prompt
+    system_prompt = load_system_prompt(args.system_prompt)
+    if not system_prompt:
+        print(f"❌ System prompt tidak ditemukan: {args.system_prompt}")
+        sys.exit(1)
+    
+    # Check if model exists
+    if not os.path.exists(args.model_path):
+        print(f"❌ Base model tidak ditemukan: {args.model_path}")
+        print("Jalankan download_model.py terlebih dahulu")
+        sys.exit(1)
+    
+    try:
+        # Load model
+        print("1️⃣ Loading model...")
+        model, tokenizer = load_finetuned_model(args.model_path, args.lora_path, system_prompt)
+        print("✅ Model loaded successfully!")
+        
+        # Test cases specific to Textilindo
+        test_cases = [
+            {
+                "prompt": "dimana lokasi textilindo?",
+                "expected": "Textilindo berkantor pusat di Jl. Raya Prancis No.39, Kosambi Tim., Kec. Kosambi, Kabupaten Tangerang, Banten 15213"
+            },
+            {
+                "prompt": "Jam berapa textilindo beroperasional?",
+                "expected": "Jam operasional Senin-Jumat 08:00-17:00, Sabtu 08:00-12:00."
+            },
+            {
+                "prompt": "Berapa ketentuan pembelian?",
+                "expected": "Minimal order 1 roll per jenis kain"
+            },
+            {
+                "prompt": "bagimana dengan pembayarannya?",
+                "expected": "Pembayaran dapat dilakukan via transfer bank atau cash on delivery"
+            },
+            {
+                "prompt": "apa ada gratis ongkir?",
+                "expected": "Gratis ongkir untuk order minimal 5 roll."
+            },
+            {
+                "prompt": "Apa bisa dikirimkan sample? apa gratis?",
+                "expected": "hallo kak untuk sampel kita bisa kirimkan gratis ya kak 😊"
+            }
+        ]
+        
+        # Choose testing mode
+        print("\n2️⃣ Pilih mode testing:")
+        print("1. Interactive mode (chat)")
+        print("2. Batch testing")
+        print("3. Custom prompt")
+        
+        choice = input("\nPilihan (1-3): ").strip()
+        
+        if choice == "1":
+            interactive_test(model, tokenizer, system_prompt)
+        elif choice == "2":
+            batch_test(model, tokenizer, system_prompt, test_cases)
+        elif choice == "3":
+            custom_prompt = input("Masukkan prompt custom: ").strip()
+            if custom_prompt:
+                response = generate_response(model, tokenizer, custom_prompt, system_prompt)
+                print(f"\n🤖 Response: {response}")
+        else:
+            print("❌ Pilihan tidak valid")
+            
+    except Exception as e:
+        logger.error(f"Error: {e}")
+        print(f"❌ Error loading model: {e}")
+
+if __name__ == "__main__":
+    main()

scripts/train_textilindo_ai.py

@@ -0,0 +1,280 @@
+#!/usr/bin/env python3
+"""
+Script untuk fine-tuning Llama 3.2 1B dengan LoRA untuk Textilindo AI Assistant
+Menggunakan system prompt dan dataset khusus Textilindo
+"""
+
+import os
+import sys
+import yaml
+import json
+import torch
+from pathlib import Path
+from transformers import (
+    AutoTokenizer, 
+    AutoModelForCausalLM,
+    TrainingArguments,
+    Trainer,
+    DataCollatorForLanguageModeling
+)
+from peft import (
+    LoraConfig,
+    get_peft_model,
+    TaskType,
+    prepare_model_for_kbit_training
+)
+from datasets import Dataset
+import logging
+from datetime import datetime
+
+# Setup logging
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+def load_config(config_path):
+    """Load configuration from YAML file"""
+    try:
+        with open(config_path, 'r') as f:
+            config = yaml.safe_load(f)
+        return config
+    except Exception as e:
+        logger.error(f"Error loading config: {e}")
+        return None
+
+def load_system_prompt(system_prompt_path):
+    """Load system prompt from markdown file"""
+    try:
+        with open(system_prompt_path, 'r', encoding='utf-8') as f:
+            content = f.read()
+        
+        # Extract SYSTEM_PROMPT from markdown
+        if 'SYSTEM_PROMPT = """' in content:
+            start = content.find('SYSTEM_PROMPT = """') + len('SYSTEM_PROMPT = """')
+            end = content.find('"""', start)
+            system_prompt = content[start:end].strip()
+        else:
+            # Fallback: use entire content
+            system_prompt = content.strip()
+        
+        return system_prompt
+    except Exception as e:
+        logger.error(f"Error loading system prompt: {e}")
+        return None
+
+def load_model_and_tokenizer(config):
+    """Load base model and tokenizer"""
+    model_path = config['model_path']
+    
+    logger.info(f"Loading model from: {model_path}")
+    
+    # Load tokenizer
+    tokenizer = AutoTokenizer.from_pretrained(
+        model_path,
+        trust_remote_code=True,
+        padding_side="right"
+    )
+    
+    if tokenizer.pad_token is None:
+        tokenizer.pad_token = tokenizer.eos_token
+    
+    # Load model
+    model = AutoModelForCausalLM.from_pretrained(
+        model_path,
+        torch_dtype=torch.float16,
+        device_map="auto",
+        trust_remote_code=True
+    )
+    
+    # Prepare model for k-bit training
+    model = prepare_model_for_kbit_training(model)
+    
+    return model, tokenizer
+
+def setup_lora_config(config):
+    """Setup LoRA configuration"""
+    lora_config = config['lora_config']
+    
+    peft_config = LoraConfig(
+        task_type=TaskType.CAUSAL_LM,
+        r=lora_config['r'],
+        lora_alpha=lora_config['lora_alpha'],
+        lora_dropout=lora_config['lora_dropout'],
+        target_modules=lora_config['target_modules'],
+        bias="none",
+    )
+    
+    return peft_config
+
+def prepare_textilindo_dataset(data_path, tokenizer, system_prompt, max_length=2048):
+    """Prepare Textilindo dataset for training with system prompt"""
+    logger.info(f"Loading dataset from: {data_path}")
+    
+    # Load JSONL dataset
+    data = []
+    with open(data_path, 'r', encoding='utf-8') as f:
+        for line_num, line in enumerate(f, 1):
+            line = line.strip()
+            if line:
+                try:
+                    json_obj = json.loads(line)
+                    data.append(json_obj)
+                except json.JSONDecodeError as e:
+                    logger.warning(f"Invalid JSON at line {line_num}: {e}")
+                    continue
+    
+    if not data:
+        raise ValueError("No valid JSON objects found in JSONL file")
+    
+    logger.info(f"Loaded {len(data)} samples from JSONL file")
+    
+    # Convert to training format with system prompt
+    training_data = []
+    for item in data:
+        # Extract instruction and output
+        instruction = item.get('instruction', '')
+        output = item.get('output', '')
+        
+        if not instruction or not output:
+            continue
+        
+        # Create training text with system prompt
+        training_text = f"<|system|>\n{system_prompt}\n<|user|>\n{instruction}\n<|assistant|>\n{output}<|end|>"
+        
+        training_data.append({
+            'text': training_text,
+            'instruction': instruction,
+            'output': output
+        })
+    
+    # Convert to Dataset
+    dataset = Dataset.from_list(training_data)
+    logger.info(f"Prepared {len(dataset)} training samples")
+    
+    def tokenize_function(examples):
+        # Tokenize the texts
+        tokenized = tokenizer(
+            examples['text'],
+            truncation=True,
+            padding=True,
+            max_length=max_length,
+            return_tensors="pt"
+        )
+        return tokenized
+    
+    # Tokenize dataset
+    tokenized_dataset = dataset.map(
+        tokenize_function,
+        batched=True,
+        remove_columns=dataset.column_names
+    )
+    
+    return tokenized_dataset
+
+def train_model(model, tokenizer, dataset, config, output_dir):
+    """Train the model with LoRA"""
+    training_config = config['training_config']
+    
+    # Setup training arguments
+    training_args = TrainingArguments(
+        output_dir=output_dir,
+        num_train_epochs=training_config['num_epochs'],
+        per_device_train_batch_size=training_config['batch_size'],
+        gradient_accumulation_steps=training_config['gradient_accumulation_steps'],
+        learning_rate=training_config['learning_rate'],
+        warmup_steps=training_config['warmup_steps'],
+        save_steps=training_config['save_steps'],
+        eval_steps=training_config['eval_steps'],
+        logging_steps=training_config.get('logging_steps', 10),
+        save_total_limit=training_config.get('save_total_limit', 3),
+        prediction_loss_only=training_config.get('prediction_loss_only', True),
+        remove_unused_columns=training_config.get('remove_unused_columns', False),
+        push_to_hub=training_config.get('push_to_hub', False),
+        report_to=training_config.get('report_to', None),
+        fp16=True,  # Enable mixed precision training
+        dataloader_pin_memory=False,  # Reduce memory usage
+    )
+    
+    # Setup data collator
+    data_collator = DataCollatorForLanguageModeling(
+        tokenizer=tokenizer,
+        mlm=False,
+    )
+    
+    # Setup trainer
+    trainer = Trainer(
+        model=model,
+        args=training_args,
+        train_dataset=dataset,
+        data_collator=data_collator,
+        tokenizer=tokenizer,
+    )
+    
+    # Start training
+    logger.info("Starting training...")
+    trainer.train()
+    
+    # Save the model
+    trainer.save_model()
+    logger.info(f"Model saved to: {output_dir}")
+
+def main():
+    print("🚀 Textilindo AI Assistant - LoRA Fine-tuning")
+    print("=" * 60)
+    
+    # Load configuration
+    config_path = "configs/training_config.yaml"
+    if not os.path.exists(config_path):
+        print(f"❌ Config file tidak ditemukan: {config_path}")
+        sys.exit(1)
+    
+    config = load_config(config_path)
+    if not config:
+        sys.exit(1)
+    
+    # Load system prompt
+    system_prompt_path = "configs/system_prompt.md"
+    if not os.path.exists(system_prompt_path):
+        print(f"❌ System prompt tidak ditemukan: {system_prompt_path}")
+        sys.exit(1)
+    
+    system_prompt = load_system_prompt(system_prompt_path)
+    if not system_prompt:
+        sys.exit(1)
+    
+    # Setup paths
+    timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
+    output_dir = Path(f"models/textilindo-ai-lora-{timestamp}")
+    output_dir.mkdir(parents=True, exist_ok=True)
+    
+    # Check if dataset exists
+    data_path = config['dataset_path']
+    if not os.path.exists(data_path):
+        print(f"❌ Dataset tidak ditemukan: {data_path}")
+        sys.exit(1)
+    
+    # Load model and tokenizer
+    print("1️⃣ Loading model and tokenizer...")
+    model, tokenizer = load_model_and_tokenizer(config)
+    
+    # Setup LoRA
+    print("2️⃣ Setting up LoRA configuration...")
+    peft_config = setup_lora_config(config)
+    model = get_peft_model(model, peft_config)
+    
+    # Print trainable parameters
+    model.print_trainable_parameters()
+    
+    # Prepare dataset
+    print("3️⃣ Preparing Textilindo dataset...")
+    dataset = prepare_textilindo_dataset(data_path, tokenizer, system_prompt, config['max_length'])
+    
+    # Train model
+    print("4️⃣ Starting training...")
+    train_model(model, tokenizer, dataset, config, output_dir)
+    
+    print("✅ Training selesai!")
+    print(f"📁 Model tersimpan di: {output_dir}")
+    print(f"🔧 Untuk testing, jalankan: python scripts/test_textilindo_ai.py --model_path {output_dir}")
+
+if __name__ == "__main__":
+    main()