ronantakizawa/sarashina2-7b-jreadability

Sarashina2-7B Difficulty-Balanced

Japanese Text Generation with Difficulty Control

Fine-tuned for difficulty-aware Japanese text generation with balanced learning and zero Simple text degradation

🎯 Model Overview

This model is a fine-tuned version of sbintuitions/sarashina2-7b designed to generate Japanese text at specified difficulty levels while maintaining high quality across both simple and complex text generation.

Key Achievement: This model achieves zero degradation on Simple text generation while simultaneously improving Complex text performance by +40%.

Source Code

📊 Performance

Metric	Baseline	Fine-tuned	Improvement
Overall Accuracy	50.0%	76.7%	+26.7%
Simple Text	93.3%	100.0%	+6.7% ✅
Complex Text	6.7%	53.3%	+46.7% ✅

What This Means

• ✅ Zero unlearning - Maintains perfect Simple text generation capability (93.3%)
• ✅ Strong Complex gains - Improved Complex text generation from 6.7% to 46.7%
• ✅ Balanced learning - Both difficulty levels perform well without trade-offs

🔬 Training Methodology

Class Weighting Strategy

The key innovation is 2x class weighting for Simple text:

class_weights = {
    'Simple': 1.5,   # 2x gradient emphasis
    'Complex': 1.0   # Normal emphasis
}

This prevents the catastrophic "unlearning" problem where models forget how to generate simple text while learning complex patterns.

Training Configuration

• Base Model: sbintuitions/sarashina2-7b (7B parameters)
• Method: LoRA (Low-Rank Adaptation) fine-tuning
• Dataset: ronantakizawa/japanese-text-difficulty-2level
• Training Examples: 1,275 texts (638 Simple, 637 Complex - perfectly balanced)
• Validation Examples: 159 texts
• Difficulty Metric: jReadability-based scoring (0-1 scale)

Key Features

1. Balanced 50/50 Dataset - Equal representation prevents bias
2. No Artificial Length Constraints - Model learns natural linguistic complexity
3. Class Weighting - Simple text receives 2x gradient influence so complex text doesn't unlearn patterns from simple text
4. LoRA Fine-tuning - Efficient training with only 40M trainable parameters (0.54%)

Hyperparameters

{
  "lora_r": 16,
  "lora_alpha": 32,
  "lora_dropout": 0.1,
  "learning_rate": 1e-4,
  "num_epochs": 3,
  "batch_size": 1,
  "gradient_accumulation_steps": 16,
  "warmup_steps": 200,
  "weight_decay": 0.01,
  "optimizer": "adamw_torch",
  "bf16": true,
  "max_length": 1500
}

💻 Usage

Installation

pip install transformers peft torch

Basic Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

# Load base model and tokenizer
base_model = AutoModelForCausalLM.from_pretrained(
    "sbintuitions/sarashina2-7b",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Load LoRA adapter
model = PeftModel.from_pretrained(
    base_model,
    "ronantakizawa/sarashina2-7b-difficulty-balanced"
)

tokenizer = AutoTokenizer.from_pretrained(
    "ronantakizawa/sarashina2-7b-difficulty-balanced"
)

# Generate Simple text
prompt = """### 指示
難易度レベル「Simple」の日本語文章を書いてください。基本的な語彙と簡単な文法を使用の文章を作成してください。

### 回答
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs,
    max_new_tokens=200,
    temperature=0.8,
    do_sample=True,
    top_p=0.9,
    repetition_penalty=1.1
)

generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(generated_text.split("### 回答\n")[1])

Generate Complex Text

prompt = """### 指示
難易度レベル「Complex」の日本語文章を書いてください。複雑な語彙と高度な文法の文章を作成してください。

### 回答
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs,
    max_new_tokens=200,
    temperature=0.8,
    do_sample=True,
    top_p=0.9,
    repetition_penalty=1.1
)

generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(generated_text.split("### 回答\n")[1])

📚 Difficulty Levels

Simple (初級レベル)

Target Audience: Beginner + Elementary Japanese learners

Characteristics:

• Basic vocabulary (基本的な語彙)
• Simple grammar structures (簡単な文法)
• Common kanji or hiragana-heavy text
• Shorter sentences
• Everyday topics

jReadability Score Range: 0.0 - 0.5

Complex (上級レベル)

Target Audience: Advanced + Expert Japanese learners

Characteristics:

• Advanced vocabulary (複雑な語彙)
• Complex grammar patterns (高度な文法)
• Advanced kanji usage
• Longer, more intricate sentences
• Abstract or technical topics

jReadability Score Range: 0.5 - 1.0

🎓 Use Cases

Language Learning

• Generate reading materials at appropriate difficulty levels
• Create personalized practice texts for learners
• Develop adaptive learning curricula

Educational Technology

• Automated content generation for Japanese learning apps
• Difficulty-graded reading comprehension exercises
• Personalized study materials

Content Creation

• Generate Japanese text for specific audiences
• Create accessibility-focused content (simple versions)
• Produce materials for different proficiency levels

Research

• Study Japanese text complexity and readability
• Analyze linguistic features across difficulty levels
• Develop difficulty assessment tools

📈 Evaluation Methodology

jReadability Scoring

The model is evaluated using jReadability, a research-backed Japanese readability metric based on Lee & Hasebe's work.

Formula:

readability = {mean words per sentence} × -0.056
            + {percentage of kango} × -0.126
            + {percentage of wago} × -0.042
            + {percentage of verbs} × -0.145
            + {percentage of particles} × -0.044
            + 11.724

Normalization: difficulty_score = (6.5 - jreadability_score) / 6.0

Accuracy Calculation

A generated text is considered "accurate" if its jReadability score falls within the target range:

• Simple: 0.0 - 0.5
• Complex: 0.5 - 1.0

This model achieves the best balance: highest overall accuracy while maintaining perfect Simple text generation.

📖 Dataset

The model was trained on ronantakizawa/japanese-text-difficulty-2level, which contains:

• 1,594 Japanese texts from Aozora Bunko (青空文庫) and Kyoto University's Basic Japanese Dataset
• Perfectly balanced: 797 Simple, 797 Complex
• jReadability-enhanced: Each text scored using research-backed metrics
• Authentic literature: Real Japanese texts, not synthetic data