Alireo-400M Model Card 📚

Model Description

Alireo-400M is a lightweight yet powerful Italian language model with 400M parameters, designed to provide efficient natural language processing capabilities while maintaining a smaller footprint compared to larger models.

Key Features ✨

Architecture: Transformer-based language model 🏗️
Parameters: 400M 📊
Context Window: 8K tokens 🪟
Training Data: Curated Italian text corpus (books, articles, web content) 📚
Model Size: ~800MB 💾

Performance 📈

Despite its compact size, Alireo-400M demonstrates impressive performance:

Benchmark Results: Outperforms Qwen 0.5B across multiple benchmarks 🏆
Language Understanding: Maintains high accuracy in Italian language understanding tasks 🎯
Speed: Efficient inference speed due to optimized architecture ⚡

Limitations ⚠️

Limited context window compared to larger models
May struggle with highly specialized technical content
Performance may vary on dialectal variations
Not suitable for multilingual tasks

Hardware Requirements 💻

Minimum RAM: 2GB
Recommended RAM: 4GB
GPU: Optional, but recommended for faster inference
Disk Space: ~1GB (including model and dependencies)

Usage Example

from transformers import AutoModelForCausalLM, AutoTokenizer

# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained("montebovi/alireo-400m")
tokenizer = AutoTokenizer.from_pretrained("montebovi/alireo-400m")

# Example text
text = "L'intelligenza artificiale sta"

# Tokenize and generate
inputs = tokenizer(text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=50)
result = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(result)

License 📜

Apache 2.0

Citation 📄

@software{alireo2024,
  author = {[Michele Montebovi]},
  title = {Alireo-400M: A Lightweight Italian Language Model},
  year = {2024},
}