SatyamSinghal/financial-ttm

Financial Transformer Time-series Model (TTM)

Overview

This project implements a Transformer-based deep learning model designed specifically for financial time series prediction. The system can forecast multiple target variables:

• Price movements for multiple future time steps
• Volatility predictions to estimate market uncertainty
• Risk classifications for trading decision support

The model leverages the Transformer architecture's ability to capture long-term dependencies and patterns in sequential data, making it well-suited for financial market prediction tasks.

Project Structure

ML_TTT/
├── main.py                 # Entry point with training orchestration
├── feature_engineering.py  # Data transformation and sequence creation
├── models.py               # Neural network architecture definitions
├── training.py             # Training loop, datasets, and optimization
├── metrics.py              # Performance evaluation metrics
├── prediction_agent.py     # Model inference and prediction interface
├── risk_manager.py         # Risk analysis utilities
├── real_time_data.py       # Data handling for live/synthetic data
└── requirements.txt        # Project dependencies

Key Components

Data Processing Pipeline

1. Data Loading: Load historical OHLCV (Open, High, Low, Close, Volume) data
2. Feature Engineering: Generate technical indicators and statistical features
3. Sequence Creation: Format data into sliding window sequences for supervised learning
4. Scaling: Normalize features to stable ranges for model training

Model Architecture

The core FinancialTTM model consists of:

• Input Projection: Linear layer to project raw features to model dimension
• Positional Encoding: Adding position information to input sequences
• Transformer Encoder: Multi-head self-attention with feed-forward networks
• Prediction Heads: Specialized output layers for different prediction tasks:
  • Price prediction head
  • Volatility prediction head
  • Confidence estimation head
  • Risk classification head

Training System

• Multi-objective Loss: Combined loss across different prediction tasks
• Early Stopping: Prevent overfitting by monitoring validation performance
• Mixed Precision Training: Optional for faster training on compatible hardware
• Comprehensive Metrics: Track model performance across different objectives

Usage Instructions

Setup

1. Install dependencies:

pip install -r requirements.txt

2. Configure the model in main.py:

config = {
    'input_dim': 20,         # Number of input features
    'd_model': 64,           # Transformer dimension
    'nhead': 4,              # Number of attention heads
    'num_layers': 2,         # Transformer encoder layers
    'lookback_window': 20,   # Sequence length for input
    'prediction_horizon': 5, # Number of future steps to predict
    'batch_size': 64,        # Training batch size
    'learning_rate': 0.001,  # Learning rate
    'epochs': 20             # Training epochs
}

Training a Model

Run the training script:

python main.py

This will:

• Generate synthetic data (or use your own data source)
• Prepare features and create sequences
• Train the model with early stopping
• Save the trained model to financial_ttm_model.pth

Making Predictions

import torch
from models import FinancialTTM

# Load model
model_data = torch.load('financial_ttm_model.pth')
model = FinancialTTM(
    input_dim=model_data['config']['input_dim'],
    d_model=model_data['config']['d_model'],
    nhead=model_data['config']['nhead'],
    num_layers=model_data['config']['num_layers'],
    prediction_horizon=model_data['config']['prediction_horizon']
)
model.load_state_dict(model_data['state_dict'])
model.eval()

# Prepare input (ensure it's preprocessed the same way as training data)
input_sequence = torch.FloatTensor(processed_input_data)

# Get predictions
with torch.no_grad():
    predictions = model(input_sequence)

# Access different predictions
price_predictions = predictions['price_prediction']
volatility_predictions = predictions['volatility_prediction']
confidence = predictions['confidence']
risk_class = predictions['risk_classification']

Use Cases

1. Algorithmic Trading: Integrate predictions into trading algorithms
2. Risk Management: Use volatility forecasts to adjust position sizing
3. Portfolio Optimization: Balance investments based on predicted market movements
4. Trading Decision Support: Use risk classifications to filter trading opportunities

Performance Metrics

Model performance is evaluated using:

• Directional Accuracy: How often price direction is correctly predicted
• Mean Absolute Error (MAE): Average absolute difference between predicted and actual prices
• Mean Squared Error (MSE): For volatility predictions
• Classification Metrics: Precision, recall, F1-score for risk classification
• Sharpe Ratio: Risk-adjusted return when used in a simulated trading strategy

Publishing to Hugging Face

1. Prepare your model for publishing

import torch

# After successful training
model_info = {
    'model_state_dict': model.state_dict(),
    'config': {
        'input_dim': 20,
        'd_model': 64,
        'nhead': 4,
        'num_layers': 2,
        'prediction_horizon': 5
    },
    'feature_columns': list(numeric_columns),  # Feature column names
    'scaler': feature_eng.scaler  # Save scaler for preprocessing
}

torch.save(model_info, "financial_ttm_model.pth")

2. Install Hugging Face Hub

pip install huggingface_hub

3. Login to Hugging Face

huggingface-cli login

4. Upload your model

from huggingface_hub import HfApi

api = HfApi()
api.create_repo(repo_id="your-username/financial-ttm", private=False)

# Upload model file
api.upload_file(
    path_or_fileobj="financial_ttm_model.pth",
    path_in_repo="financial_ttm_model.pth",
    repo_id="your-username/financial-ttm"
)

# Upload readme
api.upload_file(
    path_or_fileobj="README.md",
    path_in_repo="README.md",
    repo_id="your-username/financial-ttm"
)

# Upload example usage code
api.upload_file(
    path_or_fileobj="example_inference.py",
    path_in_repo="example_inference.py",
    repo_id="your-username/financial-ttm"
)

5. Share your model

Once published, you can share the URL: https://huggingface.co/your-username/financial-ttm

Users can then download and use your model:

from huggingface_hub import hf_hub_download

model_path = hf_hub_download(repo_id="your-username/financial-ttm", filename="financial_ttm_model.pth")
model_data = torch.load(model_path)

Contributing

Contributions are welcome! Here are some areas for potential improvement:

• Add support for more technical indicators
• Implement alternative model architectures
• Create visualization tools for predictions
• Add backtesting infrastructure for trading strategies
• Optimize for specific financial instruments or markets

License

This project is licensed under the MIT License - see the LICENSE file for details.