import subprocess import logging import gradio as gr import pandas as pd from apscheduler.schedulers.background import BackgroundScheduler from apscheduler.triggers.cron import CronTrigger from pytz import utc from tabs.trades import ( prepare_trades, get_overall_trades, get_overall_winning_trades, plot_trades_by_week, plot_winning_trades_by_week, plot_trade_details ) from tabs.tool_win import ( get_tool_winning_rate, get_overall_winning_rate, plot_tool_winnings_overall, plot_tool_winnings_by_tool ) from tabs.error import ( get_error_data, get_error_data_overall, plot_error_data, plot_tool_error_data, plot_week_error_data ) def refresh_data(): # Run the pull_data.py script and wait for it to finish try: result = subprocess.run(["python", "./scripts/pull_data.py"], check=True) logging.info("Script executed successfully: %s", result) except subprocess.CalledProcessError as e: logging.error("Failed to run script: %s", e) return # Stop execution if the script fails # Reload dataframes try: global tools_df, trades_df, error_df, error_overall_df, winning_rate_df, winning_rate_overall_df, trades_count_df, trades_winning_rate_df logging.info("Refreshing data...") tools_df = pd.read_csv("./data/tools.csv", low_memory=False) trades_df = pd.read_csv("./data/all_trades_profitability.csv") trades_df = prepare_trades(trades_df) error_df = get_error_data(tools_df=tools_df, inc_tools=INC_TOOLS) error_overall_df = get_error_data_overall(error_df=error_df) winning_rate_df = get_tool_winning_rate(tools_df=tools_df, inc_tools=INC_TOOLS) winning_rate_overall_df = get_overall_winning_rate(wins_df=winning_rate_df) trades_count_df = get_overall_trades(trades_df=trades_df) trades_winning_rate_df = get_overall_winning_trades(trades_df=trades_df) logging.info("Data refreshed.") except Exception as e: logging.error("Failed to refresh data: %s", e) tools_df = pd.read_csv("./data/tools.csv", low_memory=False) trades_df = pd.read_csv("./data/all_trades_profitability.csv") trades_df = prepare_trades(trades_df) demo = gr.Blocks() INC_TOOLS = [ 'prediction-online', 'prediction-offline', 'claude-prediction-online', 'claude-prediction-offline', 'prediction-offline-sme', 'prediction-online-sme', 'prediction-request-rag', 'prediction-request-reasoning', 'prediction-url-cot-claude', 'prediction-request-rag-claude', 'prediction-request-reasoning-claude' ] # TOOLS DATA error_df = get_error_data( tools_df=tools_df, inc_tools=INC_TOOLS ) error_overall_df = get_error_data_overall( error_df=error_df ) winning_rate_df = get_tool_winning_rate( tools_df=tools_df, inc_tools=INC_TOOLS ) winning_rate_overall_df = get_overall_winning_rate( wins_df=winning_rate_df ) trades_count_df = get_overall_trades( trades_df=trades_df ) trades_winning_rate_df = get_overall_winning_trades( trades_df=trades_df ) with demo: gr.HTML("

Olas Predict Actual Performance

") gr.Markdown("This app shows the actual performance of Olas Predict tools on the live market.") with gr.Tabs(): with gr.TabItem("🔥Trades Dashboard"): with gr.Row(): gr.Markdown("# Plot of number of trades by week") with gr.Row(): plot_trades_by_week = plot_trades_by_week( trades_df=trades_count_df ) with gr.Row(): gr.Markdown("# Plot of winning trades by week") with gr.Row(): plot_winning_trades_by_week = plot_winning_trades_by_week( trades_df=trades_winning_rate_df ) with gr.Row(): gr.Markdown("# Plot of trade details") with gr.Row(): trade_details_selector = gr.Dropdown( label="Select a trade", choices=[ "mech calls", "collateral amount", "earnings", "net earnings", "ROI" ], value="mech calls" ) with gr.Row(): trade_details_plot = plot_trade_details( trade_detail="mech calls", trades_df=trades_df ) def update_trade_details(trade_detail): return plot_trade_details( trade_detail=trade_detail, trades_df=trades_df ) trade_details_selector.change( update_trade_details, inputs=trade_details_selector, outputs=trade_details_plot ) with gr.Row(): trade_details_selector with gr.Row(): trade_details_plot with gr.TabItem("🚀 Tool Winning Dashboard"): with gr.Row(): gr.Markdown("# Plot showing overall winning rate") with gr.Row(): winning_selector = gr.Dropdown( label="Select Metric", choices=['losses', 'wins', 'total_request', 'win_perc'], value='win_perc', ) with gr.Row(): winning_plot = plot_tool_winnings_overall( wins_df=winning_rate_overall_df, winning_selector="win_perc" ) def update_tool_winnings_overall_plot(winning_selector): return plot_tool_winnings_overall( wins_df=winning_rate_overall_df, winning_selector=winning_selector ) winning_selector.change( update_tool_winnings_overall_plot, inputs=winning_selector, outputs=winning_plot ) with gr.Row(): winning_selector with gr.Row(): winning_plot with gr.Row(): gr.Markdown("# Plot showing winning rate by tool") with gr.Row(): sel_tool = gr.Dropdown( label="Select a tool", choices=INC_TOOLS, value=INC_TOOLS[0] ) with gr.Row(): plot_tool_win_rate = plot_tool_winnings_by_tool( wins_df=winning_rate_df, tool=INC_TOOLS[0] ) def update_tool_winnings_by_tool_plot(tool): return plot_tool_winnings_by_tool( wins_df=winning_rate_df, tool=tool ) sel_tool.change( update_tool_winnings_by_tool_plot, inputs=sel_tool, outputs=plot_tool_win_rate ) with gr.Row(): sel_tool with gr.Row(): plot_tool_win_rate with gr.TabItem("🏥 Tool Error Dashboard"): with gr.Row(): gr.Markdown("# Plot showing overall error") with gr.Row(): plot_error_data( error_all_df=error_overall_df ) with gr.Row(): gr.Markdown("# Plot showing error by tool") with gr.Row(): sel_tool = gr.Dropdown( label="Select a tool", choices=INC_TOOLS, value=INC_TOOLS[0] ) with gr.Row(): plot_tool_error = plot_tool_error_data( error_df=error_df, tool=INC_TOOLS[0] ) def update_tool_error_plot(tool): return plot_tool_error_data( error_df=error_df, tool=tool ) sel_tool.change( update_tool_error_plot, inputs=sel_tool, outputs=plot_tool_error ) with gr.Row(): sel_tool with gr.Row(): plot_tool_error with gr.Row(): gr.Markdown("# Plot showing error by week") with gr.Row(): choices = error_overall_df['request_month_year_week'].unique().tolist() # sort the choices by the latest week to be on the top choices = sorted(choices) sel_week = gr.Dropdown( label="Select a week", choices=choices, value=choices[-1] ) with gr.Row(): plot_week_error = plot_week_error_data( error_df=error_df, week=choices[-1] ) def update_week_error_plot(selected_week): return plot_week_error_data( error_df=error_df, week=selected_week ) sel_tool.change(update_tool_error_plot, inputs=sel_tool, outputs=plot_tool_error) sel_week.change(update_week_error_plot, inputs=sel_week, outputs=plot_week_error) with gr.Row(): sel_tool with gr.Row(): plot_tool_error with gr.Row(): sel_week with gr.Row(): plot_week_error with gr.TabItem("ℹ️ About"): with gr.Accordion("About the Benchmark"): gr.Markdown("This app shows the actual performance of Olas Predict tools on the live market.") # Create the scheduler scheduler = BackgroundScheduler(timezone=utc) scheduler.add_job(refresh_data, CronTrigger(hour=0, minute=0)) # Runs daily at 12 AM UTC scheduler.start() # scheduler = BackgroundScheduler(timezone=utc) # scheduler.add_job(refresh_data, CronTrigger(hour='*')) # Runs every hour # scheduler.start() demo.queue(default_concurrency_limit=40).launch()