(feat) update pages

pages/backtesting/README.md

@@ -0,0 +1,128 @@
+# Backtesting Module
+
+## Page Purpose and Functionality
+
+The Backtesting module enables users to test, analyze, and optimize trading strategies using historical market data. It provides a comprehensive framework for evaluating strategy performance before deploying them with real funds. The module consists of three main components: Create, Analyze, and Optimize.
+
+## Key Features
+
+### 1. Create (`/create`)
+- Design and configure backtesting scenarios for directional trading strategies
+- Set up strategy parameters including order levels, triple barrier configurations, and position sizing
+- Define backtesting periods and initial portfolio settings
+- Save configurations for future use
+
+### 2. Analyze (`/analyze`)
+- Load and examine results from Optuna optimization databases
+- Filter and compare multiple backtesting trials based on performance metrics
+- Interactive visualization of PnL vs Maximum Drawdown
+- Detailed parameter inspection and modification
+- Re-run backtests with adjusted parameters
+
+### 3. Optimize (`/optimize`)
+- Automated hyperparameter optimization using Optuna framework
+- Multi-objective optimization targeting profit, drawdown, and accuracy
+- Parallel trial execution for efficient parameter space exploration
+- Real-time optimization progress tracking
+- Export optimized configurations
+
+## User Flow
+
+1. **Strategy Creation**
+   - User selects a trading strategy controller
+   - Configures strategy parameters (e.g., technical indicators, thresholds)
+   - Sets up order levels with triple barrier configurations
+   - Defines backtesting period and initial capital
+   - Runs initial backtest
+
+2. **Optimization**
+   - User selects parameters to optimize with ranges
+   - Defines optimization objectives (maximize profit, minimize drawdown)
+   - Sets number of trials and execution parameters
+   - Monitors optimization progress in real-time
+   - Reviews Pareto-optimal solutions
+
+3. **Analysis**
+   - User loads optimization database
+   - Filters trials by performance metrics (accuracy, profit, drawdown)
+   - Selects promising trials for detailed inspection
+   - Fine-tunes parameters based on insights
+   - Exports final configurations for deployment
+
+## Technical Implementation Details
+
+### Architecture
+- **Backend Integration**: Communicates with Hummingbot's backtesting engine via the Backend API Client
+- **Data Processing**: Uses pandas for data manipulation and analysis
+- **Optimization Engine**: Leverages Optuna for Bayesian optimization
+- **Visualization**: Plotly for interactive charts and performance metrics
+
+### Key Classes and Components
+- `DirectionalTradingBacktestingEngine`: Core backtesting engine from Hummingbot
+- `OptunaDBManager`: Manages optimization databases and trial data
+- `BacktestingGraphs`: Generates performance visualizations
+- `StrategyAnalysis`: Computes strategy metrics and statistics
+
+### Data Flow
+1. Strategy configuration → Backtesting engine
+2. Historical market data → Engine simulation
+3. Trade execution results → Performance metrics
+4. Metrics → Optuna optimization
+5. Optimized parameters → Analysis and export
+
+## Component Dependencies
+
+### Internal Dependencies
+- `backend.utils.optuna_database_manager`: Database management for optimization results
+- `backend.utils.os_utils`: Controller loading utilities
+- `frontend.st_utils`: Streamlit page initialization and utilities
+- `frontend.visualization.graphs`: Chart generation for backtesting results
+- `frontend.visualization.strategy_analysis`: Performance metric calculations
+
+### External Dependencies
+- `hummingbot`: Core trading strategy framework
+- `streamlit`: Web UI framework
+- `pandas`: Data manipulation
+- `plotly`: Interactive visualizations
+- `optuna`: Hyperparameter optimization
+
+## State Management Approach
+
+### Session State Variables
+- `strategy_params`: Current strategy configuration parameters
+- `backtesting_params`: Backtesting-specific settings (period, costs, etc.)
+- `optimization_params`: Ranges and objectives for parameter optimization
+- `selected_study`: Currently selected Optuna study
+- `selected_trial`: Currently selected optimization trial
+
+### Persistent Storage
+- **Optimization Databases**: SQLite files in `data/backtesting/` directory
+- **Strategy Configurations**: YAML files in `hummingbot_files/controller_configs/`
+- **Candle Data**: Historical market data in `data/candles/`
+
+### Cache Management
+- `@st.cache_resource`: Used for database loading to prevent repeated file I/O
+- `@st.cache_data`: Applied to expensive computations like metric calculations
+- Results cached during session to improve performance when switching between trials
+
+## Best Practices
+
+1. **Data Validation**
+   - Always verify candle data availability before running backtests
+   - Validate parameter ranges to prevent invalid configurations
+   - Check for sufficient historical data for the selected period
+
+2. **Performance Optimization**
+   - Use cached resources for database operations
+   - Limit the number of simultaneous optimization trials
+   - Filter large datasets before visualization
+
+3. **User Experience**
+   - Provide clear progress indicators during long operations
+   - Display meaningful error messages for common issues
+   - Offer sensible defaults for complex parameters
+
+4. **Configuration Management**
+   - Save successful configurations with descriptive names
+   - Version control strategy configurations
+   - Document parameter choices and rationale

pages/backtesting/analyze/README.md

@@ -0,0 +1,85 @@
+# Backtesting Analysis
+
+The Backtesting Analysis page provides comprehensive tools for analyzing and comparing the performance of your trading strategy backtests.
+
+## Features
+
+### 📊 Performance Analysis
+- **Strategy Performance Metrics**: View detailed metrics including total P&L, win rate, Sharpe ratio, and maximum drawdown
+- **Trade-by-Trade Analysis**: Examine individual trades with entry/exit times, prices, and P&L
+- **Performance Visualization**: Interactive charts showing cumulative returns, drawdown periods, and trade distribution
+- **Multi-Backtest Comparison**: Compare performance across multiple backtests side-by-side
+
+### 📈 Advanced Analytics
+- **Statistical Analysis**: Distribution plots for returns, trade duration, and P&L
+- **Risk Metrics**: Comprehensive risk analysis including VaR, CVaR, and risk-adjusted returns
+- **Market Correlation**: Analyze strategy performance relative to market conditions
+- **Time-based Analysis**: Performance breakdown by hour, day, and month
+
+### 🔍 Trade Insights
+- **Trade Clustering**: Identify patterns in winning and losing trades
+- **Entry/Exit Analysis**: Evaluate the effectiveness of entry and exit signals
+- **Position Sizing**: Analyze the impact of position sizes on overall performance
+- **Fee Impact**: Understand how trading fees affect profitability
+
+## Usage Instructions
+
+### 1. Select Backtests
+- Choose one or more completed backtests from the dropdown menu
+- Filter backtests by date range, strategy type, or performance metrics
+- Load historical backtests from saved results
+
+### 2. Configure Analysis
+- Select the metrics and visualizations you want to display
+- Set date ranges for focused analysis
+- Choose comparison benchmarks (e.g., buy-and-hold, market indices)
+
+### 3. Analyze Results
+- Review performance summary cards showing key metrics
+- Explore interactive charts by zooming, panning, and hovering for details
+- Export analysis results as reports (PDF/CSV)
+- Save analysis configurations for future use
+
+### 4. Compare Strategies
+- Add multiple backtests to the comparison view
+- Align backtests by date for fair comparison
+- Identify which strategies perform best under different market conditions
+
+## Technical Notes
+
+### Data Processing
+- Backtesting results are loaded from the backend storage system
+- Large datasets are processed incrementally for optimal performance
+- Caching is implemented for frequently accessed analysis results
+
+### Visualization Components
+- **Plotly**: Interactive charts with zoom, pan, and export capabilities
+- **Pandas**: Efficient data manipulation and statistical calculations
+- **NumPy**: High-performance numerical computations
+
+### Performance Considerations
+- Analysis of large backtests (>10,000 trades) may take several seconds
+- Charts are rendered progressively to maintain UI responsiveness
+- Memory usage is optimized through data chunking
+
+## Component Structure
+
+```
+analyze/
+├── analyze.py           # Main page application
+├── components/
+│   ├── metrics.py      # Performance metric calculations
+│   ├── charts.py       # Visualization components
+│   └── comparison.py   # Multi-backtest comparison tools
+└── utils/
+    ├── data_loader.py  # Backtest data loading utilities
+    └── statistics.py   # Statistical analysis functions
+```
+
+## Error Handling
+
+The analysis page includes robust error handling for:
+- **Missing Data**: Graceful handling when backtest data is incomplete
+- **Calculation Errors**: Safe fallbacks for metric calculations
+- **Memory Limits**: Automatic data sampling for very large datasets
+- **Visualization Errors**: Alternative displays when charts fail to render

pages/backtesting/analyze/analyze.py

@@ -0,0 +1,244 @@
+import json
+import os
+from decimal import Decimal
+
+import streamlit as st
+from hummingbot.core.data_type.common import OrderType, PositionMode, TradeType
+from hummingbot.data_feed.candles_feed.candles_factory import CandlesConfig
+from hummingbot.strategy_v2.strategy_frameworks.data_types import OrderLevel, TripleBarrierConf
+from hummingbot.strategy_v2.strategy_frameworks.directional_trading import DirectionalTradingBacktestingEngine
+from hummingbot.strategy_v2.utils.config_encoder_decoder import ConfigEncoderDecoder
+
+import constants
+from backend.utils.optuna_database_manager import OptunaDBManager
+from backend.utils.os_utils import load_controllers
+from frontend.st_utils import initialize_st_page
+from frontend.visualization.graphs import BacktestingGraphs
+from frontend.visualization.strategy_analysis import StrategyAnalysis
+
+initialize_st_page(title="Analyze", icon="🔬")
+
+BASE_DATA_DIR = "data/backtesting"
+
+
+@st.cache_resource
+def get_databases():
+    sqlite_files = [db_name for db_name in os.listdir(BASE_DATA_DIR) if db_name.endswith(".db")]
+    databases_list = [OptunaDBManager(db, db_root_path=BASE_DATA_DIR) for db in sqlite_files]
+    databases_dict = {database.db_name: database for database in databases_list}
+    return [x.db_name for x in databases_dict.values() if x.status == 'OK']
+
+
+def initialize_session_state_vars():
+    if "strategy_params" not in st.session_state:
+        st.session_state.strategy_params = {}
+    if "backtesting_params" not in st.session_state:
+        st.session_state.backtesting_params = {}
+
+
+initialize_session_state_vars()
+dbs = get_databases()
+if not dbs:
+    st.warning("We couldn't find any Optuna database.")
+    selected_db_name = None
+    selected_db = None
+else:
+    # Select database from selectbox
+    selected_db = st.selectbox("Select your database:", dbs)
+    # Instantiate database manager
+    opt_db = OptunaDBManager(selected_db, db_root_path=BASE_DATA_DIR)
+    # Load studies
+    studies = opt_db.load_studies()
+    # Choose study
+    study_selected = st.selectbox("Select a study:", studies.keys())
+    # Filter trials from selected study
+    merged_df = opt_db.merged_df[opt_db.merged_df["study_name"] == study_selected]
+    filters_column, scatter_column = st.columns([1, 6])
+    with filters_column:
+        accuracy = st.slider("Accuracy", min_value=0.0, max_value=1.0, value=[0.4, 1.0], step=0.01)
+        net_profit = st.slider("Net PNL (%)", min_value=merged_df["net_pnl_pct"].min(),
+                               max_value=merged_df["net_pnl_pct"].max(),
+                               value=[merged_df["net_pnl_pct"].min(), merged_df["net_pnl_pct"].max()], step=0.01)
+        max_drawdown = st.slider("Max Drawdown (%)", min_value=merged_df["max_drawdown_pct"].min(),
+                                 max_value=merged_df["max_drawdown_pct"].max(),
+                                 value=[merged_df["max_drawdown_pct"].min(), merged_df["max_drawdown_pct"].max()],
+                                 step=0.01)
+        total_positions = st.slider("Total Positions", min_value=merged_df["total_positions"].min(),
+                                    max_value=merged_df["total_positions"].max(),
+                                    value=[merged_df["total_positions"].min(), merged_df["total_positions"].max()],
+                                    step=1)
+        net_profit_filter = (merged_df["net_pnl_pct"] >= net_profit[0]) & (merged_df["net_pnl_pct"] <= net_profit[1])
+        accuracy_filter = (merged_df["accuracy"] >= accuracy[0]) & (merged_df["accuracy"] <= accuracy[1])
+        max_drawdown_filter = (merged_df["max_drawdown_pct"] >= max_drawdown[0]) & (
+                merged_df["max_drawdown_pct"] <= max_drawdown[1])
+        total_positions_filter = (merged_df["total_positions"] >= total_positions[0]) & (
+                merged_df["total_positions"] <= total_positions[1])
+    with scatter_column:
+        bt_graphs = BacktestingGraphs(
+            merged_df[net_profit_filter & accuracy_filter & max_drawdown_filter & total_positions_filter])
+        # Show and compare all of the study trials
+        st.plotly_chart(bt_graphs.pnl_vs_maxdrawdown(), use_container_width=True)
+    # Get study trials
+    trials = studies[study_selected]
+    # Choose trial
+    trial_selected = st.selectbox("Select a trial to backtest", list(trials.keys()))
+    trial = trials[trial_selected]
+    # Transform trial config in a dictionary
+    encoder_decoder = ConfigEncoderDecoder(TradeType, OrderType, PositionMode)
+    trial_config = encoder_decoder.decode(json.loads(trial["config"]))
+
+    # Strategy parameters section
+    st.write("## Strategy parameters")
+    # Load strategies (class, config, module)
+    controllers = load_controllers(constants.CONTROLLERS_PATH)
+    # Select strategy
+    controller = controllers[trial_config["strategy_name"]]
+    # Get field schema
+    field_schema = controller["config"].schema()["properties"]
+
+    columns = st.columns(4)
+    column_index = 0
+    for field_name, properties in field_schema.items():
+        field_type = properties.get("type", "string")
+        field_value = trial_config[field_name]
+        if field_name not in ["candles_config", "order_levels", "position_mode"]:
+            with columns[column_index]:
+                if field_type in ["number", "integer"]:
+                    field_value = st.number_input(field_name,
+                                                  value=field_value,
+                                                  min_value=properties.get("minimum"),
+                                                  max_value=properties.get("maximum"),
+                                                  key=field_name)
+                elif field_type == "string":
+                    field_value = st.text_input(field_name, value=field_value)
+                elif field_type == "boolean":
+                    # TODO: Add support for boolean fields in optimize tab
+                    field_value = st.checkbox(field_name, value=field_value)
+                else:
+                    raise ValueError("Field type {field_type} not supported")
+        else:
+            if field_name == "candles_config":
+                st.write("---")
+                st.write("## Candles Config:")
+                candles = []
+                for i, candles_config in enumerate(field_value):
+                    st.write(f"#### Candle {i}:")
+                    c11, c12, c13, c14 = st.columns(4)
+                    with c11:
+                        connector = st.text_input("Connector", value=candles_config["connector"])
+                    with c12:
+                        trading_pair = st.text_input("Trading pair", value=candles_config["trading_pair"])
+                    with c13:
+                        interval = st.text_input("Interval", value=candles_config["interval"])
+                    with c14:
+                        max_records = st.number_input("Max records", value=candles_config["max_records"])
+                    st.write("---")
+                    candles.append(CandlesConfig(connector=connector, trading_pair=trading_pair, interval=interval,
+                                                 max_records=max_records))
+                field_value = candles
+            elif field_name == "order_levels":
+                new_levels = []
+                st.write("## Order Levels:")
+                for order_level in field_value:
+                    st.write(f"### Level {order_level['level']} {order_level['side'].name}")
+                    ol_c1, ol_c2 = st.columns([5, 1])
+                    with ol_c1:
+                        st.write("#### Triple Barrier config:")
+                        c21, c22, c23, c24, c25 = st.columns(5)
+                        triple_barrier_conf_level = order_level["triple_barrier_conf"]
+                        with c21:
+                            take_profit = st.number_input("Take profit",
+                                                          value=float(triple_barrier_conf_level["take_profit"]),
+                                                          key=f"{order_level['level']}_{order_level['side'].name}_tp")
+                        with c22:
+                            stop_loss = st.number_input("Stop Loss",
+                                                        value=float(triple_barrier_conf_level["stop_loss"]),
+                                                        key=f"{order_level['level']}_{order_level['side'].name}_sl")
+                        with c23:
+                            time_limit = st.number_input("Time Limit", value=triple_barrier_conf_level["time_limit"],
+                                                         key=f"{order_level['level']}_{order_level['side'].name}_tl")
+                        with c24:
+                            ts_ap = st.number_input("Trailing Stop Activation Price", value=float(
+                                triple_barrier_conf_level["trailing_stop_activation_price_delta"]),
+                                                    key=f"{order_level['level']}_{order_level['side'].name}_tsap",
+                                                    format="%.4f")
+                        with c25:
+                            ts_td = st.number_input("Trailing Stop Trailing Delta", value=float(
+                                triple_barrier_conf_level["trailing_stop_trailing_delta"]),
+                                                    key=f"{order_level['level']}_{order_level['side'].name}_tstd",
+                                                    format="%.4f")
+                    with ol_c2:
+                        st.write("#### Position config:")
+                        c31, c32 = st.columns(2)
+                        with c31:
+                            order_amount = st.number_input("Order amount USD",
+                                                           value=float(order_level["order_amount_usd"]),
+                                                           key=f"{order_level['level']}_{order_level['side'].name}_oa")
+                        with c32:
+                            cooldown_time = st.number_input("Cooldown time", value=order_level["cooldown_time"],
+                                                            key=f"{order_level['level']}_{order_level['side'].name}_cd")
+                        triple_barrier_conf = TripleBarrierConf(stop_loss=Decimal(stop_loss),
+                                                                take_profit=Decimal(take_profit),
+                                                                time_limit=time_limit,
+                                                                trailing_stop_activation_price_delta=Decimal(ts_ap),
+                                                                trailing_stop_trailing_delta=Decimal(ts_td),
+                                                                open_order_type=OrderType.MARKET)
+                        new_levels.append(OrderLevel(level=order_level["level"], side=order_level["side"],
+                                                     order_amount_usd=order_amount, cooldown_time=cooldown_time,
+                                                     triple_barrier_conf=triple_barrier_conf))
+                    st.write("---")
+
+                field_value = new_levels
+            elif field_name == "position_mode":
+                field_value = PositionMode.HEDGE
+            else:
+                field_value = None
+        st.session_state["strategy_params"][field_name] = field_value
+
+        column_index = (column_index + 1) % 4
+
+    st.write("### Backtesting period")
+    col1, col2, col3, col4 = st.columns([1, 1, 1, 0.5])
+    with col1:
+        trade_cost = st.number_input("Trade cost",
+                                     value=0.0006,
+                                     min_value=0.0001, format="%.4f", )
+    with col2:
+        initial_portfolio_usd = st.number_input("Initial portfolio usd",
+                                                value=10000.00,
+                                                min_value=1.00,
+                                                max_value=999999999.99)
+    with col3:
+        start = st.text_input("Start", value="2023-01-01")
+        end = st.text_input("End", value="2024-01-01")
+    c1, c2 = st.columns([1, 1])
+    with col4:
+        add_positions = st.checkbox("Add positions", value=True)
+        add_volume = st.checkbox("Add volume", value=True)
+        add_pnl = st.checkbox("Add PnL", value=True)
+        save_config = st.button("💾Save controller config!")
+        config = controller["config"](**st.session_state["strategy_params"])
+        controller = controller["class"](config=config)
+        if save_config:
+            encoder_decoder = ConfigEncoderDecoder(TradeType, OrderType, PositionMode)
+            encoder_decoder.yaml_dump(config.dict(),
+                                      f"hummingbot_files/controller_configs/{config.strategy_name}_{trial_selected}.yml")
+        run_backtesting_button = st.button("⚙️Run Backtesting!")
+    if run_backtesting_button:
+        try:
+            engine = DirectionalTradingBacktestingEngine(controller=controller)
+            engine.load_controller_data("./data/candles")
+            backtesting_results = engine.run_backtesting(initial_portfolio_usd=initial_portfolio_usd,
+                                                         trade_cost=trade_cost,
+                                                         start=start, end=end)
+            strategy_analysis = StrategyAnalysis(
+                positions=backtesting_results["executors_df"],
+                candles_df=backtesting_results["processed_data"],
+            )
+            metrics_container = BacktestingGraphs(backtesting_results["processed_data"]).get_trial_metrics(
+                strategy_analysis,
+                add_positions=add_positions,
+                add_volume=add_volume)
+
+        except FileNotFoundError:
+            st.warning("The requested candles could not be found.")

pages/backtesting/create/README.md

@@ -0,0 +1,105 @@
+# Backtesting Creation
+
+The Backtesting Creation page enables you to design, configure, and launch backtests for various trading strategies using historical market data.
+
+## Features
+
+### 🎯 Strategy Configuration
+- **Pre-built Strategy Templates**: Choose from popular strategies like PMM, XEMM, Grid, and Bollinger Bands
+- **Custom Parameter Settings**: Fine-tune strategy parameters including spreads, order amounts, and risk limits
+- **Multi-Exchange Support**: Backtest strategies across different exchanges and trading pairs
+- **Position Mode Selection**: Test strategies in ONE-WAY or HEDGE position modes
+
+### 📅 Backtest Setup
+- **Historical Data Selection**: Choose date ranges for backtesting with available market data
+- **Timeframe Configuration**: Select candle intervals (1m, 5m, 15m, 1h, 1d)
+- **Initial Portfolio Settings**: Set starting balances for base and quote currencies
+- **Fee Structure**: Configure maker/taker fees to match real trading conditions
+
+### 🚀 Execution Options
+- **Single Backtest**: Run individual backtests with specific configurations
+- **Batch Testing**: Queue multiple backtests with different parameters
+- **Optimization Mode**: Automatically test parameter ranges to find optimal settings
+- **Real-time Progress**: Monitor backtest execution with live progress updates
+
+## Usage Instructions
+
+### 1. Select Strategy
+- Choose a strategy type from the dropdown menu
+- Review the strategy description and requirements
+- Load a saved configuration or start with defaults
+
+### 2. Configure Parameters
+- **Trading Pair**: Select the market to backtest (e.g., BTC-USDT)
+- **Date Range**: Set start and end dates for historical data
+- **Strategy Parameters**: Adjust strategy-specific settings
+  - Spread percentages
+  - Order amounts and levels
+  - Risk management thresholds
+  - Refresh intervals
+
+### 3. Set Initial Conditions
+- **Starting Balance**: Define initial holdings in base and quote currencies
+- **Leverage**: Set leverage for perpetual/futures markets (1x for spot)
+- **Fees**: Input maker and taker fee percentages
+
+### 4. Launch Backtest
+- Review all settings in the configuration summary
+- Click "Run Backtest" to start execution
+- Monitor progress in the status panel
+- Access results in the Analyze page once complete
+
+## Technical Notes
+
+### Data Requirements
+- Historical candle data must be available for the selected date range
+- Order book snapshots are simulated based on historical spreads
+- Trade data is used for volume-weighted calculations
+
+### Execution Engine
+- **Event-Driven Simulation**: Tick-by-tick processing of market events
+- **Order Matching**: Realistic order filling based on historical liquidity
+- **Latency Simulation**: Configurable delays to model real-world conditions
+
+### Performance Optimization
+- Backtests run on the backend server for optimal performance
+- Large date ranges are processed in chunks to prevent memory issues
+- Results are streamed to the UI as they become available
+
+## Component Structure
+
+```
+create/
+├── create.py              # Main page application
+├── components/
+│   ├── strategy_selector.py   # Strategy selection interface
+│   ├── parameter_form.py      # Dynamic parameter input forms
+│   └── backtest_launcher.py   # Backtest execution controls
+└── configs/
+    ├── strategy_defaults.py   # Default configurations
+    └── validation.py          # Parameter validation rules
+```
+
+## Supported Strategies
+
+### Market Making
+- **Pure Market Making (PMM)**: Continuous bid/ask placement around mid-price
+- **Cross-Exchange Market Making (XEMM)**: Arbitrage between exchanges
+- **Perpetual Market Making**: Strategies for perpetual futures
+
+### Directional
+- **Bollinger Bands**: Mean reversion based on volatility bands
+- **MACD + Bollinger**: Combined momentum and volatility signals
+- **SuperTrend**: Trend-following with dynamic stops
+
+### Grid Trading
+- **Grid Strike**: Fixed-interval grid with customizable ranges
+- **Dynamic Grid**: Adaptive grid based on market volatility
+
+## Error Handling
+
+The creation page handles various error scenarios:
+- **Invalid Parameters**: Real-time validation with helpful error messages
+- **Insufficient Data**: Clear warnings when historical data is missing
+- **Configuration Conflicts**: Automatic detection of incompatible settings
+- **Server Errors**: Graceful fallbacks with retry options

pages/backtesting/create/create.py

@@ -0,0 +1,47 @@
+from types import SimpleNamespace
+
+import streamlit as st
+from streamlit_elements import elements, mui
+
+from frontend.components.controllers_file_explorer import ControllersFileExplorer
+from frontend.components.dashboard import Dashboard
+from frontend.components.directional_strategy_creation_card import DirectionalStrategyCreationCard
+from frontend.components.editor import Editor
+from frontend.st_utils import initialize_st_page
+
+initialize_st_page(title="Create", icon="️⚔️")
+
+# TODO:
+#    * Add videos explaining how to the triple barrier method works and how the backtesting is designed,
+#  link to video of how to create a strategy, etc in a toggle.
+#    * Add functionality to start strategy creation from scratch or by duplicating an existing one
+
+if "ds_board" not in st.session_state:
+    board = Dashboard()
+    ds_board = SimpleNamespace(
+        dashboard=board,
+        create_strategy_card=DirectionalStrategyCreationCard(board, 0, 0, 12, 1),
+        file_explorer=ControllersFileExplorer(board, 0, 2, 3, 7),
+        editor=Editor(board, 4, 2, 9, 7),
+    )
+    st.session_state.ds_board = ds_board
+
+else:
+    ds_board = st.session_state.ds_board
+
+# Add new tabs
+for tab_name, content in ds_board.file_explorer.tabs.items():
+    if tab_name not in ds_board.editor.tabs:
+        ds_board.editor.add_tab(tab_name, content["content"], content["language"])
+
+# Remove deleted tabs
+for tab_name in list(ds_board.editor.tabs.keys()):
+    if tab_name not in ds_board.file_explorer.tabs:
+        ds_board.editor.remove_tab(tab_name)
+
+with elements("directional_strategies"):
+    with mui.Paper(elevation=3, style={"padding": "2rem"}, spacing=[2, 2], container=True):
+        with ds_board.dashboard():
+            ds_board.create_strategy_card()
+            ds_board.file_explorer()
+            ds_board.editor()

pages/backtesting/optimize/README.md

@@ -0,0 +1,132 @@
+# Backtesting Optimization
+
+The Backtesting Optimization page provides powerful tools to find optimal trading strategy parameters through systematic testing and analysis.
+
+## Features
+
+### 🔧 Parameter Optimization
+- **Grid Search**: Test all combinations of parameter values systematically
+- **Random Search**: Efficiently explore large parameter spaces
+- **Genetic Algorithms**: Evolve parameters using natural selection principles
+- **Bayesian Optimization**: Smart parameter search using probabilistic models
+
+### 📊 Optimization Targets
+- **Maximize Sharpe Ratio**: Optimize for risk-adjusted returns
+- **Maximize Total P&L**: Focus on absolute profit maximization
+- **Minimize Drawdown**: Prioritize capital preservation
+- **Custom Objectives**: Define multi-objective optimization functions
+
+### 🎯 Parameter Configuration
+- **Range Definition**: Set min/max values for each parameter
+- **Step Sizes**: Define granularity of parameter search
+- **Constraints**: Apply realistic bounds and relationships
+- **Parameter Groups**: Test correlated parameters together
+
+### 📈 Results Analysis
+- **3D Surface Plots**: Visualize parameter interactions
+- **Heatmaps**: Identify optimal parameter regions
+- **Parallel Coordinates**: Explore high-dimensional results
+- **Performance Rankings**: Compare top parameter combinations
+
+## Usage Instructions
+
+### 1. Select Base Strategy
+- Choose the strategy to optimize from available backtests
+- Load the baseline configuration as starting point
+- Review historical performance metrics
+
+### 2. Define Parameter Space
+- **Select Parameters**: Choose which parameters to optimize
+- **Set Ranges**: Define minimum and maximum values
+  - Spreads: 0.1% - 5.0%
+  - Order amounts: 10% - 100%
+  - Risk limits: 0.5% - 10%
+- **Configure Steps**: Set increment sizes for each parameter
+
+### 3. Configure Optimization
+- **Algorithm**: Select optimization method
+  - Grid Search: Complete but computationally intensive
+  - Random Search: Good for initial exploration
+  - Bayesian: Efficient for expensive evaluations
+- **Objective Function**: Choose what to optimize
+- **Constraints**: Set practical limitations
+- **Iterations**: Define search budget
+
+### 4. Run Optimization
+- Review estimated runtime and resource usage
+- Start optimization process
+- Monitor real-time progress and intermediate results
+- Pause/resume long-running optimizations
+
+### 5. Analyze Results
+- View top performing parameter sets
+- Explore parameter sensitivity analysis
+- Export optimal configurations
+- Create ensemble strategies from top performers
+
+## Technical Notes
+
+### Optimization Engine
+- **Parallel Processing**: Multiple backtests run simultaneously
+- **Distributed Computing**: Leverage multiple CPU cores
+- **Memory Management**: Efficient handling of large result sets
+- **Checkpointing**: Save progress for long optimizations
+
+### Search Algorithms
+- **Grid Search**: Exhaustive search with deterministic coverage
+- **Random Search**: Monte Carlo sampling with proven efficiency
+- **Bayesian Optimization**: Gaussian Process regression for smart search
+- **Genetic Algorithms**: Population-based evolutionary optimization
+
+### Performance Metrics
+- **Primary Metrics**: Sharpe ratio, total return, maximum drawdown
+- **Risk Metrics**: VaR, CVaR, Sortino ratio, Calmar ratio
+- **Trade Metrics**: Win rate, profit factor, average trade P&L
+- **Stability Metrics**: Return consistency, strategy robustness
+
+## Component Structure
+
+```
+optimize/
+├── optimize.py              # Main optimization interface
+├── engines/
+│   ├── grid_search.py      # Grid search implementation
+│   ├── random_search.py    # Random search algorithm
+│   ├── bayesian.py         # Bayesian optimization
+│   └── genetic.py          # Genetic algorithm
+├── objectives/
+│   ├── metrics.py          # Objective function definitions
+│   └── constraints.py      # Constraint handling
+└── visualization/
+    ├── surfaces.py         # 3D parameter surfaces
+    ├── heatmaps.py         # 2D optimization heatmaps
+    └── parallel_coords.py  # Multi-dimensional plots
+```
+
+## Best Practices
+
+### Parameter Selection
+- Start with 2-3 most impactful parameters
+- Use domain knowledge to set reasonable ranges
+- Consider parameter interactions and dependencies
+- Validate results with out-of-sample data
+
+### Optimization Strategy
+- Begin with coarse grid search for exploration
+- Refine with Bayesian optimization
+- Validate top results with extended backtests
+- Test robustness with walk-forward analysis
+
+### Resource Management
+- Estimate computational requirements upfront
+- Use random search for high-dimensional spaces
+- Implement early stopping for poor performers
+- Save intermediate results frequently
+
+## Error Handling
+
+The optimization page includes comprehensive error handling:
+- **Parameter Validation**: Ensures valid parameter ranges and relationships
+- **Resource Limits**: Prevents system overload with job queuing
+- **Convergence Detection**: Identifies when optimization plateaus
+- **Result Validation**: Checks for numerical stability and outliers

pages/backtesting/optimize/optimize.py

@@ -0,0 +1,62 @@
+import time
+import webbrowser
+from types import SimpleNamespace
+
+import streamlit as st
+from streamlit_elements import elements, mui
+
+from backend.utils import os_utils
+from frontend.components.dashboard import Dashboard
+from frontend.components.editor import Editor
+from frontend.components.optimization_creation_card import OptimizationCreationCard
+from frontend.components.optimization_run_card import OptimizationRunCard
+from frontend.components.optimizations_file_explorer import OptimizationsStrategiesFileExplorer
+from frontend.st_utils import initialize_st_page
+
+initialize_st_page(title="Optimize", icon="🧪")
+
+
+def run_optuna_dashboard():
+    os_utils.execute_bash_command("optuna-dashboard sqlite:///data/backtesting/backtesting_report.db")
+    time.sleep(5)
+    webbrowser.open("http://127.0.0.1:8080/dashboard", new=2)
+
+
+if "op_board" not in st.session_state:
+    board = Dashboard()
+    op_board = SimpleNamespace(
+        dashboard=board,
+        create_optimization_card=OptimizationCreationCard(board, 0, 0, 6, 1),
+        run_optimization_card=OptimizationRunCard(board, 6, 0, 6, 1),
+        file_explorer=OptimizationsStrategiesFileExplorer(board, 0, 2, 3, 7),
+        editor=Editor(board, 4, 2, 9, 7),
+    )
+    st.session_state.op_board = op_board
+
+else:
+    op_board = st.session_state.op_board
+
+# Add new tabs
+for tab_name, content in op_board.file_explorer.tabs.items():
+    if tab_name not in op_board.editor.tabs:
+        op_board.editor.add_tab(tab_name, content["content"], content["language"])
+
+# Remove deleted tabs
+for tab_name in list(op_board.editor.tabs.keys()):
+    if tab_name not in op_board.file_explorer.tabs:
+        op_board.editor.remove_tab(tab_name)
+
+with elements("optimizations"):
+    with mui.Paper(elevation=3, style={"padding": "2rem"}, spacing=[2, 2], container=True):
+        with mui.Grid(container=True, spacing=2):
+            with mui.Grid(item=True, xs=10):
+                pass
+            with mui.Grid(item=True, xs=2):
+                with mui.Fab(variant="extended", color="primary", size="large", onClick=run_optuna_dashboard):
+                    mui.Typography("Open Optuna Dashboard", variant="body1")
+
+        with op_board.dashboard():
+            op_board.create_optimization_card()
+            op_board.run_optimization_card()
+            op_board.file_explorer()
+            op_board.editor()