(feat) update pages

2026-05-19 07:26:44 +00:00 · 2025-07-11 02:57:57 +03:00
parent 91cd3d0365
commit e44fa89283
39 changed files with 6036 additions and 780 deletions
--- a/pages/data/README.md
+++ b/pages/data/README.md
@@ -0,0 +1,202 @@
+# Data Module
+
+## Page Purpose and Functionality
+
+The Data module provides tools for accessing, downloading, and analyzing market data essential for trading strategy development and analysis. It offers interfaces for historical data retrieval, real-time market analysis, and specialized data visualizations to support informed trading decisions.
+
+## Key Features
+
+### 1. Download Candles (`/download_candles`)
+- Download historical candlestick data from multiple exchanges
+- Support for various timeframes (1m, 3m, 5m, 15m, 1h, 4h, 1d)
+- Interactive candlestick chart visualization
+- Export capabilities for offline analysis
+- Automatic data validation and gap detection
+
+### 2. Token Spreads (`/token_spreads`)
+- Real-time bid-ask spread analysis across exchanges
+- Cross-exchange arbitrage opportunity detection
+- Historical spread tracking and trends
+- Volatility analysis based on spread behavior
+- Multi-pair comparison capabilities
+
+### 3. TVL vs Market Cap (`/tvl_vs_mcap`)
+- DeFi protocol analysis comparing Total Value Locked to Market Capitalization
+- Fundamental analysis metrics for token valuation
+- Historical TVL/MCap ratio tracking
+- Protocol comparison and ranking
+- Integration with DeFi data providers
+
+## User Flow
+
+1. **Historical Data Collection**
+   - User selects exchange and trading pair
+   - Specifies date range and candle interval
+   - Downloads data with progress tracking
+   - Visualizes data quality and completeness
+   - Exports for strategy development
+
+2. **Market Analysis**
+   - User monitors real-time spreads
+   - Identifies arbitrage opportunities
+   - Analyzes market efficiency
+   - Tracks spread patterns over time
+   - Sets alerts for spread thresholds
+
+3. **Fundamental Analysis**
+   - User selects DeFi protocols or tokens
+   - Compares TVL and market cap metrics
+   - Identifies potentially undervalued assets
+   - Tracks metric changes over time
+   - Exports analysis results
+
+## Technical Implementation Details
+
+### Data Architecture
+- **Data Sources**: Direct exchange APIs and aggregated data providers
+- **Storage Format**: Optimized parquet files for efficient querying
+- **Caching Strategy**: Multi-level caching for API responses
+- **Update Mechanism**: Incremental updates to minimize API calls
+
+### API Integration
+```python
+# Exchange data retrieval pattern
+backend_api_client.market_data.get_historical_candles(
+    connector="exchange_name",
+    trading_pair="BASE-QUOTE",
+    interval="timeframe",
+    start_time=timestamp,
+    end_time=timestamp
+)
+```
+
+### Data Processing Pipeline
+1. Raw data retrieval from exchanges
+2. Data validation and cleaning
+3. Gap filling and interpolation where appropriate
+4. Aggregation and resampling
+5. Storage in optimized format
+
+## Component Dependencies
+
+### Internal Dependencies
+- `backend.services.backend_api_client`: Market data API interface
+- `frontend.st_utils`: Streamlit utilities
+- `frontend.visualization`: Chart and graph components
+
+### External Dependencies
+- `pandas`: Data manipulation and analysis
+- `plotly`: Interactive charting
+- `numpy`: Numerical computations
+- `streamlit`: Web interface
+
+### Data Storage
+- `data/candles/`: Historical candlestick data
+- `data/spreads/`: Spread analysis results
+- `data/tvl/`: TVL and market cap data
+
+## State Management Approach
+
+### Session State Variables
+- `selected_exchange`: Current exchange selection
+- `selected_pairs`: Active trading pairs
+- `date_range`: Selected time period
+- `chart_settings`: Visualization preferences
+- `cached_data`: Recently fetched data
+
+### Data Caching Strategy
+- **Memory Cache**: Recent API responses (5-minute TTL)
+- **Disk Cache**: Historical data (permanent until invalidated)
+- **Session Cache**: User-specific selections and results
+
+### Real-time Updates
+- WebSocket connections for live data
+- Polling fallback for unsupported exchanges
+- Automatic reconnection handling
+- Rate limiting compliance
+
+## Best Practices
+
+1. **Data Quality**
+   - Always validate downloaded data for gaps
+   - Check for anomalous values (e.g., zero prices)
+   - Verify timestamp consistency
+   - Handle exchange downtime gracefully
+
+2. **Performance Optimization**
+   - Batch API requests when possible
+   - Use appropriate data granularity
+   - Implement progressive loading for large datasets
+   - Optimize chart rendering for large data
+
+3. **User Experience**
+   - Show download progress clearly
+   - Provide data quality indicators
+   - Enable easy data export
+   - Cache frequently accessed data
+
+4. **Error Handling**
+   - Graceful handling of API failures
+   - Clear error messages with solutions
+   - Automatic retry with exponential backoff
+   - Fallback data sources when available
+
+## Data Specifications
+
+### Candle Data Format
+```python
+{
+    "timestamp": int,  # Unix timestamp
+    "open": float,
+    "high": float,
+    "low": float,
+    "close": float,
+    "volume": float,
+    "quote_volume": float  # Optional
+}
+```
+
+### Spread Data Format
+```python
+{
+    "timestamp": int,
+    "exchange": str,
+    "trading_pair": str,
+    "bid": float,
+    "ask": float,
+    "spread": float,  # ask - bid
+    "spread_pct": float  # spread / mid_price
+}
+```
+
+### TVL Data Format
+```python
+{
+    "timestamp": int,
+    "protocol": str,
+    "tvl_usd": float,
+    "market_cap_usd": float,
+    "tvl_mcap_ratio": float,
+    "change_24h": float  # Percentage
+}
+```
+
+## Advanced Features
+
+### Data Analysis Tools
+- Moving averages and technical indicators
+- Correlation analysis between pairs
+- Volatility calculations
+- Market microstructure metrics
+
+### Export Capabilities
+- CSV export for spreadsheet analysis
+- JSON export for programmatic access
+- Direct integration with backtesting module
+- API endpoints for external access
+
+### Visualization Options
+- Candlestick charts with overlays
+- Spread heatmaps
+- Time series comparisons
+- Distribution analysis
--- a/pages/data/download_candles/app.py
+++ b/pages/data/download_candles/app.py
@@ -27,9 +27,12 @@ with c4:
 if get_data_button:
    start_datetime = datetime.combine(start_date, time.min)
    end_datetime = datetime.combine(end_date, time.max)
+    if end_datetime < start_datetime:
+        st.error("End Date should be greater than Start Date.")
+        st.stop()

-    candles = backend_api_client.get_historical_candles(
-        connector=connector,
+    candles = backend_api_client.market_data.get_historical_candles(
+        connector_name=connector,
        trading_pair=trading_pair,
        interval=interval,
        start_time=int(start_datetime.timestamp()),
@@ -45,9 +48,7 @@ if get_data_button:
        open=candles_df['open'],
        high=candles_df['high'],
        low=candles_df['low'],
-        close=candles_df['close'],
-        increasing_line_color='#2ECC71',
-        decreasing_line_color='#E74C3C'
+        close=candles_df['close']
    )])
    fig.update_layout(
        height=1000,