maneshtrader/ManeshTrader_Architecture.md

ManeshTrader - Architecture Plan

Executive Summary

ManeshTrader is an analysis-only trading intelligence system that classifies OHLC bars into real/fake signals, derives trend state from real bars, and delivers visual insights, exports, and optional monitoring alerts. The architecture refactor moves from a single-process UI-first design to a modular, service-oriented design that improves testability, scalability, observability, and operational resilience.

System Context

flowchart LR
    Trader[Trader / Analyst] -->|Configure symbol, timeframe, filters| ManeshTrader[ManeshTrader Analysis Platform]
    ManeshTrader -->|Fetch OHLC| Yahoo[Yahoo Finance Data API]
    ManeshTrader -->|Optional notifications| Notify[Email / Push / Webhook]
    ManeshTrader -->|Export CSV/PDF| Storage[(User Downloads / Object Storage)]
    Admin[Operator] -->|Monitor health, logs, metrics| ManeshTrader

Overview: Defines external actors and dependencies around ManeshTrader.

Key Components: Trader, Operator, data provider, notification endpoints, export targets.

Relationships: Traders submit analysis requests; platform retrieves market data, computes classifications/trends, and returns charted/exported output.

Design Decisions: Keep execution out-of-scope (analysis-only boundary). External market source decoupled behind adapter interface.

NFR Considerations:

• Scalability: Stateless analysis workloads can scale horizontally.
• Performance: Cached market data and precomputed features reduce latency.
• Security: Read-only market data; no brokerage keys for execution.
• Reliability: Retry/fallback on upstream data issues.
• Maintainability: Clear system boundary reduces coupling.

Trade-offs: Dependency on third-party data availability/quality.

Risks and Mitigations:

• Risk: API throttling/outage. Mitigation: caching, backoff, alternate provider adapter.

Architecture Overview

The target architecture uses modular services with explicit boundaries:

• Presentation: Web UI / API consumers
• Application: Strategy orchestration and workflow
• Domain: Bar classification and trend state machine
• Data: Provider adapters, cache, persistence
• Platform: auth, observability, notifications, exports

Component Architecture

flowchart TB
    UI[Web UI / Streamlit Frontend]
    API[Analysis API Gateway]
    ORCH[Analysis Orchestrator]
    STRAT[Strategy Engine
(Real/Fake Classifier + Trend State Machine)]
    BT[Backtest Evaluator]
    ALERT[Alerting Service]
    EXPORT[Export Service
(CSV/PDF)]
    MKT[Market Data Adapter]
    CACHE[(Market Cache)]
    DB[(Analysis Store)]
    OBS[Observability
Logs/Metrics/Traces]

    UI --> API
    API --> ORCH
    ORCH --> STRAT
    ORCH --> BT
    ORCH --> ALERT
    ORCH --> EXPORT
    ORCH --> MKT
    MKT --> CACHE
    MKT --> ORCH
    ORCH --> DB
    API --> DB
    API --> OBS
    ORCH --> OBS
    STRAT --> OBS
    ALERT --> OBS

Overview: Internal modular decomposition for the refactored system.

Key Components:

• Analysis API Gateway: request validation and rate limiting.
• Analysis Orchestrator: coordinates data fetch, strategy execution, and response assembly.
• Strategy Engine: deterministic classification and trend transitions.
• Backtest Evaluator: lightweight historical scoring.
• Alerting/Export Services: asynchronous side effects.
• Market Data Adapter + Cache: provider abstraction and performance buffer.

Relationships: API delegates to orchestrator; orchestrator composes domain and side-effect services; observability is cross-cutting.

Design Decisions: Separate deterministic core logic from IO-heavy integrations.

NFR Considerations:

• Scalability: independent scaling for API, strategy workers, and alert/export processors.
• Performance: cache first, compute second, persist last.
• Security: centralized input validation and API policy.
• Reliability: queue-based side effects isolate failures.
• Maintainability: single-responsibility services and clear contracts.

Trade-offs: More infrastructure and operational complexity than monolith.

Risks and Mitigations:

• Risk: distributed debugging complexity. Mitigation: trace IDs and structured logs.

Deployment Architecture

flowchart TB
    subgraph Internet
      U[User Browser]
    end

    subgraph Cloud[VPC]
      LB[HTTPS Load Balancer / WAF]
      subgraph AppSubnet[Private App Subnet]
        UI[UI Service]
        API[API Service]
        WRK[Worker Service]
      end
      subgraph DataSubnet[Private Data Subnet]
        REDIS[(Redis Cache)]
        PG[(PostgreSQL)]
        MQ[(Message Queue)]
        OBJ[(Object Storage)]
      end
      OBS[Managed Monitoring]
      SECRET[Secrets Manager]
    end

    EXT[Yahoo Finance / Alt Provider]

    U --> LB --> UI
    UI --> API
    API --> REDIS
    API --> PG
    API --> MQ
    WRK --> MQ
    WRK --> PG
    WRK --> OBJ
    API --> EXT
    WRK --> EXT
    API --> OBS
    WRK --> OBS
    API --> SECRET
    WRK --> SECRET

Overview: Logical production deployment with secure segmentation.

Key Components: WAF/LB edge, stateless app services, queue-driven workers, managed data stores.

Relationships: Request path stays synchronous through UI/API; heavy export/alert tasks handled asynchronously by workers.

Design Decisions: Network segmentation and managed services for resilience and lower ops overhead.

NFR Considerations:

• Scalability: autoscaling app and worker tiers.
• Performance: Redis cache and async task offload.
• Security: private subnets, secret manager, TLS at edge.
• Reliability: managed DB backup, queue durability.
• Maintainability: environment parity across dev/staging/prod.

Trade-offs: Managed services cost vs operational simplicity.

Risks and Mitigations:

• Risk: queue backlog under spikes. Mitigation: autoscaling workers and dead-letter queues.

Data Flow

flowchart LR
    R[User Request
symbol/timeframe/filters] --> V[Input Validation]
    V --> C1{Cache Hit?}
    C1 -->|Yes| D1[Load OHLC from Cache]
    C1 -->|No| D2[Fetch OHLC from Provider]
    D2 --> D3[Normalize + Time Alignment]
    D3 --> D4[Persist to Cache]
    D1 --> P1[Classify Bars
real_bull/real_bear/fake]
    D4 --> P1
    P1 --> P2[Trend State Machine
2-bar confirmation]
    P2 --> P3[Backtest Snapshot]
    P3 --> P4[Build Response Model]
    P4 --> S1[(Analysis Store)]
    P4 --> O[UI Chart + Metrics + Events]
    P4 --> E[Export Job CSV/PDF]
    P4 --> A[Alert Job]

Overview: End-to-end data processing lifecycle for each analysis request.

Key Components: validation, cache/provider ingestion, classification/trend processing, output assembly.

Relationships: deterministic analytics pipeline with optional async exports/alerts.

Design Decisions: Normalize data before strategy logic for deterministic outcomes.

NFR Considerations:

• Scalability: compute pipeline can be parallelized per request.
• Performance: cache avoids repeated provider calls.
• Security: strict input schema and output sanitization.
• Reliability: idempotent processing and persisted analysis snapshots.
• Maintainability: explicit stage boundaries simplify test coverage.

Trade-offs: Additional persistence adds write latency.

Risks and Mitigations:

• Risk: inconsistent timestamps across providers. Mitigation: canonical UTC normalization.

Key Workflows

sequenceDiagram
    participant User
    participant UI
    participant API
    participant Data as Market Adapter
    participant Engine as Strategy Engine
    participant Alert as Alert Service
    participant Export as Export Service

    User->>UI: Submit symbol/timeframe/filters
    UI->>API: Analyze request
    API->>Data: Get closed OHLC bars
    Data-->>API: Normalized bars
    API->>Engine: Classify + detect trend
    Engine-->>API: trend state, events, metrics
    API-->>UI: chart model + events + backtest
    alt New trend/reversal event
      API->>Alert: Publish notification task
    end
    opt User requests export
      UI->>API: Export CSV/PDF
      API->>Export: Generate artifact
      Export-->>UI: Download link/blob
    end

Overview: Critical user flow from request to insight/alert/export.

Key Components: UI, API, data adapter, strategy engine, alert/export services.

Relationships: synchronous analysis, asynchronous side effects.

Design Decisions: Keep analytical response fast; move non-critical tasks to background.

NFR Considerations:

• Scalability: alert/export can scale separately.
• Performance: response prioritizes analysis payload.
• Security: permission checks around export and notification endpoints.
• Reliability: retries for failed async tasks.
• Maintainability: workflow contracts versioned via API schema.

Trade-offs: eventual consistency for async outputs.

Risks and Mitigations:

• Risk: duplicate alerts after retries. Mitigation: idempotency keys by event hash.

Additional Diagram: Domain State Model

stateDiagram-v2
    [*] --> Neutral
    Neutral --> Bullish: 2 consecutive real_bull
    Neutral --> Bearish: 2 consecutive real_bear
    Bullish --> Bullish: fake OR single real_bear fluke
    Bearish --> Bearish: fake OR single real_bull fluke
    Bullish --> Bearish: 2 consecutive real_bear
    Bearish --> Bullish: 2 consecutive real_bull

Overview: Strategy state transitions based on confirmed real-bar sequences.

Key Components: Neutral, Bullish, Bearish states; confirmation conditions.

Relationships: fake bars never reverse state; opposite single bar is non-reversal noise.

Design Decisions: enforce confirmation to reduce whipsaw.

NFR Considerations:

• Scalability: pure function state machine enables easy horizontal compute.
• Performance: O(n) per bar sequence.
• Security: deterministic logic reduces ambiguity and operator error.
• Reliability: explicit transitions avoid hidden side effects.
• Maintainability: state model is test-friendly and auditable.

Trade-offs: delayed reversals in fast inflection markets.

Risks and Mitigations:

• Risk: late entries due to confirmation lag. Mitigation: optional “early warning” non-trading signal.

Phased Development

Phase 1: Initial Implementation

• Single deployable web app with embedded analysis module.
• Basic data adapter, core strategy engine, charting, CSV export.
• Local logs and lightweight metrics.

Phase 2+: Final Architecture

• Split API, workers, and dedicated alert/export services.
• Add cache + persistent analysis store + queue-driven async tasks.
• Multi-provider market adapter and hardened observability.

Migration Path

1. Extract strategy logic into standalone domain module with unit tests.
2. Introduce API boundary and typed request/response contracts.
3. Externalize side effects (alerts/exports) into worker queue.
4. Add Redis caching and persistent analysis snapshots.
5. Enable multi-environment CI/CD and infrastructure-as-code.

Non-Functional Requirements Analysis

Scalability

Stateless API/services with autoscaling; async workers for bursty jobs; provider/caching abstraction to reduce upstream load.

Performance

Cache-first ingestion, bounded bar windows, O(n) classification/state processing, deferred heavy exports.

Security

WAF/TLS, secrets manager, strict request validation, RBAC for admin controls, audit logs for alert/export actions.

Reliability

Queue retry policies, dead-letter queues, health probes, circuit breakers for upstream data sources, backup/restore for persistent stores.

Maintainability

Layered architecture, clear contracts, domain isolation, test pyramid (unit/contract/integration), observability-first operations.

Risks and Mitigations

• Upstream data inconsistency: normalize timestamps and schema at adapter boundary.
• Alert noise: debounce and idempotency keyed by symbol/timeframe/event timestamp.
• Cost growth with scale: autoscaling guardrails, TTL caches, export retention policy.
• Strategy misinterpretation: publish explicit strategy rules and state transition docs in product UI.

Technology Stack Recommendations

• Frontend: Streamlit (MVP) then React/Next.js for multi-user production UX.
• API: FastAPI with pydantic contracts.
• Workers: Celery/RQ with Redis or managed queue.
• Storage: PostgreSQL for analysis metadata; object storage for export artifacts.
• Observability: OpenTelemetry + managed logging/metrics dashboards.
• Deployment: Containerized services on managed Kubernetes or serverless containers.

Next Steps

1. Approve the phased architecture and target operating model.
2. Define API contracts for analysis request/response and event schema.
3. Implement Phase 1 module boundaries (domain/application/infrastructure).
4. Add core test suite for classification and trend state transitions.
5. Plan Phase 2 service split and infrastructure rollout.