maneshtrader/ManeshTrader_Architecture.md

# ManeshTrader - Architecture Plan

## Executive Summary
ManeshTrader is an analysis-only trading intelligence system that classifies OHLC bars into real/fake signals, derives trend state from real bars, and delivers visual insights, exports, and optional monitoring alerts. The architecture refactor moves from a single-process UI-first design to a modular, service-oriented design that improves testability, scalability, observability, and operational resilience.

## System Context
```mermaid
flowchart LR
    Trader[Trader / Analyst] -->|Configure symbol, timeframe, filters| ManeshTrader[ManeshTrader Analysis Platform]
    ManeshTrader -->|Fetch OHLC| Yahoo[Yahoo Finance Data API]
    ManeshTrader -->|Optional notifications| Notify[Email / Push / Webhook]
    ManeshTrader -->|Export CSV/PDF| Storage[(User Downloads / Object Storage)]
    Admin[Operator] -->|Monitor health, logs, metrics| ManeshTrader
```

Overview: Defines external actors and dependencies around ManeshTrader.

Key Components: Trader, Operator, data provider, notification endpoints, export targets.

Relationships: Traders submit analysis requests; platform retrieves market data, computes classifications/trends, and returns charted/exported output.

Design Decisions: Keep execution out-of-scope (analysis-only boundary). External market source decoupled behind adapter interface.

NFR Considerations:
- Scalability: Stateless analysis workloads can scale horizontally.
- Performance: Cached market data and precomputed features reduce latency.
- Security: Read-only market data; no brokerage keys for execution.
- Reliability: Retry/fallback on upstream data issues.
- Maintainability: Clear system boundary reduces coupling.

Trade-offs: Dependency on third-party data availability/quality.

Risks and Mitigations:
- Risk: API throttling/outage. Mitigation: caching, backoff, alternate provider adapter.

## Architecture Overview
The target architecture uses modular services with explicit boundaries:
- Presentation: Web UI / API consumers
- Application: Strategy orchestration and workflow
- Domain: Bar classification and trend state machine
- Data: Provider adapters, cache, persistence
- Platform: auth, observability, notifications, exports

## Component Architecture
```mermaid
flowchart TB
    UI[Web UI / Streamlit Frontend]
    API[Analysis API Gateway]
    ORCH[Analysis Orchestrator]
    STRAT[Strategy Engine
(Real/Fake Classifier + Trend State Machine)]
    BT[Backtest Evaluator]
    ALERT[Alerting Service]
    EXPORT[Export Service
(CSV/PDF)]
    MKT[Market Data Adapter]
    CACHE[(Market Cache)]
    DB[(Analysis Store)]
    OBS[Observability
Logs/Metrics/Traces]

    UI --> API
    API --> ORCH
    ORCH --> STRAT
    ORCH --> BT
    ORCH --> ALERT
    ORCH --> EXPORT
    ORCH --> MKT
    MKT --> CACHE
    MKT --> ORCH
    ORCH --> DB
    API --> DB
    API --> OBS
    ORCH --> OBS
    STRAT --> OBS
    ALERT --> OBS
```

Overview: Internal modular decomposition for the refactored system.

Key Components:
- Analysis API Gateway: request validation and rate limiting.
- Analysis Orchestrator: coordinates data fetch, strategy execution, and response assembly.
- Strategy Engine: deterministic classification and trend transitions.
- Backtest Evaluator: lightweight historical scoring.
- Alerting/Export Services: asynchronous side effects.
- Market Data Adapter + Cache: provider abstraction and performance buffer.

Relationships: API delegates to orchestrator; orchestrator composes domain and side-effect services; observability is cross-cutting.

Design Decisions: Separate deterministic core logic from IO-heavy integrations.

NFR Considerations:
- Scalability: independent scaling for API, strategy workers, and alert/export processors.
- Performance: cache first, compute second, persist last.
- Security: centralized input validation and API policy.
- Reliability: queue-based side effects isolate failures.
- Maintainability: single-responsibility services and clear contracts.

Trade-offs: More infrastructure and operational complexity than monolith.

Risks and Mitigations:
- Risk: distributed debugging complexity. Mitigation: trace IDs and structured logs.

## Deployment Architecture
```mermaid
flowchart TB
    subgraph Internet
      U[User Browser]
    end

    subgraph Cloud[VPC]
      LB[HTTPS Load Balancer / WAF]
      subgraph AppSubnet[Private App Subnet]
        UI[UI Service]
        API[API Service]
        WRK[Worker Service]
      end
      subgraph DataSubnet[Private Data Subnet]
        REDIS[(Redis Cache)]
        PG[(PostgreSQL)]
        MQ[(Message Queue)]
        OBJ[(Object Storage)]
      end
      OBS[Managed Monitoring]
      SECRET[Secrets Manager]
    end

    EXT[Yahoo Finance / Alt Provider]

    U --> LB --> UI
    UI --> API
    API --> REDIS
    API --> PG
    API --> MQ
    WRK --> MQ
    WRK --> PG
    WRK --> OBJ
    API --> EXT
    WRK --> EXT
    API --> OBS
    WRK --> OBS
    API --> SECRET
    WRK --> SECRET
```

Overview: Logical production deployment with secure segmentation.

Key Components: WAF/LB edge, stateless app services, queue-driven workers, managed data stores.

Relationships: Request path stays synchronous through UI/API; heavy export/alert tasks handled asynchronously by workers.

Design Decisions: Network segmentation and managed services for resilience and lower ops overhead.

NFR Considerations:
- Scalability: autoscaling app and worker tiers.
- Performance: Redis cache and async task offload.
- Security: private subnets, secret manager, TLS at edge.
- Reliability: managed DB backup, queue durability.
- Maintainability: environment parity across dev/staging/prod.

Trade-offs: Managed services cost vs operational simplicity.

Risks and Mitigations:
- Risk: queue backlog under spikes. Mitigation: autoscaling workers and dead-letter queues.

## Data Flow
```mermaid
flowchart LR
    R[User Request
symbol/timeframe/filters] --> V[Input Validation]
    V --> C1{Cache Hit?}
    C1 -->|Yes| D1[Load OHLC from Cache]
    C1 -->|No| D2[Fetch OHLC from Provider]
    D2 --> D3[Normalize + Time Alignment]
    D3 --> D4[Persist to Cache]
    D1 --> P1[Classify Bars
real_bull/real_bear/fake]
    D4 --> P1
    P1 --> P2[Trend State Machine
2-bar confirmation]
    P2 --> P3[Backtest Snapshot]
    P3 --> P4[Build Response Model]
    P4 --> S1[(Analysis Store)]
    P4 --> O[UI Chart + Metrics + Events]
    P4 --> E[Export Job CSV/PDF]
    P4 --> A[Alert Job]
```

Overview: End-to-end data processing lifecycle for each analysis request.

Key Components: validation, cache/provider ingestion, classification/trend processing, output assembly.

Relationships: deterministic analytics pipeline with optional async exports/alerts.

Design Decisions: Normalize data before strategy logic for deterministic outcomes.

NFR Considerations:
- Scalability: compute pipeline can be parallelized per request.
- Performance: cache avoids repeated provider calls.
- Security: strict input schema and output sanitization.
- Reliability: idempotent processing and persisted analysis snapshots.
- Maintainability: explicit stage boundaries simplify test coverage.

Trade-offs: Additional persistence adds write latency.

Risks and Mitigations:
- Risk: inconsistent timestamps across providers. Mitigation: canonical UTC normalization.

## Key Workflows
```mermaid
sequenceDiagram
    participant User
    participant UI
    participant API
    participant Data as Market Adapter
    participant Engine as Strategy Engine
    participant Alert as Alert Service
    participant Export as Export Service

    User->>UI: Submit symbol/timeframe/filters
    UI->>API: Analyze request
    API->>Data: Get closed OHLC bars
    Data-->>API: Normalized bars
    API->>Engine: Classify + detect trend
    Engine-->>API: trend state, events, metrics
    API-->>UI: chart model + events + backtest
    alt New trend/reversal event
      API->>Alert: Publish notification task
    end
    opt User requests export
      UI->>API: Export CSV/PDF
      API->>Export: Generate artifact
      Export-->>UI: Download link/blob
    end
```

Overview: Critical user flow from request to insight/alert/export.

Key Components: UI, API, data adapter, strategy engine, alert/export services.

Relationships: synchronous analysis, asynchronous side effects.

Design Decisions: Keep analytical response fast; move non-critical tasks to background.

NFR Considerations:
- Scalability: alert/export can scale separately.
- Performance: response prioritizes analysis payload.
- Security: permission checks around export and notification endpoints.
- Reliability: retries for failed async tasks.
- Maintainability: workflow contracts versioned via API schema.

Trade-offs: eventual consistency for async outputs.

Risks and Mitigations:
- Risk: duplicate alerts after retries. Mitigation: idempotency keys by event hash.

## Additional Diagram: Domain State Model
```mermaid
stateDiagram-v2
    [*] --> Neutral
    Neutral --> Bullish: 2 consecutive real_bull
    Neutral --> Bearish: 2 consecutive real_bear
    Bullish --> Bullish: fake OR single real_bear fluke
    Bearish --> Bearish: fake OR single real_bull fluke
    Bullish --> Bearish: 2 consecutive real_bear
    Bearish --> Bullish: 2 consecutive real_bull
```

Overview: Strategy state transitions based on confirmed real-bar sequences.

Key Components: Neutral, Bullish, Bearish states; confirmation conditions.

Relationships: fake bars never reverse state; opposite single bar is non-reversal noise.

Design Decisions: enforce confirmation to reduce whipsaw.

NFR Considerations:
- Scalability: pure function state machine enables easy horizontal compute.
- Performance: O(n) per bar sequence.
- Security: deterministic logic reduces ambiguity and operator error.
- Reliability: explicit transitions avoid hidden side effects.
- Maintainability: state model is test-friendly and auditable.

Trade-offs: delayed reversals in fast inflection markets.

Risks and Mitigations:
- Risk: late entries due to confirmation lag. Mitigation: optional “early warning” non-trading signal.

## Phased Development

### Phase 1: Initial Implementation
- Single deployable web app with embedded analysis module.
- Basic data adapter, core strategy engine, charting, CSV export.
- Local logs and lightweight metrics.

### Phase 2+: Final Architecture
- Split API, workers, and dedicated alert/export services.
- Add cache + persistent analysis store + queue-driven async tasks.
- Multi-provider market adapter and hardened observability.

### Migration Path
1. Extract strategy logic into standalone domain module with unit tests.
2. Introduce API boundary and typed request/response contracts.
3. Externalize side effects (alerts/exports) into worker queue.
4. Add Redis caching and persistent analysis snapshots.
5. Enable multi-environment CI/CD and infrastructure-as-code.

## Non-Functional Requirements Analysis

### Scalability
Stateless API/services with autoscaling; async workers for bursty jobs; provider/caching abstraction to reduce upstream load.

### Performance
Cache-first ingestion, bounded bar windows, O(n) classification/state processing, deferred heavy exports.

### Security
WAF/TLS, secrets manager, strict request validation, RBAC for admin controls, audit logs for alert/export actions.

### Reliability
Queue retry policies, dead-letter queues, health probes, circuit breakers for upstream data sources, backup/restore for persistent stores.

### Maintainability
Layered architecture, clear contracts, domain isolation, test pyramid (unit/contract/integration), observability-first operations.

## Risks and Mitigations
- Upstream data inconsistency: normalize timestamps and schema at adapter boundary.
- Alert noise: debounce and idempotency keyed by symbol/timeframe/event timestamp.
- Cost growth with scale: autoscaling guardrails, TTL caches, export retention policy.
- Strategy misinterpretation: publish explicit strategy rules and state transition docs in product UI.

## Technology Stack Recommendations
- Frontend: Streamlit (MVP) then React/Next.js for multi-user production UX.
- API: FastAPI with pydantic contracts.
- Workers: Celery/RQ with Redis or managed queue.
- Storage: PostgreSQL for analysis metadata; object storage for export artifacts.
- Observability: OpenTelemetry + managed logging/metrics dashboards.
- Deployment: Containerized services on managed Kubernetes or serverless containers.

## Next Steps
1. Approve the phased architecture and target operating model.
2. Define API contracts for analysis request/response and event schema.
3. Implement Phase 1 module boundaries (domain/application/infrastructure).
4. Add core test suite for classification and trend state transitions.
5. Plan Phase 2 service split and infrastructure rollout.