specs/openspec/changes/archive/2026-10-09-reconcile-hourly-investment-sync/system-analysis/analysis.md

# System Analysis

## Requirement Coverage

| Approved delta | System decision covered here |
| --- | --- |
| `hourly-investment-sync`: hourly walk and observable result | Per-connection durable hourly attempt, no catch-up or overlap, and `lastAttempt`/Accounts `sync` semantics. |
| `tbank-data-sync`: one update owner and observable resource state | Local resource reads, two narrow first-load exceptions, and the target `{ data, meta }` contract. |
| `brokerage-accounts` | Saved Accounts and per-account summaries, plus the connection-wide hourly result exposed as `sync`. |
| `portfolio-view` and `portfolio-value-history` | Saved complete snapshots, stale fallback, connection-revision publication fence, and private retained value history. |
| `brokerage-account-ledger` | First opening of an unconfirmed historical interval may load only that interval; complete history remains local. |
| `brokerage-account-upcoming-events` | Saved forecast query/window semantics, complete-source publication, and independent failed-update status. |
| `online-instrument-information` | Online reference lookup is separate from personal-resource reads and cannot delay or alter their status. |
| `frontend-data-resilience` | A retry reissues only HTTP GET; frontend cache, server fallback, and hourly-attempt failure remain distinct. |
| `tbank-connection` | PUT starts the new connection's first walk; replacement/deletion hides old personal data and old walk status. |

## Systems and Responsibilities

| System | Responsibility | Must not do |
| --- | --- | --- |
| Frontend | Read resources, render `data` and all observable `meta` fields, retain its own successful screen cache, and retry only the GET. | Start personal sync, infer freshness from a client error, or expose a manual personal-sync action. |
| Backend resource API | Authenticate, enforce the current connection boundary, read local persisted resources, and return the target envelope. | Wait for or call T-Bank while serving ordinary resource GETs. |
| Durable hourly scheduler | Reserve at most one current walk per connection; refresh membership, each available Portfolio, mutable operations tail, then events. | Replay missed terms, queue an overlapped term, or refresh complete historical ledger intervals/candle ranges. |
| Local store | Keep last confirmed complete data, coverage, resource timestamps, hourly attempt/result, and connection revision. | Publish data or status from a superseded connection revision. |
| T-Bank and public reference sources | Supply upstream data to connection PUT validation, the hourly walk, and explicitly online reference lookups. | Become a synchronous dependency of a normal personal resource GET. |

## Interaction Flows

`sequences/read-only-resource-get.mmd` defines the normal read and retry boundary. A `200` read uses only local state; a boundary error may occur before any data lookup.

`sequences/hourly-investment-walk.mmd` defines the sole normal personal-data updater. A successful connection starts its first attempt immediately. Thereafter a next attempt is eligible no earlier than 60 minutes after the preceding attempt started. Startup performs at most one current attempt, not a replay. If a term arrives while an attempt is running, it is dropped rather than queued; the next eligible term is measured from the running attempt's start.

The walk updates current membership, every available account's Portfolio, each account's mutable operations tail, and forecasts/events for instruments in the current Portfolio. Each successful section persists independently. A failed section preserves its earlier complete result and its original success time. Complete past operation intervals, completed candles, and full market history are not refreshed by the walk.

### First-load exceptions

1. Opening an unconfirmed historical Account Ledger interval may start retrieval of only that missing interval and immediately returns local rows plus incomplete coverage/status. It does not refresh membership, Accounts, Portfolio, events, another operations interval, or candles. A confirmed complete interval is never rechecked due to a GET or the hourly walk.
2. Opening a candle range with no confirmed local result may start retrieval of only that range. It does not retrieve other ranges or all history; a confirmed range is subsequently local.

These are acquisition exceptions, not variants of manual sync. HTTP retry of either resource rereads its local state; it does not broaden or restart work.

## Contracts

`contracts/investment-resource-envelope.schema.json` is the minimal target cross-system artifact. It describes the changed resource envelope rather than duplicating backend `openapi.json`.

All affected personal resource successes use `{ data, meta }`: Accounts; account and `all` Portfolio; account and `all` operations; account and `all` events. `data` is the last confirmed complete result, except that ledger first-load may return locally held rows with `meta.completeness: "incomplete"`. Where no complete result exists, `data` is `null`; a partial result must never be represented as complete.

`meta` has five independent meanings:

| Field | Target meaning |
| --- | --- |
| `freshness` | `fresh`/`stale` applies to the timestamp of the returned confirmed result; `unknown` applies when none exists. A failed or running attempt never makes data fresh. |
| `completeness` | `complete` only after the resource's required source result is confirmed. `incomplete` describes an in-progress/failed first-load or ledger coverage gap. |
| `lastSuccessfulAt` | Timestamp of the returned complete result, or `null` when none exists. It is not replaced by a later failed attempt. |
| `lastAttempt` | The resource-relevant hourly result: `none`, `running`, `succeeded`, `partial`, or `failed`, with nullable start/end timestamps and only safe failed section/account/error-code entries. `partial` means at least one section succeeded and at least one failed. |
| `availability` | `available` has a complete result, including `[]`; `initial-loading` and `temporarily-unavailable` have no complete result; `connection-unavailable` means the current connection cannot expose the resource. |

Accounts also returns top-level `sync`: `null` if the current connection has no hourly attempt, otherwise the complete connection-wide last-walk status. Its `problems` list is the safe, scoped failure list for `membership`, `portfolio`, `operations`, and `events`; an account ID is present only for account-scoped sections. `sync` is observation only and is not a command or progress API.

Events preserves the existing account and `all` scope. `from` and `to` are optional strict `YYYY-MM-DD` UTC dates with inclusive bounds; omitted `from` is the current UTC date and omitted `to` is `from + 30` calendar days. `from > to` returns `400 INVALID_EVENTS_PERIOD`, does not read a substitute range, and starts no source work. A saved forecast retains `calculatedAt` and its original inclusive calculation window. The response filters events to the intersection of that original window and the requested window. Empty intersection and no matching events are successful empty `data` with the same forecast metadata. `all` is complete only if every included account has its required complete forecast sources; it cannot hide a missing account contribution. Events' `meta.lastAttempt` reports the resource-relevant hourly events result independently of the saved forecast, so a failed update can coexist with an available forecast.

## Data and State Semantics

An empty confirmed Accounts list, empty Portfolio, empty event intersection, and empty event list are all available complete results, never missing data. A stale complete result remains available indefinitely until a connection boundary makes it unavailable. `Portfolio.isStale` retains its snapshot-specific meaning and is not a substitute for `meta.lastAttempt` or a frontend request error.

Forecast publication requires complete broker and public-source results for the Portfolio Snapshot used to calculate it. It stores the Portfolio quantity basis, `calculatedAt`, and source window together. A failed refresh cannot publish a partial forecast or relabel an older forecast as calculated from a newer Portfolio.

Successful complete Portfolio evaluations may append one exact, per-currency value-history point at the actual successful Portfolio time. Missing valuation must not be replaced with zero and cannot create a point. Retry/publication of one attempt is idempotent. History survives connection replacement/deletion but is readable only after the current connection confirms access to the same account; no public history-read endpoint is introduced.

The connection revision is the publication fence. PUT validates the candidate token, atomically establishes a new revision and membership boundary, then triggers its first walk. DELETE is idempotent, prevents further token use, and hides current personal data. Old attempts cannot publish into a replacement connection, and neither old `sync` nor retained value history is disclosed across an unconfirmed boundary.

## Error and Recovery Semantics

Authentication errors remain HTTP boundary errors. No connection is `409 TBANK_NOT_CONNECTED`. A token known invalid/unauthorized from an hourly attempt is `400 INVALID_TBANK_TOKEN`; it retains the connection record but blocks personal-data disclosure and must not be masked by fallback data. A requested account outside current confirmed membership is `404`; unknown membership does not authorize an arbitrary account ID. Connection PUT validation failures remain `400 INVALID_TBANK_TOKEN` or `503 TBANK_UNAVAILABLE` and leave the prior connection unchanged.

Temporary upstream failure during an hourly attempt is represented in `meta.lastAttempt` and, on Accounts, `sync`; it is not a synchronous GET failure. With a previous complete result, GET returns `200` and the preserved data. Without one, GET returns `{ data: null, meta }` with `initial-loading` or `temporarily-unavailable`. `503 TBANK_UNAVAILABLE` remains reserved for an HTTP request that itself cannot be served, not a completed background failure.

Frontend recovery is limited to rereading the resource. It retains its Client Cached Snapshot during a background HTTP failure, while separately showing server stale/fallback and the failed hourly attempt where supplied. Replacing/deleting a connection invalidates personal client queries so a late response cannot revive hidden data.

## Observability Semantics

The durable hourly attempt is the observable unit: start time, terminal state, finish time, and safe failed section/account/error-code list. Absence of an attempt is `none` in resource `lastAttempt` and `null` in Accounts `sync`; it is not success. A running attempt has `finishedAt: null`. The status is scoped to the current connection revision and contains no token, upstream payload, cursor, or internal revision.

Resource `lastSuccessfulAt`, freshness, completeness, and availability describe returned `data`; the hourly result describes the attempt. This separation permits a truthful response after partial/failed refresh. Frontend status text or accessible names must expose all five resource meanings and must not rely on color alone.

## Security and NFR at System Boundaries

All personal reads require the authenticated User and current-connection access check. T-Bank tokens remain server-only, encrypted, and absent from response metadata, OpenAPI examples, logs, and status failures. Online instrument reference failure cannot prove an instrument absent or alter personal snapshot freshness.

The scheduler is a single durable reservation owner for the deployed single backend process. The target explicitly does not add multi-process scheduling. Local read latency is isolated from upstream latency because ordinary GETs do not invoke upstream personal data calls. The existing upstream limiter remains the boundary for PUT validation and hourly work.

## Compatibility / Migration

The current backend-generated `openapi.json` and frontend snapshot are identical and already expose resource envelopes, Accounts top-level `sync`, and events `sync=observe`; they are evidence of AS-IS, not the target authority. Current generated `InvestmentResourceMetaDto` uses `availability: missing|partial|complete` plus per-work `sync: idle|running|failed`, while `HourlySyncStatusDto` adds `cancelled`; these do not fully express the approved target's first-load availability, `none|succeeded|partial` attempt distinction, and resource-scoped last attempt. Current `sync=observe` and `/api/v1/tbank-sync/{id}` preserve legacy screen-work observation and must not be used to reintroduce request-started personal synchronization.

Implementation must evolve backend Swagger DTOs/controllers and regenerate backend OpenAPI. The frontend must receive that published snapshot and regenerate its local transport types. The HTTP route identities and established Problem Details codes remain compatible unless the generated target contract explicitly changes them; no copy of the full OpenAPI belongs in this change. The target schema is intentionally additive/semantic and must be reconciled with generated field names before release.

## Durable Documentation Candidates

- `docs/adr/019-hourly-reads-and-online-instrument-reference.md`: clarify that hourly status is connection-scoped and that only the two first-load exceptions may acquire personal/market data.
- `docs/architecture/cross-cutting/data-freshness.md`: replace legacy request-driven freshness table and `sync=observe` language with target resource/attempt separation and `data: null` first-load states.
- `docs/domains/portfolio/flows/portfolio-loading.md`: replace screen-triggered refresh/status flow with local GET and hourly-walk flow.
- `docs/domains/income/upcoming-events.md`: describe saved forecast source-window intersection, `calculatedAt`, and independent failed hourly update state.
- `docs/adr/017-local-tbank-reads-and-background-sync.md`: mark its request-driven/manual-refresh decisions superseded by ADR 019 rather than leaving competing guidance.

## Open Questions

None. The approved Proposal and Delta Specs determine the target semantics; implementation will reconcile field naming and route annotations through backend-generated OpenAPI without treating current generated artifacts as a second authority.