specs/openspec/changes/archive/2026-10-09-reconcile-hourly-investment-sync/design.md

Engineering Design

Context and Constraints

The deployed backend is a single NestJS process with SQLite/Prisma. Its durable hourly walk, connection revision publication fence, saved investment resources, and frontend 30-minute rereads already provide most of the approved model. The remaining mismatch is at the resource boundary: backend InvestmentResourceMetaDto and investmentResourceMeta expose legacy missing|partial|complete availability and idle|running|failed work state, while the frontend projects that legacy metadata through useResourceQuery and uses sync=observe as a transport mode. The generated backend openapi.json and frontend snapshot agree with each other, but they describe this AS-IS contract rather than the approved target.

The backend-generated OpenAPI remains the executable contract. The change-local system-analysis/contracts/investment-resource-envelope.schema.json is a semantic target and must not become a second OpenAPI source, generated client, or runtime validator. Route identities, authentication/access boundaries, Problem Details codes, and resource payload DTOs remain established unless an approved target requirement explicitly changes them.

No normal personal resource GET may wait for or call T-Bank. The only acquisition exceptions remain the first unconfirmed ledger interval and first unconfirmed candle range. The design preserves the single-process boundary; it does not add distributed scheduling, a job service, polling endpoint, manual sync command, public value-history read, or a new UI package dependency.

Technical Goals / Non-Goals

Goals

• Publish one generated HTTP envelope for Accounts, Portfolio, operations, and events: { data, meta }, with Accounts-only top-level sync.
• Make resource-result state and hourly-attempt state independently truthful: confirmed data stays available after a failed/partial attempt, while an initial result remains null until complete.
• Preserve local-read, connection-revision, and first-load invariants while migrating Swagger DTOs, mapper inputs, runtime responses, and frontend generated types together.
• Keep the existing successful TanStack Query snapshot during read retries and retain the existing 30-minute open-screen reread cadence as a local HTTP reread, not an import trigger.

Non-Goals

• Rebuild source owners, hourly scheduling, portfolio-value storage, or the legacy status endpoint beyond removing their use as a source-work command from these screens.
• Introduce a compatibility envelope, client-side DTO normalizer, duplicate cache, new route/version, manual refresh/import action, or a copy of OpenAPI in specs.
• Plan documentation edits, implementation tasks, or product behavior beyond the approved proposal, delta specs, and System Analysis.

Implementation Impact Map

frontend

affected - generated transport types, shared/api/resource-query.ts, the existing brokerage-account/portfolio/operations/events query owners, and their page VMs/views consume the target metadata. useScreenSync remains the common 30-minute open-screen reread owner, but its source-error derivation must read meta.lastAttempt rather than legacy meta.sync; retry stays HTTP-only.

backend

affected - investment resource DTOs, mapper, endpoint response DTOs, resource readers, connection/hourly-attempt projections, Swagger annotations, and generated openapi.json implement the semantic envelope. Existing durable attempt, revision, membership, snapshot, forecast, coverage, and history owners remain the data owners.

specs

affected - this Design makes the approved deltas and System Analysis implementable without duplicating the executable contract.

docs

affected - durable AS-IS references are limited to ADR 019, ADR 017, architecture/cross-cutting/data-freshness.md, domains/portfolio/flows/portfolio-loading.md, and domains/income/upcoming-events.md. Their impact is recorded here only; documentation sequencing is not designed in this artifact.

qa

N/A - no separate QA/E2E repository exists.

Decisions

Backend: one resource-meta projection and one connection-walk projection

Replace the legacy shared metadata vocabulary at the existing InvestmentResourceMetaDto/investmentResourceMeta seam. Every affected resource envelope has required meta.freshness, meta.completeness, meta.lastSuccessfulAt, meta.lastAttempt, and meta.availability. lastAttempt is a resource-scoped projection with state none|running|succeeded|partial|failed, nullable start/end times, and only safe failed section/account/error-code entries. none is a concrete status, not an omitted field or success.

The mapper derives freshness and successful time only from the returned confirmed resource result. A later running, partial, or failed walk neither changes that time nor makes data fresh. data: null pairs with unknown/ incomplete and either initial-loading, temporarily-unavailable, or connection-unavailable; a confirmed empty collection pairs with available and complete. The ledger's permitted first-load rows retain incomplete coverage and are never represented as a confirmed complete result.

Accounts additionally projects the connection-wide walk as sync: null when no attempt exists, otherwise the same complete safe attempt shape. Other resource envelopes do not gain top-level sync. This preserves the single shared metadata owner while avoiding the current ambiguity between per-resource meta.sync and Accounts-level walk status.

The existing resourceKey, dataRevision, coverage, and dependency details remain internal projection inputs or endpoint-specific metadata only where their approved behavior still requires publication; they must not substitute for any of the five required target meanings. No client must infer target state from their presence.

Backend: retain storage owners and publication fences

The durable hourly attempt remains the sole normal publisher. Resource readers read the saved current connection revision and project its persisted result; they do not reserve work or contact upstream. Each successful section can persist independently, preserving the previous confirmed resource result and its success time when a later section fails. Forecast publication remains complete-source-only, and exact per-currency value-history writes remain in the same successful Portfolio publication transaction.

Connection PUT establishes a new revision before its first walk; DELETE and replacement prevent old personal snapshots and old attempt status from being read or published. Invalid-token knowledge blocks disclosure rather than being masked by saved data. Data and storage changes must be additive when a new persisted distinction is necessary; existing snapshots, coverage, forecasts, attempts, revisions, and retained value history are not reset or reclassified by a destructive migration.

Contract: Swagger is produced at the DTO boundary

Swagger DTO annotations and endpoint response classes are the only place the wire schema is authored. The backend regenerates openapi.json from the implemented DTOs, then the frontend receives that published snapshot and runs its existing openapi-typescript generation. Hand-editing frontend schema.d.ts, adding handwritten parallel interfaces, or retaining a compatibility mapper would hide drift and is excluded.

The resulting generated contract must express nullable data, all required meta fields, nullable attempt timestamps, nullable problem fields, Accounts top-level nullable sync, and absence of top-level sync on the other resource responses. It must retain sync=observe only as a read-observation compatibility parameter while it exists in the generated route contract; it cannot cause source work and is not used to represent the new hourly state. The generated contract reconciles field spelling with the target schema before release, not through a second contract artifact.

Frontend: one query cache and metadata-aware screen state

useResourceQuery remains the sole adapter that projects resource envelopes into data, meta, and Accounts sync, preserving its existing query key, authenticated transport, access-group invalidation, and Client Cached Snapshot behavior. It must project the generated target types directly. Resource query owners remain at their existing entity slices; page VMs, not presentation views, choose initial loading, unavailable/error, empty, and loaded states.

useScreenSync remains the single 30-minute timer for open investment screens. It calls ordinary local refetch() and its retry path continues to use the existing observation read without changing cache identity. It derives server attempt warning state from meta.lastAttempt, keeps transport failures separate, and never starts personal synchronization. Existing Accounts 30-minute staleTime remains aligned with this cadence; Portfolio and other screen resources keep their current short cache freshness where already established while the open-screen timer supplies the approved reread rhythm.

All affected account screens render the five metadata meanings in text or an element with an accessible name, not color alone. A complete empty resource is normal empty content. data: null renders initial-loading or unavailable states without invented values. A failed/partial hourly attempt is an independent nonblocking warning beside confirmed data and its original time. Portfolio.isStale remains snapshot-specific and is shown separately from hourly-attempt and client HTTP-error warnings. A client snapshot stays visible during and after a failed background HTTP reread; connection replacement or deletion still invalidates personal queries.

Cross-cutting Implementation

The canonical state split is:

Concern	Owner	Observable representation
Returned confirmed resource result	Saved resource owner	data, freshness, completeness, lastSuccessfulAt, availability
Resource-relevant hourly outcome	Durable walk and resource projection	meta.lastAttempt
Connection-wide hourly outcome	Durable walk	Accounts sync only
In-flight/failed browser reread	TanStack Query	Existing query lifecycle and local retry warning
Snapshot-specific Portfolio source age	Portfolio owner	Portfolio.isStale

This split prevents a source failure from becoming a synchronous GET failure, prevents the UI from treating its cache as server fallback, and preserves safe access invalidation. Safe attempt problems contain only the approved section, optional account ID, and safe error code: no token, upstream payload, cursor, connection revision, or raw exception text is serialized or logged as status metadata.

Events retains its existing account and all scopes and its saved forecast window/filter semantics. Its metadata independently reports the latest events attempt, so a saved forecast can remain available when a later refresh fails. Operations retains its coverage semantics; only its explicitly approved missing historical first-load acquisition may work outside the hourly walk.

Data / Storage Changes

No new storage architecture is needed. Existing durable tables/records for the current connection revision, hourly attempts, snapshots, operation coverage, forecast source windows, and retained portfolio-value points are authoritative. If implementation needs persisted terminal-state or problem-detail values not already durable, it extends the existing attempt record/schema additively and backfills conservatively: absent historical attempt information is projected as none, never as succeeded.

Migrations must preserve all current user, connection, personal-resource, coverage, forecast, and history rows. They must not reset SQLite, rewrite an old partial result as complete, expose retained history across a new connection boundary, or delete old data merely to fit the new vocabulary. Compatibility is achieved by deploying the backend DTO/runtime and its generated OpenAPI as one server contract, followed by the frontend snapshot/types and rendering change; there is no claim that an old frontend understands the target envelope.

Contract Implementation

The implementation contract is the generated result of the following mapping:

Target concept	Backend source	HTTP location
Last confirmed result or no complete result	Existing local resource reader	data
Result freshness and completeness	Resource mapper from confirmed result/coverage	meta.freshness, meta.completeness
Original confirmed-result time	Saved source timestamp	meta.lastSuccessfulAt
Relevant hourly result	Durable attempt plus resource section projection	meta.lastAttempt
Whether complete data can be used	Connection/access/read result	meta.availability
Complete current-connection walk	Durable connection-wide attempt	Accounts sync

The target schema's problemSection shape is used for both scopes. For a connection-wide Accounts walk, its list may include all affected sections; for a resource-level attempt, it is restricted to failures relevant to that resource. A running attempt has finishedAt: null; terminal attempts have a completion time; none has null timestamps and no problems. These invariants are validated at runtime tests and in the generated OpenAPI, rather than by shipping the JSON Schema to application consumers.

Developer Verification Strategy

Backend verification proves the contract at DTO, mapper, controller, and SQLite boundaries: successful empty data; first load with data: null; stale fallback after a failed walk; partial walk with preserved data; resource versus Accounts attempt scoping; events forecast-window filtering; invalid-token and connection-revision disclosure fences; no upstream personal call on ordinary GET; and both first-load exceptions staying narrow. It regenerates OpenAPI and runs the existing OpenAPI contract, type, lint, and relevant E2E suites.

Frontend verification updates the supplied snapshot, runs openapi:check, and uses MSW/page-VM tests to prove generated metadata projection, no automatic retry, HTTP-only retry, 30-minute open-screen reread, cache retention after a background error, connection invalidation, and distinct server stale/hourly attempt/client transport states. UI checks cover keyboard access, accessible status text, loading/error/empty/loaded states, and narrow viewport wrapping.

Before release, verify the backend-generated OpenAPI and frontend snapshot are the same published artifact, then run each repository's configured verification commands. In this specs store, run both v2 schema validation and strict change validation; structural success is supplemented by a semantic comparison against the approved deltas and System Analysis.

UI/UX Package

Applicability and scope: new/changed. This is an inline package for the existing Accounts, account Portfolio, operations, and events screens. It adds the approved status vocabulary and preserves 30-minute local rereads; it does not introduce new composition, graphics, navigation, or controls.

Package revision: 1 (this Design). No visual files, external references,

Screens, states and interactions:

Screen/state	Presentation	Interaction
No complete result, initial attempt running	Existing structural loading state, labelled as loading	No manual sync action
No complete result, attempt failed/unavailable	Safe unavailable/error state with target status text	Retry rereads HTTP only when recovery allows
Complete empty result	Normal domain empty state and target status text	Existing navigation/actions only
Complete data with running/succeeded/partial/failed attempt	Content and original success time remain visible; attempt appears as separate accessible status	No source-work action
Client Cached Snapshot during/after HTTP failure	Existing content remains visible; transport warning remains distinct from server state	Existing retry rereads HTTP only

Responsive behavior: reuse existing Mantine/theme components and page layouts. Status text wraps without hiding account content or money values on narrow screens; color supplements an accessible label. Existing skeletons remain noninteractive and unfocusable. Existing cards, buttons, and icons are used through their public package exports; no wrapper, SVG asset, token, or component package revision is introduced.

Reused source and differences: N/A for approved visual reuse. This is a new inline status package revision 1, not an immutable reuse of a prior approved visual package. Existing cards, layouts, hooks, and UI packages are implementation inputs only; they are not the approved visual source. The package adds semantic presentation of the five target metadata fields and separate hourly-attempt/client-transport warnings. No local visual export is required.

Risks / Trade-offs

• Envelope evolution is a coordinated backend/frontend release risk. Generated artifacts prevent hidden type drift but cannot make an older client interpret new enum meanings.
• Mapping resource relevance from connection-wide attempts can over-report or under-report a failure if it is not constrained by resource/section scope; mapper tests must cover each affected resource.
• A durable none projection for old rows is intentionally less informative than inventing a historic success; it preserves truthfulness.
• The 30-minute client timer can issue local reads while an hourly walk runs; this is acceptable because reads do not contend for upstream work and retain the last confirmed result.
• Single-process scheduling remains unsuitable for multi-process deployment; adding leases, a worker, or distributed coordination requires a separate decision.
• Documentation currently contains legacy request-driven/manual-refresh descriptions. Until the identified durable references are updated, they must not be used as implementation authority over the approved behavior.

Migration / Rollout / Rollback

Rollout is backend-first at the contract source: apply only additive persistence changes if required, implement and test runtime projection/fences, regenerate and publish backend OpenAPI, then update the frontend snapshot/generated types and renderers. Do not enable the new projection alongside a second personal write owner. A frontend built from the old snapshot is not the compatibility target; rollout uses the matched generated artifact pair.

Field	Decision
Status	required
Trigger	Personal-data disclosure across connection revisions, false complete/fresh state, malformed generated contract, failure to preserve confirmed fallback, or a frontend unable to read the published contract.
Actions	Stop the affected rollout, preserve a SQLite copy, return to the last compatible backend/frontend pair, and fix forward. Do not restore screen-triggered/manual synchronization while the hourly owner remains enabled.
Data safety	Never reset production SQLite, delete additive attempt metadata, delete snapshots/coverage/forecasts/value history, or use a destructive downgrade. Old rows without target details safely project none.
Verification	On a representative copy, verify migration integrity, revision/access fences, ordinary local GET behavior, preserved fallback timestamps, generated OpenAPI/snapshot identity, and frontend rendering before traffic resumes.

Solution Assessment

Option	Assessment
Evolve the existing shared DTO/mapper and generated OpenAPI	Chosen. It is the narrowest path: one resource-meta projection already owns all affected envelope metadata, Swagger already produces the contract, and frontend types already derive from the supplied snapshot.
Add a target adapter/parallel endpoint or versioned envelope	Rejected. It duplicates mapping and cache paths, delays removal of ambiguous states, and creates an unnecessary second compatibility surface.
Handwrite frontend target types or runtime validation	Rejected. Generated paths/schemas already own transport typing; manual copies would drift and a validator adds no trust boundary.
Add a scheduler/queue/cache service to express status	Rejected. Durable single-process attempts and SQLite records already express the required state. A new service cannot improve this contract without changing the explicitly retained deployment model.
Add a new design-system package/component family	Rejected. Existing Mantine and @wealthledger/ui states cover the textual, accessible status presentation; a new package would be abstraction without a new visual primitive.

The chosen solution is intentionally a contract-and-projection reconciliation, not a new synchronization system. It minimizes code and migration surface while making the approved observable model expressible through the existing owners.