359 lines
21 KiB
Markdown
359 lines
21 KiB
Markdown
# Engineering Design
|
|
|
|
## Context and Constraints
|
|
|
|
The deployed backend is a single NestJS process with SQLite/Prisma. Its durable
|
|
hourly walk, connection revision publication fence, saved investment resources,
|
|
and frontend 30-minute rereads already provide most of the approved model.
|
|
The remaining mismatch is at the resource boundary: backend
|
|
`InvestmentResourceMetaDto` and `investmentResourceMeta` expose legacy
|
|
`missing|partial|complete` availability and `idle|running|failed` work state,
|
|
while the frontend projects that legacy metadata through `useResourceQuery` and
|
|
uses `sync=observe` as a transport mode. The generated backend `openapi.json`
|
|
and frontend snapshot agree with each other, but they describe this AS-IS
|
|
contract rather than the approved target.
|
|
|
|
The backend-generated OpenAPI remains the executable contract. The change-local
|
|
`system-analysis/contracts/investment-resource-envelope.schema.json` is a
|
|
semantic target and must not become a second OpenAPI source, generated client,
|
|
or runtime validator. Route identities, authentication/access boundaries,
|
|
Problem Details codes, and resource payload DTOs remain established unless an
|
|
approved target requirement explicitly changes them.
|
|
|
|
No normal personal resource GET may wait for or call T-Bank. The only
|
|
acquisition exceptions remain the first unconfirmed ledger interval and first
|
|
unconfirmed candle range. The design preserves the single-process boundary; it
|
|
does not add distributed scheduling, a job service, polling endpoint, manual
|
|
sync command, public value-history read, or a new UI package dependency.
|
|
|
|
## Technical Goals / Non-Goals
|
|
|
|
**Goals**
|
|
|
|
- Publish one generated HTTP envelope for Accounts, Portfolio, operations, and
|
|
events: `{ data, meta }`, with Accounts-only top-level `sync`.
|
|
- Make resource-result state and hourly-attempt state independently truthful:
|
|
confirmed data stays available after a failed/partial attempt, while an
|
|
initial result remains `null` until complete.
|
|
- Preserve local-read, connection-revision, and first-load invariants while
|
|
migrating Swagger DTOs, mapper inputs, runtime responses, and frontend
|
|
generated types together.
|
|
- Keep the existing successful TanStack Query snapshot during read retries and
|
|
retain the existing 30-minute open-screen reread cadence as a local HTTP
|
|
reread, not an import trigger.
|
|
|
|
**Non-Goals**
|
|
|
|
- Rebuild source owners, hourly scheduling, portfolio-value storage, or the
|
|
legacy status endpoint beyond removing their use as a source-work command
|
|
from these screens.
|
|
- Introduce a compatibility envelope, client-side DTO normalizer, duplicate
|
|
cache, new route/version, manual refresh/import action, or a copy of OpenAPI
|
|
in specs.
|
|
- Plan documentation edits, implementation tasks, or product behavior beyond
|
|
the approved proposal, delta specs, and System Analysis.
|
|
|
|
## Implementation Impact Map
|
|
|
|
### frontend
|
|
|
|
`affected` - generated transport types, `shared/api/resource-query.ts`, the
|
|
existing brokerage-account/portfolio/operations/events query owners, and their
|
|
page VMs/views consume the target metadata. `useScreenSync` remains the common
|
|
30-minute open-screen reread owner, but its source-error derivation must read
|
|
`meta.lastAttempt` rather than legacy `meta.sync`; retry stays HTTP-only.
|
|
|
|
### backend
|
|
|
|
`affected` - investment resource DTOs, mapper, endpoint response DTOs,
|
|
resource readers, connection/hourly-attempt projections, Swagger annotations,
|
|
and generated `openapi.json` implement the semantic envelope. Existing durable
|
|
attempt, revision, membership, snapshot, forecast, coverage, and history
|
|
owners remain the data owners.
|
|
|
|
### specs
|
|
|
|
`affected` - this Design makes the approved deltas and System Analysis
|
|
implementable without duplicating the executable contract.
|
|
|
|
### docs
|
|
|
|
`affected` - durable AS-IS references are limited to ADR 019, ADR 017,
|
|
`architecture/cross-cutting/data-freshness.md`,
|
|
`domains/portfolio/flows/portfolio-loading.md`, and
|
|
`domains/income/upcoming-events.md`. Their impact is recorded here only;
|
|
documentation sequencing is not designed in this artifact.
|
|
|
|
### qa
|
|
|
|
`N/A` - no separate QA/E2E repository exists.
|
|
|
|
## Decisions
|
|
|
|
### Backend: one resource-meta projection and one connection-walk projection
|
|
|
|
Replace the legacy shared metadata vocabulary at the existing
|
|
`InvestmentResourceMetaDto`/`investmentResourceMeta` seam. Every affected
|
|
resource envelope has required `meta.freshness`, `meta.completeness`,
|
|
`meta.lastSuccessfulAt`, `meta.lastAttempt`, and `meta.availability`.
|
|
`lastAttempt` is a resource-scoped projection with state
|
|
`none|running|succeeded|partial|failed`, nullable start/end times, and only
|
|
safe failed section/account/error-code entries. `none` is a concrete status,
|
|
not an omitted field or success.
|
|
|
|
The mapper derives freshness and successful time only from the returned
|
|
confirmed resource result. A later running, partial, or failed walk neither
|
|
changes that time nor makes data fresh. `data: null` pairs with `unknown`/
|
|
incomplete and either `initial-loading`, `temporarily-unavailable`, or
|
|
`connection-unavailable`; a confirmed empty collection pairs with available
|
|
and complete. The ledger's permitted first-load rows retain incomplete
|
|
coverage and are never represented as a confirmed complete result.
|
|
|
|
Accounts additionally projects the connection-wide walk as `sync: null` when
|
|
no attempt exists, otherwise the same complete safe attempt shape. Other
|
|
resource envelopes do not gain top-level `sync`. This preserves the single
|
|
shared metadata owner while avoiding the current ambiguity between per-resource
|
|
`meta.sync` and Accounts-level walk status.
|
|
|
|
The existing `resourceKey`, `dataRevision`, `coverage`, and dependency details
|
|
remain internal projection inputs or endpoint-specific metadata only where
|
|
their approved behavior still requires publication; they must not substitute
|
|
for any of the five required target meanings. No client must infer target state
|
|
from their presence.
|
|
|
|
### Backend: retain storage owners and publication fences
|
|
|
|
The durable hourly attempt remains the sole normal publisher. Resource readers
|
|
read the saved current connection revision and project its persisted result;
|
|
they do not reserve work or contact upstream. Each successful section can
|
|
persist independently, preserving the previous confirmed resource result and
|
|
its success time when a later section fails. Forecast publication remains
|
|
complete-source-only, and exact per-currency value-history writes remain in the
|
|
same successful Portfolio publication transaction.
|
|
|
|
Connection PUT establishes a new revision before its first walk; DELETE and
|
|
replacement prevent old personal snapshots and old attempt status from being
|
|
read or published. Invalid-token knowledge blocks disclosure rather than being
|
|
masked by saved data. Data and storage changes must be additive when a new
|
|
persisted distinction is necessary; existing snapshots, coverage, forecasts,
|
|
attempts, revisions, and retained value history are not reset or reclassified
|
|
by a destructive migration.
|
|
|
|
### Contract: Swagger is produced at the DTO boundary
|
|
|
|
Swagger DTO annotations and endpoint response classes are the only place the
|
|
wire schema is authored. The backend regenerates `openapi.json` from the
|
|
implemented DTOs, then the frontend receives that published snapshot and runs
|
|
its existing `openapi-typescript` generation. Hand-editing frontend
|
|
`schema.d.ts`, adding handwritten parallel interfaces, or retaining a
|
|
compatibility mapper would hide drift and is excluded.
|
|
|
|
The resulting generated contract must express nullable `data`, all required
|
|
`meta` fields, nullable attempt timestamps, nullable problem fields, Accounts
|
|
top-level nullable `sync`, and absence of top-level `sync` on the other
|
|
resource responses. It must retain `sync=observe` only as a read-observation
|
|
compatibility parameter while it exists in the generated route contract; it
|
|
cannot cause source work and is not used to represent the new hourly state.
|
|
The generated contract reconciles field spelling with the target schema before
|
|
release, not through a second contract artifact.
|
|
|
|
### Frontend: one query cache and metadata-aware screen state
|
|
|
|
`useResourceQuery` remains the sole adapter that projects resource envelopes
|
|
into `data`, `meta`, and Accounts `sync`, preserving its existing query key,
|
|
authenticated transport, access-group invalidation, and Client Cached Snapshot
|
|
behavior. It must project the generated target types directly. Resource query
|
|
owners remain at their existing entity slices; page VMs, not presentation
|
|
views, choose initial loading, unavailable/error, empty, and loaded states.
|
|
|
|
`useScreenSync` remains the single 30-minute timer for open investment
|
|
screens. It calls ordinary local `refetch()` and its retry path continues to
|
|
use the existing observation read without changing cache identity. It derives
|
|
server attempt warning state from `meta.lastAttempt`, keeps transport failures
|
|
separate, and never starts personal synchronization. Existing Accounts
|
|
30-minute `staleTime` remains aligned with this cadence; Portfolio and other
|
|
screen resources keep their current short cache freshness where already
|
|
established while the open-screen timer supplies the approved reread rhythm.
|
|
|
|
All affected account screens render the five metadata meanings in text or an
|
|
element with an accessible name, not color alone. A complete empty resource is
|
|
normal empty content. `data: null` renders initial-loading or unavailable
|
|
states without invented values. A failed/partial hourly attempt is an
|
|
independent nonblocking warning beside confirmed data and its original time.
|
|
`Portfolio.isStale` remains snapshot-specific and is shown separately from
|
|
hourly-attempt and client HTTP-error warnings. A client snapshot stays visible
|
|
during and after a failed background HTTP reread; connection replacement or
|
|
deletion still invalidates personal queries.
|
|
|
|
## Cross-cutting Implementation
|
|
|
|
The canonical state split is:
|
|
|
|
| Concern | Owner | Observable representation |
|
|
| --- | --- | --- |
|
|
| Returned confirmed resource result | Saved resource owner | `data`, freshness, completeness, `lastSuccessfulAt`, availability |
|
|
| Resource-relevant hourly outcome | Durable walk and resource projection | `meta.lastAttempt` |
|
|
| Connection-wide hourly outcome | Durable walk | Accounts `sync` only |
|
|
| In-flight/failed browser reread | TanStack Query | Existing query lifecycle and local retry warning |
|
|
| Snapshot-specific Portfolio source age | Portfolio owner | `Portfolio.isStale` |
|
|
|
|
This split prevents a source failure from becoming a synchronous GET failure,
|
|
prevents the UI from treating its cache as server fallback, and preserves safe
|
|
access invalidation. Safe attempt problems contain only the approved section,
|
|
optional account ID, and safe error code: no token, upstream payload, cursor,
|
|
connection revision, or raw exception text is serialized or logged as status
|
|
metadata.
|
|
|
|
Events retains its existing account and `all` scopes and its saved forecast
|
|
window/filter semantics. Its metadata independently reports the latest events
|
|
attempt, so a saved forecast can remain available when a later refresh fails.
|
|
Operations retains its coverage semantics; only its explicitly approved missing
|
|
historical first-load acquisition may work outside the hourly walk.
|
|
|
|
## Data / Storage Changes
|
|
|
|
No new storage architecture is needed. Existing durable tables/records for the
|
|
current connection revision, hourly attempts, snapshots, operation coverage,
|
|
forecast source windows, and retained portfolio-value points are authoritative.
|
|
If implementation needs persisted terminal-state or problem-detail values not
|
|
already durable, it extends the existing attempt record/schema additively and
|
|
backfills conservatively: absent historical attempt information is projected as
|
|
`none`, never as `succeeded`.
|
|
|
|
Migrations must preserve all current user, connection, personal-resource,
|
|
coverage, forecast, and history rows. They must not reset SQLite, rewrite an
|
|
old partial result as complete, expose retained history across a new connection
|
|
boundary, or delete old data merely to fit the new vocabulary. Compatibility is
|
|
achieved by deploying the backend DTO/runtime and its generated OpenAPI as one
|
|
server contract, followed by the frontend snapshot/types and rendering change;
|
|
there is no claim that an old frontend understands the target envelope.
|
|
|
|
## Contract Implementation
|
|
|
|
The implementation contract is the generated result of the following mapping:
|
|
|
|
| Target concept | Backend source | HTTP location |
|
|
| --- | --- | --- |
|
|
| Last confirmed result or no complete result | Existing local resource reader | `data` |
|
|
| Result freshness and completeness | Resource mapper from confirmed result/coverage | `meta.freshness`, `meta.completeness` |
|
|
| Original confirmed-result time | Saved source timestamp | `meta.lastSuccessfulAt` |
|
|
| Relevant hourly result | Durable attempt plus resource section projection | `meta.lastAttempt` |
|
|
| Whether complete data can be used | Connection/access/read result | `meta.availability` |
|
|
| Complete current-connection walk | Durable connection-wide attempt | Accounts `sync` |
|
|
|
|
The target schema's `problemSection` shape is used for both scopes. For a
|
|
connection-wide Accounts walk, its list may include all affected sections; for
|
|
a resource-level attempt, it is restricted to failures relevant to that
|
|
resource. A running attempt has `finishedAt: null`; terminal attempts have a
|
|
completion time; `none` has null timestamps and no problems. These invariants
|
|
are validated at runtime tests and in the generated OpenAPI, rather than by
|
|
shipping the JSON Schema to application consumers.
|
|
|
|
## Developer Verification Strategy
|
|
|
|
Backend verification proves the contract at DTO, mapper, controller, and
|
|
SQLite boundaries: successful empty data; first load with `data: null`; stale
|
|
fallback after a failed walk; partial walk with preserved data; resource versus
|
|
Accounts attempt scoping; events forecast-window filtering; invalid-token and
|
|
connection-revision disclosure fences; no upstream personal call on ordinary
|
|
GET; and both first-load exceptions staying narrow. It regenerates OpenAPI and
|
|
runs the existing OpenAPI contract, type, lint, and relevant E2E suites.
|
|
|
|
Frontend verification updates the supplied snapshot, runs `openapi:check`, and
|
|
uses MSW/page-VM tests to prove generated metadata projection, no automatic
|
|
retry, HTTP-only retry, 30-minute open-screen reread, cache retention after a
|
|
background error, connection invalidation, and distinct server stale/hourly
|
|
attempt/client transport states. UI checks cover keyboard access, accessible
|
|
status text, loading/error/empty/loaded states, and narrow viewport wrapping.
|
|
|
|
Before release, verify the backend-generated OpenAPI and frontend snapshot are
|
|
the same published artifact, then run each repository's configured verification
|
|
commands. In this specs store, run both v2 schema validation and strict change
|
|
validation; structural success is supplemented by a semantic comparison against
|
|
the approved deltas and System Analysis.
|
|
|
|
## UI/UX Package
|
|
|
|
**Applicability and scope:** `new/changed`. This is an inline package for the
|
|
existing Accounts, account Portfolio, operations, and events screens. It adds
|
|
the approved status vocabulary and preserves 30-minute local rereads; it does
|
|
not introduce new composition, graphics, navigation, or controls.
|
|
|
|
**Package revision:** `1` (this Design). No visual files, external references,
|
|
|
|
**Screens, states and interactions:**
|
|
|
|
| Screen/state | Presentation | Interaction |
|
|
| --- | --- | --- |
|
|
| No complete result, initial attempt running | Existing structural loading state, labelled as loading | No manual sync action |
|
|
| No complete result, attempt failed/unavailable | Safe unavailable/error state with target status text | Retry rereads HTTP only when recovery allows |
|
|
| Complete empty result | Normal domain empty state and target status text | Existing navigation/actions only |
|
|
| Complete data with running/succeeded/partial/failed attempt | Content and original success time remain visible; attempt appears as separate accessible status | No source-work action |
|
|
| Client Cached Snapshot during/after HTTP failure | Existing content remains visible; transport warning remains distinct from server state | Existing retry rereads HTTP only |
|
|
|
|
**Responsive behavior:** reuse existing Mantine/theme components and page
|
|
layouts. Status text wraps without hiding account content or money values on
|
|
narrow screens; color supplements an accessible label. Existing skeletons
|
|
remain noninteractive and unfocusable. Existing cards, buttons, and icons are
|
|
used through their public package exports; no wrapper, SVG asset, token, or
|
|
component package revision is introduced.
|
|
|
|
**Reused source and differences:** `N/A` for approved visual reuse. This is a
|
|
new inline status package revision `1`, not an immutable reuse of a prior
|
|
approved visual package. Existing cards, layouts, hooks, and UI packages are
|
|
implementation inputs only; they are not the approved visual source. The
|
|
package adds semantic presentation of the five target metadata fields and
|
|
separate hourly-attempt/client-transport warnings. No local visual export is
|
|
required.
|
|
|
|
## Risks / Trade-offs
|
|
|
|
- Envelope evolution is a coordinated backend/frontend release risk. Generated
|
|
artifacts prevent hidden type drift but cannot make an older client interpret
|
|
new enum meanings.
|
|
- Mapping resource relevance from connection-wide attempts can over-report or
|
|
under-report a failure if it is not constrained by resource/section scope;
|
|
mapper tests must cover each affected resource.
|
|
- A durable `none` projection for old rows is intentionally less informative
|
|
than inventing a historic success; it preserves truthfulness.
|
|
- The 30-minute client timer can issue local reads while an hourly walk runs;
|
|
this is acceptable because reads do not contend for upstream work and retain
|
|
the last confirmed result.
|
|
- Single-process scheduling remains unsuitable for multi-process deployment;
|
|
adding leases, a worker, or distributed coordination requires a separate
|
|
decision.
|
|
- Documentation currently contains legacy request-driven/manual-refresh
|
|
descriptions. Until the identified durable references are updated, they must
|
|
not be used as implementation authority over the approved behavior.
|
|
|
|
## Migration / Rollout / Rollback
|
|
|
|
Rollout is backend-first at the contract source: apply only additive persistence
|
|
changes if required, implement and test runtime projection/fences, regenerate
|
|
and publish backend OpenAPI, then update the frontend snapshot/generated types
|
|
and renderers. Do not enable the new projection alongside a second personal
|
|
write owner. A frontend built from the old snapshot is not the compatibility
|
|
target; rollout uses the matched generated artifact pair.
|
|
|
|
| Field | Decision |
|
|
| --- | --- |
|
|
| Status | `required` |
|
|
| Trigger | Personal-data disclosure across connection revisions, false complete/fresh state, malformed generated contract, failure to preserve confirmed fallback, or a frontend unable to read the published contract. |
|
|
| Actions | Stop the affected rollout, preserve a SQLite copy, return to the last compatible backend/frontend pair, and fix forward. Do not restore screen-triggered/manual synchronization while the hourly owner remains enabled. |
|
|
| Data safety | Never reset production SQLite, delete additive attempt metadata, delete snapshots/coverage/forecasts/value history, or use a destructive downgrade. Old rows without target details safely project `none`. |
|
|
| Verification | On a representative copy, verify migration integrity, revision/access fences, ordinary local GET behavior, preserved fallback timestamps, generated OpenAPI/snapshot identity, and frontend rendering before traffic resumes. |
|
|
|
|
## Solution Assessment
|
|
|
|
| Option | Assessment |
|
|
| --- | --- |
|
|
| Evolve the existing shared DTO/mapper and generated OpenAPI | Chosen. It is the narrowest path: one resource-meta projection already owns all affected envelope metadata, Swagger already produces the contract, and frontend types already derive from the supplied snapshot. |
|
|
| Add a target adapter/parallel endpoint or versioned envelope | Rejected. It duplicates mapping and cache paths, delays removal of ambiguous states, and creates an unnecessary second compatibility surface. |
|
|
| Handwrite frontend target types or runtime validation | Rejected. Generated `paths`/schemas already own transport typing; manual copies would drift and a validator adds no trust boundary. |
|
|
| Add a scheduler/queue/cache service to express status | Rejected. Durable single-process attempts and SQLite records already express the required state. A new service cannot improve this contract without changing the explicitly retained deployment model. |
|
|
| Add a new design-system package/component family | Rejected. Existing Mantine and `@wealthledger/ui` states cover the textual, accessible status presentation; a new package would be abstraction without a new visual primitive. |
|
|
|
|
The chosen solution is intentionally a contract-and-projection reconciliation,
|
|
not a new synchronization system. It minimizes code and migration surface while
|
|
making the approved observable model expressible through the existing owners.
|