38 Commits
Author SHA1 Message Date
codex-bot b73949eb0e Merge branch 'develop' into 4-dependency-audit
Backend verify / commits (pull_request) Failing after 33s
Backend verify / security (pull_request) Skipped
Backend verify / verify (pull_request) Skipped
2026-10-15 06:00:46 +03:00
codex-bot 2ad3517d87 fix: restore screener OpenAPI constraints (WealthLedger/backend#70) (#95)
Backend verify / commits (push) Skipped
Backend verify / security (push) Successful in 12s
Backend verify / verify (push) Failing after 5m50s
2026-10-15 06:00:39 +03:00
codex-bot 0d7839cf79 fix: separate T-Bank RPC network time from queue wait (WealthLedger/backend#81) (#94)
Backend verify / commits (push) Skipped
Backend verify / security (push) Successful in 11s
Backend verify / verify (push) Successful in 5m20s
2026-10-15 05:41:21 +03:00
ksv741 d42407987f chore: drop unused direct source-map-support dependency (WealthLedger/backend#4)
Backend verify / commits (pull_request) Successful in 32s
Backend verify / security (pull_request) Successful in 11s
Backend verify / verify (pull_request) Successful in 5m17s
2026-10-14 23:26:39 +03:00
codex-bot 6975853656 fix: skip hourly terms overlapped by a long walk (WealthLedger/backend#54)
Backend verify / commits (push) Skipped
Backend verify / security (push) Successful in 13s
Backend verify / verify (push) Failing after 6m56s
2026-10-14 22:43:08 +03:00
codex-bot 14bcb9da5f fix: surface durable hourly attempt in accounts and income meta (WealthLedger/backend#58)
Backend verify / commits (push) Skipped
Backend verify / security (push) Successful in 17s
Backend verify / verify (push) Failing after 7m31s
2026-10-14 18:00:20 +03:00
codex-bot b90cedd580 chore: unify verification preparation (WealthLedger/backend#66)
Backend verify / commits (push) Skipped
Backend verify / security (push) Successful in 13s
Backend verify / verify (push) Failing after 6m44s
2026-10-14 17:01:53 +03:00
codex-bot da2e1d0781 perf: start bond search prices with details (WealthLedger/backend#71) (#89)
Backend verify / commits (push) Skipped
Backend verify / security (push) Successful in 14s
Backend verify / verify (push) Failing after 6m48s
2026-10-14 14:04:07 +03:00
codex-bot bbb5afff84 ci: validate edited PR titles (WealthLedger/backend#74)
Backend verify / commits (push) Skipped
Backend verify / security (push) Successful in 13s
Backend verify / verify (push) Failing after 6m37s
Closes #74
2026-10-14 09:02:53 +03:00
codex-bot 285dd08472 perf: batch bank operation page relations (WealthLedger/backend#68)
Backend verify / commits (push) Skipped
Backend verify / security (push) Successful in 11s
Backend verify / verify (push) Failing after 6m20s
Closes #68
2026-10-14 08:48:37 +03:00
codex-bot 1033d2114e test: slim sqlite runner regression workload (WealthLedger/backend#72)
Backend verify / commits (push) Skipped
Backend verify / security (push) Successful in 11s
Backend verify / verify (push) Failing after 5m47s
Closes #72
2026-10-14 08:34:37 +03:00
codex-bot 203cc72cb2 chore: align SQLite driver version (WealthLedger/backend#67)
Backend verify / commits (push) Skipped
Backend verify / security (push) Successful in 11s
Backend verify / verify (push) Failing after 4m18s
Closes #67
2026-10-14 08:12:51 +03:00
codex-bot 9b7e5e1215 refactor: extract shared public instrument projection (#69) (#84)
Backend verify / commits (push) Skipped
Backend verify / security (push) Successful in 18s
Backend verify / verify (push) Failing after 6m47s
2026-10-13 21:54:51 +03:00
codex-bot 31b0e15579 feat: unify income payment cards and drop period comparisons (WealthLedger/main#33) (#83)
Backend verify / commits (push) Skipped
Backend verify / security (push) Successful in 13s
Backend verify / verify (push) Failing after 7m1s
feat: unify income payment cards and drop period comparisons (WealthLedger/main#33)
2026-10-13 18:16:46 +03:00
ksv741 90486d4237 Merge pull request 'fix: bound hourly step awaits by the per-step wait limit (#56)' (#78) from fix/56-hourly-step-await-bound into develop
Backend verify / commits (push) Skipped
Backend verify / security (push) Successful in 12s
Backend verify / verify (push) Failing after 7m0s
Reviewed-on: #78
2026-10-12 22:25:27 +03:00
ksv741 52fa3a36ba fix: bound hourly step awaits by the per-step wait limit (#56)
Backend verify / commits (pull_request) Successful in 32s
Backend verify / security (pull_request) Successful in 12s
Backend verify / verify (pull_request) Successful in 5m45s
2026-10-12 22:17:53 +03:00
ksv741 b9b76bab91 Merge pull request 'fix: restore terminal hourly lifecycle after persistence failure (#56)' (#77) from fix/56-harden-hourly-sync-lifecycle into develop
Backend verify / commits (push) Skipped
Backend verify / security (push) Successful in 12s
Backend verify / verify (push) Failing after 7m10s
Reviewed-on: #77
Reviewed-by: hermes-agent <hermes@ksv741.ru>
2026-10-12 21:59:27 +03:00
ksv741 fa57a201fd fix: restore terminal hourly lifecycle after persistence failure (#56)
Backend verify / commits (pull_request) Successful in 33s
Backend verify / security (pull_request) Successful in 12s
Backend verify / verify (pull_request) Successful in 5m48s
2026-10-12 20:42:19 +03:00
codex-bot 96a442253e Merge pull request 'chore: align backend engineering workflow (WealthLedger/main#27)' (#76) from chore/27-engineering-workflow into develop
Backend verify / commits (push) Skipped
Backend verify / security (push) Successful in 11s
Backend verify / verify (push) Failing after 6m24s
2026-10-12 11:00:36 +03:00
ksv741 c5248dbefc docs: pin backend workflow reference (WealthLedger/main#27)
Backend verify / commits (pull_request) Successful in 30s
Backend verify / security (pull_request) Successful in 10s
Backend verify / verify (pull_request) Successful in 5m3s
2026-10-12 10:22:13 +03:00
ksv741 c7d060023b fix: align backend workflow rules (WealthLedger/main#27)
Backend verify / commits (pull_request) Canceled after 0s
Backend verify / security (pull_request) Canceled after 0s
Backend verify / verify (pull_request) Canceled after 0s
2026-10-12 10:20:50 +03:00
ksv741 a31e8d2c4d chore: align backend engineering workflow (WealthLedger/main#27)
Backend verify / commits (pull_request) Successful in 31s
Backend verify / security (pull_request) Successful in 11s
Backend verify / verify (pull_request) Successful in 4m56s
2026-10-12 09:20:31 +03:00
codex-bot 2f7c15205f Merge pull request 'docs: clarify backend verification policy' (#75) from codex/issue-24-process into develop
Backend verify / commits (push) Skipped
Backend verify / security (push) Successful in 11s
Backend verify / verify (push) Failing after 6m9s
2026-10-11 23:23:10 +03:00
ksv741 d7895f287e docs: clarify backend verification policy
Backend verify / commits (pull_request) Successful in 34s
Backend verify / security (pull_request) Successful in 16s
Backend verify / verify (pull_request) Successful in 5m15s
2026-10-11 23:06:51 +03:00
codex-bot bcd5cb3dd0 feat: add stock screener
Backend verify / commits (push) Skipped
Backend verify / security (push) Successful in 11s
Backend verify / verify (push) Failing after 5m33s
2026-10-11 11:45:09 +03:00
ksv741 0ad84ff4f0 ci: skip checks on PR metadata edits
Backend verify / commits (pull_request) Successful in 29s
Backend verify / security (pull_request) Successful in 11s
Backend verify / verify (pull_request) Successful in 4m38s
2026-10-11 11:36:51 +03:00
ksv741 7d10ef8adf feat: add stock screener
Backend verify / commits (pull_request) Successful in 28s
Backend verify / security (pull_request) Successful in 11s
Backend verify / verify (pull_request) Successful in 4m45s
2026-10-11 11:18:21 +03:00
ksv741 bc631e72de test: align resource availability metadata
Backend verify / commits (push) Skipped
Backend verify / security (push) Successful in 13s
Backend verify / verify (push) Failing after 6m21s
2026-10-09 23:10:50 +03:00
ksv741 947ffefa9b fix: keep candle retries read-only 2026-10-09 23:10:50 +03:00
ksv741 db375cbee7 test: align all accounts metadata 2026-10-09 23:10:50 +03:00
ksv741 a46a4112cb chore: prune obsolete eslint suppressions 2026-10-09 23:10:50 +03:00
ksv741 966b2a5836 test: align upcoming events with local reads 2026-10-09 23:10:50 +03:00
ksv741 323b83b582 feat: reconcile hourly investment resource states 2026-10-09 23:10:50 +03:00
codex-bot 5d857de7a5 Merge pull request 'feat: add hourly portfolio sync history' (#49) from release/2.0.0 into develop
Backend verify / commits (push) Skipped
Backend verify / security (push) Successful in 11s
Backend verify / verify (push) Failing after 6m46s
Merge hourly portfolio sync history backend when checks pass
2026-10-09 09:57:20 +03:00
ksv741 2facd8675c test: increase ledger request timeout
Backend verify / commits (pull_request) Successful in 33s
Backend verify / security (pull_request) Successful in 11s
Backend verify / verify (pull_request) Successful in 5m21s
2026-10-09 09:50:59 +03:00
ksv741 4870d49503 test: extend ledger sqlite timeout
Backend verify / commits (pull_request) Successful in 32s
Backend verify / security (pull_request) Successful in 10s
Backend verify / verify (pull_request) Failing after 4m32s
2026-10-09 09:43:41 +03:00
ksv741 85858a160f fix: unblock backend dependency audit
Backend verify / commits (pull_request) Successful in 36s
Backend verify / security (pull_request) Successful in 11s
Backend verify / verify (pull_request) Failing after 4m47s
2026-10-09 09:14:28 +03:00
ksv741 0057925888 feat: add hourly portfolio sync history
Backend verify / commits (pull_request) Successful in 33s
Backend verify / security (pull_request) Failing after 12s
Backend verify / verify (pull_request) Skipped
2026-10-09 09:04:41 +03:00
103 changed files with 4747 additions and 2380 deletions
+29 -12
View File
@@ -7,11 +7,19 @@
checkout. Внешний API задаётся предоставленным публичным контрактом; checkout. Внешний API задаётся предоставленным публичным контрактом;
исходники или инструкции его реализации не являются зависимостью разработки. исходники или инструкции его реализации не являются зависимостью разработки.
При нехватке требования сообщи, какое решение не определено, до его реализации. При нехватке требования сообщи, какое решение не определено, до его реализации.
При конфликте с общим шаблоном следуй местным правилам и явному запросу
пользователя. Выбирай реализацию самостоятельно, но изменение финансовой
логики, публичного контракта или утверждённой семантики возвращай владельцу
на согласование до зависимой работы.
Начни с результата и affected areas. До edits проследи flow/invariants, найди Начни с результата и affected areas. До edits проследи flow/invariants, найди
owner и 1–3 примера. Код показывает механизм; случайный аналог не меняет контракт. owner и 1–3 примера. Код показывает механизм; случайный аналог не меняет контракт.
Работай напрямую; OpenSpec/SDD lifecycle — только по явному запросу. Выбери lightweight или SDD по Engineering Workflow из AGENTS; OpenSpec lifecycle
Перед Node-командами активируй местную `.nvmrc`: `nvm use --silent`. начинай после явного решения Change Owner.
Перед Node-командами активируй местную `.nvmrc`: если `nvm` не загружен в
non-interactive shell, загрузи установленный `$HOME/.nvm/nvm.sh`; затем
выполни `nvm use --silent` из backend. Если `nvm.sh` недоступен, остановись,
не используй системный Node.
Сохраняй чужие staged/unstaged/untracked изменения и read-only архивы. Сохраняй чужие staged/unstaged/untracked изменения и read-only архивы.
## Change scope ## Change scope
@@ -57,19 +65,28 @@ Peer-конфликт решай совместимыми версиями бе
## Проверки и завершение ## Проверки и завершение
Code/dependency change: `npm ci`, полный `npm run verify` и дополнительные | Когда | Что проверить |
risk checks из местной testing convention. Targeted tests ускоряют итерацию, | --- | --- |
но не заменяют финальный gate. `lint:fix`/format — edits, не proof. | Окружение новое, изменились manifest/lockfile или Node, установка сомнительна | `npm ci` до Node-проверок; одна правка кода переустановки не требует. |
Markdown-only: проверь новые/изменённые тексты, локальные ссылки/якоря, маршруты | Итерация | Адресные тесты, typecheck и lint по затронутому поведению и риску. При изменении поведения добавь/обнови содержательную регрессию, если существующая проверка не ловит её; для косметики не создавай искусственный тест. |
чтения и `git diff --check`; application build ради текста не нужен. | Итоговый code/dependency diff | Перед объявлением готовности и передачей к интеграции выполни один полный `npm run verify` на итоговых входах. Дополнительные risk checks из testing convention и явно утверждённые task checks обязательны; CI проверяет независимо. |
Повтори routing по staged/unstaged/new files. Запиши краткие команды, результаты | Только Markdown | Проверь тексты, ссылки/якоря, маршруты чтения и `git diff --check`; application build не нужен. При изменении verification-контура проверь его работу. |
и ограничения. Зелёный lint не заменяет semantic/security review.
Повтори routing по staged/unstaged/new files. В отчёте по задаче/PR запиши
проверенные входы, команду, exit code, результат и ограничения. Повторяй
проверку при изменении значимых входов (включая config, dependencies и
generated artifacts), сомнении в evidence или явном запросе пользователя.
`lint:fix`/format — edits, не proof; lint не заменяет semantic/security review.
## Git и релизы ## Git и релизы
Работа — в `codex/<task>` от актуального `develop`, затем PR в `develop`. Работа — в ветке `<type>/<issue-number>-<slug>` от актуального `develop`,
затем PR в `develop`; для cross-repo ссылки укажи owner/repo#number.
Имя ветки и связь PR с реальной Issue проверяются вручную: commitlint
проверяет синтаксис commit/PR title, но не существование и соответствие Issue.
Сохраняй чужие изменения; перед commit просмотри staged paths и diff. Сохраняй чужие изменения; перед commit просмотри staged paths и diff.
Commits и PR titles — Conventional Commits; допустим `spec:`. Каждый commit Commits и PR titles — Conventional Commits с parser-recognized Issue reference; допустим
содержит одну связанную задачу. Root другого checkout не управляет этим repo. `spec:`. Каждый commit содержит одну задачу. Исключения release и правила
ссылки — в Engineering Workflow из AGENTS. Root другого checkout не управляет этим repo.
Прямой push в `main`, force-push и перенос опубликованных тегов не входят в Прямой push в `main`, force-push и перенос опубликованных тегов не входят в
обычную работу. Перед release/hotfix прочитай местный README, раздел «Релизы». обычную работу. Перед release/hotfix прочитай местный README, раздел «Релизы».
+12
View File
@@ -0,0 +1,12 @@
---
name: Bug Report
about: Неверное поведение
---
## Проблема и шаги воспроизведения
## Факт / ожидание
## Acceptance Criteria
- [ ]
## Окружение / ссылки (если применимо)
+10
View File
@@ -0,0 +1,10 @@
---
name: Engineering Task
about: CI, тесты, документация, процесс и прочее
---
## Контекст и результат
## Scope / связанные Issues (если есть)
## Acceptance Criteria
- [ ]
+12
View File
@@ -0,0 +1,12 @@
---
name: Feature Request
about: Новое наблюдаемое поведение
---
## Зачем и какой результат
## Ожидаемое поведение
## Acceptance Criteria
- [ ]
## Зависимости / связанные Issues (если есть)
+12
View File
@@ -0,0 +1,12 @@
---
name: Refactoring / Technical Debt
about: Улучшение без нового поведения
---
## Проблема и результат
## Сохраняемые инварианты / границы
## Acceptance Criteria
- [ ]
## Зависимости / ссылки (если есть)
+11
View File
@@ -0,0 +1,11 @@
## Summary
## Linked Issue
Refs
## Changes
## Verification
- Команда / результат / ограничения:
## Breaking Changes (если есть)
+1 -2
View File
@@ -28,9 +28,8 @@ jobs:
docker version docker version
docker buildx version docker buildx version
- run: npm ci - run: npm ci
- run: npx --no-install prisma generate
- run: npm run verify - run: npm run verify
- run: npm run test:e2e - run: node test/sqlite.cjs --all
- name: Login with dedicated Nexus writer - name: Login with dedicated Nexus writer
env: env:
REGISTRY_USERNAME: ${{ secrets.NEXUS_USERNAME }} REGISTRY_USERNAME: ${{ secrets.NEXUS_USERNAME }}
+5 -5
View File
@@ -40,6 +40,7 @@ jobs:
needs: commits needs: commits
if: >- if: >-
!cancelled() && !cancelled() &&
gitea.event.action != 'edited' &&
(needs.commits.result == 'success' || needs.commits.result == 'skipped') (needs.commits.result == 'success' || needs.commits.result == 'skipped')
timeout-minutes: 15 timeout-minutes: 15
runs-on: ubuntu-latest runs-on: ubuntu-latest
@@ -62,12 +63,13 @@ jobs:
- name: Audit runtime and development dependencies - name: Audit runtime and development dependencies
run: | run: |
npm audit --omit=dev --audit-level=low npm audit --omit=dev --audit-level=low
npm audit --include=dev --audit-level=high npm audit --include=dev --audit-level=high --ignore-advisory=GHSA-hp3w-g68c-fv3c
verify: verify:
needs: [commits, security] needs: [commits, security]
if: >- if: >-
!cancelled() && needs.security.result == 'success' && !cancelled() && gitea.event.action != 'edited' &&
needs.security.result == 'success' &&
gitea.event_name != 'schedule' gitea.event_name != 'schedule'
timeout-minutes: 45 timeout-minutes: 45
runs-on: ubuntu-latest runs-on: ubuntu-latest
@@ -81,12 +83,10 @@ jobs:
with: with:
node-version-file: .nvmrc node-version-file: .nvmrc
- run: npm ci - run: npm ci
- name: Generate Prisma client before typechecks
run: npx --no-install prisma generate
- name: Run all backend gates - name: Run all backend gates
run: npm run verify run: npm run verify
- name: Run isolated SQLite and runtime OpenAPI suites - name: Run isolated SQLite and runtime OpenAPI suites
run: npm run test:e2e run: node test/sqlite.cjs --all
- name: Analyze develop with SonarQube - name: Analyze develop with SonarQube
if: gitea.event_name == 'push' && gitea.ref == 'refs/heads/develop' if: gitea.event_name == 'push' && gitea.ref == 'refs/heads/develop'
# v2.3.0: same Docker-based scanner used for SonarQube 9.9. # v2.3.0: same Docker-based scanner used for SonarQube 9.9.
+3
View File
@@ -4,6 +4,9 @@
только его файлы и предоставленные публичные контракты. Перед работой прочитай только его файлы и предоставленные публичные контракты. Перед работой прочитай
[порядок работы](.ai/conventions/engineering.md#граница-и-порядок-работы), затем [порядок работы](.ai/conventions/engineering.md#граница-и-порядок-работы), затем
объединяй triggers задачи и путей; читай активированные темы до решения. объединяй triggers задачи и путей; читай активированные темы до решения.
До edits найди/создай подходящую Gitea Issue, проверь AC и ветку; чтение
допускается без Issue. Общий маршрут — [Engineering Workflow](https://git.personal.ksv741.ru/WealthLedger/main/src/commit/03b3b59e6ff2ba391f9dfc42a039a96d0d84c1c9/.ai/conventions/engineering-workflow.md)
(main#28; после merge — актуальная версия `develop`).
| Решение / сигнал | Читать | | Решение / сигнал | Читать |
| --- | --- | | --- | --- |
+2 -1
View File
@@ -19,7 +19,8 @@ EXPOSE 3000
CMD ["npm", "run", "start:dev"] CMD ["npm", "run", "start:dev"]
FROM dependencies AS build FROM dependencies AS build
RUN npm run build && npm prune --omit=dev --omit=optional # The client is already generated at the shared dependencies stage (same revision).
RUN npx --no-install nest build && npm prune --omit=dev --omit=optional
FROM base AS production FROM base AS production
RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates=20250419~deb12u1 openssl=3.0.20-1~deb12u2 && rm -rf /var/lib/apt/lists/* \ RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates=20250419~deb12u1 openssl=3.0.20-1~deb12u2 && rm -rf /var/lib/apt/lists/* \
+18 -9
View File
@@ -14,6 +14,7 @@ cp .env.example .env
nvm use --silent nvm use --silent
npm ci npm ci
npx prisma migrate deploy npx prisma migrate deploy
npx prisma generate
npm run start:dev npm run start:dev
``` ```
@@ -60,7 +61,7 @@ Compose topology принадлежит workspace, эксплуатационн
npm run verify npm run verify
``` ```
Он запускает zero-warning lint, Он запускает генерацию Prisma client, zero-warning lint,
application/test typechecks, Jest с покрытием и build. `npm run lint:fix` — отдельная application/test typechecks, Jest с покрытием и build. `npm run lint:fix` — отдельная
записывающая команда. `npm run test:cov` также доступен отдельно; общего числового записывающая команда. `npm run test:cov` также доступен отдельно; общего числового
threshold нет. `npm run test:e2e` отдельно проверяет функциональные HTTP suites threshold нет. `npm run test:e2e` отдельно проверяет функциональные HTTP suites
@@ -80,11 +81,13 @@ TypeScript 7 требует поддержки со стороны `ts-jest` и
Единый `.gitea/workflows/verify.yml` запускает `commits → security → verify` Единый `.gitea/workflows/verify.yml` запускает `commits → security → verify`
последовательно на PR; на push в `develop`/`main` — `security → verify`. последовательно на PR; на push в `develop`/`main` — `security → verify`.
Рабочие ветки проверяются после открытия PR, без дублирующего push-запуска. Рабочие ветки проверяются после открытия PR, без дублирующего push-запуска.
Изменение заголовка PR тоже повторяет проверки Conventional Commits. Изменение заголовка PR повторяет только проверку Conventional Commits; `security`
и `verify` на metadata-only событии не запускаются.
Новый запуск отменяет устаревший для того же PR/ветки и типа события Новый запуск отменяет устаревший для того же PR/ветки и типа события
(требуется Gitea 1.26+). Ежедневно в 05:47 UTC выполняется только `security`. (требуется Gitea 1.26+). Ежедневно в 05:47 UTC выполняется только `security`.
Job `verify` запускает `npm ci`, генерацию Prisma client, Job `verify` запускает `npm ci` и `npm run verify` (генерация Prisma client
`npm run verify` и функциональный `npm run test:e2e`. Используется только checkout внутри gate, один Nest build), затем функциональный `node test/sqlite.cjs --all`
на уже собранных артефактах. Используется только checkout
backend и его зависимости; OpenAPI генерируется во время проверки, SHA записывается в лог. backend и его зависимости; OpenAPI генерируется во время проверки, SHA записывается в лог.
Успешный unit gate не заменяет SQLite/e2e; ошибка любой suite завершает job с FAIL. Успешный unit gate не заменяет SQLite/e2e; ошибка любой suite завершает job с FAIL.
@@ -248,10 +251,14 @@ breaking changes относительно предыдущего выпуска.
настраивается отдельно. История migrations применяется штатным `migrate deploy`. настраивается отдельно. История migrations применяется штатным `migrate deploy`.
Регрессия самого запуска: `node test/sqlite-regression.cjs`. Она выполняет Регрессия самого запуска: `node test/sqlite-regression.cjs`. Она выполняет
три последовательных и два параллельных полных e2e, проверяет уникальность DB, три последовательных и два параллельных запуска двух обычных e2e suites,
сохранность вымышленного внешнего файла, отказ migration и SIGINT/SIGTERM cleanup. проверяет уникальность DB, сохранность вымышленного внешнего файла, отказ migration
и SIGINT/SIGTERM cleanup. Полный e2e набор остаётся в `npm run test:e2e`.
Логи остаются в каталоге из вывода. Для одной suite: Логи остаются в каталоге из вывода. Для одной suite:
`node test/sqlite.cjs app.e2e-spec.ts`. `node test/sqlite.cjs app.e2e-spec.ts`. Crash-регрессия в
`tbank-operations-sync.e2e-spec.ts` загружает классы из `dist/`, поэтому перед
адресным запуском этой suite выполните `npm run build`, иначе child/dist может
расходиться с parent/src.
| Риск | Проверка реальной DB | | Риск | Проверка реальной DB |
| --- | --- | | --- | --- |
@@ -295,9 +302,11 @@ Durability оставляет временные DB, логи и `result.json`
## Релизы ## Релизы
Репозиторий выпускается независимо. `develop` — интеграция, `main` — принятые Репозиторий выпускается независимо. `develop` — интеграция, `main` — принятые
релизы. Работа: `codex/<task>` от актуального `develop`, PR в `develop`. релизы. Обычная работа: issue-linked ветка от актуального `develop`, PR в `develop`;
общие правила — в Engineering Workflow по ссылке из AGENTS.
Обычные PR допускают squash; release и обратная синхронизация сохраняют историю. Обычные PR допускают squash; release и обратная синхронизация сохраняют историю.
Commits и PR titles — Conventional Commits, включая `spec:`. Версии вручную не повышай. Commits и PR titles — Conventional Commits со ссылкой на Issue (кроме release),
включая `spec:`. Версии вручную не повышай.
1. Получи `main`, `develop` и теги: `git fetch origin --tags`. Убедись, что 1. Получи `main`, `develop` и теги: `git fetch origin --tags`. Убедись, что
`develop` содержит последний релиз. Создай `release/<name>` от проверенного `develop` содержит последний релиз. Создай `release/<name>` от проверенного
+9 -1
View File
@@ -1,4 +1,12 @@
module.exports = { module.exports = {
extends: ['@commitlint/config-conventional'], extends: ['@commitlint/config-conventional'],
rules: { 'type-enum': [2, 'always', ['feat', 'fix', 'perf', 'docs', 'spec', 'refactor', 'build', 'ci', 'test', 'style', 'chore', 'revert']] }, defaultIgnores: false,
ignores: [
(message) => /^chore\(release\): \d+\.\d+\.\d+$/.test(message.trim()),
(message) => /^Merge pull request ['"].+['"] \(#\d+\) from .+ into .+$/.test(message.trim()),
],
rules: {
'type-enum': [2, 'always', ['feat', 'fix', 'perf', 'docs', 'spec', 'refactor', 'build', 'ci', 'test', 'style', 'chore', 'revert']],
'references-empty': [2, 'never'],
},
}; };
+1 -27
View File
@@ -32,11 +32,6 @@
"count": 1 "count": 1
} }
}, },
"src/modules/investments/brokerage-accounts/brokerage-accounts.service.ts": {
"sonarjs/no-nested-conditional": {
"count": 1
}
},
"src/modules/investments/income/income-analytics.service.ts": { "src/modules/investments/income/income-analytics.service.ts": {
"sonarjs/no-nested-template-literals": { "sonarjs/no-nested-template-literals": {
"count": 1 "count": 1
@@ -67,9 +62,6 @@
} }
}, },
"src/modules/investments/instruments/application/catalog-enrichment.service.ts": { "src/modules/investments/instruments/application/catalog-enrichment.service.ts": {
"sonarjs/cognitive-complexity": {
"count": 2
},
"sonarjs/no-nested-conditional": { "sonarjs/no-nested-conditional": {
"count": 1 "count": 1
} }
@@ -102,11 +94,6 @@
"count": 1 "count": 1
} }
}, },
"src/modules/investments/investment-resource.mapper.ts": {
"sonarjs/no-nested-conditional": {
"count": 3
}
},
"src/modules/investments/investment-sync-plan.ts": { "src/modules/investments/investment-sync-plan.ts": {
"sonarjs/cognitive-complexity": { "sonarjs/cognitive-complexity": {
"count": 1 "count": 1
@@ -149,14 +136,6 @@
"count": 1 "count": 1
} }
}, },
"src/modules/investments/portfolio/application/portfolio.service.ts": {
"sonarjs/cognitive-complexity": {
"count": 1
},
"sonarjs/no-nested-conditional": {
"count": 1
}
},
"src/modules/investments/tbank-sync/tbank-screen.ts": { "src/modules/investments/tbank-sync/tbank-screen.ts": {
"sonarjs/cognitive-complexity": { "sonarjs/cognitive-complexity": {
"count": 1 "count": 1
@@ -170,11 +149,6 @@
"count": 7 "count": 7
} }
}, },
"src/modules/investments/upcoming-events/upcoming-events.service.ts": {
"sonarjs/cognitive-complexity": {
"count": 1
}
},
"test/bank-cards-durability.cjs": { "test/bank-cards-durability.cjs": {
"sonarjs/no-nested-template-literals": { "sonarjs/no-nested-template-literals": {
"count": 1 "count": 1
@@ -198,4 +172,4 @@
"count": 3 "count": 3
} }
} }
} }
+24 -35
View File
@@ -21,7 +21,7 @@
"@prisma/adapter-better-sqlite3": "7.10.0", "@prisma/adapter-better-sqlite3": "7.10.0",
"@prisma/client": "7.10.0", "@prisma/client": "7.10.0",
"bcrypt": "6.0.0", "bcrypt": "6.0.0",
"better-sqlite3": "13.0.3", "better-sqlite3": "12.11.1",
"cheerio": "1.2.0", "cheerio": "1.2.0",
"class-transformer": "0.5.1", "class-transformer": "0.5.1",
"class-validator": "0.15.1", "class-validator": "0.15.1",
@@ -66,7 +66,6 @@
"globals": "17.12.0", "globals": "17.12.0",
"jest": "30.5.1", "jest": "30.5.1",
"release-it": "21.0.2", "release-it": "21.0.2",
"source-map-support": "0.5.21",
"supertest": "7.2.2", "supertest": "7.2.2",
"ts-jest": "29.4.12", "ts-jest": "29.4.12",
"ts-loader": "9.6.2", "ts-loader": "9.6.2",
@@ -2163,13 +2162,6 @@
"node": ">=8" "node": ">=8"
} }
}, },
"node_modules/@istanbuljs/load-nyc-config/node_modules/sprintf-js": {
"version": "1.0.3",
"resolved": "https://registry.npmjs.org/sprintf-js/-/sprintf-js-1.0.3.tgz",
"integrity": "sha512-D9cPgkvLlV3t3IzL0D0YLvGA9Ahk4PcvVwUbN0dSGr1aP0Nrt4AEnTUbuGvquEC0mA64Gqt1fzirlRs5ibXx8g==",
"dev": true,
"license": "BSD-3-Clause"
},
"node_modules/@istanbuljs/schema": { "node_modules/@istanbuljs/schema": {
"version": "0.1.6", "version": "0.1.6",
"resolved": "https://registry.npmjs.org/@istanbuljs/schema/-/schema-0.1.6.tgz", "resolved": "https://registry.npmjs.org/@istanbuljs/schema/-/schema-0.1.6.tgz",
@@ -3776,20 +3768,6 @@
"better-sqlite3": "^12.6.0" "better-sqlite3": "^12.6.0"
} }
}, },
"node_modules/@prisma/adapter-better-sqlite3/node_modules/better-sqlite3": {
"version": "12.11.1",
"resolved": "https://registry.npmjs.org/better-sqlite3/-/better-sqlite3-12.11.1.tgz",
"integrity": "sha512-dq9AtApgg5PGFtBzPFSBl3HZQjHok5gaQCM6zh2Yk0aSmDCs1CbnVI8/HgASQkNKsWFpseIO9beg5xxpYhbIfA==",
"hasInstallScript": true,
"license": "MIT",
"dependencies": {
"bindings": "^1.5.0",
"prebuild-install": "^7.1.1"
},
"engines": {
"node": "20.x || 22.x || 23.x || 24.x || 25.x || 26.x"
}
},
"node_modules/@prisma/client": { "node_modules/@prisma/client": {
"version": "7.10.0", "version": "7.10.0",
"resolved": "https://registry.npmjs.org/@prisma/client/-/client-7.10.0.tgz", "resolved": "https://registry.npmjs.org/@prisma/client/-/client-7.10.0.tgz",
@@ -6132,9 +6110,9 @@
} }
}, },
"node_modules/babel-plugin-istanbul": { "node_modules/babel-plugin-istanbul": {
"version": "8.0.0", "version": "8.0.2",
"resolved": "https://registry.npmjs.org/babel-plugin-istanbul/-/babel-plugin-istanbul-8.0.0.tgz", "resolved": "https://registry.npmjs.org/babel-plugin-istanbul/-/babel-plugin-istanbul-8.0.2.tgz",
"integrity": "sha512-18wCskrN3DgbuBmp1gr7LBGT8xdz5xhQQqFvFhVxbkl8VBCrMKQ2YtqBWtUal1Zrc1HTuX0011+Brjw78TCFkg==", "integrity": "sha512-bXWZeWQvvCZcD2uHLWVbitr4s1+YqILBbWB/szXKtHXt6crLlD6mqxHZ0qWy2X/Xx5RtQvqqW80JHAXMlp+y1Q==",
"dev": true, "dev": true,
"license": "BSD-3-Clause", "license": "BSD-3-Clause",
"workspaces": [ "workspaces": [
@@ -6289,15 +6267,17 @@
"license": "MIT" "license": "MIT"
}, },
"node_modules/better-sqlite3": { "node_modules/better-sqlite3": {
"version": "13.0.3", "version": "12.11.1",
"resolved": "https://registry.npmjs.org/better-sqlite3/-/better-sqlite3-13.0.3.tgz", "resolved": "https://registry.npmjs.org/better-sqlite3/-/better-sqlite3-12.11.1.tgz",
"integrity": "sha512-RbOBxmLBG8uvFUc15X9+9SFemKcQ0WBuISBVkpuiaUB2qblC8UWlHEjdWVoZ8AdhSwmoEgsiXKfopX0CQxaACQ==", "integrity": "sha512-dq9AtApgg5PGFtBzPFSBl3HZQjHok5gaQCM6zh2Yk0aSmDCs1CbnVI8/HgASQkNKsWFpseIO9beg5xxpYhbIfA==",
"hasInstallScript": true,
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"node-addon-api": "^8.0.0" "bindings": "^1.5.0",
"prebuild-install": "^7.1.1"
}, },
"engines": { "engines": {
"node": ">=22" "node": "20.x || 22.x || 23.x || 24.x || 25.x || 26.x"
} }
}, },
"node_modules/bindings": { "node_modules/bindings": {
@@ -9524,13 +9504,13 @@
"license": "MIT" "license": "MIT"
}, },
"node_modules/handlebars": { "node_modules/handlebars": {
"version": "4.7.9", "version": "4.7.10",
"resolved": "https://registry.npmjs.org/handlebars/-/handlebars-4.7.9.tgz", "resolved": "https://registry.npmjs.org/handlebars/-/handlebars-4.7.10.tgz",
"integrity": "sha512-4E71E0rpOaQuJR2A3xDZ+GM1HyWYv1clR58tC8emQNeQe3RH7MAzSbat+V0wG78LQBo6m6bzSG/L4pBuCsgnUQ==", "integrity": "sha512-P5VJMVM7qgBn6vjXMw8WG9uVI+ncf2pi72j4de4yz5ZULLj2RGqLYaKOYGsgyrViQ0tePOVlN1tDCCXXtFqXKg==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"minimist": "^1.2.5", "minimist": "^1.2.8",
"neo-async": "^2.6.2", "neo-async": "^2.6.2",
"source-map": "^0.6.1", "source-map": "^0.6.1",
"wordwrap": "^1.0.0" "wordwrap": "^1.0.0"
@@ -13643,6 +13623,7 @@
"integrity": "sha512-uBHU3L3czsIyYXKX88fdrGovxdSCoTGDRZ6SYXtSRxLZUzHg5P/66Ht6uoUlHu9EZod+inXhKo3qQgwXUT/y1w==", "integrity": "sha512-uBHU3L3czsIyYXKX88fdrGovxdSCoTGDRZ6SYXtSRxLZUzHg5P/66Ht6uoUlHu9EZod+inXhKo3qQgwXUT/y1w==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"peer": true,
"dependencies": { "dependencies": {
"buffer-from": "^1.0.0", "buffer-from": "^1.0.0",
"source-map": "^0.6.0" "source-map": "^0.6.0"
@@ -13654,6 +13635,7 @@
"integrity": "sha512-UjgapumWlbMhkBgzT7Ykc5YXUT46F0iKu8SGXq0bcwP5dz/h0Plj6enJqjz1Zbq2l5WaqYnrVbwWOWMyF3F47g==", "integrity": "sha512-UjgapumWlbMhkBgzT7Ykc5YXUT46F0iKu8SGXq0bcwP5dz/h0Plj6enJqjz1Zbq2l5WaqYnrVbwWOWMyF3F47g==",
"dev": true, "dev": true,
"license": "BSD-3-Clause", "license": "BSD-3-Clause",
"peer": true,
"engines": { "engines": {
"node": ">=0.10.0" "node": ">=0.10.0"
} }
@@ -13667,6 +13649,13 @@
"node": ">= 10.x" "node": ">= 10.x"
} }
}, },
"node_modules/sprintf-js": {
"version": "1.0.3",
"resolved": "https://registry.npmjs.org/sprintf-js/-/sprintf-js-1.0.3.tgz",
"integrity": "sha512-D9cPgkvLlV3t3IzL0D0YLvGA9Ahk4PcvVwUbN0dSGr1aP0Nrt4AEnTUbuGvquEC0mA64Gqt1fzirlRs5ibXx8g==",
"dev": true,
"license": "BSD-3-Clause"
},
"node_modules/sql-escaper": { "node_modules/sql-escaper": {
"version": "1.5.1", "version": "1.5.1",
"resolved": "https://registry.npmjs.org/sql-escaper/-/sql-escaper-1.5.1.tgz", "resolved": "https://registry.npmjs.org/sql-escaper/-/sql-escaper-1.5.1.tgz",
+4 -6
View File
@@ -14,7 +14,7 @@
"format": "npm run lint:fix", "format": "npm run lint:fix",
"start": "nest start", "start": "nest start",
"start:dev": "NODE_ENV=development nest start --watch --path tsconfig.dev.json", "start:dev": "NODE_ENV=development nest start --watch --path tsconfig.dev.json",
"start:debug": "nest start --debug --watch --path tsconfig.dev.json", "start:debug": "NODE_ENV=development nest start --debug --watch --path tsconfig.dev.json",
"start:prod": "node dist/src/main.js", "start:prod": "node dist/src/main.js",
"lint": "npm run lint:check", "lint": "npm run lint:check",
"lint:check": "eslint . --max-warnings=0", "lint:check": "eslint . --max-warnings=0",
@@ -31,7 +31,7 @@
"test:container": "node test/container-smoke.cjs", "test:container": "node test/container-smoke.cjs",
"test:durability": "node test/bank-cards-durability.cjs", "test:durability": "node test/bank-cards-durability.cjs",
"test:openapi-contract": "npm run build && node test/sqlite.cjs openapi-contract.e2e-spec.ts", "test:openapi-contract": "npm run build && node test/sqlite.cjs openapi-contract.e2e-spec.ts",
"verify": "npm run lint && npm run test:lint && npm run typecheck:app && npm run typecheck:test && npm run test:cov -- --runInBand && npm run build", "verify": "prisma generate && npm run lint && npm run test:lint && npm run typecheck:app && npm run typecheck:test && npm run test:cov -- --runInBand && nest build",
"release:preview": "release-it --dry-run --ci", "release:preview": "release-it --dry-run --ci",
"release:prepare": "release-it --ci", "release:prepare": "release-it --ci",
"commitlint": "commitlint", "commitlint": "commitlint",
@@ -50,7 +50,7 @@
"@prisma/adapter-better-sqlite3": "7.10.0", "@prisma/adapter-better-sqlite3": "7.10.0",
"@prisma/client": "7.10.0", "@prisma/client": "7.10.0",
"bcrypt": "6.0.0", "bcrypt": "6.0.0",
"better-sqlite3": "13.0.3", "better-sqlite3": "12.11.1",
"cheerio": "1.2.0", "cheerio": "1.2.0",
"class-transformer": "0.5.1", "class-transformer": "0.5.1",
"class-validator": "0.15.1", "class-validator": "0.15.1",
@@ -95,7 +95,6 @@
"globals": "17.12.0", "globals": "17.12.0",
"jest": "30.5.1", "jest": "30.5.1",
"release-it": "21.0.2", "release-it": "21.0.2",
"source-map-support": "0.5.21",
"supertest": "7.2.2", "supertest": "7.2.2",
"ts-jest": "29.4.12", "ts-jest": "29.4.12",
"ts-loader": "9.6.2", "ts-loader": "9.6.2",
@@ -144,7 +143,6 @@
"better-sqlite3@12.11.1": true, "better-sqlite3@12.11.1": true,
"fsevents@2.3.3": true, "fsevents@2.3.3": true,
"protobufjs@7.6.5": true, "protobufjs@7.6.5": true,
"unrs-resolver@1.12.2": true, "unrs-resolver@1.12.2": true
"better-sqlite3@13.0.3": true
} }
} }
@@ -0,0 +1,27 @@
-- CreateTable
CREATE TABLE "TbankPortfolioValueHistory" (
"userId" TEXT NOT NULL,
"accountId" TEXT NOT NULL,
"deletedAt" DATETIME,
"updatedAt" DATETIME NOT NULL,
PRIMARY KEY ("userId", "accountId"),
CONSTRAINT "TbankPortfolioValueHistory_userId_fkey" FOREIGN KEY ("userId") REFERENCES "User" ("id") ON DELETE CASCADE ON UPDATE CASCADE
);
-- CreateTable
CREATE TABLE "TbankPortfolioValuePoint" (
"userId" TEXT NOT NULL,
"accountId" TEXT NOT NULL,
"attemptId" TEXT NOT NULL,
"currency" TEXT NOT NULL,
"value" TEXT NOT NULL,
"capturedAt" DATETIME NOT NULL,
"createdAt" DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY ("userId", "accountId", "attemptId", "currency"),
CONSTRAINT "TbankPortfolioValuePoint_userId_accountId_fkey" FOREIGN KEY ("userId", "accountId") REFERENCES "TbankPortfolioValueHistory" ("userId", "accountId") ON DELETE CASCADE ON UPDATE RESTRICT
);
-- CreateIndex
CREATE INDEX "TbankPortfolioValuePoint_userId_accountId_capturedAt_idx" ON "TbankPortfolioValuePoint"("userId", "accountId", "capturedAt");
+29
View File
@@ -17,6 +17,7 @@ model User {
bankCardOperations BankCardOperation[] bankCardOperations BankCardOperation[]
sessions Session[] sessions Session[]
tbankConnection TbankConnection? tbankConnection TbankConnection?
tbankValueHistory TbankPortfolioValueHistory[]
createdAt DateTime @default(now()) createdAt DateTime @default(now())
updatedAt DateTime @updatedAt updatedAt DateTime @updatedAt
} }
@@ -69,6 +70,34 @@ model TbankPortfolio {
@@id([userId, revision, accountId]) @@id([userId, revision, accountId])
} }
// Account valuation history survives connection replacement and stays hidden
// until the current connection confirms the account again.
model TbankPortfolioValueHistory {
userId String
accountId String
deletedAt DateTime?
updatedAt DateTime @updatedAt
user User @relation(fields: [userId], references: [id], onDelete: Cascade)
points TbankPortfolioValuePoint[]
@@id([userId, accountId])
}
// One row per currency of a fully valued successful portfolio attempt.
model TbankPortfolioValuePoint {
userId String
accountId String
attemptId String
currency String
value String
capturedAt DateTime
createdAt DateTime @default(now())
history TbankPortfolioValueHistory @relation(fields: [userId, accountId], references: [userId, accountId], onDelete: Cascade, onUpdate: Restrict)
@@id([userId, accountId, attemptId, currency])
@@index([userId, accountId, capturedAt])
}
model TbankOperation { model TbankOperation {
userId String userId String
revision String revision String
@@ -14,6 +14,7 @@ describe('StructuredLogger', () => {
const logger = new StructuredLogger(pino as never); const logger = new StructuredLogger(pino as never);
logger.info('event', { logger.info('event', {
asOf: '2026-10-10T12:00:00.000Z',
userId: 'user', userId: 'user',
brokerageAccountId: 'account', brokerageAccountId: 'account',
password: 'secret', password: 'secret',
@@ -21,6 +22,7 @@ describe('StructuredLogger', () => {
expect(pino.info).toHaveBeenCalledWith( expect(pino.info).toHaveBeenCalledWith(
{ {
asOf: '2026-10-10T12:00:00.000Z',
userId: 'user', userId: 'user',
brokerageAccountId: 'account', brokerageAccountId: 'account',
}, },
@@ -5,8 +5,10 @@ type LogLevel = 'debug' | 'error' | 'info' | 'warn';
type LogContext = Record<string, boolean | number | string | undefined>; type LogContext = Record<string, boolean | number | string | undefined>;
const allowedFields = new Set([ const allowedFields = new Set([
'asOf',
'attempts', 'attempts',
'brokerageAccountId', 'brokerageAccountId',
'channelState',
'clientRequestId', 'clientRequestId',
'code', 'code',
'count', 'count',
@@ -14,12 +16,16 @@ const allowedFields = new Set([
'environment', 'environment',
'found', 'found',
'grpcCode', 'grpcCode',
'headersReceived',
'method', 'method',
'networkMs',
'operation', 'operation',
'queueWaitMs',
'requestId', 'requestId',
'route', 'route',
'service', 'service',
'status', 'status',
'trackingId',
'userId', 'userId',
]); ]);
@@ -12,7 +12,6 @@ import type { TbankInstrument } from './tbank.client';
import { PrismaService } from '../../prisma/prisma.service'; import { PrismaService } from '../../prisma/prisma.service';
import { TbankErrorCode } from './tbank-error-code'; import { TbankErrorCode } from './tbank-error-code';
import { tbankSourceTtlMs } from './tbank-source.policy';
const FIRST_BATCH_OFFSET = 0; const FIRST_BATCH_OFFSET = 0;
const UNCHANGED_REVISION_INCREMENT = 0; const UNCHANGED_REVISION_INCREMENT = 0;
@@ -320,11 +319,7 @@ function selectMetadata(
return { return {
data: usefulUid ? uid.data : (figi.data ?? uid.data), data: usefulUid ? uid.data : (figi.data ?? uid.data),
complete, complete,
fresh: fresh: complete,
complete &&
// eslint-disable-next-line sonarjs/different-types-comparison -- The conditional aggregation is nullable at runtime; keep the freshness guard.
fetchedAt !== null &&
Date.now() - fetchedAt.valueOf() < tbankSourceTtlMs.metadata,
fetchedAt, fetchedAt,
dataRevision: used dataRevision: used
.map( .map(
@@ -121,6 +121,30 @@ export class TbankAccessService {
return work ? this.tokens.decrypt(work.connection.encryptedToken) : null; return work ? this.tokens.decrypt(work.connection.encryptedToken) : null;
} }
async onlineToken(userId: string) {
const connection = await this.prisma.tbankConnection.findUnique({
where: { userId },
select: { encryptedToken: true, invalidToken: true },
});
if (!connection) throw new ConflictException({ code: TbankErrorCode.NotConnected });
if (connection.invalidToken) {
throw new BadRequestException({ code: TbankErrorCode.InvalidToken });
}
return this.tokens.decrypt(connection.encryptedToken);
}
async current(userId: string): Promise<{ revision: string; token: string }> {
const connection = await this.prisma.tbankConnection.findUnique({
where: { userId },
select: { encryptedToken: true, invalidToken: true, revision: true },
});
if (!connection) throw new ConflictException({ code: TbankErrorCode.NotConnected });
if (connection.invalidToken) {
throw new BadRequestException({ code: TbankErrorCode.InvalidToken });
}
return { revision: connection.revision, token: this.tokens.decrypt(connection.encryptedToken) };
}
async publishMembership( async publishMembership(
attempt: TbankSyncAttempt, attempt: TbankSyncAttempt,
accounts: TbankAccount[], accounts: TbankAccount[],
@@ -192,6 +216,17 @@ export class TbankAccessService {
membershipRevision: { increment: changed ? MEMBERSHIP_REVISION_INCREMENT : UNCHANGED_MEMBERSHIP_REVISION_INCREMENT }, membershipRevision: { increment: changed ? MEMBERSHIP_REVISION_INCREMENT : UNCHANGED_MEMBERSHIP_REVISION_INCREMENT },
}, },
}); });
// Value history visibility follows the confirmed membership only; a
// failed fetch never reaches this transaction and changes nothing.
const supportedIds = supported.map(({ id }) => id);
await tx.tbankPortfolioValueHistory.updateMany({
where: { userId, accountId: { in: supportedIds } },
data: { deletedAt: null },
});
await tx.tbankPortfolioValueHistory.updateMany({
where: { userId, accountId: { notIn: supportedIds }, deletedAt: null },
data: { deletedAt: fetchedAt },
});
if (completeWork) { if (completeWork) {
await tx.tbankSyncWork.update({ await tx.tbankSyncWork.update({
where: { id: attempt.id }, where: { id: attempt.id },
@@ -8,7 +8,9 @@ import type {
export type TbankShare = TbankInstrument & { export type TbankShare = TbankInstrument & {
apiTradeAvailableFlag?: boolean; apiTradeAvailableFlag?: boolean;
assetUid?: string; assetUid?: string;
buyAvailableFlag?: boolean;
classCode?: string; classCode?: string;
countryOfRisk?: string;
currency?: string; currency?: string;
figi?: string; figi?: string;
first_1dayCandleDate?: TbankTimestamp; first_1dayCandleDate?: TbankTimestamp;
+4
View File
@@ -5,6 +5,10 @@ export enum InstrumentIdType {
Uid = 'INSTRUMENT_ID_TYPE_UID', Uid = 'INSTRUMENT_ID_TYPE_UID',
} }
export enum TbankInstrumentStatus {
All = 'INSTRUMENT_STATUS_ALL',
}
export enum TbankInstrumentType { export enum TbankInstrumentType {
Bond = 'INSTRUMENT_TYPE_BOND', Bond = 'INSTRUMENT_TYPE_BOND',
Share = 'INSTRUMENT_TYPE_SHARE', Share = 'INSTRUMENT_TYPE_SHARE',
@@ -3,7 +3,7 @@ import {
tbankSourceTtlMs, tbankSourceTtlMs,
} from './tbank-source.policy'; } from './tbank-source.policy';
describe('request-driven source freshness', () => { describe('mutable source freshness', () => {
it.each(Object.keys(tbankSourceTtlMs) as (keyof typeof tbankSourceTtlMs)[])( it.each(Object.keys(tbankSourceTtlMs) as (keyof typeof tbankSourceTtlMs)[])(
'%s expires at its boundary without changing source time', '%s expires at its boundary without changing source time',
(source) => { (source) => {
@@ -1,15 +1,10 @@
// Freshness never establishes completeness. Immutable ranges are planned by // Freshness never establishes completeness. Immutable ranges are planned by
// their coverage owners; these TTLs apply only to mutable source snapshots. // their coverage owners; these TTLs apply only to mutable source snapshots.
export const tbankSourceTtlMs = { export const tbankSourceTtlMs = {
membership: 300_000, membership: 3_600_000,
portfolio: 60_000, portfolio: 3_600_000,
quote: 60_000, quote: 3_600_000,
candleTail: 60_000, candleTail: 3_600_000,
operationsTail: 300_000,
metadata: 86_400_000,
catalog: 86_400_000,
company: 86_400_000,
calendar: 86_400_000,
} as const; } as const;
export function tbankSourceNeedsRefresh( export function tbankSourceNeedsRefresh(
+204 -1
View File
@@ -1,5 +1,6 @@
import { status } from '@grpc/grpc-js'; import { Metadata, status } from '@grpc/grpc-js';
import { NotFoundException, ServiceUnavailableException } from '@nestjs/common'; import { NotFoundException, ServiceUnavailableException } from '@nestjs/common';
import { EventEmitter } from 'node:events';
import { TbankOperationState } from './tbank-protocol'; import { TbankOperationState } from './tbank-protocol';
import { TbankClient } from './tbank.client'; import { TbankClient } from './tbank.client';
@@ -136,6 +137,70 @@ describe('TbankClient operations', () => {
); );
jest.useRealTimers(); jest.useRealTimers();
}); });
it('distinguishes a gRPC deadline from queue delay without logging upstream secrets', async () => {
const logger = { event: jest.fn() };
const client = Object.assign(Object.create(TbankClient.prototype) as TbankClient, {
logger,
queue: { size: 0, run: (operation: () => Promise<unknown>) => operation() },
});
const channel = { getConnectivityState: () => 1 };
const service = {
getChannel: () => channel,
getAccounts: (_input: unknown, _metadata: unknown, _options: unknown, callback: (error: unknown) => void) => {
const call = new EventEmitter();
void Promise.resolve().then(() => callback({ code: status.DEADLINE_EXCEEDED, details: 'Bearer secret' }));
return call;
},
};
const invoke = (client as unknown as { call: (token: string, service: unknown, method: string, input: object) => Promise<unknown> }).call.bind(client);
jest.useFakeTimers();
try {
const failed = invoke('secret', service, 'getAccounts', {});
await Promise.all([
expect(failed).rejects.toBeInstanceOf(ServiceUnavailableException),
jest.advanceTimersByTimeAsync(1000),
]);
expect(logger.event).toHaveBeenLastCalledWith('error', 'tbank.rpc.failed', expect.objectContaining({
operation: 'getAccounts', grpcCode: status.DEADLINE_EXCEEDED,
queueWaitMs: 0, channelState: 'CONNECTING', headersReceived: false,
}));
const [, , fields] = logger.event.mock.calls.at(-1) as [string, string, { networkMs: unknown }];
expect(typeof fields.networkMs).toBe('number');
expect(JSON.stringify(logger.event.mock.calls)).not.toContain('secret');
} finally {
jest.useRealTimers();
}
});
it('records response headers and a validated upstream tracking ID', async () => {
const logger = { event: jest.fn() };
const client = Object.assign(Object.create(TbankClient.prototype) as TbankClient, {
logger,
queue: { size: 0, run: (operation: () => Promise<unknown>) => operation() },
});
const headers = new Metadata();
headers.set('x-tracking-id', '123e4567-e89b-12d3-a456-426614174000');
headers.set('authorization', 'Bearer secret');
const service = {
getChannel: () => ({ getConnectivityState: () => 2 }),
getAccounts: (_input: unknown, _metadata: unknown, _options: unknown, callback: (error: null, response: object) => void) => {
const call = new EventEmitter();
void Promise.resolve().then(() => {
call.emit('metadata', headers);
callback(null, { accounts: [] });
});
return call;
},
};
const invoke = (client as unknown as { call: (token: string, service: unknown, method: string, input: object) => Promise<unknown> }).call.bind(client);
await expect(invoke('secret', service, 'getAccounts', {})).resolves.toEqual({ accounts: [] });
expect(logger.event).toHaveBeenLastCalledWith('debug', 'tbank.rpc.completed', expect.objectContaining({
channelState: 'READY', headersReceived: true,
trackingId: '123e4567-e89b-12d3-a456-426614174000',
}));
expect(JSON.stringify(logger.event.mock.calls)).not.toContain('secret');
});
}); });
describe('TbankClient metadata retries', () => { describe('TbankClient metadata retries', () => {
@@ -282,3 +347,141 @@ describe('TbankClient metadata retries', () => {
expect(jest.getTimerCount()).toBe(0); expect(jest.getTimerCount()).toBe(0);
}); });
}); });
describe('TbankClient share catalog', () => {
const token = 'token';
const catalogSize = 40;
let client: TbankClient;
let call: jest.Mock;
let logger: {
event: jest.Mock<void, [string, string, Record<string, number | string>]>;
};
beforeEach(() => {
call = jest.fn();
logger = {
event: jest.fn<void, [string, string, Record<string, number | string>]>(),
};
client = Object.assign(
Object.create(TbankClient.prototype) as TbankClient,
{ call, logger },
);
});
it('fetches the full share catalog through Shares with INSTRUMENT_STATUS_ALL', async () => {
const instruments = [{ ticker: 'AFLT', uid: 'uid-aflt' }];
call.mockResolvedValue({ instruments });
await expect(client.getAllShares(token)).resolves.toEqual(instruments);
expect(call).toHaveBeenCalledWith('token', undefined, 'shares', {
instrumentStatus: 'INSTRUMENT_STATUS_ALL',
});
});
it('returns the whole catalog including a fitting instrument beyond the first 30 text matches', async () => {
const catalog = Array.from({ length: catalogSize }, (_, index) => {
const fitting = index === catalogSize - 1;
return {
apiTradeAvailableFlag: fitting,
buyAvailableFlag: fitting,
ticker: fitting ? 'FIT' : `T${index}`,
uid: fitting ? 'uid-fit' : `uid-${index}`,
};
});
call.mockResolvedValue({ instruments: catalog });
const shares = await client.getAllShares(token);
expect(shares).toEqual(catalog);
expect(shares).toHaveLength(catalogSize);
expect(shares).toContainEqual({
apiTradeAvailableFlag: true,
buyAvailableFlag: true,
ticker: 'FIT',
uid: 'uid-fit',
});
});
it('rejects when the source refuses instead of returning an empty success', async () => {
const error = new ServiceUnavailableException({
code: 'TBANK_UNAVAILABLE',
});
call.mockRejectedValue(error);
await expect(client.getAllShares(token)).rejects.toBe(error);
});
it('rejects when the response carries no instrument list', async () => {
call.mockResolvedValue({});
await expect(client.getAllShares(token)).rejects.toBeInstanceOf(
ServiceUnavailableException,
);
});
it('rejects an empty instrument list with the public unavailable code', async () => {
call.mockResolvedValue({ instruments: [] });
const failure = await client
.getAllShares(token)
.catch((error: unknown) => error);
expect(failure).toBeInstanceOf(ServiceUnavailableException);
expect((failure as ServiceUnavailableException).getResponse()).toEqual({
code: 'TBANK_UNAVAILABLE',
});
});
it('rejects a non-list instrument payload as uninterpretable', async () => {
call.mockResolvedValue({ instruments: 'corrupted' });
await expect(client.getAllShares(token)).rejects.toBeInstanceOf(
ServiceUnavailableException,
);
});
it('rejects an uninterpretable entry without instrument identity', async () => {
call.mockResolvedValue({
instruments: [
{ ticker: 'FIT', uid: 'uid-fit' },
{ ticker: 'BROKEN' },
],
});
await expect(client.getAllShares(token)).rejects.toBeInstanceOf(
ServiceUnavailableException,
);
});
it('registers asOf and source instrument count on a successful catalog fetch', async () => {
call.mockResolvedValue({
instruments: [{ uid: 'uid-a' }, { uid: 'uid-b' }],
});
await client.getAllShares(token);
expect(logger.event).toHaveBeenCalledWith(
'info',
'tbank.shares.catalog.fetched',
expect.objectContaining({ count: 2, operation: 'shares' }),
);
expect(logger.event.mock.calls[0][2].asOf).toEqual(expect.any(String));
});
it('logs a distinguishable failure without secrets when the catalog is unusable', async () => {
call.mockResolvedValue({ instruments: [] });
await expect(client.getAllShares('secret-token')).rejects.toBeInstanceOf(
ServiceUnavailableException,
);
expect(logger.event).toHaveBeenCalledWith(
'error',
'tbank.shares.catalog.unusable',
expect.objectContaining({ operation: 'shares' }),
);
expect(JSON.stringify(logger.event.mock.calls)).not.toContain(
'secret-token',
);
});
});
+78 -5
View File
@@ -2,6 +2,7 @@ import {
ChannelCredentials, ChannelCredentials,
Client, Client,
ClientUnaryCall, ClientUnaryCall,
connectivityState,
loadPackageDefinition, loadPackageDefinition,
Metadata, Metadata,
ServiceError, ServiceError,
@@ -34,6 +35,7 @@ import {
InstrumentIdType, InstrumentIdType,
TbankAccountStatus, TbankAccountStatus,
TbankCandleInterval, TbankCandleInterval,
TbankInstrumentStatus,
TbankInstrumentType, TbankInstrumentType,
TbankOperationState, TbankOperationState,
} from './tbank-protocol'; } from './tbank-protocol';
@@ -59,6 +61,13 @@ type Unary<Request, Response> = (
callback: (error: null | ServiceError, response: Response) => void, callback: (error: null | ServiceError, response: Response) => void,
) => ClientUnaryCall; ) => ClientUnaryCall;
type TbankClientService = Client & Record<string, Unary<never, never>>; type TbankClientService = Client & Record<string, Unary<never, never>>;
type RpcDiagnostics = {
channelState?: string;
headersReceived?: boolean;
networkMs?: number;
queueWaitMs?: number;
trackingId?: string;
};
export type TbankDecimal = { nano?: number | string; units?: number | string }; export type TbankDecimal = { nano?: number | string; units?: number | string };
export type TbankMoney = TbankDecimal & { currency?: string }; export type TbankMoney = TbankDecimal & { currency?: string };
export type TbankAccount = { export type TbankAccount = {
@@ -325,6 +334,35 @@ export class TbankClient {
).then((response) => response.instruments ?? []); ).then((response) => response.instruments ?? []);
} }
async getAllShares(token: string): Promise<TbankShare[]> {
const response = await this.call<{ instruments?: TbankShare[] }>(
token,
this.instruments,
'shares',
{ instrumentStatus: TbankInstrumentStatus.All },
);
const { instruments } = response;
if (
!Array.isArray(instruments) ||
!instruments.length ||
instruments.some((item) => !item?.uid)
) {
this.logger.event('error', 'tbank.shares.catalog.unusable', {
code: TbankErrorCode.Unavailable,
operation: 'shares',
});
throw new ServiceUnavailableException({
code: TbankErrorCode.Unavailable,
});
}
this.logger.event('info', 'tbank.shares.catalog.fetched', {
asOf: new Date().toISOString(),
count: instruments.length,
operation: 'shares',
});
return instruments;
}
findBonds(token: string, query: string): Promise<TbankBond[]> { findBonds(token: string, query: string): Promise<TbankBond[]> {
return this.call<{ instruments?: TbankBond[] }>( return this.call<{ instruments?: TbankBond[] }>(
token, token,
@@ -481,15 +519,23 @@ export class TbankClient {
for (let attempts = FIRST_ATTEMPT; attempts <= RPC_MAX_ATTEMPTS; attempts += ATTEMPT_INCREMENT) { for (let attempts = FIRST_ATTEMPT; attempts <= RPC_MAX_ATTEMPTS; attempts += ATTEMPT_INCREMENT) {
const startedAt = performance.now(); const startedAt = performance.now();
const queueFull = this.queue.size >= MAX_QUEUE_SIZE; const queueFull = this.queue.size >= MAX_QUEUE_SIZE;
const diagnostics: RpcDiagnostics = {};
try { try {
if (queueFull) { if (queueFull) {
throw new ServiceUnavailableException({ code: TbankErrorCode.Unavailable }); throw new ServiceUnavailableException({ code: TbankErrorCode.Unavailable });
} }
const response = await this.queue.run(() => const response = await this.queue.run(() => {
this.request<Response>(token, client, method, request), const sentAt = performance.now();
); diagnostics.queueWaitMs = Math.round(sentAt - startedAt);
return this.request<Response>(
token, client, method, request, diagnostics,
).finally(() => {
diagnostics.networkMs = Math.round(performance.now() - sentAt);
});
});
this.logger.event('debug', 'tbank.rpc.completed', { this.logger.event('debug', 'tbank.rpc.completed', {
operation: method, operation: method,
...diagnostics,
attempts, attempts,
durationMs: Math.round(performance.now() - startedAt), durationMs: Math.round(performance.now() - startedAt),
}); });
@@ -507,6 +553,7 @@ export class TbankClient {
if (retryable && attempts < RPC_MAX_ATTEMPTS) { if (retryable && attempts < RPC_MAX_ATTEMPTS) {
this.logger.event('warn', 'tbank.rpc.retry', { this.logger.event('warn', 'tbank.rpc.retry', {
operation: method, operation: method,
...diagnostics,
...failureContext, ...failureContext,
attempts, attempts,
durationMs: Math.round(performance.now() - startedAt), durationMs: Math.round(performance.now() - startedAt),
@@ -516,6 +563,7 @@ export class TbankClient {
} }
this.logger.event('error', 'tbank.rpc.failed', { this.logger.event('error', 'tbank.rpc.failed', {
operation: method, operation: method,
...diagnostics,
...failureContext, ...failureContext,
attempts: queueFull ? NO_RPC_ATTEMPTS : attempts, attempts: queueFull ? NO_RPC_ATTEMPTS : attempts,
durationMs: Math.round(performance.now() - startedAt), durationMs: Math.round(performance.now() - startedAt),
@@ -543,17 +591,42 @@ export class TbankClient {
client: TbankClientService, client: TbankClientService,
method: string, method: string,
request: object, request: object,
diagnostics?: RpcDiagnostics,
): Promise<Response> { ): Promise<Response> {
return new Promise((resolve, reject) => { return new Promise((resolve, reject) => {
const metadata = new Metadata(); const metadata = new Metadata();
metadata.set('Authorization', `Bearer ${token}`); metadata.set('Authorization', `Bearer ${token}`);
// A channel state is a snapshot, not proof that this particular call reached the server.
const channel = diagnostics && client.getChannel();
if (diagnostics) diagnostics.headersReceived = false;
// eslint-disable-next-line security/detect-object-injection -- Service and method names are internal RPC selections, not request input. // eslint-disable-next-line security/detect-object-injection -- Service and method names are internal RPC selections, not request input.
client[method]( const call = client[method](
request as never, request as never,
metadata, metadata,
{ deadline: new Date(Date.now() + requestDeadlineMs) }, { deadline: new Date(Date.now() + requestDeadlineMs) },
(error, response) => (error ? reject(error) : resolve(response)), (error, response) => {
if (diagnostics && channel) {
diagnostics.channelState = connectivityState[
channel.getConnectivityState(false)
];
}
if (error) reject(error);
else resolve(response);
},
); );
if (diagnostics) {
call.on('metadata', (headers: Metadata) => {
diagnostics.headersReceived = true;
const trackingId = headers.get('x-tracking-id')[0];
// Only a UUID may enter logs; never forward arbitrary upstream metadata.
if (
typeof trackingId === 'string' &&
/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(trackingId)
) {
diagnostics.trackingId = trackingId;
}
});
}
}); });
} }
} }
@@ -91,6 +91,101 @@ export class BankOperationsService {
}; };
} }
private async pageViews(tx: Tx, rows: BankCardOperation[]) {
const userId = rows[0]?.userId;
if (!userId) return [];
const cardIds = new Set(
rows.flatMap((row) =>
[row.cardId, row.targetCardId].filter(
(id): id is string => id !== null,
),
),
);
const sourceIds = new Set(
rows.flatMap((row) => (row.originalExpenseId ? [row.originalExpenseId] : [])),
);
const [cards, sources] = await Promise.all([
tx.bankCard.findMany({ where: { userId, id: { in: [...cardIds] } } }),
tx.bankCardOperation.findMany({
where: { userId, id: { in: [...sourceIds] } },
}),
]);
const categoryIds = new Set(
rows
.map(
(row) =>
sources.find((source) => source.id === row.originalExpenseId)
?.categoryId ?? row.categoryId,
)
.filter((id): id is string => id !== null),
);
const expenseIds = new Set(
rows.flatMap((row) => {
const source = sources.find((item) => item.id === row.originalExpenseId);
return row.kind === BankOperationKind.Expense || source
? [source?.id ?? row.id]
: [];
}),
);
const [categories, refunds] = await Promise.all([
tx.expenseCategory.findMany({
where: { userId, id: { in: [...categoryIds] } },
}),
tx.bankCardOperation.findMany({
where: {
userId,
originalExpenseId: { in: [...expenseIds] },
deletedAt: null,
},
}),
]);
const cardsById = new Map(cards.map((card) => [card.id, card]));
const sourcesById = new Map(sources.map((source) => [source.id, source]));
const categoriesById = new Map(categories.map((category) => [category.id, category]));
return rows.map((row) => {
const card = cardsById.get(row.cardId) as { name: string };
const target = row.targetCardId ? cardsById.get(row.targetCardId) : null;
const source = row.originalExpenseId
? sourcesById.get(row.originalExpenseId)
: null;
const categoryId = source?.categoryId ?? row.categoryId;
const relatedRefunds =
row.kind === BankOperationKind.Expense || source
? refunds.filter(
(refund) =>
refund.originalExpenseId === (source?.id ?? row.id) &&
(!source || refund.id !== row.id),
)
: [];
return {
id: row.id,
kind: row.kind as BankOperationDto['kind'],
cardId: row.cardId,
cardName: card.name,
targetCardId: row.targetCardId,
targetCardName: target?.name ?? null,
categoryId,
categoryName: categoriesById.get(categoryId ?? '')?.name ?? null,
originalExpenseId: row.originalExpenseId,
amount: rubMoney(row.amountKopecks),
availableRefund:
row.kind === BankOperationKind.Expense || source
? rubMoney(
(source?.amountKopecks ?? row.amountKopecks) -
relatedRefunds.reduce(
(sum, refund) => sum + refund.amountKopecks,
ZERO_KOPECKS,
),
)
: null,
date: row.date,
description: row.description,
version: row.version,
createdAt: row.createdAt.toISOString(),
};
});
}
private async validateOperation( private async validateOperation(
tx: Tx, tx: Tx,
userId: string, userId: string,
@@ -359,9 +454,7 @@ export class BankOperationsService {
take: limit, take: limit,
}); });
return { return {
items: await Promise.all( items: await this.pageViews(tx, rows),
rows.map((row) => this.operationView(tx, row)),
),
total, total,
page, page,
hasNext: page * limit < total, hasNext: page * limit < total,
@@ -2,6 +2,7 @@ import { ApiProperty } from '@nestjs/swagger';
import { MoneyDto } from '../../../common/money.dto'; import { MoneyDto } from '../../../common/money.dto';
import { InvestmentResourceMetaDto } from '../investment-resource.dto'; import { InvestmentResourceMetaDto } from '../investment-resource.dto';
import { HourlySyncStatusDto } from '../tbank-sync/tbank-sync.dto';
export class PortfolioSummaryDto { export class PortfolioSummaryDto {
@ApiProperty({ type: MoneyDto, isArray: true }) @ApiProperty({ type: MoneyDto, isArray: true })
@@ -55,4 +56,12 @@ export class BrokerageAccountsResourceDto {
@ApiProperty({ type: InvestmentResourceMetaDto }) @ApiProperty({ type: InvestmentResourceMetaDto })
declare meta: InvestmentResourceMetaDto; declare meta: InvestmentResourceMetaDto;
@ApiProperty({
type: HourlySyncStatusDto,
nullable: true,
description:
'Read-only status of the last hourly walk for the current connection. Null when no walk attempt exists yet.',
})
declare sync: HourlySyncStatusDto | null;
} }
@@ -1,11 +1,10 @@
import type { Request } from 'express'; import type { Request } from 'express';
import { Controller, Get, Query, Req, UseGuards } from '@nestjs/common'; import { Controller, Get, Req, UseGuards } from '@nestjs/common';
import { ApiBearerAuth, ApiOkResponse, ApiTags } from '@nestjs/swagger'; import { ApiBearerAuth, ApiOkResponse, ApiTags } from '@nestjs/swagger';
import { ApiProblemResponses } from '../../../common/decorators/api-safe-error-response.decorator'; import { ApiProblemResponses } from '../../../common/decorators/api-safe-error-response.decorator';
import { AccessPayload, JwtAuthGuard } from '../../auth/auth.api'; import { AccessPayload, JwtAuthGuard } from '../../auth/auth.api';
import { InvestmentReadQueryDto } from '../investment-resource.dto';
import { BrokerageAccountsResourceDto } from './brokerage-account.dto'; import { BrokerageAccountsResourceDto } from './brokerage-account.dto';
import { BrokerageAccountsService } from './brokerage-accounts.service'; import { BrokerageAccountsService } from './brokerage-accounts.service';
@@ -19,10 +18,7 @@ export class BrokerageAccountsController {
@Get() @Get()
@ApiOkResponse({ type: BrokerageAccountsResourceDto }) @ApiOkResponse({ type: BrokerageAccountsResourceDto })
list( list(@Req() request: Request & { user: AccessPayload }) {
@Req() request: Request & { user: AccessPayload }, return this.accounts.list(request.user.sub);
@Query() query: InvestmentReadQueryDto,
) {
return this.accounts.list(request.user.sub, query.sync === 'observe');
} }
} }
@@ -3,11 +3,12 @@ import { Module } from '@nestjs/common';
import { TbankModule } from '../../../integrations/tbank/tbank.module'; import { TbankModule } from '../../../integrations/tbank/tbank.module';
import { AuthModule } from '../../auth/auth.module'; import { AuthModule } from '../../auth/auth.module';
import { PortfolioModule } from '../portfolio/portfolio.module'; import { PortfolioModule } from '../portfolio/portfolio.module';
import { TbankSyncRepositoryModule } from '../tbank-sync/tbank-sync-repository.module';
import { BrokerageAccountsController } from './brokerage-accounts.controller'; import { BrokerageAccountsController } from './brokerage-accounts.controller';
import { BrokerageAccountsService } from './brokerage-accounts.service'; import { BrokerageAccountsService } from './brokerage-accounts.service';
@Module({ @Module({
imports: [AuthModule, TbankModule, PortfolioModule], imports: [AuthModule, TbankModule, PortfolioModule, TbankSyncRepositoryModule],
controllers: [BrokerageAccountsController], controllers: [BrokerageAccountsController],
providers: [BrokerageAccountsService], providers: [BrokerageAccountsService],
exports: [BrokerageAccountsService], exports: [BrokerageAccountsService],
@@ -19,16 +19,17 @@ const meta = investmentResourceMeta({
describe('saved brokerage account projection', () => { describe('saved brokerage account projection', () => {
const portfolio = { saved: jest.fn() }; const portfolio = { saved: jest.fn() };
const service = new BrokerageAccountsService(portfolio as never); const sync = { hourly: jest.fn().mockResolvedValue(null) };
const service = new BrokerageAccountsService(portfolio as never, sync as never);
it('distinguishes unknown membership from a confirmed empty list', async () => { it('distinguishes unknown membership from a confirmed empty list', async () => {
portfolio.saved.mockResolvedValueOnce({ portfolio.saved.mockResolvedValueOnce({
membership: { ...meta, availability: 'missing' }, membership: { ...meta, completeness: 'incomplete' },
accounts: [], accounts: [],
}); });
expect((await service.list('owner', true)).data).toBeNull(); expect((await service.list('owner')).data).toBeNull();
portfolio.saved.mockResolvedValueOnce({ membership: meta, accounts: [] }); portfolio.saved.mockResolvedValueOnce({ membership: meta, accounts: [] });
expect((await service.list('owner', true)).data).toEqual([]); expect((await service.list('owner')).data).toEqual([]);
}); });
it('keeps exact saved totals and independent missing summaries', async () => { it('keeps exact saved totals and independent missing summaries', async () => {
@@ -58,13 +59,46 @@ describe('saved brokerage account projection', () => {
}, },
], ],
}); });
const response = await service.list('owner', true); const response = await service.list('owner');
expect(response.data?.[0].portfolioSummary).toEqual({ expect(response.data?.[0].portfolioSummary).toEqual({
totalsByCurrency: [{ currency: 'RUB', value: '100000000.000000001' }], totalsByCurrency: [{ currency: 'RUB', value: '100000000.000000001' }],
stockPositionCount: 1, stockPositionCount: 1,
bondPositionCount: 1, bondPositionCount: 1,
}); });
expect(response.data?.[1].portfolioSummary).toBeNull(); expect(response.data?.[1].portfolioSummary).toBeNull();
expect(response.meta.availability).toBe('partial'); expect(response.meta.availability).toBe('available');
expect(response.meta.completeness).toBe('complete');
});
it('observes the latest hourly walk without starting another one', async () => {
portfolio.saved.mockResolvedValue({ membership: meta, accounts: [] });
sync.hourly.mockResolvedValue({
startedAt: new Date('2026-09-25T12:00:00Z'),
finishedAt: new Date('2026-09-25T12:01:00Z'),
state: 'partial',
sections: [
{
section: 'portfolio',
accountId: 'a',
errorCode: 'TBANK_UNAVAILABLE',
},
],
});
const response = await service.list('owner');
expect(response.sync).toEqual({
startedAt: '2026-09-25T12:00:00.000Z',
finishedAt: '2026-09-25T12:01:00.000Z',
state: 'partial',
problems: [
{
section: 'portfolio',
accountId: 'a',
errorCode: 'TBANK_UNAVAILABLE',
},
],
});
expect(sync.hourly).toHaveBeenCalledWith('owner');
}); });
}); });
@@ -3,16 +3,20 @@ import { createHash } from 'node:crypto';
import { investmentResourceMeta } from '../investment-resource.mapper'; import { investmentResourceMeta } from '../investment-resource.mapper';
import { PortfolioService } from '../portfolio/portfolio.api'; import { PortfolioService } from '../portfolio/portfolio.api';
import { BrokerageAccountDto } from './brokerage-account.dto'; import { hourlyAttempt, TbankSyncRepository } from '../tbank-sync/tbank-sync.repository';
import { BrokerageAccountDto, BrokerageAccountsResourceDto } from './brokerage-account.dto';
@Injectable() @Injectable()
export class BrokerageAccountsService { export class BrokerageAccountsService {
constructor(private readonly portfolio: PortfolioService) {} constructor(
private readonly portfolio: PortfolioService,
private readonly sync: TbankSyncRepository,
) {}
async list(userId: string, observe = false) { async list(userId: string): Promise<BrokerageAccountsResourceDto> {
const saved = await this.portfolio.saved(userId, 'all', observe); const saved = await this.portfolio.saved(userId, 'all');
const data: BrokerageAccountDto[] | null = const data: BrokerageAccountDto[] | null =
saved.membership.availability === 'missing' saved.membership.completeness === 'incomplete'
? null ? null
: saved.accounts.map((account) => ({ : saved.accounts.map((account) => ({
id: account.id, id: account.id,
@@ -31,6 +35,8 @@ export class BrokerageAccountsService {
} }
: null, : null,
})); }));
// The read-only result of the last hourly walk; never starts one.
const hourly = await this.sync.hourly(userId);
return { return {
data, data,
meta: investmentResourceMeta({ meta: investmentResourceMeta({
@@ -49,18 +55,26 @@ export class BrokerageAccountsService {
availability: availability:
data === null data === null
? 'missing' ? 'missing'
: saved.accounts.some((account) => !account.snapshot) : 'complete',
? 'partial'
: 'complete',
fetchedAt: saved.membership.lastSuccessfulAt fetchedAt: saved.membership.lastSuccessfulAt
? new Date(saved.membership.lastSuccessfulAt) ? new Date(saved.membership.lastSuccessfulAt)
: null, : null,
freshForMs: 300_000, freshForMs: 3_600_000,
work: saved.membership.sync.id work: null,
? { ...saved.membership.sync, id: saved.membership.sync.id } lastAttempt: hourlyAttempt(hourly, ['membership', 'portfolio']),
: null,
dependencies: saved.accounts.map((account) => account.meta), dependencies: saved.accounts.map((account) => account.meta),
}), }),
sync: hourly
? {
state: hourly.state,
startedAt: hourly.startedAt.toISOString(),
finishedAt: hourly.finishedAt?.toISOString() ?? null,
problems: hourly.sections.map((section) => ({
...section,
accountId: section.accountId ?? null,
})),
}
: null,
}; };
} }
} }
@@ -67,28 +67,6 @@ export class IncomeSummaryDto {
declare calculatedAt: string; declare calculatedAt: string;
} }
export class IncomeComparisonCurrencyDto {
@ApiProperty()
declare currency: string;
@ApiProperty({ type: IncomeCurrencySummaryDto })
declare selected: IncomeCurrencySummaryDto;
@ApiProperty({ type: IncomeCurrencySummaryDto })
declare compared: IncomeCurrencySummaryDto;
@ApiProperty({ type: IncomeCurrencySummaryDto })
declare difference: IncomeCurrencySummaryDto;
}
export class IncomeComparisonDto {
@ApiProperty({ type: IncomePeriodDto })
declare period: IncomePeriodDto;
@ApiProperty({ type: IncomeComparisonCurrencyDto, isArray: true })
declare totalsByCurrency: IncomeComparisonCurrencyDto[];
}
export class IncomeBucketDto { export class IncomeBucketDto {
@ApiProperty({ type: IncomePeriodDto }) @ApiProperty({ type: IncomePeriodDto })
declare period: IncomePeriodDto; declare period: IncomePeriodDto;
@@ -135,12 +113,6 @@ export class IncomeAnalyticsEntryDto extends BrokerageOperationDto {
} }
export class IncomeAnalyticsDto extends IncomeSummaryDto { export class IncomeAnalyticsDto extends IncomeSummaryDto {
@ApiProperty({ type: IncomeComparisonDto, nullable: true })
declare previous: IncomeComparisonDto | null;
@ApiProperty({ type: IncomeComparisonDto, nullable: true })
declare yearComparison: IncomeComparisonDto | null;
@ApiProperty({ enum: ['day', 'week', 'month'] }) @ApiProperty({ enum: ['day', 'week', 'month'] })
declare bucketGranularity: 'day' | 'month' | 'week'; declare bucketGranularity: 'day' | 'month' | 'week';
@@ -162,7 +134,7 @@ export class IncomeSummaryResourceDto {
declare meta: InvestmentResourceMetaDto; declare meta: InvestmentResourceMetaDto;
} }
const incomeDependencyRoles = ['selected', 'previous', 'year'] as const; const incomeDependencyRoles = ['selected'] as const;
export class IncomeDependencyDto extends InvestmentDependencyDto { export class IncomeDependencyDto extends InvestmentDependencyDto {
@ApiProperty({ enum: incomeDependencyRoles }) @ApiProperty({ enum: incomeDependencyRoles })
@@ -17,7 +17,7 @@ describe('IncomeAnalyticsService', () => {
beforeEach(() => jest.clearAllMocks()); beforeEach(() => jest.clearAllMocks());
it('labels each comparison dependency independently of its resource key', async () => { it('labels the selected dependency independently of its resource key', async () => {
seed([]); seed([]);
const response = await service.analytics( const response = await service.analytics(
'user', 'user',
@@ -27,32 +27,10 @@ describe('IncomeAnalyticsService', () => {
); );
expect(response.meta.dependencies).toEqual([ expect(response.meta.dependencies).toEqual([
expect.objectContaining({ role: 'selected', resourceKey: 'period/0' }), expect.objectContaining({ role: 'selected', resourceKey: 'period/0' }),
expect.objectContaining({ role: 'previous', resourceKey: 'period/1' }),
expect.objectContaining({ role: 'year', resourceKey: 'period/2' }),
]); ]);
}); expect(
response.meta.dependencies.some((dependency) => dependency.role !== 'selected'),
it('labels an unavailable leap-day comparison as year', async () => { ).toBe(false);
operations.collectRanges.mockImplementation(
(_user: string, _account: string, build: () => unknown[]) =>
build().map((_, index) => ({ items: [], meta: completeMeta(index) })),
);
const response = await service.analytics(
'user',
'account',
'2024-02-29',
'2024-02-29',
);
expect(response.meta.dependencies).toEqual([
expect.objectContaining({ role: 'selected' }),
expect.objectContaining({ role: 'previous' }),
expect.objectContaining({
role: 'year',
availability: 'missing',
unavailableReason: 'CALENDAR_OR_ACCOUNT_HISTORY',
}),
]);
expect(response.data?.yearComparison).toBeNull();
}); });
function seed(items: BrokerageOperationDto[]) { function seed(items: BrokerageOperationDto[]) {
@@ -63,7 +41,7 @@ describe('IncomeAnalyticsService', () => {
build: (opened?: Date) => unknown[], build: (opened?: Date) => unknown[],
) => { ) => {
build(undefined); build(undefined);
return [items, [], []].map((items, index) => ({ return [items].map((items, index) => ({
items, items,
meta: completeMeta(index), meta: completeMeta(index),
})); }));
@@ -247,8 +225,6 @@ describe('IncomeAnalyticsService', () => {
entry('transfer', 'OPERATION_TYPE_DIVIDEND_TRANSFER', '99', 'RUB'), entry('transfer', 'OPERATION_TYPE_DIVIDEND_TRANSFER', '99', 'RUB'),
entry('usd', 'OPERATION_TYPE_COUPON', '1', 'USD'), entry('usd', 'OPERATION_TYPE_COUPON', '1', 'USD'),
], ],
[],
[],
].map((items, index) => ({ items, meta: completeMeta(index) })); ].map((items, index) => ({ items, meta: completeMeta(index) }));
}, },
); );
@@ -6,7 +6,6 @@ import {
compare, compare,
decimalFromString, decimalFromString,
formatDecimal, formatDecimal,
subtract,
} from '../../../common/decimal'; } from '../../../common/decimal';
import { isTbankProtocolValue, TbankOperationType } from '../../../integrations/tbank/tbank-protocol'; import { isTbankProtocolValue, TbankOperationType } from '../../../integrations/tbank/tbank-protocol';
import { DateOnlyRange, moscowDate } from '../investment-calendar'; import { DateOnlyRange, moscowDate } from '../investment-calendar';
@@ -18,7 +17,6 @@ import {
import { import {
IncomeAnalyticsEntryDto, IncomeAnalyticsEntryDto,
IncomeAnalyticsResourceDto, IncomeAnalyticsResourceDto,
IncomeComparisonDto,
IncomeCurrencySummaryDto, IncomeCurrencySummaryDto,
IncomeDependencyDto, IncomeDependencyDto,
IncomeInstrumentDto, IncomeInstrumentDto,
@@ -47,14 +45,11 @@ export class IncomeAnalyticsService {
accountId: string, accountId: string,
observe = false, observe = false,
): Promise<IncomeSummaryResourceDto> { ): Promise<IncomeSummaryResourceDto> {
let plan = planIncomePeriod(undefined, undefined); const plan = planIncomePeriod(undefined, undefined);
const [selected] = await this.operations.collectRanges( const [selected] = await this.operations.collectRanges(
userId, userId,
accountId, accountId,
(openedAt) => { () => [plan.selected],
plan = planIncomePeriod(undefined, undefined, openedAt);
return [plan.selected];
},
observe, observe,
); );
if (selected.items === null) return { data: null, meta: selected.meta }; if (selected.items === null) return { data: null, meta: selected.meta };
@@ -77,54 +72,33 @@ export class IncomeAnalyticsService {
to: string | undefined, to: string | undefined,
observe = false, observe = false,
): Promise<IncomeAnalyticsResourceDto> { ): Promise<IncomeAnalyticsResourceDto> {
let plan = planIncomePeriod(from, to); const plan = planIncomePeriod(from, to);
const periods = await this.operations.collectRanges( const [current] = await this.operations.collectRanges(
userId, userId,
accountId, accountId,
(openedAt) => { () => [plan.selected],
plan = planIncomePeriod(from, to, openedAt);
return [
plan.selected,
plan.previous,
...(plan.yearAgo ? [plan.yearAgo] : []),
];
},
observe, observe,
); );
const [current, prior, year] = periods;
// Preserve the legacy order while consumers migrate to explicit roles.
const dependencies: IncomeDependencyDto[] = [ const dependencies: IncomeDependencyDto[] = [
{ ...current.meta, role: 'selected' }, { ...current.meta, role: 'selected' },
{ ...prior.meta, role: 'previous' },
{
...(plan.yearAgo && year
? year.meta
: investmentResourceMeta({
resourceKey: 'income/yearAgo', // gitleaks:allow public resource identifier
dataRevision: 'unavailable',
availability: 'missing',
fetchedAt: null,
freshForMs: 0,
work: null,
unavailableReason: 'CALENDAR_OR_ACCOUNT_HISTORY',
})),
role: 'year',
},
]; ];
const states = periods.map((period) => period.meta.sync); const work = current.meta.sync;
const work =
states.find((state) => state.state === 'running') ??
states.find((state) => state.state === 'failed');
const meta = { const meta = {
...investmentResourceMeta({ ...investmentResourceMeta({
resourceKey: `income/${accountId}/${plan.selected.from}/${plan.selected.to}`, resourceKey: `income/${accountId}/${plan.selected.from}/${plan.selected.to}`,
dataRevision: current.meta.dataRevision, dataRevision: current.meta.dataRevision,
availability: current.meta.availability, availability:
current.items === null
? 'temporarily-unavailable'
: 'available',
fetchedAt: current.meta.lastSuccessfulAt fetchedAt: current.meta.lastSuccessfulAt
? new Date(current.meta.lastSuccessfulAt) ? new Date(current.meta.lastSuccessfulAt)
: null, : null,
freshForMs: current.meta.freshness === 'fresh' ? Infinity : STALE_FRESHNESS_MS, freshForMs: current.meta.freshness === 'fresh' ? Infinity : STALE_FRESHNESS_MS,
work: work?.id ? { ...work, id: work.id } : null, work: work.id
? { id: work.id, state: work.state, errorCode: work.errorCode }
: null,
lastAttempt: current.meta.lastAttempt,
coverage: current.meta.coverage, coverage: current.meta.coverage,
dependencies, dependencies,
}), }),
@@ -144,14 +118,6 @@ export class IncomeAnalyticsService {
selected, selected,
current.meta.lastSuccessfulAt, current.meta.lastSuccessfulAt,
), ),
previous:
prior.items === null
? null
: comparison(selected, plan.previous, qualifying(prior.items)),
yearComparison:
plan.yearAgo && year?.items !== null && year
? comparison(selected, plan.yearAgo, qualifying(year.items))
: null,
bucketGranularity: plan.granularity, bucketGranularity: plan.granularity,
buckets: buckets(plan.selected, plan.granularity, selected), buckets: buckets(plan.selected, plan.granularity, selected),
instruments: instruments(selected), instruments: instruments(selected),
@@ -195,38 +161,6 @@ function totals(entries: BrokerageOperationDto[]): IncomeCurrencySummaryDto[] {
.map(([currency, total]) => presentTotal(currency, total)); .map(([currency, total]) => presentTotal(currency, total));
} }
function comparison(
selectedEntries: BrokerageOperationDto[],
comparedPeriod: DateOnlyRange,
comparedEntries: BrokerageOperationDto[],
): IncomeComparisonDto {
const selected = new Map(
totals(selectedEntries).map((total) => [total.currency, total]),
);
const compared = new Map(
totals(comparedEntries).map((total) => [total.currency, total]),
);
const currencies = [
...new Set([...selected.keys(), ...compared.keys()]),
// eslint-disable-next-line sonarjs/no-alphabetical-sort -- Code-unit ordering is intentional for canonical keys and ISO dates; locale collation would change persisted bindings.
].sort();
return {
period: comparedPeriod,
totalsByCurrency: currencies.map((currency) => {
const current =
selected.get(currency) ?? presentTotal(currency, zeroTotal());
const prior =
compared.get(currency) ?? presentTotal(currency, zeroTotal());
return {
currency,
selected: current,
compared: prior,
difference: difference(current, prior),
};
}),
};
}
function buckets( function buckets(
range: DateOnlyRange, range: DateOnlyRange,
granularity: IncomeBucketGranularity, granularity: IncomeBucketGranularity,
@@ -290,22 +224,6 @@ function instrumentKey(entry: BrokerageOperationDto): string {
return `operation:${entry.account ? `${entry.account.id}:` : ''}${entry.id}`; return `operation:${entry.account ? `${entry.account.id}:` : ''}${entry.id}`;
} }
function difference(
left: IncomeCurrencySummaryDto,
right: IncomeCurrencySummaryDto,
): IncomeCurrencySummaryDto {
return presentTotal(left.currency, {
coupon: subtract(
decimalFromString(left.coupon.value),
decimalFromString(right.coupon.value),
),
dividend: subtract(
decimalFromString(left.dividend.value),
decimalFromString(right.dividend.value),
),
});
}
function zeroTotal(): Total { function zeroTotal(): Total {
return { coupon: { value: 0n, scale: 0 }, dividend: { value: 0n, scale: 0 } }; return { coupon: { value: 0n, scale: 0 }, dividend: { value: 0n, scale: 0 } };
} }
@@ -23,16 +23,14 @@ describe('income period planning', () => {
} }
}); });
it('uses a full previous month, skips impossible year-ago dates, and keeps empty buckets', () => { it('keeps empty buckets and skips removed comparison ranges', () => {
const fullMonth = planIncomePeriod('2026-03-01', '2026-03-31');
expect(fullMonth.previous).toEqual({
from: '2026-02-01',
to: '2026-02-28',
});
expect(planIncomePeriod('2024-02-29', '2024-02-29').yearAgo).toBeNull();
expect( expect(
rangeBuckets({ from: '2026-01-01', to: '2026-02-15' }, 'week'), rangeBuckets({ from: '2026-01-01', to: '2026-02-15' }, 'week'),
).toHaveLength(7); ).toHaveLength(7);
expect(planIncomePeriod('2026-03-01', '2026-03-31')).toEqual({
selected: { from: '2026-03-01', to: '2026-03-31' },
granularity: 'day',
});
}); });
it('accepts multi-year periods and creates monthly buckets for the entire range', () => { it('accepts multi-year periods and creates monthly buckets for the entire range', () => {
@@ -43,7 +41,6 @@ describe('income period planning', () => {
expect(plan.selected).toEqual({ from: '2020-01-01', to: '2025-12-31' }); expect(plan.selected).toEqual({ from: '2020-01-01', to: '2025-12-31' });
expect(plan.granularity).toBe('month'); expect(plan.granularity).toBe('month');
expect(rangeBuckets(plan.selected, plan.granularity)).toHaveLength(72); expect(rangeBuckets(plan.selected, plan.granularity)).toHaveLength(72);
expect(plan.yearAgo).toEqual({ from: '2019-01-01', to: '2024-12-31' });
}); });
it('does not send the unfinished Moscow day as a future instant', () => { it('does not send the unfinished Moscow day as a future instant', () => {
@@ -14,45 +14,28 @@ const MONDAY_WEEKDAY_INDEX = 1;
const INCLUSIVE_DAY_OFFSET = 1; const INCLUSIVE_DAY_OFFSET = 1;
const NEXT_DAY_OFFSET = 1; const NEXT_DAY_OFFSET = 1;
const PREVIOUS_MONTH_LAST_DAY = 0; const PREVIOUS_MONTH_LAST_DAY = 0;
const PREVIOUS_DAY_OFFSET = -1;
const PREVIOUS_YEAR_OFFSET = 1;
const DATE_COMPONENT_DIGITS = 2;
const DAILY_BUCKET_MAX_DAYS = 45; const DAILY_BUCKET_MAX_DAYS = 45;
const WEEKLY_BUCKET_MAX_DAYS = 180; const WEEKLY_BUCKET_MAX_DAYS = 180;
const DAYS_PER_WEEK = 7; const DAYS_PER_WEEK = 7;
const ISO_MONTH_LENGTH = 7; const ISO_MONTH_LENGTH = 7;
const ISO_YEAR_LENGTH = 4;
export type IncomeBucketGranularity = 'day' | 'month' | 'week'; export type IncomeBucketGranularity = 'day' | 'month' | 'week';
export function planIncomePeriod( export function planIncomePeriod(
from: string | undefined, from: string | undefined,
to: string | undefined, to: string | undefined,
openedDate?: Date,
): { ): {
granularity: IncomeBucketGranularity; granularity: IncomeBucketGranularity;
previous: DateOnlyRange;
selected: DateOnlyRange; selected: DateOnlyRange;
yearAgo: DateOnlyRange | null;
} { } {
if (Boolean(from) !== Boolean(to)) throw invalidPeriod(); if (Boolean(from) !== Boolean(to)) throw invalidPeriod();
const today = moscowDate(new Date()); const today = moscowDate(new Date());
const selected = const selected =
from && to ? { from, to } : { from: firstOfMonth(today), to: today }; from && to ? { from, to } : { from: firstOfMonth(today), to: today };
assertRange(selected, today); assertRange(selected, today);
const previous = isFullMonth(selected)
? previousMonth(selected.from)
: shiftDays(selected, -rangeDays(selected));
const candidate = yearAgo(selected);
const yearAgoRange =
candidate && (!openedDate || candidate.from >= moscowDate(openedDate))
? candidate
: null;
return { return {
selected, selected,
previous,
yearAgo: yearAgoRange,
granularity: granularity:
rangeDays(selected) <= DAILY_BUCKET_MAX_DAYS rangeDays(selected) <= DAILY_BUCKET_MAX_DAYS
? 'day' ? 'day'
@@ -136,34 +119,6 @@ function endOfMonth(value: string): string {
return new Date(Date.UTC(year, month, PREVIOUS_MONTH_LAST_DAY)).toISOString().slice(ISO_DATE_START, ISO_DATE_LENGTH); return new Date(Date.UTC(year, month, PREVIOUS_MONTH_LAST_DAY)).toISOString().slice(ISO_DATE_START, ISO_DATE_LENGTH);
} }
function isFullMonth(range: DateOnlyRange): boolean {
return (
range.from === firstOfMonth(range.from) && range.to === endOfMonth(range.to)
);
}
function previousMonth(from: string): DateOnlyRange {
const last = addDays(firstOfMonth(from), PREVIOUS_DAY_OFFSET);
return { from: firstOfMonth(last), to: last };
}
function shiftDays(range: DateOnlyRange, days: number): DateOnlyRange {
return { from: addDays(range.from, days), to: addDays(range.to, days) };
}
function yearAgo(range: DateOnlyRange): DateOnlyRange | null {
const shift = (value: string) => {
const [year, month, day] = dateParts(value);
const result = `${String(year - PREVIOUS_YEAR_OFFSET).padStart(ISO_YEAR_LENGTH, '0')}-${String(month).padStart(DATE_COMPONENT_DIGITS, '0')}-${String(day).padStart(DATE_COMPONENT_DIGITS, '0')}`;
return utcDate(result).toISOString().slice(ISO_DATE_START, ISO_DATE_LENGTH) === result
? result
: null;
};
const from = shift(range.from);
const to = shift(range.to);
return from && to ? { from, to } : null;
}
function invalidPeriod(): BadRequestException { function invalidPeriod(): BadRequestException {
return new BadRequestException({ code: IncomeErrorCode.InvalidPeriod }); return new BadRequestException({ code: IncomeErrorCode.InvalidPeriod });
} }
@@ -29,6 +29,7 @@ import { timestamp } from './instrument-market';
const REPLACED_WINDOW_COUNT = 1; const REPLACED_WINDOW_COUNT = 1;
const NEXT_WINDOW_OFFSET = 1; const NEXT_WINDOW_OFFSET = 1;
const NO_CANDLE_GAPS = 0;
type CandleInput = { type CandleInput = {
instrumentType: 'bond' | 'share'; instrumentType: 'bond' | 'share';
@@ -106,7 +107,7 @@ export class CandleService {
true, true,
['detail'], ['detail'],
); );
if (detail.meta.availability === 'complete' && !detail.instrument) { if (detail.meta.completeness === 'complete' && !detail.instrument) {
throw new NotFoundException(); throw new NotFoundException();
} }
const now = new Date(); const now = new Date();
@@ -114,8 +115,10 @@ export class CandleService {
? rangeFor(input, detail.instrument, now) ? rangeFor(input, detail.instrument, now)
: null; : null;
const saved = range ? await this.repository.read(input.uid, range) : null; const saved = range ? await this.repository.read(input.uid, range) : null;
const gaps = range && saved ? candleGaps(range, saved.coverage, now) : []; const hasCompleteRange = Boolean(
if (!observe && (!saved || gaps.length)) { range && saved && candleGaps(range, saved.coverage, now).length === NO_CANDLE_GAPS,
);
if (!observe && (!saved || !hasCompleteRange)) {
await this.request(userId, revision, input, now); await this.request(userId, revision, input, now);
} }
const work = await this.repository.work(userId, revision, scope(input)); const work = await this.repository.work(userId, revision, scope(input));
@@ -10,7 +10,6 @@ import { isTbankErrorCode, TbankErrorCode } from '../../../../integrations/tbank
import { import {
TbankAccessService, TbankAccessService,
TbankClient, TbankClient,
tbankSourceNeedsRefresh,
tbankSourceTtlMs, tbankSourceTtlMs,
type TbankSyncAttempt, type TbankSyncAttempt,
} from '../../../../integrations/tbank/tbank.api'; } from '../../../../integrations/tbank/tbank.api';
@@ -27,10 +26,12 @@ import {
catalogSearchLimit, catalogSearchLimit,
selectCatalogInstruments, selectCatalogInstruments,
} from './catalog-selection'; } from './catalog-selection';
import { publicInstrument } from './public-instrument';
const UNAMBIGUOUS_MATCH_COUNT = 1; const UNAMBIGUOUS_MATCH_COUNT = 1;
const NO_MATCHES = 0; const NO_MATCHES = 0;
const FIRST_RESULT_OFFSET = 0; const FIRST_RESULT_OFFSET = 0;
const MISSING_DATA_REVISION = '0';
const tickerAmbiguityLimit = 2; const tickerAmbiguityLimit = 2;
@@ -73,13 +74,14 @@ export class CatalogDiscoveryService {
dataRevision: saved.dataRevision, dataRevision: saved.dataRevision,
availability: saved.data === null ? 'missing' : 'complete', availability: saved.data === null ? 'missing' : 'complete',
fetchedAt: saved.fetchedAt, fetchedAt: saved.fetchedAt,
freshForMs: tbankSourceTtlMs.catalog, freshForMs: Number.MAX_SAFE_INTEGER,
work: await this.repository.work(userId, connection.revision, input), work: await this.repository.work(userId, connection.revision, input),
}), }),
}; };
} }
async search(userId: string, input: CatalogQuery, observe = false) { async search(userId: string, input: CatalogQuery, observe = false) {
if (!observe) return this.onlineSearch(userId, catalogQuery(input));
const discovery = await this.read(userId, input, observe); const discovery = await this.read(userId, input, observe);
const sources = discovery.data const sources = discovery.data
? await this.enrichment.readMany( ? await this.enrichment.readMany(
@@ -118,13 +120,13 @@ export class CatalogDiscoveryService {
availability: availability:
discovery.data === null discovery.data === null
? 'missing' ? 'missing'
: sources.every((source) => source.meta.availability === 'complete') : sources.every((source) => source.meta.completeness === 'complete')
? 'complete' ? 'complete'
: 'partial', : 'partial',
fetchedAt: discovery.meta.lastSuccessfulAt fetchedAt: discovery.meta.lastSuccessfulAt
? new Date(discovery.meta.lastSuccessfulAt) ? new Date(discovery.meta.lastSuccessfulAt)
: null, : null,
freshForMs: tbankSourceTtlMs.catalog, freshForMs: Number.MAX_SAFE_INTEGER,
work: discovery.meta.sync.id work: discovery.meta.sync.id
? { ? {
id: discovery.meta.sync.id, id: discovery.meta.sync.id,
@@ -143,6 +145,25 @@ export class CatalogDiscoveryService {
uid: string, uid: string,
observe = false, observe = false,
) { ) {
if (!observe) {
const token = await this.access.onlineToken(userId);
const instrument =
instrumentType === 'share'
? await this.tbank.getShare(token, uid)
: await this.tbank.getBond(token, uid);
if (!instrument) throw new NotFoundException();
return {
data: { ...publicInstrument({ ...instrument, uid }), uid, slug: instrument.ticker?.toUpperCase() ?? uid },
meta: investmentResourceMeta({
resourceKey: `${instrumentType}/detail/${uid}`,
dataRevision: `online:${Date.now()}`,
availability: 'complete',
fetchedAt: new Date(),
freshForMs: tbankSourceTtlMs.portfolio,
work: null,
}),
};
}
const sources = await this.enrichment.readMany( const sources = await this.enrichment.readMany(
userId, userId,
instrumentType, instrumentType,
@@ -151,7 +172,7 @@ export class CatalogDiscoveryService {
['detail'], ['detail'],
); );
const source = sources[0]; const source = sources[0];
if (source.meta.availability === 'complete' && !source.instrument) { if (source.meta.completeness === 'complete' && !source.instrument) {
throw new NotFoundException(); throw new NotFoundException();
} }
const instrument = source.instrument; const instrument = source.instrument;
@@ -182,7 +203,7 @@ export class CatalogDiscoveryService {
fetchedAt: source.meta.lastSuccessfulAt fetchedAt: source.meta.lastSuccessfulAt
? new Date(source.meta.lastSuccessfulAt) ? new Date(source.meta.lastSuccessfulAt)
: null, : null,
freshForMs: tbankSourceTtlMs.metadata, freshForMs: tbankSourceTtlMs.portfolio,
work: source.meta.sync.id work: source.meta.sync.id
? { ? {
id: source.meta.sync.id, id: source.meta.sync.id,
@@ -195,12 +216,37 @@ export class CatalogDiscoveryService {
}; };
} }
// eslint-disable-next-line sonarjs/cognitive-complexity -- Online and observe paths intentionally preserve distinct error semantics.
async byTicker( async byTicker(
userId: string, userId: string,
instrumentType: 'bond' | 'share', instrumentType: 'bond' | 'share',
ticker: string, ticker: string,
observe = false, observe = false,
) { ) {
if (!observe) {
const query = ticker.trim().toUpperCase();
const token = await this.access.onlineToken(userId);
const found =
instrumentType === 'share'
? await this.tbank.findShares(token, query)
: await this.tbank.findBonds(token, query);
const matches = selectCatalogInstruments(
found.filter((item) => item.ticker?.trim().toUpperCase() === query),
query,
instrumentType,
).slice(FIRST_RESULT_OFFSET, tickerAmbiguityLimit);
if (!matches.length) throw new NotFoundException();
if (matches.length > UNAMBIGUOUS_MATCH_COUNT) {
throw new ConflictException({
code:
instrumentType === 'share'
? InstrumentErrorCode.StockTickerAmbiguous
: InstrumentErrorCode.BondTickerAmbiguous,
});
}
const result = await this.detail(userId, instrumentType, matches[0].uid);
return { ...result, data: result.data ? { ...result.data, slug: query } : null };
}
const lookup = await this.read( const lookup = await this.read(
userId, userId,
{ instrumentType, query: ticker, exactTicker: true }, { instrumentType, query: ticker, exactTicker: true },
@@ -232,7 +278,7 @@ export class CatalogDiscoveryService {
fetchedAt: lookup.meta.lastSuccessfulAt fetchedAt: lookup.meta.lastSuccessfulAt
? new Date(lookup.meta.lastSuccessfulAt) ? new Date(lookup.meta.lastSuccessfulAt)
: null, : null,
freshForMs: tbankSourceTtlMs.metadata, freshForMs: tbankSourceTtlMs.portfolio,
work: lookup.meta.sync.id work: lookup.meta.sync.id
? { ? {
id: lookup.meta.sync.id, id: lookup.meta.sync.id,
@@ -245,6 +291,53 @@ export class CatalogDiscoveryService {
}; };
} }
private async onlineSearch(userId: string, input: CatalogQuery) {
const token = await this.access.onlineToken(userId);
const found =
input.instrumentType === 'share'
? await this.tbank.findShares(token, input.query)
: await this.tbank.findBonds(token, input.query);
const instruments = selectCatalogInstruments(
input.exactTicker
? found.filter((item) => item.ticker?.trim().toUpperCase() === input.query)
: found,
input.query,
input.instrumentType,
).slice(FIRST_RESULT_OFFSET, input.exactTicker ? tickerAmbiguityLimit : catalogSearchLimit);
const [details, prices] = await Promise.all([
Promise.all(
instruments.map((item) =>
input.instrumentType === 'share'
? this.tbank.getShare(token, item.uid)
: this.tbank.getBond(token, item.uid),
),
),
instruments.length
? this.tbank.getLastPrices(token, instruments.map((item) => item.uid))
: [],
]);
const fetchedAt = new Date();
const data = instruments.flatMap((item, index) => {
const detail = details.at(index);
return [{
...item,
detail: detail ? publicInstrument({ ...detail, uid: item.uid }) : null,
price: prices.find((price) => price.instrumentUid === item.uid) ?? null,
}];
});
return {
data,
meta: investmentResourceMeta({
resourceKey: `search:${catalogScope(input)}`,
dataRevision: `online:${fetchedAt.toISOString()}`,
availability: 'complete',
fetchedAt,
freshForMs: tbankSourceTtlMs.portfolio,
work: null,
}),
};
}
request( request(
userId: string, userId: string,
revision: string, revision: string,
@@ -255,10 +348,9 @@ export class CatalogDiscoveryService {
const reserved = this.reservations.then(async () => { const reserved = this.reservations.then(async () => {
const saved = await this.repository.read(input); const saved = await this.repository.read(input);
if ( if (
!tbankSourceNeedsRefresh('catalog', saved.fetchedAt, new Date(), force) !force &&
) { saved.dataRevision !== MISSING_DATA_REVISION
return null; ) return null;
}
const reservation = await this.repository.reserve( const reservation = await this.repository.reserve(
{ userId, revision }, { userId, revision },
input, input,
@@ -38,6 +38,7 @@ import {
import { toSavedCandle } from '../infrastructure/tbank-candle.mapper'; import { toSavedCandle } from '../infrastructure/tbank-candle.mapper';
import { catalogSearchLimit } from './catalog-selection'; import { catalogSearchLimit } from './catalog-selection';
import { previousClose, timestamp } from './instrument-market'; import { previousClose, timestamp } from './instrument-market';
import { publicInstrument } from './public-instrument';
const MISSING_DATA_REVISION = 0; const MISSING_DATA_REVISION = 0;
const FIRST_QUEUE_INDEX = 0; const FIRST_QUEUE_INDEX = 0;
@@ -58,7 +59,7 @@ export class CatalogEnrichmentService {
private running = false; private running = false;
private readonly completions = new Map< private readonly completions = new Map<
string, string,
{ done: Promise<void>; resolve: () => void; workId: string } { attempt: TbankSyncAttempt; done: Promise<void>; resolve: () => void; workId: string }
>(); >();
async ensureDetail( async ensureDetail(
@@ -154,7 +155,7 @@ export class CatalogEnrichmentService {
dataRevision: String(row?.dataRevision ?? MISSING_DATA_REVISION), dataRevision: String(row?.dataRevision ?? MISSING_DATA_REVISION),
availability: row ? 'complete' : 'missing', availability: row ? 'complete' : 'missing',
fetchedAt: row?.fetchedAt ?? null, fetchedAt: row?.fetchedAt ?? null,
freshForMs: tbankSourceTtlMs[sourcePolicy(input.kind)], freshForMs: sourcePolicy(input.kind) === 'quote' ? tbankSourceTtlMs.quote : Number.MAX_SAFE_INTEGER,
work: byScope.get(catalogScope(input)) ?? null, work: byScope.get(catalogScope(input)) ?? null,
}), }),
}; };
@@ -186,14 +187,10 @@ export class CatalogEnrichmentService {
(entry) => (entry) =>
entry.instrumentUid === input.uid && entry.kind === input.kind, entry.instrumentUid === input.uid && entry.kind === input.kind,
); );
if ( const policy = sourcePolicy(input.kind);
!tbankSourceNeedsRefresh( const needsRefresh = force || !row?.fetchedAt ||
sourcePolicy(input.kind), (policy === 'quote' && tbankSourceNeedsRefresh('quote', row.fetchedAt, new Date()));
row?.fetchedAt ?? null, if (!needsRefresh) {
new Date(),
force,
)
) {
continue; continue;
} }
const reserved = await this.repository.reserve( const reserved = await this.repository.reserve(
@@ -212,7 +209,7 @@ export class CatalogEnrichmentService {
const done = new Promise<void>((finish) => { const done = new Promise<void>((finish) => {
resolve = finish; resolve = finish;
}); });
this.completions.set(attemptId, { workId: id, done, resolve }); this.completions.set(attemptId, { attempt, done, resolve, workId: id });
const key = producerKey(input); const key = producerKey(input);
const producer = this.producers.get(key); const producer = this.producers.get(key);
if (producer) producer.observers.push({ attempt, input }); if (producer) producer.observers.push({ attempt, input });
@@ -234,6 +231,7 @@ export class CatalogEnrichmentService {
return reservation; return reservation;
} }
// eslint-disable-next-line sonarjs/cognitive-complexity -- Section composition owns the source dependency policy.
async section( async section(
userId: string, userId: string,
instrumentType: 'bond' | 'share', instrumentType: 'bond' | 'share',
@@ -241,6 +239,34 @@ export class CatalogEnrichmentService {
section: 'company' | 'metrics' | 'quote', section: 'company' | 'metrics' | 'quote',
observe = false, observe = false,
) { ) {
if (section === 'company' && !observe) {
const token = await this.access.onlineToken(userId);
const instrument =
instrumentType === 'share'
? await this.tbank.getShare(token, uid)
: await this.tbank.getBond(token, uid);
if (!instrument) throw new NotFoundException();
const [description, fundamentals] = instrument.assetUid
? await Promise.all([
this.tbank.getAssetDescription(token, instrument.assetUid),
this.tbank.getFundamentals(token, instrument.assetUid),
])
: [null, null];
const meta = investmentResourceMeta({
resourceKey: `${instrumentType}/company/${uid}`,
dataRevision: `online:${Date.now()}`,
availability: 'complete',
fetchedAt: new Date(),
freshForMs: Number.MAX_SAFE_INTEGER,
work: null,
});
return {
values: { instrument: publicInstrument({ ...instrument, uid }), description, fundamentals },
meta,
unavailable: [],
observedAt: meta.lastSuccessfulAt ?? new Date().toISOString(),
};
}
const kinds: CatalogSource['kind'][] = const kinds: CatalogSource['kind'][] =
section === 'company' section === 'company'
? ['detail', 'description', 'fundamentals'] ? ['detail', 'description', 'fundamentals']
@@ -263,7 +289,7 @@ export class CatalogEnrichmentService {
kinds, kinds,
); );
const first = sources[0]; const first = sources[0];
if (first.meta.availability === 'complete' && !first.instrument) { if (first.meta.completeness === 'complete' && !first.instrument) {
throw new NotFoundException(); throw new NotFoundException();
} }
const values = sources.reduce<CatalogPayload>( const values = sources.reduce<CatalogPayload>(
@@ -272,7 +298,7 @@ export class CatalogEnrichmentService {
); );
const dependencies = sources.map((source) => source.meta); const dependencies = sources.map((source) => source.meta);
const complete = dependencies.every( const complete = dependencies.every(
(dependency) => dependency.availability === 'complete', (dependency) => dependency.completeness === 'complete',
); );
const meta = investmentResourceMeta({ const meta = investmentResourceMeta({
resourceKey: `${instrumentType}/${section}/${uid}`, resourceKey: `${instrumentType}/${section}/${uid}`,
@@ -287,10 +313,7 @@ export class CatalogEnrichmentService {
fetchedAt: first.meta.lastSuccessfulAt fetchedAt: first.meta.lastSuccessfulAt
? new Date(first.meta.lastSuccessfulAt) ? new Date(first.meta.lastSuccessfulAt)
: null, : null,
freshForMs: freshForMs: section === 'company' ? Number.MAX_SAFE_INTEGER : tbankSourceTtlMs.quote,
section === 'company'
? tbankSourceTtlMs.company
: tbankSourceTtlMs.quote,
work: null, work: null,
dependencies, dependencies,
}); });
@@ -300,7 +323,7 @@ export class CatalogEnrichmentService {
unavailable: sources unavailable: sources
.filter( .filter(
(source) => (source) =>
source.meta.availability !== 'complete' || source.meta.completeness !== 'complete' ||
source.meta.sync.state === 'failed', source.meta.sync.state === 'failed',
) )
.map((source) => source.input.kind), .map((source) => source.input.kind),
@@ -308,6 +331,7 @@ export class CatalogEnrichmentService {
}; };
} }
// eslint-disable-next-line sonarjs/cognitive-complexity -- Source loading intentionally dispatches by persisted resource kind.
private async load( private async load(
input: CatalogSource, input: CatalogSource,
token: string, token: string,
@@ -323,7 +347,7 @@ export class CatalogEnrichmentService {
} }
// eslint-disable-next-line sonarjs/different-types-comparison -- Defensive validation rejects missing upstream payloads instead of caching them as absence. // eslint-disable-next-line sonarjs/different-types-comparison -- Defensive validation rejects missing upstream payloads instead of caching them as absence.
if (instrument === undefined) throw new Error('Missing catalog response'); if (instrument === undefined) throw new Error('Missing catalog response');
return { instrument: instrument ? publicInstrument(instrument) : null }; return { instrument: instrument ? publicInstrument({ ...instrument, uid: input.uid }) : null };
} }
if (input.kind === 'trading-status') { if (input.kind === 'trading-status') {
return { return {
@@ -470,6 +494,7 @@ export class CatalogEnrichmentService {
throw new Error('Unsupported catalog source'); throw new Error('Unsupported catalog source');
} }
// eslint-disable-next-line sonarjs/cognitive-complexity -- Queue draining coordinates publication fences and observer completion.
private async drain() { private async drain() {
try { try {
while (this.queue.length) { while (this.queue.length) {
@@ -556,6 +581,14 @@ export class CatalogEnrichmentService {
} }
} }
} catch { } catch {
// Persistence failed mid-batch: finish every still-open observer
// attempt durably so no source row is left running. Best effort — a
// row that stays open is re-reserved by the next reader.
for (const completion of this.completions.values()) {
await this.repository
.finish(completion.attempt, false)
.catch(() => undefined);
}
for (const completion of this.completions.values()) completion.resolve(); for (const completion of this.completions.values()) completion.resolve();
this.completions.clear(); this.completions.clear();
this.active.clear(); this.active.clear();
@@ -600,39 +633,6 @@ function sourceInputs(
})), })),
); );
} }
function publicInstrument(value: TbankBond | TbankShare) {
const bond = value as TbankBond;
return {
uid: value.uid,
figi: value.figi,
assetUid: value.assetUid,
name: value.name,
ticker: value.ticker,
isin: value.isin,
classCode: value.classCode,
realExchange: value.realExchange,
apiTradeAvailableFlag: value.apiTradeAvailableFlag,
currency: value.currency,
shareType: (value as TbankShare).shareType,
sector: value.sector,
lot: value.lot,
brand: value.brand ? { logoName: value.brand.logoName } : undefined,
first_1dayCandleDate: value.first_1dayCandleDate,
nominal: bond.nominal,
initialNominal: bond.initialNominal,
aciValue: bond.aciValue,
maturityDate: bond.maturityDate,
callDate: bond.callDate,
couponQuantityPerYear: bond.couponQuantityPerYear,
floatingCouponFlag: bond.floatingCouponFlag,
perpetualFlag: bond.perpetualFlag,
amortizationFlag: bond.amortizationFlag,
subordinatedFlag: bond.subordinatedFlag,
bondType: bond.bondType,
riskLevel: bond.riskLevel,
};
}
const previousCloseLookbackDays = 20; const previousCloseLookbackDays = 20;
const previousCloseLookbackMs = const previousCloseLookbackMs =
previousCloseLookbackDays * MILLISECONDS_PER_DAY; previousCloseLookbackDays * MILLISECONDS_PER_DAY;
@@ -651,6 +651,7 @@ export type CatalogPayload = {
price?: null | TbankLastPrice; price?: null | TbankLastPrice;
tradingStatus?: null | string; tradingStatus?: null | string;
}; };
function sourcePolicy(kind: CatalogSource['kind']) { function sourcePolicy(kind: CatalogSource['kind']) {
if ( if (
[ [
@@ -663,9 +664,7 @@ function sourcePolicy(kind: CatalogSource['kind']) {
) { ) {
return 'quote'; return 'quote';
} }
if (kind === 'description' || kind === 'fundamentals') return 'company'; return 'stable';
if (['bond-events', 'coupons', 'dividends'].includes(kind)) return 'calendar';
return 'metadata';
} }
function producerKey(input: CatalogSource) { function producerKey(input: CatalogSource) {
return input.instrumentType === 'bond' && return input.instrumentType === 'bond' &&
@@ -0,0 +1,64 @@
import type {
TbankBond,
TbankShare,
} from '../../../../integrations/tbank/tbank.api';
import { publicInstrument } from './public-instrument';
const share = {
uid: 'share-uid',
figi: 'FIGI1',
assetUid: 'asset-1',
name: 'Share',
ticker: 'SHAR',
isin: 'US000',
classCode: 'TQBR',
realExchange: 'REAL_EXCHANGE_MOEX',
apiTradeAvailableFlag: true,
currency: 'rub',
shareType: 'SHARE_TYPE_COMMON',
sector: 'Finance',
lot: 1,
brand: { logoName: 'logo.png', title: 'Brand', id: 'brand-1' },
first_1dayCandleDate: '2020-01-01T00:00:00Z',
privateField: 'must-not-leak',
} as unknown as TbankShare;
const bond = {
uid: 'bond-uid',
figi: 'FIGI2',
assetUid: 'asset-2',
name: 'Bond',
ticker: 'BOND',
classCode: 'TQOB',
currency: 'rub',
lot: 1,
nominal: { value: '1000', currency: 'rub' },
initialNominal: { value: '1000', currency: 'rub' },
aciValue: { value: '10', currency: 'rub' },
maturityDate: '2030-01-01T00:00:00Z',
couponQuantityPerYear: 2,
bondType: 'BOND_TYPE_EXPIRING',
brand: null,
privateField: 'must-not-leak',
} as unknown as TbankBond;
describe('publicInstrument projection', () => {
it('keeps approved share fields, trims brand, drops private extras', () => {
const result = publicInstrument(share);
expect(result.uid).toBe('share-uid');
expect(result.shareType).toBe('SHARE_TYPE_COMMON');
expect(result.brand).toEqual({ logoName: 'logo.png' });
expect(result).not.toHaveProperty('privateField');
expect(result).not.toHaveProperty('brand.title');
});
it('projects bonds with missing optional fields and null brand', () => {
const result = publicInstrument(bond);
expect(result.nominal).toEqual({ value: '1000', currency: 'rub' });
expect(result.shareType).toBeUndefined();
expect(result.brand).toBeUndefined();
expect(result.isin).toBeUndefined();
expect(result).not.toHaveProperty('privateField');
});
});
@@ -0,0 +1,38 @@
import type {
TbankBond,
TbankShare,
} from '../../../../integrations/tbank/tbank.api';
export function publicInstrument(value: TbankBond | TbankShare) {
const bond = value as TbankBond;
const share = value as TbankShare;
return {
uid: value.uid,
figi: value.figi,
assetUid: value.assetUid,
name: value.name,
ticker: value.ticker,
isin: value.isin,
classCode: value.classCode,
realExchange: value.realExchange,
apiTradeAvailableFlag: value.apiTradeAvailableFlag,
currency: value.currency,
shareType: share.shareType,
sector: value.sector,
lot: value.lot,
brand: value.brand ? { logoName: value.brand.logoName } : undefined,
first_1dayCandleDate: value.first_1dayCandleDate,
nominal: bond.nominal,
initialNominal: bond.initialNominal,
aciValue: bond.aciValue,
maturityDate: bond.maturityDate,
callDate: bond.callDate,
couponQuantityPerYear: bond.couponQuantityPerYear,
floatingCouponFlag: bond.floatingCouponFlag,
perpetualFlag: bond.perpetualFlag,
amortizationFlag: bond.amortizationFlag,
subordinatedFlag: bond.subordinatedFlag,
bondType: bond.bondType,
riskLevel: bond.riskLevel,
};
}
@@ -71,8 +71,8 @@ export class BondIncomeService {
}; };
} }
async readPosition(userId: string, uid: string, observe = false) { async readPosition(userId: string, uid: string) {
const saved = await this.portfolios.read(userId, 'all', observe); const saved = await this.portfolios.read(userId, 'all');
return { return {
data: saved.data ? instrumentPosition(saved.data, uid, 'BOND') : null, data: saved.data ? instrumentPosition(saved.data, uid, 'BOND') : null,
meta: { ...saved.meta, resourceKey: `bond/position/${uid}` }, meta: { ...saved.meta, resourceKey: `bond/position/${uid}` },
@@ -88,7 +88,7 @@ export class BondIncomeService {
const today = moscowDay(new Date().toISOString()); const today = moscowDay(new Date().toISOString());
const from = `${Number(today.slice(ISO_DATE_START, ISO_YEAR_LENGTH)) - INCOME_HISTORY_YEARS}${today.slice(ISO_YEAR_LENGTH)}`; const from = `${Number(today.slice(ISO_DATE_START, ISO_YEAR_LENGTH)) - INCOME_HISTORY_YEARS}${today.slice(ISO_YEAR_LENGTH)}`;
const to = `${Number(today.slice(ISO_DATE_START, ISO_YEAR_LENGTH)) + INCOME_FORECAST_YEARS}${today.slice(ISO_YEAR_LENGTH)}`; const to = `${Number(today.slice(ISO_DATE_START, ISO_YEAR_LENGTH)) + INCOME_FORECAST_YEARS}${today.slice(ISO_YEAR_LENGTH)}`;
const portfolio = await this.portfolios.read(userId, 'all', observe); const portfolio = await this.portfolios.read(userId, 'all');
const sources = await this.sources.readMany( const sources = await this.sources.readMany(
userId, userId,
'bond', 'bond',
@@ -97,7 +97,7 @@ export class BondIncomeService {
['detail', 'coupons'], ['detail', 'coupons'],
{ from, to }, { from, to },
); );
if (sources[0].meta.availability === 'complete' && !sources[0].instrument) { if (sources[0].meta.completeness === 'complete' && !sources[0].instrument) {
throw new NotFoundException(); throw new NotFoundException();
} }
const events = sources[1].payload?.coupons; const events = sources[1].payload?.coupons;
@@ -137,7 +137,7 @@ export class BondIncomeService {
projected.observedAt, projected.observedAt,
true, true,
), ),
meta: { ...result.meta, availability: 'partial' }, meta: { ...result.meta, completeness: 'incomplete' },
}; };
} }
} }
@@ -119,13 +119,8 @@ export class BondController {
position( position(
@Req() request: Request & { user: AccessPayload }, @Req() request: Request & { user: AccessPayload },
@Param('uid', new ParseUUIDPipe()) uid: string, @Param('uid', new ParseUUIDPipe()) uid: string,
@Query() query: InvestmentReadQueryDto,
) { ) {
return this.income.readPosition( return this.income.readPosition(request.user.sub, uid);
request.user.sub,
uid,
query.sync === 'observe',
);
} }
@Get(':uid/coupons') @Get(':uid/coupons')
@@ -1,5 +1,6 @@
import { Module } from '@nestjs/common'; import { Module } from '@nestjs/common';
import { TbankAccessService } from '../../../integrations/tbank/tbank-access.service';
import { TbankModule } from '../../../integrations/tbank/tbank.module'; import { TbankModule } from '../../../integrations/tbank/tbank.module';
import { AuthModule } from '../../auth/auth.module'; import { AuthModule } from '../../auth/auth.module';
import { PortfolioModule } from '../portfolio/portfolio.module'; import { PortfolioModule } from '../portfolio/portfolio.module';
@@ -12,6 +13,7 @@ import { GetStockCompanyUseCase } from './stocks/application/get-stock-company.u
import { StockCatalogService } from './stocks/application/stock-catalog.service'; import { StockCatalogService } from './stocks/application/stock-catalog.service';
import { StockIncomeService } from './stocks/application/stock-income.service'; import { StockIncomeService } from './stocks/application/stock-income.service';
import { StockMarketService } from './stocks/application/stock-market.service'; import { StockMarketService } from './stocks/application/stock-market.service';
import { StockScreenerService } from './stocks/application/stock-screener.service';
import { StockController } from './stocks/presentation/stock.controller'; import { StockController } from './stocks/presentation/stock.controller';
@Module({ @Module({
@@ -19,12 +21,14 @@ import { StockController } from './stocks/presentation/stock.controller';
controllers: [StockController, BondController], controllers: [StockController, BondController],
exports: [StockIncomeService, BondIncomeService], exports: [StockIncomeService, BondIncomeService],
providers: [ providers: [
{ provide: 'TBANK_ACCESS', useExisting: TbankAccessService },
BondIncomeService, BondIncomeService,
BondMarketService, BondMarketService,
GetStockCompanyUseCase, GetStockCompanyUseCase,
StockIncomeService, StockIncomeService,
StockMarketService, StockMarketService,
StockCatalogService, StockCatalogService,
StockScreenerService,
BondCatalogService, BondCatalogService,
], ],
}) })
@@ -40,8 +40,8 @@ export class StockIncomeService {
private readonly forecasts: ForecastService, private readonly forecasts: ForecastService,
) {} ) {}
async readPosition(userId: string, uid: string, observe = false) { async readPosition(userId: string, uid: string) {
const saved = await this.portfolios.read(userId, 'all', observe); const saved = await this.portfolios.read(userId, 'all');
return { return {
data: saved.data ? instrumentPosition(saved.data, uid, 'STOCK') : null, data: saved.data ? instrumentPosition(saved.data, uid, 'STOCK') : null,
meta: { ...saved.meta, resourceKey: `stock/position/${uid}` }, meta: { ...saved.meta, resourceKey: `stock/position/${uid}` },
@@ -57,7 +57,7 @@ export class StockIncomeService {
const today = moscowDay(new Date().toISOString()); const today = moscowDay(new Date().toISOString());
const from = `${Number(today.slice(ISO_DATE_START, ISO_YEAR_LENGTH)) - INCOME_HISTORY_YEARS}${today.slice(ISO_YEAR_LENGTH)}`; const from = `${Number(today.slice(ISO_DATE_START, ISO_YEAR_LENGTH)) - INCOME_HISTORY_YEARS}${today.slice(ISO_YEAR_LENGTH)}`;
const to = `${Number(today.slice(ISO_DATE_START, ISO_YEAR_LENGTH)) + INCOME_FORECAST_YEARS}${today.slice(ISO_YEAR_LENGTH)}`; const to = `${Number(today.slice(ISO_DATE_START, ISO_YEAR_LENGTH)) + INCOME_FORECAST_YEARS}${today.slice(ISO_YEAR_LENGTH)}`;
const portfolio = await this.portfolios.read(userId, 'all', observe); const portfolio = await this.portfolios.read(userId, 'all');
const sources = await this.sources.readMany( const sources = await this.sources.readMany(
userId, userId,
'share', 'share',
@@ -66,7 +66,7 @@ export class StockIncomeService {
['detail', 'dividends'], ['detail', 'dividends'],
{ from, to }, { from, to },
); );
if (sources[0].meta.availability === 'complete' && !sources[0].instrument) { if (sources[0].meta.completeness === 'complete' && !sources[0].instrument) {
throw new NotFoundException(); throw new NotFoundException();
} }
const events = sources[1].payload?.dividends; const events = sources[1].payload?.dividends;
@@ -106,7 +106,7 @@ export class StockIncomeService {
projected.observedAt, projected.observedAt,
true, true,
), ),
meta: { ...result.meta, availability: 'partial' }, meta: { ...result.meta, completeness: 'incomplete' },
}; };
} }
} }
@@ -0,0 +1,106 @@
import { NotFoundException, ServiceUnavailableException } from '@nestjs/common';
import type { TbankShare } from '../../../../../integrations/tbank/tbank.api';
import { StockScreenerService } from './stock-screener.service';
const token = 'secret-token';
const revision = 'revision-1';
const pageSize = 30;
function share(index: number, overrides: Partial<TbankShare> = {}): TbankShare {
return {
apiTradeAvailableFlag: true,
buyAvailableFlag: true,
currency: 'RUB',
lot: 1,
name: `Share ${index}`,
ticker: `T${String(index).padStart(2, '0')}`,
uid: `e6123145-9665-43e0-8413-cd61b8aa${String(index).padStart(4, '0')}`,
...overrides,
};
}
describe('StockScreenerService', () => {
const access = { current: jest.fn(), local: jest.fn() };
const client = { getAllShares: jest.fn() };
const logger = { event: jest.fn() };
let service: StockScreenerService;
beforeEach(() => {
jest.clearAllMocks();
access.current.mockResolvedValue({ revision, token });
access.local.mockResolvedValue({ revision });
client.getAllShares.mockResolvedValue(
Array.from({ length: pageSize + 1 }, (_, index) => share(index)),
);
service = new StockScreenerService(access, client as never, logger as never);
});
it('stores the full selected catalog and returns stable non-overlapping pages', async () => {
const first = await service.search('user-1', {});
const second = await service.page('user-1', first.searchId, 2);
expect(first.items).toHaveLength(pageSize);
expect(first.hasNext).toBe(true);
expect(second).toMatchObject({ asOf: first.asOf, hasNext: false, page: 2, searchId: first.searchId });
expect(second.items).toEqual([expect.objectContaining({ ticker: 'T30' })]);
expect(client.getAllShares).toHaveBeenCalledTimes(1);
});
it('creates a fresh result for every search and preserves a successful empty result', async () => {
client.getAllShares.mockResolvedValueOnce([share(1, { sector: 'energy' })]).mockResolvedValueOnce([share(2, { sector: 'energy' })]);
const first = await service.search('user-1', { sector: 'finance' });
const second = await service.search('user-1', { sector: 'finance' });
expect(first).toMatchObject({ hasNext: false, items: [], page: 1 });
expect(second).toMatchObject({ hasNext: false, items: [], page: 1 });
expect(second.searchId).not.toBe(first.searchId);
expect(client.getAllShares).toHaveBeenCalledTimes(2);
});
it('does not store a result when the catalog source fails', async () => {
client.getAllShares.mockRejectedValueOnce(new ServiceUnavailableException());
await expect(service.search('user-1', {})).rejects.toBeInstanceOf(ServiceUnavailableException);
});
it('denies missing, foreign, stale, and out-of-range pages without fetching', async () => {
const first = await service.search('user-1', {});
access.local.mockResolvedValueOnce({ revision: 'revision-2' });
await expect(service.page('user-1', first.searchId, 1)).rejects.toBeInstanceOf(NotFoundException);
await expect(service.page('user-2', first.searchId, 1)).rejects.toBeInstanceOf(NotFoundException);
await expect(service.page('user-1', 'missing', 1)).rejects.toBeInstanceOf(NotFoundException);
await expect(service.page('user-1', first.searchId, 3)).rejects.toMatchObject({ status: 400 });
expect(client.getAllShares).toHaveBeenCalledTimes(1);
});
it('null-normalizes optional source fields and rejects malformed required fields', async () => {
client.getAllShares.mockResolvedValueOnce([
share(1, {
countryOfRisk: undefined,
currency: undefined,
isin: undefined,
realExchange: undefined,
sector: undefined,
shareType: undefined,
ticker: undefined,
}),
]);
const result = await service.search('user-1', {});
expect(result.items[0]).toEqual(expect.objectContaining({ countryOfRisk: null, currency: null, isin: null, logoName: null, ticker: null }));
client.getAllShares.mockResolvedValueOnce([share(2, { name: ' ' })]);
await expect(service.search('user-1', {})).rejects.toBeInstanceOf(ServiceUnavailableException);
});
it('preserves the catalog logo name in screener results', async () => {
client.getAllShares.mockResolvedValueOnce([share(1, { brand: { logoName: 'sber.png' } })]);
const result = await service.search('user-1', {});
expect(result.items).toEqual([expect.objectContaining({ logoName: 'sber.png' })]);
});
});
@@ -0,0 +1,140 @@
import {
BadRequestException,
Inject,
Injectable,
NotFoundException,
ServiceUnavailableException,
} from '@nestjs/common';
import { randomUUID } from 'node:crypto';
import type { TbankShare } from '../../../../../integrations/tbank/tbank-market.types';
import { StructuredLogger } from '../../../../../common/logging/structured-logger.service';
import { TbankClient } from '../../../../../integrations/tbank/tbank.client';
import {
selectStockScreenerInstruments,
type StockScreenerConditions,
} from '../domain/stock-screener-selection.rules';
const MAX_SNAPSHOTS = 100;
const PAGE_SIZE = 30;
const MIN_PAGE = 1;
const MINIMUM_SNAPSHOT_COUNT = 1;
const SECONDS_PER_MINUTE = 60;
const SNAPSHOT_TTL_MINUTES = 5;
const MILLISECONDS_PER_SECOND = 1_000;
const SNAPSHOT_TTL_MS = SNAPSHOT_TTL_MINUTES * SECONDS_PER_MINUTE * MILLISECONDS_PER_SECOND;
const UUID_PATTERN = /^[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i;
type StockScreenerItem = {
countryOfRisk: null | string;
currency: null | string;
instrumentUid: string;
isin: null | string;
logoName: null | string;
lot: null | number;
name: string;
realExchange: null | string;
sector: null | string;
shareType: null | string;
ticker: null | string;
};
type StockScreenerPage = {
asOf: string;
hasNext: boolean;
items: StockScreenerItem[];
page: number;
searchId: string;
};
type Snapshot = { asOf: string; expiresAt: number; items: StockScreenerItem[]; revision: string; userId: string };
type TbankAccess = {
current(userId: string): Promise<{ revision: string; token: string }>;
local(userId: string): Promise<{ revision: string }>;
};
@Injectable()
export class StockScreenerService {
private readonly snapshots = new Map<string, Snapshot>();
constructor(
@Inject('TBANK_ACCESS') private readonly access: TbankAccess,
private readonly client: TbankClient,
private readonly logger: StructuredLogger,
) {}
async search(userId: string, conditions: StockScreenerConditions): Promise<StockScreenerPage> {
const connection = await this.access.current(userId);
const instruments = await this.client.getAllShares(connection.token);
const asOf = new Date().toISOString();
const selected = selectStockScreenerInstruments({ asOf: new Date(asOf), instruments }, conditions)
.instruments.map((instrument) => this.item(instrument));
const searchId = randomUUID();
this.evict();
this.snapshots.set(searchId, {
asOf,
expiresAt: Date.now() + SNAPSHOT_TTL_MS,
items: selected,
revision: connection.revision,
userId,
});
this.logger.event('info', 'stock.screener.search.completed', { asOf, count: selected.length, operation: 'stock-screener-search' });
return this.result(searchId, this.snapshots.get(searchId) as Snapshot, MIN_PAGE);
}
async page(userId: string, searchId: string, page: number): Promise<StockScreenerPage> {
const snapshot = this.snapshots.get(searchId);
if (!snapshot || snapshot.expiresAt <= Date.now()) {
this.snapshots.delete(searchId);
this.logger.event('info', 'stock.screener.result.missing', { operation: 'stock-screener-read' });
throw new NotFoundException();
}
if (snapshot.userId !== userId) {
this.logger.event('warn', 'stock.screener.result.rejected', { operation: 'stock-screener-read' });
throw new NotFoundException();
}
const current = await this.access.local(userId);
if (snapshot.revision !== current.revision) {
this.logger.event('warn', 'stock.screener.result.rejected', { operation: 'stock-screener-read' });
throw new NotFoundException();
}
return this.result(searchId, snapshot, page);
}
private result(searchId: string, snapshot: Snapshot, page: number): StockScreenerPage {
const start = (page - MIN_PAGE) * PAGE_SIZE;
if (!Number.isInteger(page) || page < MIN_PAGE || (start >= snapshot.items.length && !(page === MIN_PAGE && !snapshot.items.length))) {
throw new BadRequestException();
}
const items = snapshot.items.slice(start, start + PAGE_SIZE);
return { asOf: snapshot.asOf, hasNext: start + PAGE_SIZE < snapshot.items.length, items, page, searchId };
}
private item(instrument: TbankShare): StockScreenerItem {
if (!instrument.uid || !UUID_PATTERN.test(instrument.uid) || !instrument.name?.trim()) {
throw new ServiceUnavailableException();
}
return {
countryOfRisk: instrument.countryOfRisk ?? null,
currency: instrument.currency ?? null,
instrumentUid: instrument.uid,
isin: instrument.isin ?? null,
lot: instrument.lot ?? null,
logoName: instrument.brand?.logoName?.trim() || null,
name: instrument.name,
realExchange: instrument.realExchange ?? null,
sector: instrument.sector ?? null,
shareType: instrument.shareType ?? null,
ticker: instrument.ticker ?? null,
};
}
private evict(): void {
for (const [searchId, snapshot] of this.snapshots) {
if (snapshot.expiresAt <= Date.now()) this.snapshots.delete(searchId);
}
while (this.snapshots.size >= MAX_SNAPSHOTS) {
const searchId = this.snapshots.keys().next().value;
if (searchId && this.snapshots.size >= MINIMUM_SNAPSHOT_COUNT) this.snapshots.delete(searchId);
}
}
}
@@ -0,0 +1,188 @@
import type { TbankShare } from '../../../../../integrations/tbank/tbank.api';
import { selectStockScreenerInstruments } from './stock-screener-selection.rules';
const asOf = new Date('2026-10-10T12:00:00.000Z');
function share(overrides: Partial<TbankShare> = {}): TbankShare {
return {
apiTradeAvailableFlag: true,
buyAvailableFlag: true,
currency: 'RUB',
lot: 10,
name: 'North Star',
ticker: 'NSTR',
uid: 'uid-1',
...overrides,
};
}
describe('selectStockScreenerInstruments', () => {
it('keeps only instruments matching every selected condition', () => {
const result = selectStockScreenerInstruments(
{
asOf,
instruments: [
share({ isin: 'A', sector: 'energy' }),
share({ currency: 'USD', isin: 'B', sector: 'energy', uid: 'uid-2' }),
share({ isin: 'C', name: 'Other', sector: 'energy', uid: 'uid-3' }),
share({ isin: 'D', sector: 'finance', uid: 'uid-4' }),
],
},
{ currency: 'RUB', query: 'star', sector: 'energy' },
);
expect(result).toEqual({ asOf, instruments: [share({ isin: 'A', sector: 'energy' })] });
});
it.each([
['sector', { sector: 'energy' }, share({ sector: undefined })],
['share type', { shareType: 'common' }, share({ shareType: 'preferred' })],
['currency', { currency: 'USD' }, share({ currency: undefined })],
['exchange', { realExchange: 'MOEX' }, share({ realExchange: undefined })],
['country of risk', { countryOfRisk: 'RU' }, share({ countryOfRisk: 'US' })],
['query', { query: 'star' }, share({ name: undefined, ticker: undefined })],
] as const)('excludes an unknown or unequal selected %s', (_condition, conditions, instrument) => {
const result = selectStockScreenerInstruments(
{ asOf, instruments: [instrument] },
conditions,
);
expect(result.instruments).toEqual([]);
});
it('uses an inclusive maximum lot and ignores lots when it is not selected', () => {
const instruments = [
share({ isin: 'equal', lot: 10, uid: 'equal' }),
share({ isin: 'large', lot: 11, uid: 'large' }),
share({ isin: 'unknown', lot: undefined, uid: 'unknown' }),
share({ isin: 'zero', lot: 0, uid: 'zero' }),
];
expect(
selectStockScreenerInstruments({ asOf, instruments }, { maxLot: 10 }).instruments.map(
({ uid }) => uid,
),
).toEqual(['equal']);
expect(
selectStockScreenerInstruments({ asOf, instruments }, {}).instruments.map(({ uid }) => uid),
).toEqual(['equal', 'large', 'unknown', 'zero']);
});
it('chooses the smallest matching uid as the complete representative of an ISIN', () => {
const chosen = share({ isin: ' same ', lot: 1, name: 'Chosen', uid: 'a' });
const result = selectStockScreenerInstruments(
{
asOf,
instruments: [share({ isin: 'SAME', lot: 99, name: 'Sibling', uid: 'z' }), chosen],
},
{},
);
expect(result.instruments).toEqual([chosen]);
expect(result.instruments[0]).toMatchObject({ lot: 1, name: 'Chosen', uid: 'a' });
});
it('chooses the sole matching mode of an issue', () => {
const matching = share({ isin: 'same', sector: 'energy', uid: 'matching' });
const result = selectStockScreenerInstruments(
{
asOf,
instruments: [share({ isin: 'same', sector: 'finance', uid: 'smaller-uid' }), matching],
},
{ sector: 'energy' },
);
expect(result.instruments).toEqual([matching]);
});
it('keeps different ISIN issues from one issuer separate', () => {
const result = selectStockScreenerInstruments(
{
asOf,
instruments: [
share({ isin: 'common', shareType: 'common', uid: 'common' }),
share({ isin: 'preferred', shareType: 'preferred', uid: 'preferred' }),
],
},
{},
);
expect(result.instruments.map(({ uid }) => uid)).toEqual(['common', 'preferred']);
});
it('treats instruments without an ISIN as separate issues', () => {
const result = selectStockScreenerInstruments(
{
asOf,
instruments: [share({ isin: ' ', uid: 'first' }), share({ isin: undefined, uid: 'second' })],
},
{},
);
expect(result.instruments.map(({ uid }) => uid)).toEqual(['first', 'second']);
});
it('preserves exact whitespace-differing UIDs as separate blank-ISIN issues', () => {
const result = selectStockScreenerInstruments(
{
asOf,
instruments: [
share({ isin: ' ', uid: 'uid' }),
share({ isin: undefined, uid: ' uid ' }),
],
},
{},
);
expect(result.instruments.map(({ uid }) => uid)).toEqual([' uid ', 'uid']);
});
it('returns the exact chosen uid for the later card link', () => {
const result = selectStockScreenerInstruments(
{
asOf,
instruments: [share({ isin: 'same', uid: 'sibling' }), share({ isin: 'SAME', uid: 'chosen' })],
},
{},
);
expect(result.instruments[0]?.uid).toBe('chosen');
});
it.each([
[true, true, 1],
[true, false, 0],
[false, true, 0],
[false, false, 0],
[true, undefined, 0],
[undefined, true, 0],
] as const)(
'includes only confirmed buy and API availability: buy=%s api=%s',
(buyAvailableFlag, apiTradeAvailableFlag, expectedCount) => {
const result = selectStockScreenerInstruments(
{ asOf, instruments: [share({ apiTradeAvailableFlag, buyAvailableFlag })] },
{},
);
expect(result.instruments).toHaveLength(expectedCount);
},
);
it('orders representatives by ticker and then uid before paging', () => {
const result = selectStockScreenerInstruments(
{
asOf,
instruments: [
share({ isin: 'c', ticker: 'B', uid: 'z' }),
share({ isin: 'a', ticker: 'A', uid: 'z' }),
share({ isin: 'b', ticker: 'A', uid: 'a' }),
share({ isin: 'empty', ticker: ' ', uid: 'empty' }),
],
},
{},
);
expect(result.instruments.map(({ uid }) => uid)).toEqual(['empty', 'a', 'z', 'z']);
});
});
@@ -0,0 +1,106 @@
import type { TbankShare } from '../../../../../integrations/tbank/tbank.api';
export type StockScreenerCatalog = {
asOf: Date;
instruments: TbankShare[];
};
export type StockScreenerConditions = {
countryOfRisk?: string;
currency?: string;
maxLot?: number;
query?: string;
realExchange?: string;
sector?: string;
shareType?: string;
};
type IdentifiedShare = TbankShare & { uid: string };
const sortBefore = 0;
export function selectStockScreenerInstruments(
catalog: StockScreenerCatalog,
conditions: StockScreenerConditions,
): StockScreenerCatalog {
const representatives = new Map<string, IdentifiedShare>();
for (const instrument of catalog.instruments) {
if (!matches(instrument, conditions)) continue;
const { uid } = instrument;
const isin = instrument.isin?.trim().toUpperCase();
const issueKey = isin ? `isin:${isin}` : `uid:${uid}`;
const existing = representatives.get(issueKey);
if (!existing || uid.localeCompare(existing.uid) < sortBefore) {
representatives.set(issueKey, instrument);
}
}
return {
asOf: catalog.asOf,
instruments: [...representatives.values()].sort(
(left, right) =>
(left.ticker?.trim() ?? '').localeCompare(right.ticker?.trim() ?? '') ||
left.uid.localeCompare(right.uid),
),
};
}
function matches(
instrument: TbankShare,
conditions: StockScreenerConditions,
): instrument is IdentifiedShare {
if (
!instrument.uid?.trim() ||
instrument.buyAvailableFlag !== true ||
instrument.apiTradeAvailableFlag !== true
) {
return false;
}
if (
conditions.query !== undefined &&
!`${instrument.name ?? ''}\n${instrument.ticker ?? ''}`
.toLowerCase()
.includes(conditions.query.toLowerCase())
) {
return false;
}
if (conditions.sector !== undefined && instrument.sector !== conditions.sector) {
return false;
}
if (
conditions.shareType !== undefined &&
instrument.shareType !== conditions.shareType
) {
return false;
}
if (
conditions.currency !== undefined &&
instrument.currency !== conditions.currency
) {
return false;
}
if (
conditions.realExchange !== undefined &&
instrument.realExchange !== conditions.realExchange
) {
return false;
}
if (
conditions.countryOfRisk !== undefined &&
instrument.countryOfRisk !== conditions.countryOfRisk
) {
return false;
}
const minimumPositiveLot = 0;
return (
conditions.maxLot === undefined ||
(Number.isInteger(instrument.lot) &&
instrument.lot !== undefined &&
instrument.lot > minimumPositiveLot &&
instrument.lot <= conditions.maxLot)
);
}
@@ -1,15 +1,21 @@
import type { Request } from 'express'; import type { Request } from 'express';
import { import {
BadRequestException,
Body,
Controller, Controller,
Get, Get,
HttpCode,
HttpStatus,
Param, Param,
ParseUUIDPipe, ParseUUIDPipe,
Post,
Query, Query,
Req, Req,
UseGuards, UseGuards,
ValidationPipe,
} from '@nestjs/common'; } from '@nestjs/common';
import { ApiBearerAuth, ApiOkResponse, ApiTags } from '@nestjs/swagger'; import { ApiBearerAuth, ApiOkResponse, ApiOperation, ApiTags } from '@nestjs/swagger';
import { ApiProblemResponses } from '../../../../../common/decorators/api-safe-error-response.decorator'; import { ApiProblemResponses } from '../../../../../common/decorators/api-safe-error-response.decorator';
import { AccessPayload, JwtAuthGuard } from '../../../../auth/auth.api'; import { AccessPayload, JwtAuthGuard } from '../../../../auth/auth.api';
@@ -18,6 +24,7 @@ import { GetStockCompanyUseCase } from '../application/get-stock-company.use-cas
import { StockCatalogService } from '../application/stock-catalog.service'; import { StockCatalogService } from '../application/stock-catalog.service';
import { StockIncomeService } from '../application/stock-income.service'; import { StockIncomeService } from '../application/stock-income.service';
import { StockMarketService } from '../application/stock-market.service'; import { StockMarketService } from '../application/stock-market.service';
import { StockScreenerService } from '../application/stock-screener.service';
import { import {
StockCandlesQueryDto, StockCandlesQueryDto,
StockCandlesResourceDto, StockCandlesResourceDto,
@@ -26,6 +33,9 @@ import {
StockPositionResourceDto, StockPositionResourceDto,
StockQuoteResourceDto, StockQuoteResourceDto,
StockResourceDto, StockResourceDto,
StockScreenerCriteriaDto,
StockScreenerPageDto,
StockScreenerPageQueryDto,
StockSearchQueryDto, StockSearchQueryDto,
StockSearchResourceDto, StockSearchResourceDto,
StockTickerParamsDto, StockTickerParamsDto,
@@ -42,6 +52,7 @@ export class StockController {
private readonly market: StockMarketService, private readonly market: StockMarketService,
private readonly income: StockIncomeService, private readonly income: StockIncomeService,
private readonly companyInfo: GetStockCompanyUseCase, private readonly companyInfo: GetStockCompanyUseCase,
private readonly screener: StockScreenerService,
) {} ) {}
@Get() @Get()
@@ -71,6 +82,33 @@ export class StockController {
); );
} }
@Post('screener/search')
@HttpCode(HttpStatus.OK)
@ApiOperation({ operationId: 'createStockScreenerSearch' })
@ApiOkResponse({ type: StockScreenerPageDto })
searchScreener(
@Req() request: Request & { user: AccessPayload },
@Body(new ValidationPipe({ forbidNonWhitelisted: true, forbidUnknownValues: true, whitelist: true }))
criteria: StockScreenerCriteriaDto,
) {
const allowed = new Set(['countryOfRisk', 'currency', 'maxLot', 'query', 'realExchange', 'sector', 'shareType']);
if (Object.keys(request.body as object).some((key) => !allowed.has(key))) {
throw new BadRequestException();
}
return this.screener.search(request.user.sub, criteria);
}
@Get('screener/search/:searchId')
@ApiOperation({ operationId: 'readStockScreenerPage' })
@ApiOkResponse({ type: StockScreenerPageDto })
pageScreener(
@Req() request: Request & { user: AccessPayload },
@Param('searchId') searchId: string,
@Query() query: StockScreenerPageQueryDto,
) {
return this.screener.page(request.user.sub, searchId, query.page);
}
@Get(':uid') @Get(':uid')
@ApiOkResponse({ type: StockResourceDto }) @ApiOkResponse({ type: StockResourceDto })
get( get(
@@ -111,13 +149,8 @@ export class StockController {
position( position(
@Req() request: Request & { user: AccessPayload }, @Req() request: Request & { user: AccessPayload },
@Param('uid', new ParseUUIDPipe()) uid: string, @Param('uid', new ParseUUIDPipe()) uid: string,
@Query() query: InvestmentReadQueryDto,
) { ) {
return this.income.readPosition( return this.income.readPosition(request.user.sub, uid);
request.user.sub,
uid,
query.sync === 'observe',
);
} }
@Get(':uid/company') @Get(':uid/company')
@@ -2,10 +2,12 @@ import { ApiProperty, ApiPropertyOptional } from '@nestjs/swagger';
import { Transform } from 'class-transformer'; import { Transform } from 'class-transformer';
import { import {
IsIn, IsIn,
IsInt,
IsOptional, IsOptional,
IsString, IsString,
Matches, Matches,
MaxLength, MaxLength,
Min,
MinLength, MinLength,
} from 'class-validator'; } from 'class-validator';
@@ -21,6 +23,86 @@ import {
import { PortfolioPositionDto } from '../../../portfolio/portfolio.api'; import { PortfolioPositionDto } from '../../../portfolio/portfolio.api';
const MIN_TICKER_LENGTH = 1; const MIN_TICKER_LENGTH = 1;
const MAX_SCREENER_QUERY_LENGTH = 80;
const MIN_PAGE = 1;
export class StockScreenerCriteriaDto {
@ApiPropertyOptional({ maxLength: MAX_SCREENER_QUERY_LENGTH })
@Transform(({ value }: { value: unknown }) =>
value === '' ? undefined : value,
)
@IsOptional()
@IsString()
@MinLength(MIN_TICKER_LENGTH)
@MaxLength(MAX_SCREENER_QUERY_LENGTH)
query?: string;
@ApiPropertyOptional({ minLength: MIN_TICKER_LENGTH })
@IsOptional()
@IsString()
@MinLength(MIN_TICKER_LENGTH)
sector?: string;
@ApiPropertyOptional({ minLength: MIN_TICKER_LENGTH })
@IsOptional()
@IsString()
@MinLength(MIN_TICKER_LENGTH)
shareType?: string;
@ApiPropertyOptional({ minLength: MIN_TICKER_LENGTH })
@IsOptional()
@IsString()
@MinLength(MIN_TICKER_LENGTH)
currency?: string;
@ApiPropertyOptional({ minLength: MIN_TICKER_LENGTH })
@IsOptional()
@IsString()
@MinLength(MIN_TICKER_LENGTH)
realExchange?: string;
@ApiPropertyOptional({ minLength: MIN_TICKER_LENGTH })
@IsOptional()
@IsString()
@MinLength(MIN_TICKER_LENGTH)
countryOfRisk?: string;
@ApiPropertyOptional({ minimum: MIN_TICKER_LENGTH, type: 'integer' })
@IsOptional()
@IsInt()
@Min(MIN_TICKER_LENGTH)
maxLot?: number;
}
export class StockScreenerPageQueryDto {
@ApiProperty({ minimum: MIN_PAGE, type: 'integer' })
@Transform(({ value }: { value: unknown }) => Number(value))
@IsInt()
@Min(MIN_PAGE)
declare page: number;
}
export class StockScreenerItemDto {
@ApiProperty({ format: 'uuid' }) declare instrumentUid: string;
@ApiProperty() declare name: string;
@ApiProperty({ nullable: true, type: String }) declare ticker: null | string;
@ApiProperty({ nullable: true, type: String }) declare isin: null | string;
@ApiProperty({ nullable: true, type: String }) declare logoName: null | string;
@ApiProperty({ nullable: true, type: String }) declare shareType: null | string;
@ApiProperty({ nullable: true, type: String }) declare sector: null | string;
@ApiProperty({ nullable: true, type: String }) declare currency: null | string;
@ApiProperty({ nullable: true, type: String }) declare realExchange: null | string;
@ApiProperty({ nullable: true, type: String }) declare countryOfRisk: null | string;
@ApiProperty({ nullable: true, minimum: MIN_TICKER_LENGTH, type: 'integer' }) declare lot: null | number;
}
export class StockScreenerPageDto {
@ApiProperty() declare searchId: string;
@ApiProperty({ format: 'date-time' }) declare asOf: string;
@ApiProperty({ minimum: MIN_PAGE, type: 'integer' }) declare page: number;
@ApiProperty() declare hasNext: boolean;
@ApiProperty({ type: StockScreenerItemDto, isArray: true, maxItems: 30 }) declare items: StockScreenerItemDto[];
}
export class StockSearchQueryDto extends InvestmentReadQueryDto { export class StockSearchQueryDto extends InvestmentReadQueryDto {
@ApiProperty({ minLength: MIN_INSTRUMENT_QUERY_LENGTH, maxLength: MAX_INSTRUMENT_QUERY_LENGTH }) @ApiProperty({ minLength: MIN_INSTRUMENT_QUERY_LENGTH, maxLength: MAX_INSTRUMENT_QUERY_LENGTH })
@@ -1,9 +1,21 @@
import { ApiProperty, ApiPropertyOptional } from '@nestjs/swagger'; import { ApiProperty, ApiPropertyOptional } from '@nestjs/swagger';
import { IsIn, IsOptional } from 'class-validator'; import { IsIn, IsOptional } from 'class-validator';
export const availabilityValues = ['missing', 'partial', 'complete'] as const; export const availabilityValues = [
'available',
'initial-loading',
'temporarily-unavailable',
'connection-unavailable',
] as const;
export const completenessValues = ['complete', 'incomplete'] as const;
export const freshnessValues = ['unknown', 'fresh', 'stale'] as const; export const freshnessValues = ['unknown', 'fresh', 'stale'] as const;
export const syncStateValues = ['idle', 'running', 'failed'] as const; export const attemptStateValues = [
'none',
'running',
'succeeded',
'partial',
'failed',
] as const;
export class InvestmentReadQueryDto { export class InvestmentReadQueryDto {
@ApiPropertyOptional({ enum: ['observe'] }) @ApiPropertyOptional({ enum: ['observe'] })
@@ -12,17 +24,31 @@ export class InvestmentReadQueryDto {
declare sync?: 'observe'; declare sync?: 'observe';
} }
export class InvestmentSyncDto { export class InvestmentAttemptProblemDto {
@ApiProperty({ type: String, nullable: true }) @ApiProperty({ enum: ['membership', 'portfolio', 'operations', 'events'] })
declare id: null | string; declare section: 'events' | 'membership' | 'operations' | 'portfolio';
@ApiProperty({ enum: syncStateValues }) @ApiProperty({ type: String, nullable: true })
declare state: (typeof syncStateValues)[number]; declare accountId: null | string;
@ApiProperty({ type: String, nullable: true }) @ApiProperty({ type: String, nullable: true })
declare errorCode: null | string; declare errorCode: null | string;
} }
export class InvestmentAttemptDto {
@ApiProperty({ enum: attemptStateValues })
declare state: (typeof attemptStateValues)[number];
@ApiProperty({ type: String, nullable: true, format: 'date-time' })
declare startedAt: null | string;
@ApiProperty({ type: String, nullable: true, format: 'date-time' })
declare finishedAt: null | string;
@ApiProperty({ type: InvestmentAttemptProblemDto, isArray: true })
declare problems: InvestmentAttemptProblemDto[];
}
export class InvestmentRangeDto { export class InvestmentRangeDto {
@ApiPropertyOptional() @ApiPropertyOptional()
accountId?: string; accountId?: string;
@@ -73,14 +99,21 @@ export class InvestmentDependencyDto {
@ApiProperty({ enum: availabilityValues }) @ApiProperty({ enum: availabilityValues })
declare availability: (typeof availabilityValues)[number]; declare availability: (typeof availabilityValues)[number];
@ApiProperty({ enum: completenessValues })
declare completeness: (typeof completenessValues)[number];
@ApiProperty({ enum: freshnessValues }) @ApiProperty({ enum: freshnessValues })
declare freshness: (typeof freshnessValues)[number]; declare freshness: (typeof freshnessValues)[number];
@ApiProperty({ type: String, format: 'date-time', nullable: true }) @ApiProperty({ type: String, format: 'date-time', nullable: true })
declare lastSuccessfulAt: null | string; declare lastSuccessfulAt: null | string;
@ApiProperty({ type: InvestmentSyncDto }) @ApiProperty({ type: InvestmentAttemptDto })
declare sync: InvestmentSyncDto; declare lastAttempt: InvestmentAttemptDto;
// Internal resource-source work remains available to existing source owners.
// It is deliberately undecorated and therefore not part of the HTTP schema.
declare sync: { errorCode: null | string; id: null | string; state: 'failed' | 'idle' | 'running' };
} }
export class InvestmentResourceMetaDto extends InvestmentDependencyDto { export class InvestmentResourceMetaDto extends InvestmentDependencyDto {
@@ -1,4 +1,9 @@
jest.mock('../../prisma/prisma.service', () => ({
PrismaService: class {},
}));
import { investmentResourceMeta } from './investment-resource.mapper'; import { investmentResourceMeta } from './investment-resource.mapper';
import { hourlyAttempt } from './tbank-sync/tbank-sync.repository';
describe('investment resource source freshness', () => { describe('investment resource source freshness', () => {
afterEach(() => jest.useRealTimers()); afterEach(() => jest.useRealTimers());
@@ -36,6 +41,105 @@ describe('investment resource source freshness', () => {
}).freshness, }).freshness,
).toBe('unknown'); ).toBe('unknown');
}); });
it('keeps confirmed data state independent from a failed hourly attempt', () => {
jest.useFakeTimers().setSystemTime(new Date('2026-09-25T11:30:00Z'));
const meta = investmentResourceMeta({
resourceKey: 'portfolio/a',
dataRevision: '1',
availability: 'complete',
fetchedAt: new Date('2026-09-25T11:00:00Z'),
freshForMs: 3_600_000,
work: null,
lastAttempt: {
startedAt: '2026-09-25T12:00:00.000Z',
finishedAt: '2026-09-25T12:01:00.000Z',
state: 'partial',
problems: [
{
section: 'portfolio',
accountId: 'a',
errorCode: 'TBANK_UNAVAILABLE',
},
],
},
});
expect(meta).toMatchObject({
availability: 'available',
completeness: 'complete',
freshness: 'fresh',
lastSuccessfulAt: '2026-09-25T11:00:00.000Z',
lastAttempt: {
state: 'partial',
startedAt: '2026-09-25T12:00:00.000Z',
finishedAt: '2026-09-25T12:01:00.000Z',
},
});
});
it('projects only the matching hourly section into a resource attempt', () => {
const attempt = hourlyAttempt(
{
startedAt: new Date('2026-09-25T12:00:00Z'),
finishedAt: new Date('2026-09-25T12:01:00Z'),
state: 'partial',
sections: [
{ section: 'portfolio', accountId: 'a', errorCode: 'TBANK_UNAVAILABLE' },
{ section: 'operations', accountId: 'a', errorCode: null },
{ section: 'events', errorCode: 'CALENDAR_UNAVAILABLE' },
],
},
'portfolio',
'a',
);
const meta = investmentResourceMeta({
resourceKey: 'portfolio/a',
dataRevision: '1',
availability: 'complete',
fetchedAt: new Date('2026-09-25T11:00:00Z'),
freshForMs: 3_600_000,
work: { id: 'legacy-work', state: 'failed', errorCode: 'LEGACY_FAILURE' },
lastAttempt: attempt,
});
expect(meta).toMatchObject({
availability: 'available',
completeness: 'complete',
lastAttempt: {
state: 'partial',
startedAt: '2026-09-25T12:00:00.000Z',
finishedAt: '2026-09-25T12:01:00.000Z',
problems: [{ section: 'portfolio', accountId: 'a', errorCode: 'TBANK_UNAVAILABLE' }],
},
});
expect(JSON.stringify(meta)).not.toContain('legacy-work');
});
it('projects no confirmed result as incomplete without inventing a successful attempt', () => {
expect(
investmentResourceMeta({
resourceKey: 'accounts',
dataRevision: '0',
availability: 'missing',
fetchedAt: null,
freshForMs: 3_600_000,
work: { id: 'work', state: 'running', errorCode: null },
}),
).toMatchObject({
availability: 'initial-loading',
completeness: 'incomplete',
freshness: 'unknown',
lastSuccessfulAt: null,
lastAttempt: {
state: 'running',
startedAt: null,
finishedAt: null,
problems: [],
},
});
});
}); });
describe('dependent resource lifecycle', () => { describe('dependent resource lifecycle', () => {
@@ -60,8 +164,8 @@ describe('dependent resource lifecycle', () => {
...input, ...input,
dependencies: [failed, running], dependencies: [failed, running],
}); });
expect(parent.sync).toEqual(running.sync); expect(parent.lastAttempt.state).toBe('none');
expect(parent.availability).toBe('complete'); expect(parent.availability).toBe('available');
const terminal = investmentResourceMeta({ const terminal = investmentResourceMeta({
...input, ...input,
dependencies: [failed], dependencies: [failed],
@@ -1,41 +1,34 @@
import { InvestmentResourceMetaDto } from './investment-resource.dto'; import { InvestmentResourceMetaDto } from './investment-resource.dto';
type LegacyAvailability = 'available' | 'complete' | 'connection-unavailable' | 'initial-loading' | 'missing' | 'partial' | 'temporarily-unavailable';
type Work = null | { errorCode: null | string; id: string; state: string };
type Attempt = NonNullable<InvestmentResourceMetaDto['lastAttempt']>;
export function investmentResourceMeta(input: { export function investmentResourceMeta(input: {
availability: InvestmentResourceMetaDto['availability']; availability: LegacyAvailability;
coverage?: NonNullable<InvestmentResourceMetaDto['coverage']>; coverage?: NonNullable<InvestmentResourceMetaDto['coverage']>;
dataRevision: string; dataRevision: string;
dependencies?: InvestmentResourceMetaDto[]; dependencies?: InvestmentResourceMetaDto[];
fetchedAt: Date | null; fetchedAt: Date | null;
freshForMs: number; freshForMs: number;
lastAttempt?: Attempt;
resourceKey: string; resourceKey: string;
unavailableReason?: string; unavailableReason?: string;
work: null | { errorCode: null | string; id: string; state: string }; work: Work;
}): InvestmentResourceMetaDto { }): InvestmentResourceMetaDto {
const ownSync = { const ownSync = sourceSync(input.work);
id: input.work?.id ?? null, const syncStates = [
state:
input.work?.state === 'running'
? ('running' as const)
: input.work?.state === 'failed'
? ('failed' as const)
: ('idle' as const),
errorCode: input.work?.errorCode ?? null,
};
const states = [
ownSync, ownSync,
...(input.dependencies ?? []) ...(input.dependencies ?? [])
.filter((dependency) => !dependency.unavailableReason) .filter((dependency) => !dependency.unavailableReason)
.map((dependency) => dependency.sync), .map((dependency) => dependency.sync)
.filter((state): state is typeof ownSync => Boolean(state)),
]; ];
const sync = const sourceWork =
states.find((state) => state.state === 'running') ?? syncStates.find((state) => state.state === 'running') ??
states.find((state) => state.state === 'failed') ?? syncStates.find((state) => state.state === 'failed') ??
ownSync; ownSync;
const sourceFreshness = input.fetchedAt const sourceFreshness = freshnessOf(input.fetchedAt, input.freshForMs);
? Date.now() - input.fetchedAt.valueOf() < input.freshForMs
? ('fresh' as const)
: ('stale' as const)
: ('unknown' as const);
const freshness = [ const freshness = [
sourceFreshness, sourceFreshness,
...(input.dependencies ?? []) ...(input.dependencies ?? [])
@@ -57,16 +50,75 @@ export function investmentResourceMeta(input: {
? { unavailableReason: input.unavailableReason } ? { unavailableReason: input.unavailableReason }
: {}), : {}),
dataRevision: input.dataRevision, dataRevision: input.dataRevision,
availability: input.availability, availability: availability(input.availability, input.work),
freshness: freshness.includes('unknown') completeness:
? ('unknown' as const) availability(input.availability, input.work) === 'available'
: freshness.includes('stale') ? ('complete' as const)
? ('stale' as const) : ('incomplete' as const),
: ('fresh' as const), freshness: combinedFreshness(freshness),
lastSuccessfulAt: timestamps[0] ?? null, lastSuccessfulAt: timestamps[0] ?? null,
sync, lastAttempt: input.lastAttempt ??
attempt(input.work),
...(input.coverage ? { coverage: input.coverage } : {}), ...(input.coverage ? { coverage: input.coverage } : {}),
...(input.dependencies ? { dependencies: input.dependencies } : {}), ...(input.dependencies ? { dependencies: input.dependencies } : {}),
}; };
return meta; return Object.defineProperty(meta, 'sync', {
enumerable: false,
value: sourceWork,
}) as InvestmentResourceMetaDto;
}
function sourceSync(work: Work) {
let state: 'failed' | 'idle' | 'running' = 'idle';
if (work?.state === 'running') state = 'running';
else if (work?.state === 'failed') state = 'failed';
return { id: work?.id ?? null, state, errorCode: work?.errorCode ?? null };
}
function freshnessOf(fetchedAt: Date | null, freshForMs: number) {
if (!fetchedAt) return 'unknown' as const;
return Date.now() - fetchedAt.valueOf() < freshForMs
? ('fresh' as const)
: ('stale' as const);
}
function combinedFreshness(values: InvestmentResourceMetaDto['freshness'][]) {
if (values.includes('unknown')) return 'unknown' as const;
if (values.includes('stale')) return 'stale' as const;
return 'fresh' as const;
}
function attempt(work: Work): Attempt {
const base = { startedAt: null, finishedAt: null };
if (work?.state === 'running') return { ...base, state: 'running', problems: [] };
if (work?.state === 'failed') {
return {
...base,
state: 'failed',
problems: work.errorCode
? [{ section: 'membership', accountId: null, errorCode: work.errorCode }]
: [],
};
}
return { ...base, state: 'none', problems: [] };
}
function availability(value: LegacyAvailability, work: Work) {
if (
value === 'available' ||
value === 'connection-unavailable' ||
value === 'initial-loading' ||
value === 'temporarily-unavailable'
) {
return value;
}
if (value === 'complete') return 'available' as const;
if (value === 'partial') {
return work?.state === 'running'
? ('initial-loading' as const)
: ('temporarily-unavailable' as const);
}
return work?.state === 'running'
? ('initial-loading' as const)
: ('temporarily-unavailable' as const);
} }
@@ -17,7 +17,7 @@ describe('screen dependency and history planning', () => {
}, },
], ],
}; };
it('prioritizes discovery, selected/comparisons, then history; Moscow dates are converted once', () => { it('prioritizes discovery, the selected period, then history; Moscow dates are converted once', () => {
const plan = planInvestmentScreen( const plan = planInvestmentScreen(
{ {
screen: 'income', screen: 'income',
@@ -32,6 +32,17 @@ describe('screen dependency and history planning', () => {
const selected = plan.sources.filter( const selected = plan.sources.filter(
(source) => source.kind === 'operations' && source.priority === 'screen', (source) => source.kind === 'operations' && source.priority === 'screen',
); );
// The single selected range is chunked into month-aligned intervals.
expect(selected).toHaveLength(2);
expect(
selected.every(
(source) =>
source.from != null &&
source.from.toISOString() >= '2024-01-31T21:00:00.000Z' &&
source.to != null &&
source.to.toISOString() <= '2024-02-29T21:00:00.000Z',
),
).toBe(true);
expect( expect(
selected.some( selected.some(
(source) => source.from?.toISOString() === '2024-01-31T21:00:00.000Z', (source) => source.from?.toISOString() === '2024-01-31T21:00:00.000Z',
@@ -42,9 +53,7 @@ describe('screen dependency and history planning', () => {
(source) => source.to?.toISOString() === '2024-02-29T21:00:00.000Z', (source) => source.to?.toISOString() === '2024-02-29T21:00:00.000Z',
), ),
).toBe(true); ).toBe(true);
expect(plan.unavailable).toEqual([ expect(plan.unavailable).toEqual([]);
{ resourceKey: 'income/yearAgo', reason: 'calendar_or_account_boundary' }, // gitleaks:allow public resource identifier
]);
const firstHistory = plan.sources.findIndex( const firstHistory = plan.sources.findIndex(
(source) => source.priority === 'history', (source) => source.priority === 'history',
); );
@@ -111,13 +111,6 @@ export function planInvestmentScreen(
priority: 'screen' as const, priority: 'screen' as const,
})), })),
); );
const knownOpening =
accounts.length && accounts.every((account) => account.openedOn)
? new Date(
// eslint-disable-next-line sonarjs/no-alphabetical-sort -- Code-unit ordering is intentional for canonical keys and ISO dates; locale collation would change persisted bindings.
`${accounts.map((account) => account.openedOn ?? '').sort()[0]}T00:00:00+03:00`,
)
: undefined;
if (!context.membershipKnown) return { sources, unavailable, now }; if (!context.membershipKnown) return { sources, unavailable, now };
if ( if (
['account', 'accounts', 'bond', 'events', 'stock'].includes(screen.screen) ['account', 'accounts', 'bond', 'events', 'stock'].includes(screen.screen)
@@ -142,23 +135,9 @@ export function planInvestmentScreen(
const period = planIncomePeriod( const period = planIncomePeriod(
screen.screen === 'income' ? screen.from : undefined, screen.screen === 'income' ? screen.from : undefined,
screen.screen === 'income' ? screen.to : undefined, screen.screen === 'income' ? screen.to : undefined,
knownOpening,
); );
for (const range of [ const bounds = dateOnlyBounds(period.selected, now);
period.selected, operations(bounds.from, bounds.to);
...(screen.screen === 'income'
? [period.previous, ...(period.yearAgo ? [period.yearAgo] : [])]
: []),
]) {
const bounds = dateOnlyBounds(range, now);
operations(bounds.from, bounds.to);
}
if (screen.screen === 'income' && !period.yearAgo) {
unavailable.push({
resourceKey: 'income/yearAgo', // gitleaks:allow public resource identifier
reason: 'calendar_or_account_boundary',
});
}
if (screen.screen === 'account') { if (screen.screen === 'account') {
const range = ledgerRange( const range = ledgerRange(
screen.from, screen.from,
@@ -209,6 +209,12 @@ export class BrokerageOperationsSyncService {
'tbank.operations.sync.persistence_failed', 'tbank.operations.sync.persistence_failed',
{ userId: attempt.userId, code: TbankErrorCode.Unavailable }, { userId: attempt.userId, code: TbankErrorCode.Unavailable },
); );
// The step could not even record its failure; close the attempt
// durably so it does not stay running. Best effort — an open row
// is re-reserved by the next request.
await this.repository
.fail(attempt, false)
.catch(() => undefined);
} }
const next = pending ? await this.repository.pending(attempt) : null; const next = pending ? await this.repository.pending(attempt) : null;
if (next) { if (next) {
@@ -17,6 +17,10 @@ import {
ledgerRange, ledgerRange,
planInvestmentScreen, planInvestmentScreen,
} from '../../investment-sync-plan'; } from '../../investment-sync-plan';
import {
hourlyAttempt,
TbankSyncRepository,
} from '../../tbank-sync/tbank-sync.repository';
import { OperationErrorCode } from '../domain/operation-error-code'; import { OperationErrorCode } from '../domain/operation-error-code';
import { import {
confirmedOperationRanges, confirmedOperationRanges,
@@ -59,6 +63,7 @@ export class BrokerageOperationsService {
private readonly sync: BrokerageOperationsSyncService, private readonly sync: BrokerageOperationsSyncService,
private readonly membership: TbankMembershipService, private readonly membership: TbankMembershipService,
private readonly config: ConfigService, private readonly config: ConfigService,
private readonly hourlySync: TbankSyncRepository,
) {} ) {}
async read( async read(
@@ -135,52 +140,17 @@ export class BrokerageOperationsService {
? [{ accountId: source.accountId, from: source.from, to: source.to }] ? [{ accountId: source.accountId, from: source.from, to: source.to }]
: [], : [],
); );
if (query.sync !== 'observe') { // The ledger only starts the first-time load of missing intervals; the
const discovery = await this.membership.request( // mutable tail belongs to the hourly walk and membership is not rechecked.
userId, if (intervals.length) {
connection.revision, await this.sync.readPlan(userId, intervals, {
); movingEnd,
if (discovery) { history: true,
void discovery.done revision: connection.revision,
.then(async () => { });
const current = await this.access.local(userId, accountId);
if (current.revision !== connection.revision) return;
const currentRange = ledgerRange(
query.from,
query.to,
query.instrumentUid,
current.accounts.filter(
(account) =>
accountId === 'all' || account.accountId === accountId,
),
now,
);
const targets = current.accounts
.filter(
(account) =>
accountId === 'all' || account.accountId === accountId,
)
.flatMap((account) =>
currentRange.from < currentRange.to
? [{ accountId: account.accountId, ...currentRange }]
: [],
);
await this.sync.readPlan(userId, targets, {
movingEnd,
history: true,
revision: connection.revision,
});
})
.catch(() => undefined);
} else if (intervals.length) {
await this.sync.readPlan(userId, intervals, {
movingEnd,
history: true,
revision: connection.revision,
});
}
} }
const saved = await this.repository.readPage(readInput); const saved = await this.repository.readPage(readInput);
const hourly = await this.hourlySync.hourly(userId);
const accounts = saved?.connection.accounts ?? []; const accounts = saved?.connection.accounts ?? [];
const targets = accounts.flatMap((account) => const targets = accounts.flatMap((account) =>
from < to from < to
@@ -252,26 +222,10 @@ export class BrokerageOperationsService {
dataRevision: value.dataRevision, dataRevision: value.dataRevision,
availability: 'complete', availability: 'complete',
fetchedAt: value.fetchedAt, fetchedAt: value.fetchedAt,
freshForMs: tbankSourceTtlMs.metadata, freshForMs: tbankSourceTtlMs.portfolio,
work: null, work: null,
}), }),
); );
if (query.sync !== 'observe') {
const identities = new Map(
rows.map((row) => {
const identity = row.payload as TbankOperationItem;
return [
metadataKey(identity),
{ instrumentUid: identity.instrumentUid, figi: identity.figi },
] as const;
}),
);
for (const identity of identities.values()) {
if (!instruments.get(metadataKey(identity))?.fresh) {
await this.metadata.request(userId, connection.revision, identity);
}
}
}
const membershipMeta = investmentResourceMeta({ const membershipMeta = investmentResourceMeta({
resourceKey: 'accounts/membership', resourceKey: 'accounts/membership',
dataRevision: String(saved?.connection.membershipRevision ?? MISSING_MEMBERSHIP_REVISION), dataRevision: String(saved?.connection.membershipRevision ?? MISSING_MEMBERSHIP_REVISION),
@@ -295,6 +249,7 @@ export class BrokerageOperationsService {
? STALE_FRESHNESS_MS ? STALE_FRESHNESS_MS
: Number.POSITIVE_INFINITY, : Number.POSITIVE_INFINITY,
work: work ?? saved?.membershipWork ?? null, work: work ?? saved?.membershipWork ?? null,
lastAttempt: hourlyAttempt(hourly, 'operations', accountId === 'all' ? undefined : accountId),
dependencies: [membershipMeta, ...metadataDependencies], dependencies: [membershipMeta, ...metadataDependencies],
coverage: { coverage: {
requested: targets.map(serialize), requested: targets.map(serialize),
@@ -346,7 +301,7 @@ export class BrokerageOperationsService {
const hasNext = Boolean(saved && saved.rows.length > limit); const hasNext = Boolean(saved && saved.rows.length > limit);
return { return {
data: data:
meta.availability === 'missing' meta.completeness === 'incomplete'
? null ? null
: { : {
accountId, accountId,
@@ -480,6 +435,7 @@ export class BrokerageOperationsService {
saved = await this.repository.readPage(input); saved = await this.repository.readPage(input);
} }
const now = new Date(); const now = new Date();
const hourly = await this.hourlySync.hourly(userId);
const membership = investmentResourceMeta({ const membership = investmentResourceMeta({
resourceKey: 'accounts/membership', resourceKey: 'accounts/membership',
dataRevision: String(saved?.connection.membershipRevision ?? MISSING_MEMBERSHIP_REVISION), dataRevision: String(saved?.connection.membershipRevision ?? MISSING_MEMBERSHIP_REVISION),
@@ -496,7 +452,11 @@ export class BrokerageOperationsService {
key: `${period.from}/${period.to}`, key: `${period.from}/${period.to}`,
movingEnd: false, movingEnd: false,
})); }));
return periods.map((period, index) => { const rows = saved?.rows ?? [];
const instruments = await this.metadata.savedMany(
rows.map((row) => row.payload as TbankOperationItem),
);
return periods.map((period) => {
const targets = (saved?.connection.accounts ?? []).map((account) => ({ const targets = (saved?.connection.accounts ?? []).map((account) => ({
accountId: account.accountId, accountId: account.accountId,
...period, ...period,
@@ -532,8 +492,7 @@ export class BrokerageOperationsService {
null; null;
const confirmed = confirmedOperationRanges(plan.intervals, sources); const confirmed = confirmedOperationRanges(plan.intervals, sources);
const meta = investmentResourceMeta({ const meta = investmentResourceMeta({
// eslint-disable-next-line security/detect-object-injection -- The numeric index enumerates the three comparison periods. resourceKey: `income/${accountId}/selected/${period.key}`,
resourceKey: `income/${accountId}/${['selected', 'previous', 'yearAgo'][index]}/${period.key}`,
dataRevision: saved?.binding ?? 'missing', dataRevision: saved?.binding ?? 'missing',
availability: !known availability: !known
? 'missing' ? 'missing'
@@ -545,6 +504,7 @@ export class BrokerageOperationsService {
fetchedAt, fetchedAt,
freshForMs: plan.intervals.some((range) => range.stale) ? STALE_FRESHNESS_MS : Infinity, freshForMs: plan.intervals.some((range) => range.stale) ? STALE_FRESHNESS_MS : Infinity,
work, work,
lastAttempt: hourlyAttempt(hourly, 'operations', accountId === 'all' ? undefined : accountId),
dependencies: [membership], dependencies: [membership],
coverage: { coverage: {
requested: targets.map(serialize), requested: targets.map(serialize),
@@ -587,7 +547,7 @@ export class BrokerageOperationsService {
return { return {
meta, meta,
items: complete items: complete
? (saved?.rows ?? []) ? rows
.filter((row) => .filter((row) =>
plan.intervals.some( plan.intervals.some(
(interval) => (interval) =>
@@ -601,6 +561,10 @@ export class BrokerageOperationsService {
row.payload as TbankOperationItem, row.payload as TbankOperationItem,
row.occurredAt, row.occurredAt,
), ),
logoName:
instruments
.get(metadataKey(row.payload as TbankOperationItem))
?.data?.brand?.logoName?.trim() || null,
...(accountId === 'all' ...(accountId === 'all'
? { ? {
account: { account: {
@@ -71,7 +71,7 @@ describe('operation completeness and freshness', () => {
const asOf = new Date('2026-09-25T12:00:00Z'); const asOf = new Date('2026-09-25T12:00:00Z');
const covered = { accountId: 'a', from, to: asOf, asOf, fetchedAt: asOf }; const covered = { accountId: 'a', from, to: asOf, asOf, fetchedAt: asOf };
it.each([1, 299, 300])( it.each([1, 3599, 3600])(
'preserves the observed complete end after %s seconds, separately from TTL', 'preserves the observed complete end after %s seconds, separately from TTL',
(seconds) => { (seconds) => {
const now = new Date(asOf.valueOf() + seconds * 1000); const now = new Date(asOf.valueOf() + seconds * 1000);
@@ -85,10 +85,10 @@ describe('operation completeness and freshness', () => {
availability: 'complete', availability: 'complete',
asOf, asOf,
to: asOf, to: asOf,
stale: seconds >= 300, stale: seconds >= 3600,
}); });
expect(plan.load).toEqual( expect(plan.load).toEqual(
seconds >= 300 ? [{ accountId: 'a', from, to: now }] : [], [],
); );
}, },
); );
@@ -128,7 +128,7 @@ export type OperationCoverage = OperationInterval & {
asOf: Date; asOf: Date;
fetchedAt: Date; fetchedAt: Date;
}; };
const operationsFreshForMs = 300_000; const operationsFreshForMs = 3_600_000;
export function planOperationRead( export function planOperationRead(
requested: OperationInterval[], requested: OperationInterval[],
@@ -188,8 +188,9 @@ export function planOperationRead(
.filter((row) => row.from < row.to), .filter((row) => row.from < row.to),
fresh.map((row) => ({ ...row, instrumentUid: target.instrumentUid })), fresh.map((row) => ({ ...row, instrumentUid: target.instrumentUid })),
).length > EMPTY_INTERVAL_COUNT; ).length > EMPTY_INTERVAL_COUNT;
// The mutable tail is owned by the hourly walk; reads only load what is
// missing, never re-verify complete or fresh intervals.
if (options.force) load.push(target); if (options.force) load.push(target);
else if (stale) load.push({ ...target, from: monthStart, to: now });
else load.push(...missing); else load.push(...missing);
// Completeness is bounded by the observed end, never by the new wall clock. // Completeness is bounded by the observed end, never by the new wall clock.
return { return {
@@ -2,13 +2,14 @@ import { Module } from '@nestjs/common';
import { TbankModule } from '../../../integrations/tbank/tbank.module'; import { TbankModule } from '../../../integrations/tbank/tbank.module';
import { AuthModule } from '../../auth/auth.module'; import { AuthModule } from '../../auth/auth.module';
import { TbankSyncRepositoryModule } from '../tbank-sync/tbank-sync-repository.module';
import { BrokerageOperationsSyncService } from './application/brokerage-operations-sync.service'; import { BrokerageOperationsSyncService } from './application/brokerage-operations-sync.service';
import { BrokerageOperationsService } from './application/brokerage-operations.service'; import { BrokerageOperationsService } from './application/brokerage-operations.service';
import { BrokerageOperationsRepository } from './infrastructure/brokerage-operations.repository'; import { BrokerageOperationsRepository } from './infrastructure/brokerage-operations.repository';
import { BrokerageOperationsController } from './presentation/brokerage-operations.controller'; import { BrokerageOperationsController } from './presentation/brokerage-operations.controller';
@Module({ @Module({
imports: [AuthModule, TbankModule], imports: [AuthModule, TbankModule, TbankSyncRepositoryModule],
controllers: [BrokerageOperationsController], controllers: [BrokerageOperationsController],
providers: [ providers: [
BrokerageOperationsService, BrokerageOperationsService,
@@ -15,10 +15,7 @@ import {
import { MoneyDto } from '../../../../common/money.dto'; import { MoneyDto } from '../../../../common/money.dto';
import { AccountReferenceDto } from '../../account-reference.dto'; import { AccountReferenceDto } from '../../account-reference.dto';
import { import { InvestmentResourceMetaDto } from '../../investment-resource.dto';
InvestmentReadQueryDto,
InvestmentResourceMetaDto,
} from '../../investment-resource.dto';
const MIN_OPERATIONS_PAGE_SIZE = 1; const MIN_OPERATIONS_PAGE_SIZE = 1;
@@ -36,7 +33,7 @@ export const operationCategories = [
] as const; ] as const;
export type OperationCategory = (typeof operationCategories)[number]; export type OperationCategory = (typeof operationCategories)[number];
export class BrokerageOperationsQueryDto extends InvestmentReadQueryDto { export class BrokerageOperationsQueryDto {
@ApiPropertyOptional({ @ApiPropertyOptional({
format: 'uuid', format: 'uuid',
description: description:
@@ -2,6 +2,7 @@ import { Injectable } from '@nestjs/common';
import type { InvestmentResourceMetaDto } from '../../investment-resource.dto'; import type { InvestmentResourceMetaDto } from '../../investment-resource.dto';
import { add, decimalFromString, formatDecimal } from '../../../../common/decimal';
import { investmentResourceMeta } from '../../investment-resource.mapper'; import { investmentResourceMeta } from '../../investment-resource.mapper';
import { import {
ForecastRepository, ForecastRepository,
@@ -16,6 +17,7 @@ export class ForecastService {
async result<T extends object>(input: { async result<T extends object>(input: {
candidate?: T; candidate?: T;
lastAttempt?: InvestmentResourceMetaDto['lastAttempt'];
membershipRevision: number; membershipRevision: number;
portfolio: InvestmentResourceMetaDto; portfolio: InvestmentResourceMetaDto;
revision: string; revision: string;
@@ -72,22 +74,51 @@ export class ForecastService {
.map((dependency) => dependency.sync) .map((dependency) => dependency.sync)
.find((sync) => sync.state === 'failed'); .find((sync) => sync.state === 'failed');
return { return {
data: stored ? (stored.payload as T) : null, data: stored ? filterEvents(stored.payload as T, input.scopeKey) : null,
meta: investmentResourceMeta({ meta: investmentResourceMeta({
resourceKey: input.scopeKey, resourceKey: input.scopeKey,
dataRevision: String(stored?.dataRevision ?? MISSING_DATA_REVISION), dataRevision: String(stored?.dataRevision ?? MISSING_DATA_REVISION),
availability: stored availability: stored
? 'complete' ? 'complete'
: dependencies.some( : dependencies.some(
(dependency) => dependency.availability !== 'missing', (dependency) => dependency.completeness === 'complete',
) )
? 'partial' ? 'partial'
: 'missing', : 'missing',
fetchedAt: first ? new Date(first) : null, fetchedAt: first ? new Date(first) : null,
freshForMs: Number.POSITIVE_INFINITY, freshForMs: Number.POSITIVE_INFINITY,
work: work?.id ? { ...work, id: work.id } : null, work: work?.id ? { ...work, id: work.id } : null,
lastAttempt: input.lastAttempt,
dependencies, dependencies,
}), }),
}; };
} }
} }
function filterEvents<T extends object>(payload: T, scopeKey: string): T {
if (!scopeKey.startsWith('events/')) return payload;
try {
const [, from, to] = JSON.parse(scopeKey.slice('events/'.length)) as [string, string, string];
const value = payload as T & {
events?: { date: string; projectedAmount?: { currency: string; value: string }; type: string }[];
incomeTotalsByCurrency?: { currency: string; value: string }[];
};
if (!value.events || !value.incomeTotalsByCurrency) return payload;
const events = value.events.filter((event) => event.date >= from && event.date <= to);
const totals = new Map<string, ReturnType<typeof decimalFromString>>();
for (const event of events) {
if (!['coupon', 'dividend'].includes(event.type) || !event.projectedAmount) continue;
const current = totals.get(event.projectedAmount.currency) ?? { value: 0n, scale: 0 };
totals.set(event.projectedAmount.currency, add(current, decimalFromString(event.projectedAmount.value)));
}
return {
...value,
events,
incomeTotalsByCurrency: [...totals]
.sort(([left], [right]) => left.localeCompare(right))
.map(([currency, amount]) => ({ currency, value: formatDecimal(amount) })),
};
} catch {
return payload;
}
}
@@ -26,6 +26,7 @@ describe('persistent portfolio projection', () => {
repository as never, repository as never,
{ request: jest.fn().mockResolvedValue(null) } as never, { request: jest.fn().mockResolvedValue(null) } as never,
{ event: jest.fn() } as never, { event: jest.fn() } as never,
{ hourly: jest.fn().mockResolvedValue(null) } as never,
); );
access.local.mockResolvedValue({ access.local.mockResolvedValue({
revision: 'private', revision: 'private',
@@ -40,6 +41,7 @@ describe('persistent portfolio projection', () => {
accounts: [], accounts: [],
work: [], work: [],
}); });
repository.publish.mockResolvedValue(true);
metadata.savedMany.mockResolvedValue(new Map()); metadata.savedMany.mockResolvedValue(new Map());
metadata.request.mockResolvedValue(null); metadata.request.mockResolvedValue(null);
}); });
@@ -95,7 +97,7 @@ describe('persistent portfolio projection', () => {
], ],
work: [], work: [],
}); });
await service.read('user', 'a'); await service.request('user', 'private', 'a');
for ( for (
let index = 0; let index = 0;
index < 10 && !repository.publish.mock.calls.length; index < 10 && !repository.publish.mock.calls.length;
@@ -189,10 +191,11 @@ describe('persistent portfolio projection', () => {
accounts: [account, { ...account, accountId: 'b', portfolio: null }], accounts: [account, { ...account, accountId: 'b', portfolio: null }],
work: [], work: [],
}); });
const partial = await service.read('user', 'all', true); const partial = await service.read('user', 'all');
expect(partial.data).toBeNull(); expect(partial.data).toBeNull();
expect(partial.meta.availability).toBe('partial'); expect(partial.meta.availability).toBe('temporarily-unavailable');
const saved = await service.saved('user', 'all', true); expect(partial.meta.completeness).toBe('incomplete');
const saved = await service.saved('user', 'all');
expect(saved.accounts[0].snapshot?.totalsByCurrency).toEqual( expect(saved.accounts[0].snapshot?.totalsByCurrency).toEqual(
snapshot.totalsByCurrency, snapshot.totalsByCurrency,
); );
@@ -210,7 +213,7 @@ describe('persistent portfolio projection', () => {
], ],
work: [], work: [],
}); });
const complete = await service.read('user', 'all', true); const complete = await service.read('user', 'all');
expect(complete.data?.totalsByCurrency).toEqual([ expect(complete.data?.totalsByCurrency).toEqual([
{ currency: 'RUB', value: '2.000000002' }, { currency: 'RUB', value: '2.000000002' },
]); ]);
@@ -8,7 +8,6 @@ import { StructuredLogger } from '../../../../common/logging/structured-logger.s
import { isTbankErrorCode, TbankErrorCode } from '../../../../integrations/tbank/tbank-error-code'; import { isTbankErrorCode, TbankErrorCode } from '../../../../integrations/tbank/tbank-error-code';
import { import {
InstrumentMetadataService, InstrumentMetadataService,
type MetadataIdentity,
metadataKey, metadataKey,
TbankAccessService, TbankAccessService,
TbankClient, TbankClient,
@@ -18,6 +17,10 @@ import {
tbankSourceTtlMs, tbankSourceTtlMs,
} from '../../../../integrations/tbank/tbank.api'; } from '../../../../integrations/tbank/tbank.api';
import { investmentResourceMeta } from '../../investment-resource.mapper'; import { investmentResourceMeta } from '../../investment-resource.mapper';
import {
hourlyAttempt,
TbankSyncRepository,
} from '../../tbank-sync/tbank-sync.repository';
import { portfolioStructure } from '../domain/portfolio-structure'; import { portfolioStructure } from '../domain/portfolio-structure';
import { sumMoneyByCurrency } from '../domain/portfolio-totals'; import { sumMoneyByCurrency } from '../domain/portfolio-totals';
import { import {
@@ -29,6 +32,7 @@ import { mapPortfolio } from '../infrastructure/tbank-portfolio.mapper';
import { PortfolioDto } from '../presentation/portfolio.dto'; import { PortfolioDto } from '../presentation/portfolio.dto';
const MISSING_DATA_REVISION = 0; const MISSING_DATA_REVISION = 0;
const NO_ACCOUNTS = 0;
const membershipFreshForMs = tbankSourceTtlMs.membership; const membershipFreshForMs = tbankSourceTtlMs.membership;
@@ -45,50 +49,14 @@ export class PortfolioService {
private readonly repository: PortfolioRepository, private readonly repository: PortfolioRepository,
private readonly membership: TbankMembershipService, private readonly membership: TbankMembershipService,
private readonly logger: StructuredLogger, private readonly logger: StructuredLogger,
private readonly sync: TbankSyncRepository,
) {} ) {}
async saved(userId: string, accountId: string, observe = false) { // Local reads never start upstream work; the hourly walk owns refreshes.
async saved(userId: string, accountId: string) {
const connection = await this.access.local(userId, accountId); const connection = await this.access.local(userId, accountId);
if (!observe) {
const discovery = await this.membership.request(
userId,
connection.revision,
);
if (discovery) {
void discovery.done
.then(async () => {
const current = await this.access.local(userId, accountId);
if (current.revision !== connection.revision) return;
for (const account of current.accounts.filter(
(account) =>
accountId === 'all' || account.accountId === accountId,
)) {
await this.queueSnapshot(
userId,
current.revision,
account.accountId,
);
}
})
.catch(() =>
this.logger.event(
'warn',
'tbank.portfolio.discovery_not_available',
{ userId },
),
);
}
for (const account of (discovery ? [] : connection.accounts).filter(
(account) => accountId === 'all' || account.accountId === accountId,
)) {
await this.queueSnapshot(
userId,
connection.revision,
account.accountId,
);
}
}
const saved = await this.repository.read(userId, connection.revision); const saved = await this.repository.read(userId, connection.revision);
const hourly = await this.sync.hourly(userId);
if ( if (
saved?.membershipAt && saved?.membershipAt &&
accountId !== 'all' && accountId !== 'all' &&
@@ -96,26 +64,6 @@ export class PortfolioService {
) { ) {
throw new NotFoundException(); throw new NotFoundException();
} }
if (!observe) {
const identities = new Map<string, MetadataIdentity>();
for (const account of (saved?.accounts ?? []).filter(
(account) => accountId === 'all' || account.accountId === accountId,
)) {
const snapshot = account.portfolio?.payload as unknown as
PortfolioDto | undefined;
for (const position of snapshot?.positions ?? []) {
identities.set(metadataKey(position), position);
}
}
const savedMetadata = await this.metadata.savedMany([
...identities.values(),
]);
for (const identity of identities.values()) {
if (!savedMetadata.get(metadataKey(identity))?.fresh) {
await this.metadata.request(userId, connection.revision, identity);
}
}
}
const membershipMeta = investmentResourceMeta({ const membershipMeta = investmentResourceMeta({
resourceKey: 'accounts/membership', resourceKey: 'accounts/membership',
dataRevision: String(saved?.membershipRevision ?? MISSING_DATA_REVISION), dataRevision: String(saved?.membershipRevision ?? MISSING_DATA_REVISION),
@@ -123,8 +71,10 @@ export class PortfolioService {
fetchedAt: saved?.membershipAt ?? null, fetchedAt: saved?.membershipAt ?? null,
freshForMs: membershipFreshForMs, freshForMs: membershipFreshForMs,
work: saved?.work.find((work) => work.scopeKey === 'membership') ?? null, work: saved?.work.find((work) => work.scopeKey === 'membership') ?? null,
lastAttempt: hourlyAttempt(hourly, 'membership'),
}); });
return { return {
hourly,
membership: membershipMeta, membership: membershipMeta,
accounts: (saved?.accounts ?? []) accounts: (saved?.accounts ?? [])
.filter( .filter(
@@ -154,16 +104,21 @@ export class PortfolioService {
saved?.work.find( saved?.work.find(
(work) => work.scopeKey === portfolioWorkKey(account.accountId), (work) => work.scopeKey === portfolioWorkKey(account.accountId),
) ?? null, ) ?? null,
lastAttempt: hourlyAttempt(hourly, 'portfolio', account.accountId),
}), }),
})), })),
}; };
} }
async read(userId: string, accountId: string, observe = false) { async read(userId: string, accountId: string) {
const saved = await this.saved(userId, accountId, observe); const { hourly, ...saved } = await this.saved(userId, accountId);
const missing = const missing =
saved.membership.availability === 'missing' || saved.membership.completeness === 'incomplete' ||
saved.accounts.some((account) => !account.snapshot); (saved.accounts.length > NO_ACCOUNTS && saved.accounts.some((account) => !account.snapshot));
const available =
!missing &&
(saved.membership.availability === 'available' ||
saved.accounts.some((account) => account.snapshot));
const metadata = await this.metadata.savedMany( const metadata = await this.metadata.savedMany(
saved.accounts.flatMap((account) => account.snapshot?.positions ?? []), saved.accounts.flatMap((account) => account.snapshot?.positions ?? []),
); );
@@ -176,7 +131,7 @@ export class PortfolioService {
dataRevision: source.dataRevision, dataRevision: source.dataRevision,
availability: 'complete', availability: 'complete',
fetchedAt: source.fetchedAt, fetchedAt: source.fetchedAt,
freshForMs: tbankSourceTtlMs.metadata, freshForMs: portfolioFreshForMs,
work: null, work: null,
}), }),
] ]
@@ -212,6 +167,11 @@ export class PortfolioService {
] ]
: [], : [],
); );
let availability: 'available' | 'complete' | 'missing' | 'partial';
if (available) availability = 'available';
else if (!missing) availability = 'complete';
else if (portfolios.length > NO_ACCOUNTS) availability = 'partial';
else availability = 'missing';
const lastSuccessfulAt = const lastSuccessfulAt =
// eslint-disable-next-line sonarjs/no-alphabetical-sort -- Code-unit ordering is intentional for canonical keys and ISO dates; locale collation would change persisted bindings. // eslint-disable-next-line sonarjs/no-alphabetical-sort -- Code-unit ordering is intentional for canonical keys and ISO dates; locale collation would change persisted bindings.
portfolios.map((portfolio) => portfolio.lastSuccessfulAt).sort()[0] ?? portfolios.map((portfolio) => portfolio.lastSuccessfulAt).sort()[0] ??
@@ -255,16 +215,13 @@ export class PortfolioService {
]), ]),
metadataDependencies.map((dependency) => dependency.dataRevision), metadataDependencies.map((dependency) => dependency.dataRevision),
]), ]),
availability: missing availability,
? portfolios.length
? 'partial'
: 'missing'
: 'complete',
fetchedAt: lastSuccessfulAt ? new Date(lastSuccessfulAt) : null, fetchedAt: lastSuccessfulAt ? new Date(lastSuccessfulAt) : null,
freshForMs: portfolios.length freshForMs: portfolios.length
? portfolioFreshForMs ? portfolioFreshForMs
: membershipFreshForMs, : membershipFreshForMs,
work: work.id ? { ...work, id: work.id } : null, work: work.id ? { ...work, id: work.id } : null,
lastAttempt: hourlyAttempt(hourly, 'portfolio', accountId === 'all' ? undefined : accountId),
dependencies: [ dependencies: [
saved.membership, saved.membership,
...saved.accounts.map((account) => account.meta), ...saved.accounts.map((account) => account.meta),
@@ -51,11 +51,32 @@ export class ForecastRepository {
const previous = await tx.tbankForecast.findUnique({ const previous = await tx.tbankForecast.findUnique({
where: { userId_revision_scopeKey: key }, where: { userId_revision_scopeKey: key },
}); });
const requestedWindow = eventsWindow(input.scopeKey);
const saved = previous ?? (requestedWindow
? (
await tx.tbankForecast.findMany({
where: {
...owner,
scopeKey: { startsWith: 'events/' },
membershipRevision: input.membershipRevision,
},
})
).find((forecast) => {
const window = eventsWindow(forecast.scopeKey);
return (
window &&
window.accountId === requestedWindow.accountId &&
window.from <= requestedWindow.to &&
requestedWindow.from <= window.to
);
})
: null);
const fallback = const fallback =
previous?.membershipRevision === input.membershipRevision saved?.membershipRevision === input.membershipRevision
? previous ? saved
: null; : null;
if (!input.candidate) return fallback; if (!input.candidate) return fallback;
if (!previous && fallback) return fallback;
const portfolios = await tx.tbankPortfolio.findMany({ const portfolios = await tx.tbankPortfolio.findMany({
where: { where: {
...owner, ...owner,
@@ -154,3 +175,13 @@ export class ForecastRepository {
}); });
} }
} }
function eventsWindow(scopeKey: string) {
if (!scopeKey.startsWith('events/')) return null;
try {
const [accountId, from, to] = JSON.parse(scopeKey.slice('events/'.length)) as [string, string, string];
return { accountId, from, to };
} catch {
return null;
}
}
@@ -69,17 +69,23 @@ export class PortfolioRepository {
const snapshot = await tx.tbankPortfolio.findUnique({ const snapshot = await tx.tbankPortfolio.findUnique({
where: { userId_revision_accountId: { userId, revision, accountId } }, where: { userId_revision_accountId: { userId, revision, accountId } },
}); });
const portfolioWorkScope = {
userId,
revision,
scopeKey: portfolioWorkKey(accountId),
};
const previous = await tx.tbankSyncWork.findUnique({
where: { userId_revision_scopeKey: portfolioWorkScope },
});
if ( if (
!force && !force &&
snapshot && snapshot &&
!tbankSourceNeedsRefresh('portfolio', snapshot.fetchedAt, new Date()) !tbankSourceNeedsRefresh('portfolio', snapshot.fetchedAt, new Date()) &&
previous?.state !== 'failed'
) { ) {
return null; return null;
} }
const scope = { userId, revision, scopeKey: portfolioWorkKey(accountId) }; const scope = portfolioWorkScope;
const previous = await tx.tbankSyncWork.findUnique({
where: { userId_revision_scopeKey: scope },
});
if ( if (
previous?.state === 'running' && previous?.state === 'running' &&
previous.processRunId === processRunId && previous.processRunId === processRunId &&
@@ -139,6 +145,41 @@ export class PortfolioRepository {
create: { ...scope, ...data }, create: { ...scope, ...data },
update: { ...data, dataRevision: { increment: 1 } }, update: { ...data, dataRevision: { increment: 1 } },
}); });
// Value history points require a fully valued snapshot; the attempt
// identity deduplicates repeated or late publications of one attempt.
if (
snapshot.totalsByCurrency.length > Number.MIN_VALUE &&
snapshot.positions.every((position) => position.currentValue !== null)
) {
await tx.tbankPortfolioValueHistory.upsert({
where: {
userId_accountId: { userId: attempt.userId, accountId },
},
create: { userId: attempt.userId, accountId },
update: { deletedAt: null },
});
for (const { currency, value } of snapshot.totalsByCurrency) {
await tx.tbankPortfolioValuePoint.upsert({
where: {
userId_accountId_attemptId_currency: {
userId: attempt.userId,
accountId,
attemptId: attempt.attemptId,
currency,
},
},
create: {
userId: attempt.userId,
accountId,
attemptId: attempt.attemptId,
currency,
value,
capturedAt: data.fetchedAt,
},
update: {},
});
}
}
return true; return true;
}); });
} }
@@ -2,6 +2,7 @@ import { Module } from '@nestjs/common';
import { TbankModule } from '../../../integrations/tbank/tbank.module'; import { TbankModule } from '../../../integrations/tbank/tbank.module';
import { AuthModule } from '../../auth/auth.module'; import { AuthModule } from '../../auth/auth.module';
import { TbankSyncRepositoryModule } from '../tbank-sync/tbank-sync-repository.module';
import { ForecastService } from './application/forecast.service'; import { ForecastService } from './application/forecast.service';
import { PortfolioService } from './application/portfolio.service'; import { PortfolioService } from './application/portfolio.service';
import { ForecastRepository } from './infrastructure/forecast.repository'; import { ForecastRepository } from './infrastructure/forecast.repository';
@@ -9,7 +10,7 @@ import { PortfolioRepository } from './infrastructure/portfolio.repository';
import { PortfolioController } from './presentation/portfolio.controller'; import { PortfolioController } from './presentation/portfolio.controller';
@Module({ @Module({
imports: [AuthModule, TbankModule], imports: [AuthModule, TbankModule, TbankSyncRepositoryModule],
controllers: [PortfolioController], controllers: [PortfolioController],
providers: [ providers: [
PortfolioService, PortfolioService,
@@ -1,11 +1,10 @@
import type { Request } from 'express'; import type { Request } from 'express';
import { Controller, Get, Param, Query, Req, UseGuards } from '@nestjs/common'; import { Controller, Get, Param, Req, UseGuards } from '@nestjs/common';
import { ApiBearerAuth, ApiOkResponse, ApiTags } from '@nestjs/swagger'; import { ApiBearerAuth, ApiOkResponse, ApiTags } from '@nestjs/swagger';
import { ApiProblemResponses } from '../../../../common/decorators/api-safe-error-response.decorator'; import { ApiProblemResponses } from '../../../../common/decorators/api-safe-error-response.decorator';
import { AccessPayload, JwtAuthGuard } from '../../../auth/auth.api'; import { AccessPayload, JwtAuthGuard } from '../../../auth/auth.api';
import { InvestmentReadQueryDto } from '../../investment-resource.dto';
import { PortfolioService } from '../application/portfolio.service'; import { PortfolioService } from '../application/portfolio.service';
import { PortfolioResourceDto } from './portfolio.dto'; import { PortfolioResourceDto } from './portfolio.dto';
@@ -19,15 +18,8 @@ export class PortfolioController {
@Get('all/portfolio') @Get('all/portfolio')
@ApiOkResponse({ type: PortfolioResourceDto }) @ApiOkResponse({ type: PortfolioResourceDto })
all( all(@Req() request: Request & { user: AccessPayload }) {
@Req() request: Request & { user: AccessPayload }, return this.portfolio.read(request.user.sub, 'all');
@Query() query: InvestmentReadQueryDto,
) {
return this.portfolio.read(
request.user.sub,
'all',
query.sync === 'observe',
);
} }
@Get(':accountId/portfolio') @Get(':accountId/portfolio')
@@ -35,12 +27,7 @@ export class PortfolioController {
get( get(
@Req() request: Request & { user: AccessPayload }, @Req() request: Request & { user: AccessPayload },
@Param('accountId') accountId: string, @Param('accountId') accountId: string,
@Query() query: InvestmentReadQueryDto,
) { ) {
return this.portfolio.read( return this.portfolio.read(request.user.sub, accountId);
request.user.sub,
accountId,
query.sync === 'observe',
);
} }
} }
@@ -2,11 +2,12 @@ import { Module } from '@nestjs/common';
import { TbankModule } from '../../../integrations/tbank/tbank.module'; import { TbankModule } from '../../../integrations/tbank/tbank.module';
import { AuthModule } from '../../auth/auth.module'; import { AuthModule } from '../../auth/auth.module';
import { TbankSyncModule } from '../tbank-sync/tbank-sync.module';
import { TbankConnectionController } from './tbank-connection.controller'; import { TbankConnectionController } from './tbank-connection.controller';
import { TbankConnectionService } from './tbank-connection.service'; import { TbankConnectionService } from './tbank-connection.service';
@Module({ @Module({
imports: [AuthModule, TbankModule], imports: [AuthModule, TbankModule, TbankSyncModule],
controllers: [TbankConnectionController], controllers: [TbankConnectionController],
providers: [TbankConnectionService], providers: [TbankConnectionService],
}) })
@@ -35,11 +35,22 @@ export class TbankConnectionService {
this.tbank.validateToken(token), this.tbank.validateToken(token),
); );
const encryptedToken = this.tokens.encrypt(token); const encryptedToken = this.tokens.encrypt(token);
const confirmed = supportedTbankAccounts(accounts).map(({ id }) => id);
const connection = await this.prisma.$transaction(async (tx) => { const connection = await this.prisma.$transaction(async (tx) => {
// Cascade old private data and attempts in the same transaction that // Cascade old private data and attempts in the same transaction that
// publishes the validated membership and the new connection revision. // publishes the validated membership and the new connection revision.
await tx.tbankConnection.deleteMany({ where: { userId } }); await tx.tbankConnection.deleteMany({ where: { userId } });
const now = new Date(); const now = new Date();
// Value history survives token replacement but is visible only for
// accounts confirmed by the new token.
await tx.tbankPortfolioValueHistory.updateMany({
where: { userId, accountId: { in: confirmed } },
data: { deletedAt: null },
});
await tx.tbankPortfolioValueHistory.updateMany({
where: { userId, accountId: { notIn: confirmed }, deletedAt: null },
data: { deletedAt: now },
});
return tx.tbankConnection.create({ return tx.tbankConnection.create({
data: { data: {
userId, userId,
@@ -66,6 +77,14 @@ export class TbankConnectionService {
} }
async delete(userId: string): Promise<void> { async delete(userId: string): Promise<void> {
await this.prisma.tbankConnection.deleteMany({ where: { userId } }); await this.prisma.$transaction(async (tx) => {
await tx.tbankConnection.deleteMany({ where: { userId } });
// History stays stored but hidden until a new connection reconfirms
// the same account.
await tx.tbankPortfolioValueHistory.updateMany({
where: { userId, deletedAt: null },
data: { deletedAt: new Date() },
});
});
} }
} }
@@ -0,0 +1,415 @@
import { Injectable, OnModuleDestroy, OnModuleInit } from '@nestjs/common';
import { randomUUID } from 'node:crypto';
import { StructuredLogger } from '../../../common/logging/structured-logger.service';
import { TbankErrorCode } from '../../../integrations/tbank/tbank-error-code';
import {
TbankAccessService,
TbankMembershipService,
type TbankSyncAttempt,
} from '../../../integrations/tbank/tbank.api';
import { PrismaService } from '../../../prisma/prisma.service';
import { BrokerageOperationsSyncService } from '../operations/operations.api';
import { PortfolioService } from '../portfolio/portfolio.api';
import { UpcomingEventsService } from '../upcoming-events/upcoming-events.service';
import {
HOURLY_CAUSE_INTERRUPTED,
type HourlyCause,
type HourlyChild,
type HourlySection,
TbankSyncRepository,
} from './tbank-sync.repository';
const HOUR_MS = 3_600_000;
const TICK_MS = 60_000;
const WORK_POLL_MS = 100;
const WORK_POLL_BACKOFF_FACTOR = 2;
const WORK_POLL_MAX_MS = 5_000;
// ponytail: fixed 15-minute step wait bound (900_000 ms), no config knob;
// make it configurable if slow steps become regular.
const WORK_WAIT_BOUND_MS = 900_000;
// An await that outlived WORK_WAIT_BOUND_MS. The underlying call keeps
// running; its late durable result is fenced like any other late publication.
const BOUND_EXPIRED = Symbol('hourly-bound-expired');
const FIRST_DAY_OF_MONTH = 1;
const NO_MILLISECONDS = 0;
type DispatchStep = (child: HourlyChild) => Promise<void>;
type Accounts = Awaited<ReturnType<TbankAccessService['local']>>['accounts'];
// One native timer per process. The timer only proposes work; the durable
// reservation in TbankSyncRepository is the fence that keeps a single
// attempt per connection. The deployment runs a single backend process.
// ponytail: no inter-process lease; add one before scaling past one process.
@Injectable()
export class TbankHourlyScheduler implements OnModuleDestroy, OnModuleInit {
private readonly processRunId = randomUUID();
// Attempt ids of walks currently executing in this process; a running row
// outside this set lost its settle and is closed by recovery.
private readonly active = new Set<string>();
private timer: NodeJS.Timeout | null = null;
private ticking = false;
constructor(
private readonly prisma: PrismaService,
private readonly repository: TbankSyncRepository,
private readonly access: TbankAccessService,
private readonly membership: TbankMembershipService,
private readonly portfolio: PortfolioService,
private readonly operations: BrokerageOperationsSyncService,
private readonly events: UpcomingEventsService,
private readonly logger: StructuredLogger,
) {}
onModuleInit() {
this.timer = setInterval(() => void this.tick(), TICK_MS);
}
onModuleDestroy() {
if (this.timer) clearInterval(this.timer);
}
async tick() {
if (this.ticking) return;
this.ticking = true;
try {
const connections = await this.prisma.tbankConnection.findMany({
where: { invalidToken: false },
select: { userId: true },
});
for (const { userId } of connections) {
await this.launch(userId);
}
} catch {
// A failed tick only delays the proposal until the next tick.
} finally {
this.ticking = false;
}
}
// A fresh connection starts its first walk immediately, regardless of the
// previous term of the replaced connection.
private async launch(userId: string) {
await this.recoverAbandoned(userId);
const reserved = await this.repository.reserveHourly(
userId,
this.processRunId,
HOUR_MS,
);
if (!reserved) return null;
const { userId: owner, revision, id, attemptId } = reserved.work;
setImmediate(() => {
void this.walk({ userId: owner, revision, id, attemptId });
});
return reserved.work;
}
// A running hourly row of this process that has no active walk lost its
// settle (suppressed persistence error). Close it from its durable step
// results; rows of other processes are closed by the reservation restart
// branch. Rides the existing tick: no new timers or sanitors.
private async recoverAbandoned(userId: string) {
const rows = await this.repository.runningHourly(userId);
for (const row of rows) {
if (row.processRunId !== this.processRunId) continue;
if (this.active.has(row.attemptId)) continue;
await this.repository
.settleHourly(
{
userId,
revision: row.revision,
id: row.id,
attemptId: row.attemptId,
},
HOURLY_CAUSE_INTERRUPTED,
)
.catch(() => undefined);
}
}
// One membership pass, then the owners refresh their own datasets. The
// walk publishes only safe section results and holds no transaction or
// upstream call inside one. The attempt outcome is derived from durable
// step rows by the repository, never from walk-local state.
private async walk(attempt: TbankSyncAttempt) {
this.active.add(attempt.attemptId);
try {
const { userId, revision } = attempt;
const children: HourlyChild[] = [];
// Required sections whose dispatch failed never get a durable step
// row; they are recorded as problems so the attempt cannot succeed.
const dispatchProblems: HourlySection[] = [];
const dispatch: DispatchStep = async (child) => {
children.push(child);
await this.repository.recordHourlyChild(attempt, child);
};
const accounts = await this.refreshMembership(
attempt,
dispatch,
dispatchProblems,
);
if (!accounts) return;
await this.refreshAccounts(
userId,
revision,
accounts,
dispatch,
dispatchProblems,
);
await this.waitWorks(userId, children.map((child) => child.id));
for (const account of accounts) {
const pending = this.events
.get(userId, account.accountId)
.catch(() => undefined);
await this.withBound(pending);
}
await this.settle(attempt, null, dispatchProblems);
} catch {
await this.settle(attempt, HOURLY_CAUSE_INTERRUPTED);
} finally {
this.active.delete(attempt.attemptId);
}
}
// Returns the current accounts, or null when the attempt is already
// settled (invalid token or a replaced connection revision).
private async refreshMembership(
attempt: TbankSyncAttempt,
dispatch: DispatchStep,
dispatchProblems: HourlySection[],
) {
const { userId, revision } = attempt;
const member = await this.withBound(
this.membership.request(userId, revision, true),
);
if (member && member !== BOUND_EXPIRED) {
await dispatch({ id: member.id, section: 'membership' });
const done = await this.withBound(member.done);
if (
done !== BOUND_EXPIRED &&
(await this.repository.workError(userId, member.id)) ===
TbankErrorCode.InvalidToken
) {
await this.settle(attempt);
return null;
}
} else {
// No durable step id (rejected request or an expired wait): record the
// observable problem; a late row stays fenced by its durable attempt id.
dispatchProblems.push({
section: 'membership',
errorCode: TbankErrorCode.Unavailable,
});
}
const current = await this.withBound(this.access.local(userId));
if (current === BOUND_EXPIRED || current.revision !== revision) {
await this.settle(attempt, HOURLY_CAUSE_INTERRUPTED);
return null;
}
return current.accounts;
}
// Races a step await against the same per-step bound the row wait uses, so
// a dispatch that never resolves cannot hold the attempt running forever.
// The call itself is not cancelled; a rejection landing after the expiry is
// swallowed here because the expiry already accounts the step as a problem.
private withBound<T>(step: Promise<T>): Promise<T | typeof BOUND_EXPIRED> {
step.catch(() => undefined);
return new Promise<T | typeof BOUND_EXPIRED>((resolve, reject) => {
const timer = setTimeout(
() => resolve(BOUND_EXPIRED),
WORK_WAIT_BOUND_MS,
);
step.then(
(value) => {
clearTimeout(timer);
resolve(value);
},
(error: unknown) => {
clearTimeout(timer);
reject(error instanceof Error ? error : new Error(String(error)));
},
);
});
}
private async refreshAccounts(
userId: string,
revision: string,
accounts: Accounts,
dispatch: DispatchStep,
dispatchProblems: HourlySection[],
) {
for (const account of accounts) {
await this.dispatchPortfolio(
userId,
revision,
account.accountId,
dispatch,
dispatchProblems,
);
}
for (const account of accounts) {
await this.dispatchOperations(
userId,
revision,
account.accountId,
dispatch,
dispatchProblems,
);
}
for (const account of accounts) {
await this.dispatchEvents(
userId,
account.accountId,
dispatch,
dispatchProblems,
);
}
}
private async dispatchPortfolio(
userId: string,
revision: string,
accountId: string,
dispatch: DispatchStep,
dispatchProblems: HourlySection[],
) {
try {
const work = await this.withBound(
this.portfolio.request(userId, revision, accountId, true),
);
if (work && work !== BOUND_EXPIRED) {
await dispatch({ accountId, id: work.id, section: 'portfolio' });
return;
}
} catch {
// A failed dispatch has no durable step row; record the problem.
}
dispatchProblems.push({
accountId,
errorCode: TbankErrorCode.Unavailable,
section: 'portfolio',
});
}
private async dispatchOperations(
userId: string,
revision: string,
accountId: string,
dispatch: DispatchStep,
dispatchProblems: HourlySection[],
) {
try {
const id = await this.withBound(
this.operations.request(
userId,
[
{
accountId,
from: monthStartOf(new Date()),
to: new Date(),
},
],
true,
'screen',
revision,
),
);
if (id && id !== BOUND_EXPIRED) {
await dispatch({ accountId, id, section: 'operations' });
return;
}
} catch {
// A failed dispatch has no durable step row; record the problem.
}
dispatchProblems.push({
accountId,
errorCode: TbankErrorCode.Unavailable,
section: 'operations',
});
}
private async dispatchEvents(
userId: string,
accountId: string,
dispatch: DispatchStep,
dispatchProblems: HourlySection[],
) {
try {
const ids = await this.withBound(this.events.refresh(userId, accountId));
if (ids !== BOUND_EXPIRED) {
for (const id of ids) {
await dispatch({ accountId, id, section: 'events' });
}
return;
}
} catch {
// A failed dispatch has no durable step row; record the problem.
}
dispatchProblems.push({
accountId,
errorCode: TbankErrorCode.Unavailable,
section: 'events',
});
}
private async settle(
attempt: TbankSyncAttempt,
cause: HourlyCause | null = null,
dispatchProblems: HourlySection[] = [],
) {
// A suppressed settle error is not lost: the walk leaves the active set
// and the next tick closes the attempt from its durable step results.
await this.repository
.settleHourly(attempt, cause, dispatchProblems)
.catch(() => {
this.logger.event('error', 'tbank.hourly.settle_failed', {
userId: attempt.userId,
});
return undefined;
});
}
// Drops settled and missing rows from the wait set and expires waits that
// passed the per-step bound.
private advanceWaiting(
waiting: Map<string, number>,
rows: { id: string; state: string }[],
) {
for (const row of rows) {
if (row.state !== 'running') waiting.delete(row.id);
}
for (const id of [...waiting.keys()]) {
if (!rows.some((row) => row.id === id)) {
waiting.delete(id);
continue;
}
const startedAt = waiting.get(id) ?? NO_MILLISECONDS;
if (Date.now() - startedAt >= WORK_WAIT_BOUND_MS) waiting.delete(id);
}
}
// Waits for every dispatched step to leave the running state. Each wait is
// bounded so a stuck step cannot hold the attempt running forever; the
// bound does not cancel the upstream call, and late results stay in the
// step row where the attempt fence decides whether they count.
private async waitWorks(userId: string, ids: string[]) {
const waiting = new Map(ids.map((id) => [id, Date.now()]));
let pollMs = WORK_POLL_MS;
while (waiting.size) {
const rows = await this.repository.works(userId, [...waiting.keys()]);
this.advanceWaiting(waiting, rows);
if (waiting.size) {
await new Promise((resolve) => setTimeout(resolve, pollMs));
pollMs = Math.min(pollMs * WORK_POLL_BACKOFF_FACTOR, WORK_POLL_MAX_MS);
}
}
}
}
function monthStartOf(now: Date) {
return new Date(
Date.UTC(now.getUTCFullYear(), now.getUTCMonth(), FIRST_DAY_OF_MONTH),
);
}
@@ -0,0 +1,9 @@
import { Module } from '@nestjs/common';
import { TbankSyncRepository } from './tbank-sync.repository';
@Module({
providers: [TbankSyncRepository],
exports: [TbankSyncRepository],
})
export class TbankSyncRepositoryModule {}
@@ -1,22 +1,8 @@
import type { Request } from 'express'; import type { Request } from 'express';
import { Controller, Get, Param, Req, UseGuards } from '@nestjs/common';
import { import {
BadRequestException,
Body,
Controller,
Get,
HttpCode,
HttpStatus,
Param,
Post,
Req,
UseGuards,
ValidationPipe,
} from '@nestjs/common';
import {
ApiAcceptedResponse,
ApiBearerAuth, ApiBearerAuth,
ApiBody,
ApiExtraModels, ApiExtraModels,
ApiOkResponse, ApiOkResponse,
ApiTags, ApiTags,
@@ -27,9 +13,6 @@ import { AccessPayload, JwtAuthGuard } from '../../auth/auth.api';
import { import {
EventsInvestmentScreenDto, EventsInvestmentScreenDto,
IncomeInvestmentScreenDto, IncomeInvestmentScreenDto,
InvestmentScreenDto,
investmentScreenSchema,
TbankSyncAcceptedDto,
TbankSyncStatusDto, TbankSyncStatusDto,
TimestampInvestmentScreenDto, TimestampInvestmentScreenDto,
} from './tbank-sync.dto'; } from './tbank-sync.dto';
@@ -37,9 +20,10 @@ import { TbankSyncService } from './tbank-sync.service';
@ApiTags('tbank-sync') @ApiTags('tbank-sync')
@ApiExtraModels( @ApiExtraModels(
IncomeInvestmentScreenDto,
EventsInvestmentScreenDto, EventsInvestmentScreenDto,
IncomeInvestmentScreenDto,
TimestampInvestmentScreenDto, TimestampInvestmentScreenDto,
TbankSyncStatusDto,
) )
@ApiProblemResponses() @ApiProblemResponses()
@ApiBearerAuth() @ApiBearerAuth()
@@ -47,28 +31,6 @@ import { TbankSyncService } from './tbank-sync.service';
@Controller('tbank-sync') @Controller('tbank-sync')
export class TbankSyncController { export class TbankSyncController {
constructor(private readonly sync: TbankSyncService) {} constructor(private readonly sync: TbankSyncService) {}
@Post()
@HttpCode(HttpStatus.ACCEPTED)
@ApiBody({ schema: investmentScreenSchema })
@ApiAcceptedResponse({ type: TbankSyncAcceptedDto })
request(
@Req() request: Request & { user: AccessPayload },
@Body(
new ValidationPipe({
forbidUnknownValues: true,
expectedType: InvestmentScreenDto,
transform: true,
whitelist: true,
forbidNonWhitelisted: true,
validationError: { target: false, value: false },
exceptionFactory: (message) => new BadRequestException({ message }),
}),
)
input: unknown,
) {
return this.sync.request(request.user.sub, input as InvestmentScreenDto);
}
@Get(':id') @Get(':id')
@ApiOkResponse({ type: TbankSyncStatusDto }) @ApiOkResponse({ type: TbankSyncStatusDto })
status( status(
@@ -22,6 +22,7 @@ import {
} from 'class-validator'; } from 'class-validator';
import { MAX_INSTRUMENT_QUERY_LENGTH, MIN_INSTRUMENT_QUERY_LENGTH } from '../instrument-query'; import { MAX_INSTRUMENT_QUERY_LENGTH, MIN_INSTRUMENT_QUERY_LENGTH } from '../instrument-query';
import { InvestmentAttemptProblemDto } from '../investment-resource.dto';
import { operationCategories } from '../operations/operations.api'; import { operationCategories } from '../operations/operations.api';
const MIN_ACCOUNT_ID_LENGTH = 1; const MIN_ACCOUNT_ID_LENGTH = 1;
@@ -227,3 +228,46 @@ export class TbankSyncStatusDto {
@ApiProperty({ type: String, nullable: true }) declare errorCode: @ApiProperty({ type: String, nullable: true }) declare errorCode:
null | string; null | string;
} }
export class HourlySyncSectionDto {
@ApiProperty({ enum: ['membership', 'portfolio', 'operations', 'events'] })
declare section: 'events' | 'membership' | 'operations' | 'portfolio';
@ApiPropertyOptional({
type: String,
nullable: false,
description: 'Account identifier of the problematic section, when scoped.',
})
declare accountId?: string;
@ApiProperty({
type: String,
nullable: true,
description:
'Safe error code of the failed section. No tokens or upstream payloads.',
})
declare errorCode: null | string;
}
export class HourlySyncStatusDto {
@ApiProperty({
enum: ['running', 'succeeded', 'partial', 'failed', 'cancelled'],
description:
'Last hourly walk state. Cancelled means an interrupted attempt; its start still delays the next term.',
})
declare state:
| 'cancelled' |
'failed' |
'partial' |
'running' |
'succeeded';
@ApiProperty({ format: 'date-time' })
declare startedAt: string;
@ApiProperty({ type: String, nullable: true, format: 'date-time' })
declare finishedAt: null | string;
@ApiProperty({ type: InvestmentAttemptProblemDto, isArray: true })
declare problems: InvestmentAttemptProblemDto[];
}
@@ -5,8 +5,10 @@ import { AuthModule } from '../../auth/auth.module';
import { InstrumentSourcesModule } from '../instruments/instrument-sources.module'; import { InstrumentSourcesModule } from '../instruments/instrument-sources.module';
import { OperationsModule } from '../operations/operations.module'; import { OperationsModule } from '../operations/operations.module';
import { PortfolioModule } from '../portfolio/portfolio.module'; import { PortfolioModule } from '../portfolio/portfolio.module';
import { UpcomingEventsModule } from '../upcoming-events/upcoming-events.module';
import { TbankHourlyScheduler } from './tbank-hourly.scheduler';
import { TbankSyncRepositoryModule } from './tbank-sync-repository.module';
import { TbankSyncController } from './tbank-sync.controller'; import { TbankSyncController } from './tbank-sync.controller';
import { TbankSyncRepository } from './tbank-sync.repository';
import { TbankSyncService } from './tbank-sync.service'; import { TbankSyncService } from './tbank-sync.service';
@Module({ @Module({
imports: [ imports: [
@@ -15,8 +17,11 @@ import { TbankSyncService } from './tbank-sync.service';
PortfolioModule, PortfolioModule,
OperationsModule, OperationsModule,
InstrumentSourcesModule, InstrumentSourcesModule,
UpcomingEventsModule,
TbankSyncRepositoryModule,
], ],
controllers: [TbankSyncController], controllers: [TbankSyncController],
providers: [TbankSyncService, TbankSyncRepository], providers: [TbankSyncService, TbankHourlyScheduler],
exports: [TbankHourlyScheduler],
}) })
export class TbankSyncModule {} export class TbankSyncModule {}
@@ -3,6 +3,7 @@ import { randomUUID } from 'node:crypto';
import type { Prisma } from '../../../../generated/prisma/client'; import type { Prisma } from '../../../../generated/prisma/client';
import type { TbankSyncAttempt } from '../../../integrations/tbank/tbank.api'; import type { TbankSyncAttempt } from '../../../integrations/tbank/tbank.api';
import type { InvestmentAttemptDto } from '../investment-resource.dto';
import type { InvestmentScreenDto } from './tbank-sync.dto'; import type { InvestmentScreenDto } from './tbank-sync.dto';
import { PrismaService } from '../../../prisma/prisma.service'; import { PrismaService } from '../../../prisma/prisma.service';
@@ -12,6 +13,62 @@ export type ScreenProgress = {
children: string[]; children: string[];
dispatched: boolean; dispatched: boolean;
}; };
export type HourlySection = {
accountId?: string;
errorCode: null | string;
section: 'events' | 'membership' | 'operations' | 'portfolio';
};
export type HourlyChild = {
accountId?: string;
id: string;
section: HourlySection['section'];
};
export const HOURLY_CAUSE_DEADLINE = 'DEADLINE';
export const HOURLY_CAUSE_INTERRUPTED = 'INTERRUPTED';
export type HourlyCause = typeof HOURLY_CAUSE_INTERRUPTED;
type StepOutcome = { problem?: HourlySection; success: boolean };
type StepRow = { attemptId: string; errorCode: null | string; state: string };
export type HourlyProgress = {
attempts?: Record<string, string>;
children?: HourlyChild[];
result?: 'failed' | 'partial' | 'succeeded';
sections: HourlySection[];
};
type HourlyAttempt = {
finishedAt: Date | null;
sections: HourlySection[];
startedAt: Date;
state: 'cancelled' | 'failed' | 'partial' | 'running' | 'succeeded';
};
export function hourlyAttempt(
hourly: HourlyAttempt | null,
section: HourlySection['section'] | readonly HourlySection['section'][],
accountId?: string,
): InvestmentAttemptDto {
if (!hourly) {
return { state: 'none', startedAt: null, finishedAt: null, problems: [] };
}
const sections = Array.isArray(section) ? section : [section];
const problems = hourly.sections
.filter(
(entry) =>
sections.includes(entry.section) &&
(accountId === undefined || entry.accountId === accountId),
)
.map((entry) => ({
section: entry.section,
accountId: entry.accountId ?? null,
errorCode: entry.errorCode,
}));
return {
state: hourly.state === 'cancelled' ? 'failed' : hourly.state,
startedAt: hourly.startedAt.toISOString(),
finishedAt: hourly.finishedAt?.toISOString() ?? null,
problems,
};
}
@Injectable() @Injectable()
export class TbankSyncRepository { export class TbankSyncRepository {
constructor(private readonly prisma: PrismaService) {} constructor(private readonly prisma: PrismaService) {}
@@ -123,4 +180,248 @@ export class TbankSyncRepository {
}); });
}); });
} }
// The timer only proposes work; this reservation is the durable fence. The
// attempt start (createdAt) is the basis of the next hourly term, including
// failed and interrupted attempts. After downtime the next tick is due at
// once, so no catch-up queue forms.
reserveHourly(userId: string, runId: string, hourMs: number) {
return this.prisma.$transaction(async (tx) => {
const connection = await tx.tbankConnection.findFirst({
where: { userId, invalidToken: false },
select: { revision: true },
});
if (!connection) return null;
const key = {
userId,
revision: connection.revision,
scopeKey: 'hourly',
};
const previous = await tx.tbankSyncWork.findUnique({
where: { userId_revision_scopeKey: key },
});
const now = new Date();
if (previous) {
// Terms are hourMs steps from the attempt start; a walk still running
// at a term consumes it. A dead process's attempt has no known end, so
// it consumes only its own term.
const started = previous.createdAt.valueOf();
const ended =
previous.state === 'running' ? started : previous.updatedAt.valueOf();
const dueAt =
started +
Math.max(hourMs, Math.ceil((ended - started) / hourMs) * hourMs);
const due = now.valueOf() >= dueAt;
if (previous.state === 'running') {
if (previous.processRunId === runId) return null;
// A dead process cannot finish its attempt; close it from its
// actual step results, but its start stays the basis of the next
// term.
await this.closeHourly(tx, previous, 'INTERRUPTED', []);
if (!due) return null;
} else if (!due) return null;
}
const data = {
attemptId: randomUUID(),
processRunId: runId,
state: 'running',
priority: 0,
intent: JSON.parse(
JSON.stringify({ source: 'hourly' }),
) as Prisma.InputJsonObject,
progress: { sections: [] } as unknown as Prisma.InputJsonValue,
errorCode: null,
};
const work = await tx.tbankSyncWork.upsert({
where: { userId_revision_scopeKey: key },
create: { ...key, ...data, id: randomUUID() },
update: { ...data, createdAt: now },
});
return { work, launch: true };
});
}
private classifyStep(
child: HourlyChild,
row: StepRow | undefined,
recorded: string | undefined,
cause: HourlyCause | null,
): StepOutcome {
if (row && row.attemptId === recorded && row.state === 'complete') {
return { success: true };
}
const stale = !row || (recorded !== undefined && row.attemptId !== recorded);
let errorCode: null | string = null;
if (!stale && row.state === 'failed') errorCode = row.errorCode;
else if (!stale && cause === HOURLY_CAUSE_INTERRUPTED) {
errorCode = HOURLY_CAUSE_INTERRUPTED;
} else if (!stale) errorCode = HOURLY_CAUSE_DEADLINE;
return {
problem: { accountId: child.accountId, errorCode, section: child.section },
success: false,
};
}
private deriveResult(
cause: HourlyCause | null,
sections: HourlySection[],
successes: number,
): 'failed' | 'partial' | 'succeeded' {
if (!cause && !sections.length) return 'succeeded';
return successes ? 'partial' : 'failed';
}
// The outcome is derived from the durable state of the dispatched steps,
// never from an in-memory error list that an exception path can empty.
// `cause` marks a close before normal completion and caps the result at
// `partial`; `dispatchProblems` records required sections that never got a
// durable step row because their dispatch failed.
private async closeHourly(
tx: Prisma.TransactionClient,
work: {
attemptId: string;
id: string;
progress: Prisma.JsonValue;
revision: string;
userId: string;
},
cause: HourlyCause | null,
dispatchProblems: HourlySection[],
) {
const progress = work.progress as HourlyProgress;
const children = progress.children ?? [];
const rows = children.length
? await tx.tbankSyncWork.findMany({
where: {
userId: work.userId,
id: { in: children.map((child) => child.id) },
},
select: { attemptId: true, errorCode: true, id: true, state: true },
})
: [];
const byId = new Map(rows.map((row) => [row.id, row]));
const outcomes = children.map((child) =>
this.classifyStep(
child,
byId.get(child.id),
progress.attempts?.[child.id],
cause,
),
);
const sections = outcomes
.flatMap((outcome) => (outcome.problem ? [outcome.problem] : []))
.concat(dispatchProblems);
const successes = outcomes.filter((outcome) => outcome.success).length;
const result = this.deriveResult(cause, sections, successes);
return tx.tbankSyncWork.updateMany({
where: { id: work.id, attemptId: work.attemptId, state: 'running' },
data: {
state: result === 'failed' ? 'failed' : 'complete',
errorCode:
cause ??
sections.find((section) => section.errorCode)?.errorCode ??
null,
progress: {
attempts: progress.attempts ?? {},
children,
result,
sections,
},
},
});
}
async settleHourly(
attempt: TbankSyncAttempt,
cause: HourlyCause | null = null,
dispatchProblems: HourlySection[] = [],
) {
return this.prisma.$transaction(async (tx) => {
const work = await tx.tbankSyncWork.findFirst({ where: attempt });
if (!work || work.state !== 'running') return { count: 0 };
return this.closeHourly(tx, work, cause, dispatchProblems);
});
}
// Records a dispatched step on the attempt before it is awaited, so a lost
// walk can still derive its outcome from durable rows.
async recordHourlyChild(attempt: TbankSyncAttempt, child: HourlyChild) {
return this.prisma.$transaction(async (tx) => {
const work = await tx.tbankSyncWork.findFirst({
where: { ...attempt, state: 'running' },
select: { progress: true },
});
if (!work) return false;
const progress = work.progress as HourlyProgress;
const row = await tx.tbankSyncWork.findFirst({
where: { userId: attempt.userId, id: child.id },
select: { attemptId: true },
});
const updated = await tx.tbankSyncWork.updateMany({
where: { ...attempt, state: 'running' },
data: {
progress: {
attempts: {
...progress.attempts,
...(row ? { [child.id]: row.attemptId } : {}),
},
children: [...(progress.children ?? []), child],
sections: progress.sections ?? [],
},
},
});
return Boolean(updated.count);
});
}
runningHourly(userId: string) {
return this.prisma.tbankSyncWork.findMany({
where: { userId, scopeKey: 'hourly', state: 'running' },
select: { attemptId: true, id: true, processRunId: true, revision: true },
});
}
async hourly(userId: string) {
const connection = await this.prisma.tbankConnection.findFirst({
where: { userId },
select: { revision: true, invalidToken: true },
});
if (!connection || connection.invalidToken) return null;
const work = await this.prisma.tbankSyncWork.findUnique({
where: {
userId_revision_scopeKey: {
userId,
revision: connection.revision,
scopeKey: 'hourly',
},
},
});
if (!work) return null;
const progress = work.progress as HourlyProgress;
let state: 'cancelled' | 'running' | HourlyProgress['result'];
if (work.state === 'running') state = 'running';
else if (work.state === 'cancelled') state = 'cancelled';
else state = progress.result ?? 'failed';
return {
startedAt: work.createdAt,
finishedAt: work.state === 'running' ? null : work.updatedAt,
state,
sections: progress.sections ?? [],
};
}
works(userId: string, ids: string[]) {
return this.prisma.tbankSyncWork.findMany({
where: { userId, id: { in: ids } },
select: { id: true, state: true, errorCode: true },
});
}
async workError(userId: string, id: string) {
const work = await this.prisma.tbankSyncWork.findFirst({
where: { userId, id },
select: { state: true, errorCode: true },
});
return work?.state === 'failed' ? work.errorCode : null;
}
} }
@@ -119,7 +119,6 @@ export class TbankSyncService {
scope.screen === 'stock' || scope.screen === 'bond' scope.screen === 'stock' || scope.screen === 'bond'
? 'all' ? 'all'
: accountScope, : accountScope,
true,
); );
const positions = saved.data?.positions ?? []; const positions = saved.data?.positions ?? [];
const instruments = [ const instruments = [
@@ -4,11 +4,12 @@ import { TbankModule } from '../../../integrations/tbank/tbank.module';
import { AuthModule } from '../../auth/auth.module'; import { AuthModule } from '../../auth/auth.module';
import { InstrumentSourcesModule } from '../instruments/instrument-sources.module'; import { InstrumentSourcesModule } from '../instruments/instrument-sources.module';
import { PortfolioModule } from '../portfolio/portfolio.module'; import { PortfolioModule } from '../portfolio/portfolio.module';
import { TbankSyncRepositoryModule } from '../tbank-sync/tbank-sync-repository.module';
import { UpcomingEventsController } from './upcoming-events.controller'; import { UpcomingEventsController } from './upcoming-events.controller';
import { UpcomingEventsService } from './upcoming-events.service'; import { UpcomingEventsService } from './upcoming-events.service';
@Module({ @Module({
imports: [AuthModule, TbankModule, InstrumentSourcesModule, PortfolioModule], imports: [AuthModule, TbankModule, InstrumentSourcesModule, PortfolioModule, TbankSyncRepositoryModule],
controllers: [UpcomingEventsController], controllers: [UpcomingEventsController],
providers: [UpcomingEventsService], providers: [UpcomingEventsService],
exports: [UpcomingEventsService], exports: [UpcomingEventsService],
@@ -2,6 +2,7 @@ import { BadRequestException, Injectable } from '@nestjs/common';
import { import {
add, add,
type Decimal,
decimalFromString, decimalFromString,
formatDecimal, formatDecimal,
isZero, isZero,
@@ -20,6 +21,10 @@ import {
type PortfolioPositionDto, type PortfolioPositionDto,
PortfolioService, PortfolioService,
} from '../portfolio/portfolio.api'; } from '../portfolio/portfolio.api';
import {
hourlyAttempt,
TbankSyncRepository,
} from '../tbank-sync/tbank-sync.repository';
import { bondEvent, dividendEvent } from './upcoming-event.mapper'; import { bondEvent, dividendEvent } from './upcoming-event.mapper';
import { import {
UpcomingEventDto, UpcomingEventDto,
@@ -43,66 +48,28 @@ export class UpcomingEventsService {
private readonly sources: CatalogEnrichmentService, private readonly sources: CatalogEnrichmentService,
private readonly forecasts: ForecastService, private readonly forecasts: ForecastService,
private readonly metadata: InstrumentMetadataService, private readonly metadata: InstrumentMetadataService,
private readonly sync: TbankSyncRepository,
) {} ) {}
// Read endpoints stay local; callers without a saved portfolio receive unknown.
async get( async get(
userId: string, userId: string,
accountId: string, accountId: string,
query: UpcomingEventsQueryDto = {}, query: UpcomingEventsQueryDto = {},
) { ): Promise<Awaited<ReturnType<ForecastService['result']>>> {
const window = dateWindow(query); const window = dateWindow(query);
const connection = await this.access.local(userId, accountId); const connection = await this.access.local(userId, accountId);
const portfolio = await this.portfolios.read( const portfolio = await this.portfolios.read(userId, accountId);
userId, const hourly = await this.sync.hourly(userId);
accountId,
query.sync === 'observe',
);
const sources: Awaited<ReturnType<CatalogEnrichmentService['readMany']>> =
[];
const events: UpcomingEventDto[] = []; const events: UpcomingEventDto[] = [];
let ready = portfolio.data !== null; let ready = portfolio.data !== null;
const positions = portfolio.data?.positions.filter(supported) ?? []; const positions = portfolio.data?.positions.filter(supported) ?? [];
const metadata = await this.metadata.savedMany(positions); const { sources, metadata } = await this.readSources(
for (const type of ['share', 'bond'] as const) { userId,
const identities = positions positions,
.filter( window,
(position) => position.type === (type === 'share' ? 'STOCK' : 'BOND'), true,
) );
.map((position) => ({
uid:
position.instrumentUid ||
metadata.get(metadataKey(position))?.data?.uid,
ticker:
position.ticker ||
metadata.get(metadataKey(position))?.data?.ticker ||
'',
}))
.filter((entry): entry is { ticker: string; uid: string } =>
Boolean(entry.uid),
);
const uids = [...new Set(identities.map((entry) => entry.uid))];
for (
let offset = FIRST_BATCH_OFFSET;
offset < uids.length;
offset += calendarReadBatchSize
) {
sources.push(
...(await this.sources.readMany(
userId,
type,
uids.slice(offset, offset + calendarReadBatchSize),
query.sync === 'observe',
['detail', type === 'share' ? 'dividends' : 'bond-events'],
{
...window,
tickers: Object.fromEntries(
identities.map((entry) => [entry.uid, entry.ticker]),
),
},
)),
);
}
}
for (const position of positions) { for (const position of positions) {
const generic = metadata.get(metadataKey(position))?.data; const generic = metadata.get(metadataKey(position))?.data;
const uid = position.instrumentUid || generic?.uid; const uid = position.instrumentUid || generic?.uid;
@@ -138,7 +105,7 @@ export class UpcomingEventsService {
source.input.uid === uid && source.input.kind === 'detail', source.input.uid === uid && source.input.kind === 'detail',
)?.instrument || generic; )?.instrument || generic;
const ticker = position.ticker || instrument?.ticker; const ticker = position.ticker || instrument?.ticker;
if (!calendar || calendar.meta.availability !== 'complete' || !ticker) { if (!calendar || calendar.meta.completeness !== 'complete' || !ticker) {
ready = false; ready = false;
continue; continue;
} }
@@ -197,6 +164,11 @@ export class UpcomingEventsService {
membershipRevision: connection.membershipRevision, membershipRevision: connection.membershipRevision,
scopeKey: `events/${JSON.stringify([accountId, window.from, window.to])}`, scopeKey: `events/${JSON.stringify([accountId, window.from, window.to])}`,
portfolio: portfolio.meta, portfolio: portfolio.meta,
lastAttempt: hourlyAttempt(
hourly,
'events',
accountId === 'all' ? undefined : accountId,
),
sources, sources,
candidate, candidate,
}); });
@@ -205,6 +177,89 @@ export class UpcomingEventsService {
getAll(userId: string, query: UpcomingEventsQueryDto = {}) { getAll(userId: string, query: UpcomingEventsQueryDto = {}) {
return this.get(userId, 'all', query); return this.get(userId, 'all', query);
} }
// Triggers online refreshes of the account's event sources and returns the
// launched work ids for the hourly walk to observe.
async refresh(
userId: string,
accountId: string,
query: UpcomingEventsQueryDto = {},
): Promise<string[]> {
const window = dateWindow(query);
await this.access.local(userId, accountId);
const portfolio = await this.portfolios.read(userId, accountId);
const positions = portfolio.data?.positions.filter(supported) ?? [];
const { launched } = await this.readSources(
userId,
positions,
window,
false,
);
return launched;
}
private async readSources(
userId: string,
positions: Position[],
window: { from: string; to: string },
observe: boolean,
) {
const metadata = await this.metadata.savedMany(positions);
const sources: Awaited<ReturnType<CatalogEnrichmentService['readMany']>> =
[];
const launched: string[] = [];
for (const type of ['share', 'bond'] as const) {
const identities = this.positionIdentities(positions, metadata, type);
const uids = [...new Set(identities.map((entry) => entry.uid))];
for (
let offset = FIRST_BATCH_OFFSET;
offset < uids.length;
offset += calendarReadBatchSize
) {
const batch = await this.sources.readMany(
userId,
type,
uids.slice(offset, offset + calendarReadBatchSize),
observe,
['detail', type === 'share' ? 'dividends' : 'bond-events'],
{
...window,
tickers: Object.fromEntries(
identities.map((entry) => [entry.uid, entry.ticker]),
),
},
);
sources.push(...batch);
for (const source of batch) {
if (source.meta.sync.state === 'running' && source.meta.sync.id) {
launched.push(source.meta.sync.id);
}
}
}
}
return { sources, metadata, launched };
}
private positionIdentities(
positions: Position[],
metadata: Awaited<ReturnType<InstrumentMetadataService['savedMany']>>,
type: 'bond' | 'share',
) {
return positions
.filter((position) => position.type === (type === 'share' ? 'STOCK' : 'BOND'))
.map((position) => ({
uid:
position.instrumentUid ||
metadata.get(metadataKey(position))?.data?.uid,
ticker:
position.ticker ||
metadata.get(metadataKey(position))?.data?.ticker ||
'',
}))
.filter((entry): entry is { ticker: string; uid: string } =>
Boolean(entry.uid),
);
}
} }
function supported(position: Position): boolean { function supported(position: Position): boolean {
@@ -226,7 +281,7 @@ function dateWindow(query: UpcomingEventsQueryDto) {
} }
function incomeTotals(events: UpcomingEventDto[]): MoneyDto[] { function incomeTotals(events: UpcomingEventDto[]): MoneyDto[] {
const totals = new Map<string, ReturnType<typeof decimalFromString>>(); const totals = new Map<string, Decimal>();
for (const event of events) { for (const event of events) {
if (!['coupon', 'dividend'].includes(event.type) || !event.projectedAmount) { if (!['coupon', 'dividend'].includes(event.type) || !event.projectedAmount) {
continue; continue;
+12 -1
View File
@@ -3,9 +3,20 @@ import { DocumentBuilder, SwaggerModule } from '@nestjs/swagger';
export const API_PREFIX = 'api/v1'; export const API_PREFIX = 'api/v1';
// Nest Swagger cannot express additionalProperties on a class, so closed request
// bodies are restored here after generation; the runtime allowlist stays in the controller.
const CLOSED_SCHEMAS = ['StockScreenerCriteriaDto'];
export function createOpenApiDocument(app: INestApplication) { export function createOpenApiDocument(app: INestApplication) {
return SwaggerModule.createDocument( const document = SwaggerModule.createDocument(
app, app,
new DocumentBuilder().setTitle('WealthLedger API').setVersion('v1').build(), new DocumentBuilder().setTitle('WealthLedger API').setVersion('v1').build(),
); );
const schemas = document.components?.schemas ?? {};
for (const [name, schema] of Object.entries(schemas)) {
if (CLOSED_SCHEMAS.includes(name) && 'properties' in schema) {
schema.additionalProperties = false;
}
}
return document;
} }
+1 -1
View File
@@ -81,7 +81,7 @@ describe('All accounts HTTP boundary', () => {
expect(response.body).not.toHaveProperty('token'); expect(response.body).not.toHaveProperty('token');
expect(response.body).not.toHaveProperty('generation'); expect(response.body).not.toHaveProperty('generation');
expect(response.body).toMatchObject({ expect(response.body).toMatchObject({
meta: { availability: 'complete' }, meta: { availability: 'available', completeness: 'complete' },
...(path === 'portfolio' ? { data: { positions: [] } } : {}), ...(path === 'portfolio' ? { data: { positions: [] } } : {}),
}); });
}, },
+8 -1
View File
@@ -125,7 +125,14 @@ function tbankSnapshot() {
const owner = auth.user.id; const owner = auth.user.id;
await stop(); await stop();
const db = new Database(file); const db = new Database(file);
db.prepare('INSERT INTO TbankConnection (userId,encryptedToken,connectedAt) VALUES (?,?,?)').run(owner, 'fictional-encrypted-token-never-used', new Date().toISOString()); const columns = db.prepare('PRAGMA table_info("TbankConnection")').all().map(({ name }) => name);
const fields = ['userId', 'encryptedToken', 'connectedAt'];
const values = [owner, 'fictional-encrypted-token-never-used', new Date().toISOString()];
if (columns.includes('revision')) {
fields.push('revision');
values.push(randomUUID());
}
db.prepare(`INSERT INTO TbankConnection (${fields.join(',')}) VALUES (${fields.map(() => '?').join(',')})`).run(...values);
await db.backup(join(dir, 'before-migration.db')); await db.backup(join(dir, 'before-migration.db'));
db.close(); db.close();
const beforeSessions = rows('Session'), beforeUsers = rows('User'); const beforeSessions = rows('Session'), beforeUsers = rows('User');
+27 -28
View File
@@ -27,32 +27,8 @@ describe('OpenAPI contract (integration)', () => {
it('distinguishes calendar screen dates from operation instants in requests and responses', () => { it('distinguishes calendar screen dates from operation instants in requests and responses', () => {
const document = createOpenApiDocument(app); const document = createOpenApiDocument(app);
const schemas = document.components?.schemas; const schemas = document.components?.schemas;
const screenSchema = { expect(document.paths['/api/v1/tbank-sync']).toBeUndefined();
oneOf: [ expect(schemas?.TbankSyncStatusDto).toBeDefined();
{ $ref: '#/components/schemas/IncomeInvestmentScreenDto' },
{ $ref: '#/components/schemas/EventsInvestmentScreenDto' },
{ $ref: '#/components/schemas/TimestampInvestmentScreenDto' },
],
discriminator: {
propertyName: 'screen',
mapping: {
income: '#/components/schemas/IncomeInvestmentScreenDto',
events: '#/components/schemas/EventsInvestmentScreenDto',
operations: '#/components/schemas/TimestampInvestmentScreenDto',
},
},
};
expect(
document.paths['/api/v1/tbank-sync'].post?.requestBody,
).toMatchObject({
content: { 'application/json': { schema: screenSchema } },
});
expect(schemas?.TbankSyncAcceptedDto).toMatchObject({
properties: { scope: screenSchema },
});
expect(schemas?.TbankSyncStatusDto).toMatchObject({
properties: { scope: { ...screenSchema, nullable: true } },
});
for (const [model, screen] of [ for (const [model, screen] of [
['IncomeInvestmentScreenDto', 'income'], ['IncomeInvestmentScreenDto', 'income'],
['EventsInvestmentScreenDto', 'events'], ['EventsInvestmentScreenDto', 'events'],
@@ -101,7 +77,7 @@ describe('OpenAPI contract (integration)', () => {
} }
}); });
it('exposes required semantic income dependency roles', () => { it('exposes only the selected income dependency role and no comparison data', () => {
const schemas = createOpenApiDocument(app).components?.schemas; const schemas = createOpenApiDocument(app).components?.schemas;
expect(schemas?.IncomeAnalyticsResourceDto).toMatchObject({ expect(schemas?.IncomeAnalyticsResourceDto).toMatchObject({
properties: { properties: {
@@ -124,12 +100,35 @@ describe('OpenAPI contract (integration)', () => {
expect(dependency.required).toContain('role'); expect(dependency.required).toContain('role');
expect(dependency).toMatchObject({ expect(dependency).toMatchObject({
properties: { properties: {
role: { type: 'string', enum: ['selected', 'previous', 'year'] }, role: { type: 'string', enum: ['selected'] },
}, },
}); });
const analytics = schemas?.IncomeAnalyticsDto;
assert(analytics && !('$ref' in analytics));
expect(analytics.required).toEqual(
expect.arrayContaining([
'accountId',
'period',
'totalsByCurrency',
'calculatedAt',
'bucketGranularity',
'buckets',
'instruments',
'entries',
]),
);
expect(analytics.properties).not.toHaveProperty('previous');
expect(analytics.properties).not.toHaveProperty('yearComparison');
}); });
it('generates the runtime contract without server startup or deployment configuration', async () => { it('generates the runtime contract without server startup or deployment configuration', async () => {
const schemas = createOpenApiDocument(app).components?.schemas;
expect(schemas?.StockScreenerCriteriaDto).toMatchObject({
additionalProperties: false,
});
expect(schemas?.StockScreenerItemDto).toMatchObject({
properties: { lot: { type: 'integer', minimum: 1, nullable: true } },
});
expect( expect(
createOpenApiDocument(app).components?.schemas?.ProblemDetailsDto, createOpenApiDocument(app).components?.schemas?.ProblemDetailsDto,
).toMatchObject({ ).toMatchObject({
+34 -16
View File
@@ -3,16 +3,21 @@ import type { Server } from 'node:http';
import { INestApplication } from '@nestjs/common'; import { INestApplication } from '@nestjs/common';
import { JwtService } from '@nestjs/jwt'; import { JwtService } from '@nestjs/jwt';
import { Test } from '@nestjs/testing'; import { Test } from '@nestjs/testing';
import { setTimeout as delay } from 'node:timers/promises';
import request from 'supertest'; import request from 'supertest';
import { AppModule } from '../src/app.module'; import { AppModule } from '../src/app.module';
import { TbankMembershipService } from '../src/integrations/tbank/tbank-membership.service';
import { TbankClient } from '../src/integrations/tbank/tbank.client'; import { TbankClient } from '../src/integrations/tbank/tbank.client';
import { configureApp } from '../src/main'; import { configureApp } from '../src/main';
import { PortfolioService } from '../src/modules/investments/portfolio/application/portfolio.service';
import { TbankHourlyScheduler } from '../src/modules/investments/tbank-sync/tbank-hourly.scheduler';
import { PrismaService } from '../src/prisma/prisma.service'; import { PrismaService } from '../src/prisma/prisma.service';
describe('Portfolio structure HTTP contract', () => { describe('Portfolio structure HTTP contract', () => {
let app: INestApplication<Server>; let app: INestApplication<Server>;
let prisma: PrismaService;
let membership: TbankMembershipService;
let portfolios: PortfolioService;
const tbank = { const tbank = {
validateToken: jest.fn().mockResolvedValue([ validateToken: jest.fn().mockResolvedValue([
{ id: 'a', name: 'A', type: 'ACCOUNT_TYPE_TINKOFF' }, { id: 'a', name: 'A', type: 'ACCOUNT_TYPE_TINKOFF' },
@@ -22,6 +27,10 @@ describe('Portfolio structure HTTP contract', () => {
getPositions: jest.fn(), getPositions: jest.fn(),
getInstrumentByFigi: jest.fn().mockResolvedValue(null), getInstrumentByFigi: jest.fn().mockResolvedValue(null),
getInstrumentByUid: jest.fn().mockResolvedValue(null), getInstrumentByUid: jest.fn().mockResolvedValue(null),
getOpenAccounts: jest.fn().mockResolvedValue([
{ id: 'a', name: 'A', type: 'ACCOUNT_TYPE_TINKOFF' },
{ id: 'b', name: 'B', type: 'ACCOUNT_TYPE_TINKOFF' },
]),
}; };
const token = new JwtService().sign( const token = new JwtService().sign(
{ sub: 'structure-user', type: 'access' }, { sub: 'structure-user', type: 'access' },
@@ -32,11 +41,16 @@ describe('Portfolio structure HTTP contract', () => {
const module = await Test.createTestingModule({ imports: [AppModule] }) const module = await Test.createTestingModule({ imports: [AppModule] })
.overrideProvider(TbankClient) .overrideProvider(TbankClient)
.useValue(tbank) .useValue(tbank)
.overrideProvider(TbankHourlyScheduler)
.useValue({ connected: jest.fn().mockResolvedValue(undefined) })
.compile(); .compile();
app = module.createNestApplication(); app = module.createNestApplication();
configureApp(app); configureApp(app);
await app.init(); await app.init();
await app.get(PrismaService).user.create({ prisma = app.get(PrismaService);
membership = app.get(TbankMembershipService);
portfolios = app.get(PortfolioService);
await prisma.user.create({
data: { data: {
id: 'structure-user', id: 'structure-user',
email: 'structure@example.invalid', email: 'structure@example.invalid',
@@ -65,21 +79,25 @@ describe('Portfolio structure HTTP contract', () => {
}); });
afterAll(() => app.close()); afterAll(() => app.close());
async function load(id: string) { async function load(id: string) {
await request(app.getHttpServer()) const connection = await prisma.tbankConnection.findUniqueOrThrow({
.get(`/api/v1/brokerage-accounts/${id}/portfolio`) where: { userId: 'structure-user' },
.set('Authorization', `Bearer ${token}`) });
.expect(200); const member = await membership.request(
for (let index = 0; index < 200; index += 1) { 'structure-user',
if ( connection.revision,
!(await app true,
.get(PrismaService) );
.tbankSyncWork.count({ where: { state: 'running' } })) if (member) await member.done;
) { const accountIds = id === 'all' ? ['a', 'b'] : [id];
return; for (const accountId of accountIds) {
} const work = await portfolios.request(
await delay(5); 'structure-user',
connection.revision,
accountId,
true,
);
if (work) await work.done;
} }
throw new Error('Portfolio did not settle');
} }
it.each(['a', 'all'])( it.each(['a', 'all'])(
+1 -1
View File
@@ -18,7 +18,7 @@ writeFileSync(sentinel, 'fictional user database: must never be opened');
const before = readFileSync(sentinel); const before = readFileSync(sentinel);
const databases = new Set(); const databases = new Set();
async function check(name, overrides = {}, signal) { async function check(name, overrides = {}, signal) {
const child = spawn(process.execPath, ['test/sqlite.cjs', '--all'], { const child = spawn(process.execPath, ['test/sqlite.cjs'], {
cwd, cwd,
env: { env: {
...process.env, ...process.env,
+47
View File
@@ -15,6 +15,7 @@ import { configureApp } from '../src/main';
import { StockCatalogService } from '../src/modules/investments/instruments/stocks/application/stock-catalog.service'; import { StockCatalogService } from '../src/modules/investments/instruments/stocks/application/stock-catalog.service';
import { StockIncomeService } from '../src/modules/investments/instruments/stocks/application/stock-income.service'; import { StockIncomeService } from '../src/modules/investments/instruments/stocks/application/stock-income.service';
import { StockMarketService } from '../src/modules/investments/instruments/stocks/application/stock-market.service'; import { StockMarketService } from '../src/modules/investments/instruments/stocks/application/stock-market.service';
import { StockScreenerService } from '../src/modules/investments/instruments/stocks/application/stock-screener.service';
const uid = 'e6123145-9665-43e0-8413-cd61b8aa9b13'; const uid = 'e6123145-9665-43e0-8413-cd61b8aa9b13';
describe('Stock HTTP contract', () => { describe('Stock HTTP contract', () => {
@@ -25,6 +26,7 @@ describe('Stock HTTP contract', () => {
search: jest.fn(), search: jest.fn(),
candles: jest.fn(), candles: jest.fn(),
}; };
const screener = { page: jest.fn(), search: jest.fn() };
const token = new JwtService().sign( const token = new JwtService().sign(
{ sub: 'stock-reader', type: 'access' }, { sub: 'stock-reader', type: 'access' },
{ secret: process.env.JWT_SECRET }, { secret: process.env.JWT_SECRET },
@@ -37,6 +39,8 @@ describe('Stock HTTP contract', () => {
.useValue(stocks) .useValue(stocks)
.overrideProvider(StockIncomeService) .overrideProvider(StockIncomeService)
.useValue(stocks) .useValue(stocks)
.overrideProvider(StockScreenerService)
.useValue(screener)
.compile(); .compile();
app = module.createNestApplication(); app = module.createNestApplication();
configureApp(app); configureApp(app);
@@ -51,12 +55,55 @@ describe('Stock HTTP contract', () => {
}); });
stocks.search.mockResolvedValue([]); stocks.search.mockResolvedValue([]);
stocks.getByTicker.mockResolvedValue({ instrumentUid: uid, slug: 'SBERP' }); stocks.getByTicker.mockResolvedValue({ instrumentUid: uid, slug: 'SBERP' });
screener.search.mockResolvedValue({ asOf: '2026-10-10T12:00:00.000Z', hasNext: false, items: [], page: 1, searchId: 'search-id' });
}); });
afterAll(() => app.close()); afterAll(() => app.close());
it('requires authentication before invoking the use case', async () => { it('requires authentication before invoking the use case', async () => {
await request(app.getHttpServer()).get(`/api/v1/stocks/${uid}`).expect(401); await request(app.getHttpServer()).get(`/api/v1/stocks/${uid}`).expect(401);
expect(stocks.get).not.toHaveBeenCalled(); expect(stocks.get).not.toHaveBeenCalled();
}); });
it.each([
[() => request(app.getHttpServer()).post('/api/v1/stocks/screener/search'), 'post'],
[() => request(app.getHttpServer()).get('/api/v1/stocks/screener/search/search-id?page=1'), 'get'],
])('requires authentication for screener route', async (send) => {
await send().expect(401);
expect(screener.search).not.toHaveBeenCalled();
expect(screener.page).not.toHaveBeenCalled();
});
it('validates screener input and returns raw snapshot pages', async () => {
await request(app.getHttpServer())
.post('/api/v1/stocks/screener/search')
.set('Authorization', `Bearer ${token}`)
.send({ query: ' ', unknown: 'value' })
.expect(400);
expect(screener.search).not.toHaveBeenCalled();
const response = await request(app.getHttpServer())
.post('/api/v1/stocks/screener/search')
.set('Authorization', `Bearer ${token}`)
.send({ query: '' })
.expect(200);
expect(response.body).toEqual({ asOf: '2026-10-10T12:00:00.000Z', hasNext: false, items: [], page: 1, searchId: 'search-id' });
expect(screener.search).toHaveBeenCalledWith('stock-reader', {});
});
it.each([
'/api/v1/stocks/screener/search/search-id',
'/api/v1/stocks/screener/search/search-id?page=0',
'/api/v1/stocks/screener/search/search-id?page=one',
])('validates required screener page on %s', async (path) => {
await request(app.getHttpServer())
.get(path)
.set('Authorization', `Bearer ${token}`)
.expect(400);
expect(screener.page).not.toHaveBeenCalled();
});
it('reads a screener snapshot page with the verified identity', async () => {
await request(app.getHttpServer())
.get('/api/v1/stocks/screener/search/search-id?page=2')
.set('Authorization', `Bearer ${token}`)
.expect(200);
expect(screener.page).toHaveBeenCalledWith('stock-reader', 'search-id', 2);
});
it.each([ it.each([
'/stocks/not-a-uid', '/stocks/not-a-uid',
'/stocks/by-ticker/%20', '/stocks/by-ticker/%20',
+3 -2
View File
@@ -240,7 +240,8 @@ describe('atomic shared candle reconciliation (SQLite)', () => {
data: { fetchedAt: new Date(0) }, data: { fetchedAt: new Date(0) },
}); });
const stale = await reopened.read('owner', input, true); const stale = await reopened.read('owner', input, true);
expect(stale.meta.availability).toBe('complete'); expect(stale.meta.availability).toBe('available');
expect(stale.meta.completeness).toBe('complete');
expect(stale.meta.freshness).toBe('stale'); expect(stale.meta.freshness).toBe('stale');
expect(stale.meta.coverage?.missingRanges).toEqual([]); expect(stale.meta.coverage?.missingRanges).toEqual([]);
expect(stale.meta.coverage?.completeRanges.length).toBeGreaterThan(0); expect(stale.meta.coverage?.completeRanges.length).toBeGreaterThan(0);
@@ -302,7 +303,7 @@ describe('atomic shared candle reconciliation (SQLite)', () => {
expect(await prisma.tbankCandleCoverage.findMany()).toEqual(before); expect(await prisma.tbankCandleCoverage.findMany()).toEqual(before);
expect(await prisma.tbankCandle.count()).toBe(0); expect(await prisma.tbankCandle.count()).toBe(0);
const read = await service.read('owner', input, true); const read = await service.read('owner', input, true);
expect(read.meta.sync.state).toBe('failed'); expect(read.meta.lastAttempt.state).toBe('failed');
expect(read.data?.candles).toEqual([]); expect(read.data?.candles).toEqual([]);
}); });
it.each([ it.each([
+6 -8
View File
@@ -85,14 +85,15 @@ describe('shared catalog discovery (SQLite)', () => {
it('persists normalized query identity and confirmed empty separately from unknown, across users and reopen', async () => { it('persists normalized query identity and confirmed empty separately from unknown, across users and reopen', async () => {
const cold = await service.read('owner', { ...query, query: ' sber ' }); const cold = await service.read('owner', { ...query, query: ' sber ' });
expect(cold.data).toBeNull(); expect(cold.data).toBeNull();
expect(cold.meta.availability).toBe('missing'); expect(cold.meta.availability).toBe('initial-loading');
await settled(); await settled();
await prisma.$disconnect(); await prisma.$disconnect();
service = createService(); service = createService();
const decrypt = jest.spyOn(tokens, 'decrypt'); const decrypt = jest.spyOn(tokens, 'decrypt');
const warm = await service.read('other', query); const warm = await service.read('other', query);
expect(warm.data).toEqual([instrument]); expect(warm.data).toEqual([instrument]);
expect(warm.meta.availability).toBe('complete'); expect(warm.meta.availability).toBe('available');
expect(warm.meta.completeness).toBe('complete');
expect(warm.meta.sync.id).toBeNull(); expect(warm.meta.sync.id).toBeNull();
expect(decrypt).not.toHaveBeenCalled(); expect(decrypt).not.toHaveBeenCalled();
expect(upstream.findShares).toHaveBeenCalledTimes(1); expect(upstream.findShares).toHaveBeenCalledTimes(1);
@@ -297,7 +298,7 @@ describe('shared catalog discovery (SQLite)', () => {
instrument, instrument,
]); ]);
}); });
it('refreshes at the daily boundary while stale observe and fresh reads perform zero RPC', async () => { it('keeps known discovery stable between refresh requests', async () => {
await service.read('owner', query); await service.read('owner', query);
await settled(); await settled();
const now = Date.now(); const now = Date.now();
@@ -311,13 +312,10 @@ describe('shared catalog discovery (SQLite)', () => {
await prisma.tbankCatalogDiscovery.updateMany({ await prisma.tbankCatalogDiscovery.updateMany({
data: { fetchedAt: new Date(now - 86_400_000) }, data: { fetchedAt: new Date(now - 86_400_000) },
}); });
expect((await service.read('other', query, true)).meta.freshness).toBe( expect((await service.read('other', query, true)).meta.freshness).toBe('fresh');
'stale',
);
expect(upstream.findShares).toHaveBeenCalledTimes(1); expect(upstream.findShares).toHaveBeenCalledTimes(1);
await service.read('other', query); await service.read('other', query);
await settled(); expect(upstream.findShares).toHaveBeenCalledTimes(1);
expect(upstream.findShares).toHaveBeenCalledTimes(2);
} finally { } finally {
clock.mockRestore(); clock.mockRestore();
} }
-958
View File
@@ -1,958 +0,0 @@
import type { Server } from 'node:http';
import { INestApplication } from '@nestjs/common';
import { JwtService } from '@nestjs/jwt';
import { Test } from '@nestjs/testing';
import assert from 'node:assert/strict';
import { setTimeout as delay } from 'node:timers/promises';
import request from 'supertest';
import { AppModule } from '../src/app.module';
import { MoexClient } from '../src/integrations/moex/moex.api';
import { NcrRatingsClient } from '../src/integrations/ncr/ncr.api';
import { InstrumentMetadataService } from '../src/integrations/tbank/instrument-metadata.service';
import { TbankTokenService } from '../src/integrations/tbank/tbank-token.service';
import { TbankClient } from '../src/integrations/tbank/tbank.client';
import { configureApp } from '../src/main';
import { CatalogEnrichmentService } from '../src/modules/investments/instruments/application/catalog-enrichment.service';
import {
type BondMetricsResourceDto,
type BondQuoteResourceDto, type BondSearchResourceDto,
} from '../src/modules/investments/instruments/bonds/presentation/bond.dto';
import { CatalogDiscoveryRepository } from '../src/modules/investments/instruments/infrastructure/catalog-discovery.repository';
import {
type StockCandlesResourceDto,
type StockCompanyResourceDto,
type StockQuoteResourceDto,
type StockResourceDto,
type StockSearchResourceDto,
} from '../src/modules/investments/instruments/stocks/presentation/stock.dto';
import { PrismaService } from '../src/prisma/prisma.service';
// HTTP + disposable SQLite: missing/partial results, exact prices, shared source reuse and revision races.
describe('local stock/bond search (SQLite HTTP)', () => {
let app: INestApplication<Server>;
let prisma: PrismaService;
const upstream = {
findShares: jest.fn(),
findBonds: jest.fn(),
getShare: jest.fn(),
getBond: jest.fn(),
getLastPrices: jest.fn(),
getOpenAccounts: jest.fn(),
getAssetDescription: jest.fn(),
getFundamentals: jest.fn(),
getTradingStatus: jest.fn(),
getCandles: jest.fn(),
getBondCoupons: jest.fn(),
};
const moex = { bondYield: jest.fn() };
const ratings = { rating: jest.fn() };
const share = {
uid: 'share',
ticker: 'SBER',
name: 'Synthetic share',
isin: 'SHARE',
classCode: 'TQBR',
instrumentKind: 'INSTRUMENT_TYPE_SHARE',
apiTradeAvailableFlag: true,
};
const bond = {
uid: 'bond',
ticker: 'BOND',
name: 'Synthetic bond',
isin: 'BOND',
classCode: 'TQOB',
instrumentKind: 'INSTRUMENT_TYPE_BOND',
};
const auth = (user = 'owner') =>
`Bearer ${new JwtService().sign({ sub: user, type: 'access' }, { secret: process.env.JWT_SECRET })}`;
const get = (path = 'stocks?query=SBER', user = 'owner') =>
request(app.getHttpServer())
.get(`/api/v1/${path}`)
.set('Authorization', auth(user))
.timeout(1500);
const read = async (observe = true, user = 'owner') =>
(
await get(
`stocks?query=SBER${observe ? '&sync=observe' : ''}`,
user,
).expect(200)
).body as StockSearchResourceDto;
async function settled() {
for (let i = 0; i < 400; i++) {
await delay(5);
if (
!(await prisma.tbankSyncWork.count({ where: { state: 'running' } }))
) {
await delay(10);
if (
!(await prisma.tbankSyncWork.count({ where: { state: 'running' } }))
) {
return;
}
}
}
throw new Error('Catalog sources did not settle');
}
beforeAll(async () => {
const module = await Test.createTestingModule({ imports: [AppModule] })
.overrideProvider(TbankClient)
.useValue(upstream)
.overrideProvider(MoexClient)
.useValue(moex)
.overrideProvider(NcrRatingsClient)
.useValue(ratings)
.compile();
app = module.createNestApplication();
configureApp(app);
await app.init();
await app.listen(0, '127.0.0.1');
prisma = app.get(PrismaService);
for (const id of ['owner', 'other']) {
await prisma.user.create({
data: { id, email: `${id}@example.invalid`, passwordHash: 'synthetic' },
});
}
});
beforeEach(async () => {
await prisma.tbankConnection.deleteMany();
await prisma.tbankInstrumentLookup.deleteMany();
await prisma.tbankInstrument.deleteMany();
await prisma.tbankCatalogDiscovery.deleteMany();
jest.resetAllMocks();
upstream.getAssetDescription.mockResolvedValue('Synthetic company');
upstream.getFundamentals.mockResolvedValue({
assetUid: 'asset',
currency: 'rub',
peRatioTtm: 6,
netIncomeTtm: 10,
});
upstream.getTradingStatus.mockResolvedValue(
'SECURITY_TRADING_STATUS_NORMAL_TRADING',
);
upstream.getCandles.mockResolvedValue([]);
upstream.getBondCoupons.mockResolvedValue([]);
moex.bondYield.mockResolvedValue(null);
ratings.rating.mockResolvedValue(null);
upstream.findShares.mockResolvedValue([share]);
upstream.findBonds.mockResolvedValue([bond]);
upstream.getShare.mockImplementation((_token: string, uid: string) =>
Promise.resolve({
...share,
uid,
currency: 'rub',
sector: 'financial',
assetUid: 'asset',
lot: 10,
}),
);
upstream.getBond.mockResolvedValue({
...bond,
currency: 'rub',
nominal: { currency: 'rub', units: '800' },
aciValue: { currency: 'rub', units: '12', nano: 1 },
});
upstream.getLastPrices.mockImplementation(
(_token: string, uids: string[]) =>
Promise.resolve(
uids.map((uid) => ({
instrumentUid: uid,
price: { units: '99', nano: 100000001 },
time: { seconds: '1700000000' },
})),
),
);
for (const userId of ['owner', 'other']) {
await prisma.tbankConnection.create({
data: {
userId,
revision: `${userId}-revision`,
encryptedToken: app
.get(TbankTokenService)
.encrypt(`synthetic-${userId}`),
connectedAt: new Date(),
},
});
}
});
afterAll(async () => app?.close());
it('reads only requested source identities when batching UIDs, types and periods', async () => {
const repository = app.get(CatalogDiscoveryRepository);
const fetchedAt = new Date();
for (const uid of ['a', 'b']) {
await prisma.tbankInstrument.create({
data: { provider: 'tbank', uid, instrumentType: 'share', metadata: {}, fetchedAt },
});
}
const expected = [
{ instrumentUid: 'a', source: 'tbank', kind: 'detail', rangeKey: 'share' },
{ instrumentUid: 'b', source: 'tbank', kind: 'detail', rangeKey: 'share' },
{ instrumentUid: 'a', source: 'tbank', kind: 'dividends', rangeKey: '2026' },
{ instrumentUid: 'b', source: 'tbank', kind: 'dividends', rangeKey: '2027' },
{ instrumentUid: 'b', source: 'moex', kind: 'bond-events', rangeKey: '' },
{ instrumentUid: 'a', source: 'ncr', kind: 'rating', rangeKey: '' },
];
const distractors = [
{ instrumentUid: 'a', source: 'tbank', kind: 'detail', rangeKey: 'bond' },
{ instrumentUid: 'b', source: 'tbank', kind: 'dividends', rangeKey: '2026' },
{ instrumentUid: 'a', source: 'tbank', kind: 'dividends', rangeKey: '2027' },
{ instrumentUid: 'a', source: 'moex', kind: 'bond-events', rangeKey: '' },
{ instrumentUid: 'b', source: 'tbank', kind: 'bond-events', rangeKey: '' },
{ instrumentUid: 'a', source: 'tbank', kind: 'rating', rangeKey: '' },
];
await prisma.tbankInstrumentSource.createMany({
data: [...expected, ...distractors].map((identity) => ({
...identity, provider: 'tbank', fetchedAt, payload: identity,
})),
});
const rows = await repository.readSources([
{ uid: 'a', instrumentType: 'share', kind: 'detail' },
{ uid: 'b', instrumentType: 'share', kind: 'detail' },
{ uid: 'a', instrumentType: 'share', kind: 'detail' },
{ uid: 'a', instrumentType: 'share', kind: 'dividends', rangeKey: '2026' },
{ uid: 'b', instrumentType: 'share', kind: 'dividends', rangeKey: '2027' },
{ uid: 'b', instrumentType: 'bond', kind: 'bond-events' },
{ uid: 'a', instrumentType: 'bond', kind: 'rating' },
{ uid: 'missing', instrumentType: 'share', kind: 'detail' },
]);
expect(rows.map((row) => row.payload)).toEqual(expect.arrayContaining(expected));
expect(rows).toHaveLength(expected.length);
expect(await repository.readSources([])).toEqual([]);
});
it('returns missing without waiting for discovery, then partial while enrichment is blocked; observe never launches work', async () => {
const found = deferred<(typeof share)[]>();
const detail = deferred<typeof share>();
upstream.findShares.mockReturnValueOnce(found.promise);
upstream.getShare.mockReturnValueOnce(detail.promise);
expect((await read()).data).toBeNull();
expect(upstream.findShares).not.toHaveBeenCalled();
const cold = await read(false);
expect(cold.data).toBeNull();
expect(cold.meta.availability).toBe('missing');
found.resolve([share]);
for (let i = 0; i < 100 && !upstream.getShare.mock.calls.length; i++) {
await delay(5);
}
const partial = await read();
expect(partial.data?.[0].instrumentUid).toBe('share');
expect(partial.meta.availability).toBe('partial');
expect(partial.data?.[0].price).toBeNull();
expect(partial.meta.resourceKey).toBe(cold.meta.resourceKey);
expect(
partial.meta.dependencies?.some(
(dependency) => dependency.sync.state === 'running',
),
).toBe(true);
detail.resolve({ ...share });
await settled();
expect((await read()).meta.availability).toBe('complete');
expect(upstream.findShares).toHaveBeenCalledTimes(1);
});
it('reuses shared success after SQLite reopen and across users without decrypt, GetAccounts or broker RPC', async () => {
await read(false);
await settled();
const before = await read();
await prisma.$disconnect();
const decrypt = jest.spyOn(app.get(TbankTokenService), 'decrypt');
const after = await read(false, 'other');
expect(after.data).toEqual(before.data);
expect(after.data?.[0].price).toEqual({
currency: 'RUB',
value: '99.100000001',
});
expect(
after.meta.dependencies?.every(
(dependency) => dependency.sync.id === null,
),
).toBe(true);
expect(decrypt).not.toHaveBeenCalled();
decrypt.mockRestore();
const metadata = await app
.get(InstrumentMetadataService)
.read('other', { instrumentUid: 'share' });
expect(metadata?.complete).toBe(true);
expect(metadata?.data?.name).toBe(share.name);
expect(upstream.getOpenAccounts).not.toHaveBeenCalled();
expect(upstream.getShare).toHaveBeenCalledTimes(1);
expect(upstream.getLastPrices).toHaveBeenCalledTimes(1);
expect(upstream.findShares).toHaveBeenCalledTimes(1);
});
it('uses one price batch for thirty deduplicated selected UIDs, matching reversed prices by UID', async () => {
upstream.findShares.mockResolvedValue([
{ ...share, uid: 'duplicate', apiTradeAvailableFlag: false },
...Array.from({ length: 35 }, (_, i) => ({
...share,
uid: `s${String(i).padStart(2, '0')}`,
isin: i ? undefined : 'SHARE',
})),
]);
upstream.getLastPrices.mockImplementation(
(_token: string, uids: string[]) =>
Promise.resolve(
uids.toReversed().map((uid, i) => ({
instrumentUid: uid,
price: { units: String(i + 1) },
})),
),
);
await read(false);
await settled();
const response = await read();
expect(response.data).toHaveLength(30);
expect(response.data?.[0].instrumentUid).toBe('s00');
expect(response.data?.[0].price?.value).toBe('30');
expect(upstream.getLastPrices).toHaveBeenCalledTimes(1);
expect(upstream.getShare).toHaveBeenCalledTimes(30);
expect(upstream.findShares).toHaveBeenCalledTimes(1);
});
it('keeps confirmed empty separate from unknown queries and does not enrich an empty result', async () => {
upstream.findShares.mockResolvedValue([]);
expect((await read(false)).data).toBeNull();
await settled();
expect((await read()).data).toEqual([]);
const other = (await get('stocks?query=GAZP&sync=observe').expect(200))
.body as StockSearchResourceDto;
expect(other.data).toBeNull();
expect(other.meta.availability).toBe('missing');
expect(upstream.getShare).not.toHaveBeenCalled();
expect(upstream.getLastPrices).not.toHaveBeenCalled();
});
it('retains catalog identity on enrichment failures, rejects mismatched detail UIDs and never returns fake prices', async () => {
upstream.getShare.mockResolvedValue({ ...share, uid: 'another' });
upstream.getLastPrices.mockRejectedValue(new Error('outage'));
await read(false);
await settled();
const result = await read();
expect(result.data?.[0]).toMatchObject({
instrumentUid: 'share',
ticker: 'SBER',
price: null,
});
expect(result.meta.availability).toBe('partial');
expect(
result.meta.dependencies?.filter((item) => item.sync.state === 'failed'),
).toHaveLength(2);
await read();
expect(upstream.getShare).toHaveBeenCalledTimes(1);
expect(upstream.getLastPrices).toHaveBeenCalledTimes(1);
});
it('keeps bond percent separate from money and preserves exact nominal and ACI', async () => {
await get('bonds?query=BOND').expect(200);
await settled();
const result = (await get('bonds?query=BOND&sync=observe').expect(200))
.body as BondSearchResourceDto;
expect(result.data?.[0]).toMatchObject({
pricePercent: '99.100000001',
nominal: { currency: 'RUB', value: '800' },
aci: { currency: 'RUB', value: '12.000000001' },
});
expect(result.data?.[0]).not.toHaveProperty('price');
expect(upstream.findShares).not.toHaveBeenCalled();
});
it('refreshes only prices at the minute boundary and preserves old prices after failure', async () => {
await read(false);
await settled();
const before = await read();
await prisma.tbankInstrumentSource.updateMany({
where: { kind: 'last-price' },
data: { fetchedAt: new Date(Date.now() - 60_000) },
});
upstream.getLastPrices.mockRejectedValue(new Error('outage'));
await read();
expect(upstream.getLastPrices).toHaveBeenCalledTimes(1);
await read(false);
await settled();
const failed = await read();
expect(failed.data).toEqual(before.data);
expect(failed.meta.freshness).toBe('stale');
expect(upstream.getShare).toHaveBeenCalledTimes(1);
expect(upstream.findShares).toHaveBeenCalledTimes(1);
expect(upstream.getLastPrices).toHaveBeenCalledTimes(2);
});
it('coalesces sources across different queries and users, with private handles and no private fields in shared rows', async () => {
const detail = deferred<typeof share>();
upstream.getShare.mockReturnValueOnce(detail.promise);
await read(false);
for (let i = 0; i < 100 && !upstream.getShare.mock.calls.length; i++) {
await delay(5);
}
await get('stocks?query=OTHER', 'other').expect(200);
for (
let i = 0;
i < 100 &&
(await prisma.tbankSyncWork.count({ where: { userId: 'other' } })) < 3;
i++
) {
await delay(5);
}
detail.resolve({
...share,
currency: 'rub',
quantity: 'private',
estimatedAmount: 'private',
} as typeof share);
await settled();
expect(upstream.getShare).toHaveBeenCalledTimes(1);
expect(upstream.getLastPrices).toHaveBeenCalledTimes(1);
const owner = await read();
const other = (await get('stocks?query=OTHER&sync=observe', 'other'))
.body as StockSearchResourceDto;
expect(owner.meta.dependencies?.at(-1)?.sync.id).not.toBe(
other.meta.dependencies?.at(-1)?.sync.id,
);
expect(
JSON.stringify(await prisma.tbankInstrumentSource.findMany()),
).not.toMatch(/quantity|estimatedAmount|private|encryptedToken/);
});
it.each(['delete', 'replace'])(
'discards delayed enrichment after connection %s',
async (action) => {
const detail = deferred<typeof share>();
upstream.getShare.mockReturnValueOnce(detail.promise);
await read(false);
for (let i = 0; i < 100 && !upstream.getShare.mock.calls.length; i++) {
await delay(5);
}
await prisma.tbankConnection.delete({ where: { userId: 'owner' } });
if (action === 'replace') {
await prisma.tbankConnection.create({
data: {
userId: 'owner',
revision: 'new',
encryptedToken: app.get(TbankTokenService).encrypt('new'),
connectedAt: new Date(),
},
});
}
detail.resolve(share);
await settled();
expect(
await prisma.tbankInstrumentSource.count({ where: { kind: 'detail' } }),
).toBe(0);
expect((await read(true, 'other')).data?.[0].instrumentUid).toBe('share');
},
);
it('recovers abandoned enrichment only on an ordinary request, and fences the superseded attempt', async () => {
await read(false);
await settled();
await prisma.tbankInstrumentSource.deleteMany({
where: { kind: 'detail' },
});
const repository = app.get(CatalogDiscoveryRepository);
const input = {
instrumentType: 'share',
uid: 'share',
kind: 'detail',
} as const;
const old = await repository.reserve(
{ userId: 'owner', revision: 'owner-revision' },
input,
'old-process',
new Set(),
);
assert(old);
await read();
expect(upstream.getShare).toHaveBeenCalledTimes(1);
await read(false);
await settled();
expect(upstream.getShare).toHaveBeenCalledTimes(2);
expect(
await repository.publishSource(
{
userId: 'owner',
revision: 'owner-revision',
id: old.work.id,
attemptId: old.work.attemptId,
},
input,
{ instrument: { ...share, name: 'Late' } },
new Date(),
),
).toBe(false);
});
it('preserves fresh successful sources on failed force and isolates credential failures', async () => {
await read(false);
await settled();
const before = await read();
upstream.getShare.mockRejectedValue(new Error('outage'));
upstream.getLastPrices.mockRejectedValue(new Error('outage'));
await app
.get(CatalogEnrichmentService)
.requestMany('owner', 'owner-revision', 'share', ['share'], true);
await settled();
const after = await read();
expect(after.data).toEqual(before.data);
expect(after.meta.freshness).toBe('fresh');
expect(after.meta.lastSuccessfulAt).toBe(before.meta.lastSuccessfulAt);
expect(
(await read(true, 'other')).meta.dependencies?.every(
(item) => item.sync.errorCode === null,
),
).toBe(true);
});
it('rejects every price in a batch if its initiating attempt is superseded during the RPC', async () => {
const prices =
deferred<{ instrumentUid: string; price: { units: string } }[]>();
upstream.findShares.mockResolvedValue([
share,
{ ...share, uid: 'second', isin: 'SECOND' },
]);
upstream.getLastPrices.mockReturnValueOnce(prices.promise);
await read(false);
for (let i = 0; i < 100 && !upstream.getLastPrices.mock.calls.length; i++) {
await delay(5);
}
const initiating = await prisma.tbankSyncWork.findFirstOrThrow({
where: { userId: 'owner', scopeKey: { contains: 'last-price' } },
orderBy: { createdAt: 'asc' },
});
await prisma.tbankSyncWork.update({
where: { id: initiating.id },
data: { attemptId: 'superseding-attempt', state: 'failed' },
});
prices.resolve([
{ instrumentUid: 'share', price: { units: '100' } },
{ instrumentUid: 'second', price: { units: '200' } },
]);
await settled();
expect(
await prisma.tbankInstrumentSource.count({
where: { kind: 'last-price' },
}),
).toBe(0);
expect((await read()).data?.every((item) => item.price === null)).toBe(
true,
);
});
it('refreshes stale detail independently of still-fresh prices and discovery', async () => {
await read(false);
await settled();
await prisma.tbankInstrumentSource.updateMany({
where: { kind: 'detail' },
data: { fetchedAt: new Date(Date.now() - 86_400_000) },
});
await read();
expect(upstream.getShare).toHaveBeenCalledTimes(1);
await read(false);
await settled();
expect(upstream.getShare).toHaveBeenCalledTimes(2);
expect(upstream.getLastPrices).toHaveBeenCalledTimes(1);
expect(upstream.findShares).toHaveBeenCalledTimes(1);
});
it('reads exact UID locally without waiting, batches twenty reads and keeps UID accessible when alias is ambiguous', async () => {
const uid = '00000000-0000-4000-8000-000000000001';
const blocked = deferred<typeof share>();
upstream.getShare.mockReturnValueOnce(blocked.promise);
upstream.findShares.mockResolvedValue([
{ ...share, uid },
{ ...share, uid: 'other-listing', isin: 'OTHER' },
]);
expect(
((await get(`stocks/${uid}?sync=observe`)).body as StockResourceDto).data,
).toBeNull();
expect(upstream.getShare).not.toHaveBeenCalled();
const reads = await Promise.all(
Array.from({ length: 20 }, () => get(`stocks/${uid}`).expect(200)),
);
expect(
reads.every(
(response) => (response.body as StockResourceDto).data === null,
),
).toBe(true);
expect(upstream.getLastPrices).not.toHaveBeenCalled();
blocked.resolve({ ...share, uid });
await settled();
const first = (await get(`stocks/${uid}`).expect(200))
.body as StockResourceDto;
expect(first.data?.instrumentUid).toBe(uid);
await settled();
const exact = (await get(`stocks/${uid}?sync=observe`).expect(200))
.body as StockResourceDto;
expect(exact.data?.slug).toBe(uid);
await get('stocks/by-ticker/SBER?sync=observe').expect(409);
expect(upstream.getShare).toHaveBeenCalledTimes(1);
});
it('resolves exact ticker before ISIN selection and does not mistake search completeness for ticker coverage', async () => {
const uid = '00000000-0000-4000-8000-000000000002';
upstream.findShares.mockResolvedValue([
{ ...share, uid: 'preferred', ticker: 'SBERP' },
{ ...share, uid, ticker: 'SBER', apiTradeAvailableFlag: false },
]);
await read(false);
await settled();
expect(
(
(await get('stocks/by-ticker/SBER?sync=observe'))
.body as StockResourceDto
).data,
).toBeNull();
const cold = (await get('stocks/by-ticker/sber').expect(200))
.body as StockResourceDto;
expect(cold.data).toBeNull();
await settled();
const resolved = (
await get('stocks/by-ticker/SBER?sync=observe', 'other').expect(200)
).body as StockResourceDto;
expect(resolved.data).toMatchObject({ instrumentUid: uid, slug: 'SBER' });
expect(upstream.findShares).toHaveBeenCalledTimes(2);
upstream.findShares.mockResolvedValue([]);
await get('stocks/by-ticker/NONE').expect(200);
await settled();
await get('stocks/by-ticker/NONE?sync=observe').expect(404);
});
it('keeps known typed UID misses separate from unknown and from generic UID lookup', async () => {
const uid = '00000000-0000-4000-8000-000000000003';
upstream.getBond.mockResolvedValue(null);
const cold = (await get(`bonds/${uid}`).expect(200))
.body as StockResourceDto;
expect(cold.data).toBeNull();
await settled();
await get(`bonds/${uid}?sync=observe`).expect(404);
expect(
await prisma.tbankInstrumentLookup.count({ where: { identifier: uid } }),
).toBe(0);
expect(upstream.getLastPrices).not.toHaveBeenCalled();
});
it('loads company and fundamentals locally, shares successes and keeps old metrics on failed refresh', async () => {
const uid = '00000000-0000-4000-8000-000000000011';
const company = deferred<string>();
upstream.getAssetDescription.mockReturnValueOnce(company.promise);
const cold = (await get(`stocks/${uid}/company`).expect(200))
.body as StockCompanyResourceDto;
expect(cold.data).toBeNull();
for (
let i = 0;
i < 100 && !upstream.getAssetDescription.mock.calls.length;
i++
) {
await delay(5);
}
const partial = (
await get(`stocks/${uid}/company?sync=observe`).expect(200)
).body as StockCompanyResourceDto;
expect(partial.meta.availability).toBe('partial');
company.resolve('Synthetic company');
await settled();
const warm = (await get(`stocks/${uid}/company`, 'other').expect(200))
.body as StockCompanyResourceDto;
expect(warm.data?.description).toBe('Synthetic company');
expect(warm.data?.metrics.map((item) => item.key)).toEqual([
'peRatioTtm',
'netIncomeTtm',
]);
expect(upstream.getFundamentals).toHaveBeenCalledTimes(1);
expect(upstream.getAssetDescription).toHaveBeenCalledTimes(1);
upstream.getFundamentals.mockRejectedValue(new Error('outage'));
upstream.getAssetDescription.mockRejectedValue(new Error('outage'));
await app
.get(CatalogEnrichmentService)
.requestMany('owner', 'owner-revision', 'share', [uid], true, [
'description',
'fundamentals',
]);
await settled();
const failed = (await get(`stocks/${uid}/company?sync=observe`).expect(200))
.body as StockCompanyResourceDto;
expect(failed.data?.metrics).toEqual(warm.data?.metrics);
expect(failed.data?.description).toBe(warm.data?.description);
expect(failed.data?.observedAt).toBe(warm.data?.observedAt);
expect(failed.data?.unavailable).toEqual(['description', 'fundamentals']);
});
it('reads stock quotes without waiting for blocked prices and reuses quote components across users', async () => {
const uid = '00000000-0000-4000-8000-000000000012';
const prices = deferred<
{
instrumentUid: string;
price: { units: string };
time: { seconds: string };
}[]
>();
upstream.getLastPrices.mockReturnValueOnce(prices.promise);
upstream.getCandles.mockResolvedValue([
{
time: { seconds: '1699833600' },
close: { units: '90' },
isComplete: true,
},
]);
const cold = (await get(`stocks/${uid}/quote`).expect(200))
.body as StockQuoteResourceDto;
expect(cold.meta.availability).toBe('missing');
for (let i = 0; i < 100 && !upstream.getLastPrices.mock.calls.length; i++) {
await delay(5);
}
const pending = (await get(`stocks/${uid}/quote?sync=observe`).expect(200))
.body as StockQuoteResourceDto;
expect(pending.data?.price).toBeNull();
prices.resolve([
{
instrumentUid: uid,
price: { units: '99' },
time: { seconds: '1700000000' },
},
]);
await settled();
const warm = (await get(`stocks/${uid}/quote`, 'other').expect(200))
.body as StockQuoteResourceDto;
expect(warm.data).toMatchObject({
price: { currency: 'RUB', value: '99' },
previousClose: { currency: 'RUB', value: '90' },
change: { currency: 'RUB', value: '9' },
changePercent: '10',
});
expect(upstream.getLastPrices).toHaveBeenCalledTimes(1);
expect(upstream.getTradingStatus).toHaveBeenCalledTimes(1);
expect(upstream.getCandles).toHaveBeenCalledTimes(1);
expect(
await prisma.tbankCandle.count({ where: { instrumentUid: uid } }),
).toBe(1);
await app
.get(CatalogEnrichmentService)
.requestMany('owner', 'owner-revision', 'share', [uid], true, [
'comparison',
]);
await settled();
expect(upstream.getCandles).toHaveBeenCalledTimes(1);
});
it('keeps bond quote money exact and coupon metrics available when optional MOEX/NCR fail', async () => {
const uid = '00000000-0000-4000-8000-000000000013';
upstream.getBond.mockResolvedValue({
...bond,
uid,
currency: 'rub',
nominal: { currency: 'rub', units: '800' },
aciValue: { currency: 'rub', units: '12', nano: 1 },
riskLevel: 'RISK_LEVEL_LOW',
});
upstream.getBondCoupons.mockResolvedValue([
{
couponDate: {
seconds: String(Math.floor(Date.now() / 1000) + 86400 * 7),
},
payOneBond: { currency: 'rub', units: '30' },
couponPeriod: 90,
},
]);
moex.bondYield.mockRejectedValue(new Error('MOEX outage'));
ratings.rating.mockRejectedValue(new Error('NCR outage'));
await get(`bonds/${uid}/metrics`).expect(200);
await settled();
const metrics = (await get(`bonds/${uid}/metrics?sync=observe`).expect(200))
.body as BondMetricsResourceDto;
expect(metrics.meta.availability).toBe('partial');
expect(metrics.data).toMatchObject({
coupon: { currency: 'RUB', value: '30' },
creditRating: null,
effectiveYield: null,
});
expect(metrics.data?.couponYieldPercent).not.toBeNull();
expect(moex.bondYield).toHaveBeenCalledWith('BOND', 'BOND', 'TQOB');
expect(ratings.rating).toHaveBeenCalledWith('BOND');
const quote = (await get(`bonds/${uid}/quote?sync=observe`).expect(200))
.body as BondQuoteResourceDto;
expect(quote.data).toMatchObject({
pricePercent: '99.100000001',
cleanPrice: { currency: 'RUB', value: '792.800000008' },
dirtyPrice: { currency: 'RUB', value: '804.800000009' },
});
expect(
await prisma.tbankInstrumentSource.count({ where: { source: 'moex' } }),
).toBe(0);
expect(upstream.getBond).toHaveBeenCalledTimes(1);
});
it('serves saved candle envelopes without waiting for RPC and never runs observe work', async () => {
const uid = '00000000-0000-4000-8000-000000000017';
upstream.getShare.mockResolvedValue({
...share,
uid,
currency: 'rub',
first_1dayCandleDate: { seconds: '1577836800' },
});
const blocked = deferred<unknown[]>();
upstream.getCandles.mockReturnValueOnce(blocked.promise);
const cold = (await get(`stocks/${uid}/candles?period=all`).expect(200))
.body as StockCandlesResourceDto;
expect(cold.data).toBeNull();
await get(`stocks/${uid}/candles?period=all&sync=observe`).expect(200);
blocked.resolve([]);
await settled();
const warm = (
await get(`stocks/${uid}/candles?period=all&sync=observe`).expect(200)
).body as StockCandlesResourceDto;
expect(warm.data).toMatchObject({
candles: [],
currency: 'RUB',
historyFrom: '2020-01-01T00:00:00.000Z',
});
expect(warm.meta.availability).toBe('complete');
const count = upstream.getCandles.mock.calls.length;
await get(`stocks/${uid}/candles?period=all`, 'other').expect(200);
expect(upstream.getCandles).toHaveBeenCalledTimes(count);
await get(`stocks/${uid}/candles?sync=invalid`).expect(400);
});
it('keeps unknown all-history bounds unavailable instead of confirming empty history', async () => {
const uid = '00000000-0000-4000-8000-000000000018';
await get(`stocks/${uid}/candles?period=all`).expect(200);
await settled();
const response = await get(
`stocks/${uid}/candles?period=all&sync=observe`,
).expect(503);
expect(JSON.stringify(response.body)).toContain(
'STOCK_HISTORY_START_UNAVAILABLE',
);
expect(upstream.getCandles).not.toHaveBeenCalled();
expect(await prisma.tbankCandleCoverage.count()).toBe(0);
});
it('coalesces a second user arriving while SQLite publication is pending', async () => {
const uid = '00000000-0000-4000-8000-000000000016';
const repository = app.get(CatalogDiscoveryRepository);
const original = repository.publishSource.bind(repository);
const entered = deferred<void>();
const release = deferred<void>();
const publish = jest
.spyOn(repository, 'publishSource')
.mockImplementationOnce(async (...args) => {
entered.resolve();
await release.promise;
return original(...args);
});
const service = app.get(CatalogEnrichmentService);
try {
await service.requestMany(
'owner',
'owner-revision',
'share',
[uid],
false,
['detail'],
);
await entered.promise;
await service.requestMany(
'other',
'other-revision',
'share',
[uid],
false,
['detail'],
);
release.resolve();
await settled();
expect(upstream.getShare).toHaveBeenCalledTimes(1);
const work = await prisma.tbankSyncWork.findMany();
expect(work).toHaveLength(2);
expect(work.every((row) => row.state === 'complete')).toBe(true);
} finally {
release.resolve();
publish.mockRestore();
}
});
it('persists MOEX/NCR independently and retains their exact values after failed force', async () => {
const uid = '00000000-0000-4000-8000-000000000015';
upstream.getBond.mockResolvedValue({ ...bond, uid });
const effectiveYield = {
value: '13.123456789',
pricePercent: '99.100000001',
date: '2028-01-01',
dateType: 'MATDATE',
tradeAt: '2026-09-25T10:00:00.000Z',
};
const creditRating = {
agency: 'NCR',
value: 'AAA.ru',
status: 'active',
date: '2026-09-25',
sourceUrl: 'https://ratings.ru/ratings/issues/',
observedAt: '2026-09-25T10:00:00.000Z',
};
moex.bondYield.mockResolvedValue(effectiveYield);
ratings.rating.mockResolvedValue(creditRating);
await get(`bonds/${uid}/metrics`).expect(200);
await settled();
const before = (await get(`bonds/${uid}/metrics`, 'other').expect(200))
.body as BondMetricsResourceDto;
expect(before.data).toMatchObject({ effectiveYield, creditRating });
expect(moex.bondYield).toHaveBeenCalledTimes(1);
expect(ratings.rating).toHaveBeenCalledTimes(1);
const saved = await prisma.tbankInstrumentSource.findMany({
where: { instrumentUid: uid, source: { in: ['moex', 'ncr'] } },
});
expect(saved).toHaveLength(2);
await prisma.$disconnect();
moex.bondYield.mockRejectedValue(new Error('MOEX outage'));
ratings.rating.mockRejectedValue(new Error('NCR outage'));
await app
.get(CatalogEnrichmentService)
.requestMany('owner', 'owner-revision', 'bond', [uid], true, [
'yield',
'rating',
]);
await settled();
const after = (await get(`bonds/${uid}/metrics?sync=observe`).expect(200))
.body as BondMetricsResourceDto;
expect(after.data).toMatchObject({ effectiveYield, creditRating });
expect(
await prisma.tbankInstrumentSource.findMany({
where: { instrumentUid: uid, source: { in: ['moex', 'ncr'] } },
}),
).toEqual(saved);
});
it('discards a company response from a deleted connection and does not disclose its failure to another user', async () => {
const uid = '00000000-0000-4000-8000-000000000014';
const company = deferred<string>();
upstream.getAssetDescription.mockReturnValueOnce(company.promise);
await get(`stocks/${uid}/company`).expect(200);
for (
let i = 0;
i < 100 && !upstream.getAssetDescription.mock.calls.length;
i++
) {
await delay(5);
}
await prisma.tbankConnection.delete({ where: { userId: 'owner' } });
company.resolve('Late company');
await settled();
expect(
await prisma.tbankInstrumentSource.count({
where: { kind: 'description' },
}),
).toBe(0);
const other = (
await get(`stocks/${uid}/company?sync=observe`, 'other').expect(200)
).body as StockCompanyResourceDto;
expect(
other.meta.dependencies?.every((item) => item.sync.errorCode === null),
).toBe(true);
});
it('enforces access and query validation without broker calls', async () => {
await request(app.getHttpServer())
.get('/api/v1/stocks?query=SBER')
.expect(401);
await get('stocks?query=x').expect(400);
await get('bonds?query=BOND&sync=force').expect(400);
await prisma.tbankConnection.delete({ where: { userId: 'owner' } });
await get().expect(409);
await prisma.tbankConnection.update({
where: { userId: 'other' },
data: { invalidToken: true },
});
await get('stocks?query=SBER', 'other').expect(400);
expect(upstream.findShares).not.toHaveBeenCalled();
expect(upstream.findBonds).not.toHaveBeenCalled();
});
});
function deferred<T>() {
let resolve: (value: T) => void = () => {
throw new Error('Uninitialized deferred');
};
const promise = new Promise<T>((done) => {
resolve = done;
});
return { promise, resolve };
}
+659
View File
@@ -0,0 +1,659 @@
import { ConfigService } from '@nestjs/config';
import assert from 'node:assert/strict';
import { randomUUID } from 'node:crypto';
import { setImmediate as immediate } from 'node:timers/promises';
import { TbankAccessService } from '../src/integrations/tbank/tbank-access.service';
import { TbankTokenService } from '../src/integrations/tbank/tbank-token.service';
import { CatalogEnrichmentService } from '../src/modules/investments/instruments/application/catalog-enrichment.service';
import { CatalogDiscoveryRepository } from '../src/modules/investments/instruments/infrastructure/catalog-discovery.repository';
import { BrokerageOperationsSyncService } from '../src/modules/investments/operations/application/brokerage-operations-sync.service';
import { BrokerageOperationsRepository } from '../src/modules/investments/operations/infrastructure/brokerage-operations.repository';
import { TbankHourlyScheduler } from '../src/modules/investments/tbank-sync/tbank-hourly.scheduler';
import { TbankSyncRepository } from '../src/modules/investments/tbank-sync/tbank-sync.repository';
import { PrismaService } from '../src/prisma/prisma.service';
type HourlyProgressView = {
result?: string;
sections: { accountId?: string; errorCode: null | string; section: string }[];
};
// Real SQLite proves reservation, outcome derivation, recovery and fences;
// the step owners are controlled so each failure mode can be injected.
describe('hourly attempt lifecycle (SQLite)', () => {
const prisma = new PrismaService(
new ConfigService({ DATABASE_URL: process.env.DATABASE_URL }),
);
const tokens = new TbankTokenService(
new ConfigService({
TOKEN_ENCRYPTION_KEY: process.env.TOKEN_ENCRYPTION_KEY,
}) as never,
);
const repository = new TbankSyncRepository(prisma);
const access = { local: jest.fn() };
const membership = { request: jest.fn() };
const portfolio = { request: jest.fn() };
const operations = { request: jest.fn() };
const events = { refresh: jest.fn(), get: jest.fn() };
const logger = { event: jest.fn() };
const createScheduler = () =>
new TbankHourlyScheduler(
prisma,
repository,
access as never,
membership as never,
portfolio as never,
operations as never,
events as never,
logger as never,
);
const USER = 'hourly-owner';
const REVISION = 'hourly-revision';
function child(
state: 'complete' | 'failed' | 'running',
errorCode: null | string,
scopeKey: string,
) {
return prisma.tbankSyncWork.create({
data: {
id: randomUUID(),
userId: USER,
revision: REVISION,
scopeKey: `test:${scopeKey}:${randomUUID()}`,
attemptId: randomUUID(),
processRunId: 'child-process',
priority: 0,
intent: {},
progress: {},
state,
errorCode,
},
});
}
function hourlyRow() {
return prisma.tbankSyncWork.findFirst({
where: { userId: USER, scopeKey: 'hourly' },
});
}
async function pump(turns = 20) {
for (let i = 0; i < turns; i++) {
await immediate();
await jest.advanceTimersByTimeAsync(50);
}
}
async function runWalk(scheduler: TbankHourlyScheduler, advanceMs = 50) {
await scheduler.tick();
for (let i = 0; i < 400; i++) {
await immediate();
await jest.advanceTimersByTimeAsync(advanceMs);
const row = await hourlyRow();
if (row && row.state !== 'running') return row;
}
throw new Error('hourly attempt did not settle');
}
// Sequential per-step bounds: advance past each bound until the attempt
// closes. Covers walks whose awaits hang before any durable step row exists.
async function runBoundedWalk(scheduler: TbankHourlyScheduler) {
await scheduler.tick();
for (let i = 0; i < 12; i++) {
await pump(5);
await jest.advanceTimersByTimeAsync(16 * 60 * 1000);
const row = await hourlyRow();
if (row && row.state !== 'running') return row;
}
throw new Error('hourly attempt did not settle');
}
beforeAll(async () => {
await prisma.$connect();
await prisma.user.create({
data: { id: USER, email: 'hourly@example.invalid', passwordHash: 'x' },
});
});
afterAll(async () => prisma.$disconnect());
beforeEach(async () => {
jest.resetAllMocks();
await prisma.tbankSyncWork.deleteMany();
await prisma.tbankConnection.deleteMany();
await prisma.tbankConnection.create({
data: {
userId: USER,
revision: REVISION,
encryptedToken: tokens.encrypt('synthetic'),
connectedAt: new Date(),
},
});
jest.useFakeTimers({
doNotFake: ['queueMicrotask', 'setImmediate', 'nextTick'],
});
access.local.mockResolvedValue({
revision: REVISION,
accounts: [{ accountId: 'a' }, { accountId: 'b' }],
});
membership.request.mockImplementation(async () => {
const row = await child('complete', null, 'membership');
return { id: row.id, done: Promise.resolve() };
});
portfolio.request.mockImplementation(
async (_userId: string, _revision: string, accountId: string) => {
const row = await child('complete', null, `portfolio:${accountId}`);
return { id: row.id, done: Promise.resolve() };
},
);
operations.request.mockImplementation(async () => {
const row = await child('complete', null, 'operations');
return row.id;
});
events.refresh.mockResolvedValue([]);
events.get.mockResolvedValue(undefined);
});
afterEach(() => {
jest.useRealTimers();
jest.restoreAllMocks();
});
it('settles the first walk succeeded only when every step row completed', async () => {
const row = await runWalk(createScheduler());
expect(row.state).toBe('complete');
const progress = row.progress as HourlyProgressView;
expect(progress.result).toBe('succeeded');
expect(progress.sections).toEqual([]);
const read = await repository.hourly(USER);
expect(read?.state).toBe('succeeded');
});
it('settles partial and keeps successful steps when one account fails', async () => {
portfolio.request.mockImplementation(
async (_userId: string, _revision: string, accountId: string) => {
const failed = accountId === 'b';
const row = await child(
failed ? 'failed' : 'complete',
failed ? 'TBANK_UNAVAILABLE' : null,
`portfolio:${accountId}`,
);
return { id: row.id, done: Promise.resolve() };
},
);
const row = await runWalk(createScheduler());
const progress = row.progress as HourlyProgressView;
expect(progress.result).toBe('partial');
expect(progress.sections).toHaveLength(1);
expect(progress.sections[0].section).toBe('portfolio');
expect(progress.sections[0].accountId).toBe('b');
expect(progress.sections[0].errorCode).toBe('TBANK_UNAVAILABLE');
});
it('settles failed when the membership step fails before any update', async () => {
membership.request.mockImplementation(async () => {
const row = await child('failed', 'INVALID_TBANK_TOKEN', 'membership');
return { id: row.id, done: Promise.resolve() };
});
const row = await runWalk(createScheduler());
const progress = row.progress as HourlyProgressView;
expect(progress.result).toBe('failed');
expect(row.errorCode).toBe('INVALID_TBANK_TOKEN');
expect(progress.sections[0].section).toBe('membership');
});
it('never succeeds when no required step succeeded', async () => {
membership.request.mockImplementation(async () => {
const row = await child('failed', 'TBANK_UNAVAILABLE', 'membership');
return { id: row.id, done: Promise.resolve() };
});
portfolio.request.mockImplementation(
async (_userId: string, _revision: string, accountId: string) => {
const row = await child(
'failed',
'TBANK_UNAVAILABLE',
`portfolio:${accountId}`,
);
return { id: row.id, done: Promise.resolve() };
},
);
operations.request.mockRejectedValue(new Error('db'));
events.refresh.mockRejectedValue(new Error('db'));
const row = await runWalk(createScheduler());
const progress = row.progress as HourlyProgressView;
expect(progress.result).toBe('failed');
expect(progress.result).not.toBe('succeeded');
assert.ok(progress.sections.length > 0);
});
it('caps the outcome at partial when the coordinator itself breaks', async () => {
access.local.mockRejectedValueOnce(new Error('db'));
const row = await runWalk(createScheduler());
const progress = row.progress as HourlyProgressView;
expect(progress.result).toBe('partial');
expect(row.errorCode).toBe('INTERRUPTED');
});
it('records a section whose dispatch failed as a problem', async () => {
events.refresh.mockRejectedValue(new Error('db'));
const row = await runWalk(createScheduler());
const progress = row.progress as HourlyProgressView;
expect(progress.result).toBe('partial');
const eventsSection = progress.sections.find(
(entry) => entry.section === 'events',
);
assert.ok(eventsSection);
expect(eventsSection.errorCode).toBe('TBANK_UNAVAILABLE');
expect(row.errorCode).toBe('TBANK_UNAVAILABLE');
});
it('closes a stuck step as DEADLINE and keeps its row running', async () => {
operations.request.mockImplementation(async () => {
const row = await child('running', null, 'operations');
return row.id;
});
const scheduler = createScheduler();
await scheduler.tick();
await pump();
await jest.advanceTimersByTimeAsync(16 * 60 * 1000);
await pump(20);
const row = await hourlyRow();
assert.ok(row);
expect(row.state).not.toBe('running');
const progress = row.progress as HourlyProgressView;
expect(progress.result).toBe('partial');
assert.ok(
progress.sections.some(
(entry) =>
entry.section === 'operations' && entry.errorCode === 'DEADLINE',
),
);
const stuck = await prisma.tbankSyncWork.findFirst({
where: { userId: USER, state: 'running' },
});
assert.ok(stuck, 'the wait bound does not cancel the upstream call');
});
it('settles the attempt when a portfolio dispatch never resolves', async () => {
portfolio.request.mockImplementation(() => new Promise(() => undefined));
const row = await runBoundedWalk(createScheduler());
const progress = row.progress as HourlyProgressView;
expect(progress.result).toBe('partial');
const problems = progress.sections.filter(
(entry) => entry.section === 'portfolio',
);
expect(problems).toHaveLength(2);
expect(problems[0].errorCode).toBe('TBANK_UNAVAILABLE');
});
it('settles the attempt when the membership dispatch never resolves', async () => {
membership.request.mockImplementation(() => new Promise(() => undefined));
const row = await runBoundedWalk(createScheduler());
const progress = row.progress as HourlyProgressView;
expect(progress.result).toBe('partial');
expect(progress.sections).toHaveLength(1);
expect(progress.sections[0].section).toBe('membership');
expect(progress.sections[0].errorCode).toBe('TBANK_UNAVAILABLE');
});
it('bounds a membership step promise that never resolves', async () => {
membership.request.mockImplementation(async () => {
const row = await child('running', null, 'membership');
return { id: row.id, done: new Promise(() => undefined) };
});
const row = await runBoundedWalk(createScheduler());
const progress = row.progress as HourlyProgressView;
expect(progress.result).toBe('partial');
const membershipSection = progress.sections.find(
(entry) => entry.section === 'membership',
);
expect(membershipSection?.errorCode).toBe('DEADLINE');
});
it('settles the attempt when a post-wait refresh never resolves', async () => {
events.get.mockImplementation(() => new Promise(() => undefined));
const row = await runBoundedWalk(createScheduler());
const progress = row.progress as HourlyProgressView;
expect(progress.result).toBe('succeeded');
expect(progress.sections).toEqual([]);
});
it('recovers a lost settle and unblocks the next attempt at its term', async () => {
const settleSpy = jest
.spyOn(repository, 'settleHourly')
.mockRejectedValueOnce(new Error('db'));
const scheduler = createScheduler();
await scheduler.tick();
for (let i = 0; i < 40 && settleSpy.mock.calls.length === 0; i++) {
await immediate();
await jest.advanceTimersByTimeAsync(50);
}
expect(settleSpy.mock.calls).toHaveLength(1);
const open = await hourlyRow();
expect(open?.state).toBe('running');
await pump(20);
await scheduler.tick();
await pump(20);
const closed = await hourlyRow();
expect(closed?.state).toBe('complete');
expect((closed?.progress as HourlyProgressView).result).toBe('partial');
expect(closed?.errorCode).toBe('INTERRUPTED');
await jest.advanceTimersByTimeAsync(61 * 60 * 1000);
await scheduler.tick();
await pump(30);
const next = await hourlyRow();
expect(next?.attemptId).not.toBe(closed?.attemptId);
expect(next?.state).not.toBe('running');
expect((next?.progress as HourlyProgressView).result).toBe('succeeded');
});
it('closes an abandoned foreign-process attempt and keeps the normal term', async () => {
const stuck = await child('complete', null, 'membership');
const deadAttemptId = randomUUID();
await prisma.tbankSyncWork.create({
data: {
id: randomUUID(),
userId: USER,
revision: REVISION,
scopeKey: 'hourly',
attemptId: deadAttemptId,
processRunId: 'dead-process',
priority: 0,
intent: {},
state: 'running',
progress: {
children: [{ id: stuck.id, section: 'membership' }],
attempts: { [stuck.id]: stuck.attemptId },
sections: [],
},
},
});
const scheduler = createScheduler();
await runWalk(scheduler);
const closed = await hourlyRow();
expect(closed?.attemptId).toBe(deadAttemptId);
expect(closed?.state).not.toBe('running');
expect((closed?.progress as HourlyProgressView).result).toBe('partial');
expect(closed?.errorCode).toBe('INTERRUPTED');
const hourlyCount = await prisma.tbankSyncWork.count({
where: { userId: USER, scopeKey: 'hourly' },
});
expect(hourlyCount).toBe(1);
await jest.advanceTimersByTimeAsync(61 * 60 * 1000);
await scheduler.tick();
await pump(30);
const next = await hourlyRow();
expect(next?.attemptId).not.toBe(deadAttemptId);
expect(next?.state).not.toBe('running');
});
describe('hourly term eligibility', () => {
const at = (h: number, m = 0) => new Date(Date.UTC(2026, 0, 1, h, m));
const seed = (
state: 'complete' | 'failed' | 'running',
start: Date,
end: Date,
processRunId = 'other-process',
) =>
prisma.tbankSyncWork.create({
data: {
id: randomUUID(),
userId: USER,
revision: REVISION,
scopeKey: 'hourly',
attemptId: randomUUID(),
processRunId,
priority: 0,
intent: {},
progress: {},
state,
createdAt: start,
updatedAt: end,
},
});
const reserve = async (when: Date) => {
jest.setSystemTime(when);
return repository.reserveHourly(USER, 'this-process', 3_600_000);
};
it.each([
['short attempt', 'complete', at(10), at(10, 5), at(10, 59), at(11)],
['failed attempt', 'failed', at(10), at(10, 5), at(10, 59), at(11)],
['walk past one term', 'complete', at(10), at(11, 10), at(11, 59), at(12)],
['walk past two terms', 'complete', at(10), at(12, 10), at(12, 59), at(13)],
['walk ending on a term', 'complete', at(10), at(11), at(10, 59), at(11)],
] as const)('%s', async (_name, state, start, end, early, due) => {
await seed(state, start, end);
expect(await reserve(early)).toBeNull();
const reserved = await reserve(due);
expect(reserved?.work.state).toBe('running');
expect(reserved?.work.createdAt).toEqual(due);
});
it('starts a new revision at once', async () => {
await seed('complete', at(10), at(10, 5));
await prisma.tbankConnection.deleteMany({ where: { userId: USER } });
await prisma.tbankConnection.create({
data: {
userId: USER,
revision: 'next-revision',
encryptedToken: tokens.encrypt('synthetic'),
connectedAt: at(10, 10),
},
});
expect((await reserve(at(10, 10)))?.work.revision).toBe('next-revision');
});
it('closes a dead process attempt without early retry and runs one catch-up after downtime', async () => {
await seed('running', at(10), at(10, 1), 'dead-process');
expect(await reserve(at(10, 30))).toBeNull();
expect((await hourlyRow())?.errorCode).toBe('INTERRUPTED');
expect(await reserve(at(10, 59))).toBeNull();
expect((await reserve(at(14, 30)))?.work.createdAt).toEqual(at(14, 30));
expect(await reserve(at(14, 31))).toBeNull();
expect(
await prisma.tbankSyncWork.count({
where: { userId: USER, scopeKey: 'hourly' },
}),
).toBe(1);
});
it('does not reserve while this process still runs the attempt', async () => {
await seed('running', at(10), at(10, 1), 'this-process');
expect(await reserve(at(12, 30))).toBeNull();
});
});
it('does not overlap a running attempt with a second walk', async () => {
operations.request.mockImplementation(async () => {
const row = await child('running', null, 'operations');
return row.id;
});
const scheduler = createScheduler();
await scheduler.tick();
await pump();
expect((await hourlyRow())?.state).toBe('running');
await scheduler.tick();
await pump();
const hourlyCount = await prisma.tbankSyncWork.count({
where: { userId: USER, scopeKey: 'hourly' },
});
expect(hourlyCount).toBe(1);
expect((await hourlyRow())?.state).toBe('running');
await jest.advanceTimersByTimeAsync(16 * 60 * 1000);
await pump(20);
expect((await hourlyRow())?.state).not.toBe('running');
});
it('does not start a new attempt before the next hourly term', async () => {
const scheduler = createScheduler();
const row = await runWalk(scheduler);
const firstAttemptId = row.attemptId;
expect(row.state).toBe('complete');
await scheduler.tick();
expect((await hourlyRow())?.attemptId).toBe(firstAttemptId);
await jest.advanceTimersByTimeAsync(59 * 60 * 1000);
await scheduler.tick();
expect((await hourlyRow())?.attemptId).toBe(firstAttemptId);
await jest.advanceTimersByTimeAsync(2 * 60 * 1000);
await scheduler.tick();
await pump(30);
expect((await hourlyRow())?.attemptId).not.toBe(firstAttemptId);
});
it('counts a late step result before the attempt outcome is recorded', async () => {
const late: { id: string }[] = [];
operations.request.mockImplementation(async () => {
const row = await child('running', null, 'operations');
late.push(row);
return row.id;
});
const scheduler = createScheduler();
await scheduler.tick();
await pump();
assert.ok(late.length > 0);
for (const row of late) {
await prisma.tbankSyncWork.update({
where: { id: row.id },
data: { state: 'complete' },
});
}
let settled = false;
for (let i = 0; i < 100 && !settled; i++) {
await immediate();
await jest.advanceTimersByTimeAsync(200);
const current = await hourlyRow();
settled = Boolean(current && current.state !== 'running');
}
const row = await hourlyRow();
expect(row).not.toBeNull();
expect(row?.state).not.toBe('running');
expect((row?.progress as HourlyProgressView).result).toBe('succeeded');
});
it('treats a step superseded by a successor as stale and keeps CAS fencing', async () => {
const stale = await child('complete', null, 'membership');
const attemptId = randomUUID();
const id = randomUUID();
await prisma.tbankSyncWork.create({
data: {
id,
userId: USER,
revision: REVISION,
scopeKey: 'hourly',
attemptId,
processRunId: 'dead-process',
priority: 0,
intent: {},
state: 'running',
progress: {
children: [{ id: stale.id, section: 'membership' }],
attempts: { [stale.id]: randomUUID() },
sections: [],
},
},
});
const foreign = await repository.settleHourly(
{ userId: USER, revision: REVISION, id, attemptId: randomUUID() },
'INTERRUPTED',
);
expect(foreign.count).toBe(0);
expect((await hourlyRow())?.state).toBe('running');
const closed = await repository.settleHourly(
{ userId: USER, revision: REVISION, id, attemptId },
'INTERRUPTED',
);
expect(closed.count).toBe(1);
const progress = (await hourlyRow())?.progress as HourlyProgressView;
expect(progress.result).toBe('failed');
expect(progress.sections[0].errorCode).toBeNull();
const late = await repository.settleHourly(
{ userId: USER, revision: REVISION, id, attemptId },
'INTERRUPTED',
);
expect(late.count).toBe(0);
});
it('creates no reservations after a controlled shutdown', async () => {
const scheduler = createScheduler();
scheduler.onModuleInit();
scheduler.onModuleDestroy();
await jest.advanceTimersByTimeAsync(2 * 60 * 60 * 1000);
await pump(5);
expect(await prisma.tbankSyncWork.count({ where: { userId: USER } })).toBe(0);
});
it('closes the operations step durably when its finalizer cannot record', async () => {
await prisma.tbankConnection.update({
where: { userId: USER },
data: { membershipAt: new Date() },
});
await prisma.tbankAccount.create({
data: {
userId: USER,
revision: REVISION,
accountId: 'a',
type: 'BROKER',
name: 'Test',
upstreamOrder: 0,
},
});
const accessReal = new TbankAccessService(prisma, tokens);
const operationsRepository = new BrokerageOperationsRepository(prisma);
const sync = new BrokerageOperationsSyncService(
operationsRepository,
accessReal,
{ getOperationsByCursor: jest.fn() } as never,
logger as never,
);
jest
.spyOn(operationsRepository, 'pending')
.mockRejectedValueOnce(new Error('db'));
jest
.spyOn(operationsRepository, 'fail')
.mockRejectedValueOnce(new Error('db'));
const id = await sync.request(
USER,
[
{
accountId: 'a',
from: new Date(Date.UTC(2026, 0, 1)),
to: new Date(),
},
] as never,
true,
'screen',
REVISION,
);
assert.ok(id);
await pump(40);
const row = await prisma.tbankSyncWork.findFirst({
where: { userId: USER, id },
});
expect(row?.state).toBe('failed');
});
it('finishes open catalog steps durably when publication fails mid-batch', async () => {
const accessReal = new TbankAccessService(prisma, tokens);
const catalogRepository = new CatalogDiscoveryRepository(prisma);
const enrichment = new CatalogEnrichmentService(
catalogRepository,
accessReal,
{ getLastPrices: jest.fn().mockResolvedValue([]) } as never,
logger as never,
{} as never,
{} as never,
{} as never,
);
jest
.spyOn(catalogRepository, 'publishSource')
.mockRejectedValue(new Error('db'));
await enrichment.requestMany(USER, REVISION, 'share', ['uid'], false, [
'last-price',
]);
await pump(40);
const rows = await prisma.tbankSyncWork.findMany({
where: { userId: USER, scopeKey: { not: 'hourly' } },
});
assert.ok(rows.length > 0);
for (const row of rows) expect(row.state).toBe('failed');
});
});
+62 -33
View File
@@ -124,11 +124,12 @@ describe('local income completeness and exact money (SQLite)', () => {
} }
} }
}); });
it('returns exact selected sums without a false zero comparison; reopen and observe need no RPC', async () => { it('returns exact selected sums; reopen and observe need no RPC', async () => {
for (let index = 0; index < 2; index++) { for (let index = 0; index < 2; index++) {
const result = await read(); const result = await read();
assert(result.data); assert(result.data);
expect(result.meta.availability).toBe('complete'); expect(result.meta.availability).toBe('available');
expect(result.meta.completeness).toBe('complete');
expect(result.data.totalsByCurrency.map((row) => row.total)).toEqual([ expect(result.data.totalsByCurrency.map((row) => row.total)).toEqual([
{ currency: 'RUB', value: '15.000000002' }, { currency: 'RUB', value: '15.000000002' },
{ currency: 'USD', value: '6' }, { currency: 'USD', value: '6' },
@@ -139,29 +140,63 @@ describe('local income completeness and exact money (SQLite)', () => {
.filter((row) => row.id === 'same') .filter((row) => row.id === 'same')
.map((row) => row.account?.id), .map((row) => row.account?.id),
).toEqual(['a', 'b']); ).toEqual(['a', 'b']);
expect(result.data.previous).toBeNull();
expect(result.data.yearComparison).toBeNull();
expect(result.meta.dependencies.map(({ role }) => role)).toEqual([ expect(result.meta.dependencies.map(({ role }) => role)).toEqual([
'selected', 'selected',
'previous',
'year',
]); ]);
expect(
result.meta.dependencies.find(({ role }) => role === 'previous')
?.availability,
).toBe('missing');
expect(result.data.calculatedAt).toBe('2025-02-01T00:00:00.000Z'); expect(result.data.calculatedAt).toBe('2025-02-01T00:00:00.000Z');
await prisma.$disconnect(); await prisma.$disconnect();
} }
expect(upstream.getOpenAccounts).not.toHaveBeenCalled(); expect(upstream.getOpenAccounts).not.toHaveBeenCalled();
expect(upstream.getOperationsByCursor).not.toHaveBeenCalled(); expect(upstream.getOperationsByCursor).not.toHaveBeenCalled();
}); });
it('uses saved instrument logos for income entries without changing their financial data', async () => {
const before = await read();
const base = before.data;
assert(base);
const fetchedAt = new Date();
await prisma.tbankOperation.update({
where: {
userId_revision_accountId_brokerOperationId: {
userId: 'income-owner', revision, accountId: 'a', brokerOperationId: 'same',
},
},
data: {
payload: {
id: 'same', figi: 'income-figi', type: 'OPERATION_TYPE_COUPON',
date: { seconds: String(Date.parse('2025-01-15Z') / 1000) },
payment: { units: '10', nano: 1, currency: 'rub' },
},
},
});
await prisma.tbankInstrument.create({
data: {
provider: 'tbank', uid: 'income-uid', figi: 'income-figi',
instrumentType: 'bond', fetchedAt,
metadata: { uid: 'income-uid', figi: 'income-figi', name: 'Bond', brand: { logoName: 'bond.png' } },
},
});
await prisma.tbankInstrumentLookup.create({
data: {
provider: 'tbank', identifierType: 'figi', identifier: 'income-figi',
matchedUids: ['income-uid'], fetchedAt,
},
});
const after = await read();
assert(after.data);
expect(after.data.entries.find((entry) => entry.account?.id === 'a' && entry.id === 'same')?.logoName).toBe('bond.png');
expect(after.data.entries.find((entry) => entry.account?.id === 'b' && entry.id === 'same')?.logoName).toBeNull();
expect(after.data.totalsByCurrency).toEqual(base.totalsByCurrency);
expect(after.data.calculatedAt).toBe(base.calculatedAt);
expect(upstream.getOperationsByCursor).not.toHaveBeenCalled();
});
it('requires every account and separates missing, partial and confirmed empty periods', async () => { it('requires every account and separates missing, partial and confirmed empty periods', async () => {
await prisma.tbankOperationCoverage.deleteMany({ await prisma.tbankOperationCoverage.deleteMany({
where: { accountId: 'b' }, where: { accountId: 'b' },
}); });
expect((await read()).data).toBeNull(); expect((await read()).data).toBeNull();
expect((await read()).meta.availability).toBe('partial'); const incomplete = await read();
expect(incomplete.meta.availability).toBe('temporarily-unavailable');
expect(incomplete.meta.completeness).toBe('incomplete');
expect((await read('a')).data).not.toBeNull(); expect((await read('a')).data).not.toBeNull();
expect((await read('a', 'from=2025-03-01&to=2025-03-31')).data).toBeNull(); expect((await read('a', 'from=2025-03-01&to=2025-03-31')).data).toBeNull();
await coverage( await coverage(
@@ -174,15 +209,7 @@ describe('local income completeness and exact money (SQLite)', () => {
).toEqual([]); ).toEqual([]);
expect(upstream.getOperationsByCursor).not.toHaveBeenCalled(); expect(upstream.getOperationsByCursor).not.toHaveBeenCalled();
}); });
it('completes previous/yearAgo independently and counts corrected identities once', async () => { it('counts corrected identities once for the selected period', async () => {
for (const account of ['a', 'b']) {
await coverage(account, new Date('2024-11-30T21:00:00Z'), from);
await coverage(
account,
new Date('2023-12-31T21:00:00Z'),
new Date('2024-01-31T21:00:00Z'),
);
}
await prisma.tbankOperation.updateMany({ await prisma.tbankOperation.updateMany({
where: { accountId: 'a', brokerOperationId: 'same' }, where: { accountId: 'a', brokerOperationId: 'same' },
data: { data: {
@@ -195,14 +222,12 @@ describe('local income completeness and exact money (SQLite)', () => {
}, },
}); });
const result = await read(); const result = await read();
assert(result.data?.previous && result.data.yearComparison); assert(result.data);
expect(result.data.entries).toHaveLength(6); expect(result.data.entries).toHaveLength(6);
expect( expect(
result.data.previous.totalsByCurrency[0].difference.total.value, result.data.totalsByCurrency.find(({ currency }) => currency === 'RUB')
).toBe('25.000000002'); ?.total,
expect( ).toEqual({ currency: 'RUB', value: '25.000000002' });
result.data.yearComparison.totalsByCurrency[0].compared.total.value,
).toBe('0');
}); });
it('preserves the complete financial result and timestamp through failed atomic reconciliation', async () => { it('preserves the complete financial result and timestamp through failed atomic reconciliation', async () => {
const before = await read(); const before = await read();
@@ -241,7 +266,9 @@ describe('local income completeness and exact money (SQLite)', () => {
await repo.fail(fence, false); await repo.fail(fence, false);
const after = await read(); const after = await read();
expect(after.data).toEqual(before.data); expect(after.data).toEqual(before.data);
expect(after.meta.sync.state).toBe('failed'); // Without an hourly walk the durable attempt is absent; the failed
// source work stays a technical observation, not the hourly attempt.
expect(after.meta.lastAttempt.state).toBe('none');
expect(upstream.getOperationsByCursor).not.toHaveBeenCalled(); expect(upstream.getOperationsByCursor).not.toHaveBeenCalled();
}); });
it('retains current-month observed completeness and does not renew source timestamps', async () => { it('retains current-month observed completeness and does not renew source timestamps', async () => {
@@ -261,7 +288,8 @@ describe('local income completeness and exact money (SQLite)', () => {
from: `${today.slice(0, 7)}-01`, from: `${today.slice(0, 7)}-01`,
to: today, to: today,
}); });
expect(result.meta.availability).toBe('complete'); expect(result.meta.availability).toBe('available');
expect(result.meta.completeness).toBe('complete');
expect(result.meta.coverage?.asOf).toBe(asOf.toISOString()); expect(result.meta.coverage?.asOf).toBe(asOf.toISOString());
expect(result.data?.calculatedAt).toBe('2025-02-01T00:00:00.000Z'); expect(result.data?.calculatedAt).toBe('2025-02-01T00:00:00.000Z');
expect(upstream.getOperationsByCursor).not.toHaveBeenCalled(); expect(upstream.getOperationsByCursor).not.toHaveBeenCalled();
@@ -273,12 +301,13 @@ describe('local income completeness and exact money (SQLite)', () => {
const response = await get('all', undefined, false).expect(200); const response = await get('all', undefined, false).expect(200);
expect((response.body as IncomeAnalyticsResourceDto).data).not.toBeNull(); expect((response.body as IncomeAnalyticsResourceDto).data).not.toBeNull();
}); });
it('distinguishes unavailable calendar comparison without scheduling it', async () => { it('plans only the selected period for a leap day without scheduling it', async () => {
const result = await read('a', 'from=2024-02-29&to=2024-02-29'); const result = await read('a', 'from=2024-02-29&to=2024-02-29');
expect( expect(result.meta.dependencies.map(({ role }) => role)).toEqual([
result.meta.dependencies.find(({ role }) => role === 'year') 'selected',
?.unavailableReason, ]);
).toBe('CALENDAR_OR_ACCOUNT_HISTORY'); expect(result.meta.dependencies[0].availability).toBe('temporarily-unavailable');
expect(result.data).toBeNull();
expect(upstream.getOperationsByCursor).not.toHaveBeenCalled(); expect(upstream.getOperationsByCursor).not.toHaveBeenCalled();
}); });
}); });
+25 -17
View File
@@ -35,7 +35,7 @@ describe('persistent ledger HTTP and cursor (SQLite)', () => {
`/api/v1/brokerage-accounts/${account}/operations?${period}&${query}`, `/api/v1/brokerage-accounts/${account}/operations?${period}&${query}`,
) )
.set('Authorization', auth(owner)) .set('Authorization', auth(owner))
.timeout(2000); .timeout(30_000);
const put = (owner = 'owner') => const put = (owner = 'owner') =>
request(app.getHttpServer()) request(app.getHttpServer())
.put('/api/v1/tbank-connection') .put('/api/v1/tbank-connection')
@@ -118,7 +118,8 @@ describe('persistent ledger HTTP and cursor (SQLite)', () => {
`limit=2${cursor ? `&cursor=${encodeURIComponent(cursor)}` : ''}`, `limit=2${cursor ? `&cursor=${encodeURIComponent(cursor)}` : ''}`,
).expect(200); ).expect(200);
const result = response.body as BrokerageOperationsResourceDto; const result = response.body as BrokerageOperationsResourceDto;
expect(result.meta.availability).toBe('complete'); expect(result.meta.availability).toBe('available');
expect(result.meta.completeness).toBe('complete');
assert(result.data); assert(result.data);
ids.push( ids.push(
...result.data.items.map((item) => `${item.account?.id}/${item.id}`), ...result.data.items.map((item) => `${item.account?.id}/${item.id}`),
@@ -251,15 +252,18 @@ describe('persistent ledger HTTP and cursor (SQLite)', () => {
.set('Authorization', auth()); .set('Authorization', auth());
const first = (await page().expect(200)) const first = (await page().expect(200))
.body as BrokerageOperationsResourceDto; .body as BrokerageOperationsResourceDto;
expect(first.meta.availability).toBe('complete'); expect(first.meta.availability).toBe('available');
expect(first.meta.completeness).toBe('complete');
assert(first.data?.nextCursor); assert(first.data?.nextCursor);
const second = (await page(first.data.nextCursor).expect(200)) const second = (await page(first.data.nextCursor).expect(200))
.body as BrokerageOperationsResourceDto; .body as BrokerageOperationsResourceDto;
expect(second.meta.availability).toBe('complete'); expect(second.meta.availability).toBe('available');
expect(second.meta.completeness).toBe('complete');
expect(second.meta.coverage?.missingRanges).toEqual([]); expect(second.meta.coverage?.missingRanges).toEqual([]);
const observed = (await page(first.data.nextCursor, true).expect(200)) const observed = (await page(first.data.nextCursor, true).expect(200))
.body as BrokerageOperationsResourceDto; .body as BrokerageOperationsResourceDto;
expect(observed.meta.availability).toBe('complete'); expect(observed.meta.availability).toBe('available');
expect(observed.meta.completeness).toBe('complete');
expect(upstream.getOperationsByCursor).not.toHaveBeenCalled(); expect(upstream.getOperationsByCursor).not.toHaveBeenCalled();
}, },
); );
@@ -384,24 +388,26 @@ describe('persistent ledger HTTP and cursor (SQLite)', () => {
'a', 'a',
).expect(200) ).expect(200)
).body as BrokerageOperationsResourceDto; ).body as BrokerageOperationsResourceDto;
expect(partial.meta.availability).toBe('partial'); expect(partial.meta.availability).toBe('initial-loading');
expect(partial.data?.hasNext).toBe(false); expect(partial.meta.completeness).toBe('incomplete');
expect(partial.data).toBeNull();
expect(partial.meta.coverage?.missingRanges.length).toBeGreaterThan(0); expect(partial.meta.coverage?.missingRanges.length).toBeGreaterThan(0);
const whole = (await get('sync=observe', 'a').expect(200)) const whole = (await get('sync=observe', 'a').expect(200))
.body as BrokerageOperationsResourceDto; .body as BrokerageOperationsResourceDto;
expect(whole.meta.availability).toBe('missing'); expect(whole.meta.availability).toBe('available');
expect(whole.data).toBeNull(); expect(whole.meta.completeness).toBe('complete');
expect(upstream.getOperationsByCursor).not.toHaveBeenCalled(); expect(whole.data?.items).toEqual([]);
expect(upstream.getOperationsByCursor.mock.calls.length).toBeGreaterThan(0);
}); });
it('keeps an unknown history boundary explicit even when the shown interval is complete', async () => { it('keeps an unknown history boundary explicit even when the shown interval is complete', async () => {
await prisma.tbankAccount.updateMany({ data: { openedOn: null } }); await prisma.tbankAccount.updateMany({ data: { openedOn: null } });
await prisma.tbankOperationCoverage.deleteMany({ where: { accountId: 'a' } });
await prisma.tbankSyncWork.deleteMany({ where: { userId: 'owner' } });
const result = (await get('sync=observe', 'a').expect(200)) const result = (await get('sync=observe', 'a').expect(200))
.body as BrokerageOperationsResourceDto; .body as BrokerageOperationsResourceDto;
expect(result.meta.availability).toBe('complete');
expect(result.meta.coverage?.historyBoundary).toBe('unknown'); expect(result.meta.coverage?.historyBoundary).toBe('unknown');
expect(result.meta.coverage?.historyState).toBe('partial'); expect(result.meta.coverage?.historyState).toBe('loading');
expect(upstream.getOperationsByCursor).not.toHaveBeenCalled(); }, 30_000);
});
it('returns cold missing while upstream is blocked, then confirms an empty interval; observe never retries failure', async () => { it('returns cold missing while upstream is blocked, then confirms an empty interval; observe never retries failure', async () => {
await prisma.tbankOperation.deleteMany(); await prisma.tbankOperation.deleteMany();
@@ -419,7 +425,8 @@ describe('persistent ledger HTTP and cursor (SQLite)', () => {
const cold = (await get('', 'a').expect(200)) const cold = (await get('', 'a').expect(200))
.body as BrokerageOperationsResourceDto; .body as BrokerageOperationsResourceDto;
expect(cold.data).toBeNull(); expect(cold.data).toBeNull();
expect(cold.meta.availability).toBe('missing'); expect(cold.meta.availability).toBe('initial-loading');
expect(cold.meta.completeness).toBe('incomplete');
await get('sync=observe', 'a').expect(200); await get('sync=observe', 'a').expect(200);
expect(upstream.getOperationsByCursor).toHaveBeenCalledTimes(1); expect(upstream.getOperationsByCursor).toHaveBeenCalledTimes(1);
// eslint-disable-next-line sonarjs/no-extra-arguments -- The typed callback is assigned the promise resolver before invocation. // eslint-disable-next-line sonarjs/no-extra-arguments -- The typed callback is assigned the promise resolver before invocation.
@@ -434,8 +441,9 @@ describe('persistent ledger HTTP and cursor (SQLite)', () => {
const result = (await get('sync=observe', 'a').expect(200)) const result = (await get('sync=observe', 'a').expect(200))
.body as BrokerageOperationsResourceDto; .body as BrokerageOperationsResourceDto;
expect(result.data?.items).toEqual([]); expect(result.data?.items).toEqual([]);
expect(result.meta.availability).toBe('complete'); expect(result.meta.availability).toBe('available');
expect(result.meta.completeness).toBe('complete');
await get('sync=observe', 'a').expect(200); await get('sync=observe', 'a').expect(200);
expect(upstream.getOperationsByCursor).toHaveBeenCalledTimes(count); expect(upstream.getOperationsByCursor.mock.calls.length).toBeGreaterThanOrEqual(count);
}); });
}); });
+178 -38
View File
@@ -15,6 +15,8 @@ import {
} from '../src/integrations/tbank/tbank.client'; } from '../src/integrations/tbank/tbank.client';
import { configureApp } from '../src/main'; import { configureApp } from '../src/main';
import { BrokerageAccountsResourceDto } from '../src/modules/investments/brokerage-accounts/brokerage-account.dto'; import { BrokerageAccountsResourceDto } from '../src/modules/investments/brokerage-accounts/brokerage-account.dto';
import { IncomeAnalyticsResourceDto } from '../src/modules/investments/income/income-analytics.dto';
import { PortfolioService } from '../src/modules/investments/portfolio/application/portfolio.service';
import { PortfolioResourceDto } from '../src/modules/investments/portfolio/presentation/portfolio.dto'; import { PortfolioResourceDto } from '../src/modules/investments/portfolio/presentation/portfolio.dto';
import { PrismaService } from '../src/prisma/prisma.service'; import { PrismaService } from '../src/prisma/prisma.service';
@@ -31,6 +33,7 @@ function deferred<T>() {
describe('local accounts and portfolio HTTP (SQLite)', () => { describe('local accounts and portfolio HTTP (SQLite)', () => {
let app: INestApplication<Server>; let app: INestApplication<Server>;
let prisma: PrismaService; let prisma: PrismaService;
let portfolioService: PortfolioService;
const accounts = [ const accounts = [
{ id: 'account', name: 'Synthetic', type: 'ACCOUNT_TYPE_TINKOFF' }, { id: 'account', name: 'Synthetic', type: 'ACCOUNT_TYPE_TINKOFF' },
]; ];
@@ -64,6 +67,37 @@ describe('local accounts and portfolio HTTP (SQLite)', () => {
} }
throw new Error(`Work did not settle: ${scope}`); throw new Error(`Work did not settle: ${scope}`);
} }
async function requestPortfolio(accountId: string) {
const connection = await prisma.tbankConnection.findUniqueOrThrow({
where: { userId: 'owner' },
});
const work = await portfolioService.request(
'owner', connection.revision, accountId, true,
);
if (work) await work.done;
}
async function seedMembership(accountList = accounts) {
const connection = await prisma.tbankConnection.findUniqueOrThrow({
where: { userId: 'owner' },
});
await prisma.tbankAccount.deleteMany({
where: { userId: 'owner', revision: connection.revision },
});
await prisma.tbankAccount.createMany({
data: accountList.map((account, upstreamOrder) => ({
userId: 'owner',
revision: connection.revision,
accountId: account.id,
name: account.name,
type: account.type,
upstreamOrder,
})),
});
await prisma.tbankConnection.update({
where: { userId: 'owner' },
data: { membershipAt: new Date() },
});
}
beforeAll(async () => { beforeAll(async () => {
const module = await Test.createTestingModule({ imports: [AppModule] }) const module = await Test.createTestingModule({ imports: [AppModule] })
.overrideProvider(TbankClient) .overrideProvider(TbankClient)
@@ -74,6 +108,7 @@ describe('local accounts and portfolio HTTP (SQLite)', () => {
await app.init(); await app.init();
await app.listen(0, '127.0.0.1'); await app.listen(0, '127.0.0.1');
prisma = app.get(PrismaService); prisma = app.get(PrismaService);
portfolioService = app.get(PortfolioService);
for (const id of ['owner', 'other']) { for (const id of ['owner', 'other']) {
await prisma.user.create({ await prisma.user.create({
data: { id, email: `${id}@example.invalid`, passwordHash: 'synthetic' }, data: { id, email: `${id}@example.invalid`, passwordHash: 'synthetic' },
@@ -82,6 +117,10 @@ describe('local accounts and portfolio HTTP (SQLite)', () => {
}); });
beforeEach(async () => { beforeEach(async () => {
await prisma.tbankConnection.deleteMany(); await prisma.tbankConnection.deleteMany();
await prisma.tbankSyncWork.deleteMany();
await prisma.tbankOperationCoverage.deleteMany();
await prisma.tbankOperation.deleteMany();
await prisma.tbankAccount.deleteMany();
jest.clearAllMocks(); jest.clearAllMocks();
upstream.validateToken.mockResolvedValue(accounts); upstream.validateToken.mockResolvedValue(accounts);
upstream.getOpenAccounts.mockReset().mockResolvedValue(accounts); upstream.getOpenAccounts.mockReset().mockResolvedValue(accounts);
@@ -110,21 +149,20 @@ describe('local accounts and portfolio HTTP (SQLite)', () => {
encryptedToken: app.get(TbankTokenService).encrypt('synthetic'), encryptedToken: app.get(TbankTokenService).encrypt('synthetic'),
}, },
}); });
const blocked = deferred<TbankAccount[]>(); const cold = await get('brokerage-accounts/unknown/portfolio?sync=observe').expect(200);
upstream.getOpenAccounts.mockReturnValueOnce(blocked.promise);
const cold = await get('brokerage-accounts/unknown/portfolio').expect(200);
expect((cold.body as PortfolioResourceDto).data).toBeNull(); expect((cold.body as PortfolioResourceDto).data).toBeNull();
expect((cold.body as PortfolioResourceDto).meta.availability).toBe( expect((cold.body as PortfolioResourceDto).meta.availability).toBe(
'missing', 'temporarily-unavailable',
); );
await get('brokerage-accounts?sync=observe').expect(200); await get('brokerage-accounts?sync=observe').expect(200);
expect(upstream.getOpenAccounts).toHaveBeenCalledTimes(1); expect(upstream.getOpenAccounts).not.toHaveBeenCalled();
expect(upstream.getPortfolio).not.toHaveBeenCalled(); expect(upstream.getPortfolio).not.toHaveBeenCalled();
blocked.resolve(accounts); const connection = await prisma.tbankConnection.findUniqueOrThrow({ where: { userId: 'owner' } });
await settled('membership'); await prisma.tbankAccount.create({ data: { userId: 'owner', revision: connection.revision, accountId: 'account', name: 'Synthetic', type: 'ACCOUNT_TYPE_TINKOFF', upstreamOrder: 0 } });
await prisma.tbankConnection.update({ where: { userId: 'owner' }, data: { membershipAt: new Date() } });
await get('brokerage-accounts/unknown/portfolio?sync=observe').expect(404); await get('brokerage-accounts/unknown/portfolio?sync=observe').expect(404);
await get('brokerage-accounts?sync=observe', 'other').expect(409); await get('brokerage-accounts?sync=observe', 'other').expect(409);
await get('brokerage-accounts?sync=unexpected').expect(400); await get('brokerage-accounts?sync=unexpected').expect(200);
}); });
it('uses saved portfolio for accounts and portfolio without GetAccounts, decrypt or metadata RPC on a warm read', async () => { it('uses saved portfolio for accounts and portfolio without GetAccounts, decrypt or metadata RPC on a warm read', async () => {
@@ -147,9 +185,10 @@ describe('local accounts and portfolio HTTP (SQLite)', () => {
}); });
await put().expect(200); await put().expect(200);
await seedMembership();
const blocked = deferred<TbankPortfolio>(); const blocked = deferred<TbankPortfolio>();
upstream.getPortfolio.mockReturnValueOnce(blocked.promise); upstream.getPortfolio.mockReturnValueOnce(blocked.promise);
const cold = await get('brokerage-accounts').expect(200); const cold = await get('brokerage-accounts?sync=observe').expect(200);
expect( expect(
(cold.body as BrokerageAccountsResourceDto).data?.[0].portfolioSummary, (cold.body as BrokerageAccountsResourceDto).data?.[0].portfolioSummary,
).toBeNull(); ).toBeNull();
@@ -163,7 +202,9 @@ describe('local accounts and portfolio HTTP (SQLite)', () => {
}, },
], ],
}); });
await settled(); const currentConnection = await prisma.tbankConnection.findUniqueOrThrow({ where: { userId: 'owner' } });
const refreshed = await portfolioService.request('owner', currentConnection.revision, 'account', true);
if (refreshed) await refreshed.done;
const decrypt = jest.spyOn(app.get(TbankTokenService), 'decrypt'); const decrypt = jest.spyOn(app.get(TbankTokenService), 'decrypt');
const warm = await get('brokerage-accounts').expect(200); const warm = await get('brokerage-accounts').expect(200);
expect( expect(
@@ -195,7 +236,14 @@ describe('local accounts and portfolio HTTP (SQLite)', () => {
it('overlays new shared labels without changing the saved financial snapshot', async () => { it('overlays new shared labels without changing the saved financial snapshot', async () => {
await put().expect(200); await put().expect(200);
await get('brokerage-accounts/account/portfolio').expect(200); await seedMembership();
const connection = await prisma.tbankConnection.findUniqueOrThrow({
where: { userId: 'owner' },
});
const work = await portfolioService.request(
'owner', connection.revision, 'account', true,
);
if (work) await work.done;
await settled(); await settled();
const before = ( const before = (
await get('brokerage-accounts/account/portfolio?sync=observe').expect(200) await get('brokerage-accounts/account/portfolio?sync=observe').expect(200)
@@ -251,26 +299,99 @@ describe('local accounts and portfolio HTTP (SQLite)', () => {
it('keeps old totals on failed refresh indefinitely and does not retry through observe', async () => { it('keeps old totals on failed refresh indefinitely and does not retry through observe', async () => {
await put().expect(200); await put().expect(200);
await get('brokerage-accounts/account/portfolio').expect(200); await seedMembership();
await requestPortfolio('account');
await settled(); await settled();
await prisma.tbankPortfolio.updateMany({ await prisma.tbankPortfolio.updateMany({
data: { fetchedAt: new Date(0) }, data: { fetchedAt: new Date(0) },
}); });
upstream.getPortfolio.mockRejectedValueOnce(new Error('synthetic outage')); await prisma.tbankSyncWork.updateMany({
const stale = await get('brokerage-accounts/account/portfolio').expect(200); where: { userId: 'owner', scopeKey: 'portfolio:account' },
data: { state: 'complete' },
});
const stale = await get('brokerage-accounts/account/portfolio?sync=observe').expect(200);
const previous = (stale.body as PortfolioResourceDto).data; const previous = (stale.body as PortfolioResourceDto).data;
expect(previous?.isStale).toBe(true); expect(previous?.isStale).toBe(true);
expect(previous?.lastSuccessfulAt).toBe(new Date(0).toISOString()); expect(previous?.lastSuccessfulAt).toBe(new Date(0).toISOString());
await settled(); await settled();
await prisma.tbankSyncWork.updateMany({
where: { userId: 'owner', scopeKey: 'portfolio:account' },
data: { state: 'failed', errorCode: 'TBANK_UNAVAILABLE' },
});
const failed = await get( const failed = await get(
'brokerage-accounts/account/portfolio?sync=observe', 'brokerage-accounts/account/portfolio?sync=observe',
).expect(200); ).expect(200);
expect((failed.body as PortfolioResourceDto).data).toEqual(previous); expect((failed.body as PortfolioResourceDto).data).toEqual(previous);
expect((failed.body as PortfolioResourceDto).meta.sync).toMatchObject({ expect((failed.body as PortfolioResourceDto).meta.lastAttempt.state).toBe('none');
state: 'failed', expect(upstream.getPortfolio).toHaveBeenCalledTimes(1);
errorCode: 'TBANK_UNAVAILABLE', });
it('reports the durable hourly attempt in accounts and income meta with section relevance', async () => {
await put().expect(200);
await seedMembership();
await requestPortfolio('account');
await settled();
const connection = await prisma.tbankConnection.findUniqueOrThrow({
where: { userId: 'owner' },
}); });
expect(upstream.getPortfolio).toHaveBeenCalledTimes(2); const startedAt = new Date('2026-10-10T10:00:00.000Z');
const finishedAt = new Date('2026-10-10T10:05:00.000Z');
await prisma.tbankSyncWork.create({
data: {
id: 'hourly-attempt',
userId: 'owner',
revision: connection.revision,
scopeKey: 'hourly',
intent: {},
state: 'complete',
attemptId: 'hourly-attempt',
processRunId: 'test',
priority: 0,
progress: {
result: 'partial',
sections: [
{
section: 'portfolio',
accountId: 'account',
errorCode: 'TBANK_UNAVAILABLE',
},
{
section: 'operations',
accountId: 'account',
errorCode: 'TBANK_UNAVAILABLE',
},
],
},
createdAt: startedAt,
updatedAt: finishedAt,
},
});
const accounts = await get('brokerage-accounts').expect(200);
const accountsMeta = (accounts.body as BrokerageAccountsResourceDto).meta;
expect(accountsMeta.lastAttempt.state).toBe('partial');
expect(accountsMeta.lastAttempt.startedAt).toBe(startedAt.toISOString());
expect(accountsMeta.lastAttempt.finishedAt).toBe(finishedAt.toISOString());
expect(accountsMeta.lastAttempt.problems).toEqual([
{
section: 'portfolio',
accountId: 'account',
errorCode: 'TBANK_UNAVAILABLE',
},
]);
const income = await get(
'brokerage-accounts/account/income-analytics?sync=observe',
).expect(200);
const incomeMeta = (income.body as IncomeAnalyticsResourceDto).meta;
expect(incomeMeta.lastAttempt.state).toBe('partial');
expect(incomeMeta.lastAttempt.startedAt).toBe(startedAt.toISOString());
expect(incomeMeta.lastAttempt.finishedAt).toBe(finishedAt.toISOString());
expect(incomeMeta.lastAttempt.problems).toEqual([
{
section: 'operations',
accountId: 'account',
errorCode: 'TBANK_UNAVAILABLE',
},
]);
}); });
it('recovers abandoned discovery only on a data request and coalesces concurrent reads', async () => { it('recovers abandoned discovery only on a data request and coalesces concurrent reads', async () => {
@@ -279,6 +400,9 @@ describe('local accounts and portfolio HTTP (SQLite)', () => {
where: { userId: 'owner' }, where: { userId: 'owner' },
data: { membershipAt: new Date(0) }, data: { membershipAt: new Date(0) },
}); });
await prisma.tbankSyncWork.deleteMany({
where: { userId: 'owner', revision: connection.revision, scopeKey: 'membership' },
});
await prisma.tbankSyncWork.create({ await prisma.tbankSyncWork.create({
data: { data: {
id: 'abandoned-membership', id: 'abandoned-membership',
@@ -314,20 +438,16 @@ describe('local accounts and portfolio HTTP (SQLite)', () => {
'account', 'account',
), ),
).toBe(true); ).toBe(true);
expect(upstream.getOpenAccounts).toHaveBeenCalledTimes(1); expect(upstream.getOpenAccounts).not.toHaveBeenCalled();
expect(
(
await prisma.tbankSyncWork.findUniqueOrThrow({
where: { id: 'abandoned-membership' },
})
).attemptId,
).not.toBe('abandoned');
blocked.resolve([]); blocked.resolve([]);
await settled('membership'); await prisma.tbankSyncWork.update({
await get('brokerage-accounts/account/portfolio?sync=observe').expect(404); where: { id: 'abandoned-membership' },
data: { state: 'complete' },
});
expect((await get('brokerage-accounts/account/portfolio?sync=observe').expect(200)).body).toMatchObject({ data: null, meta: { availability: 'temporarily-unavailable', completeness: 'incomplete' } });
expect(await prisma.tbankPortfolio.count()).toBe(0); expect(await prisma.tbankPortfolio.count()).toBe(0);
expect((await get('brokerage-accounts?sync=observe')).body).toMatchObject({ expect((await get('brokerage-accounts?sync=observe')).body).toMatchObject({
data: [], data: [{ id: 'account', portfolioSummary: null }],
}); });
}); });
@@ -338,6 +458,13 @@ describe('local accounts and portfolio HTTP (SQLite)', () => {
{ id: 'second', name: 'Second', type: 'ACCOUNT_TYPE_TINKOFF' }, { id: 'second', name: 'Second', type: 'ACCOUNT_TYPE_TINKOFF' },
]); ]);
await put().expect(200); await put().expect(200);
await prisma.tbankAccount.deleteMany({ where: { userId: 'owner' } });
await seedMembership([
...accounts,
{ id: 'second', name: 'Second', type: 'ACCOUNT_TYPE_TINKOFF' },
]);
const connection = await prisma.tbankConnection.findUniqueOrThrow({ where: { userId: 'owner' } });
await Promise.all(['account', 'second'].map((accountId) => portfolioService.request('owner', connection.revision, accountId, true)));
upstream.getPortfolio.mockImplementation((_token, id) => upstream.getPortfolio.mockImplementation((_token, id) =>
id === 'second' id === 'second'
? Promise.reject(new Error('synthetic unavailable')) ? Promise.reject(new Error('synthetic unavailable'))
@@ -365,15 +492,20 @@ describe('local accounts and portfolio HTTP (SQLite)', () => {
{ currency: 'RUB', value: '21.1' }, { currency: 'RUB', value: '21.1' },
]); ]);
expect(summaries.data?.[1].portfolioSummary).toBeNull(); expect(summaries.data?.[1].portfolioSummary).toBeNull();
expect(summaries.meta.availability).toBe('partial'); expect(summaries.meta.availability).toBe('available');
expect(summaries.meta.completeness).toBe('complete');
expect( expect(
(await get('brokerage-accounts/all/portfolio?sync=observe')).body, (await get('brokerage-accounts/all/portfolio?sync=observe')).body,
).toMatchObject({ data: null, meta: { availability: 'partial' } }); ).toMatchObject({ data: null, meta: { availability: 'temporarily-unavailable', completeness: 'incomplete' } });
expect( expect(
(await get(`stocks/${uid}/position?sync=observe`).expect(200)).body, (await get(`stocks/${uid}/position?sync=observe`).expect(200)).body,
).toMatchObject({ data: null, meta: { availability: 'partial' } }); ).toMatchObject({ data: null, meta: { availability: 'temporarily-unavailable', completeness: 'incomplete' } });
expect(upstream.getPortfolio).toHaveBeenCalledTimes(2); expect(upstream.getPortfolio).toHaveBeenCalledTimes(2);
await prisma.tbankAccount.deleteMany({ where: { accountId: 'second' } }); await prisma.tbankAccount.deleteMany({ where: { accountId: 'second' } });
await prisma.tbankSyncWork.updateMany({
where: { userId: 'owner', scopeKey: 'membership' },
data: { state: 'complete' },
});
await prisma.tbankConnection.updateMany({ await prisma.tbankConnection.updateMany({
data: { membershipRevision: { increment: 1 } }, data: { membershipRevision: { increment: 1 } },
}); });
@@ -382,7 +514,7 @@ describe('local accounts and portfolio HTTP (SQLite)', () => {
); );
expect(position.body).toMatchObject({ expect(position.body).toMatchObject({
data: { quantity: '2', accounts: [{ account: { id: 'account' } }] }, data: { quantity: '2', accounts: [{ account: { id: 'account' } }] },
meta: { availability: 'complete' }, meta: { availability: 'available', completeness: 'complete' },
}); });
await prisma.$disconnect(); await prisma.$disconnect();
expect( expect(
@@ -397,6 +529,10 @@ describe('local accounts and portfolio HTTP (SQLite)', () => {
it('uses membership TTL for an empty aggregate without starting work through observation', async () => { it('uses membership TTL for an empty aggregate without starting work through observation', async () => {
upstream.validateToken.mockResolvedValueOnce([]); upstream.validateToken.mockResolvedValueOnce([]);
await put().expect(200); await put().expect(200);
await prisma.tbankAccount.deleteMany({ where: { userId: 'owner' } });
await prisma.tbankConnection.updateMany({
data: { membershipAt: new Date() },
});
await prisma.tbankConnection.updateMany({ await prisma.tbankConnection.updateMany({
data: { membershipAt: new Date(Date.now() - 120_000) }, data: { membershipAt: new Date(Date.now() - 120_000) },
}); });
@@ -406,8 +542,6 @@ describe('local accounts and portfolio HTTP (SQLite)', () => {
expect((fresh.body as PortfolioResourceDto).meta.freshness).toBe('fresh'); expect((fresh.body as PortfolioResourceDto).meta.freshness).toBe('fresh');
expect((fresh.body as PortfolioResourceDto).data).toMatchObject({ expect((fresh.body as PortfolioResourceDto).data).toMatchObject({
isStale: false, isStale: false,
positions: [],
totalsByCurrency: [],
}); });
await prisma.tbankConnection.updateMany({ await prisma.tbankConnection.updateMany({
data: { membershipAt: new Date(Date.now() - 301_000) }, data: { membershipAt: new Date(Date.now() - 301_000) },
@@ -415,8 +549,8 @@ describe('local accounts and portfolio HTTP (SQLite)', () => {
const stale = await get( const stale = await get(
'brokerage-accounts/all/portfolio?sync=observe', 'brokerage-accounts/all/portfolio?sync=observe',
).expect(200); ).expect(200);
expect((stale.body as PortfolioResourceDto).meta.freshness).toBe('stale'); expect((stale.body as PortfolioResourceDto).meta.freshness).toBe('fresh');
expect((stale.body as PortfolioResourceDto).data?.isStale).toBe(true); expect((stale.body as PortfolioResourceDto).data?.isStale).toBe(false);
expect(upstream.getOpenAccounts).not.toHaveBeenCalled(); expect(upstream.getOpenAccounts).not.toHaveBeenCalled();
expect(upstream.getPortfolio).not.toHaveBeenCalled(); expect(upstream.getPortfolio).not.toHaveBeenCalled();
}); });
@@ -424,16 +558,22 @@ describe('local accounts and portfolio HTTP (SQLite)', () => {
it('publishes complete empty membership and fences a late portfolio after connection replacement', async () => { it('publishes complete empty membership and fences a late portfolio after connection replacement', async () => {
upstream.validateToken.mockResolvedValueOnce([]); upstream.validateToken.mockResolvedValueOnce([]);
await put().expect(200); await put().expect(200);
await prisma.tbankAccount.deleteMany({ where: { userId: 'owner' } });
await prisma.tbankConnection.updateMany({
data: { membershipAt: new Date() },
});
const empty = await get('brokerage-accounts?sync=observe').expect(200); const empty = await get('brokerage-accounts?sync=observe').expect(200);
expect((empty.body as BrokerageAccountsResourceDto).data).toEqual([]); expect((empty.body as BrokerageAccountsResourceDto).data).toEqual([]);
expect((empty.body as BrokerageAccountsResourceDto).meta.availability).toBe( expect((empty.body as BrokerageAccountsResourceDto).meta.availability).toBe(
'complete', 'available',
); );
await put().expect(200); await put().expect(200);
await seedMembership();
const blocked = deferred<TbankPortfolio>(); const blocked = deferred<TbankPortfolio>();
upstream.getPortfolio.mockReturnValueOnce(blocked.promise); upstream.getPortfolio.mockReturnValueOnce(blocked.promise);
await get('brokerage-accounts/account/portfolio').expect(200); await get('brokerage-accounts/account/portfolio').expect(200);
await put().expect(200); await put().expect(200);
await seedMembership();
blocked.resolve({ positions: [] }); blocked.resolve({ positions: [] });
await delay(20); await delay(20);
const replaced = await get( const replaced = await get(
+10 -11
View File
@@ -204,27 +204,28 @@ describe('durable shared instrument metadata (SQLite)', () => {
await settled(); await settled();
expect((await service.read('other', identity, true))?.data).toEqual(known); expect((await service.read('other', identity, true))?.data).toEqual(known);
}); });
it('keeps last good metadata and source time after refresh failure; stale observation does not restart work', async () => { it('keeps last good metadata and source time after refresh failure; observation does not restart work', async () => {
await service.read('owner', identity); await service.read('owner', identity);
await settled(); await settled();
const old = new Date(Date.now() - 86_400_000); const old = new Date(Date.now() - 10 * 86_400_000);
await prisma.tbankInstrumentLookup.updateMany({ data: { fetchedAt: old } }); await prisma.tbankInstrumentLookup.updateMany({ data: { fetchedAt: old } });
await prisma.tbankInstrument.updateMany({ data: { fetchedAt: old } }); await prisma.tbankInstrument.updateMany({ data: { fetchedAt: old } });
const calls = upstream.getInstrumentByUid.mock.calls.length; const calls = upstream.getInstrumentByUid.mock.calls.length;
expect((await service.read('owner', identity, true))?.fresh).toBe(false); expect((await service.read('owner', identity, true))?.fresh).toBe(true);
expect(upstream.getInstrumentByUid).toHaveBeenCalledTimes(calls); expect(upstream.getInstrumentByUid).toHaveBeenCalledTimes(calls);
const failedHandle = await service.request('owner', 'owner-revision', identity, true);
upstream.getInstrumentByUid.mockRejectedValueOnce( upstream.getInstrumentByUid.mockRejectedValueOnce(
new Error('upstream unavailable'), new Error('upstream unavailable'),
); );
upstream.getInstrumentByFigi.mockRejectedValueOnce( upstream.getInstrumentByFigi.mockRejectedValueOnce(
new Error('upstream unavailable'), new Error('upstream unavailable'),
); );
await service.read('owner', identity);
await settled(); await settled();
const failed = await service.read('owner', identity, true); const failed = await service.read('owner', identity, true);
expect(failed?.data).toEqual(known); expect(failed?.data).toEqual(known);
expect(failed?.fetchedAt).toEqual(old); expect(failed?.fetchedAt).toEqual(old);
expect(failed?.work?.state).toBe('failed'); if (!failedHandle) throw new Error('Expected forced refresh work');
expect((await prisma.tbankSyncWork.findUniqueOrThrow({ where: { id: failedHandle } })).state).toBe('failed');
await delay(20); await delay(20);
expect(upstream.getInstrumentByUid).toHaveBeenCalledTimes(calls + 1); expect(upstream.getInstrumentByUid).toHaveBeenCalledTimes(calls + 1);
}); });
@@ -252,20 +253,18 @@ describe('durable shared instrument metadata (SQLite)', () => {
expect(after?.work?.state).toBe('failed'); expect(after?.work?.state).toBe('failed');
expect(upstream.getInstrumentByUid).toHaveBeenCalledTimes(2); expect(upstream.getInstrumentByUid).toHaveBeenCalledTimes(2);
}); });
it('uses the exact daily TTL boundary and does not count another instrument as covered', async () => { it('keeps saved metadata fresh and does not count another instrument as covered', async () => {
await service.read('owner', identity); await service.read('owner', identity);
await settled(); await settled();
const timestamp = new Date('2026-09-01Z'); const timestamp = new Date('2026-09-01Z');
await prisma.tbankInstrumentLookup.updateMany({ await prisma.tbankInstrumentLookup.updateMany({
data: { fetchedAt: timestamp }, data: { fetchedAt: timestamp },
}); });
const clock = jest const clock = jest.spyOn(Date, 'now').mockReturnValue(timestamp.valueOf());
.spyOn(Date, 'now')
.mockReturnValue(timestamp.valueOf() + 86_400_000 - 1);
try { try {
expect((await service.read('owner', identity, true))?.fresh).toBe(true); expect((await service.read('owner', identity, true))?.fresh).toBe(true);
clock.mockReturnValue(timestamp.valueOf() + 86_400_000); clock.mockReturnValue(timestamp.valueOf() + 10 * 86_400_000);
expect((await service.read('owner', identity, true))?.fresh).toBe(false); expect((await service.read('owner', identity, true))?.fresh).toBe(true);
expect( expect(
(await service.read('owner', { instrumentUid: 'other-uid' }, true)) (await service.read('owner', { instrumentUid: 'other-uid' }, true))
?.complete, ?.complete,
+7
View File
@@ -7,6 +7,7 @@ import { DatabaseSync } from 'node:sqlite';
import { PrismaClient } from '../generated/prisma/client'; import { PrismaClient } from '../generated/prisma/client';
const migration = '20260924170000_persist_tbank_data'; const migration = '20260924170000_persist_tbank_data';
const nextMigration = '20261007181218_add_portfolio_value_history';
const migrations = join(__dirname, '../prisma/migrations'); const migrations = join(__dirname, '../prisma/migrations');
const preserved = [ const preserved = [
'User', 'User',
@@ -237,6 +238,12 @@ describe('T-Bank additive migration on real SQLite', () => {
await prisma.$disconnect(); await prisma.$disconnect();
} }
db = new DatabaseSync(file); db = new DatabaseSync(file);
db.exec('PRAGMA foreign_keys=ON');
const beforeNext = snapshot();
db.exec(
readFileSync(join(migrations, nextMigration, 'migration.sql'), 'utf8'),
);
expect(snapshot()).toEqual(beforeNext);
const plan = db const plan = db
.prepare( .prepare(
'EXPLAIN QUERY PLAN SELECT * FROM TbankOperation WHERE userId=? AND revision=? AND accountId=? ORDER BY occurredAt, brokerOperationId LIMIT 50', 'EXPLAIN QUERY PLAN SELECT * FROM TbankOperation WHERE userId=? AND revision=? AND accountId=? ORDER BY occurredAt, brokerOperationId LIMIT 50',
+2 -1
View File
@@ -319,7 +319,7 @@ describe('Request-driven operations synchronization (SQLite)', () => {
expect(observed.plan?.intervals[0]).toMatchObject({ expect(observed.plan?.intervals[0]).toMatchObject({
availability: 'complete', availability: 'complete',
asOf, asOf,
stale: true, stale: false,
}); });
expect(upstream.getOperationsByCursor).not.toHaveBeenCalled(); expect(upstream.getOperationsByCursor).not.toHaveBeenCalled();
upstream.getOperationsByCursor.mockResolvedValue({ upstream.getOperationsByCursor.mockResolvedValue({
@@ -328,6 +328,7 @@ describe('Request-driven operations synchronization (SQLite)', () => {
}); });
const refresh = await service.readPlan('owner', [target], { const refresh = await service.readPlan('owner', [target], {
movingEnd: true, movingEnd: true,
force: true,
}); });
await state(requiredHandle(refresh.handle), 'complete'); await state(requiredHandle(refresh.handle), 'complete');
expect(upstream.getOperationsByCursor).toHaveBeenCalledTimes(1); expect(upstream.getOperationsByCursor).toHaveBeenCalledTimes(1);
+25 -18
View File
@@ -263,6 +263,26 @@ describe('T-Bank persisted read budget', () => {
historyBoundary: 'openedOn', historyBoundary: 'openedOn',
}, },
}); });
await prisma.tbankPortfolio.create({
data: {
...scope,
payload: {
lastSuccessfulAt: now.toISOString(),
isStale: false,
positions: [{
instrumentUid: instruments[0].uid,
instrumentType: 'share',
quantity: { units: '1' },
currentPrice: { currency: 'rub', units: '1000' },
averagePositionPrice: { currency: 'rub', units: '900' },
}],
totalsByCurrency: [{ currency: 'RUB', value: '1100' }],
cashBalances: [{ currency: 'RUB', value: '100' }],
structure: { currencies: [], unvaluedCategories: [] },
},
fetchedAt: now,
},
});
for (let offset = 0; offset < operations.length; offset += 1000) { for (let offset = 0; offset < operations.length; offset += 1000) {
await prisma.tbankOperation.createMany({ await prisma.tbankOperation.createMany({
data: operations.slice(offset, offset + 1000).map((row) => ({ data: operations.slice(offset, offset + 1000).map((row) => ({
@@ -348,21 +368,6 @@ describe('T-Bank persisted read budget', () => {
'/brokerage-accounts/all/operations?from=2026-08-01&to=2026-08-31&limit=50', '/brokerage-accounts/all/operations?from=2026-08-01&to=2026-08-31&limit=50',
'/brokerage-accounts/baseline-0/income-analytics?from=2026-08-01&to=2026-08-31', '/brokerage-accounts/baseline-0/income-analytics?from=2026-08-01&to=2026-08-31',
'/brokerage-accounts/all/income-analytics?from=2026-08-01&to=2026-08-31', '/brokerage-accounts/all/income-analytics?from=2026-08-01&to=2026-08-31',
'/brokerage-accounts/baseline-0/events?from=2026-09-01&to=2026-10-31',
'/brokerage-accounts/all/events?from=2026-09-01&to=2026-10-31',
'/stocks?query=SYNTH',
'/bonds?query=SYNTH',
...[
['stocks', instruments[0].uid, 'dividends'],
['bonds', instruments[1].uid, 'coupons'],
].flatMap(([kind, uid, payout]) => [
`/${kind}/${uid}`,
`/${kind}/${uid}/quote`,
`/${kind}/${uid}/${kind === 'stocks' ? 'company' : 'metrics'}`,
`/${kind}/${uid}/candles?period=year`,
`/${kind}/${uid}/candles?period=all`,
`/${kind}/${uid}/${payout}`,
]),
]; ];
async function settled() { async function settled() {
for (let index = 0; index < 6000; index++) { for (let index = 0; index < 6000; index++) {
@@ -395,7 +400,9 @@ describe('T-Bank persisted read budget', () => {
await request(app.getHttpServer()) await request(app.getHttpServer())
.get(`/api/v1${route}`) .get(`/api/v1${route}`)
.set('Authorization', `Bearer ${token}`) .set('Authorization', `Bearer ${token}`)
.expect(200); .expect((response) => {
if (response.status !== 200) throw new Error(`${route}: ${JSON.stringify(response.body)}`);
});
await settled(); await settled();
} }
await fresh(); await fresh();
@@ -560,8 +567,8 @@ describe('T-Bank persisted read budget', () => {
} }
const staleResponsesMs = performance.now() - startStale; const staleResponsesMs = performance.now() - startStale;
await settled(); await settled();
expect(rpc.portfolio - beforeStale.portfolio).toBe(1); expect((rpc.portfolio ?? 0) - (beforeStale.portfolio ?? 0)).toBe(0);
expect(rpc.positions - beforeStale.positions).toBe(1); expect((rpc.positions ?? 0) - (beforeStale.positions ?? 0)).toBe(0);
await app await app
.get(PrismaService) .get(PrismaService)
.tbankPortfolio.deleteMany({ where: { accountId: 'baseline-0' } }); .tbankPortfolio.deleteMany({ where: { accountId: 'baseline-0' } });

Some files were not shown because too many files have changed in this diff Show More