186 lines
8.0 KiB
Markdown
186 lines
8.0 KiB
Markdown
---
|
|
description: "Verify implementation matches change artifacts before archiving"
|
|
---
|
|
|
|
Verify that an implementation matches the change artifacts (specs, tasks, design).
|
|
|
|
**OpenSpec store (WealthLedger):** Use only `wealth-ledger-specs`, whose physical root is `specs/openspec`. Every OpenSpec command that reads or writes specs or changes MUST include `--store wealth-ledger-specs`; keep this flag on every follow-up. Do not infer a store from cwd, the nearest root, or machine configuration.
|
|
|
|
**OpenSpec version preflight:** Before the first OpenSpec command, work from the workspace root and run `bash specs/scripts/verify-openspec-version.sh`. A missing or mismatched CLI is a blocking result.
|
|
|
|
**Planning gate:** Before delivery verification run `bash specs/scripts/verify-openspec-planning.sh "<name>"`. The workspace verifier repeats this gate before affected-area checks. Structural success does not replace semantic verification or independent read-only review.
|
|
|
|
**Input**: Optionally specify a change name after `/opsx-verify` (e.g., `/opsx-verify add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
|
|
|
|
**Steps**
|
|
|
|
1. **Select the change**
|
|
|
|
If a name is provided, use it. Otherwise:
|
|
- Infer from conversation context if the user mentioned a change
|
|
- Auto-select if only one active change exists
|
|
- If ambiguous, run `openspec list --json --store wealth-ledger-specs` to get available changes and ask the user to select one
|
|
|
|
When prompting, show changes that have implementation tasks (tasks artifact exists).
|
|
Include the schema used for each change if available.
|
|
Mark changes with incomplete tasks as "(In Progress)".
|
|
|
|
Always announce: "Using change: <name>" and how to override (e.g., `/opsx-verify <other>`).
|
|
|
|
2. **Check status to understand the schema**
|
|
```bash
|
|
openspec status --change "<name>" --json --store wealth-ledger-specs
|
|
```
|
|
Parse the JSON to understand:
|
|
- `schemaName`: The workflow being used (e.g., "spec-driven")
|
|
- `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path and scope context
|
|
- Which artifacts exist for this change
|
|
|
|
3. **Get planning context and load artifacts**
|
|
|
|
```bash
|
|
openspec instructions apply --change "<name>" --json --store wealth-ledger-specs
|
|
```
|
|
|
|
This returns the change directory and `contextFiles` (artifact ID -> array of concrete file paths). Read all available artifacts from `contextFiles`.
|
|
|
|
Read the Design Delivery Map and account for frontend, backend, specs and
|
|
docs. Run the workspace verifier from the workspace root:
|
|
|
|
```bash
|
|
bash specs/scripts/verify-workspace-change.sh "<name>"
|
|
```
|
|
|
|
A missing decision or failed canonical gate for any affected delivery area
|
|
is CRITICAL. The semantic report does not replace independent review:
|
|
require evidence of a completed independent read-only review and report its
|
|
absence or any unresolved Critical/Important finding as CRITICAL.
|
|
|
|
4. **Initialize verification report structure**
|
|
|
|
Create a report structure with three dimensions:
|
|
- **Completeness**: Track tasks and spec coverage
|
|
- **Correctness**: Track requirement implementation and scenario coverage
|
|
- **Coherence**: Track design adherence and pattern consistency
|
|
|
|
Each dimension can have CRITICAL, WARNING, or SUGGESTION issues.
|
|
|
|
5. **Verify Completeness**
|
|
|
|
**Task Completion**:
|
|
- If `contextFiles.tasks` exists, read every file path in it
|
|
- Parse checkboxes: `- [ ]` (incomplete) vs `- [x]` (complete)
|
|
- Count complete vs total tasks
|
|
- If incomplete tasks exist:
|
|
- Add CRITICAL issue for each incomplete task
|
|
- Recommendation: "Complete task: <description>" or "Mark as done if already implemented"
|
|
|
|
**Spec Coverage**:
|
|
- If delta specs exist in `contextFiles.specs`:
|
|
- Extract all requirements (marked with "### Requirement:")
|
|
- For each requirement:
|
|
- Search codebase for keywords related to the requirement
|
|
- Assess if implementation likely exists
|
|
- If requirements appear unimplemented:
|
|
- Add CRITICAL issue: "Requirement not found: <requirement name>"
|
|
- Recommendation: "Implement requirement X: <description>"
|
|
|
|
6. **Verify Correctness**
|
|
|
|
**Requirement Implementation Mapping**:
|
|
- For each requirement from delta specs:
|
|
- Search codebase for implementation evidence
|
|
- If found, note file paths and line ranges
|
|
- Assess if implementation matches requirement intent
|
|
- If divergence detected:
|
|
- Add WARNING: "Implementation may diverge from spec: <details>"
|
|
- Recommendation: "Review <file>:<lines> against requirement X"
|
|
|
|
**Scenario Coverage**:
|
|
- For each scenario in delta specs (marked with "#### Scenario:"):
|
|
- Check if conditions are handled in code
|
|
- Check if tests exist covering the scenario
|
|
- If scenario appears uncovered:
|
|
- Add WARNING: "Scenario not covered: <scenario name>"
|
|
- Recommendation: "Add test or implementation for scenario: <description>"
|
|
|
|
7. **Verify Coherence**
|
|
|
|
**Design Adherence**:
|
|
- If `contextFiles.design` exists:
|
|
- Extract key decisions (look for sections like "Decision:", "Approach:", "Architecture:")
|
|
- Verify implementation follows those decisions
|
|
- If contradiction detected:
|
|
- Add WARNING: "Design decision not followed: <decision>"
|
|
- Recommendation: "Update implementation or revise design.md to match reality"
|
|
- If no design.md: Skip design adherence check, note "No design.md to verify against"
|
|
|
|
**Code Pattern Consistency**:
|
|
- Review new code for consistency with project patterns
|
|
- Check file naming, directory structure, coding style
|
|
- If significant deviations found:
|
|
- Add SUGGESTION: "Code pattern deviation: <details>"
|
|
- Recommendation: "Consider following project pattern: <example>"
|
|
|
|
8. **Generate Verification Report**
|
|
|
|
**Summary Scorecard**:
|
|
```markdown
|
|
## Verification Report: <change-name>
|
|
|
|
### Summary
|
|
| Dimension | Status |
|
|
|--------------|------------------|
|
|
| Completeness | X/Y tasks, N reqs|
|
|
| Correctness | M/N reqs covered |
|
|
| Coherence | Followed/Issues |
|
|
```
|
|
|
|
**Issues by Priority**:
|
|
|
|
1. **CRITICAL** (Must fix before archive):
|
|
- Incomplete tasks
|
|
- Missing requirement implementations
|
|
- Missing or failed affected delivery area gate
|
|
- Missing independent read-only review
|
|
- Each with specific, actionable recommendation
|
|
|
|
2. **WARNING** (Should fix):
|
|
- Spec/design divergences
|
|
- Missing scenario coverage
|
|
- Each with specific recommendation
|
|
|
|
3. **SUGGESTION** (Nice to fix):
|
|
- Pattern inconsistencies
|
|
- Minor improvements
|
|
- Each with specific recommendation
|
|
|
|
**Final Assessment**:
|
|
- If CRITICAL issues: "X critical issue(s) found. Fix before archiving."
|
|
- If only warnings: "No critical issues. Y warning(s) to consider. Ready for archive (with noted improvements)."
|
|
- If all clear: "All checks passed. Ready for archive."
|
|
|
|
**Verification Heuristics**
|
|
|
|
- **Completeness**: Focus on objective checklist items (checkboxes, requirements list)
|
|
- **Correctness**: Use keyword search, file path analysis, reasonable inference - don't require perfect certainty
|
|
- **Coherence**: Look for glaring inconsistencies, don't nitpick style
|
|
- **False Positives**: When uncertain, prefer SUGGESTION over WARNING, WARNING over CRITICAL
|
|
- **Actionability**: Every issue must have a specific recommendation with file/line references where applicable
|
|
|
|
**Graceful Degradation**
|
|
|
|
- If only tasks.md exists: verify task completion only, skip spec/design checks
|
|
- If tasks + specs exist: verify completeness and correctness, skip design
|
|
- If full artifacts: verify all three dimensions
|
|
- Always note which checks were skipped and why
|
|
|
|
**Output Format**
|
|
|
|
Use clear markdown with:
|
|
- Table for summary scorecard
|
|
- Grouped lists for issues (CRITICAL/WARNING/SUGGESTION)
|
|
- Code references in format: `file.ts:123`
|
|
- Specific, actionable recommendations
|
|
- No vague suggestions like "consider reviewing"
|