docs(phase-1): add research and validation strategy
This commit is contained in:
parent
be8bc911e0
commit
a433a2c231
1 changed files with 103 additions and 0 deletions
103
.planning/phases/01-matrix-qa-polish/01-VALIDATION.md
Normal file
103
.planning/phases/01-matrix-qa-polish/01-VALIDATION.md
Normal file
|
|
@ -0,0 +1,103 @@
|
|||
---
|
||||
phase: 1
|
||||
slug: matrix-qa-polish
|
||||
status: draft
|
||||
nyquist_compliant: false
|
||||
wave_0_complete: false
|
||||
created: 2026-04-02
|
||||
---
|
||||
|
||||
# Phase 1 — Validation Strategy
|
||||
|
||||
> Per-phase validation contract for feedback sampling during execution.
|
||||
|
||||
---
|
||||
|
||||
## Test Infrastructure
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| **Framework** | pytest + pytest-asyncio |
|
||||
| **Config file** | `pyproject.toml` |
|
||||
| **Quick run command** | `pytest tests/adapter/matrix/ -q` |
|
||||
| **Full suite command** | `pytest tests/ -q` |
|
||||
| **Estimated runtime** | ~10 seconds |
|
||||
|
||||
---
|
||||
|
||||
## Sampling Rate
|
||||
|
||||
- **After every task commit:** Run `pytest tests/adapter/matrix/ -q`
|
||||
- **After every plan wave:** Run `pytest tests/ -q`
|
||||
- **Before `/gsd:verify-work`:** Full suite must be green (96+ tests)
|
||||
- **Max feedback latency:** 15 seconds
|
||||
|
||||
---
|
||||
|
||||
## Per-Task Verification Map
|
||||
|
||||
| Task ID | Plan | Wave | Behavior | Test Type | Automated Command | Status |
|
||||
|---------|------|------|----------|-----------|-------------------|--------|
|
||||
| MAT-01 | 01 | 1 | handle_invite creates Space + Чат 1 | unit | `pytest tests/adapter/matrix/test_invite_space.py -x -q` | ⬜ pending |
|
||||
| MAT-02 | 01 | 1 | handle_invite idempotent | unit | `pytest tests/adapter/matrix/test_invite_space.py -x -q` | ⬜ pending |
|
||||
| MAT-03 | 01 | 1 | no hardcoded C1 | unit | `pytest tests/adapter/matrix/test_invite_space.py -x -q` | ⬜ pending |
|
||||
| MAT-04 | 02 | 1 | !new adds room to Space | unit | `pytest tests/adapter/matrix/test_chat_space.py -x -q` | ⬜ pending |
|
||||
| MAT-05 | 02 | 1 | !new without space_id returns error | unit | `pytest tests/adapter/matrix/test_chat_space.py -x -q` | ⬜ pending |
|
||||
| MAT-06 | 03 | 1 | OutgoingUI renders text + !yes/!no | unit | `pytest tests/adapter/matrix/test_send_outgoing.py -x -q` | ⬜ pending |
|
||||
| MAT-07 | 03 | 1 | OutgoingUI does NOT send m.reaction | unit | `pytest tests/adapter/matrix/test_send_outgoing.py -x -q` | ⬜ pending |
|
||||
| MAT-08 | 03 | 1 | pending_confirm store roundtrip | unit | `pytest tests/adapter/matrix/test_store.py -x -q` | ⬜ pending |
|
||||
| MAT-09 | 03 | 2 | !yes/!no reads pending_confirm | unit | `pytest tests/adapter/matrix/test_confirm.py -x -q` | ⬜ pending |
|
||||
| MAT-10 | 02 | 2 | !archive calls room_put_state empty | unit | `pytest tests/adapter/matrix/test_chat_space.py -x -q` | ⬜ pending |
|
||||
| MAT-11 | 04 | 2 | !settings returns dashboard | unit | `pytest tests/adapter/matrix/test_dispatcher.py -x -q` | ⬜ pending |
|
||||
| MAT-12 | 02 | 1 | RoomCreateError → user message | unit | `pytest tests/adapter/matrix/test_chat_space.py -x -q` | ⬜ pending |
|
||||
|
||||
*Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky*
|
||||
|
||||
---
|
||||
|
||||
## Wave 0 Requirements
|
||||
|
||||
- [ ] `tests/adapter/matrix/test_invite_space.py` — stubs for MAT-01..03
|
||||
- [ ] `tests/adapter/matrix/test_chat_space.py` — stubs for MAT-04..05, MAT-10, MAT-12
|
||||
- [ ] `tests/adapter/matrix/test_send_outgoing.py` — stubs for MAT-06..07
|
||||
- [ ] `tests/adapter/matrix/test_confirm.py` — stubs for MAT-09
|
||||
|
||||
Existing files to update (not create):
|
||||
- `tests/adapter/matrix/test_store.py` — add MAT-08
|
||||
- `tests/adapter/matrix/test_dispatcher.py` — add MAT-11, update broken DM-based tests
|
||||
|
||||
---
|
||||
|
||||
## Broken Tests (Must Fix)
|
||||
|
||||
These pass today but will break after the Space+rooms refactor:
|
||||
|
||||
| Test | Why it breaks | Fix |
|
||||
|------|--------------|-----|
|
||||
| `test_dispatcher.py::test_invite_event_creates_dm_room_and_sends_welcome` | Asserts `chat_id == "C1"` hardcode, DM join | Rewrite for Space creation |
|
||||
| `test_dispatcher.py::test_new_chat_creates_real_matrix_room_when_client_available` | No `room_put_state` in mock assertions | Update mock + assertions |
|
||||
| `test_reactions.py::test_build_skills_text` | Expects "Реакции 1️⃣-9️⃣" in text | Update assertion |
|
||||
| `test_reactions.py::test_build_confirmation_text` | Expects `CONFIRM_REACTION` | Update for !yes/!no |
|
||||
|
||||
---
|
||||
|
||||
## Manual-Only Verifications
|
||||
|
||||
| Behavior | Why Manual | Test Instructions |
|
||||
|----------|------------|-------------------|
|
||||
| First invite creates visible Space in Element | Element client rendering | Invite bot, check Space appears in sidebar |
|
||||
| !new creates room inside Space (not standalone) | Space membership UI | Run !new, verify room appears under Space |
|
||||
| !archive removes room from Space sidebar | Element room list | Run !archive, verify room disappears from Space |
|
||||
|
||||
---
|
||||
|
||||
## Validation Sign-Off
|
||||
|
||||
- [ ] All tasks have `<automated>` verify or Wave 0 dependencies
|
||||
- [ ] Sampling continuity: no 3 consecutive tasks without automated verify
|
||||
- [ ] Wave 0 covers all MISSING test files
|
||||
- [ ] No watch-mode flags
|
||||
- [ ] Feedback latency < 15s
|
||||
- [ ] `nyquist_compliant: true` set in frontmatter
|
||||
|
||||
**Approval:** pending
|
||||
Loading…
Add table
Add a link
Reference in a new issue