Compare commits

...
Sign in to create a new pull request.

113 commits

Author SHA1 Message Date
3340c126d6 docs: remove legacy threads and reports from planning state 2026-05-03 23:42:34 +03:00
9fc0b72ab1 docs: clean up GSD planning state and remove outdated legacy phases 2026-05-03 23:42:05 +03:00
65445f516f docs: map codebase 2026-05-03 00:31:20 +03:00
6dde5be17d docs: simplify testing section in new surface guide 2026-05-03 00:11:01 +03:00
7b2543aee7 docs: add local fullstack e2e instructions to new surface guide 2026-05-03 00:06:16 +03:00
e7e3912b5f docs: generalize new surface guide and clean up legacy docs 2026-05-03 00:01:25 +03:00
0f79494fbe feat(deploy): finalize MVP deployment and file transfer approach 2026-05-02 23:45:52 +03:00
6369721876 wip: 05-mvp-deployment paused at task 0/0 2026-04-30 18:04:24 +03:00
7e5f9c20a0 wip: Phase 05 complete, amd64 image rebuilt 2026-04-29 00:07:25 +03:00
5679b95450 wip: phase 05 paused after deployment handoff 2026-04-28 21:41:13 +03:00
5b537880ae docs(deploy): finalize multi-agent surface image handoff 2026-04-28 20:11:27 +03:00
51241d79e0 docs: fix README for platform integration — per-agent routing, compose as template 2026-04-28 03:26:09 +03:00
6d2d58f05d docs: update deploy-architecture and README for per-agent routing 2026-04-28 03:23:56 +03:00
4bbae9affa feat(deploy): per-agent base_url and workspace_path routing
- AgentDefinition gains base_url and workspace_path fields (optional)
- load_agent_registry parses them from matrix-agents.yaml
- _build_platform_from_env uses agent.base_url per agent (falls back to AGENT_BASE_URL)
- _agent_workspace_root() resolves workspace per agent from registry
- _materialize_incoming_attachments saves files to agent workspace_path/incoming/
- send_outgoing accepts workspace_root param; reads outgoing files from agent workspace_path
- dispatch loop computes workspace_root from room agent_id and passes to _send_all
- config/matrix-agents.yaml and example updated with base_url and workspace_path
2026-04-28 03:22:21 +03:00
d6b7720eca docs: add platform integration guide to README 2026-04-28 03:07:45 +03:00
b1aaa210a1 feat(deploy): platform handoff — agent routing, persistence, docs cleanup
Agent routing:
- Remove !agent command and manual agent selection flow
- Registry auto-assigns agent from user_agents mapping (fallback: agents[0])
- provision_workspace_chat and !new both write agent_id to room_meta
- Reconciliation backfills agent_id from registry on cold start
- Fix duplicate agent_id block in auth.py

Deployment stability:
- Add bot-state named volume to persist lambda_matrix.db and matrix_store
- Fix docker-compose.prod.yml duplicate environment: key (was silently losing all Matrix credentials)
- Fix MATRIX_AGENT_REGISTRY_PATH to use absolute container path /app/config/...
- Add bot-state volume declaration to docker-compose.fullstack.yml

Docs and config:
- Rewrite README.md for platform handoff (deploy table, working commands only)
- Rewrite docs/matrix-prototype.md (remove stale commands and mock descriptions)
- Remove !save/!load/!context/!agent from help text and welcome message
- Add !clear, !list, !remove, !yes/!no to help text
- Clean up .env.example (remove Telegram token, internal vars, real URLs)
- Update config/matrix-agents.example.yaml with user_agents section and comments
- Add explanatory comment to Dockerfile for --ignore-requires-python
- Remove silent uv sync fallbacks in Dockerfile
2026-04-28 03:05:11 +03:00
380961d6e9 docs(05-04): complete split deployment artifacts plan
- add phase summary for split deployment artifacts
- update state with phase 05 completion context
2026-04-28 01:18:47 +03:00
e73e13e758 docs(05-02): complete room-local clear plan
- add execution summary for room-local clear and strict routing
- update roadmap and state with plan 05-02 completion metadata
2026-04-28 01:17:48 +03:00
22a3a2b60a docs(05-04): document split deployment artifacts
- document prod vs fullstack compose usage
- align operator docs with shared /agents contract
2026-04-28 01:15:41 +03:00
85e2fda6bc feat(05-02): ship room-local clear semantics
- register clear as the room-context reset entrypoint when supported
- keep save and context bound to room platform chat ids and clear old upstream state
2026-04-28 01:15:39 +03:00
df6d8bf628 feat(05-04): split prod and fullstack compose artifacts
- add bot-only production compose contract
- add health-gated internal fullstack harness
2026-04-28 01:14:05 +03:00
ae37476ddf test(05-02): add failing regressions for clear routing
- cover room-local clear rotation and upstream disconnect behavior
- assert strict routed-platform failures on incomplete room bindings
2026-04-28 01:13:54 +03:00
8a80d004fd feat(05-01): reconcile matrix rooms before live sync
- rebuild room and user metadata from synced space topology at startup
- run reconciliation before sync_forever and persist legacy platform_chat_id backfills
2026-04-28 01:08:15 +03:00
6693d72cbd docs(05-03): complete shared-volume attachment hardening plan
- add 05-03 summary with task commits and verification details
- update roadmap and state for completed Phase 05 plan 03
2026-04-28 01:07:35 +03:00
a75b26a1cb test(05-01): add restart reconciliation regression coverage
- add startup reconciliation tests for recovery, idempotence, and startup ordering
- extend restart persistence coverage for legacy platform_chat_id backfill
2026-04-28 01:05:59 +03:00
9a0316076a fix(05-03): normalize shared-volume attachment paths
- strip /workspace and /agents roots before forwarding attachments upstream
- reuse the same normalization for send-file events returned to Matrix
2026-04-28 01:05:15 +03:00
cafb0ec9e4 test(05-03): add failing shared-volume attachment contract tests
- cover room-safe Matrix inbox paths under /agents workspaces
- assert /workspace and /agents file paths normalize to relative workspace paths
2026-04-28 01:04:31 +03:00
26eb27b01e docs(05): research mvp deployment phase 2026-04-28 00:42:24 +03:00
0f07634955 docs(05): add validation strategy 2026-04-27 23:00:33 +03:00
1a8f9cdca0 docs(05): research phase — DM-first onboarding, per-agent routing, file transfer, prod compose 2026-04-27 22:59:31 +03:00
e5c394f036 docs(05): finalize context — unauthorized users, !clear no-confirm, remove !settings 2026-04-27 22:51:49 +03:00
daa780c0b8 docs(05): single-chat arch + DM-first onboarding + !clear 2026-04-27 22:25:24 +03:00
e20634902e docs(05): update docker-compose decision — full stack with placeholder agent image 2026-04-27 22:17:34 +03:00
6553320001 docs(05): capture phase context 2026-04-27 22:13:52 +03:00
8ffbe7b6b3 wip: deployment architecture research — Phase 05 ready to plan
- docs/deploy-architecture.md: full deployment topology, agent API, file transfer via shared volume
- .planning/HANDOFF.json + .continue-here.md: session state for Phase 05 planning
2026-04-27 21:46:27 +03:00
c34db0e6c0 wip: first-chunk debug logging — paused waiting for platform-agent logs 2026-04-24 15:17:08 +03:00
2a23b30f83 chore: update STATE after Phase 04 multi-agent follow-up 2026-04-24 14:12:07 +03:00
e733119d1e feat: enforce agent routing and persist restart state
Task 4: stale room blocking + agent_id binding
- MatrixBot._check_agent_routing: blocks normal messages when user has no
  selected agent or room is bound to a different agent
- agent_routing_enabled flag on MatrixRuntime activates the check only
  in real multi-agent mode (RoutedPlatformClient)
- make_handle_new_chat now writes agent_id into new room metadata when
  user already has a selected agent

Task 5: durable restart state tests
- test_restart_persistence.py proves selected_agent_id, room agent_id,
  platform_chat_id, and the sequence counter all survive SQLiteStore
  close/reopen; also covers clean startup with no prior state
2026-04-24 14:01:49 +03:00
74cf028e8f feat: add !agent command and durable user agent selection
Users can now list available agents with !agent and select one by
number. Selection persists in user metadata (selected_agent_id). If the
current room has no agent binding yet, selecting an agent binds it
immediately so the user can start messaging without !new.

Also updates the dispatcher test to reflect that real-mode platform is
now RoutedPlatformClient, not a bare RealPlatformClient.
2026-04-24 13:54:25 +03:00
a65227e490 test: align matrix dispatch chat id contract 2026-04-24 13:29:49 +03:00
9ccba161a2 fix: require matrix agent registry in real mode 2026-04-24 13:24:56 +03:00
242f4aadd3 feat: add matrix routed platform facade 2026-04-24 13:22:05 +03:00
7627012f24 Keep Matrix registry docs preparatory 2026-04-24 13:14:52 +03:00
98caca100c Clarify Matrix agent registry docs 2026-04-24 13:12:29 +03:00
3b0401fb7c Require string agent registry fields 2026-04-24 13:11:02 +03:00
25aa5d9313 Make Matrix agent registry immutable 2026-04-24 13:08:25 +03:00
2fb6c10a5a Reject null agent registry fields 2026-04-24 13:05:26 +03:00
e801225220 Tighten Matrix agent registry validation 2026-04-24 13:02:19 +03:00
b53523ad6c Reject non-mapping agent registry entries 2026-04-24 12:57:00 +03:00
37f7ce27a2 Add Matrix agent registry loader 2026-04-24 12:54:30 +03:00
32b03becc8 docs: clarify matrix multi-agent routing specs 2026-04-24 12:42:58 +03:00
842117900a test: cover agent api base url suffix handling 2026-04-24 12:39:50 +03:00
59fbb52c20 docs: add matrix multi-agent and restart state specs 2026-04-24 12:28:53 +03:00
76230392fa fix: normalize attachments to core Attachment type in message handler
Upstream AgentApi responses can return attachment objects that don't
implement the Attachment dataclass. _to_core_attachments coerces them
via duck-typing so OutgoingMessage always carries typed Attachment
instances regardless of the upstream response shape.
2026-04-23 14:56:00 +03:00
be4607b422 wip: 04-matrix-mvp-shared-agent-context-and-context-management-comma paused at task 3/3 2026-04-23 14:53:30 +03:00
7d58dd1caf fix: use direct agent api per request 2026-04-22 15:31:28 +03:00
7d270d3d31 chore: save handoff context for next agents 2026-04-22 01:34:47 +03:00
0c2884c2b1 refactor: use thin upstream transport adapter 2026-04-22 01:25:11 +03:00
569824ead1 refactor: shrink agent api wrapper to thin adapter 2026-04-22 00:22:20 +03:00
4d917ac794 docs: add thin transport adapter plan 2026-04-22 00:17:15 +03:00
3a3fcdc695 docs: add thin transport adapter design 2026-04-22 00:11:20 +03:00
7a2ad86b88 docs: clarify matrix file sending flow 2026-04-21 23:47:06 +03:00
4524a6abc8 feat: finalize matrix platform audit and docs 2026-04-21 15:35:03 +03:00
6422c7db58 feat: support shared-workspace file flow for matrix 2026-04-21 00:26:21 +03:00
323a6d3144 feat: commit staged matrix attachments on next message 2026-04-20 21:39:37 +03:00
f111ed3348 feat: add matrix staging list and remove flow 2026-04-20 21:37:12 +03:00
83c9a1513b feat: parse matrix staged attachment commands 2026-04-20 16:26:37 +03:00
0eaf124e21 feat: add matrix staged attachment state 2026-04-20 16:21:00 +03:00
105ecc68ed docs: add matrix staged attachments design 2026-04-20 16:05:28 +03:00
8b04fcaf77 docs: add matrix shared workspace file flow design 2026-04-20 15:04:20 +03:00
e6a42d9297 wip: pause session — 3 fixes committed, file ingestion next 2026-04-19 21:22:19 +03:00
73c472ecc4 feat(matrix): implement !reset via new platform_chat_id
Instead of calling a /reset endpoint on platform-agent, !reset now
generates a new thread_id (platform_chat_id) for the room. The old
WebSocket connection is closed and the next message creates a fresh
context automatically. No platform changes required.
2026-04-19 21:20:31 +03:00
4a5260ca79 docs: clarify Matrix onboarding via DM 2026-04-19 21:12:02 +03:00
b3331464d9 docs: update README with Matrix MVP runbook and feature status
Add step-by-step setup for running Matrix surface with real platform-agent,
document all available commands, and clearly list what works vs what is
blocked (StateBackend cross-chat load, hardcoded tokens, missing /reset,
no file upload API).
2026-04-19 21:06:03 +03:00
fbcf44980e fix(sdk): correct WebSocket URL pattern for platform-agent
AgentApiWrapper._build_ws_url was building /v1/agent_ws/{chat_id}/
which does not exist in platform-agent. Fixed to /agent_ws/?thread_id={chat_id}
to match the actual endpoint and query-param isolation scheme.

Also simplify Matrix MVP settings handlers to MVP_UNAVAILABLE stubs
and add handle_unknown_command for unregistered !commands.
2026-04-19 21:05:02 +03:00
07c5078934 feat(task-7): verify matrix per-room context routing 2026-04-19 17:43:18 +03:00
c11c8ecfbf feat(task-5): scope matrix context state per room 2026-04-19 17:41:04 +03:00
03160a3b37 fix: preserve invite workspace bootstrap semantics 2026-04-19 17:34:47 +03:00
8270e5821e Assign matrix platform chat ids on creation 2026-04-19 17:31:21 +03:00
0cdee532c4 fix: ensure lazy platform chat ids before load selection 2026-04-19 17:29:36 +03:00
9cb1657d21 Add lazy platform chat IDs for Matrix rooms 2026-04-19 17:25:25 +03:00
c666d908da fix: make matrix entry-room bootstrap idempotent 2026-04-19 17:23:07 +03:00
17d580096b Serialize Matrix chat sends 2026-04-19 17:18:32 +03:00
4533118b68 Fix agent API wrapper constructor compatibility 2026-04-19 17:11:49 +03:00
730ea70f78 Fix real client chat cache compatibility 2026-04-19 17:07:52 +03:00
414a8645bd Add per-chat real client routing 2026-04-19 17:03:48 +03:00
5782001d3d fix: preserve matrix room metadata when setting platform chat id 2026-04-19 16:52:43 +03:00
f3f9b10d6b feat: add platform chat id room metadata helpers 2026-04-19 16:50:12 +03:00
9bb93fbbda docs: add matrix per-chat context design 2026-04-19 16:37:41 +03:00
430c82dba1 feat(04-01): finalize AgentApi migration 2026-04-17 16:31:48 +03:00
cd59d89617 fix(04-02): revert out-of-scope real client edit
- drop sdk/real.py change to respect requested write scope
- update phase summary file list
2026-04-17 16:12:56 +03:00
632673eaae docs(04-02): complete matrix context commands plan
- add phase summary with verification and deviations
2026-04-17 16:12:27 +03:00
b52fdc4670 feat(04-02): add matrix context management commands
- add save/load/reset/context handlers and matrix interception flows
- persist current session and last token usage in prototype state
2026-04-17 16:12:03 +03:00
da0b76882e docs(04-03): add execution summary
- record containerization decisions and verification
- document scoped deviation for uv runtime install
2026-04-17 16:07:51 +03:00
4628304979 feat(04-03): add matrix bot containerization
- add Dockerfile for matrix bot runtime
- add compose service and env template entries
2026-04-17 16:07:47 +03:00
2720ee2d6e feat(04-02): extend prototype and matrix pending state
- add saved session and last token tracking in prototype state
- add matrix load/reset pending store helpers
2026-04-17 16:07:35 +03:00
6923b801a3 wip: phase 4 planning complete, ready to execute 2026-04-17 15:36:19 +03:00
0e132849cc docs(04): create phase 4 plans — AgentApi migration, context commands, Docker 2026-04-17 15:28:40 +03:00
3f39b7002a docs: create thread — matrix dev prototype agent platform state 2026-04-16 12:01:26 +03:00
c004d96785 docs: add exact run commands for matrix prototype 2026-04-08 02:57:45 +03:00
7507b2f252 wip: 02-prototype paused at task 4/4 2026-04-08 02:55:30 +03:00
9c73266ea5 docs: add matrix direct-agent prototype runbook 2026-04-08 02:51:25 +03:00
8efc91b02b fix(matrix): accept repeat invites before provisioning 2026-04-08 02:18:11 +03:00
37643a9695 fix prototype backend review issues 2026-04-08 01:43:44 +03:00
94bdb44b93 feat: wire matrix runtime to real backend 2026-04-08 01:40:38 +03:00
9784ca6783 feat: add real platform compatibility layer 2026-04-08 01:38:28 +03:00
fabedb105b Fix prototype state user isolation 2026-04-08 01:30:37 +03:00
19c85db89a Persist canonical prototype user state 2026-04-08 01:29:02 +03:00
083be77404 fix(agent): collision-safe thread keys 2026-04-08 01:25:52 +03:00
2fad1aaa66 feat: add prototype local state store 2026-04-08 01:25:46 +03:00
de20ff638a feat: add direct agent session transport 2026-04-08 01:00:02 +03:00
1fdb5bf303 docs: add matrix direct-agent prototype design 2026-04-08 00:22:20 +03:00
b08a5e3d96 wip: matrix-restart-reconciliation-and-dev-reset-workflow paused at task 1/2 2026-04-07 18:13:06 +03:00
122 changed files with 18162 additions and 3557 deletions

22
.dockerignore Normal file
View file

@ -0,0 +1,22 @@
.git
.gitignore
.DS_Store
__pycache__/
.pytest_cache/
.ruff_cache/
.venv/
.worktrees/
external/
.planning/
docs/superpowers/
tests/
# Local runtime state must not be baked into the image.
lambda_matrix.db
matrix_store/
lambda_bot.db
config/matrix-agents.yaml
# Local environment and editor state
.env
.idea/

View file

@ -1,14 +1,32 @@
# Telegram
TELEGRAM_BOT_TOKEN=your_bot_token_here
# Matrix
MATRIX_HOMESERVER=https://matrix.org
MATRIX_USER_ID=@bot:matrix.org
# Matrix bot credentials
MATRIX_HOMESERVER=https://matrix.example.org
MATRIX_USER_ID=@lambda-bot:example.org
# Use ONE of: MATRIX_PASSWORD or MATRIX_ACCESS_TOKEN
MATRIX_PASSWORD=your_password_here
# MATRIX_ACCESS_TOKEN=your_access_token_here
# Lambda Platform
LAMBDA_PLATFORM_URL=http://localhost:8000
LAMBDA_SERVICE_TOKEN=your_service_token_here
# Backend: "real" connects to platform-agent via AgentApi; "mock" uses local stub (testing only)
MATRIX_PLATFORM_BACKEND=real
# Режим работы: "mock" или "production"
PLATFORM_MODE=mock
# Published surface image used by docker-compose.prod.yml.
# Must point to a Docker Hub/registry namespace where you have push/pull access.
SURFACES_BOT_IMAGE=mput1/surfaces-bot:latest
# platform/agent_api ref used when building a surface image
LAMBDA_AGENT_API_REF=master
# Path to agent registry inside the container (mounted via ./config:/app/config:ro)
MATRIX_AGENT_REGISTRY_PATH=/app/config/matrix-agents.yaml
# HTTP URL of the platform-agent endpoint
# Production: external agent managed by the platform
# Fullstack E2E: overridden to http://platform-agent:8000 by docker-compose.fullstack.yml
AGENT_BASE_URL=http://your-agent-host:8000
# Shared volume path inside the bot container (default: /agents).
# For multi-agent production, each agent gets a subdirectory such as /agents/0.
SURFACES_WORKSPACE_DIR=/agents
# Docker volume names (created automatically on first run)
SURFACES_SHARED_VOLUME=surfaces-agents
SURFACES_BOT_STATE_VOLUME=surfaces-bot-state

1
.gitignore vendored
View file

@ -15,6 +15,7 @@ build/
# Git worktrees (не трекаем в репо)
.worktrees/
external/
# IDE
.idea/

View file

@ -1,87 +0,0 @@
{
"version": "1.0",
"timestamp": "2026-04-04T10:13:58.720Z",
"phase": "01.1",
"phase_name": "matrix-restart-reconciliation-and-dev-reset-workflow",
"phase_dir": ".planning/phases/01.1-matrix-restart-reconciliation-and-dev-reset-workflow",
"plan": 3,
"task": 1,
"total_tasks": 2,
"status": "paused",
"completed_tasks": [],
"remaining_tasks": [
{
"id": 1,
"name": "Add a dev-only Matrix reset CLI with explicit modes",
"status": "not_started"
},
{
"id": 2,
"name": "Replace the README reset ritual with the new restart and reset workflow",
"status": "not_started"
}
],
"blockers": [
{
"description": "Phase 02 SDK integration remains blocked because platform control-plane contract is not stable yet; current platform repos only clearly expose the direct agent WebSocket layer.",
"type": "external",
"workaround": "Keep the current consumer-facing bot flows and mock-backed facade for now; use Matrix as the internal testing surface and revisit integration once master user/chat/session access is clarified."
}
],
"human_actions_pending": [
{
"action": "Confirm with the platform team the minimal control-plane contract for user/chat/session access and whether settings/attachments will exist in master.",
"context": "Current evidence shows agent_api is usable, but master is not yet a stable consumer-facing API.",
"blocking": true
}
],
"decisions": [
{
"decision": "Do not start a full rewrite of the consumer-facing bot integration yet.",
"rationale": "Platform direction is visible, but too many pieces outside the direct agent WebSocket protocol are still undefined or inconsistent.",
"phase": "02"
},
{
"decision": "Treat sdk/mock.py as a temporary local integration facade rather than a near-drop-in replacement for the real platform.",
"rationale": "The current mock assumes a unified platform API, while the real platform is split between control plane and direct agent session.",
"phase": "02"
},
{
"decision": "Use Matrix as the internal testing surface while waiting for the platform contract to stabilize.",
"rationale": "This preserves product iteration without coupling the bot too early to a moving platform backend.",
"phase": "02"
}
],
"uncommitted_files": [
".planning/config.json",
"adapter/matrix/bot.py",
"adapter/matrix/handlers/__init__.py",
"adapter/matrix/handlers/auth.py",
"adapter/matrix/handlers/chat.py",
"adapter/matrix/handlers/settings.py",
"adapter/telegram/bot.py",
"sdk/mock.py",
"tests/adapter/matrix/test_chat_space.py",
"tests/adapter/matrix/test_dispatcher.py",
"tests/adapter/matrix/test_invite_space.py",
"tests/platform/test_mock.py",
".planning/phases/01-matrix-qa-polish/01-01-SUMMARY.md",
".planning/phases/01-matrix-qa-polish/01-04-SUMMARY.md",
".planning/phases/01-matrix-qa-polish/01-05-PLAN.md",
".planning/phases/01-matrix-qa-polish/01-06-PLAN.md",
".planning/phases/01-matrix-qa-polish/01-VERIFICATION.md",
".planning/phases/01.1-matrix-restart-reconciliation-and-dev-reset-workflow/.gitkeep",
"bot-examples/",
"docs/reports/2026-04-01-surfaces-progress-report.md",
"docs/superpowers/plans/2026-03-31-matrix-adapter.md",
"docs/workflow-backup-2026-04-01.md",
"forum_topics_research.md",
"image copy 2.png",
"image copy.png",
"image.png",
"lambda_bot.db",
"lambda_matrix.db"
],
"next_action": "When resuming, either execute Phase 01.1 Plan 03 Task 1 (Matrix reset CLI) or continue the platform-integration design by defining a split MasterClient/AgentSession boundary without changing consumer adapters yet.",
"context_notes": "This session was research-heavy rather than implementation-heavy. The key conclusion is that the real platform currently exposes a direct agent WebSocket SDK plus an unfinished master control plane; our mock models a richer unified platform than what exists today. That means future work should isolate the integration boundary, not rush a full rewrite."
}

View file

@ -2,56 +2,44 @@
## What This Is
Telegram и Matrix боты для взаимодействия пользователя с AI-агентом Lambda. Каждый бот — тонкий адаптер поверх общего ядра (`core/`), изолирующего бизнес-логику от транспорта. Платформа подключается через `sdk/interface.py` Protocol; сейчас используется `MockPlatformClient`.
Surfaces (поверхности) — это тонкие адаптеры-клиенты, соединяющие мессенджеры с агентами платформы Lambda.
Текущая и главная реализация — **Matrix MVP**. Бот работает как stateless-прослойка: преобразует события Matrix во внутренний протокол `core/` и маршрутизирует их на внешние контейнеры агентов (через `AgentApi` по WebSocket).
## Core Value
Пользователь может вести диалог с Lambda-агентом через любой из поддерживаемых мессенджеров без изменения ядра системы.
Пользователь может бесшовно взаимодействовать с изолированными AI-агентами через нативные интерфейсы мессенджеров (с поддержкой пересылки файлов и работы в комнатах), в то время как сама платформа агентов не зависит от транспорта.
## Requirements
### Validated
- ✓ core/ — унифицированный протокол событий, EventDispatcher, StateStore, ChatManager, AuthManager, SettingsManager — existing
- ✓ adapter/telegram/ — forum-first адаптер (Threaded Mode), `/start`, `/new`, `/archive`, `/rename`, `/settings`, стриминг ответов — existing, QA passed
- ✓ adapter/matrix/ — DM-first адаптер, invite flow, `!new`, `!skills`, `!soul`, `!safety`, room-per-chat — existing
- ✓ sdk/mock.py — MockPlatformClient: `stream_message`, `get_or_create_user`, `get_settings`, `update_settings` — existing
- ✓ `core/` — унифицированный протокол событий, EventDispatcher, StateStore, ChatManager.
- ✓ `adapter/matrix/` — Space+rooms адаптер. Прием инвайтов, автосоздание иерархии комнат, команды `!new`, `!archive`, `!clear`, `!yes`/`!no`.
- ✓ `sdk/real.py` — интеграция с AgentApi. Поддержка WebSocket для обмена сообщениями и передачи вложений в обе стороны.
- ✓ Shared Volume — прямая передача файлов в локальные рабочие папки агентов (`/agents/`).
- ✓ Dynamic Routing — маршрутизация чатов к агентам на основе `config/matrix-agents.yaml`.
- ✓ Deployment — Разделение окружений на `docker-compose.prod.yml` (только бот) и `docker-compose.fullstack.yml` (бот + локальный агент для E2E).
### Active
### Out of Scope / Deferred
- [ ] Matrix QA — ручное тестирование Matrix адаптера, фиксация багов
- [ ] SDK integration — заменить MockPlatformClient реальным Lambda SDK (когда платформа готова)
- [ ] Production hardening — конфиг для деплоя, логирование, мониторинг
### Out of Scope
- E2EE для Matrix (python-olm не собирается на macOS/ARM) — инфраструктурная задача, отдельный трек
- Supergroup forum mode для Telegram — заменён Threaded Mode как основным режимом
- Telegram DM-first режим — заменён forum-first (Threaded Mode)
- E2EE для Matrix (отложено из-за сложностей сборки `python-olm` на кросс-платформенных средах).
- Интеграция с Master-сервисом платформы (временно используется прямое соединение с `platform-agent` через AgentApi).
- Telegram-адаптер (вынесен в легаси ветку `feat/telegram-adapter`, MVP фокусируется на Matrix).
## Context
- Python 3.11+, aiogram 3.4+, matrix-nio 0.21+, SQLite, pytest-asyncio
- Threaded Mode — Bot API 9.3, Mac клиент имеет известные баги (новые топики не сразу видны в сайдбаре)
- Lambda platform SDK ещё не готов, всё работает через MockPlatformClient
- Архитектура: Hexagonal / Ports-and-Adapters; `core/` не зависит от транспорта
## Constraints
- **Tech stack**: aiogram 3.x для Telegram, matrix-nio для Matrix — не менять без обсуждения
- **Platform**: SDK подключается только через `sdk/interface.py` Protocol — core/ и adapters не трогаются при смене реализации
- **Telegram**: Threaded Mode — единственный поддерживаемый режим; `closeForumTopic`/`deleteForumTopic` не работают в personal chat forums
- **E2EE**: python-olm не собирается на текущей среде — Matrix работает только без шифрования
- Стек: Python 3.11+, `matrix-nio`, `uv`, `pydantic`.
- Бот хранит только локальную привязку (`room_id` <-> `platform_chat_id`) в SQLite. Вся долговременная память и история диалогов хранятся на стороне агента.
- Жизненный цикл контейнеров агентов управляется платформой, а не ботом.
## Key Decisions
| Decision | Rationale | Outcome |
|----------|-----------|---------|
| Forum-first (Threaded Mode) для Telegram | Bot API 9.3 позволяет личный чат как форум — чище, без суперпруппы | ✓ Good |
| (user_id, thread_id) как PK в chats | Изоляция контекстов по топику | ✓ Good |
| MockPlatformClient через sdk/interface.py | Не ждать SDK, разрабатывать независимо | ✓ Good |
| DM-first для Matrix (не Space-first) | Space lifecycle слишком сложен для первого этапа | ✓ Good |
| Отказ от E2EE в Matrix | python-olm не собирается на macOS/ARM | — Pending |
| Space+rooms для Matrix | Room-based UX и явные чаты (по одному на тред) удобнее, чем DM-каша | ✓ Good |
| Прямая интеграция AgentApi | Master API не был готов, прямое WebSocket соединение позволяет передавать стейт и файлы | ✓ Good |
| Shared Volume для файлов | Избавляет от необходимости гонять base64 по сети, быстрый прямой доступ к файлам | ✓ Good |
| Stateless бот | Бот легко перезапускать и масштабировать, память изолирована в агентах | ✓ Good |
## Evolution
@ -61,10 +49,5 @@ Telegram и Matrix боты для взаимодействия пользова
3. New requirements emerged? → Add to Active
4. Decisions to log? → Add to Key Decisions
**After each milestone:**
1. Full review of all sections
2. Core Value check — still the right priority?
3. Update Context with current state
---
*Last updated: 2026-04-02 after initialization*
*Last updated: 2026-05-03 after codebase consolidation*

View file

@ -1,66 +1,32 @@
# Roadmap — v1.0
## Milestone: v1.0 — Production-ready surfaces
### Phase 1: Matrix QA & Polish
**Goal:** Переработать Matrix адаптер с DM-first на Space+rooms, убрать реакции в пользу !yes/!no, довести до уровня "приемлемо работает" как Telegram.
**Depends on:** Telegram QA complete
**Plans:** 6 plans
Plans:
- [x] 01-01-PLAN.md — Space+rooms infrastructure (store helpers, handle_invite rewrite, room_router)
- [x] 01-02-PLAN.md — Chat command handlers (!new, !archive, !rename) Space-aware
- [x] 01-03-PLAN.md — Reaction removal + !yes/!no confirmation + settings dashboard
- [x] 01-04-PLAN.md — Test suite (fix 4 broken + 12 new MAT-01..MAT-12)
- [x] 01-05-PLAN.md — Gap closure for Matrix `!yes` / `!no` pending-confirm scope
- [x] 01-06-PLAN.md — Remaining Phase 01 gap closure work (completed 2026-04-03)
## Milestone: v1.0 — Production-ready Matrix MVP
### Phase 01: Matrix QA & Polish
**Goal:** Переработать Matrix адаптер с DM-first на Space+rooms, убрать реакции в пользу `!yes`/`!no`.
**Status:** Completed
**Deliverables:**
- Space+rooms architecture for Matrix adapter
- !yes/!no text-based confirmation (no reactions)
- Read-only !settings dashboard
- 96+ tests green
- !yes/!no text-based confirmation
- Test suite green
### Phase 04: Matrix MVP: Agent Integration
**Goal:** Подключить реального агента через `AgentApi`, добавить команды управления контекстом (`!clear`).
**Status:** Completed
**Deliverables:**
- `sdk/real.py` — реализация `PlatformClient` через реальный SDK (`AgentApi`).
- Поддержка WebSocket стриминга.
- Команды управления контекстом.
- Обертка в Docker.
### Phase 05: MVP Deployment
**Goal:** Подготовить Matrix-бот к реальному деплою на lambda.coredump.ru с маршрутизацией по агентам и передачей файлов.
**Status:** Completed
**Deliverables:**
- Загрузка `matrix-agents.yaml` для маппинга пользователей к агентам.
- Per-room `platform_chat_id` routing.
- File transfer через shared `/agents/` volume.
- Разделение `docker-compose.prod.yml` и `docker-compose.fullstack.yml`.
---
### Phase 01.1: Matrix restart reconciliation and dev reset workflow (INSERTED)
**Goal:** Сделать Matrix-адаптер пригодным для повторяемого локального рестарта и ручного QA: бот восстанавливает минимальный local state из существующих Space/rooms и даёт явный dev reset workflow вместо ручного ritual reset.
**Requirements**: none explicitly mapped
**Depends on:** Phase 1
**Plans:** 3 plans
Plans:
- [ ] 01.1-01-PLAN.md — Non-destructive Matrix reconciliation module and tests
- [ ] 01.1-02-PLAN.md — Wire startup/bootstrap recovery into the Matrix runtime
- [ ] 01.1-03-PLAN.md — Dev reset CLI and updated Matrix restart runbook
### Phase 2: SDK Integration
**Goal:** Заменить MockPlatformClient реальным Lambda SDK — бот начинает работать с настоящим AI-агентом.
**Depends on:** Phase 1, Lambda platform SDK готов
**Deliverables:**
- `sdk/real.py` — реализация PlatformClient через реальный SDK
- `bot.py` для обоих адаптеров переключается на реальный клиент через env var
- `stream_message` работает с реальным стримингом
- Интеграционные тесты с реальным SDK (или staging)
---
### Phase 3: Production Hardening
**Goal:** Подготовить боты к реальному деплою — конфиг, логирование, мониторинг, обработка ошибок.
**Depends on:** Phase 2
**Deliverables:**
- Docker / systemd конфиг для деплоя
- Структурированное логирование в production формате
- Health-check endpoint (если нужен)
- Rate limiting и защита от спама
- Graceful shutdown
*Note: Легаси-фазы, связанные с Telegram, прототипами и Mock-платформой, были удалены из Roadmap после закрепления архитектуры MVP в ветке main.*

View file

@ -2,69 +2,48 @@
gsd_state_version: 1.0
milestone: v1.0
milestone_name: — Production-ready surfaces
status: Phase 01 Complete
last_updated: "2026-04-03T09:35:39Z"
status: MVP Deployed
last_updated: "2026-05-03T23:00:00Z"
progress:
total_phases: 3
completed_phases: 1
total_plans: 6
completed_plans: 6
completed_phases: 3
total_plans: 13
completed_plans: 13
---
# State
## Project Reference
See: .planning/PROJECT.md (updated 2026-04-02)
See: `.planning/PROJECT.md` (updated 2026-05-03)
**Core value:** Пользователь ведёт диалог с Lambda через любой мессенджер без изменения ядра
**Current focus:** Phase 02 — SDK Integration (blocked on Lambda platform SDK readiness)
**Core value:** Пользователь бесшовно взаимодействует с изолированными агентами через нативные интерфейсы (Matrix), в то время как платформа агентов не зависит от транспорта.
**Current focus:** Итерационное развитие текущей архитектуры, добавление новых фич и поверхностей (по мере необходимости).
## Current Phase
**Phase 2** of 3: SDK Integration
Текущий MVP успешно завершен. Все базовые механизмы внедрены и работают:
- Маршрутизация к `AgentApi`
- Shared Volume файловый обмен (`/agents/`)
- Dynamic config через `matrix-agents.yaml`
- Изоляция контекстов через `platform_chat_id`
Phase 1 is complete. Phase 2 remains blocked until the Lambda platform SDK is available.
Проект находится в чистом состоянии для начала нового планирования. Неактуальные легаси фазы и прототипы (Telegram, MockPlatformClient) удалены из Roadmap и трекинга.
## Decisions
- Продолжаем с Threaded Mode несмотря на баги Mac клиента (2026-04-02)
- Invite flow Matrix переведён на idempotent-проверку через `user_meta.space_id`, а не через invite-room metadata (2026-04-02)
- Неизвестные Matrix rooms больше не auto-register в роутере; используется явный fallback `unregistered:{room_id}` с warning-логом (2026-04-02)
- [Phase 01]: Use ChatContext.surface_ref as the Matrix room identifier for !rename updates.
- [Phase 01]: Keep !archive limited to core archive state in Phase 1; Space child removal remains deferred.
- [Phase 01]: Matrix OutgoingUI no longer emits reactions; confirmation state is persisted and resumed via `!yes` / `!no`.
- [Phase 01]: `!settings` now renders a dashboard snapshot instead of advertising mutable subcommands.
- [Phase 01]: Split Matrix regression coverage into dedicated invite/chat/send_outgoing/confirm test modules.
- [Phase 01]: Kept 01-04 scoped to test coverage without widening into production-code changes.
- [Phase 01]: Matrix command callbacks now include room_id in payload for !yes and !no so confirm handlers can resolve runtime state without changing core protocol types.
- [Phase 01]: Pending confirmations are stored under the D-08 composite key of matrix user id plus room id, with a narrow legacy fallback only for callers that omit room context.
- [Phase 01]: Removed Matrix reaction conversion entirely and kept command callbacks limited to !yes/!no.
- [Phase 01]: Kept !settings as a pure snapshot surface while preserving mutable subcommands outside the dashboard.
- [Phase 01]: Seeded invite and dispatcher tests with explicit next_chat_index and room ids instead of treating C1 as Matrix transport identity.
- **Space+rooms для Matrix**: Разделение тредов на отдельные Matrix-комнаты, собранные в едином Space пользователя.
- **AgentApi**: Прямая интеграция с локальным агентом без Master-прослойки по WebSocket.
- **Shared Volume**: Файлы кладутся напрямую в рабочую папку агента, избавляя от необходимости гонять их по сети в Base64.
- **Статическая маршрутизация**: На данном этапе пользователи маппятся на агентов жестко через YAML.
## Blockers
- Lambda platform SDK не готов — Phase 2 заблокирована до готовности платформы
- Отсутствуют. Проект готов к деплою (см. `docker-compose.prod.yml`).
## Accumulated Context
### Roadmap Evolution
- Phase 01.1 inserted after Phase 01: Matrix restart reconciliation and dev reset workflow (URGENT)
## Performance Metrics
| Phase | Plan | Duration | Tasks | Files | Recorded |
| --- | --- | --- | --- | --- | --- |
| 01 | 01 | 1 min | 3 | 3 | 2026-04-02T19:50:50Z |
| 01 | 02 | 1 min | 2 | 2 | 2026-04-02 |
| 01 | 03 | 3 min | 2 | 5 | 2026-04-02T19:57:34Z |
| 01 | 04 | 3 min | 2 | 7 | 2026-04-02T20:03:38Z |
| 01 | 05 | 2 min | 2 | 7 | 2026-04-03T09:28:47Z |
| 01 | 06 | 4 min | 2 | 7 | 2026-04-03T09:35:39Z |
## Session
- Last session: 2026-04-03T09:35:39Z
- Stopped at: Completed 01-06-PLAN.md
- Изначальный Roadmap включал множество ответвлений (прототипы Telegram, локальный mock-клиент). После закрепления MVP в `main` Roadmap был очищен, чтобы отражать только актуальный путь продукта.
- Следующие фазы будут добавляться по мере возникновения новых задач (например, переход от YAML-конфига к БД для реестра агентов).

View file

@ -1,134 +1,14 @@
# Architecture
# Архитектура (ARCHITECTURE.md)
**Analysis Date:** 2026-04-01
## Паттерн "Thin Adapter" (Тонкая поверхность)
## Pattern Overview
Система разделена на три логических слоя:
1. **Транспортный слой (Adapter)**: Подключается к внешней платформе (Matrix). Занимается конвертацией нативных событий (`room.message`) во внутренние структуры (`IncomingMessage`).
2. **Ядро (Core)**: Предоставляет единый протокол (`core/protocol.py`), не зависящий от конкретной реализации (Matrix, Telegram и т.д.).
3. **Платформенный слой (SDK)**: `RealPlatformClient` инкапсулирует подключение по WebSocket к реальным агентам (AgentApi).
**Overall:** Hexagonal / Ports-and-Adapters
## Routing & Registry
Бот может обслуживать множество агентов (multi-tenant). Маршрутизация настраивается статически через `config/matrix-agents.yaml`. Каждый пользователь (`@user:server`) привязан к конкретному `agent_id`, у которого есть свой HTTP URL и свой изолированный `workspace_path` (например, `/agents/1/`).
**Key Characteristics:**
- A platform-neutral `core/` defines all business logic and unified event types
- Adapters (`adapter/telegram/`, `adapter/matrix/`) translate platform-specific events into core types and back
- The AI platform SDK is hidden behind a `PlatformClient` Protocol; the current implementation (`sdk/mock.py`) is swappable without touching core or adapters
- All state is stored through a `StateStore` Protocol, with `InMemoryStore` for tests and `SQLiteStore` for production
## Layers
**Protocol Layer:**
- Purpose: Defines every data structure crossing layer boundaries
- Location: `core/protocol.py`
- Contains: `IncomingMessage`, `IncomingCommand`, `IncomingCallback`, `OutgoingMessage`, `OutgoingUI`, `OutgoingNotification`, `OutgoingTyping`, `ChatContext`, `AuthFlow`, `SettingsAction`, type aliases `IncomingEvent` and `OutgoingEvent`
- Depends on: Python stdlib only
- Used by: All other layers
**Core / Business Logic Layer:**
- Purpose: Handles all domain logic independent of any platform
- Location: `core/`
- Contains:
- `core/handler.py``EventDispatcher`: routes `IncomingEvent` to registered handler functions; returns `list[OutgoingEvent]`
- `core/handlers/` — one module per event category (`start`, `message`, `chat`, `settings`, `callback`)
- `core/store.py``StateStore` Protocol + `InMemoryStore` + `SQLiteStore`
- `core/chat.py``ChatManager`: creates/renames/archives chat workspaces (C1/C2/C3); persists via `StateStore`
- `core/auth.py``AuthManager`: tracks auth flow state (`pending``confirmed`); persists via `StateStore`
- `core/settings.py``SettingsManager`: fetches/caches user settings from SDK; invalidates on write
- Depends on: `core/protocol.py`, `sdk/interface.py` (Protocol only), `core/store.py`
- Used by: Adapters
**SDK / Platform Layer:**
- Purpose: Wraps the external Lambda AI platform; isolated behind a Protocol
- Location: `sdk/`
- Contains:
- `sdk/interface.py``PlatformClient` Protocol: `get_or_create_user`, `send_message`, `stream_message`, `get_settings`, `update_settings`; also `WebhookReceiver` Protocol, Pydantic models (`User`, `MessageResponse`, `MessageChunk`, `UserSettings`, `AgentEvent`)
- `sdk/mock.py``MockPlatformClient`: full in-memory implementation with simulated latency; supports both sync (`send_message`) and streaming (`stream_message`, currently returns single chunk); includes webhook simulation via `simulate_agent_event()`
- Depends on: `sdk/interface.py`
- Used by: `core/` managers, adapters during bot startup
**Adapter Layer:**
- Purpose: Translates platform-native events into `IncomingEvent` and `OutgoingEvent` back to platform-native calls
- Location: `adapter/matrix/`, adapter/telegram/ (in `.worktrees/telegram/`)
- Contains per adapter: `bot.py` (entry point + send logic), `converter.py` (native event → protocol), `handlers/` (adapter-specific handler overrides registered on top of core handlers), optional `store.py` / `room_router.py` / `reactions.py` for adapter state
- Depends on: `core/`, `sdk/`, platform SDK (aiogram or matrix-nio)
- Used by: `__main__` / `asyncio.run(main())`
## Data Flow
**Incoming Message (Matrix example):**
1. `matrix-nio` fires `RoomMessageText` callback → `MatrixBot.on_room_message()` in `adapter/matrix/bot.py`
2. `resolve_chat_id()` in `adapter/matrix/room_router.py` maps `room_id` → logical `chat_id` (e.g. `C1`), persisted in `StateStore`
3. `from_room_event()` in `adapter/matrix/converter.py` converts the nio event to `IncomingMessage` or `IncomingCommand`
4. `EventDispatcher.dispatch(incoming)` in `core/handler.py` selects the handler by routing key (command name, callback action, or `"*"` for messages)
5. Handler (e.g. `core/handlers/message.py:handle_message`) calls `platform.send_message()` on `MockPlatformClient`, receives `MessageResponse`
6. Handler returns `list[OutgoingEvent]` (e.g. `[OutgoingTyping(..., False), OutgoingMessage(...)]`)
7. `MatrixBot._send_all()` iterates the list; `send_outgoing()` converts each to a `client.room_send()` / `client.room_typing()` call
**Incoming Reaction (Matrix):**
1. `ReactionEvent` callback → `MatrixBot.on_reaction()`
2. `from_reaction()` maps emoji key to `IncomingCallback` with `action="confirm"`, `"cancel"`, or `"toggle_skill"`
3. Dispatch → `core/handlers/callback.py`
**Command Routing:**
The `EventDispatcher` uses a routing key per event type:
- `IncomingCommand``event.command` (e.g. `"start"`, `"new"`, `"settings"`)
- `IncomingCallback``event.action` (e.g. `"confirm"`, `"toggle_skill"`)
- `IncomingMessage``"*"` (catch-all), or `event.attachments[0].type` if attachments present
Adapters call `register_all(dispatcher)` first (core handlers), then `register_matrix_handlers(dispatcher, ...)` to override or add platform-specific variants (e.g. `!new` creates a real Matrix room via the nio client).
**State Management:**
- All persistent state goes through `StateStore` (key-value, async interface)
- Key namespaces: `chat:{user_id}:{chat_id}`, `auth:{user_id}`, `settings:{user_id}`, `matrix_room:{room_id}`, `matrix_user:{matrix_user_id}`, `matrix_state:{room_id}`, `matrix_skills_msg:{room_id}`
- Production uses `SQLiteStore` (row-per-key, JSON-serialised values); tests use `InMemoryStore`
## Key Abstractions
**EventDispatcher (`core/handler.py`):**
- Purpose: Single dispatch table for all event types; decouples handler logic from transport
- Pattern: Registry (map of `event_type → {key → HandlerFn}`); wildcard `"*"` as fallback
- Handler signature: `async def handler(event, chat_mgr, auth_mgr, settings_mgr, platform) → list[OutgoingEvent]`
**StateStore Protocol (`core/store.py`):**
- Purpose: Pluggable persistence behind a minimal `get/set/delete/keys` interface
- Implementations: `InMemoryStore` (tests/dev), `SQLiteStore` (production)
- Key pattern: `"{namespace}:{discriminator}"`
**PlatformClient Protocol (`sdk/interface.py`):**
- Purpose: Contracts the entire surface of the Lambda AI SDK
- Current implementation: `MockPlatformClient` in `sdk/mock.py`
- Swap path: Replace `sdk/mock.py` with a real SDK client; no changes needed elsewhere
**Converter functions (`adapter/matrix/converter.py`):**
- Purpose: One-way transformation from platform-native event to `IncomingEvent`
- Always produce canonical protocol types; adapters never pass raw library objects to core
## Entry Points
**Matrix Bot:**
- Location: `adapter/matrix/bot.py:main()`
- Run: `python -m adapter.matrix.bot`
- Startup sequence: load `.env` → build `AsyncClient``build_runtime()` → register callbacks → `client.sync_forever()`
**Telegram Bot:**
- Location: `.worktrees/telegram/adapter/telegram/bot.py` (feature branch, not merged to main yet)
- Run: `python -m adapter.telegram.bot`
## Error Handling
**Strategy:** Errors propagate up to the adapter's event callback. The adapter logs and drops the event; the bot keeps running.
**Patterns:**
- `EventDispatcher.dispatch()` returns `[]` (empty list) when no handler is found and logs a warning
- `AuthManager` and `ChatManager` raise `ValueError` for not-found entities; callers are responsible for catching
- `MockPlatformClient` raises `PlatformError` (defined in `sdk/interface.py`) on unexpected states
## Cross-Cutting Concerns
**Logging:** `structlog` throughout; all managers and the dispatcher use `structlog.get_logger(__name__)`
**Validation:** Pydantic models in `sdk/interface.py` for SDK responses; plain dataclasses in `core/protocol.py` for internal events
**Authentication:** `AuthManager.is_authenticated()` is checked in `handle_message` before forwarding to platform; unauthenticated users receive a prompt to run `!start` / `/start`
---
*Architecture analysis: 2026-04-01*
## Файловый контракт
Файлы не передаются агенту в base64. Бот сохраняет вложение напрямую в локальную директорию (общий volume), и передает агенту только относительный путь (`workspace_path`).

View file

@ -1,235 +1,6 @@
# Codebase Concerns
# Известные проблемы (CONCERNS.md)
**Analysis Date:** 2026-04-01
---
## Tech Debt
### Telegram adapter not merged to main
- Issue: The entire `adapter/telegram/` directory exists only in the `feat/telegram-adapter` branch (worktree at `.worktrees/telegram/`). `main` has no Telegram adapter at all.
- Files: `.worktrees/telegram/adapter/telegram/` and remote branch `origin/feat/telegram-adapter`
- Impact: Running `python -m adapter.telegram.bot` from `main` fails with ImportError. Tests referencing `adapter/telegram/` (e.g., `tests/adapter/test_forum_db.py`) only exist in the worktree and are absent from `main`.
- Fix approach: Merge `feat/telegram-adapter` into `main` after final manual QA pass. The branch is ahead of main by 5 commits (`a1b7a14` being the most recent).
### Divergent core/handlers between main and feat/telegram-adapter
- Issue: `feat/telegram-adapter` removed platform-awareness from `core/handlers/chat.py` and `core/handlers/message.py` — the `_command()` and `_start_command()` helpers that format Matrix `!cmd` vs Telegram `/cmd` prompts were deleted. The branch hardcodes `/start` everywhere.
- Files: `core/handlers/chat.py`, `core/handlers/message.py` (differ between branches)
- Impact: If the Matrix adapter relies on these platform-aware helpers being in `main`'s version of core, merging `feat/telegram-adapter` will break Matrix `!start` prompt text for unauthenticated users.
- Fix approach: Before merging, decide which version of `core/handlers/` is canonical. The Matrix adapter in `main` currently passes because `main` still has the platform-aware helpers.
### SQLiteStore uses blocking I/O in async context
- Issue: `core/store.py` `SQLiteStore` methods are declared `async` but perform synchronous blocking `sqlite3.connect()` calls without `asyncio.to_thread` or `aiosqlite`.
- Files: `core/store.py` lines 4673
- Impact: Each database call blocks the asyncio event loop. Under any concurrent load (e.g., two Matrix users sending messages simultaneously) this will cause visible latency spikes and potential event loop starvation.
- Fix approach: Replace `sqlite3` calls with `aiosqlite` or wrap each call in `asyncio.to_thread()`.
### Telegram adapter has its own separate SQLite database layer
- Issue: `adapter/telegram/db.py` is a fully independent SQLite database (file: `lambda_bot.db`) with its own schema (`tg_users`, `chats`). Meanwhile, `core/store.py` has `SQLiteStore` with a KV schema (`lambda_matrix.db` for Matrix). The two stores are incompatible and do not share data.
- Files: `.worktrees/telegram/adapter/telegram/db.py`, `core/store.py`
- Impact: There is no unified storage layer. Chat state is split across two databases. A user's Telegram chats cannot be seen from Matrix and vice versa (even conceptually). Violates the "single core" architecture principle from CLAUDE.md.
- Fix approach: This is a fundamental design gap. Either extend `StateStore` to support the Telegram-specific data model, or accept separate stores as intentional for the prototype stage and document the constraint.
### MockPlatformClient hardcoded throughout — no production path wired
- Issue: Both `adapter/matrix/bot.py` and `.worktrees/telegram/adapter/telegram/bot.py` instantiate `MockPlatformClient()` directly. `PLATFORM_MODE` is defined in `.env.example` but is never read or acted upon anywhere in the codebase.
- Files: `adapter/matrix/bot.py` line 71, `sdk/mock.py`, `sdk/interface.py`
- Impact: There is no runtime switch to connect a real SDK. Switching to production requires code changes, not configuration.
- Fix approach: Add a factory function in `sdk/` that reads `PLATFORM_MODE` and returns either `MockPlatformClient` or a real `PlatformClient`. Both bot entrypoints should use this factory.
### MatrixRuntime type annotation leaks MockPlatformClient
- Issue: `adapter/matrix/bot.py` `MatrixRuntime.platform` is typed as `MockPlatformClient` (not `PlatformClient`). `build_event_dispatcher` and `build_runtime` signatures also use `MockPlatformClient` as the parameter type.
- Files: `adapter/matrix/bot.py` lines 46, 54, 67
- Impact: The isolation promise ("replace only `sdk/mock.py` when real SDK arrives") is broken — the bot layer is coupled to the mock concrete type, not the Protocol.
- Fix approach: Change type annotations to `PlatformClient` from `sdk.interface`.
---
## Known Bugs / Open Issues
### Telegram forum: global commands visible inside topic context
- Issue: Telegram shows the full bot command menu (including `/chats`, `/new`, `/settings`) even when the user is inside a forum topic. The code blocks `switch` and `new_chat` callbacks inside topics but the commands themselves still appear in the UI.
- Files: `.worktrees/telegram/adapter/telegram/handlers/forum.py`, `.worktrees/telegram/adapter/telegram/bot.py`
- Impact: Users can tap `/settings` or `/chats` inside a topic and get confusing behavior.
- Tracked: Issue `#15``Telegram forum topics: remaining UX and synchronization gaps`
### Telegram forum: `/new <name>` inside linked topic does not rename the Telegram topic
- Issue: Running `/new <name>` inside a forum topic that is already linked to a chat renames the internal chat record but does not call `edit_forum_topic` to rename the actual Telegram topic.
- Files: `.worktrees/telegram/adapter/telegram/handlers/forum.py`
- Impact: Topic name in Telegram goes out of sync with internal chat name.
- Tracked: Issue `#15`
### Matrix: `handle_invite` hardcodes `chat_id = "C1"` for all new rooms
- Issue: `adapter/matrix/handlers/auth.py` `handle_invite()` always assigns `chat_id = "C1"` regardless of how many rooms the user already has. If a user invites the bot into a second room before using `!new`, both rooms get `C1`.
- Files: `adapter/matrix/handlers/auth.py` line 26
- Impact: Two rooms mapped to the same `chat_id` causes routing collisions.
- Fix approach: Call `next_chat_id(store, user_id)` here instead of hardcoding `"C1"`.
### Matrix: `remove_reaction` uses non-standard `undo` field
- Issue: `adapter/matrix/reactions.py` `remove_reaction()` sends a `"undo": True` field in the reaction event body. This is not part of the Matrix spec for reaction redaction. The correct approach is to redact the original reaction event via `client.room_redact()`.
- Files: `adapter/matrix/reactions.py` lines 5668
- Impact: Reaction "undo" will silently fail on compliant homeservers.
### Matrix: E2EE not supported (blocked by `python-olm`)
- Issue: `matrix-nio` E2EE requires `python-olm`, which fails to build on macOS/ARM. No encrypted DM support.
- Files: `adapter/matrix/bot.py`
- Impact: The bot cannot operate in encrypted rooms. Users who have DM encryption enforced cannot use the Matrix bot.
- Status: Documented as a known infrastructure constraint in `docs/reports/2026-04-01-surfaces-progress-report.md`. Needs a separate infrastructure task.
---
## Security Considerations
### SQLite database files not in .gitignore
- Risk: `lambda_bot.db` and `lambda_matrix.db` are present in the working tree (shown in `git status`) but not listed in `.gitignore`. These files may contain user data including chat content and display names.
- Files: `lambda_bot.db`, `lambda_matrix.db`, `.gitignore`
- Current mitigation: Files are currently untracked (not yet staged) but nothing prevents them from being accidentally committed.
- Recommendation: Add `*.db` or specific filenames to `.gitignore` immediately.
### Auth flow is auto-confirmed in mock — no real validation exists
- Issue: `core/auth.py` `confirm()` automatically sets `state = "confirmed"` and generates a fake `platform_user_id`. There is no real verification step, no code exchange, no token validation.
- Files: `core/auth.py` lines 3948
- Impact: The auth layer is decorative for the prototype. Any user who sends `!start` or `/start` is immediately authenticated. If the real SDK auth requires a different flow (e.g., OAuth, code), the current `AuthManager` interface may not match.
- Current mitigation: Acceptable for mock stage. Must be re-evaluated before production use.
### Matrix room metadata stored without access control
- Issue: `adapter/matrix/store.py` stores room metadata keyed by `room_id`. Any call that can supply an arbitrary `room_id` can read or overwrite another user's room metadata.
- Files: `adapter/matrix/store.py`, `adapter/matrix/room_router.py`
- Impact: In the current single-process bot this is not exploitable. If the store is ever shared across processes or users, room metadata can be poisoned.
---
## Fragile Areas
### `core/chat.py` scan-by-suffix fallback is O(N) and collision-prone
- Issue: `ChatManager.get()` when called without `user_id` scans all `chat:*` keys and matches by suffix (e.g., `":C1"`). If two users both have a chat named `C1` (which is always the case), this returns the first one found, non-deterministically.
- Files: `core/chat.py` lines 7682
- Impact: Functions like `rename` and `archive` that call `chat_mgr.get(chat_id)` without `user_id` will operate on the wrong user's chat in a multi-user scenario.
- Fix approach: Audit all callers and always pass `user_id`. The scan-by-suffix fallback should be removed or explicitly guarded.
### `adapter/matrix/handlers/chat.py` chat_id counter races under concurrency
- Issue: `make_handle_new_chat` calls `chat_mgr.list_active()` and uses `len(chats) + 1` to compute a new `chat_id`. This is not atomic. Two concurrent `!new` commands from the same user can produce the same `chat_id`.
- Files: `adapter/matrix/handlers/chat.py` line 17
- Impact: Duplicate `chat_id` values (`C2`, `C2`) for the same user, leading to state corruption.
- Fix approach: Use `next_chat_id()` from `adapter/matrix/store.py` which increments an atomic counter in the store. The `next_chat_id()` function already exists but is not used here.
### `conftest.py` contains a fragile stdlib `platform` module workaround
- Issue: `conftest.py` patches `sys.modules` to remove the Python stdlib `platform` module so local `platform/` (which no longer exists — renamed to `sdk/`) doesn't shadow it. The comment still refers to `platform/` but the directory was renamed to `sdk/` in commit `41660fe`.
- Files: `conftest.py` lines 113
- Impact: The workaround is now a no-op (there is no `platform/` package to shadow) but adds confusion. The comment is incorrect. If someone creates a `platform/` directory again, unexpected behavior can return.
- Fix approach: Remove the `sys.modules` patching entirely since `sdk/` does not conflict with stdlib. Update the comment.
### Forum onboarding `chat_shared` constructs a fake `Chat` object
- Issue: `adapter/telegram/handlers/forum.py` handles `chat_shared` by constructing `Chat(id=..., type="supergroup", is_forum=True)` and passing it to `_complete_group_link()`. The `is_forum=True` is hardcoded — the real value from Telegram is not verified. This means the check `if getattr(forwarded_chat, "is_forum", None) is False` in the forwarding fallback path is bypassed entirely.
- Files: `.worktrees/telegram/adapter/telegram/handlers/forum.py` lines 162168
- Impact: A user could link a regular supergroup (without Topics enabled) via `chat_shared`, which would succeed in linking but fail when the bot tries to create forum topics.
---
## Gaps Between CLAUDE.md and Actual Code
### CLAUDE.md says `platform/` — code uses `sdk/`
- CLAUDE.md architecture diagram shows `platform/interface.py` and `platform/mock.py`
- Actual code uses `sdk/interface.py` and `sdk/mock.py` (renamed in commit `41660fe`)
- Files: `CLAUDE.md` (project instructions), `sdk/interface.py`, `sdk/mock.py`
- Also: Agent config files at `.claude/agents/core-developer.md` still reference `platform/` throughout
- Impact: New contributors reading CLAUDE.md will look for a `platform/` directory that does not exist.
### CLAUDE.md lists `core/handlers/` sub-handlers that partially do not exist
- CLAUDE.md lists handler modules but the actual `core/handlers/` only has: `start.py`, `message.py`, `chat.py`, `settings.py`, `callback.py`
- No `voice.py` handler exists; voice is handled as a fallback inside `core/handlers/message.py` (returns stub response)
- No `payment.py` handler exists; `PaymentRequired` dataclass is defined in `core/protocol.py` but never dispatched
- Files: `core/protocol.py` (PaymentRequired defined), `core/handlers/` (no payment or voice handlers)
### CLAUDE.md workflow describes `@reviewer` agent but agent file references old patterns
- `.claude/agents/core-developer.md` still says "Твоя зона — `core/` и `platform/`"
- The old Haiku/Sonnet researcher-developer workflow is captured in `docs/workflow-backup-2026-04-01.md`, but `.claude/agents/` configs were not updated to match
### `tests/adapter/test_forum_db.py` is untracked on main
- This test file exists in the working tree (visible in `git status`) but is not committed to `main`. It tests `adapter/telegram/db.py` which also does not exist on `main`.
- Files: `tests/adapter/test_forum_db.py`
- Impact: Running `pytest tests/` from main currently includes this test, which imports `adapter.telegram.db`. This import succeeds only because the test auto-reloads the module from an untracked file. This is fragile — if the file is deleted, tests silently pass with fewer tests counted.
---
## Missing Critical Features
### No streaming response support in adapters
- Both adapters use `platform.send_message()` (sync) not `platform.stream_message()` (streaming)
- `sdk/interface.py` defines `stream_message` returning `AsyncIterator[MessageChunk]`
- No adapter sends a typing indicator before the response arrives and then streams chunks
- Impact: User experience with slow AI responses will show nothing until the full response is ready
- Files: `core/handlers/message.py` line 28, `sdk/interface.py` lines 8388
### No webhook/push notification handling
- `sdk/interface.py` defines `WebhookReceiver` Protocol with `on_agent_event()`
- `sdk/mock.py` has `register_webhook_receiver()` and `simulate_agent_event()`
- Neither bot entrypoint registers a `WebhookReceiver`
- Impact: Push notifications from the platform (task completions, background jobs) cannot reach the user
- Files: `sdk/interface.py` lines 9597, `adapter/matrix/bot.py`, no registration present
### Telegram adapter uses InMemoryStore for core state
- `.worktrees/telegram/adapter/telegram/bot.py` calls `InMemoryStore()` for the `EventDispatcher`'s state
- All `core/` state (auth, chat metadata in the KV layer) is lost on bot restart
- `adapter/telegram/db.py` SQLite is used only for Telegram-specific data
- Impact: On restart, authenticated users are logged out; core chat context is wiped
- Files: `.worktrees/telegram/adapter/telegram/bot.py` line 46
### No multi-user isolation in Matrix store
- `adapter/matrix/store.py` keys are global (`matrix_room:ROOMID`, `matrix_user:USERID`)
- There is no namespace or tenant isolation
- Impact: At scale, any key collision would corrupt state. For a single-user prototype this is acceptable, but it is an architectural constraint to document before expanding scope.
---
## Test Coverage Gaps
### No tests for `adapter/telegram/` in main test suite
- `tests/adapter/` on main only contains `matrix/` tests and the untracked `test_forum_db.py`
- All Telegram adapter tests live in the worktree at `.worktrees/telegram/tests/`
- Files: `tests/adapter/` (missing `telegram/` subdirectory on main)
- Risk: Merging `feat/telegram-adapter` without also merging its tests leaves Telegram untested on main
- Priority: High
### No tests for `core/handlers/callback.py` confirm/cancel real behavior
- `core/handlers/callback.py` `handle_confirm` and `handle_cancel` return stub text with `action_id`
- No test verifies that a real confirmation flow (dispatch → confirm → side effect) works end to end
- Files: `core/handlers/callback.py`, `tests/core/test_dispatcher.py`
- Priority: Medium
### No tests for `adapter/matrix/handlers/auth.py` multi-room invite scenario
- The hardcoded `C1` bug (see Known Bugs section) is not caught by any test
- Files: `adapter/matrix/handlers/auth.py`, `tests/adapter/matrix/test_dispatcher.py`
- Priority: Medium
---
*Concerns audit: 2026-04-01*
- **Отсутствие E2E шифрования в Matrix**: На данный момент бот не поддерживает зашифрованные комнаты, так как библиотека `matrix-nio` требует нативной сборки `python-olm`, что усложняет кросс-платформенный деплой.
- **Потеря стейта агентов**: Так как текущий `platform-agent` часто работает с `MemorySaver`, его стейт теряется при перезапусках. Это проблема внешнего агента, но она напрямую влияет на UX поверхности.
- **Общий том (Shared Volume)**: Контракт обязывает бота и агента запускаться на одном физическом хосте (или иметь распределенный сетевой диск), что может стать бутылочным горлышком при сильном масштабировании.
- **Hardcoded роутинг**: `matrix-agents.yaml` требует ручного редактирования и перезапуска бота при добавлении новых агентов. Желательно вынести этот процесс в динамическую БД или API.

View file

@ -1,195 +1,7 @@
# Coding Conventions
# Конвенции (CONVENTIONS.md)
**Analysis Date:** 2026-04-01
## Linting and Formatting
**Tool:** ruff (configured in `pyproject.toml`)
**Settings:**
- Line length: 100 characters
- Target: Python 3.11
- Active rule sets: `E` (pycodestyle errors), `F` (pyflakes), `I` (isort), `UP` (pyupgrade), `B` (bugbear)
**Type checking:** mypy (available as dev dependency; not enforced in CI at this time)
Run linting:
```bash
ruff check .
ruff format .
```
## File Naming
- Module files: `snake_case.py` (e.g., `room_router.py`, `test_dispatcher.py`)
- Each module starts with a comment declaring its path: `# core/handler.py`
- Test files: `test_<module>.py` (e.g., `test_store.py`, `test_converter.py`)
- No index/barrel files except `__init__.py` for package registration
## Class Naming
- `PascalCase` for all classes (e.g., `EventDispatcher`, `MockPlatformClient`, `MatrixBot`)
- Protocol/interface classes named after the capability: `StateStore`, `PlatformClient`, `WebhookReceiver`
- Manager classes suffixed with `Manager`: `ChatManager`, `AuthManager`, `SettingsManager`
- Dataclasses follow the same `PascalCase` rule: `IncomingMessage`, `OutgoingUI`, `MatrixRuntime`
## Function and Method Naming
- `snake_case` for all functions and methods
- Private helpers prefixed with single underscore: `_to_dict`, `_from_dict`, `_routing_key`, `_latency`
- Handler functions named `handle_<action>`: `handle_start`, `handle_message`, `handle_new_chat`
- Builder functions named `build_<thing>`: `build_runtime`, `build_event_dispatcher`, `build_skills_text`
- Converter functions named `from_<source>`: `from_room_event`, `from_command`, `from_reaction`
- Predicate functions named `is_<state>`: `is_authenticated`, `is_new`
## Variable Naming
- `snake_case` for all variables and parameters
- Internal state attributes prefixed with `_`: `self._store`, `self._platform`, `self._handlers`
- Store key prefixes are module-level constants in `UPPER_SNAKE_CASE`:
```python
ROOM_META_PREFIX = "matrix_room:"
USER_META_PREFIX = "matrix_user:"
```
- Constants for reaction strings are module-level: `CONFIRM_REACTION = "👍"`, `PLATFORM = "matrix"`
## Type Annotations
All files use `from __future__ import annotations` at the top for deferred evaluation.
**Annotation style:**
- Use built-in generics (`list[str]`, `dict[str, Any]`) — not `List`, `Dict` from `typing`
- Union types written with `|`: `str | None`, `IncomingCallback | None`
- Type aliases at module level: `IncomingEvent = IncomingMessage | IncomingCommand | IncomingCallback`
- Callable types use `typing.Callable` and `typing.Awaitable`:
```python
HandlerFn = Callable[..., Awaitable[list[OutgoingEvent]]]
```
- Handler functions use loose `list` return type without generics (consistent across `core/handlers/`)
- Protocol classes use `...` as body for abstract methods:
```python
async def get(self, key: str) -> dict | None: ...
```
**Pydantic vs dataclasses:**
- `core/protocol.py` — plain `@dataclass` with `field(default_factory=...)` for mutable defaults
- `sdk/interface.py` — Pydantic `BaseModel` for all SDK-facing models (`User`, `MessageResponse`, `UserSettings`)
- Choose `@dataclass` for internal protocol structs, `BaseModel` for SDK boundary models
## Import Organization
Order (enforced by ruff `I` rules):
1. `from __future__ import annotations`
2. Standard library imports (grouped)
3. Third-party imports (grouped)
4. Local imports from project packages (grouped)
Example from `adapter/matrix/bot.py`:
```python
from __future__ import annotations
import asyncio
import os
from dataclasses import dataclass
from pathlib import Path
import structlog
from nio import AsyncClient, ...
from dotenv import load_dotenv
from adapter.matrix.converter import from_reaction, from_room_event
from core.auth import AuthManager
from core.protocol import OutgoingEvent, ...
from sdk.mock import MockPlatformClient
```
No relative imports; all imports use absolute package paths from the project root.
## Async Patterns
All I/O methods are `async def`. There are no sync wrappers around async code.
**Handler signature pattern** (used uniformly across `core/handlers/`):
```python
async def handle_<action>(event: IncomingEvent, auth_mgr, platform, chat_mgr, settings_mgr) -> list:
```
Note: manager parameters are untyped in handler signatures (accepted as `**kwargs` at call site in `EventDispatcher.dispatch`).
**Awaiting store calls:**
```python
stored = await self._store.get(f"auth:{user_id}")
await self._store.set(f"auth:{user_id}", _to_dict(flow))
```
**SQLiteStore uses sync sqlite3** inside `async def` methods — blocking I/O is not off-loaded to a thread executor. This is a known limitation (see CONCERNS.md).
**Mock latency simulation:**
```python
await self._latency(200, 600) # min_ms, max_ms
```
## Logging
**Library:** `structlog`
**Pattern:**
```python
import structlog
logger = structlog.get_logger(__name__)
logger.info("Chat created", chat_id=chat_id, user_id=user_id)
logger.warning("No handler registered", event_type=event_type.__name__, key=key)
```
- Always pass structured keyword arguments — never use f-strings in log calls
- Logger created at module level with `structlog.get_logger(__name__)`
## Error Handling
- Raise `ValueError` for invalid domain state (e.g., chat not found in `ChatManager.rename`)
- `sdk/interface.py` defines `PlatformError(Exception)` with a `code` field for SDK-level errors
- Handler functions never raise — they return `[]` or a fallback `OutgoingMessage`
- No `try/except` blocks in core handlers; errors from the platform are expected to propagate
## Comments
- Module-level comment declaring file path at top: `# core/handler.py`
- Docstrings for classes with non-obvious behavior:
```python
class MockPlatformClient:
"""
Заглушка SDK платформы Lambda.
...
"""
```
- Inline comments for non-obvious blocks:
```python
# Scan by chat_id suffix when user_id unknown (slower)
```
- Comments in Russian are normal and acceptable throughout the codebase
## Serialization Pattern
Dataclasses are serialized/deserialized via private module-level functions, not class methods:
```python
def _to_dict(ctx: ChatContext) -> dict:
return { "chat_id": ctx.chat_id, ... }
def _from_dict(d: dict) -> ChatContext:
return ChatContext(chat_id=d["chat_id"], ...)
```
This pattern is used in `core/auth.py` and `core/chat.py`. Follow this pattern for any new manager that persists to `StateStore`.
## Module Design
- No barrel `__init__.py` exports except `core/handlers/__init__.py` which exposes `register_all`
- Manager classes take `(platform, store)` as constructor args; `platform` is often stored as `object` or not stored at all if unused
- `@dataclass` is preferred for plain data containers, not NamedTuple or TypedDict
- Store key namespacing follows `<namespace>:<user_id>:<entity_id>` pattern:
`"chat:u1:C1"`, `"auth:u1"`, `"matrix_room:!r:m.org"`
---
*Convention analysis: 2026-04-01*
- **Асинхронность**: Весь код бота асинхронный (`asyncio`). Вызовы SDK и Matrix-клиента выполняются через `await`. Блокирующие вызовы (если они есть) должны выноситься в тредпул.
- **Обработка ошибок**: Бот не должен падать из-за ошибок отдельного агента. Ошибки SDK (например, `PlatformError`) отлавливаются в боте и возвращаются пользователю в виде системных сообщений или уведомлений.
- **Стейтлесс-подход**: Поверхность хранит минимальный стейт (только локальный SQLite для связки `room_id` <-> `platform_chat_id`). Вся история сообщений и память лежат на стороне агентов.
- **Переменные окружения**: Бот полностью конфигурируется через `.env` (префиксы `MATRIX_` и `SURFACES_`).
- **Добавление новой поверхности**: Новая поверхность должна быть самостоятельной папкой в `adapter/`, реализовывать `converter.py`, и переиспользовать `sdk/real.py` и `core/protocol.py`.

View file

@ -1,173 +1,15 @@
# External Integrations
# Интеграции (INTEGRATIONS.md)
**Analysis Date:** 2026-04-01
## Platform Agent API
- **Тип**: WebSocket (через `AgentApi` SDK)
- **Назначение**: Связь между Matrix-адаптером и внешней LLM-платформой.
- **Контракт**: Surface выступает "тупым" клиентом. Он отправляет `platform_chat_id` и `user_id` вместе с сообщениями. Платформа/Агент отвечает текстом и вложениями. Контейнерами агентов бот не управляет.
## Bot Platform APIs
## Matrix Homeserver
- **Тип**: HTTP/HTTPS API (via `matrix-nio`)
- **Назначение**: Пользовательский интерфейс и транспорт сообщений для бота.
- **Ограничения**: Поддерживается только нешифрованное (unencrypted) взаимодействие.
**Telegram Bot API:**
- Purpose: Primary messaging surface for user ↔ Lambda agent interaction
- Client library: `aiogram` 3.26.0 (async, wraps Telegram Bot API v7+)
- Authentication: Bot token via `TELEGRAM_BOT_TOKEN`
- Entry point: `adapter/telegram/bot.py` (planned; aiogram worktree branch `feat/telegram-adapter`)
- Transport: Long-polling or webhook (aiogram supports both; mode not yet locked in)
- Bot API docs: https://core.telegram.org/bots/api
**Matrix Client-Server API:**
- Purpose: Secondary messaging surface (Matrix/Element clients)
- Client library: `matrix-nio` 0.25.2 (async)
- Authentication: password login or pre-existing access token (`MATRIX_ACCESS_TOKEN`)
- Login flow in `adapter/matrix/bot.py` `main()`:
- If `MATRIX_ACCESS_TOKEN` is set → assigned directly to `client.access_token`
- Else if `MATRIX_PASSWORD` is set → `client.login(password=..., device_name="surfaces-bot")`
- Sync method: `client.sync_forever(timeout=30000)` (30-second long-poll)
- E2EE store: nio file-based store at path from `MATRIX_STORE_PATH` (default: `"matrix_store"`)
- Matrix C-S API docs: https://spec.matrix.org/latest/client-server-api/
### Matrix Room Model
Rooms are mapped to Lambda chat slots (C1, C2, C3…) via `adapter/matrix/room_router.py`:
- First message in a room → assigns next chat ID (C1, C2, …) and persists mapping to store
- Room metadata stored under key `matrix_room:<room_id>` in `StateStore`
- User metadata (next chat index) stored under `matrix_user:<matrix_user_id>`
### Matrix Event Types Handled
| nio Event Class | Handler | Action |
|--------------------|-----------------------------|-------------------------------|
| `RoomMessageText` | `MatrixBot.on_room_message` | Dispatch to `EventDispatcher` |
| `ReactionEvent` | `MatrixBot.on_reaction` | Button confirmation / skill toggle |
| `InviteMemberEvent`| `MatrixBot.on_member` | Accept room invite |
| `RoomMemberEvent` | `MatrixBot.on_member` | Membership change handling |
## Lambda Platform (Internal SDK)
**Purpose:** AI agent backend — processes user messages, manages user accounts, returns responses
**Interface:** `sdk/interface.py``PlatformClient` Protocol
**Current Implementation:** `sdk/mock.py``MockPlatformClient`
- Simulates network latency (1080 ms default, 200600 ms for message calls)
- In-process in-memory state (users, messages, settings dicts)
- Supports webhook simulation via `simulate_agent_event()`
**Production Integration (future):**
- URL: `LAMBDA_PLATFORM_URL` (default: `http://localhost:8000`)
- Auth: `LAMBDA_SERVICE_TOKEN` (bearer token)
- Mode switch: `PLATFORM_MODE=mock` vs `PLATFORM_MODE=production`
- Swap path: replace `sdk/mock.py` only; no changes to `core/` or `adapter/`
**Platform API Methods (from `sdk/interface.py`):**
```python
async def get_or_create_user(external_id, platform, display_name) -> User
async def send_message(user_id, chat_id, text, attachments) -> MessageResponse
async def stream_message(user_id, chat_id, text, attachments) -> AsyncIterator[MessageChunk]
async def get_settings(user_id) -> UserSettings
async def update_settings(user_id, action) -> None
```
**Webhook / Push (outbound from platform → bot):**
- Interface: `WebhookReceiver` Protocol (`sdk/interface.py`)
- Registration: `MockPlatformClient.register_webhook_receiver(receiver)`
- Event types: `task_done`, `task_error`, `task_progress` (modelled in `AgentEvent`)
- Production implementation not yet wired; mock supports `simulate_agent_event()` for testing
## Data Storage
**Databases:**
*SQLite (primary persistence):*
- Client: stdlib `sqlite3` (synchronous, called from async code without `asyncio.to_thread`)
- Schema: single key-value table: `kv (key TEXT PRIMARY KEY, value TEXT NOT NULL)`
- JSON serialization for values (`json.dumps` / `json.loads`)
- Matrix bot DB path: `MATRIX_DB_PATH` (default: `"lambda_matrix.db"`)
- Telegram bot DB path: implicit `"lambda_bot.db"` (file present in repo root — development artifact)
- Implementation: `core/store.py` `SQLiteStore`
*In-Memory (testing / development):*
- `InMemoryStore` — plain Python dict, no persistence across restarts
- `MockPlatformClient` internal state — also in-memory dicts
**File Storage:**
- Matrix nio E2EE store: local filesystem directory at `MATRIX_STORE_PATH` (default: `"matrix_store/"`)
- No object storage (S3/GCS/etc.) currently; mock client has `attachment_mode` flag (`"url"` | `"binary"` | `"s3"`) reserved for future real SDK
**Caching:**
- None — no Redis or external cache layer
## Authentication & Identity
**Telegram Auth:**
- Bot token → passed to aiogram dispatcher at startup
- User identity: Telegram user ID mapped to platform `external_id`
**Matrix Auth:**
- Password or access token (see above)
- User identity: Matrix user ID (e.g. `@user:matrix.org`) mapped to platform `external_id`
**Lambda Platform User Identity:**
- `get_or_create_user(external_id, platform)` → returns `User` with internal `user_id`
- External IDs are platform-prefixed in mock: `"{platform}:{external_id}"`
## Monitoring & Observability
**Logging:**
- `structlog` 25.5.0 — structured logging (key=value pairs)
- Logger instantiation: `structlog.get_logger(__name__)` in each module
- Log calls use keyword arguments: `logger.info("event_name", key=value, ...)`
- No log shipping / aggregation configured (local stdout only)
**Error Tracking:**
- None — no Sentry, Datadog, or similar integration
**Metrics:**
- None — `MockPlatformClient.get_stats()` returns basic in-memory counters (not exported)
## CI/CD & Deployment
**Hosting:**
- Not specified — no Dockerfile, docker-compose, or cloud config files present
**CI Pipeline:**
- None detected — no `.github/workflows/`, `.gitlab-ci.yml`, etc.
## Environment Configuration
**Required variables (from `.env.example`):**
| Variable | Required | Default | Purpose |
|-----------------------|----------|--------------------|--------------------------------------|
| `TELEGRAM_BOT_TOKEN` | Yes* | — | Telegram Bot API token |
| `MATRIX_HOMESERVER` | Yes* | — | Matrix homeserver URL (e.g. `https://matrix.org`) |
| `MATRIX_USER_ID` | Yes* | — | Bot's Matrix user ID |
| `MATRIX_PASSWORD` | Cond. | — | Login password (if no access token) |
| `MATRIX_ACCESS_TOKEN` | Cond. | — | Pre-issued access token (preferred) |
| `MATRIX_DEVICE_ID` | No | `""` | Matrix device ID |
| `MATRIX_DB_PATH` | No | `"lambda_matrix.db"` | SQLite DB file path (Matrix bot) |
| `MATRIX_STORE_PATH` | No | `"matrix_store"` | nio E2EE store directory |
| `LAMBDA_PLATFORM_URL` | No** | `http://localhost:8000` | Lambda platform base URL |
| `LAMBDA_SERVICE_TOKEN`| No** | — | Service auth token for Lambda API |
| `PLATFORM_MODE` | No | `"mock"` | `"mock"` or `"production"` |
\* Required for the respective bot to function.
\*\* Only required when `PLATFORM_MODE=production`.
**Secrets location:**
- `.env` file (gitignored)
- Never committed — `.env.example` provides template
- Loaded via `python-dotenv` at module import in each `bot.py` entry point
## Webhooks & Callbacks
**Incoming (platform → bot):**
- `WebhookReceiver.on_agent_event(event: AgentEvent)` — receives async task completion notifications
- Not yet wired to an HTTP endpoint; `MockPlatformClient.simulate_agent_event()` used for testing
**Outgoing (bot → external):**
- Telegram: all via `aiogram` polling or webhook (no direct outbound HTTP beyond Telegram API)
- Matrix: all via `matrix-nio` `AsyncClient.room_send()`, `room_typing()`, etc.
- Platform: via `PlatformClient` send/stream methods
---
*Integration audit: 2026-04-01*
## Файловая система (Shared Volume)
- **Тип**: Docker Shared Volume (`/agents/`)
- **Назначение**: Прямая передача файлов между поверхностью и агентами в обход сети. Поверхность пишет файлы в поддиректорию конкретного агента, агент их читает, и наоборот.

View file

@ -1,113 +1,14 @@
# Technology Stack
# Технологический стек (STACK.md)
**Analysis Date:** 2026-04-01
## Язык и Runtime
- **Python**: 3.11-slim (используется в Docker-образах)
- **Пакетный менеджер**: `uv` (используется для быстрой и строгой установки зависимостей, frozen lockfiles).
## Languages
## Ключевые библиотеки
- **matrix-nio**: Асинхронный клиент для Matrix (события, синхронизация, отправка).
- **pydantic**: Для валидации структур данных (события из AgentApi).
- **structlog**: Структурированное логирование (json/console).
**Primary:**
- Python 3.11+ — all application code (enforced via `pyproject.toml` `requires-python = ">=3.11"`)
**Type Annotations:**
- Full `from __future__ import annotations` usage throughout
- `typing.Protocol` used for dependency inversion (`core/store.py`, `sdk/interface.py`)
## Runtime
**Environment:**
- CPython — runtime (development host currently runs 3.14.3)
- Minimum: Python 3.11 (uses `match`-compatible union syntax, `Self`, `X | Y` type hints)
**Package Manager:**
- `uv` 0.9.30 (Homebrew)
- Lockfile: `uv.lock` present and committed
- Install: `uv sync`
## Frameworks
**Telegram Bot:**
- `aiogram` 3.26.0 — async Telegram Bot API framework
- Used in `adapter/telegram/` (planned; directory not yet present in main branch)
- Brings in `aiohttp` 3.13.3 as its HTTP transport
**Matrix Bot:**
- `matrix-nio` 0.25.2 — async Matrix Client-Server API client
- Used in `adapter/matrix/bot.py`
- Key classes: `AsyncClient`, `AsyncClientConfig`, `RoomMessageText`, `ReactionEvent`, `InviteMemberEvent`, `RoomMemberEvent`, `MatrixRoom`
- Long-polling via `client.sync_forever(timeout=30000)`
**Data Validation:**
- `pydantic` 2.12.5 — data models in `sdk/interface.py`
- `User`, `Attachment`, `MessageResponse`, `MessageChunk`, `UserSettings`, `AgentEvent`, `PlatformError`
- Core protocol structs (`core/protocol.py`) use plain `dataclasses` instead
**Build/Dev:**
- `setuptools` ≥68 + `setuptools-scm` + `wheel` — build backend (`pyproject.toml`)
- `ruff` 0.15.8 — linting and import sorting (`line-length = 100`, `target-version = "py311"`, rules: E, F, I, UP, B)
- `mypy` 1.19.1 — static type checking
## Key Dependencies
**Critical:**
- `aiogram>=3.4,<4` (resolved: 3.26.0) — Telegram adapter; pin avoids breaking v4 API
- `matrix-nio>=0.21` (resolved: 0.25.2) — Matrix adapter; async-only client
- `pydantic>=2.5` (resolved: 2.12.5) — SDK interface models; v2 required (v1 incompatible)
**Infrastructure:**
- `structlog` 25.5.0 — structured logging throughout; used via `structlog.get_logger(__name__)`
- `python-dotenv` 1.2.2 — loads `.env` at bot startup (`load_dotenv(Path(...) / ".env")`)
- `httpx` 0.28.1 — available for HTTP calls (future SDK integration, not yet used in core logic)
**Async I/O:**
- `aiohttp` 3.13.3 — transitive via aiogram; provides HTTP session to Telegram Bot API
- `asyncio` — stdlib; used directly in `sdk/mock.py` (`asyncio.sleep` for latency simulation) and all bot entry points (`asyncio.run(main())`)
## Testing
**Runner:**
- `pytest` 9.0.2
- `pytest-asyncio` 1.3.0 — `asyncio_mode = "auto"` (set in `pyproject.toml`)
- `pytest-cov` 7.1.0 — coverage reporting
**Configuration:**
- `pyproject.toml` `[tool.pytest.ini_options]`: `testpaths = ["tests"]`, `pythonpath = ["."]`
- `conftest.py` at project root
## Internal Module Structure
**Core (no external deps except stdlib + pydantic via sdk):**
- `core/protocol.py``dataclasses`-based unified event types
- `core/store.py``StateStore` Protocol + `InMemoryStore` (dict) + `SQLiteStore` (stdlib `sqlite3`)
- `core/handler.py``EventDispatcher`
- `core/auth.py`, `core/chat.py`, `core/settings.py` — domain managers
**SDK Layer:**
- `sdk/interface.py``PlatformClient` Protocol (pydantic models)
- `sdk/mock.py``MockPlatformClient` in-process stub; simulates latency via `asyncio.sleep`
**Adapters:**
- `adapter/matrix/` — matrix-nio integration (active)
- `adapter/telegram/` — aiogram integration (referenced in deps, worktree branch exists)
## Configuration
**Environment:**
- Loaded from `.env` via `python-dotenv` at startup
- See `INTEGRATIONS.md` for full variable list
**Build:**
- `pyproject.toml` — single source of truth for deps, build, lint, test config
## Platform Requirements
**Development:**
- Python ≥3.11
- `uv` for dependency management
**Production:**
- Any environment with Python ≥3.11
- Matrix bot: requires writable filesystem path for `matrix_store/` (nio E2EE store) and SQLite DB
- Telegram bot: stateless beyond env vars (or optionally SQLite for persistence)
---
*Stack analysis: 2026-04-01*
## Инфраструктура
- **Docker / Docker Compose**: Используется для локального (fullstack) и продакшн развертывания.
- **SQLite**: Легковесная локальная база данных для хранения маппингов пользователей/комнат (`adapter/matrix/store.py`).

View file

@ -1,210 +1,18 @@
# Codebase Structure
# Структура (STRUCTURE.md)
**Analysis Date:** 2026-04-01
## Directory Layout
```
surfaces-bot/
├── adapter/
│ ├── __init__.py
│ └── matrix/ # matrix-nio adapter (merged to main)
│ ├── __init__.py
│ ├── bot.py # Entry point, MatrixBot class, send_outgoing()
│ ├── converter.py # nio Event → IncomingEvent
│ ├── reactions.py # Emoji constants, skills text builder
│ ├── room_router.py # room_id → chat_id resolution
│ ├── store.py # Matrix-specific StateStore helpers (room meta, user meta)
│ └── handlers/
│ ├── __init__.py # register_matrix_handlers()
│ ├── auth.py # handle_invite (invite member event)
│ ├── chat.py # Chat creation (creates real Matrix rooms)
│ ├── confirm.py # Confirmation flow callbacks
│ └── settings.py # Settings sub-commands and toggle_skill
├── core/
│ ├── auth.py # AuthManager: start_flow, confirm, is_authenticated
│ ├── chat.py # ChatManager: get_or_create, list_active, rename, archive
│ ├── handler.py # EventDispatcher: register, dispatch, _routing_key
│ ├── protocol.py # All shared dataclasses and type aliases
│ ├── settings.py # SettingsManager: get (cached), apply (invalidates cache)
│ ├── store.py # StateStore Protocol, InMemoryStore, SQLiteStore
│ └── handlers/
│ ├── __init__.py # register_all() — binds all core handlers to dispatcher
│ ├── callback.py # handle_confirm, handle_cancel, handle_toggle_skill
│ ├── chat.py # handle_new_chat, handle_rename, handle_archive, handle_list_chats
│ ├── message.py # handle_message — auth guard + platform.send_message
│ ├── settings.py # handle_settings — displays settings menu
│ └── start.py # handle_start — get_or_create_user + welcome message
├── sdk/
│ ├── __init__.py
│ ├── interface.py # PlatformClient Protocol, WebhookReceiver Protocol, Pydantic models
│ └── mock.py # MockPlatformClient — full in-memory implementation
├── tests/
│ ├── __init__.py
│ ├── conftest.py # (root conftest — sys.path fix for local sdk/ shadowing stdlib)
│ ├── adapter/
│ │ ├── __init__.py
│ │ ├── matrix/
│ │ │ ├── __init__.py
│ │ │ ├── test_converter.py
│ │ │ ├── test_dispatcher.py
│ │ │ ├── test_reactions.py
│ │ │ └── test_store.py
│ │ └── test_forum_db.py # untracked — forum DB exploration
│ ├── core/
│ │ ├── test_auth.py
│ │ ├── test_chat.py
│ │ ├── test_dispatcher.py
│ │ ├── test_integration.py
│ │ ├── test_protocol.py
│ │ ├── test_settings.py
│ │ ├── test_store.py
│ │ └── test_voice_slot.py
│ └── platform/
│ └── test_mock.py
├── docs/ # All human documentation
├── .planning/ # GSD planning artefacts
│ └── codebase/ # Codebase map documents (this directory)
├── .claude/
│ └── agents/ # Agent configuration files
├── .worktrees/
│ └── telegram/ # Telegram adapter on feat/telegram-adapter branch
│ └── ... # Mirrors main layout; merged separately
├── conftest.py # Root pytest conftest: sys.path hack for local sdk/
├── pyproject.toml # Project metadata, dependencies, ruff + pytest config
├── uv.lock # Lockfile (uv)
├── lambda_matrix.db # SQLite DB written by Matrix bot (gitignored)
└── .env.example # Environment variable template
```
## Directory Purposes
**`core/`:**
- Purpose: Platform-neutral business logic. Never imports from `adapter/`.
- Key files: `protocol.py` (all shared types), `handler.py` (dispatcher), `store.py` (persistence interface)
- Add new domain logic here; keep it free of aiogram/matrix-nio imports
**`core/handlers/`:**
- Purpose: One async function per command/callback/message type. Each returns `list[OutgoingEvent]`.
- Registration: `register_all()` in `core/handlers/__init__.py` binds them to the dispatcher
- Adapters can override any key by calling `dispatcher.register(event_type, key, fn)` after `register_all()`
**`sdk/`:**
- Purpose: Contract (`interface.py`) and mock (`mock.py`) for the Lambda AI platform SDK
- Note: The directory is named `sdk/` in actual code (not `platform/` as CLAUDE.md describes); `handler.py` imports from `sdk.interface`
- When real SDK arrives: replace `sdk/mock.py` only; `sdk/interface.py` must not change unless the contract changes
**`adapter/matrix/`:**
- Purpose: Everything matrix-nio-specific. Translates between nio and core protocol.
- `bot.py` owns `MatrixBot`, `build_runtime()`, `send_outgoing()`, and `main()`
- `store.py` provides key-namespaced helpers on top of `StateStore` (not a separate store implementation)
- `room_router.py` maintains the `room_id → chat_id` mapping persisted in `StateStore`
**`adapter/telegram/`:**
- Purpose: aiogram 3.x adapter. Lives in `.worktrees/telegram/` on `feat/telegram-adapter` branch.
- Uses aiogram FSM states (`states.py`) and inline keyboards (`keyboards/`)
- Not yet merged to `main`
**`tests/`:**
- Purpose: pytest test suite mirroring the source tree
- `tests/core/` — unit tests for each core module
- `tests/adapter/matrix/` — Matrix adapter tests (converter, dispatcher, reactions, store)
- `tests/platform/` — MockPlatformClient tests
**`docs/`:**
- Purpose: Human-readable design documents; not consumed by code
- Key docs: `docs/surface-protocol.md` (unification rationale), `docs/api-contract.md` (SDK contract), `docs/telegram-prototype.md`, `docs/matrix-prototype.md`
## Key File Locations
**Entry Points:**
- `adapter/matrix/bot.py` — Matrix bot `main()`, run via `python -m adapter.matrix.bot`
- `.worktrees/telegram/adapter/telegram/bot.py` — Telegram bot entry (feature branch)
**Shared Protocol:**
- `core/protocol.py` — single source of truth for all inter-layer data types
**SDK Contract:**
- `sdk/interface.py``PlatformClient` Protocol; defines the API surface for the real SDK
- `sdk/mock.py``MockPlatformClient`; current runtime implementation
**Dispatcher Registration:**
- `core/handlers/__init__.py``register_all()` for platform-agnostic handlers
- `adapter/matrix/handlers/__init__.py``register_matrix_handlers()` for Matrix overrides
**Persistence:**
- `core/store.py``StateStore` Protocol, `InMemoryStore`, `SQLiteStore`
- `adapter/matrix/store.py` — Matrix-specific store helper functions (not a store implementation)
**Configuration:**
- `pyproject.toml` — dependencies, pytest config (`asyncio_mode = "auto"`, `pythonpath = ["."]`), ruff config
- `conftest.py``sys.path` insert so local `sdk/` shadows stdlib `platform` module
## Naming Conventions
**Files:**
- Modules: `snake_case.py`
- Entry points: `bot.py` per adapter
- Converter: `converter.py` per adapter
- Handlers directory: `handlers/` per layer
**Classes:**
- Managers: `{Domain}Manager` (e.g. `ChatManager`, `AuthManager`, `SettingsManager`)
- Bot runtime: `{Platform}Bot` (e.g. `MatrixBot`)
- Protocol types: PascalCase dataclasses (e.g. `IncomingMessage`, `OutgoingUI`)
- SDK types: PascalCase Pydantic models (e.g. `MessageResponse`, `UserSettings`)
**Handler functions:**
- `handle_{command}` for command handlers (e.g. `handle_start`, `handle_new_chat`)
- `make_handle_{command}` for factory functions that close over adapter state (e.g. `make_handle_new_chat(client, store)`)
**State keys:**
- `"{namespace}:{discriminator}"` — always use the prefix constants defined in `adapter/matrix/store.py`
## Where to Add New Code
**New core command handler:**
1. Add `async def handle_{cmd}(event, chat_mgr, auth_mgr, settings_mgr, platform) -> list` in `core/handlers/{category}.py`
2. Register it in `core/handlers/__init__.py:register_all()` with `dispatcher.register(IncomingCommand, "{cmd}", handle_{cmd})`
3. Write tests in `tests/core/test_dispatcher.py` or a dedicated `tests/core/test_{category}.py`
**New Matrix-specific handler (needs nio client or matrix store):**
1. Add handler in `adapter/matrix/handlers/{category}.py`
2. Register in `adapter/matrix/handlers/__init__.py:register_matrix_handlers()` — this overrides the core handler for that key
**New protocol type:**
- Add dataclass to `core/protocol.py`; update `IncomingEvent` or `OutgoingEvent` union aliases if it crosses layer boundaries
- Update `EventDispatcher._routing_key()` if it requires a new dispatch strategy
**New StateStore key namespace:**
- Add prefix constant and helper functions in `adapter/matrix/store.py` (for Matrix-specific state) or directly in the relevant manager (for core state)
**New test:**
- Unit tests for core logic: `tests/core/test_{module}.py`
- Adapter tests: `tests/adapter/matrix/test_{module}.py`
- Use `InMemoryStore` as the store; use `MockPlatformClient` as the platform client
## Special Directories
**`.worktrees/telegram/`:**
- Purpose: Git worktree for `feat/telegram-adapter` branch; full copy of the repo root
- Generated: Yes (via `git worktree add`)
- Committed: No (worktrees are local)
**`.planning/`:**
- Purpose: GSD planning artefacts — phase plans and codebase maps
- Generated: Yes (by `/gsd:` commands)
- Committed: Yes (tracked with the repo)
**`.claude/agents/`:**
- Purpose: Agent role configuration files for the multi-agent workflow
- Committed: Yes
**`src/`:**
- Purpose: Contains only `surfaces_bot.egg-info/` (setuptools build artefact); no source code
- Generated: Yes
- Committed: No
---
*Structure analysis: 2026-04-01*
- `core/`:
- `protocol.py` — Унифицированные структуры данных (сообщения, файлы, UI).
- `adapter/matrix/`:
- `bot.py` — Главный event-loop Matrix.
- `converter.py` — Конвертация событий Matrix-nio ⇄ `core/protocol.py`.
- `agent_registry.py` — Парсинг `matrix-agents.yaml`.
- `files.py` — Работа с вложениями и shared volume.
- `store.py` — SQLite база для маппинга чатов Matrix и `platform_chat_id`.
- `routed_platform.py` — Динамический роутинг вызовов к нужным агентам на лету.
- `sdk/`:
- `interface.py` — Интерфейс PlatformClient.
- `real.py` — Имплементация WebSocket клиента (`AgentApi`).
- `mock.py` — Мок-клиент для E2E тестов без платформы.
- `config/`: Конфиги маршрутизации (YAML).
- `docs/`: Актуальная документация по развертыванию и архитектуре.
- `docker-compose*.yml`: Продакшн и локальные манифесты для сборки.

View file

@ -1,210 +1,17 @@
# Testing Patterns
# Тестирование (TESTING.md)
**Analysis Date:** 2026-04-01
## Unit-тесты
Расположены в `tests/`. Покрытие сфокусировано на логике Matrix адаптера (пока он является основной поверхностью):
- Файловый контракт (`test_files.py`)
- Диспетчер и конвертация (`test_dispatcher.py`)
- Взаимодействие с PlatformClient (`test_routed_platform.py`)
- Работа с контекстными командами бота (`test_context_commands.py`)
## Test Framework
## E2E тестирование
Локально тестируется через запуск контейнеров из `docker-compose.fullstack.yml`, который поднимает один инстанс бота и один локальный `platform-agent`. Это позволяет имитировать полную цепочку взаимодействия (Matrix -> Бот -> Агент) с общим каталогом для файлов.
**Runner:** pytest 8.x
**Config:** `pyproject.toml` `[tool.pytest.ini_options]`
```toml
[tool.pytest.ini_options]
asyncio_mode = "auto"
testpaths = ["tests"]
pythonpath = ["."]
```
**Async support:** pytest-asyncio with `asyncio_mode = "auto"` — all `async def` test functions run automatically without decorators.
**Coverage:** pytest-cov (available but no minimum threshold configured)
**Run commands:**
## Запуск тестов
```bash
pytest tests/ -v # all tests
pytest tests/core/ -v # core layer only
pytest tests/adapter/telegram/ -v # telegram adapter only
pytest tests/adapter/matrix/ -v # matrix adapter only
pytest tests/ --cov=. --cov-report=term # with coverage report
# Запуск юнит-тестов (только для Matrix адаптера)
pytest tests/adapter/matrix/ -v
```
## Test Directory Structure
```
tests/
├── __init__.py
├── core/
│ ├── test_auth.py — AuthManager unit tests
│ ├── test_chat.py — ChatManager unit tests
│ ├── test_dispatcher.py — EventDispatcher routing tests
│ ├── test_integration.py — full flow smoke tests (dispatcher + managers + mock)
│ ├── test_protocol.py — dataclass defaults and construction
│ ├── test_settings.py — SettingsManager unit tests
│ ├── test_store.py — InMemoryStore + SQLiteStore tests
│ └── test_voice_slot.py — handle_message() handler unit tests
├── adapter/
│ ├── __init__.py
│ ├── test_forum_db.py — Telegram SQLite DB helpers (untracked, new)
│ └── matrix/
│ ├── __init__.py
│ ├── test_converter.py — matrix-nio event → IncomingEvent converter
│ ├── test_dispatcher.py — full Matrix bot integration (build_runtime)
│ ├── test_reactions.py — reaction text builders and emoji mapping
│ └── test_store.py — Matrix store helper functions
└── platform/
└── test_mock.py — MockPlatformClient behavior
```
Tests mirror the source tree. New tests for `adapter/telegram/` go in `tests/adapter/telegram/` (directory exists in `.worktrees/telegram` branch, not yet merged to main).
## conftest.py
`conftest.py` at project root (`/Users/a/MAI/sem2/lambda/surfaces-bot/conftest.py`) handles a sys.path conflict: the project has a local `platform/` (now `sdk/`) package that shadows Python's stdlib `platform` module. It inserts the project root at `sys.path[0]` and removes the cached stdlib `platform` module.
No shared fixtures are defined in `conftest.py`. All fixtures are local to test files.
## Test Structure
**Fixture pattern — local to each test file:**
```python
@pytest.fixture
def mgr():
return AuthManager(MockPlatformClient(), InMemoryStore())
@pytest.fixture
def store() -> InMemoryStore:
return InMemoryStore()
```
**Async tests require no decorator** (asyncio_mode = "auto"):
```python
async def test_not_authenticated_initially(mgr):
assert await mgr.is_authenticated("u1") is False
```
**Sync tests** are used for pure-function tests (protocol dataclass construction, reaction text builders):
```python
def test_incoming_message_defaults():
msg = IncomingMessage(user_id="u1", platform="telegram", chat_id="C1", text="hi")
assert msg.attachments == []
```
**Integration fixture pattern** — builds full runtime in-process:
```python
@pytest.fixture
def dispatcher():
platform = MockPlatformClient()
store = InMemoryStore()
d = EventDispatcher(
platform=platform,
chat_mgr=ChatManager(platform, store),
auth_mgr=AuthManager(platform, store),
settings_mgr=SettingsManager(platform, store),
)
register_all(d)
return d
```
## Mocking Strategy
**Primary mock: `MockPlatformClient`** from `sdk/mock.py`
All tests use `MockPlatformClient()` directly — it is the real mock for the SDK layer. No unittest.mock patching of `MockPlatformClient` is needed.
**`unittest.mock.AsyncMock`** is used only when testing integration with external clients (matrix-nio `AsyncClient`):
```python
from unittest.mock import AsyncMock
client = SimpleNamespace(
room_create=AsyncMock(return_value=SimpleNamespace(room_id="!r2:example"))
)
```
**`types.SimpleNamespace`** is used to fabricate matrix-nio event objects without importing the full nio library:
```python
def text_event(body: str, sender: str = "@a:m.org", event_id: str = "$e1"):
return SimpleNamespace(
sender=sender, body=body, event_id=event_id, msgtype="m.text", replyto_event_id=None
)
```
This is the pattern for all matrix converter tests — define factory functions at module level that return `SimpleNamespace` objects.
**`tmp_path` pytest fixture** is used for SQLiteStore tests to get a throwaway database file:
```python
async def test_sqlite_set_and_get(tmp_path):
store = SQLiteStore(str(tmp_path / "test.db"))
```
**`monkeypatch.setenv`** is used in `tests/adapter/test_forum_db.py` to inject `DB_PATH` env var and reload the module with a fresh database:
```python
@pytest.fixture(autouse=True)
def fresh_db(tmp_path, monkeypatch):
db_file = str(tmp_path / "test.db")
monkeypatch.setenv("DB_PATH", db_file)
import importlib
import adapter.telegram.db as db_mod
importlib.reload(db_mod)
db_mod.init_db()
return db_mod
```
**What NOT to mock:**
- `InMemoryStore` — use it directly; it's a real in-memory implementation
- `MockPlatformClient` — use it directly; patching it defeats the purpose
- Core manager classes (`AuthManager`, `ChatManager`, `SettingsManager`) — always instantiate real ones
## Test Data Patterns
**User IDs:** short strings like `"u1"`, `"u2"`, `"tg_123"`, `"@alice:m.org"`
**Chat IDs:** `"C1"`, `"C2"`, `"C3"` — matches the workspace slot naming
**Platform strings:** literal `"telegram"` or `"matrix"`
**Room IDs:** `"!r:m.org"`, `"!dm:example.org"` — valid Matrix room ID format
No shared factories or fixtures files. Test data is constructed inline or via simple factory functions local to the test module.
## What Is Tested
| Area | Status |
|------|--------|
| `core/protocol.py` — dataclass defaults | Covered (`test_protocol.py`) |
| `core/store.py` — InMemoryStore + SQLiteStore | Covered (`test_store.py`) |
| `core/auth.py` — AuthManager | Covered (`test_auth.py`) |
| `core/chat.py` — ChatManager | Covered (`test_chat.py`) |
| `core/settings.py` — SettingsManager | Covered (`test_settings.py`) |
| `core/handler.py` — EventDispatcher routing | Covered (`test_dispatcher.py`) |
| `core/handlers/message.py` — handle_message | Covered (`test_voice_slot.py`) |
| Full dispatcher + all core handlers integration | Covered (`test_integration.py`) |
| `sdk/mock.py` — MockPlatformClient | Covered (`test_mock.py`) |
| `adapter/matrix/converter.py` — event parsing | Covered (`test_converter.py`) |
| `adapter/matrix/store.py` — store helpers | Covered (`test_store.py`) |
| `adapter/matrix/reactions.py` — text builders | Covered (`test_reactions.py`) |
| `adapter/matrix/bot.py` — MatrixBot + build_runtime | Covered (`test_dispatcher.py`) |
| `adapter/telegram/db.py` — SQLite helpers | Covered (`test_forum_db.py`, untracked) |
## Coverage Gaps
**Telegram adapter handlers** — `adapter/telegram/handlers/` (`auth.py`, `chat.py`, `confirm.py`, `forum.py`, `settings.py`) have no tests in `main`. Tests exist only in `.worktrees/telegram` branch (not yet merged).
**Telegram converter** — `adapter/telegram/converter.py` has no tests in `main`.
**`core/handlers/callback.py` and `core/handlers/settings.py`** — tested indirectly through integration tests but lack dedicated unit tests.
**`adapter/matrix/room_router.py`** — `resolve_chat_id` has no direct unit tests; exercised only through `MatrixBot.on_room_message` integration path.
**`adapter/matrix/handlers/`** — individual handler files (`auth.py`, `chat.py`, `confirm.py`, `settings.py`) are tested only via `test_dispatcher.py` integration; no isolated unit tests.
**`sdk/mock.py` streaming** — `stream_message` is not tested; only `send_message` is covered.
**Error paths** — `ChatManager.rename` raises `ValueError` when chat not found; no test exercises this path. Same for `ChatManager.archive`.
## Naming Conventions
- Test functions: `test_<behavior_under_test>` — descriptive, no abbreviations
- Fixture names match the object they create: `mgr`, `store`, `dispatcher`, `deps`
- Factory functions in converter tests: `text_event()`, `file_event()`, `image_event()`, `reaction_event()`
---
*Testing analysis: 2026-04-01*

View file

@ -1,48 +0,0 @@
---
phase: 01.1-matrix-restart-reconciliation-and-dev-reset-workflow
task: 1
total_tasks: 2
status: paused
last_updated: 2026-04-04T10:13:58.720Z
---
<current_state>
Formally, the most recently active GSD artifact is `01.1-03-PLAN.md`, which has not been executed yet. In parallel, an out-of-band research pass compared the local mock SDK against platform repos and concluded that Phase 02 SDK integration is still blocked on an unstable control-plane contract.
</current_state>
<completed_work>
- Session research: inspected local `sdk/interface.py`, `sdk/mock.py`, core message/settings usage, and platform repos `agent_api`, `agent`, `master`, `docs`.
- Established that the real platform currently provides a direct WebSocket `agent_api` for talking to the agent, while `master` is still mostly a control-plane skeleton rather than a stable consumer-facing API.
- Confirmed that the current local mock assumes a richer unified platform API than what is actually implemented today.
- Concluded that consumer adapters should not be deeply rewritten yet; Matrix remains the right internal testing surface for now.
</completed_work>
<remaining_work>
- Task 1: Implement `adapter.matrix.reset` with `local-only`, `server-leave-forget`, and `--dry-run`, plus tests.
- Task 2: Update `README.md` so restart vs explicit reset workflow is documented and the old manual reset ritual is removed.
- Phase 02 follow-up, once platform stabilizes: split the current platform boundary into control-plane and direct-agent-session abstractions instead of keeping a single `PlatformClient`.
</remaining_work>
<decisions_made>
- Keep the current consumer-facing bot logic largely intact for now; do not force an early rewrite around the incomplete platform backend.
- Treat `sdk/mock.py` as a temporary local integration facade, not as a near-drop-in simulation of the real platform.
- Use Matrix for internal testing while waiting for the platform team to finalize the minimal control-plane contract.
</decisions_made>
<blockers>
- Platform contract blocker: `agent_api` is concrete enough to study, but `master` still does not expose a stable user/chat/session/settings API for surfaces.
- Product contract blocker: attachments, settings, webhook-style long task events, and exact session bootstrap flow are still unclear on the platform side.
</blockers>
<context>
The key mental model from this session: our mock pretends the platform is already a complete backend, but the real platform today is split. There is a usable direct agent WebSocket protocol, and there is a developing master control plane, but they have not converged into the unified SDK shape that the bot currently assumes. Because of that, the right near-term move is not to rush integration, but to preserve momentum with Matrix/internal testing and keep the future integration boundary explicit.
</context>
<next_action>
Start with one of these, depending on priority:
1. Execute `01.1-03-PLAN.md` Task 1 and build the Matrix reset CLI.
2. If returning to platform research, write a concrete draft interface for `MasterClient` + `AgentSession` while leaving consumer adapters unchanged.
</next_action>

View file

@ -1,157 +0,0 @@
---
phase: 01.1-matrix-restart-reconciliation-and-dev-reset-workflow
plan: 01
type: execute
wave: 1
depends_on: []
files_modified:
- adapter/matrix/reconcile.py
- tests/adapter/matrix/test_reconcile.py
autonomous: true
requirements: []
must_haves:
truths:
- "A normal Matrix restart can rebuild missing local metadata from already joined Space/chat rooms instead of requiring a destructive reset."
- "Reconciliation restores the minimal local state needed for routing and chat operations: `matrix_user:*`, `matrix_room:*`, and missing `chat:{user}:{chat_id}` rows."
- "Reconciliation never provisions new Matrix rooms or Spaces while repairing local state."
- "Recovered users get `next_chat_index` advanced past the highest recovered `C*` chat id."
artifacts:
- path: "adapter/matrix/reconcile.py"
provides: "Matrix bootstrap reconciliation helpers and structured report objects."
- path: "tests/adapter/matrix/test_reconcile.py"
provides: "Regression coverage for startup and single-room reconciliation behavior."
key_links:
- from: "adapter/matrix/reconcile.py"
to: "adapter/matrix/store.py"
via: "set_user_meta and set_room_meta restore Matrix metadata"
pattern: "set_(user|room)_meta"
- from: "adapter/matrix/reconcile.py"
to: "core/chat.py"
via: "chat_mgr.get_or_create repairs missing `chat:*` rows"
pattern: "chat_mgr\\.get_or_create"
---
<objective>
Create the non-destructive Matrix reconciliation layer that Phase 01.1 depends on.
Purpose: Per D-01 through D-07, the adapter must stop treating local SQLite state as the only truth in dev. Startup and recovery code need a single helper module that can rebuild local metadata from the homeserver room graph without creating duplicate Spaces or chats.
Output: `adapter/matrix/reconcile.py` with full-run and single-room recovery helpers, plus targeted pytest coverage.
</objective>
<execution_context>
@/Users/a/.codex/get-shit-done/workflows/execute-plan.md
@/Users/a/.codex/get-shit-done/templates/summary.md
</execution_context>
<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/01.1-matrix-restart-reconciliation-and-dev-reset-workflow/01.1-CONTEXT.md
@.planning/phases/01.1-matrix-restart-reconciliation-and-dev-reset-workflow/01.1-RESEARCH.md
@.planning/phases/01-matrix-qa-polish/01-01-SUMMARY.md
@adapter/matrix/store.py
@adapter/matrix/handlers/auth.py
@core/chat.py
@tests/adapter/matrix/test_invite_space.py
<interfaces>
From `adapter/matrix/store.py`:
```python
async def get_room_meta(store: StateStore, room_id: str) -> dict | None
async def set_room_meta(store: StateStore, room_id: str, meta: dict) -> None
async def get_user_meta(store: StateStore, matrix_user_id: str) -> dict | None
async def set_user_meta(store: StateStore, matrix_user_id: str, meta: dict) -> None
```
From `core/chat.py`:
```python
async def get_or_create(
self,
user_id: str,
chat_id: str,
platform: str,
surface_ref: str,
name: str | None = None,
) -> ChatContext
```
From Phase 01 room metadata shape:
```python
{
"room_type": "chat",
"chat_id": "C4",
"display_name": "Чат 4",
"matrix_user_id": "@alice:example.org",
"space_id": "!space:example.org",
}
```
</interfaces>
</context>
<tasks>
<task type="auto" tdd="true">
<name>Task 1: Add reconciliation module for startup and single-room recovery</name>
<files>adapter/matrix/reconcile.py, tests/adapter/matrix/test_reconcile.py</files>
<read_first>adapter/matrix/store.py, adapter/matrix/handlers/auth.py, core/chat.py, tests/adapter/matrix/test_invite_space.py, .planning/phases/01.1-matrix-restart-reconciliation-and-dev-reset-workflow/01.1-CONTEXT.md</read_first>
<behavior>
- Test 1: `reconcile_matrix_state(...)` recreates missing `matrix_user:*`, `matrix_room:*`, and `chat:*` entries from joined Matrix rooms without calling `room_create`.
- Test 2: `reconcile_matrix_state(...)` leaves already-correct local metadata intact and reports restored vs skipped/conflicting rooms.
- Test 3: `reconcile_single_room(...)` can repair one `unregistered:{room_id}` chat room on demand and recompute `next_chat_index` for that user.
- Test 4: Space rooms or unrelated joined rooms are skipped, not converted into chat rows.
</behavior>
<action>
Create `adapter/matrix/reconcile.py` as the authoritative recovery module for this phase. Implement a small, explicit API that Plan 02 can wire directly:
```python
async def reconcile_matrix_state(client: Any, store: StateStore, chat_mgr: ChatManager) -> dict: ...
async def reconcile_single_room(
client: Any, store: StateStore, chat_mgr: ChatManager, room_id: str, matrix_user_id: str
) -> dict: ...
```
Inside this module, add focused private helpers as needed for room classification, extracting room names, parsing `C<number>` ids, and recomputing `next_chat_index`. Keep the logic non-destructive per D-04:
- never call `room_create`, `room_invite`, or provisioning code from `handlers/auth.py`
- prefer already-hydrated room data from the post-sync client object, and only fall back to explicit room-state fetches if required for room classification
- rebuild only the minimal metadata required by D-03: `matrix_user:*`, `matrix_room:*`, and missing `chat:{user}:{chat_id}` records
- if `chat:*` exists but points at the wrong `surface_ref`, repair it from Matrix room metadata and include the fix in the returned report
- derive `next_chat_index` from the highest recovered `C<number>` for that user instead of trusting stale local counters
Return a structured reconciliation report with stable keys such as:
`joined_rooms`, `restored_user_meta`, `restored_room_meta`, `restored_chat_rows`, `repaired_chat_rows`, `skipped_rooms`, and `conflicts`.
Write `tests/adapter/matrix/test_reconcile.py` with lightweight `SimpleNamespace`/fake-client fixtures following the existing Matrix test style. Cover both full startup reconciliation and `reconcile_single_room(...)`. Assert that no provisioning calls are made during reconciliation, because D-04 forbids creating new Space/room topology while recovering local state.
</action>
<verify>
<automated>cd /Users/a/MAI/sem2/lambda/surfaces-bot && pytest tests/adapter/matrix/test_reconcile.py -q</automated>
</verify>
<acceptance_criteria>
- `adapter/matrix/reconcile.py` exports `reconcile_matrix_state` and `reconcile_single_room`.
- Reconciliation restores missing `matrix_user:*`, `matrix_room:*`, and `chat:*` entries for already-joined Matrix chat rooms per D-02 and D-03.
- Reconciliation does not call `room_create` or otherwise provision new server-side rooms per D-04.
- The report returned by reconciliation clearly distinguishes restored items, skipped rooms, and conflicts.
- `tests/adapter/matrix/test_reconcile.py` proves `next_chat_index` is recomputed from recovered chat ids rather than stale local state.
</acceptance_criteria>
<done>The repository has an executable, tested reconciliation layer that can rebuild local Matrix metadata after dev-state loss without duplicating server-side rooms.</done>
</task>
</tasks>
<verification>
Run `pytest tests/adapter/matrix/test_reconcile.py -q` and confirm startup and single-room reconciliation paths are covered.
</verification>
<success_criteria>
- Matrix recovery logic exists as a dedicated module instead of being scattered through handlers.
- Reconciliation is idempotent, non-destructive, and sufficient to restore routing/chat metadata from existing Matrix rooms.
- Plan 02 can wire startup and first-access recovery by calling exported functions rather than inventing new recovery logic.
</success_criteria>
<output>
After completion, create `.planning/phases/01.1-matrix-restart-reconciliation-and-dev-reset-workflow/01.1-01-SUMMARY.md`
</output>

View file

@ -1,167 +0,0 @@
---
phase: 01.1-matrix-restart-reconciliation-and-dev-reset-workflow
plan: 02
type: execute
wave: 2
depends_on: ["01.1-01"]
files_modified:
- adapter/matrix/bot.py
- tests/adapter/matrix/test_dispatcher.py
autonomous: true
requirements: []
must_haves:
truths:
- "The Matrix bot performs an initial sync and reconciliation before entering steady-state `sync_forever()`."
- "If a room still arrives as `unregistered:{room_id}` after startup, the bot makes one targeted recovery attempt before dispatching or failing."
- "When reconciliation cannot repair a room, the bot logs a clear diagnostic reason instead of crashing on downstream commands like `!rename`."
artifacts:
- path: "adapter/matrix/bot.py"
provides: "Startup bootstrap flow with initial sync, reconciliation, and targeted runtime retry."
- path: "tests/adapter/matrix/test_dispatcher.py"
provides: "Matrix runtime coverage for pre-sync reconcile and on-message recovery behavior."
key_links:
- from: "adapter/matrix/bot.py"
to: "adapter/matrix/reconcile.py"
via: "startup bootstrap and single-room recovery calls"
pattern: "reconcile_(matrix_state|single_room)"
- from: "adapter/matrix/bot.py"
to: "adapter/matrix/room_router.py"
via: "unregistered room detection before dispatch"
pattern: "unregistered:"
---
<objective>
Wire the new reconciliation layer into the actual Matrix runtime.
Purpose: D-05 through D-07 require restart recovery to be the default developer path. The bot must bootstrap itself from existing Matrix rooms on startup and make one on-demand repair attempt before routing an unknown room through the dispatcher.
Output: `adapter/matrix/bot.py` performs initial sync + reconciliation before `sync_forever()`, and runtime tests prove the bot recovers or logs clearly instead of blindly dispatching broken state.
</objective>
<execution_context>
@/Users/a/.codex/get-shit-done/workflows/execute-plan.md
@/Users/a/.codex/get-shit-done/templates/summary.md
</execution_context>
<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/01.1-matrix-restart-reconciliation-and-dev-reset-workflow/01.1-CONTEXT.md
@.planning/phases/01.1-matrix-restart-reconciliation-and-dev-reset-workflow/01.1-RESEARCH.md
@.planning/phases/01.1-matrix-restart-reconciliation-and-dev-reset-workflow/01.1-01-PLAN.md
@adapter/matrix/bot.py
@adapter/matrix/room_router.py
@adapter/matrix/reconcile.py
@tests/adapter/matrix/test_dispatcher.py
<interfaces>
From `adapter/matrix/bot.py`:
```python
class MatrixBot:
async def on_room_message(self, room: MatrixRoom, event: RoomMessageText) -> None
async def main() -> None
```
From `adapter/matrix/reconcile.py`:
```python
async def reconcile_matrix_state(client: Any, store: StateStore, chat_mgr: ChatManager) -> dict
async def reconcile_single_room(
client: Any, store: StateStore, chat_mgr: ChatManager, room_id: str, matrix_user_id: str
) -> dict
```
From `adapter/matrix/room_router.py`:
```python
async def resolve_chat_id(store: StateStore, room_id: str, matrix_user_id: str) -> str
```
</interfaces>
</context>
<tasks>
<task type="auto" tdd="true">
<name>Task 1: Run initial sync and reconciliation before the long-poll loop</name>
<files>adapter/matrix/bot.py, tests/adapter/matrix/test_dispatcher.py</files>
<read_first>adapter/matrix/bot.py, adapter/matrix/reconcile.py, tests/adapter/matrix/test_dispatcher.py, .planning/phases/01.1-matrix-restart-reconciliation-and-dev-reset-workflow/01.1-RESEARCH.md</read_first>
<behavior>
- Test 1: `main()` performs `client.sync(timeout=0, full_state=True)` before `sync_forever()`.
- Test 2: `main()` calls `reconcile_matrix_state(...)` after the initial sync and logs the returned report.
- Test 3: startup still reaches `sync_forever()` when reconciliation reports recoverable skips/conflicts instead of fatal failure.
</behavior>
<action>
Modify `adapter/matrix/bot.py` so normal startup follows the two-phase bootstrap recommended in research:
1. build client and runtime
2. authenticate
3. register callbacks
4. run `await client.sync(timeout=0, full_state=True)`
5. run `await reconcile_matrix_state(client, runtime.store, runtime.chat_mgr)`
6. log a structured `matrix_reconcile_complete` event with the report fields
7. enter `await client.sync_forever(timeout=30000)`
Do not move provisioning logic into startup. The startup step only rehydrates local state from server-side rooms per D-02 through D-04.
Update or add focused tests in `tests/adapter/matrix/test_dispatcher.py` using `monkeypatch`/fake-client patterns already used in the repo so the verify command proves the call order and logging-safe behavior. The test should fail if `sync_forever()` starts before reconciliation.
</action>
<verify>
<automated>cd /Users/a/MAI/sem2/lambda/surfaces-bot && pytest tests/adapter/matrix/test_dispatcher.py -q</automated>
</verify>
<acceptance_criteria>
- `adapter/matrix/bot.py` runs an initial full-state sync before steady-state polling.
- `adapter/matrix/bot.py` invokes `reconcile_matrix_state(...)` exactly once during startup.
- Startup logs a structured reconciliation summary instead of silently skipping the recovery step.
- `tests/adapter/matrix/test_dispatcher.py` asserts the bootstrap order explicitly.
</acceptance_criteria>
<done>Normal Matrix bot startup now includes a recovery pass before the event loop begins handling user traffic.</done>
</task>
<task type="auto" tdd="true">
<name>Task 2: Retry unknown-room routing once before dispatching broken state</name>
<files>adapter/matrix/bot.py, tests/adapter/matrix/test_dispatcher.py</files>
<read_first>adapter/matrix/bot.py, adapter/matrix/room_router.py, adapter/matrix/reconcile.py, tests/adapter/matrix/test_dispatcher.py, .planning/phases/01.1-matrix-restart-reconciliation-and-dev-reset-workflow/01.1-CONTEXT.md</read_first>
<behavior>
- Test 1: `MatrixBot.on_room_message(...)` detects `unregistered:{room_id}`, runs `reconcile_single_room(...)`, then retries `resolve_chat_id(...)`.
- Test 2: if retry succeeds, the event is dispatched against the recovered logical chat id.
- Test 3: if retry still fails, the bot does not crash; it logs a clear warning and sends a user-facing diagnostic message to that room.
</behavior>
<action>
Extend `MatrixBot.on_room_message(...)` so D-07 is satisfied even when startup could not repair a room yet. Keep `resolve_chat_id(...)` as the room-router source of truth, but treat `unregistered:{room_id}` as a recovery trigger rather than a stable runtime identity:
- first call `resolve_chat_id(...)`
- if the result starts with `unregistered:`, call `reconcile_single_room(client, runtime.store, runtime.chat_mgr, room.room_id, event.sender)`
- immediately retry `resolve_chat_id(...)`
- only dispatch once a concrete logical chat id exists
- if the retry still returns `unregistered:{room_id}`, log a structured warning with room id, matrix user id, and reconciliation report, then send a short `OutgoingMessage`-equivalent Matrix text explaining that local state could not be restored automatically and a dev reset/restart may be required
Do not invent a new fallback chat id and do not auto-create rooms here; that would violate D-04. Keep this change inside `adapter/matrix/bot.py` so file ownership stays isolated for this plan.
</action>
<verify>
<automated>cd /Users/a/MAI/sem2/lambda/surfaces-bot && pytest tests/adapter/matrix/test_dispatcher.py -q</automated>
</verify>
<acceptance_criteria>
- Unknown Matrix rooms trigger one targeted reconciliation attempt before dispatch.
- Successful targeted recovery leads to normal dispatch with a real logical `chat_id`.
- Failed targeted recovery logs a clear diagnostic and avoids a handler crash on missing chat state per D-06.
- No code path in this task provisions new Matrix rooms or Spaces.
</acceptance_criteria>
<done>The runtime treats unknown rooms as recoverable state drift first, not as a silent routing failure or crash path.</done>
</task>
</tasks>
<verification>
Run `pytest tests/adapter/matrix/test_dispatcher.py -q` and confirm both startup-bootstrap and first-access recovery behaviors are covered.
</verification>
<success_criteria>
- A standard Matrix restart now attempts recovery before the bot starts processing live events.
- Unknown-room events are diagnosable and recoverable instead of falling straight into broken command handling.
- The runtime never provisions new server-side rooms during restart reconciliation.
</success_criteria>
<output>
After completion, create `.planning/phases/01.1-matrix-restart-reconciliation-and-dev-reset-workflow/01.1-02-SUMMARY.md`
</output>

View file

@ -1,149 +0,0 @@
---
phase: 01.1-matrix-restart-reconciliation-and-dev-reset-workflow
plan: 03
type: execute
wave: 1
depends_on: []
files_modified:
- adapter/matrix/reset.py
- tests/adapter/matrix/test_reset.py
- README.md
autonomous: true
requirements: []
must_haves:
truths:
- "Developers have an explicit dev-only reset command instead of relying on memory or ad hoc shell history."
- "The default reset mode clears only local Matrix state and explains the manual Matrix-client cleanup that may still be needed."
- "Optional server cleanup is clearly named around leave/forget semantics and supports dry-run output."
artifacts:
- path: "adapter/matrix/reset.py"
provides: "Dev reset CLI for local-only, server-leave-forget, and dry-run workflows."
- path: "tests/adapter/matrix/test_reset.py"
provides: "CLI coverage for local reset behavior and printed operator guidance."
- path: "README.md"
provides: "Updated developer instructions for normal restart vs explicit reset."
key_links:
- from: "adapter/matrix/reset.py"
to: "README.md"
via: "documented invocation and manual Matrix cleanup guidance"
pattern: "adapter\\.matrix\\.reset"
---
<objective>
Ship the dev reset workflow that complements normal restart reconciliation.
Purpose: D-08 through D-10 require a repeatable, explicit reset path for clean-room QA without making destructive cleanup the default restart flow. This plan creates the tool and updates the runbook developers actually use.
Output: `adapter/matrix/reset.py`, pytest coverage, and README instructions that replace the old `rm -f lambda_matrix.db` ritual.
</objective>
<execution_context>
@/Users/a/.codex/get-shit-done/workflows/execute-plan.md
@/Users/a/.codex/get-shit-done/templates/summary.md
</execution_context>
<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/01.1-matrix-restart-reconciliation-and-dev-reset-workflow/01.1-CONTEXT.md
@.planning/phases/01.1-matrix-restart-reconciliation-and-dev-reset-workflow/01.1-RESEARCH.md
@README.md
@adapter/matrix/bot.py
@core/store.py
<interfaces>
From `adapter/matrix/bot.py` env usage:
```python
db_path = os.environ.get("MATRIX_DB_PATH", "lambda_matrix.db")
store_path = os.environ.get("MATRIX_STORE_PATH", "matrix_store")
homeserver = os.environ.get("MATRIX_HOMESERVER")
user_id = os.environ.get("MATRIX_USER_ID")
```
From `core/store.py`:
```python
class SQLiteStore:
def __init__(self, db_path: str) -> None: ...
```
</interfaces>
</context>
<tasks>
<task type="auto" tdd="true">
<name>Task 1: Add a dev-only Matrix reset CLI with explicit modes</name>
<files>adapter/matrix/reset.py, tests/adapter/matrix/test_reset.py</files>
<read_first>adapter/matrix/bot.py, core/store.py, .planning/phases/01.1-matrix-restart-reconciliation-and-dev-reset-workflow/01.1-RESEARCH.md</read_first>
<behavior>
- Test 1: `--mode local-only` deletes the configured local DB/store paths or reports what would be deleted in dry-run mode.
- Test 2: `--mode server-leave-forget --dry-run` prints the exact rooms it would leave/forget and does not mutate local files.
- Test 3: when server cleanup is not executed, the command prints the manual Matrix-client steps required by D-10.
</behavior>
<action>
Create `adapter/matrix/reset.py` as a CLI entrypoint runnable via `uv run python -m adapter.matrix.reset`. Use `argparse` and keep the tool explicitly dev-only in its help text and logs.
Implement the following modes from research and locked decisions:
- `local-only` (default destructive mode for local QA): remove `MATRIX_DB_PATH` and `MATRIX_STORE_PATH` if they exist; if not, report that they were already absent
- `server-leave-forget`: for the bot account only, log in using the same Matrix env vars as `adapter/matrix/bot.py`, inspect joined rooms, and call `room_leave()` + `room_forget()` for each joined room; support `--dry-run` so the operator can inspect the target set before mutation
- `--dry-run` must work with both modes and print a structured summary instead of mutating files or Matrix membership
Always print a post-run summary that distinguishes:
- what local files/directories were deleted or would be deleted
- what server-side leave/forget actions were executed or would be executed
- the manual Matrix client steps still required for a true clean-room QA rerun (leave/archive old rooms or Space in Element, accept fresh invites, etc.) when those actions are outside this phase
Write `tests/adapter/matrix/test_reset.py` to cover local-only deletion, dry-run output, and server-leave-forget dry-run behavior with fake clients/temporary directories. Follow the repos existing lightweight async test style.
</action>
<verify>
<automated>cd /Users/a/MAI/sem2/lambda/surfaces-bot && pytest tests/adapter/matrix/test_reset.py -q</automated>
</verify>
<acceptance_criteria>
- `adapter/matrix/reset.py` supports `local-only`, `server-leave-forget`, and `--dry-run`.
- `local-only` reset targets both `lambda_matrix.db` and `matrix_store` via env-aware paths per D-09.
- The tool never claims to globally delete Matrix rooms; it uses leave/forget semantics or prints manual cleanup instructions per D-10.
- `tests/adapter/matrix/test_reset.py` proves dry-run mode is non-destructive.
</acceptance_criteria>
<done>The repository contains a repeatable dev reset tool that replaces the undocumented shell ritual and names server-side cleanup honestly.</done>
</task>
<task type="auto">
<name>Task 2: Replace the README reset ritual with the new restart and reset workflow</name>
<files>README.md</files>
<read_first>README.md, .planning/phases/01.1-matrix-restart-reconciliation-and-dev-reset-workflow/01.1-CONTEXT.md, .planning/phases/01.1-matrix-restart-reconciliation-and-dev-reset-workflow/01.1-RESEARCH.md</read_first>
<action>
Update `README.md` so Matrix development instructions reflect Phase 01.1 instead of the old destructive reset ritual. Replace the current manual QA block that tells developers to `rm -f lambda_matrix.db` with a short, explicit split:
- normal restart: `PYTHONPATH=. uv run python -m adapter.matrix.bot` now performs reconciliation automatically
- explicit clean-room reset: `PYTHONPATH=. uv run python -m adapter.matrix.reset --mode local-only`
- optional server cleanup preview: `PYTHONPATH=. uv run python -m adapter.matrix.reset --mode server-leave-forget --dry-run`
State clearly that normal restart is the default path per D-05, and that full server-side cleanup may still require manual steps in the Matrix client. Keep the README concise; do not add production guidance or Phase 2 SDK content.
</action>
<verify>
<automated>cd /Users/a/MAI/sem2/lambda/surfaces-bot && python -m adapter.matrix.reset --help >/tmp/matrix-reset-help.txt && rg -n "adapter.matrix.reset|local-only|server-leave-forget|reconciliation" README.md /tmp/matrix-reset-help.txt</automated>
</verify>
<acceptance_criteria>
- `README.md` no longer recommends raw `rm -f lambda_matrix.db` as the default Matrix restart workflow.
- `README.md` documents the normal restart path and the explicit reset path separately.
- The documented reset commands match the CLI implemented in `adapter/matrix/reset.py`.
</acceptance_criteria>
<done>Developers can follow a repeatable README workflow for ordinary restart and clean-room QA reset without relying on tribal knowledge.</done>
</task>
</tasks>
<verification>
Run `pytest tests/adapter/matrix/test_reset.py -q` and `python -m adapter.matrix.reset --help`, then confirm the README commands and help text stay aligned.
</verification>
<success_criteria>
- Dev reset is an explicit tool, not a remembered shell sequence.
- Local-only reset is automated and documented.
- Server cleanup semantics are honest, dry-runnable, and accompanied by manual Matrix-client guidance where needed.
</success_criteria>
<output>
After completion, create `.planning/phases/01.1-matrix-restart-reconciliation-and-dev-reset-workflow/01.1-03-SUMMARY.md`
</output>

View file

@ -1,121 +0,0 @@
# Phase 01.1: Matrix restart reconciliation and dev reset workflow - Context
**Gathered:** 2026-04-03
**Status:** Ready for planning
<domain>
## Phase Boundary
Сделать Matrix-адаптер пригодным для повторяемой локальной разработки и ручного QA без ручного “ритуала” удаления БД, выхода из всех комнат и пересоздания пользователя.
В scope этой фазы:
- безопасный restart flow для Matrix-бота после потери локального state
- reconciliation локального store с уже существующими Matrix rooms / Space
- отдельный dev reset workflow для controlled clean-room QA
- диагностируемое поведение при несогласованности local state и server-side Matrix state
Вне scope:
- реальный Lambda SDK
- новые пользовательские Matrix features
- E2EE
- production-grade multi-user migration framework
</domain>
<decisions>
## Implementation Decisions
### Matrix state lifecycle
- **D-01:** Локальный SQLite store больше не должен считаться единственной точкой истины для Matrix runtime в dev workflow.
- **D-02:** При старте бот должен пытаться восстановить минимально необходимое локальное состояние из уже существующих Matrix rooms / Space, а не требовать full reset.
- **D-03:** Reconciliation должен восстанавливать как минимум `matrix_user:*`, `matrix_room:*` и missing `chat:{user}:{chat_id}` записи, если серверные комнаты уже существуют.
- **D-04:** Reconciliation не должен создавать новые Space/rooms, если задача — именно восстановление локального state после рестарта.
### Dev restart behavior
- **D-05:** Обычный restart бота должен быть основным путём для разработки; удаление `lambda_matrix.db` и `matrix_store` не должно быть обязательным для проверки workflow.
- **D-06:** Если local state неполон, бот должен либо восстановить его, либо логировать понятную причину, а не падать на командах вроде `!rename`.
- **D-07:** Несогласованность между `room_meta` и `ChatManager` должна обнаруживаться и устраняться автоматически на startup или при первом обращении.
### Dev reset workflow
- **D-08:** Нужен отдельный dev-only reset tool/script для controlled QA, вместо ручного набора shell-команд.
- **D-09:** Reset workflow должен как минимум поддерживать `local-only` reset: удаление `lambda_matrix.db` и `matrix_store` с понятной инструкцией, что делать с server-side Matrix rooms.
- **D-10:** Если full server-side cleanup не автоматизируется в этой фазе, tool должен явно печатать, какие ручные шаги обязательны в Matrix client.
### The agent's Discretion
- Точное место вызова reconciliation в startup flow
- Внутренняя структура helper-модуля (`bootstrap.py`, `reconcile.py` или аналог)
- Формат dev reset script и уровень автоматизации server-side cleanup
- Детали debug-logging и dry-run режима, если они помогают без раздувания scope
</decisions>
<specifics>
## Specific Ideas
- Главный критерий: после обычного restart бот не должен ломаться только потому, что local DB была сброшена или частично потеряна.
- Reset workflow должен быть явным и repeatable, а не завязанным на память разработчика.
- Нужно различать две ситуации:
- broken because code is wrong
- broken because local dev state was deliberately reset and requires reconciliation
</specifics>
<canonical_refs>
## Canonical References
**Downstream agents MUST read these before planning or implementing.**
### Matrix phase artifacts
- `.planning/phases/01-matrix-qa-polish/01-CONTEXT.md` — locked Matrix decisions from Phase 1
- `.planning/phases/01-matrix-qa-polish/01-VERIFICATION.md` — what Phase 1 already validated and what manual QA still expects
- `.planning/phases/01-matrix-qa-polish/01-HUMAN-UAT.md` — remaining real-client Matrix checks
### Current Matrix runtime
- `adapter/matrix/bot.py` — startup flow, sync loop, runtime wiring, DB/store env vars
- `adapter/matrix/store.py` — persisted Matrix metadata and pending confirmation keys
- `adapter/matrix/room_router.py` — room_id to chat_id resolution and current unregistered fallback
- `adapter/matrix/handlers/auth.py` — invite bootstrap that creates Space and first chat room
- `core/chat.py``ChatManager` persistence contract that currently breaks when local state is missing
### Supporting docs
- `docs/matrix-prototype.md` — intended Matrix UX and architecture direction
- `README.md` — current run instructions and existing manual QA/reset habits
</canonical_refs>
<code_context>
## Existing Code Insights
### Reusable Assets
- `adapter/matrix/store.py` already persists room/user metadata and is the obvious place to anchor reconciliation inputs.
- `adapter/matrix/room_router.py` already detects unknown rooms via `unregistered:{room_id}` fallback; this is a useful reconciliation trigger point.
- `core/chat.py` already has `get_or_create`, `rename`, `archive`, `list_active`; missing chat records can be rebuilt through this API instead of inventing a parallel format.
### Established Patterns
- Matrix runtime uses `SQLiteStore` for adapter-local metadata and `matrix-nio` room callbacks for transport events.
- Phase 1 already moved Matrix to Space+rooms and command-only confirmations, so this phase must preserve that model rather than reverting to DM-first simplifications.
### Integration Points
- Startup path in `adapter/matrix/bot.py:main()` is the natural place to run reconciliation before `sync_forever`.
- Invite/bootstrap path in `adapter/matrix/handlers/auth.py` is the existing source of truth for what metadata a healthy first room should have.
- `ChatManager` records and `matrix_room:*` metadata must stay consistent enough that commands like `!rename`, `!archive`, and `!chats` work after restart.
</code_context>
<deferred>
## Deferred Ideas
- Full production-grade migration of historical Matrix state across schema versions
- Automatic server-side deletion/leave for all Matrix rooms and Space during reset, if it requires broader admin semantics
- Any Phase 2 SDK integration work
</deferred>
---
*Phase: 01.1-matrix-restart-reconciliation-and-dev-reset-workflow*
*Context gathered: 2026-04-03*

View file

@ -1,350 +0,0 @@
# Phase 01.1: Matrix restart reconciliation and dev reset workflow - Research
**Researched:** 2026-04-03
**Domain:** Matrix adapter restart reconciliation, local state recovery, dev reset workflow
**Confidence:** HIGH
<user_constraints>
## User Constraints (from CONTEXT.md)
### Locked Decisions
- **D-01:** Локальный SQLite store больше не должен считаться единственной точкой истины для Matrix runtime в dev workflow.
- **D-02:** При старте бот должен пытаться восстановить минимально необходимое локальное состояние из уже существующих Matrix rooms / Space, а не требовать full reset.
- **D-03:** Reconciliation должен восстанавливать как минимум `matrix_user:*`, `matrix_room:*` и missing `chat:{user}:{chat_id}` записи, если серверные комнаты уже существуют.
- **D-04:** Reconciliation не должен создавать новые Space/rooms, если задача — именно восстановление локального state после рестарта.
- **D-05:** Обычный restart бота должен быть основным путём для разработки; удаление `lambda_matrix.db` и `matrix_store` не должно быть обязательным для проверки workflow.
- **D-06:** Если local state неполон, бот должен либо восстановить его, либо логировать понятную причину, а не падать на командах вроде `!rename`.
- **D-07:** Несогласованность между `room_meta` и `ChatManager` должна обнаруживаться и устраняться автоматически на startup или при первом обращении.
- **D-08:** Нужен отдельный dev-only reset tool/script для controlled QA, вместо ручного набора shell-команд.
- **D-09:** Reset workflow должен как минимум поддерживать `local-only` reset: удаление `lambda_matrix.db` и `matrix_store` с понятной инструкцией, что делать с server-side Matrix rooms.
- **D-10:** Если full server-side cleanup не автоматизируется в этой фазе, tool должен явно печатать, какие ручные шаги обязательны в Matrix client.
### Claude's Discretion
- Точное место вызова reconciliation в startup flow
- Внутренняя структура helper-модуля (`bootstrap.py`, `reconcile.py` или аналог)
- Формат dev reset script и уровень автоматизации server-side cleanup
- Детали debug-logging и dry-run режима, если они помогают без раздувания scope
### Deferred Ideas (OUT OF SCOPE)
- Full production-grade migration of historical Matrix state across schema versions
- Automatic server-side deletion/leave for all Matrix rooms and Space during reset, if it requires broader admin semantics
- Any Phase 2 SDK integration work
</user_constraints>
## Summary
Phase 01.1 should be planned as a bootstrap/recovery phase, not as another chat-feature phase. The current Matrix adapter has no startup reconciliation path: `adapter/matrix/bot.py` logs in and goes directly to `sync_forever()`, while routing and command handlers assume `matrix_room:*`, `matrix_user:*`, and `chat:*` keys already exist. That means local DB loss currently produces logical corruption, not just missing cache.
The safe standard approach is: perform a first sync that hydrates joined-room state, inspect the bot's current joined rooms and room state from the homeserver, rebuild the minimal local metadata needed for command routing, and only then enter the long-running sync loop. Reconciliation should be non-destructive and idempotent: if local keys already exist and match server state, leave them alone; if they are missing, recreate them; if they conflict, prefer the server room topology for Matrix-specific metadata and recreate missing `ChatManager` rows from that.
For reset, separate two workflows explicitly. `local-only` reset is the default and should be automated. Optional server-side cleanup may leave/forget rooms for the bot account, but it cannot promise global deletion of Matrix rooms for all members; if that is not automated, the tool must print the exact manual steps for the Matrix client.
**Primary recommendation:** Add a startup `reconcile_matrix_state()` step before `sync_forever()`, and ship a dev-only reset CLI with `local-only`, `server-leave-forget`, and `dry-run` modes.
## Project Constraints (from CLAUDE.md)
- Do not treat missing Lambda SDK as a blocker.
- Keep all platform calls behind `platform/interface.py`.
- Current runtime implementation is `platform/mock.py`; recommendations must work with that.
- Prefer architecture changes in adapters and core without coupling to future SDK internals.
- Use pytest-based verification.
- Do not recommend committing `.env`.
- Respect dependency order: `core/` first, then `platform/`, then adapters.
## Standard Stack
### Core
| Library | Version | Purpose | Why Standard |
|---------|---------|---------|--------------|
| Python | 3.14.3 installed | Runtime for bot and scripts | Already available locally; codebase targets `>=3.11`. |
| `matrix-nio` | 0.25.2, published 2024-10-04 | Matrix client, sync, room membership/state APIs | Already installed; exposes the exact bootstrap/reset APIs this phase needs. |
| `SQLiteStore` (repo) | local | Adapter/core KV persistence | Existing persistence contract for `matrix_user:*`, `matrix_room:*`, and `chat:*`. |
| Matrix Client-Server API | spec latest | Authoritative room membership/state semantics | Needed to reason about restart recovery and leave/forget behavior correctly. |
### Supporting
| Library | Version | Purpose | When to Use |
|---------|---------|---------|-------------|
| `pytest` | 9.0.2, published 2025-12-06 | Test runner | For targeted adapter/bootstrap regression tests. |
| `pytest-asyncio` | 1.3.0, published 2025-11-10 | Async test execution | For async reconciliation/reset flows. |
| `structlog` | 25.5.0, published 2025-10-27 | Diagnostics | For reconciliation summaries and conflict logging. |
| `python-dotenv` | 1.2.2, published 2026-03-01 | Env loading | Already used by `adapter/matrix/bot.py` for Matrix config. |
### Alternatives Considered
| Instead of | Could Use | Tradeoff |
|------------|-----------|----------|
| Startup reconciliation from joined rooms + state | Force developers to wipe local DB and recreate rooms | Simpler code, but directly violates D-01, D-02, D-05. |
| Non-destructive local rebuild | Full auto-recreate of Space/rooms on missing local state | Easier to implement, but causes duplicate Matrix rooms and breaks D-04. |
| Dev reset script | README-only manual ritual | Lower code cost, but not repeatable and fails D-08..D-10. |
**Installation:**
```bash
uv sync
```
**Version verification:** Verified via installed environment and PyPI metadata on 2026-04-03:
- `matrix-nio` `0.25.2` - 2024-10-04
- `pytest` `9.0.2` - 2025-12-06
- `pytest-asyncio` `1.3.0` - 2025-11-10
- `structlog` `25.5.0` - 2025-10-27
- `python-dotenv` `1.2.2` - 2026-03-01
## Architecture Patterns
### Recommended Project Structure
```text
adapter/matrix/
├── bot.py # startup flow calls reconciliation before sync loop
├── reconcile.py # bootstrap/rebuild logic from Matrix server state
├── reset.py # dev-only reset CLI / entrypoint
├── room_router.py # room_id -> chat_id with recovery hook
├── store.py # metadata helpers, prefix scans, derived counters
└── handlers/
├── auth.py # first-time provisioning only
└── chat.py # uses recovered state, no provisioning fallback
```
### Pattern 1: Two-Phase Startup Bootstrap
**What:** Split startup into `login -> initial sync/full_state -> reconcile -> steady-state sync_forever`.
**When to use:** Always for Matrix bot startup when local DB may be missing or stale.
**Example:**
```python
# Source: matrix-nio AsyncClient docs/source + repo startup flow
client = AsyncClient(...)
runtime = build_runtime(store=SQLiteStore(db_path), client=client)
await login_or_restore_session(client)
await client.sync(timeout=0, full_state=True)
report = await reconcile_matrix_state(client, runtime.store, runtime.chat_mgr)
logger.info("matrix_reconcile_complete", **report)
await client.sync_forever(timeout=30000)
```
### Pattern 2: Rebuild Local Metadata From Joined Rooms
**What:** Enumerate joined rooms, inspect local hydrated room objects or room state, and recreate missing `matrix_room:*`, `matrix_user:*`, and `chat:*` records.
**When to use:** On startup and optionally on `unregistered:{room_id}` fallback at runtime.
**Example:**
```python
# Source: matrix-nio AsyncClient.joined_rooms/room_get_state + repo store contracts
joined = await client.joined_rooms()
for room_id in joined.rooms:
state = await client.room_get_state(room_id)
# detect: space room vs chat room, owner user, child relationship, display name
# rebuild matrix_room:{room_id}
# rebuild chat:{matrix_user_id}:{chat_id} if absent
```
### Pattern 3: Non-Destructive Reconciliation Report
**What:** Return a structured report: scanned rooms, restored rooms, restored chats, conflicts, skipped rooms.
**When to use:** Every reconciliation run, including dry-run.
**Example:**
```python
{
"joined_rooms": 4,
"restored_user_meta": 1,
"restored_room_meta": 3,
"restored_chat_rows": 3,
"conflicts": [],
"skipped_rooms": ["!dm:example.org"],
}
```
### Pattern 4: Reset Modes Are Explicit
**What:** Separate `local-only`, `server-leave-forget`, and `dry-run`.
**When to use:** For dev/QA only. Never mix destructive server cleanup into normal startup.
**Example:**
```bash
uv run python -m adapter.matrix.reset --mode local-only
uv run python -m adapter.matrix.reset --mode server-leave-forget --dry-run
```
### Anti-Patterns to Avoid
- **Provisioning during reconciliation:** Do not create a new Space or new rooms while trying to recover missing local state.
- **Treating `next_chat_index` as primary truth:** Derive it from recovered `chat_id` values after scan; do not trust a missing or stale counter.
- **Routing unknown rooms straight through:** `unregistered:{room_id}` is a signal to reconcile, not a stable runtime identity.
- **Destructive reset by default:** Startup must never leave/forget rooms automatically.
- **Blindly trusting local `surface_ref`:** If `chat:*` and `matrix_room:*` disagree, rebuild from Matrix room metadata and repair the chat row.
## Don't Hand-Roll
| Problem | Don't Build | Use Instead | Why |
|---------|-------------|-------------|-----|
| Room discovery | Custom DB-only reconstruction heuristics | `AsyncClient.joined_rooms()` plus synced room state | Server already knows which rooms the bot joined. |
| Space membership detection | Naming-convention parsing of room names | Matrix state: `m.room.create.type`, `m.space.child`, `m.space.parent` | Names are mutable and non-authoritative. |
| Room cleanup semantics | Custom “delete room” assumptions | `room_leave()` + `room_forget()` semantics | Client API supports leave/forget, not guaranteed global deletion. |
| Chat ID recovery | Hardcoded `C1/C2/...` reset | Rebuild from existing `matrix_room:*`/server state and compute next index | Prevents collisions after partial DB loss. |
| Diagnostic output | Ad hoc `print()` strings | Structured reconciliation/reset report via `structlog` | Easier manual QA and failure triage. |
**Key insight:** The homeserver already persists the bots room graph. This phase should rehydrate local cache from that graph, not attempt to replace it with a second custom truth model.
## Common Pitfalls
### Pitfall 1: Joining the sync loop before reconciliation
**What goes wrong:** Commands arrive while local metadata is still missing, producing `unregistered:{room_id}` routing or `ChatManager` misses.
**Why it happens:** Current `main()` enters `sync_forever()` immediately after login.
**How to avoid:** Perform initial sync and reconciliation first.
**Warning signs:** `unregistered_room` logs immediately after restart; `ValueError("Chat ... not found")` on `!rename` or `!archive`.
### Pitfall 2: Recovering room metadata but not chat rows
**What goes wrong:** Room routing works, but `ChatManager.rename/archive/list_active` still fails because `chat:{user}:{chat_id}` rows were not recreated.
**Why it happens:** Matrix adapter metadata and core chat metadata live in different keyspaces.
**How to avoid:** Reconciliation must repair both stores in one pass.
**Warning signs:** `matrix_room:*` exists but `chat:*` keys do not.
### Pitfall 3: Trusting stale `next_chat_index`
**What goes wrong:** New chats reuse existing `C` IDs after local recovery.
**Why it happens:** `next_chat_id()` increments a persisted counter that may be absent or behind.
**How to avoid:** After scan, set `next_chat_index = max(recovered_chat_numbers) + 1`.
**Warning signs:** New room gets `C1` even though Space already contains prior rooms.
### Pitfall 4: Assuming room names identify chat rooms safely
**What goes wrong:** Reconciliation binds the wrong room because a user renamed a room or Space.
**Why it happens:** Names are user-facing labels, not stable identifiers.
**How to avoid:** Prefer room state and existing `chat_id` metadata; use display names only as fallback.
**Warning signs:** Duplicate “Чат 1” names or renamed rooms break matching.
### Pitfall 5: Over-promising full cleanup
**What goes wrong:** Reset script claims a “clean slate” but rooms still exist in Element or for other members.
**Why it happens:** Leaving/forgetting affects the bot accounts membership/history, not necessarily global room deletion.
**How to avoid:** Name the mode accurately and print the manual client steps when needed.
**Warning signs:** QA reruns still show old rooms in the users client.
## Code Examples
Verified patterns from official sources and the installed library surface:
### Initial Sync Before Reconcile
```python
# Source: matrix-nio AsyncClient.sync/sync_forever
await client.sync(timeout=0, full_state=True)
report = await reconcile_matrix_state(client, store, chat_mgr)
await client.sync_forever(timeout=30000)
```
### Space Child Link Creation
```python
# Source: Matrix client-server API state event + current auth/new-chat flow
await client.room_put_state(
room_id=space_id,
event_type="m.space.child",
content={"via": [homeserver]},
state_key=chat_room_id,
)
```
### Bot-Side Leave/Forget Cleanup
```python
# Source: matrix-nio AsyncClient.room_leave / room_forget
for room_id in room_ids:
await client.room_leave(room_id)
await client.room_forget(room_id)
```
### Router Recovery Trigger
```python
# Source: repo room_router contract
chat_id = await resolve_chat_id(store, room_id, matrix_user_id)
if chat_id.startswith("unregistered:"):
await reconcile_single_room(client, store, chat_mgr, room_id, matrix_user_id)
```
## State of the Art
| Old Approach | Current Approach | When Changed | Impact |
|--------------|------------------|--------------|--------|
| Local adapter DB treated as the operational truth | Rebuildable local cache from server room graph | Mature Matrix client practice; supported by current Matrix CS API and `matrix-nio` | Restart no longer requires destructive local reset. |
| Manual room cleanup in client after experiments | Scripted leave/forget plus explicit manual instructions | Current `matrix-nio` 0.25.x API surface | QA becomes repeatable and auditable. |
| Immediate steady-state sync after login | Initial sync/full-state bootstrap before long polling | Supported by current `AsyncClient.sync()` / `sync_forever()` behavior | Reconciliation can run before any user traffic is handled. |
**Deprecated/outdated:**
- `README.md` Matrix manual QA instruction `rm -f lambda_matrix.db` as the primary restart flow: outdated for this phase.
- DM-first Matrix recovery assumptions in `docs/matrix-prototype.md`: outdated relative to Phase 1 Space+rooms decisions.
## Open Questions
1. **How exactly should reconciliation identify the owning Matrix user for a recovered room when local `matrix_room:*` is gone?**
- What we know: the bot can enumerate joined rooms and fetch room state; current healthy metadata stores `matrix_user_id` and `space_id`.
- What's unclear: whether Phase 1-created rooms also expose enough server-side structure to recover owner deterministically without existing local metadata in every case.
- Recommendation: Plan a proof test against a real homeserver/client. If room-state-only ownership is ambiguous, persist a tiny bot-authored marker state event going forward, but keep that addition narrowly scoped.
2. **Should runtime recovery happen only on startup, or also lazily on first unknown room access?**
- What we know: startup repair satisfies D-02/D-07 for common restart loss; `room_router` already surfaces unknown rooms cleanly.
- What's unclear: whether partial DB corruption during runtime is common enough to justify lazy single-room repair in Phase 01.1.
- Recommendation: Make startup reconciliation required, lazy room repair optional if it stays small.
3. **How much of server cleanup should Phase 01.1 automate?**
- What we know: `room_leave()` and `room_forget()` are available; global room deletion is not what the client API guarantees.
- What's unclear: whether automating bot-side leave/forget is worth the extra risk for this urgent phase.
- Recommendation: Keep `local-only` mandatory. Make server cleanup optional and clearly labeled experimental/dev-only if included.
## Environment Availability
| Dependency | Required By | Available | Version | Fallback |
|------------|------------|-----------|---------|----------|
| Python | Runtime, scripts, tests | ✓ | 3.14.3 | — |
| `uv` | Standard install/run workflow | ✓ | 0.9.30 | `python -m` + existing venv |
| `pytest` | Automated verification | ✓ | 9.0.2 | `uv run pytest` |
| Matrix homeserver credentials | Real restart/reset manual QA | ✗ in current shell | — | Manual-only after `.env` is configured |
| Matrix bot local DB/store paths | Reset workflow | ✓ | defaults in code | Can override with `MATRIX_DB_PATH` / `MATRIX_STORE_PATH` |
**Missing dependencies with no fallback:**
- Live Matrix credentials for real manual reconciliation/reset QA.
**Missing dependencies with fallback:**
- None for repository-only implementation and tests.
## Validation Architecture
### Test Framework
| Property | Value |
|----------|-------|
| Framework | `pytest 9.0.2` + `pytest-asyncio 1.3.0` |
| Config file | `pyproject.toml` |
| Quick run command | `pytest tests/adapter/matrix -v` |
| Full suite command | `pytest tests/ -v` |
### Phase Requirements → Test Map
| Req ID | Behavior | Test Type | Automated Command | File Exists? |
|--------|----------|-----------|-------------------|-------------|
| PH01.1-BOOT | Startup rebuilds missing `matrix_user:*`, `matrix_room:*`, and `chat:*` from existing rooms without creating new rooms | unit/integration | `pytest tests/adapter/matrix/test_reconcile.py -v` | ❌ Wave 0 |
| PH01.1-ROUTER | Unknown room fallback can trigger repair or yields diagnosable warning without crashing commands | unit | `pytest tests/adapter/matrix/test_room_router_reconcile.py -v` | ❌ Wave 0 |
| PH01.1-COUNTER | Reconciliation resets `next_chat_index` to recovered max + 1 | unit | `pytest tests/adapter/matrix/test_reconcile.py -k next_chat_index -v` | ❌ Wave 0 |
| PH01.1-RESET | Dev reset `local-only` removes local DB/store paths and prints next steps | unit/smoke | `pytest tests/adapter/matrix/test_reset.py -v` | ❌ Wave 0 |
| PH01.1-NONDESTRUCTIVE | Reconciliation never calls room creation APIs | unit | `pytest tests/adapter/matrix/test_reconcile.py -k no_create -v` | ❌ Wave 0 |
### Sampling Rate
- **Per task commit:** `pytest tests/adapter/matrix -v`
- **Per wave merge:** `pytest tests/ -v`
- **Phase gate:** Full suite green before `/gsd:verify-work`
### Wave 0 Gaps
- [ ] `tests/adapter/matrix/test_reconcile.py` - startup reconciliation scenarios
- [ ] `tests/adapter/matrix/test_reset.py` - CLI/script reset modes and output
- [ ] `tests/adapter/matrix/test_room_router_reconcile.py` - lazy recovery or warning behavior
- [ ] Integration fixture for a fake `AsyncClient` response surface matching `joined_rooms()` and `room_get_state()`
## Sources
### Primary (HIGH confidence)
- Matrix Client-Server API - room state, leave, forget, joined rooms, Spaces semantics: https://spec.matrix.org/latest/client-server-api/index.html
- `matrix-nio` installed 0.25.2 API surface verified locally on 2026-04-03 via `AsyncClient.sync`, `sync_forever`, `joined_rooms`, `room_get_state`, `room_leave`, `room_forget`
- Repo code: [adapter/matrix/bot.py](/Users/a/MAI/sem2/lambda/surfaces-bot/adapter/matrix/bot.py), [adapter/matrix/store.py](/Users/a/MAI/sem2/lambda/surfaces-bot/adapter/matrix/store.py), [adapter/matrix/room_router.py](/Users/a/MAI/sem2/lambda/surfaces-bot/adapter/matrix/room_router.py), [adapter/matrix/handlers/auth.py](/Users/a/MAI/sem2/lambda/surfaces-bot/adapter/matrix/handlers/auth.py), [core/chat.py](/Users/a/MAI/sem2/lambda/surfaces-bot/core/chat.py)
- PyPI release metadata: https://pypi.org/project/matrix-nio/ , https://pypi.org/project/pytest/ , https://pypi.org/project/pytest-asyncio/ , https://pypi.org/project/structlog/ , https://pypi.org/project/python-dotenv/
### Secondary (MEDIUM confidence)
- [README.md](/Users/a/MAI/sem2/lambda/surfaces-bot/README.md) - current manual reset habit and run commands
- [docs/matrix-prototype.md](/Users/a/MAI/sem2/lambda/surfaces-bot/docs/matrix-prototype.md) - original Matrix UX intent, noting outdated DM/reaction sections
- [01-CONTEXT.md](/Users/a/MAI/sem2/lambda/surfaces-bot/.planning/phases/01-matrix-qa-polish/01-CONTEXT.md) - locked Phase 1 Matrix decisions
- [01-VERIFICATION.md](/Users/a/MAI/sem2/lambda/surfaces-bot/.planning/phases/01-matrix-qa-polish/01-VERIFICATION.md) - what has already been verified and what still needs human Matrix QA
### Tertiary (LOW confidence)
- None
## Metadata
**Confidence breakdown:**
- Standard stack: HIGH - verified against installed environment, PyPI metadata, and official Matrix spec
- Architecture: HIGH - directly grounded in current repo flow plus current `matrix-nio`/Matrix capabilities
- Pitfalls: HIGH - derived from concrete gaps in current startup/store/router code
**Research date:** 2026-04-03
**Valid until:** 2026-05-03

View file

@ -1,80 +0,0 @@
---
phase: 01.1
slug: matrix-restart-reconciliation-and-dev-reset-workflow
status: draft
nyquist_compliant: false
wave_0_complete: false
created: 2026-04-03
---
# Phase 01.1 — Validation Strategy
> Per-phase validation contract for feedback sampling during execution.
---
## Test Infrastructure
| Property | Value |
|----------|-------|
| **Framework** | `pytest 9.0.2` + `pytest-asyncio 1.3.0` |
| **Config file** | `pyproject.toml` |
| **Quick run command** | `pytest tests/adapter/matrix -v` |
| **Full suite command** | `pytest tests/ -v` |
| **Estimated runtime** | ~20 seconds |
---
## Sampling Rate
- **After every task commit:** Run `pytest tests/adapter/matrix -v`
- **After every plan wave:** Run `pytest tests/ -v`
- **Before `$gsd-verify-work`:** Full suite must be green
- **Max feedback latency:** 20 seconds
---
## Per-Task Verification Map
| Task ID | Plan | Wave | Requirement | Test Type | Automated Command | File Exists | Status |
|---------|------|------|-------------|-----------|-------------------|-------------|--------|
| 01.1-01-01 | 01 | 1 | PH01.1-BOOT | unit/integration | `pytest tests/adapter/matrix/test_reconcile.py -v` | ❌ W0 | ⬜ pending |
| 01.1-01-01 | 01 | 1 | PH01.1-COUNTER | unit | `pytest tests/adapter/matrix/test_reconcile.py -k next_chat_index -v` | ❌ W0 | ⬜ pending |
| 01.1-01-01 | 01 | 1 | PH01.1-NONDESTRUCTIVE | unit | `pytest tests/adapter/matrix/test_reconcile.py -k no_create -v` | ❌ W0 | ⬜ pending |
| 01.1-02-01 | 02 | 2 | PH01.1-BOOT | unit | `pytest tests/adapter/matrix/test_dispatcher.py -k startup -v` | ✅ | ⬜ pending |
| 01.1-02-02 | 02 | 2 | PH01.1-ROUTER | unit | `pytest tests/adapter/matrix/test_dispatcher.py -k reconcile -v` | ✅ | ⬜ pending |
| 01.1-03-01 | 03 | 1 | PH01.1-RESET | unit/smoke | `pytest tests/adapter/matrix/test_reset.py -v` | ❌ W0 | ⬜ pending |
| 01.1-03-02 | 03 | 1 | PH01.1-RESET | smoke | `python -m adapter.matrix.reset --help` | ❌ W0 | ⬜ pending |
*Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky*
---
## Wave 0 Requirements
- [ ] `tests/adapter/matrix/test_reconcile.py` — startup reconciliation scenarios, `next_chat_index`, and no-provisioning assertions
- [ ] `tests/adapter/matrix/test_reset.py` — CLI reset modes, dry-run behavior, and operator guidance output
- [ ] `tests/adapter/matrix/test_dispatcher.py` — startup bootstrap order and targeted unknown-room recovery coverage
- [ ] Fake `AsyncClient` fixture surface for joined rooms, room state, leave, and forget behavior
---
## Manual-Only Verifications
| Behavior | Requirement | Why Manual | Test Instructions |
|----------|-------------|------------|-------------------|
| Reconciled Space/chat rooms render correctly in a real Matrix client after restart | PH01.1-BOOT | Client UX and homeserver state cannot be fully trusted from fake nio fixtures | 1. Start the bot with existing Space/chat rooms. 2. Verify the bot does not create duplicate Space or chat rooms. 3. Send a command in a recovered room and confirm it routes normally. |
| Server-side cleanup leaves the account in a usable Element state after `server-leave-forget` | PH01.1-RESET | Element/archive behavior and homeserver retention are client/server integration concerns | 1. Run `python -m adapter.matrix.reset --mode server-leave-forget --dry-run`. 2. Run without `--dry-run` on a test account. 3. Confirm joined rooms disappear for the bot and fresh invites can be accepted cleanly. |
---
## Validation Sign-Off
- [ ] All tasks have `<automated>` verify or Wave 0 dependencies
- [ ] Sampling continuity: no 3 consecutive tasks without automated verify
- [ ] Wave 0 covers all MISSING references
- [ ] No watch-mode flags
- [ ] Feedback latency < 20s
- [ ] `nyquist_compliant: true` set in frontmatter
**Approval:** pending

View file

@ -0,0 +1,626 @@
---
phase: 04-matrix-mvp-shared-agent-context-and-context-management-comma
plan: 01
type: execute
wave: 1
depends_on: []
files_modified:
- sdk/agent_api_wrapper.py
- sdk/agent_session.py
- sdk/real.py
- adapter/matrix/bot.py
- tests/platform/test_agent_session.py
- tests/platform/test_real.py
- tests/adapter/matrix/test_dispatcher.py
autonomous: true
requirements:
- Replace AgentSessionClient with AgentApi
- Wire AgentApi lifecycle into MatrixBot
must_haves:
truths:
- "RealPlatformClient uses AgentApiWrapper (wraps AgentApi), not AgentSessionClient"
- "AgentApiWrapper is connected before sync_forever and closed in finally block of main()"
- "build_thread_key and AgentSessionClient are gone from sdk/"
- "stream_message() yields MessageChunk objects including a final chunk with tokens_used from last_tokens_used"
- "AGENT_WS_URL is used unchanged (no thread_id query param)"
- "MATRIX_PLATFORM_BACKEND=real still works end-to-end without test crash"
- "All existing tests pass after the swap"
artifacts:
- path: "sdk/agent_api_wrapper.py"
provides: "AgentApiWrapper subclass of AgentApi with last_tokens_used tracking"
contains: "AgentApiWrapper"
- path: "sdk/real.py"
provides: "RealPlatformClient wrapping AgentApiWrapper"
contains: "AgentApiWrapper"
- path: "adapter/matrix/bot.py"
provides: "main() awaits agent_api.connect() and agent_api.close()"
contains: "agent_api.connect"
- path: "tests/platform/test_real.py"
provides: "Updated tests using FakeAgentApi instead of FakeAgentSessionClient"
key_links:
- from: "adapter/matrix/bot.py main()"
to: "RealPlatformClient._agent_api"
via: "runtime.platform.agent_api property"
pattern: "agent_api\\.connect"
- from: "sdk/real.py stream_message()"
to: "agent_api.last_tokens_used"
via: "attribute read after async-for loop"
pattern: "last_tokens_used"
---
<objective>
Replace the custom per-request AgentSessionClient with a thin AgentApiWrapper that
subclasses AgentApi from lambda_agent_api and adds last_tokens_used tracking. Remove
build_thread_key and AgentSessionClient entirely. Wire AgentApiWrapper connect/close
into bot.py main(). Update all tests that referenced the old client.
Do NOT modify any file under external/. The external/ directory is managed by the
platform team. All customisation goes in sdk/agent_api_wrapper.py.
Purpose: The existing AgentSessionClient creates a new WebSocket per message and
injects thread_id into the URL — both incompatible with origin/main platform-agent.
AgentApi maintains a single persistent WS connection managed via connect()/close()
and exposes send_message() as an AsyncIterator. We capture tokens_used in a thin
subclass so sdk/real.py can include it in the final MessageChunk without touching
the upstream library.
Output: sdk/agent_api_wrapper.py (new), sdk/real.py (rewritten), sdk/agent_session.py
(stubbed), adapter/matrix/bot.py updated, tests green.
</objective>
<execution_context>
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
@$HOME/.claude/get-shit-done/templates/summary.md
</execution_context>
<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/phases/04-matrix-mvp-shared-agent-context-and-context-management-comma/04-CONTEXT.md
@.planning/phases/04-matrix-mvp-shared-agent-context-and-context-management-comma/04-RESEARCH.md
</context>
<interfaces>
<!-- Key types the executor needs. Read from source before touching anything. -->
<!-- IMPORTANT: external/ files are READ-ONLY — do not modify them. -->
From external/platform-agent_api/lambda_agent_api/agent_api.py (READ ONLY):
```python
class AgentApi:
def __init__(self, agent_id: str, url: str,
callback=None, on_disconnect=None): ...
async def connect(self) -> None: ... # opens WS, awaits MsgStatus, starts _listen task
async def close(self) -> None: ... # cancels _listen, closes WS+session
async def send_message(self, text: str) -> AsyncIterator[AgentEventUnion]:
# yields MsgEventTextChunk only; breaks on MsgEventEnd (does NOT yield it)
# MsgEventEnd.tokens_used is consumed internally at the break point
...
async def _listen(self) -> None:
# internal task: receives WS frames, puts AgentEventUnion into self._queue
# on MsgEventEnd: puts it in queue then breaks
...
# AgentEventUnion = Union[MsgEventTextChunk, MsgEventEnd] per server.py
```
From external/platform-agent_api/lambda_agent_api/server.py (READ ONLY):
```python
class MsgEventTextChunk(BaseModel):
type: Literal[EServerMessage.AGENT_EVENT_TEXT_CHUNK]
text: str
class MsgEventEnd(BaseModel):
type: Literal[EServerMessage.AGENT_EVENT_END]
tokens_used: int
```
New file to create — sdk/agent_api_wrapper.py:
```python
class AgentApiWrapper(AgentApi):
"""Thin subclass of AgentApi that captures tokens_used from MsgEventEnd.
AgentApi.send_message() yields only MsgEventTextChunk and breaks silently
on MsgEventEnd without storing tokens_used. This wrapper overrides _listen()
to intercept MsgEventEnd and store tokens_used before it is discarded.
"""
last_tokens_used: int = 0
async def _listen(self) -> None:
# Override: same as parent, but capture MsgEventEnd.tokens_used
...
```
From sdk/interface.py (unchanged):
```python
class MessageChunk(BaseModel):
message_id: str
delta: str
finished: bool
tokens_used: int = 0
class PlatformClient(Protocol):
async def send_message(self, user_id, chat_id, text, attachments=None) -> MessageResponse: ...
async def stream_message(self, user_id, chat_id, text, attachments=None) -> AsyncIterator[MessageChunk]: ...
```
</interfaces>
<tasks>
<task type="auto" tdd="true">
<name>Task 1: Create sdk/agent_api_wrapper.py; rewrite sdk/real.py; stub sdk/agent_session.py</name>
<read_first>
- sdk/real.py (full file — being replaced)
- sdk/agent_session.py (full file — being stubbed)
- external/platform-agent_api/lambda_agent_api/agent_api.py (full file — READ ONLY, inspect _listen and send_message to understand override point)
- external/platform-agent_api/lambda_agent_api/server.py (full file — READ ONLY, MsgEventEnd.tokens_used)
- sdk/interface.py (MessageChunk, PlatformClient Protocol)
</read_first>
<files>sdk/agent_api_wrapper.py, sdk/real.py, sdk/agent_session.py</files>
<behavior>
- Create sdk/agent_api_wrapper.py with class AgentApiWrapper(AgentApi):
- __init__: calls super().__init__(...) and adds self.last_tokens_used: int = 0
- Override _listen(): copy the parent _listen() logic verbatim, then at the point where MsgEventEnd is received (before or as it is put into the queue), set self.last_tokens_used = chunk.tokens_used
- Do NOT modify agent_api.py in external/ — subclass only
- RealPlatformClient.__init__ accepts agent_api: AgentApiWrapper, prototype_state: PrototypeStateStore, platform: str = "matrix"
- RealPlatformClient exposes agent_api as property self.agent_api so bot.py main() can call connect/close
- stream_message() iterates agent_api.send_message(text) yielding MessageChunk per MsgEventTextChunk chunk; after loop yields final MessageChunk(finished=True, delta="", tokens_used=agent_api.last_tokens_used)
- send_message() collects all chunks from stream_message() and returns MessageResponse
- No thread_key, no build_thread_key references anywhere in sdk/real.py
- sdk/agent_session.py: replace file contents with single comment stub (keep file to avoid import errors until tests are updated in Task 2)
</behavior>
<action>
1. Read external/platform-agent_api/lambda_agent_api/agent_api.py fully to find the exact _listen() implementation and the line where MsgEventEnd is handled.
2. Create sdk/agent_api_wrapper.py:
```python
from __future__ import annotations
import sys
from pathlib import Path
# Ensure lambda_agent_api is importable (same sys.path trick as bot.py)
_api_root = Path(__file__).resolve().parents[1] / "external" / "platform-agent_api"
if str(_api_root) not in sys.path:
sys.path.insert(0, str(_api_root))
from lambda_agent_api.agent_api import AgentApi
from lambda_agent_api.server import MsgEventEnd
class AgentApiWrapper(AgentApi):
"""Thin subclass of AgentApi that captures tokens_used from MsgEventEnd.
AgentApi.send_message() yields MsgEventTextChunk events and breaks on
MsgEventEnd without storing tokens_used. This wrapper overrides _listen()
to intercept MsgEventEnd and set self.last_tokens_used before the event
is discarded, so RealPlatformClient can include it in the final MessageChunk.
Do NOT modify external/platform-agent_api — subclass only.
"""
def __init__(self, agent_id: str, url: str, **kwargs) -> None:
super().__init__(agent_id=agent_id, url=url, **kwargs)
self.last_tokens_used: int = 0
async def _listen(self) -> None:
# Copy parent _listen() logic.
# Read external/platform-agent_api/lambda_agent_api/agent_api.py _listen()
# and reproduce it here, adding:
# if isinstance(event, MsgEventEnd):
# self.last_tokens_used = event.tokens_used
# at the point where MsgEventEnd is processed.
#
# IMPORTANT: after reading agent_api.py, replace this entire method body
# with the exact parent implementation + the tokens_used capture line.
# Do not call super()._listen() — the parent creates a task; we need the
# override to run in the same task context.
raise NotImplementedError(
"Executor: replace this body with the copied _listen() from AgentApi "
"plus `self.last_tokens_used = event.tokens_used` at the MsgEventEnd branch."
)
```
IMPORTANT NOTE FOR EXECUTOR: The `_listen()` body above is a placeholder.
After reading agent_api.py, copy the actual _listen() implementation from AgentApi
into AgentApiWrapper._listen() and insert `self.last_tokens_used = event.tokens_used`
at the MsgEventEnd branch. The final file must NOT contain the NotImplementedError.
3. Rewrite sdk/real.py entirely:
```python
from __future__ import annotations
from typing import TYPE_CHECKING, AsyncIterator
from sdk.interface import Attachment, MessageChunk, MessageResponse, PlatformClient, User, UserSettings
from sdk.prototype_state import PrototypeStateStore
if TYPE_CHECKING:
from sdk.agent_api_wrapper import AgentApiWrapper
class RealPlatformClient(PlatformClient):
def __init__(
self,
agent_api: "AgentApiWrapper",
prototype_state: PrototypeStateStore,
platform: str = "matrix",
) -> None:
self._agent_api = agent_api
self._prototype_state = prototype_state
self._platform = platform
@property
def agent_api(self) -> "AgentApiWrapper":
return self._agent_api
async def get_or_create_user(
self,
external_id: str,
platform: str,
display_name: str | None = None,
) -> User:
return await self._prototype_state.get_or_create_user(
external_id=external_id,
platform=platform,
display_name=display_name,
)
async def send_message(
self,
user_id: str,
chat_id: str,
text: str,
attachments: list[Attachment] | None = None,
) -> MessageResponse:
parts: list[str] = []
tokens_used = 0
async for chunk in self.stream_message(user_id, chat_id, text, attachments):
if chunk.delta:
parts.append(chunk.delta)
if chunk.finished:
tokens_used = chunk.tokens_used
return MessageResponse(
message_id=user_id,
response="".join(parts),
tokens_used=tokens_used,
finished=True,
)
async def stream_message(
self,
user_id: str,
chat_id: str,
text: str,
attachments: list[Attachment] | None = None,
) -> AsyncIterator[MessageChunk]:
from lambda_agent_api.server import MsgEventTextChunk
async for event in self._agent_api.send_message(text):
if isinstance(event, MsgEventTextChunk):
yield MessageChunk(
message_id=user_id,
delta=event.text,
finished=False,
)
yield MessageChunk(
message_id=user_id,
delta="",
finished=True,
tokens_used=self._agent_api.last_tokens_used,
)
async def get_settings(self, user_id: str) -> UserSettings:
return await self._prototype_state.get_settings(user_id)
async def update_settings(self, user_id: str, action) -> None:
await self._prototype_state.update_settings(user_id, action)
```
4. Replace sdk/agent_session.py content with:
```python
# Deleted in Phase 4 — replaced by AgentApiWrapper from sdk/agent_api_wrapper.py
# File kept as stub to avoid import errors during migration; remove after test_agent_session.py is updated.
```
</action>
<verify>
<automated>cd /Users/a/MAI/sem2/lambda/surfaces-bot && python -c "from sdk.real import RealPlatformClient; print('import ok')"</automated>
</verify>
<done>
- sdk/agent_api_wrapper.py exists with AgentApiWrapper(AgentApi), __init__ sets self.last_tokens_used = 0, _listen() override captures MsgEventEnd.tokens_used
- sdk/real.py imports AgentApiWrapper (not AgentSessionClient or AgentApi directly), exposes self.agent_api property
- sdk/real.py stream_message yields final chunk with tokens_used from agent_api.last_tokens_used
- external/ directory has NO modifications
- sdk/agent_session.py contains only a comment stub (no class definitions)
- `python -c "from sdk.real import RealPlatformClient"` exits 0
- `grep "AgentApiWrapper" sdk/real.py` returns a match
- `grep "last_tokens_used" sdk/agent_api_wrapper.py` returns a match
</done>
</task>
<task type="auto" tdd="true">
<name>Task 2: Wire AgentApiWrapper lifecycle into bot.py main(); update all broken tests</name>
<read_first>
- adapter/matrix/bot.py (full file — _build_platform_from_env and main() need changes)
- tests/platform/test_agent_session.py (full file — delete or rewrite)
- tests/platform/test_real.py (full file — FakeAgentSessionClient → FakeAgentApi)
- tests/adapter/matrix/test_dispatcher.py (test_build_runtime_uses_real_platform — needs update)
</read_first>
<files>adapter/matrix/bot.py, tests/platform/test_agent_session.py, tests/platform/test_real.py, tests/adapter/matrix/test_dispatcher.py</files>
<behavior>
- _build_platform_from_env() returns a RealPlatformClient with an unconnected AgentApiWrapper (connect() NOT called here — called in main())
- main() calls await runtime.platform.agent_api.connect() after build_runtime() (only when backend is "real"; mock has no agent_api); wrap in `if hasattr(runtime.platform, "agent_api")` guard
- main() finally block: await agent_api.close() before await client.close()
- AGENT_WS_URL env var is passed unchanged to AgentApiWrapper(url=ws_url) — no query param manipulation
- test_agent_session.py: completely rewritten — remove all build_thread_key tests, remove AgentSessionClient tests; replace with 2 tests: (1) import check for lambda_agent_api module, (2) stub test that documents the deletion
- test_real.py: FakeAgentSessionClient replaced with FakeAgentApi that has send_message(text: str) -> AsyncIterator and last_tokens_used: int = 0; tests updated to construct RealPlatformClient(agent_api=FakeAgentApi(), prototype_state=PrototypeStateStore()); test_send_message no longer checks thread_key in message_id (now uses user_id); test_stream_message checks final chunk tokens_used comes from FakeAgentApi.last_tokens_used
- test_dispatcher.py: test_build_runtime_uses_real_platform_when_matrix_backend_is_real must NOT call agent_api.connect() (build_runtime only constructs, does not connect); update test to mock AgentApiWrapper so it does not attempt a real WS connection; assert isinstance(runtime.platform, RealPlatformClient) still passes
</behavior>
<action>
1. Edit adapter/matrix/bot.py:
a. Remove imports: `from sdk.agent_session import AgentSessionClient, AgentSessionConfig`
b. In _build_platform_from_env(), use AgentApiWrapper with lazy import:
```python
def _build_platform_from_env() -> PlatformClient:
backend = os.environ.get("MATRIX_PLATFORM_BACKEND", "mock").strip().lower()
if backend == "real":
import sys
_api_root = Path(__file__).resolve().parents[2] / "external" / "platform-agent_api"
if str(_api_root) not in sys.path:
sys.path.insert(0, str(_api_root))
from sdk.agent_api_wrapper import AgentApiWrapper
ws_url = os.environ["AGENT_WS_URL"]
agent_api = AgentApiWrapper(agent_id="matrix-bot", url=ws_url)
return RealPlatformClient(
agent_api=agent_api,
prototype_state=PrototypeStateStore(),
platform="matrix",
)
return MockPlatformClient()
```
c. In main(), after `runtime = build_runtime(store=SQLiteStore(db_path), client=client)`, add:
```python
if hasattr(runtime.platform, "agent_api"):
await runtime.platform.agent_api.connect()
```
d. In main() finally block, add before `await client.close()`:
```python
if hasattr(runtime.platform, "agent_api"):
await runtime.platform.agent_api.close()
```
2. Rewrite tests/platform/test_agent_session.py:
```python
"""
test_agent_session.py — stub after Phase 4 migration.
AgentSessionClient and build_thread_key were removed in Phase 4.
The platform client is now AgentApiWrapper wrapping AgentApi from lambda_agent_api.
See tests/platform/test_real.py for RealPlatformClient tests.
"""
import sys
from pathlib import Path
_api_root = Path(__file__).resolve().parents[2] / "external" / "platform-agent_api"
if str(_api_root) not in sys.path:
sys.path.insert(0, str(_api_root))
def test_lambda_agent_api_module_importable():
from lambda_agent_api.agent_api import AgentApi # noqa: F401
from lambda_agent_api.server import MsgEventTextChunk, MsgEventEnd # noqa: F401
assert True
def test_agent_session_module_is_stub():
"""Ensure old module no longer exposes AgentSessionClient or build_thread_key."""
import sdk.agent_session as mod
assert not hasattr(mod, "AgentSessionClient"), "AgentSessionClient should be removed"
assert not hasattr(mod, "build_thread_key"), "build_thread_key should be removed"
```
3. Rewrite tests/platform/test_real.py:
```python
from __future__ import annotations
import sys
from pathlib import Path
from typing import AsyncIterator
import pytest
from core.protocol import SettingsAction
from sdk.interface import MessageChunk, MessageResponse, UserSettings
from sdk.prototype_state import PrototypeStateStore
from sdk.real import RealPlatformClient
_api_root = Path(__file__).resolve().parents[2] / "external" / "platform-agent_api"
if str(_api_root) not in sys.path:
sys.path.insert(0, str(_api_root))
from lambda_agent_api.server import MsgEventTextChunk, EServerMessage # noqa: E402
class FakeAgentApi:
"""Minimal fake for AgentApiWrapper — no real WebSocket."""
def __init__(self) -> None:
self.last_tokens_used: int = 0
self.send_calls: list[str] = []
async def send_message(self, text: str) -> AsyncIterator[MsgEventTextChunk]:
self.send_calls.append(text)
self.last_tokens_used = 7
yield MsgEventTextChunk(type=EServerMessage.AGENT_EVENT_TEXT_CHUNK, text=text[:2])
yield MsgEventTextChunk(type=EServerMessage.AGENT_EVENT_TEXT_CHUNK, text=text[2:])
# send_message() in real AgentApi breaks on MsgEventEnd without yielding it;
# FakeAgentApi mirrors this by not yielding MsgEventEnd — last_tokens_used is set directly.
@pytest.mark.asyncio
async def test_real_platform_client_get_or_create_user_uses_local_state():
client = RealPlatformClient(
agent_api=FakeAgentApi(),
prototype_state=PrototypeStateStore(),
)
first = await client.get_or_create_user("u1", "matrix", "Alice")
second = await client.get_or_create_user("u1", "matrix")
assert first.user_id == "usr-matrix-u1"
assert first.is_new is True
assert second.user_id == first.user_id
assert second.is_new is False
assert second.display_name == "Alice"
@pytest.mark.asyncio
async def test_real_platform_client_send_message_calls_agent_with_text():
fake = FakeAgentApi()
client = RealPlatformClient(agent_api=fake, prototype_state=PrototypeStateStore())
result = await client.send_message("@alice:example.org", "C1", "hello")
assert result.response == "hello"
assert result.tokens_used == 7
assert fake.send_calls == ["hello"]
@pytest.mark.asyncio
async def test_real_platform_client_stream_message_yields_chunks_and_final_with_tokens():
fake = FakeAgentApi()
client = RealPlatformClient(agent_api=fake, prototype_state=PrototypeStateStore())
chunks = []
async for chunk in client.stream_message("@alice:example.org", "C1", "hello"):
chunks.append(chunk)
assert chunks[-1].finished is True
assert chunks[-1].tokens_used == 7
assert "".join(c.delta for c in chunks) == "hello"
@pytest.mark.asyncio
async def test_real_platform_client_settings_are_local():
client = RealPlatformClient(
agent_api=FakeAgentApi(),
prototype_state=PrototypeStateStore(),
)
await client.update_settings(
"usr-matrix-u1",
SettingsAction(action="toggle_skill", payload={"skill": "browser", "enabled": True}),
)
settings = await client.get_settings("usr-matrix-u1")
assert isinstance(settings, UserSettings)
assert settings.skills["browser"] is True
```
4. Edit tests/adapter/matrix/test_dispatcher.py — update `test_build_runtime_uses_real_platform_when_matrix_backend_is_real`:
- Add sys.path setup for lambda_agent_api (same pattern as above)
- Mock AgentApiWrapper so it does not open a real WS:
```python
async def test_build_runtime_uses_real_platform_when_matrix_backend_is_real(monkeypatch):
import sys
from pathlib import Path
_api_root = Path(__file__).resolve().parents[3] / "external" / "platform-agent_api"
if str(_api_root) not in sys.path:
sys.path.insert(0, str(_api_root))
monkeypatch.setenv("MATRIX_PLATFORM_BACKEND", "real")
monkeypatch.setenv("AGENT_WS_URL", "ws://agent.example/agent_ws/")
# Patch AgentApiWrapper to avoid real WS connection during build_runtime
import sdk.agent_api_wrapper as _mod
class _FakeAgentApiWrapper:
def __init__(self, agent_id, url, **kw):
self.last_tokens_used = 0
async def connect(self): pass
async def close(self): pass
async def send_message(self, text):
return; yield # empty async generator
monkeypatch.setattr(_mod, "AgentApiWrapper", _FakeAgentApiWrapper)
from adapter.matrix.bot import build_runtime
from sdk.real import RealPlatformClient
runtime = build_runtime()
assert isinstance(runtime.platform, RealPlatformClient)
```
</action>
<verify>
<automated>cd /Users/a/MAI/sem2/lambda/surfaces-bot && python -m pytest tests/platform/test_agent_session.py tests/platform/test_real.py tests/adapter/matrix/test_dispatcher.py -v 2>&1 | tail -20</automated>
</verify>
<done>
- All tests in test_agent_session.py, test_real.py, test_dispatcher.py pass
- main() in bot.py has agent_api.connect() call guarded by hasattr check
- main() finally block closes agent_api before matrix client
- grep confirms no "AgentSessionClient" or "build_thread_key" remain in sdk/real.py or adapter/matrix/bot.py
- grep confirms no modifications to any file under external/
</done>
</task>
</tasks>
<threat_model>
## Trust Boundaries
| Boundary | Description |
|----------|-------------|
| bot → platform-agent WS | Outbound WS to agent service; input is user text |
| env vars → bot config | AGENT_WS_URL, MATRIX_PLATFORM_BACKEND read from environment |
## STRIDE Threat Register
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|-----------|----------|-----------|-------------|-----------------|
| T-04-01-01 | Tampering | AgentApiWrapper.send_message() text | accept | Single-user prototype; text originates from authenticated Matrix user |
| T-04-01-02 | Denial of Service | AgentBusyException from concurrent sends | mitigate | AgentApi._request_lock already prevents concurrent sends; bot must surface error to user instead of crashing |
| T-04-01-03 | Information Disclosure | AGENT_WS_URL in env | accept | Internal service URL; not exposed to users |
</threat_model>
<verification>
Run full test suite after both tasks complete:
```bash
cd /Users/a/MAI/sem2/lambda/surfaces-bot && python -m pytest tests/ -v 2>&1 | tail -30
```
Grep checks:
```bash
# No old imports should remain
grep -r "AgentSessionClient\|build_thread_key" sdk/ adapter/ tests/ --include="*.py" | grep -v "stub\|Deleted\|removed"
# AgentApiWrapper wired in bot.py
grep "agent_api.connect\|agent_api.close" adapter/matrix/bot.py
# last_tokens_used set in wrapper
grep "last_tokens_used" sdk/agent_api_wrapper.py
# No external/ files modified
git diff --name-only external/
```
</verification>
<success_criteria>
- `pytest tests/platform/ tests/adapter/matrix/test_dispatcher.py -v` exits 0 with no failures
- `grep -r "AgentSessionClient" sdk/ adapter/` returns empty (or only the stub comment)
- `grep -r "build_thread_key" sdk/ adapter/` returns empty
- `grep "agent_api.connect" adapter/matrix/bot.py` returns a match
- `grep "last_tokens_used" sdk/agent_api_wrapper.py` returns the assignment line
- `git diff --name-only external/` returns empty (external/ untouched)
</success_criteria>
<output>
After completion, create `.planning/phases/04-matrix-mvp-shared-agent-context-and-context-management-comma/04-01-SUMMARY.md`
</output>

View file

@ -0,0 +1,29 @@
# 04-01 Summary
## Outcome
Replaced the Matrix real backend's custom `AgentSessionClient` path with a shared
`AgentApiWrapper` over upstream `lambda_agent_api.AgentApi`.
## Changes
- Added `sdk/agent_api_wrapper.py` to capture `MsgEventEnd.tokens_used` without
modifying `external/`.
- Rewrote `sdk/real.py` to use a shared `agent_api`, stream text chunks from
`AgentApi.send_message()`, and emit a final `MessageChunk` with
`last_tokens_used`.
- Updated `adapter/matrix/bot.py` to construct `RealPlatformClient` with
`AgentApiWrapper`, keep `AGENT_WS_URL` unchanged, and manage
`agent_api.connect()` / `agent_api.close()` around `sync_forever()`.
- Stubbed `sdk/agent_session.py` as a compatibility placeholder.
- Updated Matrix/runtime tests away from `thread_key` and per-request websocket
assumptions.
## Verification
- `pytest tests/platform/test_real.py -q`
- `pytest tests/adapter/matrix/test_dispatcher.py -q`
- `pytest tests/core/test_integration.py -q`
- `pytest tests/platform/test_agent_session.py -q`
All listed commands passed locally.

View file

@ -0,0 +1,865 @@
---
phase: 04-matrix-mvp-shared-agent-context-and-context-management-comma
plan: 02
type: execute
wave: 2
depends_on:
- 04-01-PLAN.md
files_modified:
- sdk/prototype_state.py
- adapter/matrix/store.py
- adapter/matrix/handlers/__init__.py
- adapter/matrix/handlers/context_commands.py
- adapter/matrix/bot.py
- tests/adapter/matrix/test_context_commands.py
- tests/platform/test_prototype_state.py
autonomous: true
requirements:
- Implement !save, !load, !reset, !context commands
- PrototypeStateStore saved sessions storage
- !load pending state in Matrix store
- !reset pending state in Matrix store
- Numeric input interception for !load
must_haves:
truths:
- "!save sends a save prompt to the agent and records session name in PrototypeStateStore"
- "!load shows a numbered list of saved sessions; numeric reply selects a session"
- "!reset shows a confirmation dialog; !yes calls POST /reset; !no cancels"
- "!context returns current session name, last tokens_used, and list of saved sessions"
- "Numeric input intercepted in on_room_message before dispatcher.dispatch when load_pending is set"
- "!yes in reset_pending context calls POST {AGENT_BASE_URL}/reset and reports unavailable on 404"
- "All context command tests pass"
artifacts:
- path: "adapter/matrix/handlers/context_commands.py"
provides: "make_handle_save, make_handle_load, make_handle_reset, make_handle_context"
- path: "adapter/matrix/store.py"
provides: "get_load_pending, set_load_pending, clear_load_pending, get_reset_pending, set_reset_pending, clear_reset_pending"
- path: "sdk/prototype_state.py"
provides: "add_saved_session, list_saved_sessions, get_last_tokens_used, set_last_tokens_used"
- path: "tests/adapter/matrix/test_context_commands.py"
provides: "tests for all four commands"
key_links:
- from: "adapter/matrix/bot.py on_room_message()"
to: "adapter/matrix/store.get_load_pending()"
via: "check before dispatcher.dispatch"
pattern: "get_load_pending"
- from: "adapter/matrix/handlers/context_commands.py make_handle_reset"
to: "httpx.AsyncClient.post(AGENT_BASE_URL + '/reset')"
via: "!yes handler inside reset_pending flow"
pattern: "httpx"
- from: "sdk/real.py stream_message()"
to: "prototype_state.set_last_tokens_used()"
via: "call after final chunk"
pattern: "set_last_tokens_used"
---
<objective>
Add four context management commands to the Matrix bot: !save, !load, !reset, !context.
Extend PrototypeStateStore with saved sessions and last_tokens_used tracking. Add
load_pending and reset_pending state keys to Matrix store. Wire numeric input
interception in on_room_message. Register all handlers.
Purpose: Users need to save, load, and reset agent context, and inspect current context
state — essential for a shared-context MVP where one agent container persists across
Matrix sessions.
Output: context_commands.py handler module, store.py extensions, prototype_state.py
extensions, bot.py updated, full test coverage.
</objective>
<execution_context>
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
@$HOME/.claude/get-shit-done/templates/summary.md
</execution_context>
<context>
@.planning/phases/04-matrix-mvp-shared-agent-context-and-context-management-comma/04-CONTEXT.md
@.planning/phases/04-matrix-mvp-shared-agent-context-and-context-management-comma/04-RESEARCH.md
@.planning/phases/04-matrix-mvp-shared-agent-context-and-context-management-comma/04-01-SUMMARY.md
</context>
<interfaces>
<!-- Key contracts executor needs. Read source files before touching anything. -->
From adapter/matrix/store.py (existing pattern):
```python
PENDING_CONFIRM_PREFIX = "matrix_pending_confirm:"
def _pending_confirm_key(user_id: str, room_id: str | None = None) -> str: ...
async def get_pending_confirm(store, user_id, room_id=None) -> dict | None: ...
async def set_pending_confirm(store, user_id, room_id, meta) -> None: ...
async def clear_pending_confirm(store, user_id, room_id=None) -> None: ...
```
New store keys to add (same pattern):
```python
LOAD_PENDING_PREFIX = "matrix_load_pending:"
RESET_PENDING_PREFIX = "matrix_reset_pending:"
# Keys: f"{PREFIX}{user_id}:{room_id}"
# load_pending data: {"saves": [{"name": str, "created_at": str}, ...], "display": str}
# reset_pending data: {"active": True}
```
From adapter/matrix/handlers/__init__.py (existing registration):
```python
def register_matrix_handlers(dispatcher: EventDispatcher, client=None, store=None) -> None:
dispatcher.register(IncomingCommand, "new", make_handle_new_chat(client, store))
...
```
Handler closure signature (all existing handlers follow this):
```python
async def handle_X(event: IncomingCommand, auth_mgr, platform, chat_mgr, settings_mgr) -> list[OutgoingEvent]:
```
New handlers use make_handle_X(agent_api, store, prototype_state) closures:
```python
async def _inner(event: IncomingCommand, auth_mgr, platform, chat_mgr, settings_mgr) -> list[OutgoingEvent]:
...
return _inner
```
From sdk/prototype_state.py (PrototypeStateStore to extend):
```python
class PrototypeStateStore:
def __init__(self) -> None:
self._users: dict[str, User] = {}
self._settings: dict[str, dict[str, Any]] = {}
# Add:
# self._saved_sessions: dict[str, list[dict]] = {}
# self._last_tokens_used: dict[str, int] = {}
```
From core/protocol.py:
```python
@dataclass
class IncomingCommand:
user_id: str; platform: str; chat_id: str; command: str; args: list[str]
@dataclass
class OutgoingMessage:
chat_id: str; text: str
@dataclass
class OutgoingUI:
chat_id: str; text: str; buttons: list[UIButton]
```
From sdk/real.py (after Plan 01):
```python
class RealPlatformClient:
async def stream_message(self, user_id, chat_id, text, ...) -> AsyncIterator[MessageChunk]:
# yields chunks; last chunk has finished=True, tokens_used=agent_api.last_tokens_used
```
SAVE_PROMPT template (Claude's Discretion):
```python
SAVE_PROMPT = (
"Summarize our conversation and save to /workspace/contexts/{name}.md. "
"Reply only with: Saved: {name}"
)
LOAD_PROMPT = (
"Load context from /workspace/contexts/{name}.md and use it as background "
"for our conversation. Reply: Loaded: {name}"
)
```
Auto-name format for !save without args: `context-{YYYYMMDD-HHMMSS}` UTC.
HTTP client for POST /reset: httpx.AsyncClient (already in pyproject.toml deps).
AGENT_BASE_URL env var: `os.environ.get("AGENT_BASE_URL", "http://127.0.0.1:8000")`
</interfaces>
<tasks>
<task type="auto" tdd="true">
<name>Task 1: Extend PrototypeStateStore and Matrix store with pending state helpers</name>
<read_first>
- sdk/prototype_state.py (full file — adding saved_sessions and last_tokens_used)
- adapter/matrix/store.py (full file — adding load_pending and reset_pending helpers)
- tests/platform/test_prototype_state.py (full file — adding new test cases)
</read_first>
<files>sdk/prototype_state.py, adapter/matrix/store.py, tests/platform/test_prototype_state.py</files>
<behavior>
- PrototypeStateStore.__init__ adds: self._saved_sessions: dict[str, list[dict]] = {} and self._last_tokens_used: dict[str, int] = {}
- add_saved_session(user_id: str, name: str) -> None: appends {"name": name, "created_at": datetime.now(UTC).isoformat()} to _saved_sessions[user_id]
- list_saved_sessions(user_id: str) -> list[dict]: returns copy of _saved_sessions.get(user_id, [])
- get_last_tokens_used(user_id: str) -> int: returns _last_tokens_used.get(user_id, 0)
- set_last_tokens_used(user_id: str, tokens: int) -> None: sets _last_tokens_used[user_id] = tokens
- adapter/matrix/store.py adds LOAD_PENDING_PREFIX and RESET_PENDING_PREFIX constants
- get_load_pending(store, user_id, room_id) -> dict | None
- set_load_pending(store, user_id, room_id, data: dict) -> None
- clear_load_pending(store, user_id, room_id) -> None
- get_reset_pending(store, user_id, room_id) -> dict | None
- set_reset_pending(store, user_id, room_id, data: dict) -> None
- clear_reset_pending(store, user_id, room_id) -> None
- test_prototype_state.py gets 4 new tests: add/list saved sessions, last_tokens_used get/set
</behavior>
<action>
1. Edit sdk/prototype_state.py — add to __init__ and add four new async methods:
In __init__ after existing attributes:
```python
self._saved_sessions: dict[str, list[dict]] = {}
self._last_tokens_used: dict[str, int] = {}
```
After update_settings() method, add:
```python
async def add_saved_session(self, user_id: str, name: str) -> None:
sessions = self._saved_sessions.setdefault(user_id, [])
sessions.append({"name": name, "created_at": datetime.now(UTC).isoformat()})
async def list_saved_sessions(self, user_id: str) -> list[dict]:
return list(self._saved_sessions.get(user_id, []))
async def get_last_tokens_used(self, user_id: str) -> int:
return self._last_tokens_used.get(user_id, 0)
async def set_last_tokens_used(self, user_id: str, tokens: int) -> None:
self._last_tokens_used[user_id] = tokens
```
2. Edit adapter/matrix/store.py — add after existing constants and helpers:
After PENDING_CONFIRM_PREFIX line, add:
```python
LOAD_PENDING_PREFIX = "matrix_load_pending:"
RESET_PENDING_PREFIX = "matrix_reset_pending:"
```
After clear_pending_confirm(), add:
```python
def _load_pending_key(user_id: str, room_id: str) -> str:
return f"{LOAD_PENDING_PREFIX}{user_id}:{room_id}"
async def get_load_pending(store: StateStore, user_id: str, room_id: str) -> dict | None:
return await store.get(_load_pending_key(user_id, room_id))
async def set_load_pending(store: StateStore, user_id: str, room_id: str, data: dict) -> None:
await store.set(_load_pending_key(user_id, room_id), data)
async def clear_load_pending(store: StateStore, user_id: str, room_id: str) -> None:
await store.delete(_load_pending_key(user_id, room_id))
def _reset_pending_key(user_id: str, room_id: str) -> str:
return f"{RESET_PENDING_PREFIX}{user_id}:{room_id}"
async def get_reset_pending(store: StateStore, user_id: str, room_id: str) -> dict | None:
return await store.get(_reset_pending_key(user_id, room_id))
async def set_reset_pending(store: StateStore, user_id: str, room_id: str, data: dict) -> None:
await store.set(_reset_pending_key(user_id, room_id), data)
async def clear_reset_pending(store: StateStore, user_id: str, room_id: str) -> None:
await store.delete(_reset_pending_key(user_id, room_id))
```
3. Edit tests/platform/test_prototype_state.py — append four new tests:
```python
@pytest.mark.asyncio
async def test_saved_sessions_add_and_list():
store = PrototypeStateStore()
await store.add_saved_session("u1", "my-save")
await store.add_saved_session("u1", "another-save")
sessions = await store.list_saved_sessions("u1")
assert len(sessions) == 2
assert sessions[0]["name"] == "my-save"
assert "created_at" in sessions[0]
assert sessions[1]["name"] == "another-save"
@pytest.mark.asyncio
async def test_saved_sessions_list_returns_copy():
store = PrototypeStateStore()
await store.add_saved_session("u1", "my-save")
sessions = await store.list_saved_sessions("u1")
sessions.append({"name": "injected"})
sessions2 = await store.list_saved_sessions("u1")
assert len(sessions2) == 1
@pytest.mark.asyncio
async def test_last_tokens_used_default_zero():
store = PrototypeStateStore()
assert await store.get_last_tokens_used("u1") == 0
@pytest.mark.asyncio
async def test_last_tokens_used_set_and_get():
store = PrototypeStateStore()
await store.set_last_tokens_used("u1", 42)
assert await store.get_last_tokens_used("u1") == 42
```
</action>
<verify>
<automated>cd /Users/a/MAI/sem2/lambda/surfaces-bot && python -m pytest tests/platform/test_prototype_state.py -v 2>&1 | tail -15</automated>
</verify>
<done>
- PrototypeStateStore has add_saved_session, list_saved_sessions, get_last_tokens_used, set_last_tokens_used
- adapter/matrix/store.py has LOAD_PENDING_PREFIX, RESET_PENDING_PREFIX, and 6 new helper functions
- All test_prototype_state.py tests pass (including 4 new ones)
- `grep "add_saved_session\|list_saved_sessions" sdk/prototype_state.py` returns matches
- `grep "LOAD_PENDING_PREFIX\|RESET_PENDING_PREFIX" adapter/matrix/store.py` returns matches
</done>
</task>
<task type="auto" tdd="true">
<name>Task 2: Implement context_commands.py handlers, wire into __init__.py and bot.py, update tokens_used tracking in real.py</name>
<read_first>
- adapter/matrix/handlers/__init__.py (full file — adding registrations)
- adapter/matrix/handlers/confirm.py (full file — example of make_handle_X closure pattern with store)
- adapter/matrix/bot.py (full file — on_room_message and build_runtime need changes)
- sdk/real.py (after Plan 01 — add set_last_tokens_used call after stream_message)
- adapter/matrix/store.py (after Task 1 — load_pending/reset_pending helpers now available)
- sdk/prototype_state.py (after Task 1 — saved_sessions methods available)
</read_first>
<files>
adapter/matrix/handlers/context_commands.py,
adapter/matrix/handlers/__init__.py,
adapter/matrix/bot.py,
sdk/real.py,
tests/adapter/matrix/test_context_commands.py
</files>
<behavior>
- context_commands.py exports: make_handle_save, make_handle_load, make_handle_reset, make_handle_context
- make_handle_save(agent_api, store, prototype_state) -> handler:
!save with no args: auto-name = f"context-{datetime.now(UTC).strftime('%Y%m%d-%H%M%S')}"
!save [name]: use args[0] as name
sends SAVE_PROMPT via platform.send_message (NOT stream — simple blocking send)
calls prototype_state.add_saved_session(event.user_id, name)
returns [OutgoingMessage(chat_id=event.chat_id, text=f"Сохранение запущено: {name}")]
- make_handle_load(agent_api, store, prototype_state) -> handler:
!load: fetches sessions = await prototype_state.list_saved_sessions(event.user_id)
if empty: returns [OutgoingMessage(chat_id=..., text="Нет сохранённых сессий. Используй !save [имя].")]
else: builds numbered display text, stores load_pending via set_load_pending(store, event.user_id, room_id, {"saves": sessions})
room_id is in event.chat_id (in Matrix adapter, chat_id == room_id for commands)
returns [OutgoingMessage(chat_id=..., text=display_text + "\nВведи номер или 0 / !cancel для отмены.")]
- Numeric input interception in MatrixBot.on_room_message():
Before dispatcher.dispatch, check load_pending = await get_load_pending(runtime.store, sender, room_id)
If load_pending and msg text is digit: handle_load_selection(pending, selection, ...)
handle_load_selection: if text == "0" or "!cancel" → clear_load_pending, return [OutgoingMessage("Отменено")]
if valid index → clear_load_pending, send LOAD_PROMPT via platform.send_message, add session as current_session in prototype_state (store in dict), return [OutgoingMessage("Загрузка: {name}")]
if invalid index → return [OutgoingMessage("Неверный номер. Введи от 1 до N или 0 для отмены.")]
- make_handle_reset(store, agent_base_url) -> handler:
!reset: set reset_pending, return [OutgoingMessage with text:
"Сбросить контекст агента? Выбери:\n !yes — сбросить\n !save [имя] — сохранить и сбросить\n !no — отмена")]
!yes in reset_pending: call POST {AGENT_BASE_URL}/reset via httpx; if 404 or connection error → "Reset endpoint недоступен. Обратитесь к администратору."; else "Контекст сброшен."; clear reset_pending
!no in reset_pending: clear reset_pending, return [OutgoingMessage("Отменено.")]
!save имя in reset_pending: delegate to save logic, then POST /reset (same fallback)
Reset_pending check must happen BEFORE pending_confirm in handler priority — implement by checking reset_pending in the !yes and !no handlers (make_handle_confirm must check reset_pending first)
- make_handle_context(store, prototype_state) -> handler:
reads current_session from prototype_state._current_session dict (keyed by user_id) if it exists
reads tokens = await prototype_state.get_last_tokens_used(event.user_id)
reads sessions = await prototype_state.list_saved_sessions(event.user_id)
formats: "Контекст:\n Сессия: {name or 'не загружена'}\n Токены (последний ответ): {tokens}\n Сохранения ({len}):\n {list}"
returns [OutgoingMessage(chat_id=..., text=formatted)]
- sdk/real.py: after the final yield in stream_message, call await self._prototype_state.set_last_tokens_used(user_id, self._agent_api.last_tokens_used) — needs prototype_state reference already present in RealPlatformClient
- PrototypeStateStore gets one more dict: self._current_session: dict[str, str] = {} and methods get_current_session(user_id) -> str | None, set_current_session(user_id, name) -> None
- register_matrix_handlers() updated to accept agent_api and agent_base_url params; registers save/load/reset/context
</behavior>
<action>
1. Add to sdk/prototype_state.py __init__: `self._current_session: dict[str, str] = {}`
Add methods:
```python
async def get_current_session(self, user_id: str) -> str | None:
return self._current_session.get(user_id)
async def set_current_session(self, user_id: str, name: str) -> None:
self._current_session[user_id] = name
```
2. Create adapter/matrix/handlers/context_commands.py:
```python
from __future__ import annotations
import os
from datetime import UTC, datetime
from typing import TYPE_CHECKING
import httpx
import structlog
from core.protocol import IncomingCommand, OutgoingEvent, OutgoingMessage
if TYPE_CHECKING:
from lambda_agent_api.agent_api import AgentApi
from sdk.prototype_state import PrototypeStateStore
from core.store import StateStore
logger = structlog.get_logger(__name__)
SAVE_PROMPT = (
"Summarize our conversation and save to /workspace/contexts/{name}.md. "
"Reply only with: Saved: {name}"
)
LOAD_PROMPT = (
"Load context from /workspace/contexts/{name}.md and use it as background "
"for our conversation. Reply: Loaded: {name}"
)
def make_handle_save(agent_api: "AgentApi", store: "StateStore", prototype_state: "PrototypeStateStore"):
async def handle_save(
event: IncomingCommand, auth_mgr, platform, chat_mgr, settings_mgr
) -> list[OutgoingEvent]:
if event.args:
name = event.args[0]
else:
name = f"context-{datetime.now(UTC).strftime('%Y%m%d-%H%M%S')}"
prompt = SAVE_PROMPT.format(name=name)
try:
await platform.send_message(event.user_id, event.chat_id, prompt)
except Exception as exc:
logger.warning("save_agent_call_failed", error=str(exc))
return [OutgoingMessage(chat_id=event.chat_id, text=f"Ошибка при сохранении: {exc}")]
await prototype_state.add_saved_session(event.user_id, name)
return [OutgoingMessage(chat_id=event.chat_id, text=f"Сохранение запущено: {name}")]
return handle_save
def make_handle_load(store: "StateStore", prototype_state: "PrototypeStateStore"):
async def handle_load(
event: IncomingCommand, auth_mgr, platform, chat_mgr, settings_mgr
) -> list[OutgoingEvent]:
from adapter.matrix.store import set_load_pending
sessions = await prototype_state.list_saved_sessions(event.user_id)
if not sessions:
return [OutgoingMessage(
chat_id=event.chat_id,
text="Нет сохранённых сессий. Используй !save [имя].",
)]
lines = ["Сохранённые сессии:"]
for i, s in enumerate(sessions, start=1):
created = s.get("created_at", "")[:10]
lines.append(f" {i}. {s['name']} ({created})")
lines.append("\nВведи номер или 0 / !cancel для отмены.")
display = "\n".join(lines)
await set_load_pending(store, event.user_id, event.chat_id, {"saves": sessions})
return [OutgoingMessage(chat_id=event.chat_id, text=display)]
return handle_load
def make_handle_reset(store: "StateStore", agent_base_url: str):
async def handle_reset(
event: IncomingCommand, auth_mgr, platform, chat_mgr, settings_mgr
) -> list[OutgoingEvent]:
from adapter.matrix.store import set_reset_pending
await set_reset_pending(store, event.user_id, event.chat_id, {"active": True})
text = (
"Сбросить контекст агента? Выбери:\n"
" !yes — сбросить\n"
" !save [имя] — сохранить и сбросить\n"
" !no — отмена"
)
return [OutgoingMessage(chat_id=event.chat_id, text=text)]
return handle_reset
async def _call_reset_endpoint(agent_base_url: str, chat_id: str) -> list[OutgoingEvent]:
try:
async with httpx.AsyncClient() as http:
resp = await http.post(f"{agent_base_url}/reset", timeout=5.0)
if resp.status_code == 404:
return [OutgoingMessage(chat_id=chat_id, text="Reset endpoint недоступен. Обратитесь к администратору.")]
return [OutgoingMessage(chat_id=chat_id, text="Контекст сброшен.")]
except (httpx.ConnectError, httpx.TimeoutException) as exc:
logger.warning("reset_endpoint_unreachable", error=str(exc))
return [OutgoingMessage(chat_id=chat_id, text="Reset endpoint недоступен. Обратитесь к администратору.")]
def make_handle_context(store: "StateStore", prototype_state: "PrototypeStateStore"):
async def handle_context(
event: IncomingCommand, auth_mgr, platform, chat_mgr, settings_mgr
) -> list[OutgoingEvent]:
session_name = await prototype_state.get_current_session(event.user_id) or "не загружена"
tokens = await prototype_state.get_last_tokens_used(event.user_id)
sessions = await prototype_state.list_saved_sessions(event.user_id)
lines = [
"Контекст:",
f" Сессия: {session_name}",
f" Токены (последний ответ): {tokens}",
f" Сохранения ({len(sessions)}):",
]
for s in sessions:
created = s.get("created_at", "")[:10]
lines.append(f" • {s['name']} ({created})")
if not sessions:
lines.append(" (нет)")
return [OutgoingMessage(chat_id=event.chat_id, text="\n".join(lines))]
return handle_context
```
3. Edit adapter/matrix/handlers/__init__.py:
- Add import at top: `from adapter.matrix.handlers.context_commands import make_handle_save, make_handle_load, make_handle_reset, make_handle_context`
- Change signature: `def register_matrix_handlers(dispatcher, client=None, store=None, agent_api=None, prototype_state=None, agent_base_url="http://127.0.0.1:8000") -> None:`
- Add at bottom of function before the last line:
```python
if agent_api is not None and prototype_state is not None:
dispatcher.register(IncomingCommand, "save", make_handle_save(agent_api, store, prototype_state))
dispatcher.register(IncomingCommand, "load", make_handle_load(store, prototype_state))
dispatcher.register(IncomingCommand, "reset", make_handle_reset(store, agent_base_url))
dispatcher.register(IncomingCommand, "context", make_handle_context(store, prototype_state))
```
4. Edit adapter/matrix/bot.py:
a. Add imports: `from adapter.matrix.store import get_load_pending, clear_load_pending, get_reset_pending, clear_reset_pending`
b. In build_event_dispatcher() and build_runtime(), extract prototype_state from platform if RealPlatformClient, otherwise create new one:
In build_runtime() after creating platform:
```python
prototype_state = getattr(platform, "_prototype_state", None)
agent_api = getattr(platform, "_agent_api", None)
agent_base_url = os.environ.get("AGENT_BASE_URL", "http://127.0.0.1:8000")
```
Pass these to register_matrix_handlers:
```python
register_matrix_handlers(dispatcher, client=client, store=store,
agent_api=agent_api, prototype_state=prototype_state,
agent_base_url=agent_base_url)
```
c. In MatrixBot.on_room_message(), before `incoming = from_room_event(...)`:
```python
sender = getattr(event, "sender", None)
# !load numeric interception
load_pending = await get_load_pending(self.runtime.store, sender, room.room_id)
if load_pending is not None:
text = getattr(event, "body", "").strip()
if text.isdigit() or text == "0" or text == "!cancel":
outgoing = await self._handle_load_selection(
sender, room.room_id, text, load_pending
)
await self._send_all(room.room_id, outgoing)
return
```
d. Add _handle_load_selection method to MatrixBot:
```python
async def _handle_load_selection(
self, user_id: str, room_id: str, text: str, pending: dict
) -> list[OutgoingEvent]:
from adapter.matrix.store import clear_load_pending
saves = pending.get("saves", [])
if text == "0" or text == "!cancel":
await clear_load_pending(self.runtime.store, user_id, room_id)
return [OutgoingMessage(chat_id=room_id, text="Отменено.")]
idx = int(text) - 1
if idx < 0 or idx >= len(saves):
return [OutgoingMessage(chat_id=room_id, text=f"Неверный номер. Введи от 1 до {len(saves)} или 0 для отмены.")]
name = saves[idx]["name"]
await clear_load_pending(self.runtime.store, user_id, room_id)
prototype_state = getattr(self.runtime.platform, "_prototype_state", None)
if prototype_state is not None:
await prototype_state.set_current_session(user_id, name)
prompt = f"Load context from /workspace/contexts/{name}.md and use it as background for our conversation. Reply: Loaded: {name}"
try:
await self.runtime.platform.send_message(user_id, room_id, prompt)
except Exception as exc:
logger.warning("load_agent_call_failed", error=str(exc))
return [OutgoingMessage(chat_id=room_id, text=f"Ошибка при загрузке: {exc}")]
return [OutgoingMessage(chat_id=room_id, text=f"Загрузка: {name}")]
```
e. In MatrixBot.on_room_message(), also add reset_pending check for !yes/!no/!save commands:
In the block after load_pending check, before calling dispatcher.dispatch:
```python
# !reset pending interception for !yes, !no, !save commands
reset_pending = await get_reset_pending(self.runtime.store, sender, room.room_id)
if reset_pending is not None:
body = getattr(event, "body", "").strip()
if body == "!yes" or body.startswith("!save ") or body == "!no":
outgoing = await self._handle_reset_selection(sender, room.room_id, body)
await self._send_all(room.room_id, outgoing)
return
```
f. Add _handle_reset_selection method to MatrixBot:
```python
async def _handle_reset_selection(
self, user_id: str, room_id: str, text: str
) -> list[OutgoingEvent]:
from adapter.matrix.store import clear_reset_pending
from adapter.matrix.handlers.context_commands import _call_reset_endpoint
agent_base_url = os.environ.get("AGENT_BASE_URL", "http://127.0.0.1:8000")
await clear_reset_pending(self.runtime.store, user_id, room_id)
if text == "!no":
return [OutgoingMessage(chat_id=room_id, text="Отменено.")]
if text.startswith("!save "):
name = text[len("!save "):].strip()
prototype_state = getattr(self.runtime.platform, "_prototype_state", None)
prompt = f"Summarize our conversation and save to /workspace/contexts/{name}.md. Reply only with: Saved: {name}"
try:
await self.runtime.platform.send_message(user_id, room_id, prompt)
if prototype_state:
await prototype_state.add_saved_session(user_id, name)
except Exception as exc:
logger.warning("save_before_reset_failed", error=str(exc))
return await _call_reset_endpoint(agent_base_url, room_id)
```
5. Edit sdk/real.py — in stream_message(), after the final yield, add:
```python
await self._prototype_state.set_last_tokens_used(user_id, self._agent_api.last_tokens_used)
```
(This must come after `yield MessageChunk(finished=True, ...)` — use a local variable to store tokens_used before yield, then call set_last_tokens_used after the generator resumes.)
Actually: put it before the final yield:
```python
await self._prototype_state.set_last_tokens_used(user_id, self._agent_api.last_tokens_used)
yield MessageChunk(
message_id=user_id,
delta="",
finished=True,
tokens_used=self._agent_api.last_tokens_used,
)
```
6. Create tests/adapter/matrix/test_context_commands.py:
```python
from __future__ import annotations
from typing import AsyncIterator
from unittest.mock import AsyncMock, patch
import pytest
from adapter.matrix.bot import MatrixBot, build_runtime
from core.protocol import IncomingCommand, OutgoingMessage
from sdk.mock import MockPlatformClient
from sdk.prototype_state import PrototypeStateStore
def make_runtime_with_prototype_state():
proto = PrototypeStateStore()
platform = MockPlatformClient()
# Inject prototype_state into platform so handlers can find it
platform._prototype_state = proto
runtime = build_runtime(platform=platform)
return runtime, proto
@pytest.mark.asyncio
async def test_save_command_auto_name_records_session():
proto = PrototypeStateStore()
platform = MockPlatformClient()
platform._prototype_state = proto
from adapter.matrix.handlers.context_commands import make_handle_save
from core.store import InMemoryStore
store = InMemoryStore()
handler = make_handle_save(agent_api=None, store=store, prototype_state=proto)
event = IncomingCommand(user_id="u1", platform="matrix", chat_id="!room:example", command="save", args=[])
class FakePlatform:
async def send_message(self, *a, **kw): pass
result = await handler(event, None, FakePlatform(), None, None)
assert any(isinstance(r, OutgoingMessage) and "Сохранение запущено" in r.text for r in result)
sessions = await proto.list_saved_sessions("u1")
assert len(sessions) == 1
assert sessions[0]["name"].startswith("context-")
@pytest.mark.asyncio
async def test_save_command_with_name_uses_given_name():
proto = PrototypeStateStore()
from adapter.matrix.handlers.context_commands import make_handle_save
from core.store import InMemoryStore
store = InMemoryStore()
handler = make_handle_save(agent_api=None, store=store, prototype_state=proto)
event = IncomingCommand(user_id="u1", platform="matrix", chat_id="!r:e", command="save", args=["my-session"])
class FakePlatform:
async def send_message(self, *a, **kw): pass
await handler(event, None, FakePlatform(), None, None)
sessions = await proto.list_saved_sessions("u1")
assert sessions[0]["name"] == "my-session"
@pytest.mark.asyncio
async def test_load_command_shows_numbered_list():
proto = PrototypeStateStore()
await proto.add_saved_session("u1", "session-A")
await proto.add_saved_session("u1", "session-B")
from adapter.matrix.handlers.context_commands import make_handle_load
from core.store import InMemoryStore
store = InMemoryStore()
handler = make_handle_load(store=store, prototype_state=proto)
event = IncomingCommand(user_id="u1", platform="matrix", chat_id="!r:e", command="load", args=[])
result = await handler(event, None, None, None, None)
assert len(result) == 1
text = result[0].text
assert "1." in text and "session-A" in text
assert "2." in text and "session-B" in text
assert "0" in text
@pytest.mark.asyncio
async def test_load_command_empty_sessions():
proto = PrototypeStateStore()
from adapter.matrix.handlers.context_commands import make_handle_load
from core.store import InMemoryStore
store = InMemoryStore()
handler = make_handle_load(store=store, prototype_state=proto)
event = IncomingCommand(user_id="u1", platform="matrix", chat_id="!r:e", command="load", args=[])
result = await handler(event, None, None, None, None)
assert "Нет сохранённых сессий" in result[0].text
@pytest.mark.asyncio
async def test_reset_command_shows_dialog():
proto = PrototypeStateStore()
from adapter.matrix.handlers.context_commands import make_handle_reset
from core.store import InMemoryStore
store = InMemoryStore()
handler = make_handle_reset(store=store, agent_base_url="http://127.0.0.1:8000")
event = IncomingCommand(user_id="u1", platform="matrix", chat_id="!r:e", command="reset", args=[])
result = await handler(event, None, None, None, None)
text = result[0].text
assert "!yes" in text
assert "!save" in text
assert "!no" in text
@pytest.mark.asyncio
async def test_reset_yes_reports_unavailable_when_endpoint_missing():
from adapter.matrix.handlers.context_commands import _call_reset_endpoint
with patch("httpx.AsyncClient") as MockClient:
import httpx
MockClient.return_value.__aenter__ = AsyncMock(return_value=MockClient.return_value)
MockClient.return_value.__aexit__ = AsyncMock(return_value=False)
MockClient.return_value.post = AsyncMock(side_effect=httpx.ConnectError("refused"))
result = await _call_reset_endpoint("http://127.0.0.1:8000", "!r:e")
assert "недоступен" in result[0].text
@pytest.mark.asyncio
async def test_context_command_shows_snapshot():
proto = PrototypeStateStore()
await proto.set_last_tokens_used("u1", 99)
await proto.add_saved_session("u1", "my-save")
from adapter.matrix.handlers.context_commands import make_handle_context
from core.store import InMemoryStore
store = InMemoryStore()
handler = make_handle_context(store=store, prototype_state=proto)
event = IncomingCommand(user_id="u1", platform="matrix", chat_id="!r:e", command="context", args=[])
result = await handler(event, None, None, None, None)
text = result[0].text
assert "99" in text
assert "my-save" in text
assert "не загружена" in text
```
</action>
<verify>
<automated>cd /Users/a/MAI/sem2/lambda/surfaces-bot && python -m pytest tests/adapter/matrix/test_context_commands.py tests/platform/test_prototype_state.py -v 2>&1 | tail -20</automated>
</verify>
<done>
- adapter/matrix/handlers/context_commands.py exists with make_handle_save, make_handle_load, make_handle_reset, make_handle_context, _call_reset_endpoint
- register_matrix_handlers() signature accepts agent_api, prototype_state, agent_base_url; registers save/load/reset/context handlers when agent_api is not None
- MatrixBot.on_room_message() checks load_pending and reset_pending before dispatcher.dispatch
- sdk/real.py calls set_last_tokens_used before final yield
- All tests in test_context_commands.py pass
- Full test suite still passes: `pytest tests/ -v` exits 0
</done>
</task>
</tasks>
<threat_model>
## Trust Boundaries
| Boundary | Description |
|----------|-------------|
| Matrix user → command args | !save [name] arg is user-controlled; used in file paths |
| bot → agent (save/load prompts) | Prompt text contains user-supplied name |
| bot → POST /reset endpoint | HTTP call to AGENT_BASE_URL (internal service) |
## STRIDE Threat Register
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|-----------|----------|-----------|-------------|-----------------|
| T-04-02-01 | Tampering | !save name arg used in /workspace/contexts/{name}.md path | mitigate | Sanitize name: only allow [a-zA-Z0-9_-] characters; reject path traversal attempts (names containing "/" or "..") |
| T-04-02-02 | Information Disclosure | !context exposes tokens_used and session names | accept | Single-user prototype; data is the user's own |
| T-04-02-03 | Denial of Service | !load numeric interception could loop if load_pending never clears | mitigate | clear_load_pending on selection (any) or disconnect; pending data is volatile in-memory |
| T-04-02-04 | Spoofing | !yes intercepted by reset_pending could conflict with pending_confirm | mitigate | Reset_pending check in on_room_message before dispatcher — takes priority; documented in code comment |
| T-04-02-05 | Tampering | POST /reset to AGENT_BASE_URL | accept | Internal service URL from env; timeout=5.0 prevents hanging |
</threat_model>
<verification>
Run full suite after both tasks:
```bash
cd /Users/a/MAI/sem2/lambda/surfaces-bot && python -m pytest tests/ -v 2>&1 | tail -30
```
Grep checks:
```bash
# Handlers registered
grep "save\|load\|reset\|context" adapter/matrix/handlers/__init__.py
# Numeric interception in bot
grep "get_load_pending\|_handle_load_selection" adapter/matrix/bot.py
# tokens tracking in real.py
grep "set_last_tokens_used" sdk/real.py
# context_commands module
ls adapter/matrix/handlers/context_commands.py
```
</verification>
<success_criteria>
- `pytest tests/adapter/matrix/test_context_commands.py -v` exits 0 with 7+ tests passing
- `pytest tests/platform/test_prototype_state.py -v` exits 0 (including 4 new tests)
- `pytest tests/ -v` exits 0
- !save, !load, !reset, !context all registered in register_matrix_handlers
- load_pending and reset_pending helpers exist in adapter/matrix/store.py
- MatrixBot.on_room_message contains numeric interception for !load
</success_criteria>
<output>
After completion, create `.planning/phases/04-matrix-mvp-shared-agent-context-and-context-management-comma/04-02-SUMMARY.md`
</output>

View file

@ -0,0 +1,40 @@
# Phase 04 Plan 02: Matrix Context Commands Summary
## Outcome
Added Matrix context management for `!save`, `!load`, `!reset`, and `!context`, plus
pending-state interception in the Matrix bot and prototype-state tracking for saved
sessions, current session, and last token usage.
## Commits
- `2720ee2` `feat(04-02): extend prototype and matrix pending state`
- `b52fdc4` `feat(04-02): add matrix context management commands`
## Verification
- `pytest tests/platform/test_prototype_state.py -q`
- `pytest tests/adapter/matrix/test_context_commands.py tests/platform/test_prototype_state.py -q`
- `pytest tests/adapter/matrix/test_dispatcher.py tests/adapter/matrix/test_confirm.py -q`
## Deviations from Plan
### Auto-fixed Issues
1. `[Rule 2 - Critical functionality]` Sanitized save names before using them in save/load prompts.
This rejects names outside `[A-Za-z0-9_-]` and prevents path-like input from flowing into `/workspace/contexts/{name}.md`.
2. `[Rule 2 - Critical functionality]` Cleared the active current-session marker on reset.
Without this, `!context` could report a stale loaded session after `!reset`.
## Files Changed
- `sdk/prototype_state.py`
- `adapter/matrix/store.py`
- `adapter/matrix/handlers/__init__.py`
- `adapter/matrix/handlers/context_commands.py`
- `adapter/matrix/bot.py`
- `tests/adapter/matrix/test_context_commands.py`
- `tests/platform/test_prototype_state.py`
## Self-Check: PASSED

View file

@ -0,0 +1,193 @@
---
phase: 04-matrix-mvp-shared-agent-context-and-context-management-comma
plan: 03
type: execute
wave: 2
depends_on:
- 04-01-PLAN.md
files_modified:
- Dockerfile
- docker-compose.yml
- .env.example
autonomous: true
requirements:
- Dockerfile for Matrix bot
- docker-compose.yml with matrix-bot service
- .env.example updated with AGENT_BASE_URL and MATRIX_PLATFORM_BACKEND
must_haves:
truths:
- "Dockerfile builds successfully with python:3.11-slim base"
- "lambda_agent_api installed in container despite Python version constraint"
- "PYTHONPATH=/app set so adapter/matrix/bot.py is runnable as module"
- "docker-compose.yml defines matrix-bot service with env_file: .env"
- ".env.example contains AGENT_BASE_URL and MATRIX_PLATFORM_BACKEND=real"
- "CMD runs python -m adapter.matrix.bot"
artifacts:
- path: "Dockerfile"
provides: "Matrix bot container image"
contains: "python:3.11-slim"
- path: "docker-compose.yml"
provides: "Service definition for matrix-bot"
contains: "matrix-bot"
- path: ".env.example"
provides: "Updated env template"
contains: "AGENT_BASE_URL"
key_links:
- from: "Dockerfile"
to: "external/platform-agent_api"
via: "COPY + pip install with --ignore-requires-python"
pattern: "ignore-requires-python"
---
<objective>
Package the Matrix bot in a Docker container. Create Dockerfile using python:3.11-slim,
install lambda_agent_api from the local external/ directory (bypassing the Python 3.14
version constraint), and define a docker-compose.yml for running the matrix-bot service.
Update .env.example with new variables.
Purpose: Enable reproducible MVP deployment of the Matrix bot in a container alongside
the separately-run platform-agent.
Output: Dockerfile, docker-compose.yml, updated .env.example.
</objective>
<execution_context>
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
@$HOME/.claude/get-shit-done/templates/summary.md
</execution_context>
<context>
@.planning/phases/04-matrix-mvp-shared-agent-context-and-context-management-comma/04-CONTEXT.md
@.planning/phases/04-matrix-mvp-shared-agent-context-and-context-management-comma/04-RESEARCH.md
</context>
<tasks>
<task type="auto">
<name>Task 1: Create Dockerfile and docker-compose.yml</name>
<read_first>
- .env.example (full file — adding new vars)
- external/platform-agent_api/lambda_agent_api/ (ls — verify files to copy)
- pyproject.toml (verify uv is the package manager used)
</read_first>
<files>Dockerfile, docker-compose.yml, .env.example</files>
<action>
1. Check if pyproject.toml uses uv or pip. The project uses `uv sync` per CLAUDE.md. However, in the Docker container we can use pip for simplicity since uv's lockfile-based install may need the lockfile present. Use pip for the base install of surfaces-bot deps, and install lambda_agent_api separately.
Actually: the project uses uv. Use uv in Docker to be consistent:
- Install uv via pip (pip install uv)
- Run uv sync to install project deps
- Install lambda_agent_api with pip --ignore-requires-python
2. Create Dockerfile:
```dockerfile
FROM python:3.11-slim
WORKDIR /app
# Install uv
RUN pip install --no-cache-dir uv
# Copy dependency manifests first for layer caching
COPY pyproject.toml uv.lock* ./
# Install project dependencies via uv (no project install yet, just deps)
RUN uv sync --no-install-project --frozen 2>/dev/null || uv sync --no-install-project
# Copy project source
COPY . .
# Install the project itself
RUN uv sync --frozen 2>/dev/null || uv sync
# Install lambda_agent_api, bypassing Python version constraint
RUN pip install --no-cache-dir --ignore-requires-python /app/external/platform-agent_api
ENV PYTHONPATH=/app
ENV PYTHONUNBUFFERED=1
CMD ["python", "-m", "adapter.matrix.bot"]
```
3. Create docker-compose.yml:
```yaml
services:
matrix-bot:
build: .
env_file: .env
restart: unless-stopped
# platform-agent runs separately — not included in this compose file
```
4. Read current .env.example, then append new variables. Current file likely has MATRIX_* vars. Add:
- AGENT_WS_URL=ws://127.0.0.1:8000/agent_ws/
- AGENT_BASE_URL=http://127.0.0.1:8000
- MATRIX_PLATFORM_BACKEND=real
Read .env.example first to see what's there, then write the full updated file.
</action>
<done>
- `grep "python:3.11-slim" Dockerfile` returns a match
- `grep "ignore-requires-python" Dockerfile` returns a match (lambda_agent_api install)
- `grep "PYTHONPATH=/app" Dockerfile` returns a match
- `grep "adapter.matrix.bot" Dockerfile` returns a match (CMD)
- `grep "matrix-bot" docker-compose.yml` returns a match
- `grep "env_file" docker-compose.yml` returns a match
- `grep "AGENT_BASE_URL" .env.example` returns a match
- `grep "MATRIX_PLATFORM_BACKEND" .env.example` returns a match
- Dockerfile exists with python:3.11-slim, uv install, lambda_agent_api pip install --ignore-requires-python, PYTHONPATH=/app, CMD python -m adapter.matrix.bot
- docker-compose.yml exists with matrix-bot service, env_file: .env, restart: unless-stopped
- .env.example contains AGENT_WS_URL, AGENT_BASE_URL, MATRIX_PLATFORM_BACKEND=real
</done>
<verify>
<automated>grep "python:3.11-slim" /Users/a/MAI/sem2/lambda/surfaces-bot/Dockerfile && grep "ignore-requires-python" /Users/a/MAI/sem2/lambda/surfaces-bot/Dockerfile && grep "AGENT_BASE_URL" /Users/a/MAI/sem2/lambda/surfaces-bot/.env.example && echo "All checks passed"</automated>
</verify>
</task>
</tasks>
<threat_model>
## Trust Boundaries
| Boundary | Description |
|----------|-------------|
| container → host env | .env file mounts secrets into container |
| container → platform-agent | Network call to AGENT_WS_URL / AGENT_BASE_URL |
## STRIDE Threat Register
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|-----------|----------|-----------|-------------|-----------------|
| T-04-03-01 | Information Disclosure | .env file with secrets mounted in container | mitigate | .env in .gitignore; .env.example committed with placeholder values only, never real secrets |
| T-04-03-02 | Tampering | lambda_agent_api installed from local path via --ignore-requires-python | accept | Local package under version control; no external supply chain risk |
| T-04-03-03 | Denial of Service | restart: unless-stopped could loop on crash | accept | Expected behavior; operator can `docker compose stop` |
</threat_model>
<verification>
```bash
# Verify files exist and contain expected content
grep "python:3.11-slim" /Users/a/MAI/sem2/lambda/surfaces-bot/Dockerfile
grep "ignore-requires-python" /Users/a/MAI/sem2/lambda/surfaces-bot/Dockerfile
grep "AGENT_BASE_URL" /Users/a/MAI/sem2/lambda/surfaces-bot/.env.example
grep "matrix-bot" /Users/a/MAI/sem2/lambda/surfaces-bot/docker-compose.yml
```
</verification>
<success_criteria>
- Dockerfile, docker-compose.yml, .env.example all exist in project root
- Dockerfile builds without errors when platform-agent_api dir is present (docker build . exits 0)
- .env.example contains AGENT_BASE_URL, AGENT_WS_URL, MATRIX_PLATFORM_BACKEND
- docker-compose.yml service named matrix-bot uses env_file: .env
</success_criteria>
<output>
After completion, create `.planning/phases/04-matrix-mvp-shared-agent-context-and-context-management-comma/04-03-SUMMARY.md`
</output>

View file

@ -0,0 +1,106 @@
---
phase: 04-matrix-mvp-shared-agent-context-and-context-management-comma
plan: 03
subsystem: infra
tags: [docker, docker-compose, matrix, uv, lambda-agent-api]
requires:
- phase: 04-01
provides: Matrix MVP runtime and environment model
provides:
- Matrix bot Docker image definition
- Single-service docker-compose setup for matrix-bot
- Env template entries for Agent API base URLs and real backend selection
affects: [deployment, matrix, local-dev]
tech-stack:
added: [Dockerfile, docker-compose]
patterns: [uv-managed container install with system Python runtime, local path install for lambda_agent_api]
key-files:
created: [Dockerfile, docker-compose.yml]
modified: [.env.example]
key-decisions:
- "Kept compose scoped to matrix-bot only; platform-agent remains external to this stack."
- "Set UV_PROJECT_ENVIRONMENT=/usr/local so uv-installed dependencies are available to CMD [\"python\", \"-m\", \"adapter.matrix.bot\"]."
patterns-established:
- "Install project dependencies with uv inside the container, then install lambda_agent_api from external/platform-agent_api via pip --ignore-requires-python."
requirements-completed: [Dockerfile for Matrix bot, docker-compose.yml with matrix-bot service, .env.example updated with AGENT_BASE_URL and MATRIX_PLATFORM_BACKEND]
duration: 6min
completed: 2026-04-17
---
# Phase 4 Plan 03: Matrix Bot Containerization Summary
**Python 3.11 Matrix bot container with uv-managed app dependencies, local lambda_agent_api install bypass, and a single-service compose entrypoint**
## Performance
- **Duration:** 6 min
- **Started:** 2026-04-17T13:01:00Z
- **Completed:** 2026-04-17T13:07:04Z
- **Tasks:** 1
- **Files modified:** 4
## Accomplishments
- Added a root `Dockerfile` for the Matrix bot using `python:3.11-slim`.
- Added `docker-compose.yml` with a single `matrix-bot` service using `env_file: .env`.
- Extended `.env.example` with `AGENT_WS_URL`, `AGENT_BASE_URL`, and `MATRIX_PLATFORM_BACKEND=real`.
## Files Created/Modified
- `Dockerfile` - Builds the Matrix bot image, installs project dependencies with `uv`, and installs `lambda_agent_api` from the local `external/` tree.
- `docker-compose.yml` - Defines the `matrix-bot` service with restart policy and `.env` loading.
- `.env.example` - Documents the agent WebSocket URL, agent HTTP base URL, and real backend selector.
## Decisions Made
- Kept the compose scope limited to the Matrix bot, matching the phase boundary and excluding platform services.
- Added `UV_PROJECT_ENVIRONMENT=/usr/local` as a correctness fix so `uv sync` installs are visible to the final `python` runtime.
## Deviations from Plan
### Auto-fixed Issues
**1. [Rule 2 - Missing Critical] Ensured uv installs are available to the container runtime**
- **Found during:** Task 1 (Create Dockerfile and docker-compose.yml)
- **Issue:** The plan sketch used `uv sync` plus `CMD ["python", ...]`; by default, `uv sync` would install into a virtual environment that system `python` would not use.
- **Fix:** Set `UV_PROJECT_ENVIRONMENT=/usr/local` in the Dockerfile before running `uv sync`.
- **Files modified:** `Dockerfile`
- **Verification:** Required grep checks passed and the generated compose config remained valid.
---
**Total deviations:** 1 auto-fixed (1 missing critical)
**Impact on plan:** Narrow correctness fix only. No scope expansion.
## Issues Encountered
- `docker compose config` resolved values from the local `.env`, so verification was kept to config rendering and grep-style checks rather than a full image build.
## User Setup Required
- Create `.env` from `.env.example` with real Matrix and agent values before running `docker compose up`.
## Next Phase Readiness
- Matrix bot container packaging is in place and ready for operator-supplied secrets plus an external platform-agent deployment.
- No code changes were made outside the allowed containerization files.
## Verification
- `grep 'python:3.11-slim' Dockerfile`
- `grep 'ignore-requires-python' Dockerfile`
- `grep 'PYTHONPATH=/app' Dockerfile`
- `grep 'adapter.matrix.bot' Dockerfile`
- `grep 'matrix-bot' docker-compose.yml`
- `grep 'env_file' docker-compose.yml`
- `grep 'AGENT_BASE_URL' .env.example`
- `grep 'AGENT_WS_URL' .env.example`
- `grep 'MATRIX_PLATFORM_BACKEND' .env.example`
- `docker compose -f docker-compose.yml config`
## Self-Check: PASSED
- Found `Dockerfile`
- Found `docker-compose.yml`
- Found updated `.env.example`
- Found `.planning/phases/04-matrix-mvp-shared-agent-context-and-context-management-comma/04-03-SUMMARY.md`

View file

@ -0,0 +1,136 @@
# Phase 4: Matrix MVP — Agent Context + Context Management — Context
**Gathered:** 2026-04-16
**Status:** Ready for planning
**Source:** Conversation context (2026-04-16 design session)
<domain>
## Phase Boundary
Привести Matrix-бот к рабочему состоянию для MVP-деплоя в контейнер:
- Убрать наш кастомный `AgentSessionClient` и thread_id патч из `platform-agent`, перейти на актуальный origin/main платформы с `AgentApi` из `lambda_agent_api`
- Добавить 4 команды управления контекстом агента: `!save`, `!load`, `!reset`, `!context`
- Упаковать Matrix-бот в Docker-контейнер
НЕ входит в фазу:
- Изменения в platform-agent (это задача команды платформы)
- Telegram адаптер
- E2EE
- Skills system (ждём платформу)
</domain>
<decisions>
## Implementation Decisions
### Архитектура платформы (locked)
- **Один контейнер = один чат**: `AgentService` с `thread_id = "default"` — намеренная архитектура. Изоляция на уровне контейнеров, не thread_id. Не менять.
- **Убрать thread_id патч**: наш коммит `1dca2c1` в `external/platform-agent` удаляем. Переходим на `origin/main` platform-agent.
- **Удалить `build_thread_key`**: функция больше не нужна. Убрать из `sdk/agent_session.py` и `sdk/real.py`.
- **Заменить `AgentSessionClient` на `AgentApi`**: использовать `AgentApi` из `external/platform-agent_api/lambda_agent_api/agent_api.py`. Он уже правильно обрабатывает все event-типы (неизвестные → `logger.warning`, без краша).
### !save (locked)
- Синтаксис: `!save` (автоимя по дате/времени) или `!save [имя]`
- Механизм: Matrix-бот посылает агенту текстовое сообщение "Summarize our conversation and save to /workspace/contexts/[name].md. Reply only with: Saved: [name]"
- Имена сохранений хранятся в `PrototypeStateStore` (список для `!load`)
- Агент сам пишет файл через свои инструменты (`write_file`)
### !load (locked)
- `!load` без аргументов → бот показывает нумерованный список сохранений
- Пользователь вводит **число** (1, 2, 3...) для выбора
- Выход из состояния: `0` или `!cancel`
- После выбора: бот посылает агенту "Load context from /workspace/contexts/[name].md and use it as background for our conversation. Reply: Loaded: [name]"
- Состояние ожидания выбора хранится в Matrix store (аналогично pending_confirm)
### !reset (locked)
- Показывает confirmation-диалог:
```
Сбросить контекст агента? Выбери:
!yes — сбросить
!save [имя] — сохранить и сбросить
!no — отмена
```
- `!yes` → вызвать `POST {AGENT_BASE_URL}/reset` (resets AgentService singleton)
- `!save имя` → сначала выполняется логика !save, затем POST /reset
- `!no` → отмена
- Fallback если `/reset` endpoint недоступен (404): вернуть пользователю "Reset endpoint недоступен. Обратитесь к администратору."
- `AGENT_BASE_URL` — новая env переменная (HTTP base URL агента, отдельно от `AGENT_WS_URL`)
### !context (locked)
- Показывает: имя текущей сессии (если загружали через `!load`), токены из последнего ответа (`tokens_used` из `MsgEventEnd`), список сохранений (имена + даты)
- Не делает никаких вызовов к агенту
### Dockerfile + docker-compose (locked)
- `Dockerfile` для Matrix-бота (`adapter/matrix/bot.py`)
- `docker-compose.yml` с сервисом `matrix-bot`
- Env переменные через `.env` файл
- Platform-agent запускается отдельно (не входит в compose этой фазы)
### Claude's Discretion
- Структура хранения saved sessions в PrototypeStateStore (dict name→timestamp)
- Формат автоимени для !save без аргументов
- HTTP клиент для POST /reset (aiohttp или httpx)
- Точный формат промптов к агенту для save/load
</decisions>
<canonical_refs>
## Canonical References
**Downstream agents MUST read these before planning or implementing.**
### Platform клиент (заменяем)
- `sdk/agent_session.py` — текущий AgentSessionClient, УДАЛЯЕМ/ЗАМЕНЯЕМ
- `sdk/real.py` — RealPlatformClient, обновляем под AgentApi
- `external/platform-agent_api/lambda_agent_api/agent_api.py` — новый клиент AgentApi
- `external/platform-agent_api/lambda_agent_api/server.py` — типы сообщений (MsgStatus, MsgEventTextChunk, MsgEventEnd, etc.)
- `external/platform-agent_api/lambda_agent_api/client.py` — MsgUserMessage
### Matrix адаптер (расширяем)
- `adapter/matrix/bot.py` — точка входа, MatrixBot, build_runtime
- `adapter/matrix/handlers/` — существующие обработчики команд
- `adapter/matrix/store.py` — get_room_meta, set_pending_confirm (паттерн для !load state)
- `sdk/prototype_state.py` — PrototypeStateStore, расширяем для saved sessions
### Состояние платформы
- `.planning/threads/matrix-dev-prototype-agent-platform-state.md` — исследование от 2026-04-14
### Существующая архитектура команд
- `core/protocol.py` — IncomingCommand, OutgoingMessage, OutgoingUI
- `core/handlers/` — паттерны регистрации обработчиков
</canonical_refs>
<specifics>
## Specific Ideas
- `AgentApi` требует явного `connect()` при старте и `close()` при завершении — lifecycle нужно встроить в `MatrixBot`
- `AgentApi.send_message()` — AsyncIterator, возвращает `MsgEventTextChunk` чанки и `MsgEventEnd`
- Для `!load` состояние "ожидаем число" хранить по ключу `load_pending:{matrix_user_id}:{room_id}` в store (аналог pending_confirm)
- `AGENT_BASE_URL` — HTTP URL, например `http://127.0.0.1:8000`; `AGENT_WS_URL` = `ws://127.0.0.1:8000/agent_ws/`
- platform-agent origin/main: `POST /reset` эндпоинта нет — это нужно запросить у команды платформы. До тех пор `!reset` возвращает "Reset endpoint недоступен"
</specifics>
<deferred>
## Deferred Ideas
- Замена `PrototypeStateStore` на реальный control-plane из platform-master (Phase 3)
- Skills интеграция через SkillsMiddleware (ждём платформу)
- E2EE для Matrix
- `!reset` через docker restart (заменяется на /reset endpoint когда платформа добавит)
- Суммаризация контекста (агент сам решает как писать в файл)
</deferred>
---
*Phase: 04-matrix-mvp-shared-agent-context-and-context-management-comma*
*Context gathered: 2026-04-16 via conversation design session*

View file

@ -0,0 +1,546 @@
# Phase 4: Matrix MVP — Shared Agent Context + Context Management — Research
**Researched:** 2026-04-16
**Domain:** Matrix bot, AgentApi WebSocket client, context management commands, Docker packaging
**Confidence:** HIGH (all findings verified against actual source files in this repo)
---
<user_constraints>
## User Constraints (from CONTEXT.md)
### Locked Decisions
**Архитектура платформы:**
- Один контейнер = один чат: `AgentService` с `thread_id = "default"` — намеренная архитектура. Не менять.
- Убрать thread_id патч: наш коммит `1dca2c1` в `external/platform-agent` удаляем. Переходим на `origin/main` platform-agent.
- Удалить `build_thread_key`: функция больше не нужна. Убрать из `sdk/agent_session.py` и `sdk/real.py`.
- Заменить `AgentSessionClient` на `AgentApi`: использовать `AgentApi` из `external/platform-agent_api/lambda_agent_api/agent_api.py`.
**!save:** Синтаксис `!save` (автоимя) или `!save [имя]`. Механизм: посылаем агенту текстовое сообщение. Имена сохранений хранятся в `PrototypeStateStore`.
**!load:** `!load` без аргументов → нумерованный список. Пользователь вводит число. Выход: `0` или `!cancel`. После выбора — посылаем агенту текстовое сообщение. Состояние ожидания в Matrix store.
**!reset:** Confirmation-диалог с `!yes`/`!save имя`/`!no`. `!yes``POST {AGENT_BASE_URL}/reset`. Fallback если 404: сообщение пользователю.
**!context:** Показывает имя сессии, токены, список сохранений. Без вызовов агента.
**Dockerfile + docker-compose:** Для Matrix-бота. Env через `.env`. Platform-agent — отдельно.
### Claude's Discretion
- Структура хранения saved sessions в PrototypeStateStore (dict name→timestamp)
- Формат автоимени для !save без аргументов
- HTTP клиент для POST /reset (aiohttp или httpx)
- Точный формат промптов к агенту для save/load
### Deferred Ideas (OUT OF SCOPE)
- Замена `PrototypeStateStore` на реальный control-plane из platform-master
- Skills интеграция через SkillsMiddleware
- E2EE для Matrix
- `!reset` через docker restart
- Суммаризация контекста
</user_constraints>
---
## Summary
Phase 4 replaces the custom `AgentSessionClient` with the production `AgentApi` from `lambda_agent_api`, adds four context management commands to the Matrix bot, and packages it in Docker. All findings are verified directly against source files.
**Primary recommendation:** Wire `AgentApi` as a persistent connection in `MatrixBot.__init__` (connect on start, close in finally block of `main()`). Expose it through `RealPlatformClient`. The four commands follow the existing handler registration pattern in `adapter/matrix/handlers/__init__.py`.
The `platform-agent` at `origin/main` already works with `AgentApi` — it does NOT require `thread_id` query param. Our local patch (`1dca2c1`) must be discarded and `external/platform-agent` reset to `origin/main`.
---
## Project Constraints (from CLAUDE.md)
- **Tech stack:** matrix-nio for Matrix — do not change without discussion
- **Platform client:** connected only via `sdk/interface.py` Protocol — core/ and adapters untouched when swapping implementation
- **No E2EE** — matrix-nio without python-olm
- **Hotfixes < 20 lines** → Claude Code directly; implementation → Codex via GSD
- **MATRIX_PLATFORM_BACKEND env var** controls mock vs real
---
## Standard Stack
### Core (verified)
| Library | Version | Purpose | Source |
|---------|---------|---------|--------|
| `lambda_agent_api` | local (external/platform-agent_api) | AgentApi WebSocket client | [VERIFIED: file read] |
| `aiohttp` | >=3.9 (surfaces-bot), >=3.13.4 (agent_api) | WebSocket transport inside AgentApi | [VERIFIED: pyproject.toml] |
| `pydantic` | >=2.5 | Message serialization (MsgUserMessage, MsgEventEnd, etc.) | [VERIFIED: server.py/client.py] |
| `httpx` | >=0.27 | HTTP client for POST /reset (already in deps) | [VERIFIED: pyproject.toml] |
| `structlog` | >=24.1 | Logging (existing pattern) | [VERIFIED: pyproject.toml] |
### Supporting
| Library | Version | Purpose | When to Use |
|---------|---------|---------|-------------|
| `aiohttp` | already a dep | Alternative HTTP for POST /reset | Could use instead of httpx — both available |
**Installation:** No new packages needed. `lambda_agent_api` is installed as a local path package (currently accessed via `sys.path` injection in tests; for production use, add to pyproject.toml as path dep or install via `pip install -e external/platform-agent_api`).
**Critical:** `lambda_agent_api` requires Python >=3.14 per its own `pyproject.toml`. The surfaces-bot requires Python >=3.11. [VERIFIED: pyproject.toml of both]. This is a **version mismatch** — see Pitfalls.
---
## Architecture Patterns
### AgentApi Constructor (verified)
```python
# Source: external/platform-agent_api/lambda_agent_api/agent_api.py
AgentApi(
agent_id: str, # arbitrary string ID, used in logs
url: str, # WebSocket URL, e.g. "ws://127.0.0.1:8000/agent_ws/"
callback: Optional[Callable[[ServerMessage], None]] = None, # for orphaned msgs
on_disconnect: Optional[Callable[['AgentApi'], None]] = None # called on WS close
)
```
### AgentApi Lifecycle (verified)
```python
# Source: external/platform-agent_api/lambda_agent_api/agent_api.py
agent = AgentApi(agent_id="matrix-bot", url=ws_url)
await agent.connect() # opens WS, waits for MsgStatus, starts _listen() task
# ... use agent ...
await agent.close() # cancels _listen task, closes WS and session
```
`connect()` blocks until `MsgStatus` is received from server (5s timeout). After `connect()`, a background `_listen()` asyncio task runs continuously, routing server messages to an internal `asyncio.Queue`.
### AgentApi.send_message() semantics (verified)
```python
# Source: external/platform-agent_api/lambda_agent_api/agent_api.py, line 134
async def send_message(self, text: str) -> AsyncIterator[AgentEventUnion]:
```
- `AgentEventUnion = Union[MsgEventTextChunk, MsgEventEnd]`**but** the generator `yield`s only `MsgEventTextChunk` chunks; it `break`s (stops) on `MsgEventEnd` without yielding it.
- `MsgEventEnd` carries `tokens_used: int` — to capture this, the caller must intercept the queue or handle `MsgEventEnd` in the `_listen` loop. **Currently `send_message` discards `tokens_used`.** This affects `!context` which needs tokens.
**Resolution:** In `RealPlatformClient.stream_message()`, after iterating through `send_message()`, `tokens_used` won't be directly available. Options:
1. Store `tokens_used` in a shared attribute after each response (add `self._last_tokens_used` to `AgentApi` or a wrapper).
2. Use the `callback` parameter to capture `MsgEventEnd` events from the `_listen` loop.
[ASSUMED] The simplest approach: wrap `AgentApi` in a thin `AgentApiAdapter` class that intercepts `_listen` output and exposes `last_tokens_used`. Or: store tokens in `PrototypeStateStore` after each message.
### AgentApi concurrency constraint (verified)
`AgentApi._request_lock` prevents parallel `send_message()` calls — second call raises `AgentBusyException`. In the single-user Matrix prototype this is acceptable. The bot must not dispatch two messages concurrently to the same agent.
### Wiring AgentApi into MatrixBot (integration pattern)
The `AgentApi` must be a persistent connection (not per-message connect/disconnect) because:
1. `_listen()` task runs in background and routes server push events.
2. Per-message connect/disconnect would recreate the aiohttp session each time and discard LangGraph thread state.
**Recommended wiring:**
```python
# adapter/matrix/bot.py — main() function
agent_api = AgentApi(agent_id="matrix-bot", url=ws_url)
await agent_api.connect()
runtime = build_runtime(store=SQLiteStore(db_path), client=client, agent_api=agent_api)
try:
await client.sync_forever(timeout=30000, since=since_token)
finally:
await client.close()
await agent_api.close()
```
`_build_platform_from_env()` currently instantiates everything synchronously. It must be refactored to `async` or split: construct `AgentApi` synchronously, call `connect()` in `main()` before starting sync loop.
### RealPlatformClient updates
`RealPlatformClient` currently imports `AgentSessionClient` and calls `build_thread_key`. Both are removed. The updated class:
```python
class RealPlatformClient(PlatformClient):
def __init__(
self,
agent_api: AgentApi, # replaces agent_sessions: AgentSessionClient
prototype_state: PrototypeStateStore,
platform: str = "matrix",
) -> None:
```
`send_message()` and `stream_message()` call `agent_api.send_message(text)` directly — no `thread_key` needed.
### platform-agent origin/main: what changes (verified)
Our patch `1dca2c1` added `thread_id` query param handling to `external/platform-agent/src/api/external.py`. On `origin/main`, the `process_message()` function does NOT use `thread_id` — it calls `agent_service.astream(msg.text)` without `thread_id`. The WS URL becomes simply `ws://host:port/agent_ws/` — no query params.
### Existing command registration pattern (verified)
```python
# adapter/matrix/handlers/__init__.py — register_matrix_handlers()
dispatcher.register(IncomingCommand, "new", make_handle_new_chat(client, store))
dispatcher.register(IncomingCommand, "settings", handle_settings)
dispatcher.register(IncomingCallback, "confirm", make_handle_confirm(store))
```
Handler signature (all existing handlers follow this):
```python
async def handle_X(
event: IncomingCommand,
auth_mgr,
platform,
chat_mgr,
settings_mgr,
) -> list[OutgoingEvent]:
```
New context commands need access to `agent_api` (for `!save`, `!load`) and `store` (for `!context`, `!load` pending state). Pattern: use `make_handle_X(agent_api, store)` closures — same as `make_handle_new_chat(client, store)`.
### !load pending state pattern (verified)
Existing `PENDING_CONFIRM_PREFIX = "matrix_pending_confirm:"` in `adapter/matrix/store.py`.
New key for load pending state:
```python
LOAD_PENDING_PREFIX = "matrix_load_pending:"
def _load_pending_key(user_id: str, room_id: str) -> str:
return f"{LOAD_PENDING_PREFIX}{user_id}:{room_id}"
```
Stored data structure:
```python
{
"saves": [{"name": "my-save", "ts": "2026-04-16T12:00:00Z"}, ...],
"display": "1. my-save (2026-04-16)\n2. other..."
}
```
The numeric input `1`, `2`, etc. is intercepted in `MatrixBot.on_room_message()` BEFORE dispatching as `IncomingMessage` — check if `load_pending` exists for this user+room, resolve to save name, dispatch the load command internally.
**Alternative (recommended):** Handle numeric input in the `IncomingMessage` handler via a pre-dispatch interceptor, or register a special numeric-input check in the dispatcher for messages that are pure integers.
### !reset confirmation dialog pattern
!reset reuses the `OutgoingUI` + `pending_confirm` mechanism or a simpler custom state. Since the dialog options are `!yes`, `!save имя`, `!no` (not just yes/no), it cannot reuse `pending_confirm` directly without extension.
Simplest approach: store `reset_pending:{user_id}:{room_id}` key (boolean) and check for `!yes`/`!no`/`!save` commands from the `IncomingCommand` dispatcher when reset_pending is set.
### saved sessions storage in PrototypeStateStore
New dict attribute on `PrototypeStateStore`:
```python
self._saved_sessions: dict[str, list[dict]] = {}
# Key: matrix_user_id
# Value: [{"name": "my-save", "created_at": "2026-04-16T12:00:00Z"}, ...]
```
Methods to add:
```python
async def add_saved_session(self, user_id: str, name: str) -> None: ...
async def list_saved_sessions(self, user_id: str) -> list[dict]: ...
```
### !context tokens_used tracking
`MsgEventEnd.tokens_used: int` is available from `server.py`. Since `AgentApi.send_message()` drops it, the planner must decide how to surface it. Recommended: store in `PrototypeStateStore` as `_last_tokens_used: dict[str, int]` keyed by user_id, updated after each successful agent response in `RealPlatformClient`.
### Prompts for !save / !load (Claude's Discretion)
```python
# !save
SAVE_PROMPT = (
"Summarize our conversation and save to /workspace/contexts/{name}.md. "
"Reply only with: Saved: {name}"
)
# !load
LOAD_PROMPT = (
"Load context from /workspace/contexts/{name}.md and use it as background "
"for our conversation. Reply: Loaded: {name}"
)
```
Auto-name format (Claude's Discretion): `context-{YYYYMMDD-HHMMSS}` (UTC, no spaces, no special chars, safe as filename).
### POST /reset endpoint
Confirmed absent in `origin/main` platform-agent. Only endpoint is `GET /agent_ws/` (WebSocket). The `main.py` has no HTTP routes beyond what FastAPI provides by default (`/docs`, `/openapi.json`).
`!reset` with `!yes``POST {AGENT_BASE_URL}/reset` → expect 404 → return "Reset endpoint недоступен. Обратитесь к администратору."
HTTP client for this: **httpx** (already in `pyproject.toml`):
```python
import httpx
async with httpx.AsyncClient() as client:
response = await client.post(f"{agent_base_url}/reset", timeout=5.0)
if response.status_code == 404:
return [OutgoingMessage(chat_id=..., text="Reset endpoint недоступен...")]
```
### Dockerfile
```dockerfile
FROM python:3.11-slim
WORKDIR /app
COPY pyproject.toml .
RUN pip install -e .
COPY . .
ENV PYTHONUNBUFFERED=1
CMD ["python", "-m", "adapter.matrix.bot"]
```
`lambda_agent_api` must be installed in the container. Options:
1. `COPY external/platform-agent_api /app/external/platform-agent_api` + `pip install -e /app/external/platform-agent_api`
2. Include `lambda_agent_api` package directly in `surfaces-bot` package (copy source files)
Option 1 is cleaner.
### docker-compose.yml structure
```yaml
services:
matrix-bot:
build: .
env_file: .env
restart: unless-stopped
```
Platform-agent runs separately — not in this compose file.
---
## Don't Hand-Roll
| Problem | Don't Build | Use Instead | Why |
|---------|-------------|-------------|-----|
| WebSocket lifecycle with reconnect | Custom WS manager | `AgentApi` from `lambda_agent_api` | Already handles connect/close/listen loop, error routing, queue management |
| Message deserialization | Custom JSON parsing | `ServerMessage.validate_json()` (Pydantic TypeAdapter) | Discriminated union handles all message types |
| HTTP async client | `aiohttp.ClientSession` directly | `httpx.AsyncClient` | Already in deps, cleaner API for one-shot POST |
| Concurrent request guard | Custom lock | `AgentApi._request_lock` | Already implemented, raises `AgentBusyException` |
---
## Common Pitfalls
### Pitfall 1: lambda_agent_api Python version mismatch
**What goes wrong:** `lambda_agent_api/pyproject.toml` declares `requires-python = ">=3.14"`. The surfaces-bot runs on Python 3.11+. If `pip install -e external/platform-agent_api` is run with Python 3.11 it may fail or emit warnings.
**Why it happens:** The `lambda_agent_api` was developed under Python 3.14 (seen in `.venv` path: `python3.14`). The code itself uses no 3.14-specific syntax — it is pure aiohttp + pydantic which run on 3.11.
**How to avoid:** Change `requires-python = ">=3.11"` in `external/platform-agent_api/pyproject.toml` before building the Docker image, or install with `--ignore-requires-python`. Alternatively, copy the three source files directly into the surfaces-bot package.
**Warning signs:** `pip install` failure with "requires Python >=3.14".
### Pitfall 2: AgentApi.send_message() drops MsgEventEnd (tokens_used lost)
**What goes wrong:** The generator yields only `MsgEventTextChunk` objects and breaks on `MsgEventEnd` without yielding it. Any downstream code that tries to get `tokens_used` from the iterator gets nothing.
**Why it happens:** The generator `break`s on `MsgEventEnd` (line 172 of agent_api.py) without yielding it. This is intentional for streaming UX but loses token info.
**How to avoid:** Before streaming, set `self._last_tokens_used = 0`. In `_listen()`, `MsgEventEnd` is put into `_current_queue` (line 241). The `send_message()` generator reads from that queue but does `break` — the `MsgEventEnd` object is consumed but not returned to caller. The only way to capture it is to subclass `AgentApi` or read from `_current_queue` directly before the break.
**Practical fix:** Add `self.last_tokens_used: int = 0` to `AgentApi` and intercept the queue in the `finally` block of `send_message()` — or store it in a wrapper class.
### Pitfall 3: AgentApi persistent connection vs sync_forever loop
**What goes wrong:** If `agent_api.connect()` is called inside `_build_platform_from_env()` (sync function), it creates an `asyncio.Task` for `_listen()` outside the event loop context.
**Why it happens:** `_build_platform_from_env()` is called synchronously from `build_runtime()`. `connect()` is a coroutine.
**How to avoid:** Do NOT call `agent_api.connect()` inside `_build_platform_from_env()`. Instead:
1. `_build_platform_from_env()` creates `RealPlatformClient` with an unconnected `AgentApi`
2. `main()` awaits `agent_api.connect()` explicitly after constructing runtime
Expose `agent_api` from `RealPlatformClient` via a property so `main()` can call `connect()` on it.
### Pitfall 4: !load numeric input interception
**What goes wrong:** When user types `1` in response to `!load` menu, it is dispatched as `IncomingMessage` (not a command) and routed to the platform — the agent receives "1" as a user message.
**Why it happens:** The Matrix converter (`from_room_event`) produces `IncomingMessage` for plain text, `IncomingCommand` only for `!`-prefixed text.
**How to avoid:** In `MatrixBot.on_room_message()`, before calling `dispatcher.dispatch()`, check if `load_pending` state exists for this user+room. If yes and the message text is a digit (or `0`/`!cancel`), handle it as a load selection instead of routing to agent.
### Pitfall 5: platform-agent thread_id removal breaks existing tests
**What goes wrong:** `tests/platform/test_agent_session.py` imports `build_thread_key` and tests `process_message` with `thread_id` in query params. After the patch is removed, these tests will fail.
**Why it happens:** Tests were written against our patched `external.py`.
**How to avoid:** The plan must include updating `test_agent_session.py` — remove `build_thread_key` tests, update `process_message` tests to reflect origin/main signature (no `thread_id` param).
### Pitfall 6: !reset dialog conflicts with existing !yes/!no flow
**What goes wrong:** The existing `pending_confirm` flow uses `!yes`/`!no`. If both `reset_pending` and `pending_confirm` are active simultaneously, `!yes` could trigger the wrong handler.
**Why it happens:** Both flows listen for the same commands.
**How to avoid:** `!reset` dialog uses a separate state key `reset_pending:{user_id}:{room_id}`. The handler for `!yes` must check `reset_pending` first, then `pending_confirm`. Document priority in handler code.
---
## Code Examples
### Invoking AgentApi.send_message() in stream_message
```python
# Source: external/platform-agent_api/lambda_agent_api/agent_api.py
async def stream_message(self, user_id: str, chat_id: str, text: str, ...) -> AsyncIterator[MessageChunk]:
async for event in self._agent_api.send_message(text):
if isinstance(event, MsgEventTextChunk):
yield MessageChunk(
message_id=user_id,
delta=event.text,
finished=False,
)
# After loop ends, MsgEventEnd was consumed internally
yield MessageChunk(message_id=user_id, delta="", finished=True, tokens_used=self._agent_api.last_tokens_used)
```
### Handler registration pattern
```python
# Source: adapter/matrix/handlers/__init__.py
def register_matrix_handlers(dispatcher: EventDispatcher, client=None, store=None, agent_api=None) -> None:
# existing...
dispatcher.register(IncomingCommand, "save", make_handle_save(agent_api, store))
dispatcher.register(IncomingCommand, "load", make_handle_load(agent_api, store))
dispatcher.register(IncomingCommand, "reset", make_handle_reset(store))
dispatcher.register(IncomingCommand, "context", make_handle_context(store))
```
### !load pending key
```python
# New in adapter/matrix/store.py
LOAD_PENDING_PREFIX = "matrix_load_pending:"
async def get_load_pending(store: StateStore, user_id: str, room_id: str) -> dict | None:
return await store.get(f"{LOAD_PENDING_PREFIX}{user_id}:{room_id}")
async def set_load_pending(store: StateStore, user_id: str, room_id: str, data: dict) -> None:
await store.set(f"{LOAD_PENDING_PREFIX}{user_id}:{room_id}", data)
async def clear_load_pending(store: StateStore, user_id: str, room_id: str) -> None:
await store.delete(f"{LOAD_PENDING_PREFIX}{user_id}:{room_id}")
```
### platform-agent origin/main process_message (no thread_id)
```python
# Source: git show origin/main:src/api/external.py in external/platform-agent
async def process_message(ws: WebSocket, msg, agent_service: AgentService):
match msg:
case MsgUserMessage():
async for chunk in agent_service.astream(msg.text): # no thread_id arg
await ws.send_text(chunk.model_dump_json())
await ws.send_text(MsgEventEnd(tokens_used=0).model_dump_json())
```
---
## Assumptions Log
| # | Claim | Section | Risk if Wrong |
|---|-------|---------|---------------|
| A1 | `tokens_used` can be captured by storing in `AgentApi.last_tokens_used` attribute during `_listen()` before it's queued | Architecture Patterns | If `_listen` timing means value is read before queue, token count would be wrong — low risk, easy to test |
| A2 | Python 3.11 can run `lambda_agent_api` despite `>=3.14` constraint in pyproject.toml | Standard Stack | If code uses 3.14-specific syntax, would fail at runtime — actual code inspected: no 3.14 syntax found |
| A3 | httpx is preferred over aiohttp for POST /reset (one-shot HTTP) | Standard Stack | Either works; httpx already in deps |
---
## Open Questions
1. **tokens_used capture from AgentApi**
- What we know: `MsgEventEnd.tokens_used` is put into `_current_queue` but consumed (not yielded) by `send_message()` generator
- What's unclear: Cleanest interception point without modifying `lambda_agent_api` source
- Recommendation: Add `last_tokens_used: int = 0` attribute to `AgentApi` and set it in `send_message()`'s `finally` block when draining orphan queue, OR set it in `_listen()` before putting `MsgEventEnd` in queue
2. **!load numeric input dispatch**
- What we know: Plain text `1`, `2` arrives as `IncomingMessage`, not `IncomingCommand`
- What's unclear: Where to intercept — in `on_room_message()` (bot layer) or in dispatcher pre-hook
- Recommendation: Intercept in `MatrixBot.on_room_message()` before `dispatcher.dispatch()`. Keeps dispatcher clean.
3. **lambda_agent_api install in Docker**
- What we know: It's a local package in `external/platform-agent_api/`
- What's unclear: Whether to install as editable or copy sources
- Recommendation: `COPY external/platform-agent_api /build/lambda_agent_api && pip install /build/lambda_agent_api` in Dockerfile
---
## Environment Availability
| Dependency | Required By | Available | Version | Fallback |
|------------|-------------|-----------|---------|----------|
| Python 3.11+ | All | ✓ | System | — |
| aiohttp | AgentApi WS | ✓ | >=3.9 in deps | — |
| httpx | POST /reset | ✓ | >=0.27 in deps | aiohttp |
| matrix-nio | Matrix bot | ✓ | >=0.21 in deps | — |
| lambda_agent_api | AgentApi | local only | 0.1.0 | — |
| Docker | Container build | [ASSUMED] standard dev env | — | — |
| platform-agent (running) | Integration test | local clone | origin/main needed | — |
---
## Validation Architecture
### Test Framework
| Property | Value |
|----------|-------|
| Framework | pytest + pytest-asyncio (asyncio_mode = "auto") |
| Config file | pyproject.toml `[tool.pytest.ini_options]` |
| Quick run command | `pytest tests/platform/test_real.py tests/adapter/matrix/test_dispatcher.py -v` |
| Full suite command | `pytest tests/ -v` |
### Phase Requirements → Test Map
| Req | Behavior | Test Type | File |
|-----|----------|-----------|------|
| Remove build_thread_key | Function gone from sdk/ | unit | `tests/platform/test_agent_session.py` — update/remove |
| AgentApi replaces AgentSessionClient | `RealPlatformClient` uses `AgentApi` | unit | `tests/platform/test_real.py` — update |
| !save sends prompt to agent | Command dispatches agent message | unit | `tests/adapter/matrix/test_dispatcher.py` — add |
| !load shows list | Command returns numbered list | unit | `tests/adapter/matrix/test_dispatcher.py` — add |
| !load numeric select | Bot intercepts digit, sends load prompt | unit | `tests/adapter/matrix/test_dispatcher.py` — add |
| !reset shows dialog | Command returns confirmation UI | unit | `tests/adapter/matrix/test_dispatcher.py` — add |
| !context returns snapshot | Command returns session info | unit | `tests/adapter/matrix/test_dispatcher.py` — add |
| PrototypeStateStore saved sessions | add/list saved sessions | unit | `tests/platform/test_prototype_state.py` — add |
### Wave 0 Gaps
- [ ] `tests/platform/test_agent_api_integration.py` — unit tests for `RealPlatformClient` with mocked `AgentApi`
- [ ] `tests/adapter/matrix/test_context_commands.py` — dedicated module for !save/!load/!reset/!context handlers
---
## Sources
### Primary (HIGH confidence — verified by file read in this session)
- `external/platform-agent_api/lambda_agent_api/agent_api.py` — AgentApi constructor, connect/close/send_message, _listen loop
- `external/platform-agent_api/lambda_agent_api/server.py` — MsgEventTextChunk, MsgEventEnd, MsgStatus, AgentEventUnion types
- `external/platform-agent_api/lambda_agent_api/client.py` — MsgUserMessage type
- `external/platform-agent/src/api/external.py` — current (patched) and origin/main versions verified via git show
- `adapter/matrix/handlers/__init__.py` — handler registration pattern
- `adapter/matrix/store.py` — pending_confirm key pattern
- `adapter/matrix/bot.py` — MatrixBot, build_runtime, _build_platform_from_env
- `sdk/agent_session.py` — current AgentSessionClient (to be replaced)
- `sdk/real.py` — RealPlatformClient (to be updated)
- `sdk/prototype_state.py` — PrototypeStateStore (to be extended)
- `core/protocol.py` — IncomingCommand, OutgoingMessage types
- `pyproject.toml` — dependency versions
- `external/platform-agent_api/pyproject.toml` — Python version constraint
### Tertiary (LOW confidence)
- Docker best practices for Python apps [ASSUMED] — standard industry pattern
---
## Metadata
**Confidence breakdown:**
- AgentApi interface: HIGH — read source directly
- platform-agent origin/main diff: HIGH — verified via `git show origin/main`
- handler registration pattern: HIGH — read all handler files
- pending_confirm key pattern: HIGH — read store.py directly
- tokens_used interception: MEDIUM — pattern clear but implementation needs care
- Docker/docker-compose: MEDIUM — standard pattern, not verified against specific matrix-nio requirements
**Research date:** 2026-04-16
**Valid until:** 2026-05-16 (lambda_agent_api is local — stable until platform team updates it)

View file

@ -0,0 +1,158 @@
---
phase: 05-mvp-deployment
plan: 01
type: execute
wave: 1
depends_on: []
files_modified:
- adapter/matrix/reconciliation.py
- adapter/matrix/bot.py
- tests/adapter/matrix/test_reconciliation.py
- tests/adapter/matrix/test_restart_persistence.py
autonomous: true
requirements:
- PH05-01
- PH05-03
must_haves:
truths:
- "On restart, existing Matrix Space and child-room topology is rebuilt before live sync begins."
- "Restart recovery preserves Space+rooms UX instead of creating duplicate DM-style working rooms."
- "Recovered rooms regain user metadata, room metadata, and chat bindings needed for normal routing."
- "Legacy working rooms missing `platform_chat_id` are backfilled deterministically during startup before strict routing handles traffic."
artifacts:
- path: "adapter/matrix/reconciliation.py"
provides: "Authoritative restart reconciliation from Matrix topology into local metadata"
- path: "adapter/matrix/bot.py"
provides: "Startup wiring that runs reconciliation before sync_forever"
- path: "tests/adapter/matrix/test_reconciliation.py"
provides: "Regression coverage for startup recovery and idempotence"
key_links:
- from: "adapter/matrix/bot.py"
to: "adapter/matrix/reconciliation.py"
via: "startup bootstrap before sync_forever"
pattern: "reconcil"
- from: "adapter/matrix/reconciliation.py"
to: "core/chat.py"
via: "chat manager rebuild for recovered rooms"
pattern: "get_or_create"
---
<objective>
Rebuild Matrix-local routing state from authoritative Space topology before the bot processes live traffic.
Purpose: Preserve the Phase 01 Space+rooms contract after restart even if SQLite metadata is partial or missing.
Output: A startup reconciliation module, bot wiring, and regression tests proving no DM-first duplication on restart.
</objective>
<execution_context>
@/Users/a/.codex/get-shit-done/workflows/execute-plan.md
@/Users/a/.codex/get-shit-done/templates/summary.md
</execution_context>
<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/05-mvp-deployment/05-RESEARCH.md
@.planning/phases/05-mvp-deployment/05-VALIDATION.md
@.planning/phases/04-matrix-mvp-shared-agent-context-and-context-management-comma/04-02-SUMMARY.md
@adapter/matrix/bot.py
@adapter/matrix/store.py
@adapter/matrix/handlers/auth.py
@tests/adapter/matrix/test_invite_space.py
@tests/adapter/matrix/test_chat_space.py
@tests/adapter/matrix/test_restart_persistence.py
<interfaces>
From `adapter/matrix/bot.py`:
```python
async def prepare_live_sync(client: AsyncClient) -> str | None:
response = await client.sync(timeout=0, full_state=True)
if isinstance(response, SyncResponse):
return response.next_batch
return None
```
```python
class MatrixBot:
async def _bootstrap_unregistered_room(
self,
room: MatrixRoom,
sender: str,
) -> list[OutgoingEvent] | None: ...
```
From `adapter/matrix/store.py`:
```python
async def get_room_meta(store: StateStore, room_id: str) -> dict | None: ...
async def set_room_meta(store: StateStore, room_id: str, meta: dict) -> None: ...
async def get_user_meta(store: StateStore, matrix_user_id: str) -> dict | None: ...
async def set_user_meta(store: StateStore, matrix_user_id: str, meta: dict) -> None: ...
async def next_platform_chat_id(store: StateStore) -> str: ...
```
</interfaces>
</context>
<tasks>
<task type="auto" tdd="true">
<name>Task 1: Add restart reconciliation regression coverage</name>
<files>tests/adapter/matrix/test_reconciliation.py, tests/adapter/matrix/test_restart_persistence.py</files>
<read_first>tests/adapter/matrix/test_invite_space.py, tests/adapter/matrix/test_chat_space.py, tests/adapter/matrix/test_restart_persistence.py, adapter/matrix/bot.py, adapter/matrix/handlers/auth.py, .planning/phases/05-mvp-deployment/05-RESEARCH.md</read_first>
<behavior>
- Test 1: startup recovery rebuilds user space metadata, room metadata, and chat bindings from Matrix topology without creating new working rooms (per D-Phase05-reset and PH05-01).
- Test 2: reconciliation is idempotent and safe when local SQLite state is already present.
- Test 3: reconciliation happens before lazy `_bootstrap_unregistered_room()` would run for existing rooms (per PH05-03).
- Test 4: legacy room metadata missing `platform_chat_id` is backfilled deterministically at startup and persisted before routed handling begins.
</behavior>
<acceptance_criteria>
- `tests/adapter/matrix/test_reconciliation.py` exists and names reconciliation entrypoints explicitly.
- The new tests assert restored `space_id`, `chat_id`, `matrix_user_id`, and `platform_chat_id` values for recovered rooms.
- The regression slice also proves existing Space onboarding behavior still passes by running `test_invite_space.py` and `test_chat_space.py`.
- The automated command in `<verify>` fails before implementation or would fail if reconciliation is removed.
</acceptance_criteria>
<action>Create a dedicated `tests/adapter/matrix/test_reconciliation.py` module and extend restart persistence coverage so Phase 05 has a real Wave 0 contract. Model the recovered topology after the Phase 01 Space+rooms onboarding tests, not a DM-first flow, and explicitly keep those onboarding regressions in the verification slice so restart hardening cannot break provisioning UX. Cover recovery of `user_meta`, `room_meta`, `ChatManager` bindings, and room-local routing fields from Matrix-side state before live callbacks begin, including deterministic backfill for legacy rooms that predate `platform_chat_id`. Keep temporary UX state out of scope, per research.</action>
<verify>
<automated>pytest tests/adapter/matrix/test_invite_space.py tests/adapter/matrix/test_chat_space.py tests/adapter/matrix/test_reconciliation.py tests/adapter/matrix/test_restart_persistence.py -v</automated>
</verify>
<done>Phase 05 has failing-or-red-before-code tests that define authoritative restart reconciliation behavior and exclude duplicate room provisioning.</done>
</task>
<task type="auto" tdd="true">
<name>Task 2: Implement authoritative startup reconciliation and wire it before live sync</name>
<files>adapter/matrix/reconciliation.py, adapter/matrix/bot.py</files>
<read_first>adapter/matrix/bot.py, adapter/matrix/store.py, adapter/matrix/handlers/auth.py, tests/adapter/matrix/test_reconciliation.py, tests/adapter/matrix/test_restart_persistence.py, .planning/phases/05-mvp-deployment/05-RESEARCH.md</read_first>
<behavior>
- Test 1: startup rebuild runs after login and initial full-state fetch, but before `sync_forever()` processes live events.
- Test 2: recovered rooms keep their existing Space+rooms identity and do not trigger `_bootstrap_unregistered_room()` unless the room is genuinely new.
- Test 3: local metadata can be rebuilt from Matrix topology when SQLite entries are missing, while existing valid metadata remains stable.
- Test 4: startup repair assigns a deterministic `platform_chat_id` to legacy rooms missing that field and persists it before routed platform calls can occur.
</behavior>
<acceptance_criteria>
- `adapter/matrix/reconciliation.py` exports a focused reconciliation entrypoint used by startup code.
- `adapter/matrix/bot.py` invokes reconciliation before `client.sync_forever(...)`.
- Recovered room metadata includes `room_type`, `chat_id`, `space_id`, `matrix_user_id`, and `platform_chat_id` where available or rebuildable.
- Legacy rooms missing `platform_chat_id` follow one documented startup backfill path rather than ad hoc routing fallbacks.
</acceptance_criteria>
<action>Implement a restart recovery module that treats Matrix topology as authoritative, per the Phase 05 reset and research notes. Rebuild missing local metadata for Space-owned working rooms, deterministically backfill missing `platform_chat_id` values for legacy rooms, and re-create `ChatManager` entries needed by routing, while keeping SQLite as a rebuildable cache rather than the source of truth. Wire the new reconciliation step into startup after the initial full-state sync and before live sync begins, and keep the onboarding regression slice green while doing it. Do not widen into timeline scraping, new storage backends, or DM-first fallbacks.</action>
<verify>
<automated>pytest tests/adapter/matrix/test_invite_space.py tests/adapter/matrix/test_chat_space.py tests/adapter/matrix/test_reconciliation.py tests/adapter/matrix/test_restart_persistence.py tests/adapter/matrix/test_dispatcher.py -v</automated>
</verify>
<done>Restart recovery restores the minimum durable state for existing Space rooms before live traffic, and the guarded regression suite passes.</done>
</task>
</tasks>
<verification>
Run the onboarding, reconciliation, restart-persistence, and Matrix dispatcher slices together. Confirm startup now has a deterministic pre-sync recovery and legacy-room backfill step instead of relying on lazy room bootstrap or routing-time fallbacks for existing topology.
</verification>
<success_criteria>
The bot can restart with partial or empty local room metadata, rebuild managed Space rooms before live sync, and continue handling those rooms without creating duplicate onboarding rooms.
</success_criteria>
<output>
After completion, create `.planning/phases/05-mvp-deployment/05-01-SUMMARY.md`
</output>

View file

@ -0,0 +1,99 @@
---
phase: 05-mvp-deployment
plan: 01
subsystem: infra
tags: [matrix, reconciliation, sqlite, startup, testing]
requires:
- phase: 01-matrix-mvp
provides: Space+rooms onboarding, room metadata, and Matrix dispatcher behavior
- phase: 04-matrix-mvp-shared-agent-context-and-context-management-comma
provides: durable platform_chat_id and restart persistence primitives
provides:
- authoritative startup reconciliation from Matrix room topology into local metadata
- pre-sync startup wiring that repairs managed rooms before live traffic
- restart regression coverage for reconciliation, idempotence, and legacy platform_chat_id backfill
affects: [matrix, startup, deployment, restart-persistence]
tech-stack:
added: []
patterns: [matrix-topology-as-source-of-truth, sqlite-cache-rebuild, pre-sync-reconciliation]
key-files:
created: [adapter/matrix/reconciliation.py, tests/adapter/matrix/test_reconciliation.py]
modified: [adapter/matrix/bot.py, tests/adapter/matrix/test_restart_persistence.py]
key-decisions:
- "Treat synced Matrix parent/child topology as authoritative for managed room recovery; keep SQLite rebuildable."
- "Backfill missing platform_chat_id values during startup reconciliation instead of routing-time fallbacks."
patterns-established:
- "Startup runs full-state sync, then reconciliation, then sync_forever."
- "Recovered Matrix rooms rebuild user_meta, room_meta, auth state, and ChatManager bindings idempotently."
requirements-completed: [PH05-01, PH05-03]
duration: 8min
completed: 2026-04-27
---
# Phase 05 Plan 01: Restart Reconciliation Summary
**Matrix startup now rebuilds Space-owned working rooms into durable local routing state before live sync begins**
## Performance
- **Duration:** 8 min
- **Started:** 2026-04-27T22:00:47Z
- **Completed:** 2026-04-27T22:08:47Z
- **Tasks:** 2
- **Files modified:** 4
## Accomplishments
- Added a dedicated reconciliation module that restores `user_meta`, `room_meta`, auth state, chat bindings, and missing `platform_chat_id` values from the synced Matrix room graph.
- Wired startup to run reconciliation immediately after the initial full-state sync and before `sync_forever()`.
- Added regression coverage for recovery, idempotence, pre-sync ordering, onboarding compatibility, and legacy restart backfill.
## Task Commits
Each task was committed atomically:
1. **Task 1: Add restart reconciliation regression coverage** - `a75b26a` (test)
2. **Task 2: Implement authoritative startup reconciliation and wire it before live sync** - `8a80d00` (feat)
## Files Created/Modified
- `adapter/matrix/reconciliation.py` - Startup recovery from Matrix topology into local room and user metadata.
- `adapter/matrix/bot.py` - Startup wiring that runs reconciliation after the bootstrap sync and before live sync.
- `tests/adapter/matrix/test_reconciliation.py` - Recovery, idempotence, and startup-order regression coverage.
- `tests/adapter/matrix/test_restart_persistence.py` - Legacy `platform_chat_id` backfill persistence coverage.
## Decisions Made
- Used the synced Matrix room graph as the authoritative source for restart recovery, while preserving existing local metadata whenever it is already valid.
- Kept legacy `platform_chat_id` repair on a single startup path so routed handling never needs ad hoc fallback creation for existing rooms.
## Deviations from Plan
### Auto-fixed Issues
**1. [Rule 3 - Blocking] Switched verification to a clean `uv run pytest` environment**
- **Found during:** Task 1 and Task 2 verification
- **Issue:** The default `pytest` path used a mismatched virtualenv without repo dependencies, and `.env` injected Matrix backend variables that polluted mock-mode tests.
- **Fix:** Ran the verification slice through `uv run pytest` with `UV_CACHE_DIR=/tmp/uv-cache-surfaces` and blank `MATRIX_AGENT_REGISTRY_PATH` / `MATRIX_PLATFORM_BACKEND` values to match the intended test environment.
- **Files modified:** None
- **Verification:** `uv run pytest` slice passed with 50/50 tests green
- **Committed in:** not applicable (verification-only adjustment)
---
**Total deviations:** 1 auto-fixed (1 blocking)
**Impact on plan:** Verification needed an environment correction, but code scope stayed within the plan and owned files.
## Issues Encountered
- The shell environment loaded deployment-oriented Matrix backend settings from `.env`; these had to be neutralized for the mock-mode regression slice.
## User Setup Required
None - no external service configuration required.
## Next Phase Readiness
- Restart recovery is in place for existing Space rooms, including deterministic legacy `platform_chat_id` repair.
- Remaining Phase 05 plans can build on a stable pre-sync recovery path instead of lazy bootstrap for existing topology.
## Self-Check: PASSED
---
*Phase: 05-mvp-deployment*
*Completed: 2026-04-27*

View file

@ -0,0 +1,156 @@
---
phase: 05-mvp-deployment
plan: 02
type: execute
wave: 2
depends_on:
- 05-01
files_modified:
- adapter/matrix/handlers/__init__.py
- adapter/matrix/handlers/context_commands.py
- adapter/matrix/routed_platform.py
- tests/adapter/matrix/test_context_commands.py
- tests/adapter/matrix/test_routed_platform.py
autonomous: true
requirements:
- PH05-02
must_haves:
truths:
- "Each working Matrix room uses its own durable `platform_chat_id` as the real agent context boundary."
- "`!clear` resets only the current room by rotating its `platform_chat_id` and disconnecting the old upstream chat."
- "Save, load, context, and routed send paths resolve through room-local platform context, not shared user state."
- "Strict room routing assumes startup reconciliation has already repaired legacy rooms missing `platform_chat_id`."
artifacts:
- path: "adapter/matrix/handlers/context_commands.py"
provides: "Room-local `!clear`, save/load/context resolution, and upstream disconnect behavior"
- path: "adapter/matrix/routed_platform.py"
provides: "Strict room -> agent_id + platform_chat_id routing"
- path: "tests/adapter/matrix/test_context_commands.py"
provides: "Regression coverage for `!clear` and room-local context commands"
key_links:
- from: "adapter/matrix/handlers/__init__.py"
to: "adapter/matrix/handlers/context_commands.py"
via: "IncomingCommand registration for `clear`"
pattern: "\"clear\""
- from: "adapter/matrix/routed_platform.py"
to: "adapter/matrix/store.py"
via: "room metadata lookup"
pattern: "platform_chat_id"
---
<objective>
Make room-local platform context explicit and user-facing by shipping real `!clear` semantics and strict per-room routing.
Purpose: Phase 05 must preserve Space+rooms UX while giving each room a true upstream context boundary.
Output: Updated command wiring, room-local context reset behavior, and routing regressions tied to `platform_chat_id`.
</objective>
<execution_context>
@/Users/a/.codex/get-shit-done/workflows/execute-plan.md
@/Users/a/.codex/get-shit-done/templates/summary.md
</execution_context>
<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/05-mvp-deployment/05-RESEARCH.md
@.planning/phases/05-mvp-deployment/05-VALIDATION.md
@.planning/phases/04-matrix-mvp-shared-agent-context-and-context-management-comma/04-02-SUMMARY.md
@adapter/matrix/handlers/__init__.py
@adapter/matrix/handlers/context_commands.py
@adapter/matrix/routed_platform.py
@tests/adapter/matrix/test_context_commands.py
@tests/adapter/matrix/test_routed_platform.py
<interfaces>
From `adapter/matrix/handlers/__init__.py`:
```python
dispatcher.register(
IncomingCommand,
"reset",
make_handle_reset(store, prototype_state)
if prototype_state is not None
else handle_settings,
)
```
From `adapter/matrix/handlers/context_commands.py`:
```python
async def _resolve_context_scope(
event: IncomingCommand,
store: StateStore,
chat_mgr,
) -> tuple[str, str | None]: ...
```
From `adapter/matrix/routed_platform.py`:
```python
async def _resolve_delegate(self, user_id: str, local_chat_id: str) -> tuple[PlatformClient, str]:
...
```
</interfaces>
</context>
<tasks>
<task type="auto" tdd="true">
<name>Task 1: Expand room-local context and clear-command tests</name>
<files>tests/adapter/matrix/test_context_commands.py, tests/adapter/matrix/test_routed_platform.py</files>
<read_first>tests/adapter/matrix/test_context_commands.py, tests/adapter/matrix/test_routed_platform.py, adapter/matrix/handlers/__init__.py, adapter/matrix/handlers/context_commands.py, adapter/matrix/routed_platform.py, .planning/phases/05-mvp-deployment/05-VALIDATION.md</read_first>
<behavior>
- Test 1: `!clear` rotates only the current room's `platform_chat_id` and disconnects only the old upstream chat (per PH05-02).
- Test 2: `!clear` is the supported command name; `!reset` may remain as a compatibility alias but must not be the only registered path.
- Test 3: routed send/stream paths fail fast when room metadata lacks `agent_id` or `platform_chat_id` instead of silently sharing context.
- Test 4: routed behavior uses startup-repaired room metadata and does not introduce a second fallback path that invents `platform_chat_id` during message handling.
</behavior>
<acceptance_criteria>
- Tests explicitly mention `clear` in command registration or command invocation.
- The context-command tests assert old and new `platform_chat_id` values and upstream disconnect behavior.
- The routed-platform tests assert room-local IDs are passed to delegates unchanged.
</acceptance_criteria>
<action>Extend the current Matrix context-command and routed-platform regressions so Phase 05 has direct coverage for `!clear`, room-local `platform_chat_id` rotation, and fail-fast routing when room bindings are incomplete. Treat startup reconciliation from `05-01` as the only supported repair path for legacy rooms missing `platform_chat_id`; the routed path must consume repaired metadata, not synthesize new room identities on demand. Preserve the Phase 04 prototype-state behavior where it still fits, but anchor new checks on per-room context isolation rather than shared session assumptions.</action>
<verify>
<automated>pytest tests/adapter/matrix/test_context_commands.py tests/adapter/matrix/test_routed_platform.py -v</automated>
</verify>
<done>The tests define the Phase 05 room-local contract for reset/clear and for routed upstream calls.</done>
</task>
<task type="auto" tdd="true">
<name>Task 2: Ship real room-local `!clear` semantics and strict routing</name>
<files>adapter/matrix/handlers/__init__.py, adapter/matrix/handlers/context_commands.py, adapter/matrix/routed_platform.py</files>
<read_first>adapter/matrix/handlers/__init__.py, adapter/matrix/handlers/context_commands.py, adapter/matrix/routed_platform.py, tests/adapter/matrix/test_context_commands.py, tests/adapter/matrix/test_routed_platform.py, .planning/phases/05-mvp-deployment/05-RESEARCH.md</read_first>
<behavior>
- Test 1: command registration exposes `!clear` as the real context-reset entrypoint for Matrix rooms.
- Test 2: only the active room's `platform_chat_id` rotates, and only the old upstream chat session is disconnected.
- Test 3: all room-local context commands resolve through recovered room metadata instead of falling back to shared user scope.
- Test 4: strict routing stays strict at runtime because legacy-room repair was already handled at startup by `05-01`, not by hidden message-path fallbacks.
</behavior>
<acceptance_criteria>
- `adapter/matrix/handlers/__init__.py` registers `clear`; if `reset` remains, it is clearly a compatibility alias.
- `adapter/matrix/handlers/context_commands.py` resolves and rotates room-local platform context without touching other rooms.
- `adapter/matrix/routed_platform.py` keeps explicit `MATRIX_ROUTE_INCOMPLETE` behavior when bindings are missing.
</acceptance_criteria>
<action>Update the Matrix context command surface to match the Phase 05 contract: real `!clear`, room-local `platform_chat_id` rotation, and upstream disconnect scoped to the old room context. Keep `save`, `load`, and `context` anchored to the same room-local identity. Tighten routed-platform behavior only where needed to preserve fail-fast semantics after startup reconciliation has repaired legacy rooms; do not reintroduce shared chat state, user-level reset behavior, or message-time backfill of missing `platform_chat_id`.</action>
<verify>
<automated>pytest tests/adapter/matrix/test_context_commands.py tests/adapter/matrix/test_routed_platform.py tests/adapter/matrix/test_dispatcher.py -v</automated>
</verify>
<done>Users can clear one working room without affecting others, and all routed upstream calls stay bound to room-local platform context.</done>
</task>
</tasks>
<verification>
Run the context and routed-platform slices plus Matrix dispatcher smoke coverage to confirm the exposed command name and room-local routing behavior are consistent.
</verification>
<success_criteria>
Every working Matrix room has an independent upstream context boundary, and `!clear` resets only the room where it is invoked.
</success_criteria>
<output>
After completion, create `.planning/phases/05-mvp-deployment/05-02-SUMMARY.md`
</output>

View file

@ -0,0 +1,106 @@
---
phase: 05-mvp-deployment
plan: 02
subsystem: matrix
tags: [matrix, routing, context, platform-chat-id, testing]
requires:
- phase: 05-01
provides: startup reconciliation for room metadata before live routing
provides:
- room-local `!clear` coverage and command registration
- strict room-local context resolution for save/context flows
- fail-fast routed-platform regressions for incomplete room bindings
affects: [matrix-dispatcher, routed-platform, startup-reconciliation]
tech-stack:
added: []
patterns: [per-room platform context, compatibility alias registration, fail-fast routing]
key-files:
created: []
modified:
- adapter/matrix/handlers/__init__.py
- adapter/matrix/handlers/context_commands.py
- tests/adapter/matrix/test_context_commands.py
- tests/adapter/matrix/test_routed_platform.py
key-decisions:
- "Expose `clear` only when prototype room-context support is available, while keeping `reset` as a compatibility alias."
- "Require recovered `platform_chat_id` for save/context/clear flows instead of falling back to shared local chat ids."
patterns-established:
- "Matrix room context commands resolve through room metadata repaired at startup, not message-time backfill."
- "Reset semantics rotate one room's upstream chat id and disconnect only that old upstream session."
requirements-completed: [PH05-02]
duration: 16 min
completed: 2026-04-27
---
# Phase 05 Plan 02: Room-local clear and strict Matrix routing Summary
**Room-local `!clear` command coverage, strict per-room `platform_chat_id` context resolution, and fail-fast Matrix routing regressions**
## Performance
- **Duration:** 16 min
- **Started:** 2026-04-27T22:00:00Z
- **Completed:** 2026-04-27T22:15:58Z
- **Tasks:** 2
- **Files modified:** 4
## Accomplishments
- Added RED/GREEN regression coverage for `!clear`, room-local chat-id rotation, and strict routed-platform failure modes.
- Registered `clear` as the supported reset entrypoint when room-context support exists, while preserving `reset` as a compatibility alias.
- Removed shared-context fallbacks from save/context handling and fixed reset to clear the old upstream prototype session before rotation.
## Task Commits
Each task was committed atomically:
1. **Task 1: Expand room-local context and clear-command tests** - `ae37476` (test)
2. **Task 2: Ship real room-local `!clear` semantics and strict routing** - `85e2fda` (feat)
## Files Created/Modified
- `adapter/matrix/handlers/__init__.py` - registers `clear` for runtimes with prototype room-context support and keeps `reset` as the compatibility alias.
- `adapter/matrix/handlers/context_commands.py` - requires room-local `platform_chat_id` for save/context/clear flows and disconnects the old upstream context on clear.
- `tests/adapter/matrix/test_context_commands.py` - covers room-local clear rotation, upstream disconnect, and clear registration.
- `tests/adapter/matrix/test_routed_platform.py` - covers incomplete-room fail-fast behavior and repaired metadata routing.
## Decisions Made
- Exposed `clear` only on runtimes that actually have prototype room-context support so Phase 05 semantics do not break older MVP smoke tests.
- Treated startup-repaired room metadata as the only supported source of `platform_chat_id` for context commands instead of falling back to local chat ids.
## Deviations from Plan
### Auto-fixed Issues
**1. [Rule 1 - Bug] Clear path was wiping the new context instead of the old upstream session**
- **Found during:** Task 2 (Ship real room-local `!clear` semantics and strict routing)
- **Issue:** `make_handle_reset()` cleared prototype session state for the newly generated chat id, leaving the previous upstream context intact.
- **Fix:** Cleared prototype state for the old `platform_chat_id` before rotating to the new room-local id, and kept the new context empty as well.
- **Files modified:** `adapter/matrix/handlers/context_commands.py`
- **Verification:** `MATRIX_AGENT_REGISTRY_PATH='' MATRIX_PLATFORM_BACKEND='' UV_CACHE_DIR=/tmp/uv-cache-surfaces uv run pytest tests/adapter/matrix/test_context_commands.py tests/adapter/matrix/test_routed_platform.py tests/adapter/matrix/test_dispatcher.py -v`
- **Committed in:** `85e2fda`
---
**Total deviations:** 1 auto-fixed (1 bug)
**Impact on plan:** The auto-fix was required for correct room-local clear behavior. No scope creep.
## Issues Encountered
- Plain `pytest` used an environment without `PyYAML`; verification was switched to `uv run pytest` with `UV_CACHE_DIR=/tmp/uv-cache-surfaces`.
- Shared shell env exposed `MATRIX_AGENT_REGISTRY_PATH=/app/config/matrix-agents.yaml`, which broke unrelated Matrix smoke tests. Verification was run with `MATRIX_AGENT_REGISTRY_PATH='' MATRIX_PLATFORM_BACKEND=''` to match the intended mock-runtime test setup.
## User Setup Required
None - no external service configuration required.
## Next Phase Readiness
- Matrix room-local clear semantics and routing contracts are now explicit and covered.
- Phase 05 follow-on work can assume startup reconciliation remains the only supported repair path for missing room routing metadata.
---
*Phase: 05-mvp-deployment*
*Completed: 2026-04-27*
## Self-Check: PASSED
- Found `.planning/phases/05-mvp-deployment/05-02-SUMMARY.md`
- Found commit `ae37476`
- Found commit `85e2fda`

View file

@ -0,0 +1,145 @@
---
phase: 05-mvp-deployment
plan: 03
type: execute
wave: 1
depends_on: []
files_modified:
- adapter/matrix/files.py
- sdk/real.py
- tests/adapter/matrix/test_files.py
- tests/platform/test_real.py
autonomous: true
requirements:
- PH05-04
must_haves:
truths:
- "Incoming Matrix attachments are written into a room-safe shared-volume path and passed upstream as relative workspace paths."
- "Agent-emitted files can be returned to Matrix users without inventing a separate file proxy."
- "The shared-volume contract works with the Phase 05 `/agents` deployment shape."
artifacts:
- path: "adapter/matrix/files.py"
provides: "Room-safe shared-volume path building and path resolution"
- path: "sdk/real.py"
provides: "Attachment path passthrough and send-file normalization"
- path: "tests/adapter/matrix/test_files.py"
provides: "Regression coverage for shared-volume path construction"
key_links:
- from: "adapter/matrix/files.py"
to: "sdk/real.py"
via: "relative `workspace_path` transport"
pattern: "workspace_path"
- from: "sdk/real.py"
to: "adapter/matrix/bot.py"
via: "OutgoingMessage attachments rendered back to Matrix"
pattern: "MsgEventSendFile"
---
<objective>
Harden the Matrix attachment path contract around the shared deployment volume instead of custom transport shims.
Purpose: Phase 05 file handling must survive real deployment with room-safe paths and outbound file delivery through the existing shared-volume model.
Output: Attachment-path regressions and any targeted runtime fixes needed for `/agents`-backed shared volume behavior.
</objective>
<execution_context>
@/Users/a/.codex/get-shit-done/workflows/execute-plan.md
@/Users/a/.codex/get-shit-done/templates/summary.md
</execution_context>
<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/05-mvp-deployment/05-RESEARCH.md
@.planning/phases/05-mvp-deployment/05-VALIDATION.md
@docs/deploy-architecture.md
@docs/superpowers/specs/2026-04-20-matrix-shared-workspace-file-flow-design.md
@adapter/matrix/files.py
@sdk/real.py
@tests/adapter/matrix/test_files.py
@tests/platform/test_real.py
<interfaces>
From `adapter/matrix/files.py`:
```python
def build_workspace_attachment_path(
*,
workspace_root: Path,
matrix_user_id: str,
room_id: str,
filename: str,
timestamp: str | None = None,
) -> tuple[str, Path]: ...
```
From `sdk/real.py`:
```python
@staticmethod
def _attachment_paths(attachments: list[Attachment] | None) -> list[str]: ...
@staticmethod
def _attachment_from_send_file_event(event: MsgEventSendFile) -> Attachment: ...
```
</interfaces>
</context>
<tasks>
<task type="auto" tdd="true">
<name>Task 1: Add shared-volume file contract tests for `/agents` deployment</name>
<files>tests/adapter/matrix/test_files.py, tests/platform/test_real.py</files>
<read_first>tests/adapter/matrix/test_files.py, tests/platform/test_real.py, adapter/matrix/files.py, sdk/real.py, docs/deploy-architecture.md, .planning/phases/05-mvp-deployment/05-RESEARCH.md</read_first>
<behavior>
- Test 1: incoming Matrix files land under a room-safe `surfaces/matrix/.../inbox/...` path that remains relative to the agent workspace contract.
- Test 2: upstream file events normalize `/workspace/...` and `/agents/...`-style absolute paths into relative `workspace_path` values.
- Test 3: attachment forwarding never switches to inline blobs or HTTP shim URLs (per PH05-04).
</behavior>
<acceptance_criteria>
- `tests/adapter/matrix/test_files.py` asserts the path namespace includes sanitized user and room components.
- `tests/platform/test_real.py` contains explicit coverage for send-file path normalization.
- The automated test command in `<verify>` exercises both inbound and outbound sides of the shared-volume contract.
</acceptance_criteria>
<action>Expand the file-flow regressions around the real deployment contract described in research and `docs/deploy-architecture.md`. Keep the tests centered on relative `workspace_path` transport and room-safe on-disk layout. Do not introduce proxy URLs, base64 payload transport, or new platform endpoints.</action>
<verify>
<automated>pytest tests/adapter/matrix/test_files.py tests/platform/test_real.py -v</automated>
</verify>
<done>Phase 05 has direct test coverage for `/agents`-backed shared-volume behavior across inbound and outbound file paths.</done>
</task>
<task type="auto" tdd="true">
<name>Task 2: Tighten attachment path handling for the shared volume contract</name>
<files>adapter/matrix/files.py, sdk/real.py</files>
<read_first>adapter/matrix/files.py, sdk/real.py, tests/adapter/matrix/test_files.py, tests/platform/test_real.py, docs/deploy-architecture.md</read_first>
<behavior>
- Test 1: inbound attachment helpers keep returning relative paths even when the bot writes into `/agents`.
- Test 2: outbound file normalization accepts absolute paths from the agent runtime but strips them back to relative workspace paths for Matrix rendering.
- Test 3: no code path emits non-relative attachment references to the upstream agent API.
</behavior>
<acceptance_criteria>
- `sdk/real.py` only forwards relative attachment paths to the agent API.
- `sdk/real.py` normalizes both `/workspace` and `/agents` absolute roots if present in send-file events.
- `adapter/matrix/files.py` remains the single source of truth for room-safe attachment path construction.
</acceptance_criteria>
<action>Implement only the minimum runtime changes needed to satisfy the shared-volume tests. Keep `adapter/matrix/files.py` as the single place that builds surface-owned attachment paths, and keep `sdk/real.py` responsible only for attachment passthrough and send-file normalization. Do not widen this plan into compose edits, registry redesign, or bot command changes.</action>
<verify>
<automated>pytest tests/adapter/matrix/test_files.py tests/platform/test_real.py tests/adapter/matrix/test_send_outgoing.py -v</automated>
</verify>
<done>Incoming and outgoing file references stay compatible with the real shared-volume deployment contract, and the targeted file/path regressions pass.</done>
</task>
</tasks>
<verification>
Run the Matrix file helper, real platform client, and outgoing-send slices together so the shared-volume contract is validated from write path through return-to-user rendering.
</verification>
<success_criteria>
The Matrix bot and agent runtime can exchange file references through the shared volume using only relative workspace paths and room-safe storage layout.
</success_criteria>
<output>
After completion, create `.planning/phases/05-mvp-deployment/05-03-SUMMARY.md`
</output>

View file

@ -0,0 +1,103 @@
---
phase: 05-mvp-deployment
plan: 03
subsystem: infra
tags: [matrix, attachments, shared-volume, agents, pytest]
requires:
- phase: 04-matrix-mvp-shared-agent-context-and-context-management-comma
provides: direct AgentApi integration and Matrix outgoing file rendering
provides:
- shared-volume attachment path regressions for /agents deployment
- relative workspace-path normalization for upstream attachment transport
- send-file event normalization for Matrix outbound file rendering
affects: [matrix, deployment, shared-volume, file-transfer]
tech-stack:
added: []
patterns: [relative workspace_path transport, shared-volume root normalization]
key-files:
created: []
modified:
- tests/adapter/matrix/test_files.py
- tests/platform/test_real.py
- sdk/real.py
key-decisions:
- "Keep adapter/matrix/files.py as the only path-construction source; sdk/real.py only normalizes attachment references crossing the shared-volume boundary."
- "Normalize both /workspace and /agents absolute paths back to relative workspace paths before forwarding to the agent API or rendering Matrix send-file events."
patterns-established:
- "Matrix attachment transport stays relative even when the bot sees absolute shared-volume paths."
- "Agent-emitted file paths are rendered back to Matrix without HTTP proxy shims or inline blobs."
requirements-completed: [PH05-04]
duration: 3 min
completed: 2026-04-27
---
# Phase 05 Plan 03: Shared-volume attachment path hardening Summary
**Room-safe Matrix inbox paths and relative shared-volume attachment normalization for `/agents` deployment**
## Performance
- **Duration:** 3 min
- **Started:** 2026-04-27T22:02:34Z
- **Completed:** 2026-04-27T22:05:41Z
- **Tasks:** 2
- **Files modified:** 3
## Accomplishments
- Added regression coverage for room-safe `surfaces/matrix/.../inbox/...` attachment layout under `/agents` workspaces.
- Added explicit tests for `/workspace/...` and `/agents/...` attachment-path normalization on both upstream send and downstream send-file rendering.
- Tightened `RealPlatformClient` so only relative `workspace_path` values are forwarded to the agent API.
## Task Commits
Each task was committed atomically:
1. **Task 1: Add shared-volume file contract tests for `/agents` deployment** - `cafb0ec` (test)
2. **Task 2: Tighten attachment path handling for the shared volume contract** - `9a03160` (fix)
## Files Created/Modified
- `tests/adapter/matrix/test_files.py` - covers room-safe shared-volume inbox paths under an `/agents` workspace root.
- `tests/platform/test_real.py` - covers relative attachment forwarding and send-file normalization for `/workspace` and `/agents` paths.
- `sdk/real.py` - normalizes shared-volume roots before attachments cross the upstream or Matrix rendering boundary.
## Decisions Made
- Kept `adapter/matrix/files.py` unchanged so path construction remains centralized there.
- Reused one normalization helper in `sdk/real.py` for both `_attachment_paths()` and `_attachment_from_send_file_event()` to keep inbound and outbound behavior aligned.
## Deviations from Plan
### Auto-fixed Issues
**1. [Rule 3 - Blocking] Adjusted verification to use the project uv environment**
- **Found during:** Task 2 (Tighten attachment path handling for the shared volume contract)
- **Issue:** The plan's raw `pytest` command collected with an interpreter missing `PyYAML`, which blocked `tests/adapter/matrix/test_send_outgoing.py` before runtime assertions could execute.
- **Fix:** Re-ran verification with `UV_CACHE_DIR=/tmp/uv-cache uv run pytest ...`, matching the repo's `CLAUDE.md` guidance and the project `.venv`.
- **Files modified:** None
- **Verification:** `UV_CACHE_DIR=/tmp/uv-cache uv run pytest tests/adapter/matrix/test_files.py tests/platform/test_real.py tests/adapter/matrix/test_send_outgoing.py -v`
- **Committed in:** None (verification-environment adjustment only)
---
**Total deviations:** 1 auto-fixed (1 blocking)
**Impact on plan:** No scope creep. The deviation only changed the verification entrypoint so the planned test slice could run in the correct environment.
## Issues Encountered
- The ambient `pytest` resolved to a different virtualenv than the project `.venv`, so direct verification was unreliable for dependency-backed Matrix tests.
## User Setup Required
None - no external service configuration required.
## Next Phase Readiness
- Shared-volume file references now stay relative across inbound bot downloads, agent API attachment transport, and outbound Matrix send-file rendering.
- Phase 05 compose and deployment work can assume the `/agents` contract without adding file proxy infrastructure.
## Self-Check: PASSED
- Found `.planning/phases/05-mvp-deployment/05-03-SUMMARY.md`
- Verified commit `cafb0ec` exists in git history
- Verified commit `9a03160` exists in git history
---
*Phase: 05-mvp-deployment*
*Completed: 2026-04-27*

View file

@ -0,0 +1,128 @@
---
phase: 05-mvp-deployment
plan: 04
type: execute
wave: 2
depends_on:
- 05-03
files_modified:
- docker-compose.prod.yml
- docker-compose.fullstack.yml
- Dockerfile
- .env.example
- README.md
- docs/deploy-architecture.md
autonomous: true
requirements:
- PH05-05
must_haves:
truths:
- "Production handoff uses a bot-only compose artifact instead of the internal full-stack harness."
- "Internal E2E compose brings up the bot, platform-agent, and shared volume with explicit health-gated startup."
- "Deployment docs and env examples match the split compose artifacts and shared `/agents` contract."
artifacts:
- path: "docker-compose.prod.yml"
provides: "Bot-only deployment handoff artifact"
- path: "docker-compose.fullstack.yml"
provides: "Internal E2E harness with shared volume and dependency gating"
- path: ".env.example"
provides: "Documented runtime contract for Phase 05 deployment"
key_links:
- from: "docker-compose.fullstack.yml"
to: "docker-compose.prod.yml"
via: "shared service definition or explicit duplication"
pattern: "matrix-bot"
- from: "docs/deploy-architecture.md"
to: "docker-compose.prod.yml"
via: "operator handoff instructions"
pattern: "prod"
---
<objective>
Split deployment artifacts by operational intent so operator handoff and internal E2E testing stop sharing the same compose contract.
Purpose: Phase 05 needs an explicit bot-only production artifact and a separate full-stack compose harness aligned with the shared-volume design.
Output: `docker-compose.prod.yml`, `docker-compose.fullstack.yml`, and updated env/docs describing when to use each.
</objective>
<execution_context>
@/Users/a/.codex/get-shit-done/workflows/execute-plan.md
@/Users/a/.codex/get-shit-done/templates/summary.md
</execution_context>
<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/05-mvp-deployment/05-RESEARCH.md
@.planning/phases/05-mvp-deployment/05-VALIDATION.md
@.planning/phases/04-matrix-mvp-shared-agent-context-and-context-management-comma/04-03-SUMMARY.md
@docs/deploy-architecture.md
@docker-compose.yml
@Dockerfile
@.env.example
<interfaces>
Current root compose contract:
```yaml
services:
platform-agent:
...
matrix-bot:
build: .
env_file: .env
environment:
AGENT_BASE_URL: http://platform-agent:8000
SURFACES_WORKSPACE_DIR: /workspace
```
</interfaces>
</context>
<tasks>
<task type="auto">
<name>Task 1: Create split prod and fullstack compose artifacts</name>
<files>docker-compose.prod.yml, docker-compose.fullstack.yml, Dockerfile, .env.example</files>
<read_first>docker-compose.yml, Dockerfile, .env.example, docs/deploy-architecture.md, .planning/phases/05-mvp-deployment/05-RESEARCH.md, .planning/phases/05-mvp-deployment/05-VALIDATION.md</read_first>
<acceptance_criteria>
- `docker-compose.prod.yml` defines only the bot-side runtime required for operator handoff.
- `docker-compose.fullstack.yml` includes the internal platform-agent service, shared volume mounts, and health-gated startup rather than sleep-based sequencing.
- `.env.example` documents the Phase 05 env contract without requiring the reader to inspect the old root compose file.
</acceptance_criteria>
<action>Create two explicit compose artifacts per PH05-05: a bot-only `docker-compose.prod.yml` for deployment handoff and a `docker-compose.fullstack.yml` for internal E2E runs. Align mounts and env values with the Phase 05 shared `/agents` volume direction from research. Reuse the existing Dockerfile unless a small compatibility edit is required. Do not keep the old single-file compose setup as the only documented runtime.</action>
<verify>
<automated>docker compose -f docker-compose.prod.yml config > /tmp/phase05-prod-compose.yml && docker compose -f docker-compose.fullstack.yml config > /tmp/phase05-fullstack-compose.yml && rg -n "^services:$|^ matrix-bot:$|^volumes:$|/agents" /tmp/phase05-prod-compose.yml && test -z "$(rg -n "^ platform-agent:$" /tmp/phase05-prod-compose.yml)" && rg -n "^ matrix-bot:$|^ platform-agent:$|condition: service_healthy|healthcheck:|/agents" /tmp/phase05-fullstack-compose.yml</automated>
</verify>
<done>Both compose files render successfully and express distinct operational roles: prod handoff vs internal full-stack testing.</done>
</task>
<task type="auto">
<name>Task 2: Update deployment docs and operator guidance for the split artifacts</name>
<files>README.md, docs/deploy-architecture.md</files>
<read_first>README.md, docs/deploy-architecture.md, docker-compose.prod.yml, docker-compose.fullstack.yml, .env.example</read_first>
<acceptance_criteria>
- README or deploy doc tells the operator exactly which compose file to use for production vs internal E2E.
- The docs describe the shared `/agents` volume behavior and reference the relevant env vars.
- The old root `docker-compose.yml` is no longer the primary documented deployment path.
</acceptance_criteria>
<action>Update the repo docs so the Phase 05 deployment story is executable without inference: production handoff stays bot-only, full-stack compose is for internal E2E, and shared-volume file behavior is described in the same terms as the runtime artifacts. Keep documentation narrowly scoped to the shipped compose split; do not widen into platform-master or future storage design.</action>
<verify>
<automated>rg -n "docker-compose\\.prod|docker-compose\\.fullstack|/agents|prod handoff|full-stack" README.md docs/deploy-architecture.md .env.example && rg -n "production|deploy" README.md docs/deploy-architecture.md | rg "docker-compose\\.prod" && test -z "$(rg -n "docker compose up|docker-compose\\.yml" README.md docs/deploy-architecture.md | rg "production|deploy")"</automated>
</verify>
<done>The docs and env guidance match the new compose artifacts and no longer imply a single shared deployment file.</done>
</task>
</tasks>
<verification>
Render both compose files, then grep the docs for the new artifact names and `/agents` references so the operator contract is explicit and consistent.
</verification>
<success_criteria>
An operator can deploy the Matrix bot with the bot-only compose file, while developers can run the internal end-to-end harness separately without reinterpreting the deployment docs.
</success_criteria>
<output>
After completion, create `.planning/phases/05-mvp-deployment/05-04-SUMMARY.md`
</output>

View file

@ -0,0 +1,93 @@
---
phase: 05-mvp-deployment
plan: 04
subsystem: infra
tags: [docker-compose, matrix, deployment, agents, docs]
requires:
- phase: 05-03
provides: "Shared /agents attachment contract and path normalization for Matrix runtime"
provides:
- "docker-compose.prod.yml bot-only deployment handoff artifact"
- "docker-compose.fullstack.yml internal E2E harness with health-gated platform-agent startup"
- "README and deploy architecture docs aligned to the split compose contract"
affects: [mvp-deployment, operator-handoff, internal-e2e]
tech-stack:
added: [Docker Compose]
patterns: [split-compose-by-operational-intent, shared-agents-volume-contract]
key-files:
created: [docker-compose.prod.yml, docker-compose.fullstack.yml]
modified: [.env.example, README.md, docs/deploy-architecture.md]
key-decisions:
- "Split Compose artifacts by runtime intent: bot-only prod handoff vs internal full-stack verification."
- "Document /agents as the bot-side shared volume root while internal platform-agent keeps /workspace on the same volume."
patterns-established:
- "Production operators use docker-compose.prod.yml and provide an external AGENT_BASE_URL."
- "Internal verification uses docker-compose.fullstack.yml with service_healthy gating instead of sleep-based startup."
requirements-completed: [PH05-05]
duration: 3 min
completed: 2026-04-27
---
# Phase 05 Plan 04: Split deployment artifacts Summary
**Bot-only production compose handoff plus a separate health-gated full-stack harness aligned to the shared `/agents` volume contract**
## Performance
- **Duration:** 3 min
- **Started:** 2026-04-27T22:12:42Z
- **Completed:** 2026-04-27T22:16:09Z
- **Tasks:** 2
- **Files modified:** 5
## Accomplishments
- Added `docker-compose.prod.yml` as the operator-facing bot-only runtime artifact.
- Added `docker-compose.fullstack.yml` for internal E2E runs with `platform-agent` health-gated startup.
- Updated env and deployment docs so `/agents` and the split compose flow are explicit without relying on the old root compose file.
## Task Commits
Each task was committed atomically:
1. **Task 1: Create split prod and fullstack compose artifacts** - `df6d8bf` (feat)
2. **Task 2: Update deployment docs and operator guidance for the split artifacts** - `22a3a2b` (docs)
**Plan metadata:** pending final docs commit after state updates
## Files Created/Modified
- `docker-compose.prod.yml` - bot-only deployment handoff with `/agents` volume contract
- `docker-compose.fullstack.yml` - internal harness extending the bot service and adding health-checked `platform-agent`
- `.env.example` - Phase 05 runtime variables for prod handoff and internal full-stack defaults
- `README.md` - operator-facing instructions for choosing the correct compose artifact
- `docs/deploy-architecture.md` - deployment topology and shared-volume guidance aligned to the split artifacts
## Decisions Made
- Split deployment packaging by operational intent instead of overloading one compose file for both prod handoff and internal testing.
- Kept the bots absolute shared path rooted at `/agents` in docs and env examples while letting the internal `platform-agent` consume the same named volume at `/workspace`.
## Deviations from Plan
None - plan executed exactly as written.
## Issues Encountered
- The docs verification grep treated `deploy` inside the filename `docs/deploy-architecture.md` as a false positive when root compose was still named explicitly. The fix was to remove the literal root compose filename from deploy-path wording while keeping the deprecation clear.
## User Setup Required
None - no external service configuration required beyond populating `.env` from `.env.example`.
## Next Phase Readiness
- Operators now have a bot-only compose artifact for handoff, and developers have a separate internal E2E harness.
- Remaining Phase 05 work can rely on the split runtime contract without reinterpreting deployment docs.
## Self-Check: PASSED
- Summary file exists at `.planning/phases/05-mvp-deployment/05-04-SUMMARY.md`
- Commit `df6d8bf` found in git history
- Commit `22a3a2b` found in git history
---
*Phase: 05-mvp-deployment*
*Completed: 2026-04-27*

View file

@ -0,0 +1,411 @@
# Phase 05: MVP Deployment - Research
**Researched:** 2026-04-28
**Domain:** Matrix bot production deployment, restart reconciliation, per-room context isolation, shared-volume file transfer
**Confidence:** HIGH
## Project Constraints (from CLAUDE.md)
- All platform calls must stay behind `platform/interface.py` (`PlatformClient` protocol).
- Current platform implementation is a mock / replaceable adapter; architecture must not depend on unfinished upstream SDK.
- Keep architecture decisions inside this repo and document contracts locally.
- Prefer async, adapter/core separation, and do not bypass the existing `core/` and `adapter/` layering.
- Use `uv sync` for dependency installation.
- Use `pytest tests/ -v` and adapter-specific pytest slices for verification.
- Never commit `.env`.
- Dependency order remains fixed: `core/` first, `platform/` second, adapters after that.
## Summary
Phase 05 should not introduce a new stack. The established implementation path is to harden the existing `matrix-nio + SQLiteStore + RoutedPlatformClient + shared workspace volume` design so production restart behavior matches the current Space+rooms UX. The main architectural rule is: Matrix topology is authoritative for room existence, while local SQLite metadata is authoritative only after reconciliation has rebuilt it.
The production-safe approach is to bind every working Matrix room to its own durable `platform_chat_id`, rotate only that identifier for `!clear`, and make restart recovery idempotent. Reconciliation should rebuild `user_meta`, `room_meta`, `ChatManager` entries, and missing routing fields from Matrix Space membership and room state before `sync_forever()` begins processing live traffic. Unknown rooms must be reconciled first, not silently converted into new chats.
For files, keep the current shared-volume contract and relative `workspace_path` transport. Do not build HTTP file shims or embed file payloads in bot-side state. For deployment artifacts, split runtime intent explicitly: `docker-compose.prod.yml` is a bot-only handoff contract, while `docker-compose.fullstack.yml` is the internal E2E harness that brings up platform services and shared volumes together.
**Primary recommendation:** Implement Phase 05 as a reconciliation-and-deploy hardening pass on the current Matrix stack, with Matrix Space state as source of truth and per-room `platform_chat_id` as the routing key.
## Standard Stack
### Core
| Library | Version | Purpose | Why Standard |
|---------|---------|---------|--------------|
| `matrix-nio` | 0.25.2 | Async Matrix client, Spaces, media upload/download, token login, sync loop | Already in repo; official docs confirm support for Spaces, token login, `room_put_state`, `upload`, `download`, and `sync_forever` |
| `sqlite3` / `SQLiteStore` | stdlib / repo-local | Durable bot metadata (`room_meta`, `user_meta`, routing state) | Small, local, restart-safe KV layer already used by runtime and tests |
| `PyYAML` | 6.0.3 | Agent registry / deployment config parsing | Current repo standard for `config/matrix-agents.yaml`-style artifacts |
| `httpx` | 0.28.1 | Async HTTP for auxiliary platform calls | Already used; fits async runtime and current codebase |
| Docker Compose | v2 spec; local install `v2.40.3` | Prod/fullstack topology, shared named volumes, health-gated startup | Officially supports multi-file overlays, named volumes, and `service_healthy` gating |
### Supporting
| Library | Version | Purpose | When to Use |
|---------|---------|---------|-------------|
| `structlog` | 25.5.0 | Structured runtime logging | Use for reconciliation summaries, routing mismatches, and deploy diagnostics |
| `pydantic` | 2.13.3 | Typed config / payload validation | Use for any new deployment config or reconciliation report structures |
| `python-dotenv` | 1.2.2 | Local env loading | Keep for local and compose-driven runtime config |
| `pytest` | 9.0.3 | Test runner | Full phase verification and regression slices |
| `pytest-asyncio` | 1.3.0 | Async test execution | Required for reconciliation/runtime tests |
### Alternatives Considered
| Instead of | Could Use | Tradeoff |
|------------|-----------|----------|
| `matrix-nio` | Synapse Admin / raw Matrix HTTP calls | Worse fit; repo already depends on nio abstractions and tests |
| repo-local `SQLiteStore` | Redis/Postgres | Unnecessary operational scope increase for MVP deployment |
| shared volume file flow | custom file proxy / presigned URLs | More moving parts, more auth/cleanup edge cases, no need for MVP |
| split compose files | one overloaded compose file with profiles | Harder operator handoff; less explicit prod vs internal-test intent |
**Installation:**
```bash
uv sync
```
**Version verification:** Verified on 2026-04-28 from PyPI and local environment.
| Package | Verified Version | Publish Date | Source |
|---------|------------------|--------------|--------|
| `matrix-nio` | 0.25.2 | 2024-10-04 | PyPI |
| `httpx` | 0.28.1 | 2024-12-06 | PyPI |
| `structlog` | 25.5.0 | 2025-10-27 | PyPI |
| `pydantic` | 2.13.3 | 2026-04-20 | PyPI |
| `aiohttp` | 3.13.5 | 2026-03-31 | PyPI |
| `PyYAML` | 6.0.3 | 2025-09-25 | PyPI |
| `python-dotenv` | 1.2.2 | 2026-03-01 | PyPI |
| `pytest` | 9.0.3 | 2026-04-07 | PyPI |
| `pytest-asyncio` | 1.3.0 | 2025-11-10 | PyPI |
## Architecture Patterns
### Recommended Project Structure
```text
adapter/matrix/
├── bot.py # startup, sync bootstrap, live callbacks
├── reconciliation.py # new: restart recovery from Matrix state
├── files.py # shared-volume path building / materialization
├── routed_platform.py # room -> agent_id + platform_chat_id routing
├── store.py # room_meta/user_meta helpers and counters
└── handlers/
├── auth.py # Space + first room provisioning
├── chat.py # !new / !archive / !rename
└── context_commands.py # !save / !load / !clear / !context
deploy/
├── docker-compose.prod.yml # bot-only handoff
└── docker-compose.fullstack.yml # internal E2E stack
```
### Pattern 1: Matrix Space State Is Canonical, SQLite Is Rebuildable
**What:** Treat Matrix Space membership and child-room state as the source of truth for room topology; use local SQLite metadata as a cached routing index that reconciliation can rebuild.
**When to use:** Startup, DB loss, stale local metadata, and any deployment where rooms may outlive the bot process.
**Example:**
```python
# Source: repo pattern from adapter/matrix/store.py + Matrix Space state
room_meta = {
"room_type": "chat",
"chat_id": "C7",
"display_name": "Research",
"matrix_user_id": "@alice:example.org",
"space_id": "!space:example.org",
"agent_id": "agent-1",
"platform_chat_id": "42",
}
await set_room_meta(store, room_id, room_meta)
await chat_mgr.get_or_create(
user_id=room_meta["matrix_user_id"],
chat_id=room_meta["chat_id"],
platform="matrix",
surface_ref=room_id,
name=room_meta["display_name"],
)
```
### Pattern 2: Per-Room `platform_chat_id` Is the Only Real Context Boundary
**What:** Route every working Matrix room to its own durable `platform_chat_id`.
**When to use:** Normal messaging, `!save`, `!load`, `!context`, `!clear`, restart restoration.
**Example:**
```python
# Source: adapter/matrix/routed_platform.py + adapter/matrix/handlers/context_commands.py
old_chat_id = room_meta["platform_chat_id"]
new_chat_id = await next_platform_chat_id(store)
await set_platform_chat_id(store, room_id, new_chat_id)
disconnect = getattr(platform, "disconnect_chat", None)
if callable(disconnect):
await disconnect(old_chat_id)
```
### Pattern 3: `!clear` Means Chat-ID Rotation, Not Global Wipe
**What:** Implement real clear by rotating only the current room's `platform_chat_id` and disconnecting the old upstream chat session.
**When to use:** User-triggered context reset for one room.
**Example:**
```python
# Source: adapter/matrix/handlers/context_commands.py
room_id = await _resolve_room_id(event, chat_mgr)
old_chat_id = (room_meta or {}).get("platform_chat_id") or room_id
new_chat_id = await next_platform_chat_id(store)
await set_platform_chat_id(store, room_id, new_chat_id)
```
### Pattern 4: Shared-Volume File Handoff Uses Relative Workspace Paths
**What:** Persist incoming Matrix media into a room-scoped path under the shared workspace, and pass only relative paths to the agent.
**When to use:** User uploads, staged attachments, agent-emitted files.
**Example:**
```python
# Source: adapter/matrix/files.py
relative_path = (
Path("surfaces") / "matrix" / safe_user / safe_room / "inbox" / f"{stamp}-{safe_name}"
)
return Attachment(
type=attachment.type,
url=attachment.url,
filename=filename,
mime_type=attachment.mime_type,
workspace_path=relative_path.as_posix(),
)
```
### Pattern 5: Compose Split By Operational Intent
**What:** Keep one compose artifact for operator handoff and one for internal full-stack testing.
**When to use:** Deployment packaging.
**Example:**
```yaml
# docker-compose.prod.yml
services:
matrix-bot:
image: surfaces-bot:latest
env_file: .env
volumes:
- agents:/agents
# docker-compose.fullstack.yml
services:
matrix-bot:
extends:
file: docker-compose.prod.yml
service: matrix-bot
platform-agent:
...
volumes:
agents:
```
### Anti-Patterns to Avoid
- **Lazy bootstrap as restart strategy:** `_bootstrap_unregistered_room()` is acceptable for first-contact repair, not as the primary restart recovery path in production.
- **Per-user context identity:** a user-level or DM-level chat id breaks Space+rooms isolation and makes `!clear` incorrect.
- **Global reset endpoint semantics:** `!clear` must not wipe other rooms or all agent state for a user.
- **Absolute attachment paths in platform payloads:** keep agent attachment references relative to its workspace contract.
- **Sleep-based service readiness:** use Compose healthchecks and dependency conditions, not shell `sleep`.
## Don't Hand-Roll
| Problem | Don't Build | Use Instead | Why |
|---------|-------------|-------------|-----|
| Matrix room/Space protocol | Raw custom HTTP wrappers for state events | `matrix-nio` `room_create`, `room_put_state`, `space_get_hierarchy`, `sync_forever`, `upload`, `download` | Official support already exists and repo tests are built around nio |
| Restart topology discovery | Ad hoc timeline scraping | Full-state sync plus room state / Space child reconciliation | Timeline replay is noisy and brittle; state is the stable source |
| File transfer bus | Base64 blobs or custom bot-side file API | Shared `/agents/` volume with relative `workspace_path` | Lower operational complexity and already matches upstream agent contract |
| Compose startup sequencing | Shell loops / sleeps | `healthcheck` + `depends_on: condition: service_healthy` | Official Compose behavior is deterministic and observable |
| Context reset | Deleting all SQLite rows or resetting the whole user | Rotate current room `platform_chat_id` and drop that room's live agent connection | Preserves other rooms and matches user expectation |
**Key insight:** The deceptively hard problems in this phase are already solved by the current stack: Matrix room state, nio media handling, named volumes, and service health gating. Custom alternatives add more failure modes than value.
## Common Pitfalls
### Pitfall 1: Unknown room after restart creates a duplicate working chat
**What goes wrong:** The bot treats an existing room as unregistered and provisions a fresh room/tree.
**Why it happens:** Local SQLite metadata is missing, but Matrix topology still exists.
**How to avoid:** Run reconciliation before live sync callbacks; only allow lazy bootstrap for genuinely new first-contact rooms.
**Warning signs:** New `Чат N` rooms appear after restart without a matching user action.
### Pitfall 2: `!clear` resets the wrong scope
**What goes wrong:** Clearing one room also clears another room, or does nothing because the upstream session key did not change.
**Why it happens:** Context is keyed by user or local `chat_id` instead of durable room-local `platform_chat_id`.
**How to avoid:** Always resolve room -> `platform_chat_id`, rotate it, and disconnect only the old upstream chat.
**Warning signs:** Two rooms share response history or `!context` reports the same platform context id.
### Pitfall 3: Space child linkage is incomplete
**What goes wrong:** Rooms exist but do not appear correctly under the user's Space.
**Why it happens:** Missing or malformed `m.space.child` state, especially missing `via` data.
**How to avoid:** Persist `space_id`, write `m.space.child` with `state_key=room_id`, and reconcile child links on startup.
**Warning signs:** Element shows the room outside the Space, or not at all in the hierarchy.
### Pitfall 4: Shared volume works locally but fails in deployment
**What goes wrong:** Agent-generated files cannot be read by the bot, or bot-downloaded files are unreadable by the agent.
**Why it happens:** Mount mismatch, wrong root (`/workspace` vs `/agents`), or container user/group permissions.
**How to avoid:** Standardize one shared root, keep relative workspace paths, and align container permissions with Compose volume configuration.
**Warning signs:** Attachment paths exist in metadata but not on disk inside the other container.
### Pitfall 5: Compose `depends_on` starts too early
**What goes wrong:** Bot starts before dependent services are actually ready.
**Why it happens:** Short-form `depends_on` only waits for container start, not health.
**How to avoid:** Use healthchecks and long-form `depends_on` with `service_healthy` in the full-stack compose file.
**Warning signs:** First requests fail after fresh `docker compose up`, then succeed on retry.
## Code Examples
Verified patterns from official sources and current repo:
### Create a Space with `matrix-nio`
```python
# Source: matrix-nio API docs
space_resp = await client.room_create(
name=f"Lambda — {display_name}",
visibility=RoomVisibility.private,
invite=[matrix_user_id],
space=True,
)
```
### Add a child room to a Space
```python
# Source: current repo pattern + Matrix spec
await client.room_put_state(
room_id=space_id,
event_type="m.space.child",
content={"via": [homeserver]},
state_key=chat_room_id,
)
```
### Persist room-scoped attachment paths
```python
# Source: adapter/matrix/files.py
relative_path, absolute_path = build_workspace_attachment_path(
workspace_root=workspace_root,
matrix_user_id=matrix_user_id,
room_id=room_id,
filename=filename,
)
absolute_path.parent.mkdir(parents=True, exist_ok=True)
absolute_path.write_bytes(body)
```
### Health-gated startup in Compose
```yaml
# Source: Docker Compose docs
services:
matrix-bot:
depends_on:
platform-agent:
condition: service_healthy
platform-agent:
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
interval: 10s
timeout: 5s
retries: 5
```
## State of the Art
| Old Approach | Current Approach | When Changed | Impact |
|--------------|------------------|--------------|--------|
| Per-user or single shared platform context | Per-room `platform_chat_id` | Repo direction corrected on 2026-04-28 | Enables true room isolation and correct `!clear` |
| Single overloaded compose runtime | Separate prod handoff and full-stack E2E compose files | Current Phase 05 scope | Reduces operator ambiguity |
| Unknown room auto-bootstrap as recovery | Explicit reconciliation before live traffic | Recommended for Phase 05 | Prevents duplicate chat trees after restart |
| File payloads treated as transport concern | Shared-volume relative path contract | Already present in repo | Keeps bot/platform contract simple and durable |
**Deprecated/outdated:**
- Single-chat / DM-first deployment direction: explicitly discarded in Phase 05 reset.
- Global reset semantics for Matrix context commands: does not match Space+rooms UX.
- Using only local store as truth for restart recovery: unsafe once deployed rooms outlive the process.
## Open Questions
1. **What exact Matrix state should reconciliation trust for `chat_id` labels?**
- What we know: `room_meta.chat_id` is local and not derivable from Matrix protocol by default.
- What's unclear: whether chat labels should be reconstructed from room names, stored custom state, or cached local metadata when present.
- Recommendation: persist `chat_id` in local SQLite, but make reconciliation able to regenerate a stable fallback label and avoid blocking routing if the label is missing.
2. **What readiness probe exists for `platform-agent` in the full-stack compose?**
- What we know: Compose health gating is the right pattern.
- What's unclear: whether upstream agent image already exposes a reliable health endpoint.
- Recommendation: inspect upstream container and add a bot-facing probe before finalizing `docker-compose.fullstack.yml`.
3. **Should prod mount root remain `/workspace` or be renamed to `/agents` externally?**
- What we know: current code defaults to `SURFACES_WORKSPACE_DIR=/workspace`, while deployment docs describe shared `/agents/`.
- What's unclear: whether external handoff wants a host path named `/agents` while containers still use `/workspace`.
- Recommendation: keep one in-container canonical path and let host-side naming vary only in Compose mounts.
## Environment Availability
| Dependency | Required By | Available | Version | Fallback |
|------------|------------|-----------|---------|----------|
| Python | bot runtime | ✓ | 3.14.3 | — |
| `uv` | dependency install | ✓ | 0.9.30 | `pip` |
| `pytest` | validation | ✓ | 9.0.2 installed | `python -m pytest` |
| Docker Engine | deployment packaging / E2E compose | ✓ | 29.1.3 | none |
| Docker Compose | split runtime orchestration | ✓ | 2.40.3 | none |
**Missing dependencies with no fallback:**
- None
**Missing dependencies with fallback:**
- None
## Validation Architecture
### Test Framework
| Property | Value |
|----------|-------|
| Framework | `pytest` + `pytest-asyncio` |
| Config file | `pyproject.toml` |
| Quick run command | `pytest tests/adapter/matrix/test_restart_persistence.py -v` |
| Full suite command | `pytest tests/ -v` |
### Phase Requirements → Test Map
| Req ID | Behavior | Test Type | Automated Command | File Exists? |
|--------|----------|-----------|-------------------|-------------|
| PH05-01 | Space+rooms onboarding remains primary UX | integration | `pytest tests/adapter/matrix/test_invite_space.py tests/adapter/matrix/test_chat_space.py -v` | ✅ |
| PH05-02 | Per-room `platform_chat_id` isolates routing and powers real clear | integration | `pytest tests/adapter/matrix/test_routed_platform.py tests/adapter/matrix/test_context_commands.py -v` | ✅ |
| PH05-03 | Restart reconciliation restores routing metadata | integration | `pytest tests/adapter/matrix/test_restart_persistence.py -v` | ❌ new reconciliation tests needed |
| PH05-04 | Shared-volume file transfer is room-safe | integration | `pytest tests/adapter/matrix/test_files.py tests/platform/test_real.py -v` | ✅ partial |
| PH05-05 | Split prod/fullstack compose artifacts stay coherent | smoke | `docker compose -f docker-compose.prod.yml config && docker compose -f docker-compose.fullstack.yml config` | ❌ Wave 0 |
### Sampling Rate
- **Per task commit:** `pytest tests/adapter/matrix/test_restart_persistence.py -v`
- **Per wave merge:** `pytest tests/adapter/matrix/ -v`
- **Phase gate:** `pytest tests/ -v` plus both compose files passing `docker compose ... config`
### Wave 0 Gaps
- [ ] `tests/adapter/matrix/test_reconciliation.py` — startup recovery of user/room metadata from Matrix state
- [ ] `tests/adapter/matrix/test_context_commands.py` additions — `!clear` command contract and room-local rotation semantics
- [ ] `tests/adapter/matrix/test_compose_artifacts.py` or equivalent smoke command documentation — split compose validation
- [ ] `tests/adapter/matrix/test_files.py` additions — cross-room attachment path isolation and shared-root consistency
## Sources
### Primary (HIGH confidence)
- Local repo code and tests:
- `adapter/matrix/bot.py`
- `adapter/matrix/store.py`
- `adapter/matrix/files.py`
- `adapter/matrix/routed_platform.py`
- `adapter/matrix/handlers/auth.py`
- `adapter/matrix/handlers/context_commands.py`
- `tests/adapter/matrix/test_restart_persistence.py`
- `tests/adapter/matrix/test_files.py`
- `tests/platform/test_real.py`
- Matrix-nio API docs: https://matrix-nio.readthedocs.io/en/latest/nio.html
- Matrix-nio async client docs: https://matrix-nio.readthedocs.io/en/latest/_modules/nio/client/async_client.html
- Matrix-nio PyPI release page: https://pypi.org/project/matrix-nio/
- Matrix spec Spaces / hierarchy: https://spec.matrix.org/v1.18/server-server-api/
- Matrix spec changelog note on `via` for `m.space.child`: https://spec.matrix.org/v1.16/changelog/v1.9/
- Docker Compose CLI reference: https://docs.docker.com/reference/cli/docker/compose/
- Docker Compose services reference: https://docs.docker.com/reference/compose-file/services/
### Secondary (MEDIUM confidence)
- `docs/deploy-architecture.md` — repo-local deployment contract clarified on 2026-04-27
- `docs/research/matrix-spaces.md` — prior internal research aligned with spec, but not treated as primary
- `README.md` runtime notes for current Matrix backend and shared workspace behavior
### Tertiary (LOW confidence)
- None
## Metadata
**Confidence breakdown:**
- Standard stack: HIGH - current repo stack verified against official docs and package registries
- Architecture: HIGH - recommendations align with existing runtime boundaries and official Matrix / Compose behavior
- Pitfalls: HIGH - derived from current code paths, existing tests, and official protocol/runtime semantics
**Research date:** 2026-04-28
**Valid until:** 2026-05-28

View file

@ -0,0 +1,83 @@
---
phase: 05
slug: mvp-deployment
status: revised
nyquist_compliant: true
wave_0_complete: false
created: 2026-04-28
---
# Phase 05 — Validation Strategy
> Per-phase validation contract for feedback sampling during execution.
---
## Test Infrastructure
| Property | Value |
|----------|-------|
| **Framework** | `pytest` + `pytest-asyncio` |
| **Config file** | `pyproject.toml` |
| **Quick run command** | `pytest tests/adapter/matrix/test_reconciliation.py tests/adapter/matrix/test_restart_persistence.py -v` |
| **Full suite command** | `pytest tests/ -v` |
| **Estimated runtime** | targeted slices < 60 seconds each; full suite longer |
---
## Sampling Rate
- **After every task commit:** Run the exact `<automated>` command from the task that just changed
- **After every plan wave:** Run `pytest tests/adapter/matrix/ -v`
- **Before `$gsd-verify-work`:** Full suite must be green
- **Max feedback latency:** 60 seconds for task-level slices
---
## Per-Task Verification Map
| Task ID | Plan | Wave | Requirement | Test Type | Automated Command | File Exists | Status |
|---------|------|------|-------------|-----------|-------------------|-------------|--------|
| 05-01-01 | 01 | 1 | PH05-01 | integration | `pytest tests/adapter/matrix/test_invite_space.py tests/adapter/matrix/test_chat_space.py tests/adapter/matrix/test_reconciliation.py tests/adapter/matrix/test_restart_persistence.py -v` | ❌ W0 | ⬜ pending |
| 05-01-02 | 01 | 1 | PH05-03 | integration | `pytest tests/adapter/matrix/test_invite_space.py tests/adapter/matrix/test_chat_space.py tests/adapter/matrix/test_reconciliation.py tests/adapter/matrix/test_restart_persistence.py tests/adapter/matrix/test_dispatcher.py -v` | ❌ W0 | ⬜ pending |
| 05-02-01 | 02 | 2 | PH05-02 | integration | `pytest tests/adapter/matrix/test_context_commands.py tests/adapter/matrix/test_routed_platform.py -v` | ✅ partial | ⬜ pending |
| 05-02-02 | 02 | 2 | PH05-02 | integration | `pytest tests/adapter/matrix/test_context_commands.py tests/adapter/matrix/test_routed_platform.py tests/adapter/matrix/test_dispatcher.py -v` | ✅ partial | ⬜ pending |
| 05-03-01 | 03 | 1 | PH05-04 | integration | `pytest tests/adapter/matrix/test_files.py tests/platform/test_real.py -v` | ✅ partial | ⬜ pending |
| 05-03-02 | 03 | 1 | PH05-04 | integration | `pytest tests/adapter/matrix/test_files.py tests/platform/test_real.py tests/adapter/matrix/test_send_outgoing.py -v` | ✅ partial | ⬜ pending |
| 05-04-01 | 04 | 2 | PH05-05 | smoke | `docker compose -f docker-compose.prod.yml config && docker compose -f docker-compose.fullstack.yml config` | ❌ W0 | ⬜ pending |
| 05-04-02 | 04 | 2 | PH05-05 | docs smoke | `rg -n "docker-compose\\.prod|docker-compose\\.fullstack|/agents|prod handoff|full-stack" README.md docs/deploy-architecture.md .env.example` | ✅ | ⬜ pending |
*Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky*
---
## Wave 0 Requirements
- [ ] `tests/adapter/matrix/test_reconciliation.py` — startup recovery of user and room metadata from Matrix state
- [ ] `tests/adapter/matrix/test_restart_persistence.py` additions — deterministic backfill for legacy rooms missing `platform_chat_id`
- [ ] `tests/adapter/matrix/test_context_commands.py` additions — room-local `!clear` rotation semantics
- [ ] `tests/adapter/matrix/test_files.py` additions — cross-room attachment isolation and shared-root consistency
- [ ] Compose smoke coverage or documented verification command for `docker-compose.prod.yml` and `docker-compose.fullstack.yml`
---
## Manual-Only Verifications
| Behavior | Requirement | Why Manual | Test Instructions |
|----------|-------------|------------|-------------------|
| Restart after real Matrix room topology exists | PH05-03 | Full recovery depends on live Space hierarchy and persisted homeserver state | Start the bot, provision a Space and chat rooms, stop the bot, remove local SQLite metadata, restart, confirm routing and room labels are rebuilt before live messages are handled |
| Shared `/agents` volume behavior across bot and platform containers | PH05-04 | Container mounts and permissions are environment-dependent | Run `docker compose -f docker-compose.fullstack.yml up`, upload a file in Matrix, confirm the agent sees the relative `workspace_path`, then confirm an agent-created file is readable back from the bot side |
| Operator handoff of prod compose | PH05-05 | Final deploy contract depends on real env files and target host conventions | Run `docker compose -f docker-compose.prod.yml config` on the target deployment checkout and confirm only bot services, required env vars, and shared volumes are present |
---
## Validation Sign-Off
- [ ] All tasks have `<automated>` verify or Wave 0 dependencies
- [ ] Sampling continuity: no 3 consecutive tasks without automated verify
- [ ] Wave 0 covers all MISSING references
- [ ] No watch-mode flags
- [x] Feedback latency target tightened to task slices under 60s
- [x] `nyquist_compliant: true` set in frontmatter
**Approval:** pending

46
Dockerfile Normal file
View file

@ -0,0 +1,46 @@
FROM python:3.11-slim AS base
WORKDIR /app
RUN useradd -u 1000 -m appuser
USER appuser
ENV PYTHONUNBUFFERED=1
ENV PYTHONPATH=/app
ENV UV_PROJECT_ENVIRONMENT=/usr/local
# Install uv and git for reproducible platform SDK installation.
RUN apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates git \
&& rm -rf /var/lib/apt/lists/* \
&& pip install --no-cache-dir uv
# Copy dependency manifests first for layer caching.
COPY pyproject.toml uv.lock* ./
# Install project dependencies into the system environment.
RUN uv sync --no-dev --no-install-project --frozen
FROM base AS development
COPY . .
RUN uv sync --no-dev --frozen
# Local fullstack/dev builds can override the SDK with a checked-out agent_api
# build context, matching platform-agent's development Dockerfile pattern.
COPY --from=agent_api . /agent_api/
RUN python -m pip install --no-cache-dir --ignore-requires-python -e /agent_api/
CMD ["python", "-m", "adapter.matrix.bot"]
FROM base AS production
COPY . .
RUN uv sync --no-dev --frozen
# Production builds follow the platform-agent pattern: install the API SDK from
# the platform Git repository instead of relying on local external/ clones.
ARG LAMBDA_AGENT_API_REF=master
RUN python -m pip install --no-cache-dir --ignore-requires-python \
"git+https://git.lambda.coredump.ru/platform/agent_api.git@${LAMBDA_AGENT_API_REF}"
CMD ["python", "-m", "adapter.matrix.bot"]

344
README.md
View file

@ -1,25 +1,54 @@
# Lambda Lab 3.0 — Surfaces
Команда поверхностей. Telegram и Matrix боты для взаимодействия пользователя с AI-агентом Lambda.
Matrix-бот для взаимодействия пользователя с AI-агентом Lambda.
## Статус
## Интеграция для платформы
Прототип в разработке. SDK платформы ещё не готов — работаем через `MockPlatformClient`.
Бот — это один Docker-контейнер (`matrix-bot`), который вы добавляете в свою инфраструктуру рядом с агентами. Production target — один surface container на 25-30 внешних agent containers/services.
| Поверхность | Статус | Описание |
|---|---|---|
| Telegram | 🔨 В разработке | DM + Forum Topics mode, активная реализация сейчас в отдельном worktree |
| Matrix | 🔨 В разработке | Незашифрованные комнаты: бот создаёт private Space и отдельную room на каждый чат |
### Что бот ожидает от вас
**1. HTTP-эндпоинт агента**
Бот подключается к агенту через `lambda_agent_api.AgentApi` по адресу `AGENT_BASE_URL`.
Протокол — WebSocket поверх HTTP, контракт описан в `docs/deploy-architecture.md`.
**2. Shared volume с per-agent поддиректориями**
Shared volume монтируется в бот как `/agents`. Каждый агент видит свою поддиректорию.
```
Bot container Agent containers
/agents/0/ ←── volume ──→ agent_0: /workspace/
/agents/1/ ←── volume ──→ agent_1: /workspace/
/agents/N/ ←── volume ──→ agent_N: /workspace/
```
- Бот сохраняет входящий файл прямо в `{workspace_path}/{file}` и передаёт агенту `attachments=["{file}"]`
- Если файл с таким именем уже есть, бот сохраняет следующий как `file (1).ext`, `file (2).ext`, как в Windows
- Агент пишет исходящий файл прямо в свой `/workspace/file`, бот читает его из `{workspace_path}/file`
- `workspace_path` для каждого агента задаётся в `config/matrix-agents.yaml`
**3. Конфиг агентов**
Файл `config/matrix-agents.yaml` — маппинг Matrix-пользователей на агентов. Вы заполняете его под свою инфраструктуру. Пример в `config/matrix-agents.example.yaml`.
### Что бот не делает
- Не управляет lifecycle агент-контейнеров (запуск/остановка — на вашей стороне)
- Не хранит историю разговоров (это в памяти агента)
- Не обрабатывает аутентификацию пользователей — любой Matrix-пользователь, который пишет боту, получает доступ
### Минимальный чеклист
- [ ] Взять опубликованный image `SURFACES_BOT_IMAGE` или собрать production image из этого репозитория
- [ ] Заполнить `config/matrix-agents.yaml` — ID агентов, `base_url` каждого, `workspace_path`, маппинг пользователей
- [ ] Задать переменные окружения (см. `.env.example`): Matrix credentials, `MATRIX_PLATFORM_BACKEND=real`, `MATRIX_AGENT_REGISTRY_PATH=/app/config/matrix-agents.yaml`
- [ ] Смонтировать в бот-контейнер shared volume как `/agents` — каждый агент должен видеть свою поддиректорию как `/workspace`
- [ ] Добавить bot-only service в свой compose; агенты в этот compose не входят и управляются платформой
---
## Концепция
## Статус
Пользователь получает персонального AI-агента через привычный мессенджер.
Агент выполняет реальные задачи: разбирает почту, ищет информацию, работает с файлами, управляет календарём.
**Поверхности** — тонкие клиенты. Вся бизнес-логика на стороне платформы.
Задача команды: сделать интерфейс удобным, надёжным и легко расширяемым.
Matrix MVP готов к деплою. Telegram — в отдельном worktree, не входит в этот handoff.
---
@ -30,121 +59,224 @@ surfaces-bot/
core/ — общее ядро, не зависит от транспорта
protocol.py — унифицированные структуры (IncomingMessage, OutgoingUI, ...)
handler.py — EventDispatcher: IncomingEvent → OutgoingEvent
handlers/ — обработчики по типам событий
store.py — StateStore Protocol + InMemoryStore + SQLiteStore
chat.py — ChatManager: метаданные чатов C1/C2/C3
auth.py — AuthManager: аутентификация
settings.py — SettingsManager: коннекторы, скиллы, SOUL, безопасность
chat.py — ChatManager
auth.py — AuthManager
settings.py — SettingsManager
adapter/
telegram/ — aiogram 3.x адаптер
matrix/ — matrix-nio адаптер
sdk/
interface.py — PlatformClient Protocol (контракт к SDK)
mock.py — MockPlatformClient (заглушка)
real.py — RealPlatformClient (через AgentApi)
mock.py — MockPlatformClient (заглушка для тестов)
config/
matrix-agents.yaml — реестр агентов
docs/ — документация
.claude/agents/ — агенты для Claude Code
```
**Ключевой принцип:** добавить новую поверхность = написать один адаптер-конвертер.
Ядро (`core/`) не трогается. Подробнее: [`docs/surface-protocol.md`](docs/surface-protocol.md)
Подробнее: [`docs/surface-protocol.md`](docs/surface-protocol.md)
---
## Функционал прототипа
## Деплой
### Telegram ([подробнее](docs/telegram-prototype.md))
- **Чаты** — основной Telegram UX сейчас развивается в отдельном worktree `feat/telegram-adapter`
- **Forum Topics mode** — бот умеет подключать forum-группу через `/forum`; чат может быть привязан к отдельной теме
- **DM-режим** — базовый диалог и переключение чатов сохраняются
- **Аутентификация** — привязка Telegram аккаунта к аккаунту платформы
- **Диалог** — typing indicator, передача файлов, подтверждение опасных действий через inline-кнопки
- **Настройки** через `/settings`: коннекторы (Gmail, GitHub, Notion...), скиллы, личность агента (SOUL), безопасность, подписка
### Matrix ([подробнее](docs/matrix-prototype.md))
- **Онбординг** — при первом invite бот создаёт private Space `Lambda — {display_name}` и первую комнату `Чат 1`, сразу приглашая туда пользователя
- **Чаты**`!new`, `!chats`, `!rename`, `!archive`, `!help`; новые комнаты регистрируются в локальном `ChatManager`
- **Диалог** — сообщения, вложения, подтверждения `!yes` / `!no` и routing через `EventDispatcher`
- **Стабильность** — перед `sync_forever()` бот делает bootstrap sync и стартует с `since`, чтобы не переигрывать старую timeline после рестарта
- **Текущее ограничение** — encrypted DM пока не поддержан; ручное тестирование Matrix ведётся в незашифрованных комнатах и зависит от локального state-store бота
---
## Замена SDK
Вся работа с платформой идёт через `PlatformClient` Protocol:
```python
class PlatformClient(Protocol):
async def get_or_create_user(self, external_id: str, platform: str, ...) -> User: ...
async def send_message(self, user_id: str, chat_id: str, text: str, ...) -> MessageResponse: ...
async def get_settings(self, user_id: str) -> UserSettings: ...
async def update_settings(self, user_id: str, action: Any) -> None: ...
```
Бот не управляет lifecycle контейнеров — это делает Master (платформа).
Бот передаёт `user_id` + `chat_id` + сообщение; платформа сама решает нужно ли поднять контейнер.
Сейчас: `MockPlatformClient` в `sdk/mock.py`.
Когда SDK готов: добавляем `SdkPlatformClient`, меняем одну строку в DI. Адаптеры и ядро не трогаем.
---
## Быстрый старт
### Переменные окружения
```bash
# Зависимости
uv sync # или: pip install -e ".[dev]"
cp .env.example .env
```
# Тесты
| Переменная | Обязательна | Описание |
|---|---|---|
| `MATRIX_HOMESERVER` | ✓ | URL Matrix-сервера |
| `MATRIX_USER_ID` | ✓ | `@bot:example.org` |
| `MATRIX_PASSWORD` | ✓ | пароль (или `MATRIX_ACCESS_TOKEN`) |
| `MATRIX_PLATFORM_BACKEND` | ✓ | `real` для продакшна |
| `SURFACES_BOT_IMAGE` | ✓ | Docker image поверхности: `mput1/surfaces-bot:latest` |
| `AGENT_BASE_URL` | | Fallback URL агента если `base_url` не задан в `matrix-agents.yaml` |
| `MATRIX_AGENT_REGISTRY_PATH` | ✓ | путь к реестру внутри контейнера: `/app/config/matrix-agents.yaml` |
| `SURFACES_WORKSPACE_DIR` | | путь к shared volume в контейнере (по умолчанию `/agents`) |
| `SURFACES_SHARED_VOLUME` | | имя Docker volume (по умолчанию `surfaces-agents`) |
### Реестр агентов
`config/matrix-agents.yaml` — статический маппинг пользователей на агентов:
```yaml
user_agents:
"@user0:matrix.lambda.coredump.ru": agent-0
"@user1:matrix.lambda.coredump.ru": agent-1
agents:
- id: agent-0
label: "Agent 0"
base_url: "http://lambda.coredump.ru:7000/agent_0/"
workspace_path: "/agents/0"
- id: agent-1
label: "Agent 1"
base_url: "http://lambda.coredump.ru:7000/agent_1/"
workspace_path: "/agents/1"
- id: agent-2
label: "Agent 2"
base_url: "http://lambda.coredump.ru:7000/agent_2/"
workspace_path: "/agents/2"
```
- `user_agents` — маппинг Matrix user_id → agent_id. Если пользователь не найден — используется первый агент.
- `base_url` — HTTP URL агент-эндпоинта (path-based routing через reverse proxy).
- `workspace_path` — путь к воркспейсу агента внутри бот-контейнера на shared volume.
Бот сохраняет входящие файлы прямо в `{workspace_path}/`, агент пишет исходящие прямо в свой `/workspace/`.
- Для 25-30 агентов продолжайте тот же паттерн: `/agent_17/` + `/agents/17`, `/agent_29/` + `/agents/29`.
Полный пример с комментариями: `config/matrix-agents.example.yaml`
### Production (bot-only)
`docker-compose.prod.yml` — bot-only handoff через published image. Платформа добавляет этот сервис в свой compose рядом с agent containers, монтирует shared volume и задаёт переменные окружения. Этот compose не создаёт и не собирает агент-контейнеры.
Перед redeploy можно проверить реальные agent routes из той же сети, где будет работать бот:
```bash
PYTHONPATH=. uv run python -m tools.check_matrix_agents \
--config config/matrix-agents.yaml \
--timeout 5
```
Проверка открывает фактический WebSocket URL каждого агента (`.../v1/agent_ws/{chat_id}/`) и ждёт первый `STATUS`. Для проверки полного запроса к агенту добавьте `--message "ping"`.
Для запуска опубликованного image:
```bash
export SURFACES_BOT_IMAGE=mput1/surfaces-bot:latest
docker compose --env-file .env -f docker-compose.prod.yml up -d
```
Опубликованный image:
```text
mput1/surfaces-bot:latest
sha256:2f135f3535f7765d4377b440cdabe41195ad2efbc3e175def159ae4689ef90bd
```
Для сборки и публикации surface image:
```bash
docker login
export SURFACES_BOT_IMAGE=mput1/surfaces-bot:latest
docker build --target production \
--build-arg LAMBDA_AGENT_API_REF=master \
-t "$SURFACES_BOT_IMAGE" .
docker push "$SURFACES_BOT_IMAGE"
```
Если push возвращает `insufficient_scope`, текущий Docker login не имеет доступа к namespace/repository. Создайте repository в нужном Docker Hub namespace или перетегируйте image в namespace пользователя, под которым выполнен `docker login`.
### Fullstack E2E (bot + agent)
```bash
docker compose --env-file .env -f docker-compose.fullstack.yml up --build
```
Поднимает `matrix-bot` вместе с локальным `platform-agent`. Это internal harness, не production topology на 25-30 агентов. `matrix-bot` собирается через Dockerfile target `development` и локальный `external/platform-agent_api` context, как в dev-сборке `platform-agent`. `AGENT_BASE_URL` перекрывается на `http://platform-agent:8000`. Shared volume виден как `/agents` в боте и `/workspace` в агенте.
### Сброс состояния (локально)
```bash
rm -f lambda_matrix.db && rm -rf matrix_store
```
---
## Shared volume: передача файлов
```
Bot (/agents) Agent (/workspace = /agents/N/)
/agents/0/report.pdf ←──── одно и то же хранилище ────→ /workspace/report.pdf
/agents/0/result.txt ←────────────────────────────────→ /workspace/result.txt
```
- **Входящий файл** (пользователь → агент): бот сохраняет в `{workspace_path}/{file}`, например `/agents/17/report.pdf`, и передаёт агенту `attachments=["report.pdf"]`
- **Коллизии имён**: если `/agents/17/report.pdf` уже существует, бот сохранит следующий файл как `/agents/17/report (1).pdf`, затем `/agents/17/report (2).pdf`
- **Исходящий файл** (агент → пользователь): агент пишет в `/workspace/file`, бот читает из `{workspace_path}/file`, например `/agents/17/result.txt`, и отправляет пользователю как Matrix file message
- `workspace_path` для каждого агента задаётся в `config/matrix-agents.yaml`
---
## Онбординг пользователя
1. Пользователь приглашает бота в личные сообщения (DM) на Matrix-сервере
2. Бот создаёт private Space `Lambda — {display_name}` и комнату `Чат 1`
3. Дальнейшее общение — в рабочих комнатах, не в DM
**Требование:** незашифрованные комнаты. E2EE не поддержан.
---
## Команды Matrix
### Работающие
| Команда | Действие |
|---|---|
| *(любое сообщение)* | Диалог с агентом, стриминг ответа |
| `!new [название]` | Создать новый чат |
| `!chats` | Список активных чатов |
| `!rename <название>` | Переименовать текущую комнату |
| `!archive` | Архивировать чат |
| `!clear` | Сбросить контекст текущего чата |
| `!yes` / `!no` | Подтвердить / отменить действие агента |
| `!list` | Файлы в очереди вложений |
| `!remove <n>` / `!remove all` | Удалить вложение из очереди |
| `!help` | Справка |
### Не работают / заглушки
| Команда | Статус |
|---|---|
| `!save` / `!load` / `!context` | Нестабильны: зависят от агента, сессии теряются при рестарте |
| `!settings` и подкоманды | Заглушки MVP, требуют готового SDK платформы |
---
## Отправка файлов агенту
Matrix-клиент отправляет файлы и текст отдельными событиями. Файл без текстовой инструкции ставится в очередь.
```
[отправил файл]
!list
1. report.pdf
прочитай и сделай summary ← файл уйдёт агенту вместе с этим текстом
```
---
## Известные ограничения
| Проблема | Причина |
|---|---|
| История теряется при рестарте агента | `platform-agent` использует `MemorySaver` (in-memory) |
| E2EE | `python-olm` не собирается на macOS/ARM |
---
## Разработка
```bash
uv sync
pytest tests/ -v
# Запустить Matrix бота
cp .env.example .env # заполнить MATRIX_* переменные
PYTHONPATH=. uv run python -m adapter.matrix.bot
pytest tests/adapter/matrix/ -v # только Matrix
```
### Telegram worktree
Текущая Telegram-разработка идёт в отдельном worktree:
```bash
cd .worktrees/telegram
export BOT_TOKEN=...
PYTHONPATH=. python -m adapter.telegram.bot
```
### Matrix manual QA
Пока Matrix-бот тестируется в незашифрованных комнатах:
```bash
cd /path/to/surfaces-bot
rm -f lambda_matrix.db
rm -rf matrix_store
PYTHONPATH=. uv run python -m adapter.matrix.bot
```
---
## Документация
| Файл | Содержание |
|---|---|
| [`docs/surface-protocol.md`](docs/surface-protocol.md) | Унификация поверхностей — все структуры, как добавить новую поверхность |
| [`docs/telegram-prototype.md`](docs/telegram-prototype.md) | Функционал Telegram прототипа |
| [`docs/matrix-prototype.md`](docs/matrix-prototype.md) | Функционал Matrix прототипа |
| [`docs/api-contract.md`](docs/api-contract.md) | Контракт к SDK платформы |
| [`docs/user-flow.md`](docs/user-flow.md) | FSM и user journey |
| [`docs/claude-code-guide.md`](docs/claude-code-guide.md) | Гайд по работе с Claude Code |
---
## Команда
Поверхности и интеграции
Lambda Lab 3.0, МАИ
| [`docs/deploy-architecture.md`](docs/deploy-architecture.md) | **Читать первым.** Топология деплоя, volume-контракт, AgentApi, конфигурация |
| [`docs/matrix-prototype.md`](docs/matrix-prototype.md) | Команды бота, UX, передача файлов |
| [`docs/known-limitations.md`](docs/known-limitations.md) | Известные ограничения и обходные пути |
| [`docs/surface-protocol.md`](docs/surface-protocol.md) | Внутренний протокол событий (для расширения) |
| [`docs/new-surface-guide.md`](docs/new-surface-guide.md) | Руководство по созданию новой поверхности (Discord, Slack и др.) |

View file

@ -0,0 +1,125 @@
from __future__ import annotations
from collections.abc import Mapping
from dataclasses import dataclass, field
from pathlib import Path
from typing import Literal
import yaml
class AgentRegistryError(ValueError):
pass
@dataclass(frozen=True)
class AgentDefinition:
agent_id: str
label: str
base_url: str = field(default="")
workspace_path: str = field(default="")
@dataclass(frozen=True)
class AgentAssignment:
agent_id: str | None
source: Literal["configured", "default", "none"]
@property
def is_default(self) -> bool:
return self.source == "default"
class AgentRegistry:
def __init__(
self,
agents: list[AgentDefinition],
user_agents: Mapping[str, str] | None = None,
) -> None:
self.agents = tuple(agents)
self._by_id = {agent.agent_id: agent for agent in self.agents}
self._user_agents: dict[str, str] = dict(user_agents or {})
def get(self, agent_id: str) -> AgentDefinition:
try:
return self._by_id[agent_id]
except KeyError as exc:
raise AgentRegistryError(f"unknown agent id: {agent_id}") from exc
def get_agent_id_for_user(self, matrix_user_id: str) -> str | None:
return self._user_agents.get(matrix_user_id)
def resolve_agent_for_user(self, matrix_user_id: str) -> AgentAssignment:
agent_id = self.get_agent_id_for_user(matrix_user_id)
if agent_id is not None:
return AgentAssignment(agent_id=agent_id, source="configured")
if self.agents:
return AgentAssignment(agent_id=self.agents[0].agent_id, source="default")
return AgentAssignment(agent_id=None, source="none")
def _required_text(entry: Mapping[str, object], key: str) -> str:
value = entry.get(key)
if not isinstance(value, str):
raise AgentRegistryError("each agent entry requires id and label")
text = value.strip()
if not text:
raise AgentRegistryError("each agent entry requires id and label")
return text
def _optional_text(entry: Mapping[str, object], key: str) -> str:
value = entry.get(key)
if value is None:
return ""
if not isinstance(value, str):
raise AgentRegistryError(f"agent entry field '{key}' must be a string")
return value.strip()
def _load_registry_data(path: str | Path) -> dict[str, object]:
try:
raw = yaml.safe_load(Path(path).read_text(encoding="utf-8"))
except yaml.YAMLError as exc:
raise AgentRegistryError("invalid agent registry YAML") from exc
if raw is None:
return {}
if not isinstance(raw, Mapping):
raise AgentRegistryError("agent registry must be a mapping with an agents list")
return dict(raw)
def load_agent_registry(path: str | Path) -> AgentRegistry:
raw = _load_registry_data(path)
entries = raw.get("agents")
if not isinstance(entries, list) or not entries:
raise AgentRegistryError("agents registry must contain a non-empty agents list")
agents: list[AgentDefinition] = []
seen: set[str] = set()
for entry in entries:
if not isinstance(entry, Mapping):
raise AgentRegistryError("each agent entry requires id and label")
agent_id = _required_text(entry, "id")
label = _required_text(entry, "label")
base_url = _optional_text(entry, "base_url")
workspace_path = _optional_text(entry, "workspace_path")
if agent_id in seen:
raise AgentRegistryError(f"duplicate agent id: {agent_id}")
seen.add(agent_id)
agents.append(
AgentDefinition(
agent_id=agent_id,
label=label,
base_url=base_url,
workspace_path=workspace_path,
)
)
user_agents = raw.get("user_agents")
if user_agents is not None:
if not isinstance(user_agents, Mapping):
raise AgentRegistryError("user_agents must be a mapping of user_id to agent_id")
user_agents = {str(k): str(v) for k, v in user_agents.items()}
return AgentRegistry(agents, user_agents)

View file

@ -1,32 +1,71 @@
from __future__ import annotations
import asyncio
import logging
import os
import re
from dataclasses import dataclass
from pathlib import Path
from urllib.parse import urlsplit, urlunsplit
import structlog
from dotenv import load_dotenv
from nio import (
AsyncClient,
AsyncClientConfig,
InviteMemberEvent,
MatrixRoom,
RoomMemberEvent,
RoomMessage,
RoomMessageAudio,
RoomMessageFile,
RoomMessageImage,
RoomMessageText,
RoomMessageVideo,
)
from nio.responses import SyncResponse
from dotenv import load_dotenv
from adapter.matrix.agent_registry import AgentRegistry, AgentRegistryError, load_agent_registry
from adapter.matrix.converter import from_room_event
from adapter.matrix.files import (
download_matrix_attachment,
matrix_msgtype_for_attachment,
resolve_workspace_attachment_path,
)
from adapter.matrix.handlers import register_matrix_handlers
from adapter.matrix.handlers.auth import handle_invite
from adapter.matrix.handlers.auth import (
default_agent_notice,
handle_invite,
provision_workspace_chat,
restore_workspace_access,
)
from adapter.matrix.handlers.context_commands import (
LOAD_PROMPT,
)
from adapter.matrix.reconciliation import reconcile_startup_state
from adapter.matrix.room_router import resolve_chat_id
from adapter.matrix.store import get_room_meta, set_pending_confirm
from adapter.matrix.routed_platform import RoutedPlatformClient
from adapter.matrix.store import (
add_staged_attachment,
clear_load_pending,
clear_staged_attachments,
get_load_pending,
get_room_meta,
get_staged_attachments,
next_platform_chat_id,
remove_staged_attachment_at,
set_pending_confirm,
set_platform_chat_id,
set_room_meta,
)
from core.auth import AuthManager
from core.chat import ChatManager
from core.handler import EventDispatcher
from core.handlers import register_all
from core.protocol import (
Attachment,
IncomingCommand,
IncomingMessage,
OutgoingEvent,
OutgoingMessage,
OutgoingNotification,
@ -35,7 +74,10 @@ from core.protocol import (
)
from core.settings import SettingsManager
from core.store import InMemoryStore, SQLiteStore, StateStore
from sdk.interface import PlatformClient, PlatformError
from sdk.mock import MockPlatformClient
from sdk.prototype_state import PrototypeStateStore
from sdk.real import RealPlatformClient
logger = structlog.get_logger(__name__)
@ -44,41 +86,161 @@ load_dotenv(Path(__file__).resolve().parents[2] / ".env")
@dataclass
class MatrixRuntime:
platform: MockPlatformClient
platform: PlatformClient
store: StateStore
chat_mgr: ChatManager
auth_mgr: AuthManager
settings_mgr: SettingsManager
dispatcher: EventDispatcher
agent_routing_enabled: bool = False
registry: AgentRegistry | None = None
def build_event_dispatcher(platform: MockPlatformClient, store: StateStore) -> EventDispatcher:
def build_event_dispatcher(platform: PlatformClient, store: StateStore) -> EventDispatcher:
chat_mgr = ChatManager(platform, store)
auth_mgr = AuthManager(platform, store)
settings_mgr = SettingsManager(platform, store)
prototype_state = getattr(platform, "_prototype_state", None)
agent_base_url = _agent_base_url_from_env()
registry = _load_agent_registry_from_env()
dispatcher = EventDispatcher(
platform=platform, chat_mgr=chat_mgr, auth_mgr=auth_mgr, settings_mgr=settings_mgr
)
register_all(dispatcher)
register_matrix_handlers(dispatcher, store=store)
register_matrix_handlers(
dispatcher,
store=store,
registry=registry,
prototype_state=prototype_state,
agent_base_url=agent_base_url,
)
return dispatcher
def _normalize_agent_base_url(url: str) -> str:
parsed = urlsplit(url)
path = re.sub(r"(?:/v1)?/agent_ws(?:/[^/]+)?/?$", "", parsed.path.rstrip("/"))
return urlunsplit((parsed.scheme, parsed.netloc, path, "", ""))
def _ws_debug_enabled() -> bool:
value = os.environ.get("SURFACES_DEBUG_WS", "")
return value.strip().lower() in {"1", "true", "yes", "on"}
def _configure_debug_logging() -> None:
if not _ws_debug_enabled():
return
root_logger = logging.getLogger()
if not root_logger.handlers:
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s [%(levelname)-8s] %(name)s %(message)s",
)
elif root_logger.level > logging.INFO:
root_logger.setLevel(logging.INFO)
logging.getLogger("lambda_agent_api").setLevel(logging.INFO)
logging.getLogger("lambda_agent_api.agent_api").setLevel(logging.INFO)
def _agent_base_url_from_env() -> str:
if base_url := os.environ.get("AGENT_BASE_URL"):
return base_url
if ws_url := os.environ.get("AGENT_WS_URL"):
return _normalize_agent_base_url(ws_url)
return "http://127.0.0.1:8000"
def _load_agent_registry_from_env(required: bool = False) -> AgentRegistry | None:
registry_path = os.environ.get("MATRIX_AGENT_REGISTRY_PATH", "").strip()
if not registry_path:
if required:
raise RuntimeError(
"MATRIX_AGENT_REGISTRY_PATH is required when MATRIX_PLATFORM_BACKEND=real"
)
return None
try:
registry = load_agent_registry(registry_path)
except (AgentRegistryError, OSError) as exc:
raise RuntimeError(f"failed to load matrix agent registry: {registry_path}") from exc
if _ws_debug_enabled():
logger.warning(
"matrix_agent_registry_loaded",
registry_path=registry_path,
agent_count=len(registry.agents),
)
for agent in registry.agents:
logger.warning(
"matrix_agent_registry_entry",
registry_path=registry_path,
agent_id=agent.agent_id,
label=agent.label,
configured_base_url=agent.base_url,
normalized_base_url=_normalize_agent_base_url(agent.base_url)
if agent.base_url
else "",
workspace_path=agent.workspace_path,
)
return registry
def _build_platform_from_env(*, store: StateStore, chat_mgr: ChatManager) -> PlatformClient:
backend = os.environ.get("MATRIX_PLATFORM_BACKEND", "mock").strip().lower()
if _ws_debug_enabled():
logger.warning(
"matrix_platform_backend_selected",
backend=backend,
global_agent_base_url=_agent_base_url_from_env(),
registry_path=os.environ.get("MATRIX_AGENT_REGISTRY_PATH", "").strip(),
)
if backend == "real":
prototype_state = PrototypeStateStore()
registry = _load_agent_registry_from_env(required=True)
assert registry is not None
global_base_url = _agent_base_url_from_env()
delegates = {
agent.agent_id: RealPlatformClient(
agent_id=agent.agent_id,
agent_base_url=agent.base_url or global_base_url,
prototype_state=prototype_state,
platform="matrix",
)
for agent in registry.agents
}
return RoutedPlatformClient(
chat_mgr=chat_mgr,
store=store,
delegates=delegates,
)
return MockPlatformClient()
def build_runtime(
platform: MockPlatformClient | None = None,
platform: PlatformClient | None = None,
store: StateStore | None = None,
client: AsyncClient | None = None,
) -> MatrixRuntime:
platform = platform or MockPlatformClient()
store = store or InMemoryStore()
chat_mgr = ChatManager(platform, store)
platform = platform or _build_platform_from_env(store=store, chat_mgr=chat_mgr)
chat_mgr = ChatManager(platform, store)
auth_mgr = AuthManager(platform, store)
settings_mgr = SettingsManager(platform, store)
prototype_state = getattr(platform, "_prototype_state", None)
agent_base_url = _agent_base_url_from_env()
registry = _load_agent_registry_from_env()
dispatcher = EventDispatcher(
platform=platform, chat_mgr=chat_mgr, auth_mgr=auth_mgr, settings_mgr=settings_mgr
)
register_all(dispatcher)
register_matrix_handlers(dispatcher, client=client, store=store)
register_matrix_handlers(
dispatcher,
client=client,
store=store,
registry=registry,
prototype_state=prototype_state,
agent_base_url=agent_base_url,
)
return MatrixRuntime(
platform=platform,
store=store,
@ -86,6 +248,8 @@ def build_runtime(
auth_mgr=auth_mgr,
settings_mgr=settings_mgr,
dispatcher=dispatcher,
agent_routing_enabled=isinstance(platform, RoutedPlatformClient),
registry=registry,
)
@ -94,15 +258,524 @@ class MatrixBot:
self.client = client
self.runtime = runtime
async def _ensure_platform_chat_id(self, room_id: str, room_meta: dict | None) -> None:
if not room_meta:
return
if room_meta.get("redirect_room_id"):
return
if room_meta.get("platform_chat_id"):
return
await set_platform_chat_id(
self.runtime.store,
room_id,
await next_platform_chat_id(self.runtime.store),
)
async def _refresh_room_agent_assignment(
self, room_id: str, matrix_user_id: str, room_meta: dict | None
) -> tuple[dict | None, bool]:
if not room_meta or room_meta.get("redirect_room_id") or self.runtime.registry is None:
return room_meta, False
assignment = self.runtime.registry.resolve_agent_for_user(matrix_user_id)
updated = dict(room_meta)
should_warn_default = False
if assignment.source == "configured" and (
updated.get("agent_id") != assignment.agent_id
or updated.get("agent_assignment") != "configured"
):
updated["agent_id"] = assignment.agent_id
updated["agent_assignment"] = "configured"
updated.pop("default_agent_notice_sent", None)
elif assignment.source == "default":
if not updated.get("agent_id"):
updated["agent_id"] = assignment.agent_id
if updated.get("agent_id") == assignment.agent_id:
updated["agent_assignment"] = "default"
should_warn_default = not updated.get("default_agent_notice_sent")
updated["default_agent_notice_sent"] = True
if updated != room_meta:
await set_room_meta(self.runtime.store, room_id, updated)
return updated, should_warn_default
return room_meta, should_warn_default
async def on_room_message(self, room: MatrixRoom, event: RoomMessageText) -> None:
if getattr(event, "sender", None) == self.client.user_id:
return
chat_id = await resolve_chat_id(self.runtime.store, room.room_id, event.sender)
incoming = from_room_event(event, room_id=room.room_id, chat_id=chat_id)
sender = getattr(event, "sender", None)
body = (getattr(event, "body", None) or "").strip()
room_meta = await get_room_meta(self.runtime.store, room.room_id)
if room_meta is not None and not room_meta.get("redirect_room_id"):
await self._ensure_platform_chat_id(room.room_id, room_meta)
room_meta, warn_default_agent = await self._refresh_room_agent_assignment(
room.room_id, sender, room_meta
)
if warn_default_agent and not body.startswith("!"):
await self._send_all(
room.room_id,
[OutgoingMessage(chat_id=room.room_id, text=default_agent_notice())],
)
load_pending = await get_load_pending(self.runtime.store, sender, room.room_id)
if load_pending is not None and (body.isdigit() or body == "!cancel"):
outgoing = await self._handle_load_selection(sender, room.room_id, body, load_pending)
await self._send_all(room.room_id, outgoing)
return
if room_meta is None:
outgoing = await self._bootstrap_unregistered_room(room, sender)
if outgoing:
await self._send_all(room.room_id, outgoing)
return
elif room_meta.get("redirect_room_id"):
display_name = getattr(room, "display_name", None) or sender
if body == "!new":
try:
created = await provision_workspace_chat(
self.client,
sender,
display_name,
self.runtime.platform,
self.runtime.store,
self.runtime.auth_mgr,
self.runtime.chat_mgr,
registry=self.runtime.registry,
)
except Exception as exc:
logger.warning(
"matrix_entry_room_new_chat_failed",
room_id=room.room_id,
sender=sender,
error=str(exc),
)
await self._send_all(
room.room_id,
[
OutgoingMessage(
chat_id=room.room_id,
text="Не удалось создать новый рабочий чат.",
)
],
)
return
welcome = f"Создал новый рабочий чат {created['room_name']}."
if created.get("agent_assignment") == "default":
welcome = f"{welcome}\n\n{default_agent_notice()}"
await self.client.room_send(
created["chat_room_id"],
"m.room.message",
{"msgtype": "m.text", "body": welcome},
)
await set_room_meta(
self.runtime.store,
room.room_id,
{
**room_meta,
"redirect_room_id": created["chat_room_id"],
"redirect_chat_id": created["chat_id"],
},
)
await self._send_all(
room.room_id,
[
OutgoingMessage(
chat_id=room.room_id,
text=(
f"Создал рабочий чат {created['room_name']} "
f"({created['chat_id']}) и отправил приглашение."
),
)
],
)
return
restored = await restore_workspace_access(
self.client,
sender,
display_name,
self.runtime.platform,
self.runtime.store,
self.runtime.auth_mgr,
self.runtime.chat_mgr,
registry=self.runtime.registry,
)
redirect_room_id = room_meta["redirect_room_id"]
redirect_chat_id = room_meta.get("redirect_chat_id", "рабочий чат")
if restored.get("created_new_chat"):
text = (
f"Создал новый рабочий чат {restored['room_name']} "
f"({restored['chat_id']}) и отправил приглашение."
)
else:
text = (
f"Рабочий чат уже создан: {redirect_chat_id}. "
"Я повторно отправил приглашения в пространство Lambda и рабочие чаты. "
"Чтобы создать новый чат, напишите !new здесь."
)
await self._send_all(
room.room_id,
[
OutgoingMessage(
chat_id=room.room_id,
text=text,
)
],
)
logger.info(
"matrix_redirect_entry_room",
room_id=room.room_id,
redirect_room_id=redirect_room_id,
user=sender,
)
return
if not body.startswith("!") and self.runtime.agent_routing_enabled:
pass
local_chat_id = await resolve_chat_id(self.runtime.store, room.room_id, sender)
incoming = from_room_event(event, room_id=room.room_id, chat_id=local_chat_id)
if incoming is None:
return
outgoing = await self.runtime.dispatcher.dispatch(incoming)
await self._send_all(room.room_id, outgoing)
if isinstance(incoming, IncomingCommand) and incoming.command in {
"matrix_list_attachments",
"matrix_remove_attachment",
}:
outgoing = await self._handle_staged_attachment_command(
room.room_id,
sender,
incoming,
)
await self._send_all(room.room_id, outgoing)
return
if self._is_file_only_event(event, incoming):
materialized = await self._materialize_incoming_attachments(
room.room_id,
sender,
incoming,
)
await self._stage_attachments(room.room_id, sender, materialized.attachments)
return
if isinstance(incoming, IncomingMessage) and incoming.attachments:
incoming = await self._materialize_incoming_attachments(
room.room_id,
sender,
incoming,
)
clear_staged_after_dispatch = False
if isinstance(incoming, IncomingMessage) and incoming.text:
incoming, clear_staged_after_dispatch = await self._merge_staged_attachments(
room.room_id,
sender,
incoming,
)
agent_id = (room_meta or {}).get("agent_id")
if _ws_debug_enabled() and not body.startswith("!"):
logger.warning(
"matrix_incoming_message_route",
room_id=room.room_id,
sender=sender,
local_chat_id=local_chat_id,
agent_id=agent_id,
platform_chat_id=(room_meta or {}).get("platform_chat_id"),
)
workspace_root = self._agent_workspace_root(agent_id)
try:
outgoing = await self.runtime.dispatcher.dispatch(incoming)
except PlatformError as exc:
logger.warning(
"matrix_message_platform_error",
room_id=room.room_id,
sender=getattr(event, "sender", None),
code=exc.code,
error=str(exc),
)
outgoing = [
OutgoingMessage(
chat_id=local_chat_id,
text="Сервис временно недоступен. Попробуйте ещё раз позже.",
)
]
else:
if clear_staged_after_dispatch:
await clear_staged_attachments(self.runtime.store, room.room_id, sender)
await self._send_all(room.room_id, outgoing, workspace_root=workspace_root)
def _is_file_only_event(
self, event: RoomMessage, incoming: IncomingMessage | IncomingCommand
) -> bool:
return (
isinstance(incoming, IncomingMessage)
and bool(incoming.attachments)
and not isinstance(event, RoomMessageText)
)
async def _stage_attachments(
self,
room_id: str,
user_id: str,
attachments: list,
) -> None:
for attachment in attachments:
await add_staged_attachment(
self.runtime.store,
room_id,
user_id,
{
"type": attachment.type,
"url": attachment.url,
"filename": attachment.filename,
"mime_type": attachment.mime_type,
"workspace_path": attachment.workspace_path,
},
)
async def _format_staged_attachments(
self,
room_id: str,
user_id: str,
*,
include_hint: bool = False,
) -> str:
attachments = await get_staged_attachments(self.runtime.store, room_id, user_id)
if not attachments:
return "Нет сохраненных вложений."
lines = ["Вложения в очереди:"]
for index, attachment in enumerate(attachments, start=1):
lines.append(f"{index}. {attachment.get('filename') or 'attachment'}")
if include_hint:
lines.extend(
[
"",
"Следующее сообщение отправит файлы агенту.",
"Команды: !list, !remove <n>, !remove all",
]
)
return "\n".join(lines)
async def _handle_staged_attachment_command(
self,
room_id: str,
user_id: str,
incoming: IncomingCommand,
) -> list[OutgoingEvent]:
if incoming.command == "matrix_list_attachments":
return [
OutgoingMessage(
chat_id=incoming.chat_id,
text=await self._format_staged_attachments(room_id, user_id),
)
]
arg = incoming.args[0] if incoming.args else ""
if arg == "all":
await clear_staged_attachments(self.runtime.store, room_id, user_id)
return [OutgoingMessage(chat_id=incoming.chat_id, text="Все вложения удалены.")]
try:
index = int(arg) - 1
except ValueError:
return [OutgoingMessage(chat_id=incoming.chat_id, text="Нет такого вложения.")]
removed = await remove_staged_attachment_at(self.runtime.store, room_id, user_id, index)
if removed is None:
return [OutgoingMessage(chat_id=incoming.chat_id, text="Нет такого вложения.")]
return [
OutgoingMessage(
chat_id=incoming.chat_id,
text=await self._format_staged_attachments(room_id, user_id),
)
]
async def _merge_staged_attachments(
self,
room_id: str,
user_id: str,
incoming: IncomingMessage,
) -> tuple[IncomingMessage, bool]:
staged = await get_staged_attachments(self.runtime.store, room_id, user_id)
if not staged:
return incoming, False
attachments = [
Attachment(
type=item.get("type", "document"),
url=item.get("url"),
filename=item.get("filename"),
mime_type=item.get("mime_type"),
workspace_path=item.get("workspace_path"),
)
for item in staged
]
return (
IncomingMessage(
user_id=incoming.user_id,
platform=incoming.platform,
chat_id=incoming.chat_id,
text=incoming.text,
attachments=attachments,
reply_to=incoming.reply_to,
),
True,
)
def _agent_workspace_root(self, agent_id: str | None) -> Path:
default = Path(os.environ.get("SURFACES_WORKSPACE_DIR", "/workspace"))
if agent_id is None or self.runtime.registry is None:
return default
try:
agent = self.runtime.registry.get(agent_id)
if agent.workspace_path:
return Path(agent.workspace_path)
except Exception:
pass
return default
async def _materialize_incoming_attachments(
self,
room_id: str,
matrix_user_id: str,
incoming: IncomingMessage,
) -> IncomingMessage:
room_meta = await get_room_meta(self.runtime.store, room_id)
agent_id = (room_meta or {}).get("agent_id")
workspace_root = self._agent_workspace_root(agent_id)
materialized = []
for attachment in incoming.attachments:
materialized.append(
await download_matrix_attachment(
client=self.client,
workspace_root=workspace_root,
matrix_user_id=matrix_user_id,
room_id=room_id,
attachment=attachment,
)
)
return IncomingMessage(
user_id=incoming.user_id,
platform=incoming.platform,
chat_id=incoming.chat_id,
text=incoming.text,
attachments=materialized,
reply_to=incoming.reply_to,
)
async def _bootstrap_unregistered_room(
self,
room: MatrixRoom,
sender: str,
) -> list[OutgoingEvent] | None:
if not hasattr(self.client, "room_create") or not hasattr(self.client, "room_put_state"):
return None
display_name = getattr(room, "display_name", None) or sender
try:
created = await provision_workspace_chat(
self.client,
sender,
display_name,
self.runtime.platform,
self.runtime.store,
self.runtime.auth_mgr,
self.runtime.chat_mgr,
registry=self.runtime.registry,
)
except Exception as exc:
logger.warning(
"matrix_unregistered_room_bootstrap_failed",
room_id=room.room_id,
sender=sender,
error=str(exc),
)
return [
OutgoingMessage(
chat_id=room.room_id,
text="Не удалось подготовить рабочий чат. Попробуйте ещё раз позже.",
)
]
welcome = (
f"Привет, {created['user'].display_name or sender}! Пиши — я здесь.\n\n"
"Команды: !new · !chats · !rename · !archive · !context · !save · !load · !help"
)
if created.get("agent_assignment") == "default":
welcome = f"{welcome}\n\n{default_agent_notice()}"
await set_room_meta(
self.runtime.store,
room.room_id,
{
"matrix_user_id": sender,
"redirect_room_id": created["chat_room_id"],
"redirect_chat_id": created["chat_id"],
},
)
await self.client.room_send(
created["chat_room_id"],
"m.room.message",
{"msgtype": "m.text", "body": welcome},
)
return [
OutgoingMessage(
chat_id=room.room_id,
text=(
f"Создал рабочий чат {created['room_name']} ({created['chat_id']}) "
"и добавил его в пространство Lambda. "
"Открой приглашённую комнату для продолжения."
),
)
]
async def _handle_load_selection(
self,
user_id: str,
room_id: str,
text: str,
pending: dict,
) -> list[OutgoingEvent]:
saves = pending.get("saves", [])
if text in {"0", "!cancel"}:
await clear_load_pending(self.runtime.store, user_id, room_id)
return [OutgoingMessage(chat_id=room_id, text="Отменено.")]
index = int(text) - 1
if index < 0 or index >= len(saves):
return [
OutgoingMessage(
chat_id=room_id,
text=f"Неверный номер. Введи от 1 до {len(saves)} или 0 для отмены.",
)
]
name = saves[index]["name"]
await clear_load_pending(self.runtime.store, user_id, room_id)
prototype_state = getattr(self.runtime.platform, "_prototype_state", None)
if prototype_state is not None:
room_meta = await get_room_meta(self.runtime.store, room_id)
context_keys = []
if room_meta is not None:
platform_chat_id = room_meta.get("platform_chat_id")
if platform_chat_id:
context_keys.append(platform_chat_id)
chat_id = room_meta.get("chat_id")
if chat_id:
context_keys.append(chat_id)
if not context_keys:
context_keys.append(room_id)
for context_key in dict.fromkeys(context_keys):
await prototype_state.set_current_session(context_key, name)
try:
await self.runtime.platform.send_message(
user_id,
room_id,
LOAD_PROMPT.format(name=name),
)
except Exception as exc:
logger.warning("load_agent_call_failed", error=str(exc))
return [OutgoingMessage(chat_id=room_id, text=f"Ошибка при загрузке: {exc}")]
return [
OutgoingMessage(chat_id=room_id, text=f"Запрос на загрузку отправлен агенту: {name}")
]
async def on_member(self, room: MatrixRoom, event: RoomMemberEvent) -> None:
if getattr(event, "sender", None) == self.client.user_id:
@ -117,11 +790,23 @@ class MatrixBot:
self.runtime.store,
self.runtime.auth_mgr,
self.runtime.chat_mgr,
self.runtime.registry,
)
async def _send_all(self, room_id: str, outgoing: list[OutgoingEvent]) -> None:
async def _send_all(
self,
room_id: str,
outgoing: list[OutgoingEvent],
workspace_root: Path | None = None,
) -> None:
for event in outgoing:
await send_outgoing(self.client, room_id, event, store=self.runtime.store)
await send_outgoing(
self.client,
room_id,
event,
store=self.runtime.store,
workspace_root=workspace_root,
)
async def prepare_live_sync(client: AsyncClient) -> str | None:
@ -130,11 +815,13 @@ async def prepare_live_sync(client: AsyncClient) -> str | None:
return response.next_batch
return None
async def send_outgoing(
client: AsyncClient,
room_id: str,
event: OutgoingEvent,
store: StateStore | None = None,
workspace_root: Path | None = None,
) -> None:
if isinstance(event, OutgoingTyping):
await client.room_typing(room_id, event.is_typing, timeout=25000)
@ -144,7 +831,39 @@ async def send_outgoing(
await client.room_send(room_id, "m.room.message", {"msgtype": "m.text", "body": body})
return
if isinstance(event, OutgoingMessage):
await client.room_send(room_id, "m.room.message", {"msgtype": "m.text", "body": event.text})
if event.text:
await client.room_send(
room_id, "m.room.message", {"msgtype": "m.text", "body": event.text}
)
if event.attachments:
workspace_root = workspace_root or Path(
os.environ.get("SURFACES_WORKSPACE_DIR", "/workspace")
)
for attachment in event.attachments:
if not attachment.workspace_path:
continue
file_path = resolve_workspace_attachment_path(
workspace_root, attachment.workspace_path
)
with file_path.open("rb") as handle:
upload_response, _ = await client.upload(
handle,
content_type=attachment.mime_type or "application/octet-stream",
filename=attachment.filename or file_path.name,
filesize=file_path.stat().st_size,
)
content_uri = getattr(upload_response, "content_uri", None)
if not content_uri:
raise RuntimeError(f"Matrix upload failed for {file_path}")
await client.room_send(
room_id,
"m.room.message",
{
"msgtype": matrix_msgtype_for_attachment(attachment),
"body": attachment.filename or file_path.name,
"url": content_uri,
},
)
return
if isinstance(event, OutgoingUI):
lines = [event.text]
@ -176,6 +895,7 @@ async def send_outgoing(
async def main() -> None:
_configure_debug_logging()
homeserver = os.environ.get("MATRIX_HOMESERVER")
user_id = os.environ.get("MATRIX_USER_ID")
device_id = os.environ.get("MATRIX_DEVICE_ID", "")
@ -207,9 +927,19 @@ async def main() -> None:
await client.login(password=password, device_name="surfaces-bot")
since_token = await prepare_live_sync(client)
await reconcile_startup_state(client, runtime)
bot = MatrixBot(client, runtime)
client.add_event_callback(bot.on_room_message, RoomMessageText)
client.add_event_callback(
bot.on_room_message,
(
RoomMessageText,
RoomMessageFile,
RoomMessageImage,
RoomMessageVideo,
RoomMessageAudio,
),
)
client.add_event_callback(bot.on_member, (InviteMemberEvent, RoomMemberEvent))
logger.info(
@ -219,9 +949,21 @@ async def main() -> None:
store_path=store_path,
request_timeout=client_config.request_timeout,
)
if _ws_debug_enabled():
logger.warning(
"matrix_ws_debug_enabled",
homeserver=homeserver,
user_id=user_id,
backend=os.environ.get("MATRIX_PLATFORM_BACKEND", "mock").strip().lower(),
global_agent_base_url=_agent_base_url_from_env(),
registry_path=os.environ.get("MATRIX_AGENT_REGISTRY_PATH", "").strip(),
)
try:
await client.sync_forever(timeout=30000, since=since_token)
finally:
close = getattr(runtime.platform, "close", None)
if callable(close):
await close()
await client.close()

View file

@ -14,42 +14,53 @@ PLATFORM = "matrix"
def extract_attachments(event: Any) -> list[Attachment]:
source = getattr(event, "source", {}) or {}
content = source.get("content", {}) or getattr(event, "content", {}) or {}
msgtype = getattr(event, "msgtype", None)
if msgtype is None:
content = getattr(event, "content", {}) or {}
msgtype = content.get("msgtype")
url = content.get("url") or getattr(event, "url", None)
filename = content.get("body") or getattr(event, "body", None)
mime_type = content.get("mimetype") or getattr(event, "mimetype", None)
if mime_type is None:
info = content.get("info") or {}
if isinstance(info, dict):
mime_type = info.get("mimetype")
if msgtype == "m.image":
return [
Attachment(
type="image",
url=getattr(event, "url", None),
mime_type=getattr(event, "mimetype", None),
url=url,
filename=filename,
mime_type=mime_type,
)
]
if msgtype == "m.file":
return [
Attachment(
type="document",
url=getattr(event, "url", None),
filename=getattr(event, "body", None),
mime_type=getattr(event, "mimetype", None),
url=url,
filename=filename,
mime_type=mime_type,
)
]
if msgtype == "m.audio":
return [
Attachment(
type="audio",
url=getattr(event, "url", None),
mime_type=getattr(event, "mimetype", None),
url=url,
filename=filename,
mime_type=mime_type,
)
]
if msgtype == "m.video":
return [
Attachment(
type="video",
url=getattr(event, "url", None),
mime_type=getattr(event, "mimetype", None),
url=url,
filename=filename,
mime_type=mime_type,
)
]
return []
@ -75,6 +86,24 @@ def from_command(body: str, sender: str, chat_id: str, room_id: str | None = Non
},
)
if command == "list" and not args:
return IncomingCommand(
user_id=sender,
platform=PLATFORM,
chat_id=chat_id,
command="matrix_list_attachments",
args=[],
)
if command == "remove" and len(args) == 1:
return IncomingCommand(
user_id=sender,
platform=PLATFORM,
chat_id=chat_id,
command="matrix_remove_attachment",
args=[args[0]],
)
aliases = {
"skills": "settings_skills",
"connectors": "settings_connectors",

114
adapter/matrix/files.py Normal file
View file

@ -0,0 +1,114 @@
from __future__ import annotations
import mimetypes
import re
from pathlib import Path, PurePosixPath
from core.protocol import Attachment
def _sanitize_filename(value: str) -> str:
filename = PurePosixPath(str(value).replace("\\", "/")).name.strip()
cleaned = re.sub(r"[\x00-\x1f\x7f<>:\"/\\|?*]+", "_", filename)
cleaned = cleaned.strip(" .")
return cleaned or "attachment.bin"
def _default_filename(attachment: Attachment) -> str:
if attachment.filename:
return attachment.filename
extension = mimetypes.guess_extension(attachment.mime_type or "") or ""
base = {
"image": "image",
"audio": "audio",
"video": "video",
"document": "attachment",
}.get(attachment.type, "attachment")
return f"{base}{extension}"
def _with_copy_index(filename: str, index: int) -> str:
path = Path(filename)
suffix = path.suffix
stem = path.stem if suffix else filename
return f"{stem} ({index}){suffix}"
def _unique_workspace_relative_path(workspace_root: Path, filename: str) -> tuple[str, Path]:
safe_name = _sanitize_filename(filename)
candidate = workspace_root / safe_name
if not candidate.exists():
return safe_name, candidate
index = 1
while True:
indexed_name = _with_copy_index(safe_name, index)
candidate = workspace_root / indexed_name
if not candidate.exists():
return indexed_name, candidate
index += 1
def build_agent_workspace_path(
*,
workspace_root: Path,
filename: str,
) -> tuple[str, Path]:
"""Saves user files directly to {workspace_root}/{filename}.
The returned relative path is what gets passed to agent.send_message(attachments=[...]).
"""
return _unique_workspace_relative_path(workspace_root, filename)
async def download_matrix_attachment(
*,
client,
workspace_root: Path,
matrix_user_id: str,
room_id: str,
attachment: Attachment,
timestamp: str | None = None,
) -> Attachment:
if not attachment.url:
return attachment
filename = _default_filename(attachment)
del matrix_user_id, room_id, timestamp
relative_path, absolute_path = build_agent_workspace_path(
workspace_root=workspace_root,
filename=filename,
)
absolute_path.parent.mkdir(parents=True, exist_ok=True)
response = await client.download(attachment.url)
body = getattr(response, "body", None)
if body is None:
raise RuntimeError(f"Matrix download response for {attachment.url} has no body")
absolute_path.write_bytes(body)
return Attachment(
type=attachment.type,
url=attachment.url,
filename=filename,
mime_type=attachment.mime_type,
workspace_path=relative_path,
)
def resolve_workspace_attachment_path(workspace_root: Path, workspace_path: str) -> Path:
path = Path(workspace_path)
if path.is_absolute():
return path
return workspace_root / path
def matrix_msgtype_for_attachment(attachment: Attachment) -> str:
return {
"image": "m.image",
"audio": "m.audio",
"video": "m.video",
}.get(attachment.type, "m.file")

View file

@ -7,6 +7,12 @@ from adapter.matrix.handlers.chat import (
make_handle_rename,
)
from adapter.matrix.handlers.confirm import make_handle_cancel, make_handle_confirm
from adapter.matrix.handlers.context_commands import (
make_handle_context,
make_handle_load,
make_handle_reset,
make_handle_save,
)
from adapter.matrix.handlers.settings import (
handle_help,
handle_settings,
@ -18,18 +24,32 @@ from adapter.matrix.handlers.settings import (
handle_settings_status,
handle_settings_whoami,
handle_toggle_skill,
handle_unknown_command,
)
from core.handler import EventDispatcher
from core.protocol import IncomingCallback, IncomingCommand
def register_matrix_handlers(dispatcher: EventDispatcher, client=None, store=None) -> None:
dispatcher.register(IncomingCommand, "new", make_handle_new_chat(client, store))
def register_matrix_handlers(
dispatcher: EventDispatcher,
client=None,
store=None,
registry=None,
prototype_state=None,
agent_base_url: str = "http://127.0.0.1:8000",
) -> None:
dispatcher.register(IncomingCommand, "new", make_handle_new_chat(client, store, registry))
dispatcher.register(IncomingCommand, "chats", handle_list_chats)
dispatcher.register(IncomingCommand, "rename", make_handle_rename(client, store))
dispatcher.register(IncomingCommand, "archive", make_handle_archive(client, store))
dispatcher.register(IncomingCommand, "help", handle_help)
dispatcher.register(IncomingCommand, "settings", handle_settings)
if prototype_state is not None:
clear_handler = make_handle_reset(store, prototype_state)
dispatcher.register(IncomingCommand, "clear", clear_handler)
dispatcher.register(IncomingCommand, "reset", clear_handler)
else:
dispatcher.register(IncomingCommand, "reset", handle_settings)
dispatcher.register(IncomingCommand, "settings_skills", handle_settings_skills)
dispatcher.register(IncomingCommand, "settings_connectors", handle_settings_connectors)
dispatcher.register(IncomingCommand, "settings_soul", handle_settings_soul)
@ -41,3 +61,13 @@ def register_matrix_handlers(dispatcher: EventDispatcher, client=None, store=Non
dispatcher.register(IncomingCallback, "confirm", make_handle_confirm(store))
dispatcher.register(IncomingCallback, "cancel", make_handle_cancel(store))
dispatcher.register(IncomingCallback, "toggle_skill", handle_toggle_skill)
dispatcher.register(IncomingCommand, "*", handle_unknown_command)
if prototype_state is not None:
dispatcher.register(
IncomingCommand,
"save",
make_handle_save(None, store, prototype_state),
)
dispatcher.register(IncomingCommand, "load", make_handle_load(store, prototype_state))
dispatcher.register(IncomingCommand, "context", make_handle_context(store, prototype_state))

View file

@ -1,14 +1,15 @@
from __future__ import annotations
import structlog
from typing import Any
import structlog
from nio.api import RoomVisibility
from nio.responses import RoomCreateError
from adapter.matrix.agent_registry import AgentRegistry
from adapter.matrix.store import (
get_user_meta,
next_chat_id,
next_platform_chat_id,
set_room_meta,
set_user_meta,
)
@ -16,16 +17,47 @@ from adapter.matrix.store import (
logger = structlog.get_logger(__name__)
async def handle_invite(client: Any, room: Any, event: Any, platform, store, auth_mgr, chat_mgr) -> None:
matrix_user_id = getattr(event, "sender", "")
display_name = getattr(room, "display_name", None) or matrix_user_id
def _default_room_name(chat_id: str) -> str:
suffix = chat_id[1:] if chat_id.startswith("C") else chat_id
return f"Чат {suffix}"
existing = await get_user_meta(store, matrix_user_id)
if existing and existing.get("space_id"):
return
await client.join(room.room_id)
def default_agent_notice() -> str:
return (
"Внимание: ваш Matrix ID не найден в конфиге агентов. "
"Пока используется агент по умолчанию. После добавления вас в конфиг "
"бот переключит существующие комнаты на назначенного агента."
)
async def _invite_if_possible(client: Any, room_id: str, matrix_user_id: str) -> bool:
room_invite = getattr(client, "room_invite", None)
if not callable(room_invite):
return False
try:
await room_invite(room_id, matrix_user_id)
return True
except Exception as exc:
logger.warning(
"matrix_workspace_reinvite_failed",
room_id=room_id,
user=matrix_user_id,
error=str(exc),
)
return False
async def provision_workspace_chat(
client: Any,
matrix_user_id: str,
display_name: str,
platform,
store,
auth_mgr,
chat_mgr,
room_name_override: str | None = None,
registry: AgentRegistry | None = None,
) -> dict:
user = await platform.get_or_create_user(
external_id=matrix_user_id,
platform="matrix",
@ -34,24 +66,41 @@ async def handle_invite(client: Any, room: Any, event: Any, platform, store, aut
await auth_mgr.confirm(matrix_user_id)
homeserver = matrix_user_id.split(":")[-1]
user_meta = await get_user_meta(store, matrix_user_id) or {}
space_id = user_meta.get("space_id")
space_resp = await client.room_create(
name=f"Lambda — {display_name}",
space=True,
visibility=RoomVisibility.private,
invite=[matrix_user_id],
)
if isinstance(space_resp, RoomCreateError):
logger.error(
"space creation failed",
user=matrix_user_id,
error=getattr(space_resp, "status_code", None),
if not space_id:
space_resp = await client.room_create(
name=f"Lambda — {display_name}",
space=True,
visibility=RoomVisibility.private,
invite=[matrix_user_id],
)
return
space_id = space_resp.room_id
if isinstance(space_resp, RoomCreateError):
logger.error(
"space creation failed",
user=matrix_user_id,
error=getattr(space_resp, "status_code", None),
)
raise RuntimeError("Не удалось создать Space.")
space_id = space_resp.room_id
user_meta["space_id"] = space_id
await set_user_meta(store, matrix_user_id, user_meta)
next_chat_index = int(user_meta.get("next_chat_index", 1))
chat_id = f"C{next_chat_index}"
platform_chat_id = await next_platform_chat_id(store)
room_name = room_name_override or _default_room_name(chat_id)
agent_id = None
agent_assignment = "none"
if registry is not None:
assignment = registry.resolve_agent_for_user(matrix_user_id)
agent_id = assignment.agent_id
agent_assignment = assignment.source
chat_resp = await client.room_create(
name="Чат 1",
name=room_name,
visibility=RoomVisibility.private,
is_direct=False,
invite=[matrix_user_id],
@ -62,7 +111,7 @@ async def handle_invite(client: Any, room: Any, event: Any, platform, store, aut
user=matrix_user_id,
error=getattr(chat_resp, "status_code", None),
)
return
raise RuntimeError("Не удалось создать рабочий чат.")
chat_room_id = chat_resp.room_id
await client.room_put_state(
@ -72,10 +121,8 @@ async def handle_invite(client: Any, room: Any, event: Any, platform, store, aut
state_key=chat_room_id,
)
chat_id = await next_chat_id(store, matrix_user_id)
user_meta = await get_user_meta(store, matrix_user_id) or {}
user_meta["space_id"] = space_id
user_meta["next_chat_index"] = next_chat_index + 1
await set_user_meta(store, matrix_user_id, user_meta)
await set_room_meta(
@ -84,9 +131,12 @@ async def handle_invite(client: Any, room: Any, event: Any, platform, store, aut
{
"room_type": "chat",
"chat_id": chat_id,
"display_name": "Чат 1",
"display_name": room_name,
"matrix_user_id": matrix_user_id,
"space_id": space_id,
"platform_chat_id": platform_chat_id,
"agent_id": agent_id,
"agent_assignment": agent_assignment,
},
)
await chat_mgr.get_or_create(
@ -94,15 +144,142 @@ async def handle_invite(client: Any, room: Any, event: Any, platform, store, aut
chat_id=chat_id,
platform="matrix",
surface_ref=chat_room_id,
name="Чат 1",
name=room_name,
)
return {
"user": user,
"space_id": space_id,
"chat_room_id": chat_room_id,
"chat_id": chat_id,
"room_name": room_name,
"agent_assignment": agent_assignment,
"agent_id": agent_id,
}
async def restore_workspace_access(
client: Any,
matrix_user_id: str,
display_name: str,
platform,
store,
auth_mgr,
chat_mgr,
registry: AgentRegistry | None = None,
) -> dict:
user_meta = await get_user_meta(store, matrix_user_id) or {}
space_id = user_meta.get("space_id")
if not space_id:
created = await provision_workspace_chat(
client,
matrix_user_id,
display_name,
platform,
store,
auth_mgr,
chat_mgr,
room_name_override="Чат 1",
registry=registry,
)
return {**created, "reinvited_rooms": [], "created_new_chat": True}
await auth_mgr.confirm(matrix_user_id)
await _invite_if_possible(client, space_id, matrix_user_id)
chats = await chat_mgr.list_active(matrix_user_id)
if not chats:
created = await provision_workspace_chat(
client,
matrix_user_id,
display_name,
platform,
store,
auth_mgr,
chat_mgr,
registry=registry,
)
return {**created, "reinvited_rooms": [], "created_new_chat": True}
reinvited_rooms = []
for chat in chats:
if chat.surface_ref:
if await _invite_if_possible(client, chat.surface_ref, matrix_user_id):
reinvited_rooms.append(chat.surface_ref)
return {
"space_id": space_id,
"reinvited_rooms": reinvited_rooms,
"created_new_chat": False,
}
async def handle_invite(
client: Any,
room: Any,
event: Any,
platform,
store,
auth_mgr,
chat_mgr,
registry: AgentRegistry | None = None,
) -> None:
matrix_user_id = getattr(event, "sender", "")
display_name = getattr(room, "display_name", None) or matrix_user_id
await client.join(room.room_id)
existing = await get_user_meta(store, matrix_user_id)
if existing and existing.get("space_id"):
restored = await restore_workspace_access(
client,
matrix_user_id,
display_name,
platform,
store,
auth_mgr,
chat_mgr,
registry=registry,
)
body = "Я отправил повторные приглашения в пространство Lambda и рабочие чаты."
if restored.get("created_new_chat"):
body = (
f"Создал новый рабочий чат {restored['room_name']} "
f"({restored['chat_id']}) и отправил приглашение."
)
if restored.get("agent_assignment") == "default":
body = f"{body}\n\n{default_agent_notice()}"
await client.room_send(
room.room_id,
"m.room.message",
{"msgtype": "m.text", "body": body},
)
return
try:
created = await provision_workspace_chat(
client,
matrix_user_id,
display_name,
platform,
store,
auth_mgr,
chat_mgr,
room_name_override="Чат 1",
registry=registry,
)
except RuntimeError as exc:
logger.error("invite_workspace_provision_failed", user=matrix_user_id, error=str(exc))
return
welcome = (
f"Привет, {user.display_name or matrix_user_id}! Пиши — я здесь.\n\n"
"Команды: !new · !chats · !rename · !archive · !skills · !soul · !safety · !settings"
f"Привет, {created['user'].display_name or matrix_user_id}! Пиши — я здесь.\n\n"
"Команды: !new · !chats · !rename · !archive · !clear · !help"
)
if created.get("agent_assignment") == "default":
welcome = f"{welcome}\n\n{default_agent_notice()}"
await client.room_send(
chat_room_id,
created["chat_room_id"],
"m.room.message",
{"msgtype": "m.text", "body": welcome},
)

View file

@ -1,12 +1,20 @@
from __future__ import annotations
from typing import Any, Awaitable, Callable
from collections.abc import Awaitable, Callable
from typing import Any
import structlog
from nio.api import RoomVisibility
from nio.responses import RoomCreateError
from adapter.matrix.store import get_user_meta, next_chat_id, set_room_meta
from adapter.matrix.agent_registry import AgentRegistry
from adapter.matrix.handlers.auth import default_agent_notice
from adapter.matrix.store import (
get_user_meta,
next_chat_id,
next_platform_chat_id,
set_room_meta,
)
from core.protocol import IncomingCommand, OutgoingMessage
logger = structlog.get_logger(__name__)
@ -42,6 +50,7 @@ async def _fallback_new_chat(
def make_handle_new_chat(
client: Any | None,
store: Any | None,
registry: AgentRegistry | None = None,
) -> Callable[..., Awaitable[list]]:
async def handle_new_chat(
event: IncomingCommand, auth_mgr, platform, chat_mgr, settings_mgr
@ -69,6 +78,7 @@ def make_handle_new_chat(
name = " ".join(event.args).strip() if event.args else ""
chat_id = await next_chat_id(store, event.user_id)
platform_chat_id = await next_platform_chat_id(store)
room_name = name or f"Чат {chat_id}"
response = await client.room_create(
@ -97,17 +107,24 @@ def make_handle_new_chat(
state_key=room_id,
)
await set_room_meta(
store,
room_id,
{
"room_type": "chat",
"chat_id": chat_id,
"display_name": room_name,
"matrix_user_id": event.user_id,
"space_id": space_id,
},
)
agent_id = None
agent_assignment = "none"
if registry is not None:
assignment = registry.resolve_agent_for_user(event.user_id)
agent_id = assignment.agent_id
agent_assignment = assignment.source
room_meta: dict = {
"room_type": "chat",
"chat_id": chat_id,
"display_name": room_name,
"matrix_user_id": event.user_id,
"space_id": space_id,
"platform_chat_id": platform_chat_id,
"agent_id": agent_id,
"agent_assignment": agent_assignment,
}
await set_room_meta(store, room_id, room_meta)
ctx = await chat_mgr.get_or_create(
user_id=event.user_id,
chat_id=chat_id,
@ -115,10 +132,13 @@ def make_handle_new_chat(
surface_ref=room_id,
name=room_name,
)
text = f"Создан чат: {ctx.display_name} ({ctx.chat_id})"
if agent_assignment == "default":
text = f"{text}\n\n{default_agent_notice()}"
return [
OutgoingMessage(
chat_id=event.chat_id,
text=f"Создан чат: {ctx.display_name} ({ctx.chat_id})",
text=text,
)
]
@ -150,7 +170,10 @@ def make_handle_rename(
return [
OutgoingMessage(
chat_id=event.chat_id,
text="Этот чат не найден в локальном состоянии бота. Открой зарегистрированную комнату или создай новый чат через !new.",
text=(
"Этот чат не найден в локальном состоянии бота. "
"Открой зарегистрированную комнату или создай новый чат через !new."
),
)
]
@ -180,7 +203,10 @@ def make_handle_archive(
return [
OutgoingMessage(
chat_id=event.chat_id,
text="Этот чат не найден в локальном состоянии бота. Создай новый чат через !new.",
text=(
"Этот чат не найден в локальном состоянии бота. "
"Создай новый чат через !new."
),
)
]
ctx = await chat_mgr.get(event.chat_id, user_id=event.user_id)

View file

@ -0,0 +1,230 @@
from __future__ import annotations
import re
from datetime import UTC, datetime
from typing import TYPE_CHECKING
import httpx
import structlog
from adapter.matrix.store import (
get_room_meta,
next_platform_chat_id,
set_load_pending,
set_platform_chat_id,
)
from core.protocol import IncomingCommand, OutgoingEvent, OutgoingMessage
if TYPE_CHECKING:
from core.store import StateStore
from sdk.prototype_state import PrototypeStateStore
logger = structlog.get_logger(__name__)
SAVE_PROMPT = (
"Summarize our conversation and save to /workspace/contexts/{name}.md. "
"Reply only with: Saved: {name}"
)
LOAD_PROMPT = (
"Load context from /workspace/contexts/{name}.md and use it as background "
"for our conversation. Reply: Loaded: {name}"
)
_VALID_NAME = re.compile(r"^[A-Za-z0-9_-]+$")
def _sanitize_session_name(raw_name: str) -> str | None:
name = raw_name.strip()
if not name or not _VALID_NAME.fullmatch(name):
return None
return name
async def _resolve_room_id(event: IncomingCommand, chat_mgr) -> str:
if chat_mgr is None:
return event.chat_id
ctx = await chat_mgr.get(event.chat_id, user_id=event.user_id)
if ctx is not None and ctx.surface_ref:
return ctx.surface_ref
return event.chat_id
async def _resolve_context_scope(
event: IncomingCommand,
store: StateStore,
chat_mgr,
) -> tuple[str, str | None]:
room_id = await _resolve_room_id(event, chat_mgr)
room_meta = await get_room_meta(store, room_id)
platform_chat_id = room_meta.get("platform_chat_id") if room_meta else None
return room_id, platform_chat_id
async def _require_platform_context(
event: IncomingCommand,
store: StateStore,
chat_mgr,
) -> tuple[str, str]:
room_id, platform_chat_id = await _resolve_context_scope(event, store, chat_mgr)
if not platform_chat_id:
raise RuntimeError(f"matrix room context is incomplete: {room_id}")
return room_id, platform_chat_id
def make_handle_save(agent_api, store: StateStore, prototype_state: PrototypeStateStore):
async def handle_save(
event: IncomingCommand, auth_mgr, platform, chat_mgr, settings_mgr
) -> list[OutgoingEvent]:
if event.args:
name = _sanitize_session_name(event.args[0])
if name is None:
return [
OutgoingMessage(
chat_id=event.chat_id,
text="Имя сохранения может содержать только буквы, цифры, _ и -.",
)
]
else:
name = f"context-{datetime.now(UTC).strftime('%Y%m%d-%H%M%S')}"
try:
await platform.send_message(
event.user_id,
event.chat_id,
SAVE_PROMPT.format(name=name),
)
except Exception as exc:
logger.warning("save_agent_call_failed", error=str(exc))
return [OutgoingMessage(chat_id=event.chat_id, text=f"Ошибка при сохранении: {exc}")]
try:
_, platform_chat_id = await _require_platform_context(event, store, chat_mgr)
except RuntimeError as exc:
logger.warning("save_context_incomplete", error=str(exc))
return [OutgoingMessage(chat_id=event.chat_id, text="Контекст комнаты не готов. Попробуй позже.")]
await prototype_state.add_saved_session(
event.user_id,
name,
source_context_id=platform_chat_id,
)
return [
OutgoingMessage(
chat_id=event.chat_id,
text=f"Запрос на сохранение отправлен агенту: {name}",
)
]
return handle_save
def make_handle_load(store: StateStore, prototype_state: PrototypeStateStore):
async def handle_load(
event: IncomingCommand, auth_mgr, platform, chat_mgr, settings_mgr
) -> list[OutgoingEvent]:
sessions = await prototype_state.list_saved_sessions(event.user_id)
if not sessions:
return [
OutgoingMessage(
chat_id=event.chat_id,
text="Нет сохранённых сессий. Используй !save [имя].",
)
]
room_id, _ = await _resolve_context_scope(event, store, chat_mgr)
lines = ["Сохранённые сессии:"]
for index, session in enumerate(sessions, start=1):
created = session.get("created_at", "")[:10]
lines.append(f" {index}. {session['name']} ({created})")
lines.append("")
lines.append("Введи номер или 0 / !cancel для отмены.")
await set_load_pending(store, event.user_id, room_id, {"saves": sessions})
return [OutgoingMessage(chat_id=event.chat_id, text="\n".join(lines))]
return handle_load
def make_handle_reset(store: StateStore, prototype_state: PrototypeStateStore):
async def handle_reset(
event: IncomingCommand, auth_mgr, platform, chat_mgr, settings_mgr
) -> list[OutgoingEvent]:
try:
room_id, old_chat_id = await _require_platform_context(event, store, chat_mgr)
except RuntimeError as exc:
logger.warning("clear_context_incomplete", error=str(exc))
return [OutgoingMessage(chat_id=event.chat_id, text="Контекст комнаты не готов. Попробуй позже.")]
new_chat_id = await next_platform_chat_id(store)
await set_platform_chat_id(store, room_id, new_chat_id)
disconnect = getattr(platform, "disconnect_chat", None)
if callable(disconnect):
await disconnect(old_chat_id)
await prototype_state.clear_current_session(old_chat_id)
await prototype_state.clear_current_session(new_chat_id)
return [
OutgoingMessage(
chat_id=event.chat_id,
text="Контекст сброшен. Агент не помнит предыдущий разговор.",
)
]
return handle_reset
async def _call_reset_endpoint(agent_base_url: str, chat_id: str) -> list[OutgoingEvent]:
try:
async with httpx.AsyncClient() as client:
response = await client.post(f"{agent_base_url}/reset", timeout=5.0)
except (httpx.ConnectError, httpx.TimeoutException) as exc:
logger.warning("reset_endpoint_unreachable", error=str(exc))
return [
OutgoingMessage(
chat_id=chat_id,
text="Reset endpoint недоступен. Обратитесь к администратору.",
)
]
if response.status_code == 404:
return [
OutgoingMessage(
chat_id=chat_id,
text="Reset endpoint недоступен. Обратитесь к администратору.",
)
]
return [OutgoingMessage(chat_id=chat_id, text="Контекст сброшен.")]
def make_handle_context(store: StateStore, prototype_state: PrototypeStateStore):
async def handle_context(
event: IncomingCommand, auth_mgr, platform, chat_mgr, settings_mgr
) -> list[OutgoingEvent]:
try:
_, platform_chat_id = await _require_platform_context(event, store, chat_mgr)
except RuntimeError as exc:
logger.warning("context_scope_incomplete", error=str(exc))
return [OutgoingMessage(chat_id=event.chat_id, text="Контекст комнаты не готов. Попробуй позже.")]
current_session = await prototype_state.get_current_session(platform_chat_id)
tokens_used = await prototype_state.get_last_tokens_used(platform_chat_id)
sessions = await prototype_state.list_saved_sessions(event.user_id)
lines = [
"Контекст:",
f" Контекст чата: {platform_chat_id}",
f" Сессия: {current_session or 'не загружена'}",
f" Токены (последний ответ): {tokens_used}",
f" Сохранения ({len(sessions)}):",
]
if sessions:
for session in sessions:
created = session.get("created_at", "")[:10]
lines.append(f" - {session['name']} ({created})")
else:
lines.append(" (нет)")
return [OutgoingMessage(chat_id=event.chat_id, text="\n".join(lines))]
return handle_context

View file

@ -1,8 +1,6 @@
from __future__ import annotations
from adapter.matrix.reactions import build_skills_text
from core.protocol import IncomingCommand, OutgoingMessage, SettingsAction
from core.protocol import IncomingCommand, OutgoingMessage
HELP_TEXT = "\n".join(
[
@ -12,186 +10,87 @@ HELP_TEXT = "\n".join(
"!chats список активных чатов",
"!rename <название> переименовать текущий чат",
"!archive архивировать текущий чат",
"!settings общий обзор настроек",
"!skills список навыков",
"!soul [поле значение] показать или изменить личность",
"!safety [триггер on/off] показать или изменить безопасность",
"!status краткий статус",
"!whoami показать ваш id",
"",
"!clear сбросить контекст текущего чата",
"",
"!list показать файлы в очереди",
"!remove <n> удалить файл из очереди",
"!remove all очистить очередь файлов",
"",
"!yes / !no подтвердить или отменить действие",
"!help эта справка",
]
)
def _render_mapping(title: str, data: dict | None) -> str:
data = data or {}
lines = [title]
if not data:
lines.append("Нет данных.")
else:
for key, value in data.items():
lines.append(f"{key}: {value}")
return "\n".join(lines)
def _parse_bool(value: str) -> bool:
return value.lower() in {"1", "true", "yes", "on", "enable", "enabled"}
MVP_UNAVAILABLE_TEXT = (
"Эта команда скрыта в MVP и сейчас недоступна. "
"Используй !help для списка поддерживаемых команд."
)
async def handle_settings(
event: IncomingCommand, auth_mgr, platform, chat_mgr, settings_mgr
) -> list:
settings = await settings_mgr.get(event.user_id)
chats = await chat_mgr.list_active(event.user_id)
skills_lines = []
for name, enabled in settings.skills.items():
state = "on" if enabled else "off"
skills_lines.append(f" {state} {name}")
skills_text = "\n".join(skills_lines) if skills_lines else " нет навыков"
soul_lines = []
for key, value in (settings.soul or {}).items():
soul_lines.append(f" {key}: {value}")
soul_text = "\n".join(soul_lines) if soul_lines else " по умолчанию"
safety_lines = []
for key, value in (settings.safety or {}).items():
state = "on" if value else "off"
safety_lines.append(f" {state} {key}")
safety_text = "\n".join(safety_lines) if safety_lines else " по умолчанию"
chat_lines = [f" {chat.display_name} ({chat.chat_id})" for chat in chats]
chats_text = "\n".join(chat_lines) if chat_lines else " нет активных чатов"
dashboard = "\n".join(
[
"Настройки",
"",
"Скиллы:",
skills_text,
"",
"Личность:",
soul_text,
"",
"Безопасность:",
safety_text,
"",
f"Активные чаты ({len(chats)}):",
chats_text,
]
)
return [OutgoingMessage(chat_id=event.chat_id, text=dashboard)]
return [OutgoingMessage(chat_id=event.chat_id, text=MVP_UNAVAILABLE_TEXT)]
async def handle_help(
event: IncomingCommand, auth_mgr, platform, chat_mgr, settings_mgr
) -> list:
async def handle_help(event: IncomingCommand, auth_mgr, platform, chat_mgr, settings_mgr) -> list:
return [OutgoingMessage(chat_id=event.chat_id, text=HELP_TEXT)]
async def handle_settings_skills(
event: IncomingCommand, auth_mgr, platform, chat_mgr, settings_mgr
) -> list:
settings = await settings_mgr.get(event.user_id)
return [OutgoingMessage(chat_id=event.chat_id, text=build_skills_text(settings))]
return [OutgoingMessage(chat_id=event.chat_id, text=MVP_UNAVAILABLE_TEXT)]
async def handle_settings_connectors(
event: IncomingCommand, auth_mgr, platform, chat_mgr, settings_mgr
) -> list:
settings = await settings_mgr.get(event.user_id)
return [
OutgoingMessage(
chat_id=event.chat_id, text=_render_mapping("🔗 Коннекторы", settings.connectors)
)
]
return [OutgoingMessage(chat_id=event.chat_id, text=MVP_UNAVAILABLE_TEXT)]
async def handle_settings_soul(
event: IncomingCommand, auth_mgr, platform, chat_mgr, settings_mgr
) -> list:
if len(event.args) >= 2:
field = event.args[0]
value = " ".join(event.args[1:])
await settings_mgr.apply(
event.user_id,
SettingsAction(action="set_soul", payload={"field": field, "value": value}),
)
return [
OutgoingMessage(chat_id=event.chat_id, text=f"Личность обновлена: {field} = {value}")
]
settings = await settings_mgr.get(event.user_id)
return [
OutgoingMessage(chat_id=event.chat_id, text=_render_mapping("🧠 Личность", settings.soul))
]
return [OutgoingMessage(chat_id=event.chat_id, text=MVP_UNAVAILABLE_TEXT)]
async def handle_settings_safety(
event: IncomingCommand, auth_mgr, platform, chat_mgr, settings_mgr
) -> list:
if len(event.args) >= 2:
trigger = event.args[0]
enabled = _parse_bool(event.args[1])
await settings_mgr.apply(
event.user_id,
SettingsAction(action="set_safety", payload={"trigger": trigger, "enabled": enabled}),
)
state = "включена" if enabled else "выключена"
return [OutgoingMessage(chat_id=event.chat_id, text=f"Безопасность {trigger} {state}")]
settings = await settings_mgr.get(event.user_id)
return [
OutgoingMessage(
chat_id=event.chat_id, text=_render_mapping("🔒 Безопасность", settings.safety)
)
]
return [OutgoingMessage(chat_id=event.chat_id, text=MVP_UNAVAILABLE_TEXT)]
async def handle_settings_plan(
event: IncomingCommand, auth_mgr, platform, chat_mgr, settings_mgr
) -> list:
settings = await settings_mgr.get(event.user_id)
return [OutgoingMessage(chat_id=event.chat_id, text=_render_mapping("💳 План", settings.plan))]
return [OutgoingMessage(chat_id=event.chat_id, text=MVP_UNAVAILABLE_TEXT)]
async def handle_settings_status(
event: IncomingCommand, auth_mgr, platform, chat_mgr, settings_mgr
) -> list:
chats = await chat_mgr.list_active(event.user_id)
settings = await settings_mgr.get(event.user_id)
text = "\n".join(
[
"📊 Статус",
f"Активных чатов: {len(chats)}",
f"Скиллов: {len(settings.skills)}",
f"Коннекторов: {len(settings.connectors)}",
]
)
return [OutgoingMessage(chat_id=event.chat_id, text=text)]
return [OutgoingMessage(chat_id=event.chat_id, text=MVP_UNAVAILABLE_TEXT)]
async def handle_settings_whoami(
event: IncomingCommand, auth_mgr, platform, chat_mgr, settings_mgr
) -> list:
return [OutgoingMessage(chat_id=event.chat_id, text=f"👤 {event.platform}:{event.user_id}")]
return [OutgoingMessage(chat_id=event.chat_id, text=MVP_UNAVAILABLE_TEXT)]
async def handle_toggle_skill(event, auth_mgr, platform, chat_mgr, settings_mgr) -> list:
settings = await settings_mgr.get(event.user_id)
keys = list(settings.skills.keys())
skill = event.payload.get("skill")
if not skill:
idx = event.payload.get("skill_index")
if isinstance(idx, int) and 1 <= idx <= len(keys):
skill = keys[idx - 1]
if not skill:
return [OutgoingMessage(chat_id=event.chat_id, text="Ошибка: не удалось определить навык.")]
return [OutgoingMessage(chat_id=event.chat_id, text=MVP_UNAVAILABLE_TEXT)]
enabled = not bool(settings.skills.get(skill, False))
await settings_mgr.apply(
event.user_id,
SettingsAction(action="toggle_skill", payload={"skill": skill, "enabled": enabled}),
)
state = "включён" if enabled else "выключен"
return [OutgoingMessage(chat_id=event.chat_id, text=f"Навык {skill} {state}.")]
async def handle_unknown_command(
event: IncomingCommand, auth_mgr, platform, chat_mgr, settings_mgr
) -> list:
return [
OutgoingMessage(
chat_id=event.chat_id,
text="Неизвестная команда. Используй !help для списка поддерживаемых команд.",
)
]

View file

@ -0,0 +1,180 @@
from __future__ import annotations
import re
from dataclasses import dataclass
from adapter.matrix.store import (
get_room_meta,
get_user_meta,
next_platform_chat_id,
set_room_meta,
set_user_meta,
)
_CHAT_ID_PATTERNS = (
re.compile(r"\bC(?P<index>\d+)\b", re.IGNORECASE),
re.compile(r"^Чат\s+(?P<index>\d+)$", re.IGNORECASE),
)
@dataclass(slots=True)
class ReconciliationResult:
recovered_rooms: int = 0
repaired_rooms: int = 0
backfilled_platform_chat_ids: int = 0
def _room_name(room: object) -> str | None:
for attr in ("name", "display_name"):
value = getattr(room, attr, None)
if isinstance(value, str) and value.strip():
return value.strip()
return None
def _chat_id_from_room(room: object, existing_meta: dict | None) -> str | None:
chat_id = (existing_meta or {}).get("chat_id")
if isinstance(chat_id, str) and chat_id:
return chat_id
name = _room_name(room)
if not name:
return None
for pattern in _CHAT_ID_PATTERNS:
match = pattern.search(name)
if match:
return f"C{int(match.group('index'))}"
return None
def _space_id_for_room(
room: object, rooms_by_id: dict[str, object], existing_meta: dict | None
) -> str | None:
existing_space_id = (existing_meta or {}).get("space_id")
if isinstance(existing_space_id, str) and existing_space_id:
return existing_space_id
parents = getattr(room, "parents", None)
if not parents:
parents = getattr(room, "space_parents", None)
if not parents:
return None
for parent_id in parents:
parent = rooms_by_id.get(parent_id)
if parent is None:
continue
if getattr(parent, "room_type", None) == "m.space" or getattr(parent, "children", None):
return parent_id
return parent_id
return None
def _matrix_user_id_for_room(
room: object, bot_user_id: str | None, existing_meta: dict | None
) -> str | None:
existing_user_id = (existing_meta or {}).get("matrix_user_id")
if isinstance(existing_user_id, str) and existing_user_id:
return existing_user_id
users = getattr(room, "users", None) or {}
for user_id in users:
if user_id != bot_user_id:
return user_id
return None
async def reconcile_startup_state(client: object, runtime: object) -> ReconciliationResult:
rooms_by_id = getattr(client, "rooms", None) or {}
bot_user_id = getattr(client, "user_id", None)
result = ReconciliationResult()
max_chat_index_by_user: dict[str, int] = {}
recovered_space_by_user: dict[str, str] = {}
for room_id, room in rooms_by_id.items():
if getattr(room, "room_type", None) == "m.space":
continue
existing_meta = await get_room_meta(runtime.store, room_id)
if existing_meta and existing_meta.get("redirect_room_id"):
continue
space_id = _space_id_for_room(room, rooms_by_id, existing_meta)
chat_id = _chat_id_from_room(room, existing_meta)
matrix_user_id = _matrix_user_id_for_room(room, bot_user_id, existing_meta)
if not space_id or not chat_id or not matrix_user_id:
continue
recovered_space_by_user[matrix_user_id] = space_id
chat_index = int(chat_id[1:])
max_chat_index_by_user[matrix_user_id] = max(
max_chat_index_by_user.get(matrix_user_id, 0),
chat_index,
)
display_name = (existing_meta or {}).get("display_name") or _room_name(room) or chat_id
room_meta = dict(existing_meta or {})
room_meta.update(
{
"room_type": "chat",
"chat_id": chat_id,
"display_name": display_name,
"matrix_user_id": matrix_user_id,
"space_id": space_id,
}
)
if not room_meta.get("platform_chat_id"):
room_meta["platform_chat_id"] = await next_platform_chat_id(runtime.store)
result.backfilled_platform_chat_ids += 1
if not room_meta.get("agent_id"):
registry = getattr(runtime, "registry", None)
if registry is not None:
assignment = registry.resolve_agent_for_user(matrix_user_id)
if assignment.agent_id:
room_meta["agent_id"] = assignment.agent_id
room_meta["agent_assignment"] = assignment.source
else:
registry = getattr(runtime, "registry", None)
if registry is not None:
assignment = registry.resolve_agent_for_user(matrix_user_id)
if assignment.source == "configured" and (
room_meta.get("agent_id") != assignment.agent_id
or room_meta.get("agent_assignment") != "configured"
):
room_meta["agent_id"] = assignment.agent_id
room_meta["agent_assignment"] = "configured"
elif (
assignment.source == "default"
and room_meta.get("agent_id") == assignment.agent_id
and not room_meta.get("agent_assignment")
):
room_meta["agent_assignment"] = "default"
if existing_meta is None:
result.recovered_rooms += 1
elif room_meta != existing_meta:
result.repaired_rooms += 1
await set_room_meta(runtime.store, room_id, room_meta)
await runtime.auth_mgr.confirm(matrix_user_id)
await runtime.chat_mgr.get_or_create(
user_id=matrix_user_id,
chat_id=chat_id,
platform="matrix",
surface_ref=room_id,
name=display_name,
)
for matrix_user_id, recovered_space_id in recovered_space_by_user.items():
user_meta = dict(await get_user_meta(runtime.store, matrix_user_id) or {})
user_meta["space_id"] = user_meta.get("space_id") or recovered_space_id
next_chat_index = max_chat_index_by_user[matrix_user_id] + 1
user_meta["next_chat_index"] = max(
int(user_meta.get("next_chat_index", 1)), next_chat_index
)
await set_user_meta(runtime.store, matrix_user_id, user_meta)
return result

View file

@ -0,0 +1,133 @@
from __future__ import annotations
import os
from collections.abc import AsyncIterator, Mapping
import structlog
from adapter.matrix.store import get_room_meta
from core.chat import ChatManager
from core.store import StateStore
from sdk.interface import (
Attachment,
MessageChunk,
MessageResponse,
PlatformClient,
PlatformError,
User,
UserSettings,
)
logger = structlog.get_logger(__name__)
def _ws_debug_enabled() -> bool:
value = os.environ.get("SURFACES_DEBUG_WS", "")
return value.strip().lower() in {"1", "true", "yes", "on"}
class RoutedPlatformClient(PlatformClient):
def __init__(
self,
*,
chat_mgr: ChatManager,
store: StateStore,
delegates: Mapping[str, PlatformClient],
) -> None:
if not delegates:
raise ValueError("RoutedPlatformClient requires at least one delegate")
self._chat_mgr = chat_mgr
self._store = store
self._delegates = dict(delegates)
self._default_client = next(iter(self._delegates.values()))
self._prototype_state = getattr(self._default_client, "_prototype_state", None)
async def get_or_create_user(
self,
external_id: str,
platform: str,
display_name: str | None = None,
) -> User:
return await self._default_client.get_or_create_user(
external_id=external_id,
platform=platform,
display_name=display_name,
)
async def send_message(
self,
user_id: str,
chat_id: str,
text: str,
attachments: list[Attachment] | None = None,
) -> MessageResponse:
delegate, platform_chat_id = await self._resolve_delegate(user_id, chat_id)
return await delegate.send_message(user_id, platform_chat_id, text, attachments)
async def stream_message(
self,
user_id: str,
chat_id: str,
text: str,
attachments: list[Attachment] | None = None,
) -> AsyncIterator[MessageChunk]:
delegate, platform_chat_id = await self._resolve_delegate(user_id, chat_id)
async for chunk in delegate.stream_message(user_id, platform_chat_id, text, attachments):
yield chunk
async def get_settings(self, user_id: str) -> UserSettings:
return await self._default_client.get_settings(user_id)
async def update_settings(self, user_id: str, action) -> None:
await self._default_client.update_settings(user_id, action)
async def close(self) -> None:
for delegate in self._delegates.values():
close = getattr(delegate, "close", None)
if callable(close):
await close()
async def _resolve_delegate(
self, user_id: str, local_chat_id: str
) -> tuple[PlatformClient, str]:
chat = await self._chat_mgr.get(local_chat_id, user_id)
if chat is None:
raise PlatformError(
f"unknown matrix chat id: {local_chat_id}",
code="MATRIX_CHAT_NOT_FOUND",
)
room_meta = await get_room_meta(self._store, chat.surface_ref)
if room_meta is None:
raise PlatformError(
f"matrix room is not bound: {chat.surface_ref}",
code="MATRIX_ROOM_NOT_BOUND",
)
agent_id = room_meta.get("agent_id")
platform_chat_id = room_meta.get("platform_chat_id")
if not agent_id or not platform_chat_id:
raise PlatformError(
f"matrix room routing is incomplete: {chat.surface_ref}",
code="MATRIX_ROUTE_INCOMPLETE",
)
delegate = self._delegates.get(str(agent_id))
if delegate is None:
raise PlatformError(
f"unknown matrix agent id: {agent_id}",
code="MATRIX_AGENT_NOT_FOUND",
)
if _ws_debug_enabled():
logger.warning(
"matrix_route_resolved",
user_id=user_id,
local_chat_id=local_chat_id,
surface_ref=chat.surface_ref,
agent_id=str(agent_id),
platform_chat_id=str(platform_chat_id),
delegate_type=type(delegate).__name__,
)
return delegate, str(platform_chat_id)

View file

@ -1,5 +1,8 @@
from __future__ import annotations
import asyncio
from weakref import WeakValueDictionary
from core.store import StateStore
ROOM_META_PREFIX = "matrix_room:"
@ -7,6 +10,12 @@ USER_META_PREFIX = "matrix_user:"
ROOM_STATE_PREFIX = "matrix_state:"
SKILLS_MSG_PREFIX = "matrix_skills_msg:"
PENDING_CONFIRM_PREFIX = "matrix_pending_confirm:"
LOAD_PENDING_PREFIX = "matrix_load_pending:"
RESET_PENDING_PREFIX = "matrix_reset_pending:"
STAGED_ATTACHMENTS_PREFIX = "matrix_staged_attachments:"
PLATFORM_CHAT_SEQ_KEY = "matrix_platform_chat_seq"
_STAGED_ATTACHMENTS_LOCKS: WeakValueDictionary[str, asyncio.Lock] = WeakValueDictionary()
_PLATFORM_CHAT_SEQ_LOCK = asyncio.Lock()
async def get_room_meta(store: StateStore, room_id: str) -> dict | None:
@ -17,6 +26,17 @@ async def set_room_meta(store: StateStore, room_id: str, meta: dict) -> None:
await store.set(f"{ROOM_META_PREFIX}{room_id}", meta)
async def get_platform_chat_id(store: StateStore, room_id: str) -> str | None:
meta = await get_room_meta(store, room_id)
return meta.get("platform_chat_id") if meta else None
async def set_platform_chat_id(store: StateStore, room_id: str, platform_chat_id: str) -> None:
meta = dict(await get_room_meta(store, room_id) or {})
meta["platform_chat_id"] = platform_chat_id
await set_room_meta(store, room_id, meta)
async def get_user_meta(store: StateStore, matrix_user_id: str) -> dict | None:
return await store.get(f"{USER_META_PREFIX}{matrix_user_id}")
@ -25,6 +45,12 @@ async def set_user_meta(store: StateStore, matrix_user_id: str, meta: dict) -> N
await store.set(f"{USER_META_PREFIX}{matrix_user_id}", meta)
async def set_room_agent_id(store: StateStore, room_id: str, agent_id: str) -> None:
meta = dict(await get_room_meta(store, room_id) or {})
meta["agent_id"] = agent_id
await set_room_meta(store, room_id, meta)
async def get_room_state(store: StateStore, room_id: str) -> str:
data = await store.get(f"{ROOM_STATE_PREFIX}{room_id}")
return data["state"] if data else "idle"
@ -51,16 +77,29 @@ async def next_chat_id(store: StateStore, matrix_user_id: str) -> str:
return f"C{index}"
async def next_platform_chat_id(store: StateStore) -> str:
async with _PLATFORM_CHAT_SEQ_LOCK:
data = await store.get(PLATFORM_CHAT_SEQ_KEY)
index = int((data or {}).get("next_platform_chat_index", 1))
await store.set(
PLATFORM_CHAT_SEQ_KEY,
{"next_platform_chat_index": index + 1},
)
return str(index)
def _pending_confirm_key(user_id: str, room_id: str | None = None) -> str:
if room_id is None:
return f"{PENDING_CONFIRM_PREFIX}{user_id}"
return f"{PENDING_CONFIRM_PREFIX}{user_id}:{room_id}"
async def get_pending_confirm(
store: StateStore, user_id: str, room_id: str | None = None
) -> dict | None:
return await store.get(_pending_confirm_key(user_id, room_id))
async def set_pending_confirm(
store: StateStore, user_id: str, room_id: str | dict, meta: dict | None = None
) -> None:
@ -74,3 +113,95 @@ async def clear_pending_confirm(
store: StateStore, user_id: str, room_id: str | None = None
) -> None:
await store.delete(_pending_confirm_key(user_id, room_id))
def _load_pending_key(user_id: str, room_id: str) -> str:
return f"{LOAD_PENDING_PREFIX}{user_id}:{room_id}"
async def get_load_pending(store: StateStore, user_id: str, room_id: str) -> dict | None:
return await store.get(_load_pending_key(user_id, room_id))
async def set_load_pending(store: StateStore, user_id: str, room_id: str, data: dict) -> None:
await store.set(_load_pending_key(user_id, room_id), data)
async def clear_load_pending(store: StateStore, user_id: str, room_id: str) -> None:
await store.delete(_load_pending_key(user_id, room_id))
def _reset_pending_key(user_id: str, room_id: str) -> str:
return f"{RESET_PENDING_PREFIX}{user_id}:{room_id}"
async def get_reset_pending(store: StateStore, user_id: str, room_id: str) -> dict | None:
return await store.get(_reset_pending_key(user_id, room_id))
async def set_reset_pending(
store: StateStore,
user_id: str,
room_id: str,
data: dict,
) -> None:
await store.set(_reset_pending_key(user_id, room_id), data)
async def clear_reset_pending(store: StateStore, user_id: str, room_id: str) -> None:
await store.delete(_reset_pending_key(user_id, room_id))
def _staged_attachments_key(room_id: str, user_id: str) -> str:
return f"{STAGED_ATTACHMENTS_PREFIX}{room_id}:{user_id}"
def _staged_attachments_lock(room_id: str, user_id: str) -> asyncio.Lock:
key = _staged_attachments_key(room_id, user_id)
lock = _STAGED_ATTACHMENTS_LOCKS.get(key)
if lock is None:
lock = asyncio.Lock()
_STAGED_ATTACHMENTS_LOCKS[key] = lock
return lock
async def get_staged_attachments(store: StateStore, room_id: str, user_id: str) -> list[dict]:
data = await store.get(_staged_attachments_key(room_id, user_id))
if not isinstance(data, dict):
return []
attachments = data.get("attachments")
if not isinstance(attachments, list):
return []
return [attachment for attachment in attachments if isinstance(attachment, dict)]
async def add_staged_attachment(
store: StateStore, room_id: str, user_id: str, attachment: dict
) -> None:
async with _staged_attachments_lock(room_id, user_id):
attachments = await get_staged_attachments(store, room_id, user_id)
attachments.append(attachment)
await store.set(_staged_attachments_key(room_id, user_id), {"attachments": attachments})
async def remove_staged_attachment_at(
store: StateStore, room_id: str, user_id: str, index: int
) -> dict | None:
async with _staged_attachments_lock(room_id, user_id):
attachments = await get_staged_attachments(store, room_id, user_id)
if index < 0 or index >= len(attachments):
return None
removed = attachments.pop(index)
if attachments:
await store.set(_staged_attachments_key(room_id, user_id), {"attachments": attachments})
else:
await store.delete(_staged_attachments_key(room_id, user_id))
return removed
async def clear_staged_attachments(store: StateStore, room_id: str, user_id: str) -> None:
async with _staged_attachments_lock(room_id, user_id):
await store.delete(_staged_attachments_key(room_id, user_id))

View file

@ -0,0 +1,44 @@
# Agent registry for the Matrix bot.
# Production target: one surface bot routes to 25-30 externally managed agents.
# Keep adding entries with the same base_url/workspace_path pattern.
#
# user_agents: maps a Matrix user ID to an agent ID.
# If a user is not listed, the bot uses the first agent from the list below.
# Omit this section entirely for a single-agent setup.
#
# agents: list of available agents.
# id — must match the agent ID known to the platform
# label — human-readable name (shown in logs)
# base_url — HTTP/WS URL of this agent's endpoint
# (overrides the global AGENT_BASE_URL env var for this agent)
# workspace_path — absolute path to this agent's workspace directory inside the bot container
# (the bot saves incoming files directly here and reads outgoing files from here)
# Example: /agents/0 means the bot mounts the shared volume at /agents/
# and this agent's files live under /agents/0/
user_agents:
"@user0:matrix.example.org": agent-0
"@user1:matrix.example.org": agent-1
"@user2:matrix.example.org": agent-2
agents:
- id: agent-0
label: "Agent 0"
base_url: "http://lambda.coredump.ru:7000/agent_0/"
workspace_path: "/agents/0"
- id: agent-1
label: "Agent 1"
base_url: "http://lambda.coredump.ru:7000/agent_1/"
workspace_path: "/agents/1"
- id: agent-2
label: "Agent 2"
base_url: "http://lambda.coredump.ru:7000/agent_2/"
workspace_path: "/agents/2"
# Continue the same pattern through agent-29 for a 25-30 agent deployment:
# - id: agent-29
# label: "Agent 29"
# base_url: "http://lambda.coredump.ru:7000/agent_29/"
# workspace_path: "/agents/29"

View file

@ -0,0 +1,10 @@
agents:
- id: agent-0
label: "Smoke Agent 0"
base_url: "http://agent-proxy:7000/agent_0/"
workspace_path: "/agents/0"
- id: agent-1
label: "Smoke Agent 1"
base_url: "http://agent-proxy:7000/agent_1/"
workspace_path: "/agents/1"

View file

@ -0,0 +1,8 @@
# Single-agent configuration for MVP deployment.
# For multi-agent setup with per-user routing, see config/matrix-agents.example.yaml.
agents:
- id: agent-1
label: Surface
base_url: "http://lambda.coredump.ru:7000/agent_1/"
workspace_path: "/agents/1"

View file

@ -1,7 +1,35 @@
# core/handlers/message.py
from __future__ import annotations
from core.protocol import IncomingMessage, OutgoingMessage, OutgoingTyping
from core.protocol import Attachment, IncomingMessage, OutgoingMessage, OutgoingTyping
def _infer_attachment_type(mime_type: str | None) -> str:
if not mime_type:
return "document"
if mime_type.startswith("image/"):
return "image"
if mime_type.startswith("audio/"):
return "audio"
if mime_type.startswith("video/"):
return "video"
return "document"
def _to_core_attachments(raw: list) -> list[Attachment]:
result = []
for a in raw:
if isinstance(a, Attachment):
result.append(a)
else:
result.append(Attachment(
type=getattr(a, "type", None) or _infer_attachment_type(getattr(a, "mime_type", None)),
url=getattr(a, "url", None),
filename=getattr(a, "filename", None),
mime_type=getattr(a, "mime_type", None),
workspace_path=getattr(a, "workspace_path", None),
))
return result
def _start_command(platform: str) -> str:
@ -29,10 +57,15 @@ async def handle_message(event: IncomingMessage, auth_mgr, platform, chat_mgr, s
user_id=event.user_id,
chat_id=event.chat_id,
text=event.text,
attachments=[],
attachments=event.attachments,
)
return [
OutgoingTyping(chat_id=event.chat_id, is_typing=False),
OutgoingMessage(chat_id=event.chat_id, text=response.response, parse_mode="markdown"),
OutgoingMessage(
chat_id=event.chat_id,
text=response.response,
parse_mode="markdown",
attachments=_to_core_attachments(getattr(response, "attachments", [])),
),
]

View file

@ -12,6 +12,7 @@ class Attachment:
content: bytes | None = None
filename: str | None = None
mime_type: str | None = None
workspace_path: str | None = None
@dataclass

View file

@ -0,0 +1,61 @@
services:
matrix-bot:
extends:
file: docker-compose.prod.yml
service: matrix-bot
build:
context: .
dockerfile: Dockerfile
target: development
args:
LAMBDA_AGENT_API_REF: ${LAMBDA_AGENT_API_REF:-master}
additional_contexts:
agent_api: ./external/platform-agent_api
tags:
- ${SURFACES_BOT_DEV_IMAGE:-surfaces-bot:dev}
environment:
AGENT_BASE_URL: http://platform-agent:8000
depends_on:
platform-agent:
condition: service_healthy
platform-agent:
build:
context: ./external/platform-agent
target: development
additional_contexts:
agent_api: ./external/platform-agent_api
environment:
PYTHONUNBUFFERED: "1"
AGENT_ID: ${AGENT_ID:-matrix-dev}
PROVIDER_MODEL: ${PROVIDER_MODEL:-openai/gpt-4o-mini}
PROVIDER_URL: ${PROVIDER_URL:-}
PROVIDER_API_KEY: ${PROVIDER_API_KEY:-}
COMPOSIO_API_KEY: ${COMPOSIO_API_KEY:-}
volumes:
- ./external/platform-agent/src:/app/src
- ./external/platform-agent_api:/agent_api
- agents:/workspace
command: >
sh -lc "
mkdir -p /workspace &&
chown -R agent:agent /workspace &&
exec /app/.venv/bin/uvicorn src.main:app --host 0.0.0.0 --port 8000 --no-access-log
"
ports:
- "8000:8000"
healthcheck:
test:
- CMD-SHELL
- python -c "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/openapi.json', timeout=2).read()"
interval: 60s
timeout: 5s
retries: 5
start_period: 15s
restart: unless-stopped
volumes:
agents:
name: ${SURFACES_SHARED_VOLUME:-surfaces-agents}
bot-state:
name: ${SURFACES_BOT_STATE_VOLUME:-surfaces-bot-state}

26
docker-compose.prod.yml Normal file
View file

@ -0,0 +1,26 @@
services:
matrix-bot:
image: "${SURFACES_BOT_IMAGE:?Set SURFACES_BOT_IMAGE to the pushed image, e.g. mput1/surfaces-bot:latest}"
environment:
MATRIX_HOMESERVER: ${MATRIX_HOMESERVER:-}
MATRIX_USER_ID: ${MATRIX_USER_ID:-}
MATRIX_PASSWORD: ${MATRIX_PASSWORD:-}
MATRIX_ACCESS_TOKEN: ${MATRIX_ACCESS_TOKEN:-}
MATRIX_PLATFORM_BACKEND: ${MATRIX_PLATFORM_BACKEND:-real}
MATRIX_AGENT_REGISTRY_PATH: ${MATRIX_AGENT_REGISTRY_PATH:-/app/config/matrix-agents.yaml}
AGENT_BASE_URL: ${AGENT_BASE_URL:-}
SURFACES_WORKSPACE_DIR: ${SURFACES_WORKSPACE_DIR:-/agents}
MATRIX_DB_PATH: /app/state/lambda_matrix.db
MATRIX_STORE_PATH: /app/state/matrix_store
PYTHONUNBUFFERED: "1"
volumes:
- agents:/agents
- bot-state:/app/state
- ./config:/app/config:ro
restart: unless-stopped
volumes:
agents:
name: ${SURFACES_SHARED_VOLUME:-surfaces-agents}
bot-state:
name: ${SURFACES_BOT_STATE_VOLUME:-surfaces-bot-state}

View file

@ -0,0 +1,18 @@
services:
agent-proxy:
volumes:
- ./docker/nginx/smoke-agents-timeout.conf:/etc/nginx/nginx.conf:ro
depends_on:
agent-no-status:
condition: service_started
agent-no-status:
build:
context: .
dockerfile: Dockerfile
target: production
args:
LAMBDA_AGENT_API_REF: ${LAMBDA_AGENT_API_REF:-master}
environment:
PYTHONUNBUFFERED: "1"
command: ["python", "-m", "tools.no_status_agent", "--host", "0.0.0.0", "--port", "8000"]

109
docker-compose.smoke.yml Normal file
View file

@ -0,0 +1,109 @@
services:
surface-smoke:
build:
context: .
dockerfile: Dockerfile
target: production
args:
LAMBDA_AGENT_API_REF: ${LAMBDA_AGENT_API_REF:-master}
environment:
PYTHONUNBUFFERED: "1"
SMOKE_TIMEOUT: ${SMOKE_TIMEOUT:-5}
volumes:
- agents:/agents
- ./config:/app/config:ro
depends_on:
agent-proxy:
condition: service_healthy
command: >
sh -lc "
python -m tools.check_matrix_agents --config /app/config/matrix-agents.smoke.yaml --timeout ${SMOKE_TIMEOUT:-5}
"
agent-proxy:
image: nginx:1.27-alpine
volumes:
- ./docker/nginx/smoke-agents.conf:/etc/nginx/nginx.conf:ro
healthcheck:
test:
- CMD-SHELL
- nc -z 127.0.0.1 7000
interval: 2s
timeout: 2s
retries: 15
start_period: 2s
depends_on:
agent-0:
condition: service_healthy
agent-1:
condition: service_healthy
ports:
- "${SMOKE_PROXY_PORT:-7000}:7000"
agent-0:
build:
context: ./external/platform-agent
target: development
additional_contexts:
agent_api: ./external/platform-agent_api
environment:
PYTHONUNBUFFERED: "1"
AGENT_ID: ${AGENT_0_ID:-agent-0}
PROVIDER_MODEL: ${PROVIDER_MODEL:-debug-model}
PROVIDER_URL: ${PROVIDER_URL:-http://provider.invalid/v1}
PROVIDER_API_KEY: ${PROVIDER_API_KEY:-debug-key}
volumes:
- ./external/platform-agent/src:/app/src
- ./external/platform-agent_api:/agent_api
- agents:/shared-agents
healthcheck:
test:
- CMD-SHELL
- python -c "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/openapi.json', timeout=2).read()"
interval: 5s
timeout: 3s
retries: 12
start_period: 5s
command: >
sh -lc "
mkdir -p /shared-agents/0 &&
rm -rf /workspace &&
ln -s /shared-agents/0 /workspace &&
exec /app/.venv/bin/uvicorn src.main:app --host 0.0.0.0 --port 8000 --no-access-log
"
agent-1:
build:
context: ./external/platform-agent
target: development
additional_contexts:
agent_api: ./external/platform-agent_api
environment:
PYTHONUNBUFFERED: "1"
AGENT_ID: ${AGENT_1_ID:-agent-1}
PROVIDER_MODEL: ${PROVIDER_MODEL:-debug-model}
PROVIDER_URL: ${PROVIDER_URL:-http://provider.invalid/v1}
PROVIDER_API_KEY: ${PROVIDER_API_KEY:-debug-key}
volumes:
- ./external/platform-agent/src:/app/src
- ./external/platform-agent_api:/agent_api
- agents:/shared-agents
healthcheck:
test:
- CMD-SHELL
- python -c "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/openapi.json', timeout=2).read()"
interval: 5s
timeout: 3s
retries: 12
start_period: 5s
command: >
sh -lc "
mkdir -p /shared-agents/1 &&
rm -rf /workspace &&
ln -s /shared-agents/1 /workspace &&
exec /app/.venv/bin/uvicorn src.main:app --host 0.0.0.0 --port 8000 --no-access-log
"
volumes:
agents:
name: ${SURFACES_SMOKE_VOLUME:-surfaces-smoke-agents}

39
docker-compose.yml Normal file
View file

@ -0,0 +1,39 @@
services:
platform-agent:
build:
context: ./external/platform-agent
target: development
additional_contexts:
agent_api: ./external/platform-agent_api
env_file: .env
environment:
PYTHONUNBUFFERED: "1"
volumes:
- ./external/platform-agent/src:/app/src
- ./external/platform-agent_api:/agent_api
- workspace:/workspace
command: >
sh -lc "
mkdir -p /workspace &&
chown -R agent:agent /workspace &&
exec /app/.venv/bin/uvicorn src.main:app --host 0.0.0.0 --port 8000
"
ports:
- "8000:8000"
restart: unless-stopped
matrix-bot:
build: .
env_file: .env
environment:
AGENT_BASE_URL: http://platform-agent:8000
SURFACES_WORKSPACE_DIR: /workspace
depends_on:
- platform-agent
volumes:
- workspace:/workspace
- ./config:/app/config:ro
restart: unless-stopped
volumes:
workspace:

View file

@ -0,0 +1,28 @@
events {}
http {
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
server {
listen 7000;
location /agent_0/ {
proxy_pass http://agent-0:8000/;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
location /agent_1/ {
proxy_pass http://agent-no-status:8000/;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
}
}

View file

@ -0,0 +1,28 @@
events {}
http {
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
server {
listen 7000;
location /agent_0/ {
proxy_pass http://agent-0:8000/;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
location /agent_1/ {
proxy_pass http://agent-1:8000/;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
}
}

View file

@ -1,143 +0,0 @@
# API Contract — Lambda Platform
> **Статус:** ЧЕРНОВИК — проектируем сами, уточняем с Азаматом когда SDK будет готов
> **Последнее обновление:** 2026-03-29
---
## Архитектурный контекст
Каждому пользователю выделяется **один LXC-контейнер** с workspace 10 ГБ.
Workspace содержит директории чатов: `C1/`, `C2/`, `C3/` — файлы + `history.db` в каждом.
**Master** управляет lifecycle контейнера (запуск, заморозка, пробуждение).
Бот **не управляет lifecycle** — он передаёт `user_id` + `chat_id` + сообщение.
Master сам решает: нужно ли поднять контейнер, смонтировать нужный чат, запустить агента.
---
## Base URL
```
https://api.lambda-platform.io/v1
```
## Аутентификация
```
Authorization: Bearer {SERVICE_TOKEN}
```
Сервисный токен выдаётся команде поверхностей. Не путать с токеном пользователя.
---
## Users
### GET /users/{external_id}?platform={platform}
Получает или создаёт пользователя.
**Query params:**
- `platform``telegram` | `matrix`
**Response 200:**
```json
{
"user_id": "usr_abc123",
"external_id": "12345678",
"platform": "telegram",
"display_name": "Иван Иванов",
"created_at": "2025-01-15T10:30:00Z",
"is_new": false
}
```
---
## Messages
Бот не управляет сессиями явно. Отправка сообщения — единственная операция.
Master решает: нужен ли новый контейнер, или разбудить существующий.
### POST /users/{user_id}/chats/{chat_id}/messages
Отправляет сообщение пользователя агенту. Master поднимает/размораживает контейнер,
монтирует нужный чат (`C1/`, `C2/`...), запускает агента.
**Request:**
```json
{
"text": "Привет, что ты умеешь?",
"attachments": []
}
```
**Response 200:**
```json
{
"message_id": "msg_qwe012",
"response": "Я AI-агент Lambda...",
"tokens_used": 142,
"finished": true
}
```
---
## Settings
### GET /users/{user_id}/settings
Настройки пользователя: скиллы, коннекторы, SOUL, безопасность, план.
**Response 200:**
```json
{
"skills": {"web-search": true, "browser": false},
"connectors": {"gmail": {"connected": true, "email": "user@gmail.com"}},
"soul": {"name": "Лямбда", "style": "friendly"},
"safety": {"email-send": true, "file-delete": true},
"plan": {"name": "Beta", "tokens_used": 800, "tokens_limit": 1000}
}
```
### POST /users/{user_id}/settings
Применяет действие над настройками.
**Request:**
```json
{
"action": "toggle_skill",
"payload": {"skill": "browser", "enabled": true}
}
```
**Response 200:**
```json
{"ok": true}
```
---
## Error format
```json
{
"error": "ERROR_CODE",
"message": "Human readable description",
"details": {}
}
```
Коды ошибок: `USER_NOT_FOUND`, `RATE_LIMITED`, `PLATFORM_ERROR`, `CONTAINER_UNAVAILABLE`
---
## Открытые вопросы к команде платфрмы (SDK)
- [ ] Точный формат эндпоинта отправки сообщения — URL, поля
- [ ] Как передавать вложения (файлы, изображения)? Через S3 pre-signed URL или напрямую?
- [ ] Стриминговый ответ (SSE / WebSocket) или только sync?
- [ ] Формат `SettingsAction` — совпадает с нашим или другой?

197
docs/deploy-architecture.md Normal file
View file

@ -0,0 +1,197 @@
# Deployment Architecture — Matrix Bot + Agents
> Сформировано 2026-04-27 по итогам обсуждения с платформой.
---
## Compose Artifacts
- **Production deploy:** `docker-compose.prod.yml`
Bot-only handoff через published image (`SURFACES_BOT_IMAGE`). Поднимает только `matrix-bot`, монтирует shared volume в `/agents`.
Платформа предоставляет 25-30 agent containers/services отдельно; бот подключается к ним через `base_url` из `matrix-agents.yaml`.
- **Internal full-stack E2E:** `docker-compose.fullstack.yml`
Внутренний harness для тестирования. Локально собирает `matrix-bot` через Dockerfile target `development`, поднимает один `platform-agent`, health-gated startup.
Production operators: `docker-compose.prod.yml`. Internal E2E: `docker-compose.fullstack.yml`.
---
## Топология
```
lambda.coredump.ru
├── :7000 (reverse proxy, path-based routing)
│ ├── /agent_0/ → agent_0 container
│ ├── /agent_1/ → agent_1 container
│ └── /agent_N/ → agent_N container
└── Matrix bot instance (один инстанс на всех)
└── volume /agents/ (shared с агентами)
├── /agents/0/ ← workspace agent_0
├── /agents/1/ ← workspace agent_1
└── /agents/N/
```
- **Один инстанс Matrix-бота** обслуживает всех пользователей.
- **Один агент-контейнер на пользователя.** Production scale target: 25-30 внешних агентов. Изоляция по `agent_id`, не через один общий agent instance.
- **Shared volume** `/agents/` смонтирован в Matrix-бот. В internal full-stack harness тот же volume mounted as `/workspace` inside `platform-agent`, чтобы bot-side absolute paths и agent workspace относились к одному и тому же хранилищу.
---
## Конфиг (два словаря)
```yaml
# config/matrix-agents.yaml
user_agents:
"@user0:matrix.lambda.coredump.ru": agent-0
"@user1:matrix.lambda.coredump.ru": agent-1
"@user2:matrix.lambda.coredump.ru": agent-2
agents:
- id: agent-0
label: "Agent 0"
base_url: "http://lambda.coredump.ru:7000/agent_0/"
workspace_path: "/agents/0"
- id: agent-1
label: "Agent 1"
base_url: "http://lambda.coredump.ru:7000/agent_1/"
workspace_path: "/agents/1"
- id: agent-2
label: "Agent 2"
base_url: "http://lambda.coredump.ru:7000/agent_2/"
workspace_path: "/agents/2"
```
- `user_agents` — маппинг Matrix user_id → agent_id. Если пользователь не найден — используется первый агент из списка.
- `agents[].base_url` — HTTP URL агент-эндпоинта. Бот подключается через AgentApi.
- `agents[].workspace_path` — абсолютный путь к воркспейсу агента **внутри контейнера бота** (т.е. на shared volume).
Бот сохраняет входящие файлы прямо в `{workspace_path}/`, читает исходящие из `{workspace_path}/`.
- Для 25-30 агентов продолжайте тот же паттерн до нужного номера: `/agent_17/` + `/agents/17`, `/agent_29/` + `/agents/29`.
## Surface Image Build Contract
Production image содержит только Matrix surface. Он не содержит `platform-agent` и не требует локального `external/` build context.
```bash
docker login
export SURFACES_BOT_IMAGE=mput1/surfaces-bot:latest
docker build --target production \
--build-arg LAMBDA_AGENT_API_REF=master \
-t "$SURFACES_BOT_IMAGE" .
docker push "$SURFACES_BOT_IMAGE"
```
Published image:
```text
mput1/surfaces-bot:latest
sha256:2f135f3535f7765d4377b440cdabe41195ad2efbc3e175def159ae4689ef90bd
```
`SURFACES_BOT_IMAGE` должен указывать на registry namespace, куда текущий Docker account может пушить. Ошибка `insufficient_scope` означает, что пользователь не залогинен в этот namespace, repository не создан, или у аккаунта нет push-доступа.
Production Dockerfile ставит `platform/agent_api` из Git по тому же принципу, что и `platform-agent` production image:
```bash
git+https://git.lambda.coredump.ru/platform/agent_api.git
```
Локальный `docker-compose.fullstack.yml` остаётся dev/E2E harness: он использует target `development` и `additional_contexts.agent_api=./external/platform-agent_api`, чтобы можно было тестировать surface вместе с локальным checkout SDK.
---
## Agent API (используем master ветку `platform/agent_api`)
```python
from lambda_agent_api.agent_api import AgentApi
connected_agents: dict[tuple[str, int], AgentApi] = {}
def on_agent_disconnect(agent: AgentApi):
connected_agents.pop((agent.id, agent.chat_id), None)
async def on_message(matrix_user_id: str, matrix_room_id: str, text: str):
agent_id = get_agent_id_by_user(matrix_user_id) # из user_agents конфига
platform_chat_id = get_room_platform_chat_id(matrix_room_id)
agent = connected_agents.get((agent_id, platform_chat_id))
if not agent:
agent = AgentApi(
agent_id,
get_agent_base_url(agent_id), # ws://lambda.coredump.ru:7000/agent_0/
on_disconnect=on_agent_disconnect,
chat_id=platform_chat_id, # отдельный thread на Matrix room
)
await agent.connect()
connected_agents[(agent_id, platform_chat_id)] = agent
async for event in agent.send_message(text):
...
```
**Параметры конструктора (master):**
```python
AgentApi(
agent_id: str,
base_url: str, # ws://host:port/agent_N/
chat_id: int = 0, # surfaces must supply per-room platform_chat_id
on_disconnect: callable,
)
```
**Lifecycle:** агент автоматически отключается после нескольких минут бездействия.
`on_disconnect` удаляет из пула → следующее сообщение создаёт новое соединение.
---
## Передача файлов
### Пользователь → Агент (входящий файл)
1. Matrix-бот получает файл от пользователя
2. Сохраняет в workspace агента: `/agents/{N}/{filename}`
3. Если файл уже существует, выбирает следующее имя: `filename (1).ext`, `filename (2).ext`
4. Вызывает `agent.send_message(text, attachments=["filename"])`
— путь относительно `/workspace` агента
### Агент → Пользователь (исходящий файл)
1. Агент эмитит `MsgEventSendFile(path="report.pdf")`
2. Matrix-бот читает файл: `/agents/{N}/report.pdf`
3. Отправляет как Matrix file message пользователю
**Ключевое:** production handoff через `docker-compose.prod.yml` и internal E2E через `docker-compose.fullstack.yml` используют один и тот же `/agents` contract на стороне поверхности. Прямой HTTP-доступ к файлам не нужен.
---
## Текущее состояние platform-agent (main)
- Composio интегрирован в main (`#9-интеграция-composIO`)
- Агент требует в `.env`: `AGENT_ID`, `COMPOSIO_API_KEY`
- Backend: `IsolatedShellBackend` (main) / `CompositeBackend` (ветка `#19`, не merged)
- Memory: `MemorySaver` — история слетает при рестарте контейнера (known limitation)
---
## platform-master (будущее, пока не используем)
Ветка `feat/storage` реализует реальный Master-сервис:
- `POST /api/v1/create {chat_id}` → поднимает/переиспользует sandbox-контейнер
- TTL-based lifecycle (300с default, конфигурируемо)
- `ChatStorage` — API для upload/download файлов через Master
- Auth + p2p lease — вне текущего scope MVP
**Для деплоя MVP используем статический конфиг без Master.**
При готовности Master: `get_agent_url()` будет вызывать `POST /api/v1/create`, URL возвращается в ответе.
---
## Открытые вопросы
- `platform-agent_api #9-clientside-tool-call` убирает `attachments` и `MsgEventSendFile` — пока используем master. Уточнить у платформы сроки мержа перед деплоем.
- История при рестарте агента теряется — `platform-agent` использует `MemorySaver` (in-memory). Ограничение платформы.
- `AGENT_ID` и `COMPOSIO_API_KEY` для каждого агент-контейнера — значения предоставляет платформа.

View file

@ -0,0 +1,301 @@
# Matrix Direct-Agent Prototype
> **ВНИМАНИЕ: Это исторический документ.**
> Описанный здесь прототип был интегрирован в `main` и стал основой для Matrix MVP, но архитектура претерпела значительные изменения. Для актуальной информации по деплою смотрите `docs/deploy-architecture.md`, а для создания новой поверхности — `docs/max-surface-guide.md`.
Русскоязычная заметка по прототипу Matrix surface, который ходит не в `MockPlatformClient`, а напрямую в живой agent backend по WebSocket.
## Что сделали
В этой ветке собран рабочий Matrix-only прототип с минимальным вмешательством в существующую архитектуру.
Ключевая идея:
- Matrix-адаптер и `core/` остаются на старом контракте `PlatformClient`
- вместо `sdk/mock.py` можно включить `sdk/real.py`
- `sdk/real.py` внутри разделяет две ответственности:
- `sdk/agent_session.py` — прямое общение с agent по WebSocket
- `sdk/prototype_state.py` — локальный user/settings state для прототипа
Это позволило не переписывать Matrix-логику под нестабильный `platform/master` и при этом подключить живого агента вместо мока.
## Что поменялось в `surfaces-bot`
Добавлено:
- `sdk/agent_session.py`
- `sdk/prototype_state.py`
- `sdk/real.py`
- тесты для transport/state/real backend
Изменено:
- `adapter/matrix/bot.py`
- `adapter/matrix/handlers/auth.py`
- `README.md`
- интеграционные и Matrix dispatcher тесты
Функционально это дало:
- переключение Matrix backend через env:
- `MATRIX_PLATFORM_BACKEND=mock`
- `MATRIX_PLATFORM_BACKEND=real`
- прямую отправку текста в live agent через `AGENT_BASE_URL`
- локальное хранение settings и user mapping
- изоляцию backend memory по `thread_id`
- исправление повторных invite: бот теперь сначала `join()`, а уже потом решает, нужно ли пере-провиженить Space/chat tree
## Что поменяли в `platform-agent`
Для прототипа потребовался минимальный локальный патч в клонированном `external/platform-agent`.
Изменения:
- `src/api/external.py`
- `src/agent/service.py`
Смысл патча:
- agent больше не использует один общий hardcoded `thread_id="default"`
- `thread_id` читается из query parameter WebSocket-соединения
- дальше этот `thread_id` передаётся в config memory/checkpointer
Локальный commit в clone:
- `1dca2c1``feat: support websocket thread ids`
Важно:
- этот commit живёт в `external/platform-agent`
- он не входит в git-историю `surfaces-bot`
- если прототип должен запускаться у других людей без ручных патчей, этот commit надо отдельно запушить или повторить в platform repo
## Текущая архитектура прототипа
Поток сообщения сейчас такой:
1. Matrix room event попадает в `adapter/matrix`
2. адаптер переводит его в `IncomingMessage` / `IncomingCommand`
3. `EventDispatcher` вызывает handler из `core/`
4. handler вызывает `PlatformClient`
5. при real backend это `RealPlatformClient`
6. `RealPlatformClient` строит `thread_key`
7. `AgentSessionClient` открывает WebSocket на `agent_ws/?thread_id=...`
8. ответ агента возвращается обратно в Matrix
Что остаётся локальным в v1:
- `!settings`
- `!skills`
- `!soul`
- `!safety`
- user registration mapping
Что реально идёт в живого агента:
- обычные текстовые сообщения
- память по чатам через `thread_id`
## Ограничения прототипа
Сейчас это не полный platform integration, а рабочий direct-agent prototype.
Ограничения:
- только текстовый чат
- без attachments в agent
- без async task callbacks/webhooks
- без реального control-plane из `platform/master`
- encrypted Matrix rooms пока не поддержаны
- repeat invite не создаёт новую Space-структуру, если user уже был провиженен локально
- backend/provider ошибки пока не везде деградируют в user-facing reply; часть ошибок всё ещё может уронить процесс surface
## Как запускать
Нужно поднять два процесса:
- patched `platform-agent`
- Matrix bot из `surfaces-bot`
### 1. Подготовить `platform-agent`
Локальный clone:
- [external/platform-agent](/Users/a/MAI/sem2/lambda/surfaces-bot/external/platform-agent)
И связанный SDK clone:
- [external/platform-agent_api](/Users/a/MAI/sem2/lambda/surfaces-bot/external/platform-agent_api)
Первичная подготовка:
```bash
cd /Users/a/MAI/sem2/lambda/surfaces-bot/external/platform-agent
uv sync
uv pip install --python .venv/bin/python -e ../platform-agent_api
```
Если у вас был активирован чужой venv, сначала сделайте:
```bash
deactivate
```
Иначе `uv pip install` может поставить пакет не в тот interpreter.
### 2. Запустить agent backend
Пример с OpenRouter:
```bash
cd /Users/a/MAI/sem2/lambda/surfaces-bot/external/platform-agent
export PROVIDER_URL=https://openrouter.ai/api/v1
export PROVIDER_API_KEY='YOUR_OPENROUTER_KEY'
export PROVIDER_MODEL='qwen/qwen3.5-122b-a10b'
uv run uvicorn src.main:app --host 0.0.0.0 --port 8000
```
После этого WebSocket endpoint должен быть доступен по:
```text
ws://127.0.0.1:8000/agent_ws/
```
### 3. Запустить Matrix bot
В отдельном терминале:
```bash
cd /Users/a/MAI/sem2/lambda/surfaces-bot
export MATRIX_PLATFORM_BACKEND=real
export AGENT_BASE_URL=http://127.0.0.1:8000
export MATRIX_HOMESERVER=https://matrix.lambda.coredump.ru
export MATRIX_USER_ID=@lambda_surface_test_bot:matrix.lambda.coredump.ru
export MATRIX_PASSWORD='YOUR_PASSWORD'
PYTHONPATH=. uv run python -m adapter.matrix.bot
```
Если всё ок, в логах будет что-то вроде:
```text
Matrix bot starting ...
```
## Точные команды
Ниже команды в том виде, в котором реально поднимался рабочий прототип.
### Platform / agent backend
```bash
cd /Users/a/MAI/sem2/lambda/surfaces-bot/external/platform-agent
deactivate 2>/dev/null || true
uv sync
uv pip install --python .venv/bin/python -e ../platform-agent_api
export PROVIDER_URL=https://openrouter.ai/api/v1
export PROVIDER_API_KEY='YOUR_OPENROUTER_KEY'
export PROVIDER_MODEL='qwen/qwen3.5-122b-a10b'
uv run uvicorn src.main:app --host 0.0.0.0 --port 8000
```
### Matrix bot
```bash
cd /Users/a/MAI/sem2/lambda/surfaces-bot
export MATRIX_PLATFORM_BACKEND=real
export AGENT_BASE_URL=http://127.0.0.1:8000
export MATRIX_HOMESERVER=https://matrix.lambda.coredump.ru
export MATRIX_USER_ID=@lambda_surface_test_bot:matrix.lambda.coredump.ru
export MATRIX_PASSWORD='YOUR_PASSWORD'
PYTHONPATH=. uv run python -m adapter.matrix.bot
```
### Перезапуск Matrix state с нуля
```bash
cd /Users/a/MAI/sem2/lambda/surfaces-bot
rm -f lambda_matrix.db
rm -rf matrix_store
PYTHONPATH=. uv run python -m adapter.matrix.bot
```
## Smoke test
Рекомендуемый сценарий ручной проверки:
1. Пригласить бота в fresh unencrypted room
2. Дождаться join
3. Если это первый invite для данного локального state:
- бот создаст private Space
- бот создаст room `Чат 1`
4. Открыть `Чат 1`
5. Отправить `!start`
6. Отправить обычное текстовое сообщение
7. Проверить, что ответ пришёл от live backend, а не от `[MOCK]`
8. Проверить `!new`
9. Проверить, что память разделяется между чатами
Если бот уже был однажды провиженен и локальный state не очищался:
- повторный invite не создаст новую Space-структуру
- бот просто зайдёт в room и будет отвечать там
Это нормальное поведение текущей реализации.
## Сброс локального Matrix state
Если нужно повторно проверить именно first-invite provisioning:
```bash
cd /Users/a/MAI/sem2/lambda/surfaces-bot
rm -f lambda_matrix.db
rm -rf matrix_store
PYTHONPATH=. uv run python -m adapter.matrix.bot
```
После этого можно снова приглашать бота как "с нуля".
## Частые проблемы
### 1. `ModuleNotFoundError: lambda_agent_api`
Значит `platform-agent_api` не установлен в `.venv` агента.
Исправление:
```bash
cd /Users/a/MAI/sem2/lambda/surfaces-bot/external/platform-agent
uv pip install --python .venv/bin/python -e ../platform-agent_api
```
### 2. `CERTIFICATE_VERIFY_FAILED` при запуске Matrix bot
Это не ошибка surface logic. Это TLS trust problem до Matrix homeserver.
Нужно:
- либо установить системные/Python certificates
- либо передать корпоративный CA через `SSL_CERT_FILE`
### 3. Бот заходит в room, но не создаёт новую Space
Скорее всего user уже есть в локальном state.
Варианты:
- это ожидаемо для repeat invite
- либо очистить `lambda_matrix.db` и `matrix_store`
### 4. Бот падает после message send
Значит backend/provider вернул ошибку, которая ещё не была деградирована в user-facing ответ.
Пример уже встречавшегося кейса:
- неверный model id
- key не имеет доступа к model
Сначала проверяйте:
- `PROVIDER_URL`
- `PROVIDER_MODEL`
- `PROVIDER_API_KEY`
## Полезные ссылки внутри repo
- [README.md](/Users/a/MAI/sem2/lambda/surfaces-bot/README.md)
- [adapter/matrix/bot.py](/Users/a/MAI/sem2/lambda/surfaces-bot/adapter/matrix/bot.py)
- [sdk/agent_session.py](/Users/a/MAI/sem2/lambda/surfaces-bot/sdk/agent_session.py)
- [sdk/real.py](/Users/a/MAI/sem2/lambda/surfaces-bot/sdk/real.py)
- [sdk/prototype_state.py](/Users/a/MAI/sem2/lambda/surfaces-bot/sdk/prototype_state.py)
- [2026-04-08-matrix-direct-agent-prototype-design.md](/Users/a/MAI/sem2/lambda/surfaces-bot/docs/superpowers/specs/2026-04-08-matrix-direct-agent-prototype-design.md)
- [2026-04-08-matrix-direct-agent-prototype.md](/Users/a/MAI/sem2/lambda/surfaces-bot/docs/superpowers/plans/2026-04-08-matrix-direct-agent-prototype.md)

View file

@ -4,263 +4,101 @@
Один бот, каждый чат — отдельная комната, все комнаты собраны в personal Space.
При первом входе бот создаёт для пользователя личное пространство (Space) —
это как папка в Element. Внутри Space бот создаёт комнату для каждого нового
чата с агентом. Пользователь видит аккуратную структуру: одно пространство,
внутри — список чатов. История хранится нативно в Matrix — это часть протокола,
ничего дополнительно делать не нужно.
При первом invite бот создаёт для пользователя личное пространство (Space) и первую рабочую комнату.
История хранится нативно в Matrix. UX прагматичный: явные `!`-команды, локальный state-store, нативные Matrix rooms.
Matrix выбран как внутренняя поверхность: команды лаборатории, тестировщики,
разработчики скиллов. Поэтому UX здесь прагматичный: минимум магии, явные
команды `!`, локальный state-store и нативные Matrix rooms.
Matrix — внутренняя поверхность: команда лаборатории, тестировщики, разработчики скиллов.
---
## Аутентификация
## Онбординг
### Флоу
1. Пользователь приглашает бота в личные сообщения или пишет в общей комнате
2. Бот проверяет `@user:matrix.org` — есть ли аккаунт на платформе
3. Если нет — бот отправляет одноразовый код или ссылку
4. Пользователь подтверждает, платформа возвращает токен
5. Бот сохраняет привязку `matrix_user_id → platform_user_id`
1. Пользователь приглашает бота в личные сообщения (DM) на Matrix-сервере
2. Бот принимает invite, создаёт Space `Lambda — {display_name}` и первую комнату `Чат 1`
3. Приглашает пользователя в `Чат 1` и пишет приветствие
4. Дальнейшее общение ведётся в рабочих комнатах, не в DM
### В моке
- Любой пользователь проходит аутентификацию автоматически
- Бот отвечает: «Добро пожаловать, {display_name}. Создаю ваше пространство...»
- Демонстрирует флоу без реальной платформы
---
## Чаты через Space + комнаты (вариант Б)
### Структура
```
Space: «Lambda — {display_name}»
├── 💬 Чат 1 ← первый чат, создаётся автоматически
├── 💬 Чат 1 ← создаётся автоматически при invite
├── 💬 Чат 2
└── 💬 Исследование рынка ← пользователь сам называет
└── 💬 Исследование рынка ← пользователь называет сам через !new
```
### Создание Space
При первом входе бот:
1. Создаёт Space `Lambda — {display_name}`
2. Создаёт первую комнату-чат `Чат 1`
3. Передаёт `invite=[matrix_user_id]` прямо в `room_create(...)` для Space и комнаты
4. Привязывает `chat_id ↔ room_id` в локальном состоянии
5. Пишет приветствие в `Чат 1`
**Требование:** незашифрованные комнаты. E2EE не поддержан (инфраструктурное ограничение).
---
## Работающие команды
### Управление чатами
Команды работают в зарегистрированных комнатах бота:
| Команда | Действие |
|---|---|
| `!new` | Создать новый чат (новую комнату в Space) |
| `!new Название` | Создать чат с именем |
| `!help` | Показать шпаргалку по доступным командам |
| `!rename Название` | Переименовать текущую комнату |
| `!archive` | Архивировать чат и вывести бота из комнаты |
| `!chats` | Показать список чатов |
| `!settings`, `!skills`, `!soul`, `!safety`, `!plan`, `!status`, `!whoami` | Настройки и диагностика |
| `!chats` | Список активных чатов |
| `!rename <название>` | Переименовать текущую комнату |
| `!archive` | Архивировать чат |
| `!help` | Справка |
### Создание нового чата
1. Пользователь пишет `!new` или `!new Анализ конкурентов`
2. Бот создаёт новую комнату в Space
3. Сразу приглашает пользователя через `room_create(..., invite=[user_id])`
4. Регистрирует комнату в локальном состоянии и `ChatManager`
5. Пользователь переходит в новую комнату — начинает диалог
### Контекст
### В моке
- Space и комнаты создаются реально через matrix-nio
- Сообщения передаются в MockPlatformClient с `chat_id` (C1, C2...)
- История хранится в Matrix нативно
- Дефолтные `skills`, `safety`, `soul`, `plan` подмешиваются даже после частичных локальных обновлений настроек
| Команда | Действие |
|---|---|
| `!clear` | Сбросить контекст текущего чата (создаёт новый thread у агента) |
| `!reset` | Псевдоним для `!clear` |
### Переименование и архивирование
### Подтверждения
- `!rename` обновляет имя комнаты через state event `m.room.name`
- `!archive` архивирует чат в `ChatManager` и делает `room_leave(...)`
- Если бот потерял локальное состояние и видит комнату как `unregistered:*`, то `!rename` и `!archive` возвращают защитное сообщение вместо сломанного действия
| Команда | Действие |
|---|---|
| `!yes` | Подтвердить действие агента |
| `!no` | Отменить действие агента |
### Вложения (файловая очередь)
Matrix-клиенты отправляют файлы и текст отдельными событиями. Файл без текстовой инструкции ставится в очередь, а не уходит агенту сразу.
| Команда | Действие |
|---|---|
| `!list` | Показать файлы в очереди |
| `!remove <n>` | Удалить файл из очереди по номеру |
| `!remove all` | Очистить всю очередь |
Как отправить файлы агенту:
1. Отправь один или несколько файлов в рабочую комнату
2. Напиши текстовое сообщение с инструкцией, например: `что на изображении?`
3. Бот отправит агенту текст вместе со всеми файлами из очереди
---
## Основной диалог
## Диалог
### Флоу сообщения
1. Пользователь пишет текст в комнату-чат
2. Бот показывает typing (m.typing event)
3. Запрос уходит в платформу (MockPlatformClient)
4. Бот отвечает в той же комнате
### Вложения
- Файлы, изображения отправляются как Matrix media events
- Бот принимает `m.file`, `m.image`, `m.audio`
- Передаёт в платформу как `attachments` через `IncomingMessage`
- В моке: подтверждение получения + заглушка-ответ
### Реакции как действия
Matrix поддерживает реакции на сообщения (`m.reaction`).
Используем это для подтверждения действий агента:
```
Агент: Хочу отправить письмо на vasya@mail.ru
Тема: «Отчёт за неделю»
👍 — подтвердить ❌ — отменить
```
Пользователь ставит реакцию — бот обрабатывает. Нативно и удобно.
### Треды для длинных задач
Если агент выполняет долгую задачу (deep research, генерация документа),
бот создаёт тред от своего первого ответа и пишет промежуточные статусы туда.
Основной чат не засоряется.
```
Бот: Начинаю исследование по теме «AI агенты 2025» [→ в треде]
└── Ищу источники... (1/4)
└── Анализирую статьи... (2/4)
└── Формирую отчёт... (3/4)
└── Готово. Отчёт: [...]
```
- Любое текстовое сообщение уходит агенту, бот показывает typing-индикатор
- Ответ стримится по WebSocket и выводится в ту же комнату
- Каждая комната имеет свой `platform_chat_id` — контексты изолированы между комнатами
---
## Настройки и диагностика
## Передача файлов
Отдельной комнаты `Настройки` в текущей версии нет. Команды вызываются как обычные
`!`-команды из зарегистрированных комнат бота, а `!settings` отдаёт сводный dashboard
по скиллам, личности, безопасности и активным чатам.
### Пользователь → Агент
Бот сохраняет файл в shared volume: `{workspace_path}/{filename}`
и передаёт агенту относительный путь как `workspace_path`.
### Коннекторы
```
!connectors — показать список
!connect gmail — подключить Gmail (OAuth ссылка)
!connect github — подключить GitHub
!connect calendar — подключить Google Calendar
!connect notion — подключить Notion
!disconnect gmail — отключить
```
Статус:
```
Коннекторы:
✅ Gmail — подключён (user@gmail.com)
❌ GitHub — не подключён → !connect github
❌ Google Calendar — не подключён
❌ Notion — не подключён
```
В моке: OAuth ссылка-заглушка → «Подключено ✓»
### Скиллы
```
!skills — показать список
!skill on browser — включить Browser Use
!skill off browser — выключить
```
Статус:
```
Скиллы:
✅ web-search — поиск в интернете
✅ fetch-url — чтение веб-страниц
✅ email — чтение почты (требует Gmail)
❌ browser — управление браузером
❌ image-gen — генерация изображений
❌ video-gen — генерация видео
✅ files — работа с файлами
❌ calendar — календарь (требует Google Calendar)
```
В моке: состояние хранится локально.
### Личность агента
```
!soul — показать текущий SOUL.md
!soul name Лямбда — задать имя агента
!soul style brief — стиль: brief | friendly | formal
!soul priority «разбирать почту утром» — приоритетная задача
!soul reset — сбросить к дефолту
```
В моке: SOUL.md генерируется и хранится локально, агент обращается по имени.
### Безопасность
```
!safety — показать настройки
!safety on email-send — требовать подтверждение перед отправкой письма
!safety off calendar-create — не спрашивать для создания событий
```
Статус:
```
Подтверждение требуется для:
✅ отправка письма
✅ удаление файлов
✅ публикация в соцсетях
❌ создание события в календаре
❌ поиск в интернете
```
### Подписка
```
!plan — показать текущий план
```
```
Подписка: Beta (бесплатно)
Токены этот месяц: 800 / 1000
━━━━━━━━░░ 80%
```
Заглушка, реализует другая команда.
### Статус и диагностика
```
!status — состояние платформы и чатов
!whoami — текущий аккаунт платформы
```
```
Статус:
Платформа: ✅ доступна
Аккаунт: user@lambda.lab
Активных чатов: 3
```
### Агент → Пользователь
Агент эмитит путь к файлу в своём workspace. Бот читает файл из `/agents/...`
и отправляет пользователю как Matrix file message.
---
## FSM состояния
## Известные ограничения
```
[Invite] → AuthPending → AuthConfirmed
SpaceSetup → Idle (в комнате Настройки)
[новая комната] → ChatCreated → Idle (в чате)
ReceivingMessage → WaitingResponse → Idle
WaitingReaction (confirm) → [✅/❌] → Idle
LongTask → [тред со статусами] → Done → Idle
```
---
## Стек
- Python 3.11+
- matrix-nio (async) — Matrix клиент
- MockPlatformClient → `platform/interface.py`
- structlog для логирования
- SQLite / in-memory store для хранения `matrix_user_id → platform_user_id`, состояния скиллов и маппинга `chat_id → room_id`
---
## Ограничения текущей версии
- Ручной QA и текущая разработка идут только в незашифрованных комнатах
- После рестарта бот делает bootstrap sync и стартует с `since`, поэтому старые события не должны переигрываться повторно
- Если удалить локальную БД/стор, старые Matrix rooms останутся, но команды, завязанные на локальную регистрацию чатов, перестанут работать для этих комнат до повторного онбординга
| Проблема | Причина |
|---|---|
| `!save` / `!load` / `!context` | Нестабильны: `!save` зависит от агента (пишет файл в workspace), сессии хранятся in-memory и теряются при рестарте |
| Первый чанк ответа иногда пропадает после tool/file flow | Баг в upstream `platform-agent`; подробности: `docs/reports/2026-04-22-platform-streaming-final-bug-report-ru.md` |
| Персистентность истории между рестартами агента | `platform-agent` использует `MemorySaver` (in-memory) |
| E2EE комнаты | `python-olm` не собирается на macOS/ARM |
| `!settings` и настройки скиллов/SOUL/безопасности | Заглушки MVP, требуют готового SDK платформы |

313
docs/new-surface-guide.md Normal file
View file

@ -0,0 +1,313 @@
# Руководство по созданию новой поверхности
Этот документ описывает, как написать новую новую поверхность (например, Discord, Slack или Custom Web) по образцу текущей Matrix-поверхности в ветке `main`.
Он основан на актуальной реализации Matrix surface в репозитории и отражает текущую продакшн-логику, а не устаревший легаси.
---
## 1. Общая архитектура
### 1.1. Что такое поверхность
Поверхность — это тонкий адаптер между конкретной платформой (Платформа) и общим ядром бота.
В репозитории есть разделение:
- `core/` — общее ядро и бизнес-логика
- `adapter/<platform>/` — реализация конкретной поверхности
- `sdk/real.py` — работа с реальной платформой / агентом
- `config/` — статическая конфигурация агентов
- `docs/surface-protocol.md` — общий контракт поверхностей
### 1.2. Как это работает
Поверхность должна:
- принимать нативные события от Платформа
- преобразовывать их в единый внутренний контракт (`IncomingMessage`, `IncomingCommand`, `IncomingCallback`)
- передавать их в `core`
- получать ответы из `core` (`OutgoingMessage`, `OutgoingUI`, `OutgoingTyping`, `OutgoingNotification`)
- преобразовывать ответы обратно в нативные нативные сообщения
Поверхность не должна:
- управлять жизненным циклом агентских контейнеров
- хранить долгую историю бесед вне `core`/платформы
- аутентифицировать пользователей сама (если это не часть Платформа API)
---
## 2. Структура новой поверхности
### 2.1. Основные каталоги
Рекомендуемая структура для новой платформы:
```
adapter/<platform>/
bot.py
converter.py
agent_registry.py
files.py
handlers/
store.py
```
### 2.2. Принцип reuse
По примеру Matrix surface, New surface должен переиспользовать общий `core` и общий `sdk`.
Не дублируйте бизнес-логику, а реализуйте только адаптер:
- `adapter/<platform>/converter.py` — конвертация событий платформы ⇄ внутренние структуры
- `adapter/<platform>/bot.py` — основной runtime, старт Платформа client, loop, отправка/прием
- `adapter/<platform>/agent_registry.py` — загрузка `config/<platform>-agents.yaml`
- `adapter/<platform>/files.py` — хранение входящих/исходящих вложений
---
## 3. Контракт входящих/исходящих событий
### 3.1. Внутренний формат
Смотрите `core/protocol.py`. Основные типы:
- `IncomingMessage` — обычное текстовое сообщение + вложения
- `IncomingCommand` — управляющая команда
- `IncomingCallback` — подтверждение / интерактивные действия
- `OutgoingMessage` — ответ пользователю
- `OutgoingUI` — интерфейсные элементы (кнопки и т.п.)
- `OutgoingTyping` — индикатор печати
- `OutgoingNotification` — системное уведомление
### 3.2. Пример конверсии Matrix
В Matrix-реализации `adapter/matrix/converter.py`:
- текст `!yes` / `!no` превращается в `IncomingCallback` с `action: confirm/cancel`
- `!list`/`!remove` говорят не агенту, а surface-процессу
- вложения `m.file`, `m.image`, `m.audio`, `m.video` нормализуются в `Attachment`
Для Платформа реализуйте аналогичную логику для native команд вашего клиента.
---
## 4. Реестр агентов и маршрутизация
### 4.1. Что хранит реестр
В текущей Matrix реализации есть `config/matrix-agents.yaml` и `adapter/matrix/agent_registry.py`.
Структура:
```yaml
user_agents:
"@user0:matrix.example.org": agent-0
"@user1:matrix.example.org": agent-1
agents:
- id: agent-0
label: "Agent 0"
base_url: "http://lambda.coredump.ru:7000/agent_0/"
workspace_path: "/agents/0"
```
### 4.2. Логика выбора агента
- `user_agents` маппит конкретного пользователя на `agent_id`
- если user_id не найден, используется первый агент из списка
- `agents[].base_url` определяет URL агента
- `agents[].workspace_path` определяет путь внутри surface-контейнера для этого агента
Это важно: именно на этом контракте строится разделение агентов по рабочим каталогам.
### 4.3. Рекомендуемая Версия для новой платформы
Создайте `config/<platform>-agents.yaml` с тем же смыслом.
- `user_agents` — маппинг external user_id → agent_id
- `agents` — список агентов
- `workspace_path` для каждого агента должен быть абсолютным путем внутри surface-контейнера, например `/agents/0`
---
## 5. Файловый контракт
### 5.1. Shared volume
Текущее Matrix-решение использует shared volume:
- surface монтирует общий том как `/agents`
- каждый агент видит свою поддиректорию как `/workspace`
Топология:
```
Bot (/agents) Agent (/workspace = /agents/N/)
/agents/0/report.pdf ←──→ /workspace/report.pdf
```
### 5.2. Правила записи файлов
В `adapter/matrix/files.py` реализовано:
- входящий файл сохраняется прямо в `{workspace_root}/{filename}`
- возвращается путь `workspace_path` относительный внутри рабочего каталога агента
- при коллизии имен создаётся `file (1).ext`, `file (2).ext`
- `Attachment.workspace_path` передаётся агенту
Для исходящих файлов:
- surface читает файл из `workspace_root / workspace_path`
- загружает его в платформу
### 5.3. Пример поведения
- Пользователь отправляет файл → surface скачивает файл и кладёт его в agent workspace
- Агент получает `attachments=["report.pdf"]` и работает с относительным `workspace_path`
- Агент пишет результат в `/workspace/result.txt`
- surface читает `/agents/{N}/result.txt` и отправляет файл пользователю
---
## 6. Чат-менеджмент и контекст
### 6.1. `platform_chat_id`
Matrix-реализация использует `platform_chat_id` как стабильный идентификатор чата на стороне агента.
- `room_meta.platform_chat_id` определяется и сохраняется в `adapter/matrix/store.py`
- `reconcile_startup_state()` восстанавливает отсутствующие `platform_chat_id` при рестарте
- `RoutedPlatformClient` перенаправляет запросы агенту по `agent_id` + `platform_chat_id`
Для New surface тот же принцип:
- каждая внешняя беседа должна привязываться к одному внутреннему `chat_id`
- этот `chat_id` используется для вызовов агента
- если в Платформа есть несколько комнат/топиков, каждая должна иметь свой `surface_ref`
### 6.2. Команды управления чатами
Matrix поддерживает следующие команды, которые нужно сохранить в Платформа:
- `!new [название]` — создать новый чат
- `!chats` — список активных чатов
- `!rename <название>` — переименовать текущий чат
- `!archive` — архивировать чат
- `!clear` / `!reset` — сбросить контекст текущего чата
- `!yes` / `!no` — подтвердить или отменить действие агента
- `!list` — показать очередь вложений
- `!remove <n>` / `!remove all` — удалить вложение из очереди
- `!help` — справка
Эти команды реализованы в Matrix через `adapter/matrix/handlers/`.
### 6.3. Очередь вложений
Matrix surface поддерживает staged attachments:
- файл может быть отправлен без текста
- surface сохраняет файл в `staged_attachments` для конкретного room_id + user_id
- следующий текст отправляется агенту вместе со всеми файлами из очереди
В Платформа можно реализовать ту же модель:
- `!list` показывает текущую очередь
- `!remove` удаляет файл из очереди
- команда-индикатор или следующее текстовое сообщение отправляет queued attachments агенту
---
## 7. Runtime и окружение
### 7.1. Переменные среды
Для Matrix surface текущий runtime ожидает:
- `MATRIX_HOMESERVER` — URL Matrix-сервера
- `MATRIX_USER_ID``@bot:example.org`
- `MATRIX_PASSWORD` или `MATRIX_ACCESS_TOKEN`
- `MATRIX_PLATFORM_BACKEND` — должно быть `real` для продакшна
- `MATRIX_AGENT_REGISTRY_PATH` — путь к `config/matrix-agents.yaml`
- `AGENT_BASE_URL` — fallback URL агента
- `SURFACES_WORKSPACE_DIR` — путь к shared volume внутри контейнера (по умолчанию `/workspace` в коде, но в docs рекомендуют `/agents`)
Для New surface используйте аналогичные переменные:
- `PLATFORM_PLATFORM_BACKEND=real`
- `PLATFORM_AGENT_REGISTRY_PATH=/app/config/<platform>-agents.yaml`
- `SURFACES_WORKSPACE_DIR=/agents`
- `AGENT_BASE_URL` — если хотите общий fallback
### 7.2. Environment contract
В коде `adapter/matrix/bot.py`:
- `_agent_base_url_from_env()` читает `AGENT_BASE_URL` или `AGENT_WS_URL`
- `_load_agent_registry_from_env()` читает `MATRIX_AGENT_REGISTRY_PATH`
- `_build_platform_from_env()` выбирает `RealPlatformClient` при `MATRIX_PLATFORM_BACKEND=real`
В New surface реализуйте ту же логику, заменив префиксы на `PLATFORM_`.
---
## 8. Локальное тестирование
Для тестирования новой поверхности вместе с одним локальным агентом используйте паттерн `docker-compose.fullstack.yml`.
В этом режиме:
- Запускается 1 контейнер вашей поверхности
- Запускается 1 контейнер `platform-agent`
- Поднимается локальный shared volume (`surfaces-agents`)
- Поверхность настроена маршрутизировать запросы на `http://platform-agent:8000` (через `AGENT_BASE_URL`)
- Пользователь общается с ботом, а бот напрямую общается с локальным агентом, разделяя с ним общую папку для файлов.
Это самый быстрый способ проверить интеграцию новой платформы без внешнего бэкенда.
---
## 9. Реализация шаг за шагом
1. Скопировать `adapter/matrix/` как шаблон для `adapter/<platform>/`.
2. Сделать `adapter/<platform>/converter.py`:
- превратить native нативные сообщения в `IncomingMessage`
- превратить команды в `IncomingCommand`
- превратить yes/no-подтверждения в `IncomingCallback`
3. Сделать `adapter/<platform>/agent_registry.py` на основе `adapter/matrix/agent_registry.py`.
4. Сделать `adapter/<platform>/files.py` на основе `adapter/matrix/files.py`.
5. Сделать `adapter/<platform>/bot.py`:
- инстанцировать runtime
- читать env vars `PLATFORM_*`
- загружать реестр агентов
- обрабатывать входящие события
- отправлять `Outgoing*` обратно в Платформа
6. Реализовать команды управления чатами и очередь вложений.
7. Прописать `config/<platform>-agents.yaml`.
8. Прописать `docker-compose.platform.yml` или аналог, чтобы surface монтировал `/agents`.
9. Написать тесты по аналогии с `tests/adapter/matrix/`.
10. Проверить, что все env vars читаются из окружения и не зависят от устаревших Matrix-переменных.
---
## 10. Важные замечания
- Текущий Matrix surface на ветке `main` — активная реализация, а не устаревший легаси.
- Документация и код согласованы: `agent_registry`, `files`, `routed_platform`, `reconciliation` работают вместе.
- Обязательно явно задавайте `SURFACES_WORKSPACE_DIR=/agents` в production, если `workspace_path` в реестре указывает на `/agents/*`.
- Для New surface сохраните ту же архитектуру: surface = thin adapter, агенты = внешние сервисы.
- Не пытайтесь в surface реализовывать логику запуска/стопа агент-контейнеров.
---
## 11. Полезные ссылки внутри репозитория
- `README.md`
- `docs/deploy-architecture.md`
- `docs/surface-protocol.md`
- `adapter/matrix/bot.py`
- `adapter/matrix/converter.py`
- `adapter/matrix/agent_registry.py`
- `adapter/matrix/files.py`
- `adapter/matrix/routed_platform.py`
- `adapter/matrix/reconciliation.py`
- `tests/adapter/matrix/`

View file

@ -0,0 +1,245 @@
# Баг-репорт: регрессия стриминга платформы после file/tool flow
## Кратко
После обновления до текущих upstream-версий платформы стриминг ответов стал нестабильным в сценариях с вложениями и tool/file flow.
Наблюдаемые симптомы:
- первый текстовый chunk ответа может приходить уже обрезанным
- соседние ответы могут "протекать" друг в друга
- после некоторых запросов бот перестаёт присылать финальный ответ
- платформа присылает дублирующий `END`
До обновления платформы этот класс ошибок у нас не воспроизводился.
## Версии платформы
В рантайме используются upstream-репозитории без локальных правок:
- `platform-agent`: `5e7c2df954cc3cd2f5bf8ae688e10a20038dde61`
- `platform-agent_api`: `aa480bbec5bbf8e006284dd03aed1c2754e9bbee`
## Контекст интеграции
- поверхность: Matrix
- транспорт к платформе: websocket через `platform-agent_api`
- `chat_id` на платформу отправляется как стабильный числовой surrogate id
- shared workspace: `/workspace`
Важно: vendored platform repos чистые, совпадают с upstream. Проблема воспроизводится без локальных patch'ей в платформу.
## Пользовательские симптомы
Примеры из живого диалога:
- ожидалось: `Моя ошибка: ...`
- фактически пришло: `оя ошибка: ...`
- ожидалось начало ответа вида `По фото IMG_3183.png ...`
- фактически пришло: `IMG_3183.png**) — это ...`
Также наблюдалось:
- после вопросов по изображениям бот иногда вообще перестаёт отвечать
- в том же чате, до attachment/tool flow, ответы приходят корректно
## Шаги воспроизведения
1. Поднять `platform-agent` и Matrix surface на версиях выше.
2. Отправить несколько обычных текстовых сообщений.
3. Убедиться, что начальные ответы стримятся корректно.
4. Отправить изображения/файлы и задать вопросы вида:
- `что изображено на фото`
- уточняющие follow-up вопросы по тем же вложениям
5. Затем отправить ещё одно обычное текстовое сообщение.
6. Наблюдать один или несколько симптомов:
- первый chunk начинается с середины слова
- ответ начинается с середины фразы
- хвост прошлого ответа загрязняет следующий
- видимого финального ответа нет вообще
## Что удалось доказать
По debug-логам Matrix surface видно, что обрезание присутствует уже в первом chunk, полученном от платформы.
Корректные первые chunk'и до attachment/tool flow:
- `Hey! How`
- `Я`
- `Первый файл не найден — возможно, ...`
Некорректные первые chunk'и после attachment/tool flow:
- `IMG_3183.png**) — это ю...`
- `оя ошибка: в первом запросе...`
Это логируется сразу после десериализации websocket event на клиентской стороне, до Matrix rendering и до финальной сборки текста ответа. Из этого следует, что повреждение происходит на стороне платформенного стриминга, а не в Matrix sender.
## Дополнительное наблюдение по протоколу
Платформа сейчас отправляет дублирующий `END`.
Релевантные места в upstream:
- `external/platform-agent/src/agent/service.py`
- уже `yield MsgEventEnd(...)`
- `external/platform-agent/src/api/external.py`
- после завершения цикла дополнительно отправляет ещё один `MsgEventEnd(...)`
В живых логах это видно как:
- первый `END`
- второй `END`
- клиентская suppression логика, которая гасит дубликат
Это само по себе делает границы ответа неоднозначными и повышает риск "протекания" поздних событий в следующий запрос.
## Предполагаемая первопричина
Похоже, что на стороне платформы одновременно есть две проблемы.
### 1. Двойной сигнал завершения стрима
Для одного ответа генерируется два `END`.
Вероятные последствия:
- нечёткая граница ответа
- поздние события могут относиться не к тому запросу
- соседние ответы могут смешиваться
### 2. Некорректное извлечение текстового chunk'а
В текущем upstream `platform-agent/src/agent/service.py` текст форвардится из `chunk.content` внутри `on_chat_model_stream`.
Однако в документации LangChain для `astream_events()` примеры используют `event["data"]["chunk"].text`, а в Deep Agents stream дополнительно есть `ns`/`source`, которые важны для отделения main-agent output от tool/subagent/model-internal stream.
Потенциальные последствия:
- первый видимый chunk может быть неполным
- во внешний клиент может попадать не только финальный пользовательский текст
- attachment/tool flow сильнее деградирует поведение стрима
## Почему проблема считается платформенной
С нашей стороны были проверены и исключены базовые причины:
- вложения корректно сохраняются в `/workspace`
- контейнер `platform-agent` видит эти файлы
- Matrix surface получает уже обрезанный первый chunk от платформы
- обрезание происходит до сборки финального ответа
- эксперимент с reconnect на каждый запрос не исправил проблему
- платформенные vendored repos сейчас совпадают с upstream
## Ожидаемое поведение
Для каждого пользовательского запроса:
- текстовые chunk'и должны начинаться с реального начала ответа модели
- должен приходить ровно один terminal `END`
- границы ответов должны быть однозначными
- file/tool flow не должен ломать следующий ответ
## Фактическое поведение
После attachment/tool flow:
- первый text chunk может быть уже обрезан
- `END` приходит дважды
- следующий ответ может начаться с середины слова или фразы
- отдельные запросы могут не завершаться видимым ответом
## Дополнительный failure mode: большие изображения
В отдельном воспроизведении текстовые запросы до файлового сценария работали корректно, а сбой возникал только после попытки анализа изображений.
По логам видно уже не только stream corruption, но и конкретный image-path failure:
- `platform-agent` рвёт websocket с `1009 (message too big)`
- провайдер возвращает `400` с причиной:
- `Exceeded limit on max bytes per data-uri item : 10485760`
Характерный фрагмент:
```text
websockets.exceptions.ConnectionClosedError: received 1009 (message too big); then sent 1009 (message too big)
...
Agent error (INTERNAL_ERROR): Error code: 400 - {
'error': {
'message': 'Provider returned error',
'metadata': {
'raw': '{"error":{"message":"Exceeded limit on max bytes per data-uri item : 10485760"...}}'
}
}
}
```
Из этого следует:
- текстовый path сам по себе работоспособен
- image-analysis path в платформе сейчас передаёт изображение как data URI
- для достаточно больших изображений это утыкается в provider limit `10,485,760` байт на один data-uri item
- параллельно websocket path между surface и platform-agent неустойчив к этому failure scenario и закрывается с `1009`
То есть в платформе есть как минимум ещё одна отдельная проблема помимо некорректного стриминга:
- отсутствует безопасная обработка больших изображений до отправки в provider
- отсутствует аккуратная деградация без разрыва websocket-сессии
## Что стоит исправить в платформе
1. Отправлять ровно один `MsgEventEnd` на один ответ.
2. Перепроверить extraction текста из `on_chat_model_stream`:
- вероятно, должен использоваться `chunk.text`, а не `chunk.content`.
3. Учитывать `ns`/`source` и форвардить наружу только main assistant output.
4. Перед отправкой изображений в provider проверять размер payload и не пытаться слать oversized data-uri.
5. Для больших изображений:
- либо делать resize/compression,
- либо возвращать контролируемую user-facing ошибку без разрыва websocket.
6. В идеале добавить более жёсткую request boundary в websocket protocol, чтобы late events не могли прилипать к следующему запросу.
## Наши временные mitigation'ы на стороне surface
Они не исправляют корень, только снижают ущерб:
- suppression duplicate `END`
- короткий post-`END` drain window
- idle timeout для зависшего стрима
- transport failures нормализуются в `PlatformError`, чтобы Matrix bot не падал процессом
Эти меры понадобились только потому, что в текущем сценарии platform stream contract нестабилен.
## Приложение: характерный фрагмент логов
```text
[matrix-bot] text chunk queue=True text='Первый файл не найден — возможно,'
[matrix-bot] ...
[matrix-bot] end event queue=True tokens=0
[matrix-bot] end event queue=True tokens=0
[matrix-bot] dropped duplicate END tokens=0
[matrix-bot] text chunk queue=True text='IMG_3183.png**) — это ю'
[matrix-bot] ...
[matrix-bot] end event queue=True tokens=0
[matrix-bot] end event queue=True tokens=0
[matrix-bot] dropped duplicate END tokens=0
[matrix-bot] text chunk queue=True text='оя ошибка: в первом запросе я случайно постав'
```
Этот фрагмент показывает две вещи:
- duplicate `END` действительно приходит от платформы
- следующий первый chunk уже приходит в клиента обрезанным
## Приложение: характерный фрагмент логов для больших изображений
```text
platform-agent-1 | websockets.exceptions.ConnectionClosedError: received 1009 (message too big); then sent 1009 (message too big)
...
matrix-bot-1 | Agent error (INTERNAL_ERROR): Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"Exceeded limit on max bytes per data-uri item : 10485760","type":"invalid_request_error"...}}}}
```
Этот фрагмент показывает ещё две вещи:
- image path в платформе реально упирается в лимит провайдера на размер data URI
- при этом ошибка не изолируется корректно и сопровождается разрывом websocket-соединения

View file

@ -0,0 +1,294 @@
# Финальный баг-репорт: потеря начала ответа и сбои streaming/image path в `platform-agent`
## Статус
Это финальный отчёт после полного аудита интеграции `surfaces -> platform-agent_api -> platform-agent`.
Итог:
- текущая реализация `surfaces` рабочая, но проблемная из-за upstream-дефектов платформы
- баг с пропадающим началом ответа на текущем состоянии **не локализуется в `surfaces`**
- в воспроизведённом сценарии повреждённый первый текстовый chunk рождается уже внутри `platform-agent`
- помимо этого подтверждены ещё два независимых platform-side дефекта:
- duplicate `END`
- некорректная обработка больших изображений (`data-uri > 10 MB`, `WS 1009`)
## Версии и состояние кода
Рантайм воспроизводился на vendored upstream-репозиториях без локальных platform patchей:
- `platform-agent`: `5e7c2df954cc3cd2f5bf8ae688e10a20038dde61`
- `platform-agent_api`: `aa480bbec5bbf8e006284dd03aed1c2754e9bbee`
Со стороны `surfaces` transport layer был предварительно очищен:
- убрана локальная stream-semantics из `sdk/agent_api_wrapper.py`
- `sdk/real.py` переведён на pinned upstream `platform-agent_api.AgentApi`
- больше нет локального post-END drain, custom listener и wrapper-owned reclassification events
Это важно: баг воспроизводился **после** удаления наших транспортных костылей.
## Контекст интеграции
- поверхность: Matrix
- транспорт к платформе: WebSocket через upstream `platform-agent_api.AgentApi`
- `chat_id` на платформу: стабильный числовой surrogate id, выдаваемый со стороны `surfaces`
- файловый контракт: shared `/workspace`, вложения передаются как относительные пути в `attachments`
## Пользовательские симптомы
Наблюдались несколько классов сбоев:
1. Начало ответа может пропасть
- ожидалось: `Моя ошибка: ...`
- фактически: `оя ошибка: ...`
- ожидалось: `На двух изображениях: ...`
- фактически: ` двух изображениях: ...`
2. После tool/file flow ответы могут вести себя нестабильно
- следующий ответ стартует с середины фразы
- в некоторых сценариях после image/tool path платформа отвечает ошибкой или замолкает
3. На больших изображениях image path падает совсем
- provider error `Exceeded limit on max bytes per data-uri item : 10485760`
- websocket закрывается с `1009 (message too big)`
## Что было проверено на стороне `surfaces`
Ниже перечислено, что именно было перепроверено в нашем коде и почему это не выглядит корнем проблемы.
### 1. Мы больше не режем и не переклассифицируем stream локально
В текущем `surfaces`:
- `sdk/agent_api_wrapper.py` — thin construction/factory shim над upstream `AgentApi`
- `sdk/real.py` — просто итерирует upstream events и склеивает `MsgEventTextChunk.text`
- `adapter/matrix/bot.py` — отправляет `OutgoingMessage.text` в Matrix без `strip/lstrip`
Наблюдение:
- в текущем коде не осталось места, где строка могла бы превратиться из `Моя ошибка` в `оя ошибка` через локальный slicing
### 2. Сборка ответа у нас линейная и тупая
`sdk/real.py` делает только следующее:
- если пришёл `MsgEventTextChunk` — добавляет `event.text` в `response_parts`
- если пришёл `MsgEventSendFile` — превращает его в `Attachment`
- не пытается “восстанавливать” поток после `END`
Следствие:
- если начало уже отсутствует в `event.text`, мы его не можем ни потерять, ни вернуть
### 3. Matrix sender не модифицирует текст
`adapter/matrix/bot.py` передаёт текст дальше как есть.
Следствие:
- Matrix renderer не является объяснением пропажи первого куска
## Что было проверено в `platform-agent_api`
Upstream client всё ещё имеет спорную queue-архитектуру:
- одна активная `_current_queue`
- `MsgEventEnd` съедается внутри `send_message()`
- в `finally` очередь отвязывается и дренится orphan messages
Это архитектурно хрупко и может быть источником других boundary bugs.
Но в конкретном воспроизведении этот слой не был точкой порчи текста.
Почему:
- в raw logs клиент получил **ровно тот же** первый text chunk, который сервер уже отправил
- queue/dequeue не изменили его содержимое
## Что удалось доказать по raw logs
Для финальной проверки была временно добавлена точечная диагностика в:
- `external/platform-agent/src/agent/service.py`
- `external/platform-agent/src/api/external.py`
- `external/platform-agent_api/lambda_agent_api/agent_api.py`
Эта диагностика **не входила** в рабочую реализацию и использовалась только для локализации бага.
### Ключевое наблюдение
На проблемном запросе после tool/file flow сервер сам yieldил уже обрезанный первый chunk:
```text
platform-agent-1 | [raw-stream][server-yield] chat=1 event=TEXT text=' двух изображениях:\n\n**Первое изображение'
platform-agent-1 | [raw-stream][ws-send] chat=1 event=AGENT_EVENT_TEXT_CHUNK text=' двух изображениях:\n\n**Первое изображение' path=None
matrix-bot-1 | [raw-stream][client-listen] agent=matrix-bot chat=1 queue_active=True AGENT_EVENT_TEXT_CHUNK text=' двух изображениях:\n\n**Первое изображение'
matrix-bot-1 | [raw-stream][client-dequeue] agent=matrix-bot chat=1 request=2 AGENT_EVENT_TEXT_CHUNK text=' двух изображениях:\n\n**Первое изображение'
```
Это означает:
- порча произошла **до** websocket-клиента
- `surfaces` transport layer не является источником именно этого дефекта
- `platform-agent_api` не исказил этот конкретный chunk по дороге
Дополнительно тот же паттерн виден и вне image-сценария:
```text
platform-agent-1 | [raw-stream][server-yield] chat=1 event=TEXT text='сё работает напрямую'
...
matrix-bot-1 | [raw-stream][client-dequeue] ... text='сё работает напрямую'
```
То есть сервер уже выдаёт `сё`, а не `Всё`.
## Наиболее вероятный root cause
Главный подозреваемый — `external/platform-agent/src/agent/service.py`.
Сейчас он делает следующее:
- читает `self._agent.astream_events(...)`
- обрабатывает только `kind == "on_chat_model_stream"`
- берёт `chunk = event["data"]["chunk"]`
- если `chunk.content`, форвардит `MsgEventTextChunk(text=chunk.content)`
Проблема в том, что это очень грубое преобразование raw event stream в пользовательский текст.
### Почему именно это место выглядит корнем
1. Первый битый chunk уже рождается на server-side
- это подтверждено логами выше
2. Код берёт только `chunk.content`
- если начало ответа приходит в другой форме, поле или raw event, оно просто теряется
3. Код не учитывает `ns` / `source`
- после tool/vision flow у Deep Agents / LangChain может быть более сложная структура потока
- текущий adapter flattenит её слишком агрессивно
4. Код никак не валидирует, что наружу уходит именно main assistant output
Итоговая гипотеза:
> После tool/file/vision flow `platform-agent` неправильно адаптирует `astream_events()` в `MsgEventTextChunk`. Начало итогового пользовательского ответа может находиться не в том raw event, который сейчас читается через `chunk.content`, либо теряться из-за упрощённой фильтрации потока.
## Подтверждённый отдельный баг: duplicate `END`
Это отдельный platform-side дефект.
Сейчас:
- `external/platform-agent/src/agent/service.py` уже делает `yield MsgEventEnd(...)`
- `external/platform-agent/src/api/external.py` после завершения цикла дополнительно отправляет ещё один `MsgEventEnd(...)`
По логам это выглядит так:
```text
platform-agent-1 | [raw-stream][server-yield] chat=1 event=END
platform-agent-1 | [raw-stream][ws-send] chat=1 event=AGENT_EVENT_END text=None path=None
platform-agent-1 | [raw-stream][ws-send] chat=1 event=AGENT_EVENT_END duplicate_end=true
matrix-bot-1 | ... AGENT_EVENT_END tokens_used=0
matrix-bot-1 | ... AGENT_EVENT_END tokens_used=0
```
Независимая оценка:
- duplicate `END` — реальный баг платформы
- он делает границу ответа менее надёжной
- но в текущем воспроизведении он **не объясняет** сам факт потери первого text chunk
То есть это важный, но вторичный дефект.
## Подтверждённый отдельный баг: большие изображения ломают image path
В отдельном воспроизведении платформа падала на анализе изображений с provider error:
```text
Exceeded limit on max bytes per data-uri item : 10485760
```
И параллельно websocket рвался с:
```text
received 1009 (message too big); then sent 1009 (message too big)
```
Это означает:
- image path отправляет в provider oversized `data:` URI
- безопасной предвалидации / деградации нет
- failure scenario сопровождается разрывом websocket-соединения
Независимая оценка:
- это отдельный platform-side bug
- он не объясняет потерю первого чанка в текстовом сценарии напрямую
- но подтверждает, что path `tool/file/image -> platform stream` в целом сейчас нестабилен
## Что мы считаем исключённым
С достаточной уверенностью можно исключить:
1. Локальный slicing текста в `surfaces`
2. Локальную “умную” реконструкцию потока, потому что она была удалена
3. Matrix sender как источник потери первого чанка
4. `platform-agent_api` queue/dequeue как primary root cause именно в этом воспроизведении
## Финальная независимая оценка
Текущая оценка вероятностей:
- `75%` — ошибка в `platform-agent`, в адаптере `astream_events() -> MsgEventTextChunk`
- `15%` — provider/model stream приносит начало ответа в другой raw event/field, а `platform-agent` его некорректно интерпретирует
- `10%` — вторичные platform-side boundary defects (`duplicate END`, queue semantics и т.д.)
- `~0-5%` — ошибка в `surfaces`
Итоговый вывод:
> На текущем состоянии кода баг с пропадающим началом ответа следует считать platform-side дефектом. Воспроизведение после cleanup transport layer показывает, что первый повреждённый text chunk формируется уже внутри `platform-agent` до отправки в websocket.
## Что нужно исправить в платформе
### Обязательно
1. Убрать duplicate `END`
- один ответ должен завершаться ровно одним `MsgEventEnd`
2. Перепроверить адаптацию `astream_events()` в `service.py`
- логировать и проанализировать raw `event["event"]`
- проверить `event.get("name")`
- смотреть `event.get("ns")`
- сравнить `chunk.content` с тем, что реально лежит в `chunk.text` / raw chunk repr
3. Форвардить наружу только финальный main assistant output
- не flattenить весь поток без учёта `ns/source`
### Желательно
4. Сделать image path устойчивым к oversized payload
- preflight check размера
- resize/compress или controlled error без разрыва WS
5. Улучшить client/server protocol boundary
- более строгая корреляция запроса и ответа
- более однозначная semantics конца ответа
## Что мы сделали со своей стороны
Со стороны `surfaces` уже выполнено следующее:
- transport layer очищен до thin adapter над upstream `AgentApi`
- локальные stream-workaroundы удалены
- рабочая интеграция сохранена
- known issue задокументирован
То есть текущая интеграция не “идеальна”, но её поведение теперь достаточно чистое, чтобы platform bug было можно локализовать без смешения ответственности.
## Приложение: короткий диагноз
Если нужна самая короткая формулировка для issue tracker:
> После cleanup transport layer в `surfaces` и воспроизведения на clean upstream platform repos видно, что `platform-agent` иногда сам порождает первый `MsgEventTextChunk` уже обрезанным, особенно после tool/file flow. Дополнительно подтверждены duplicate `END` и отдельный image-path failure на больших `data:` URI.

View file

@ -0,0 +1,515 @@
# Matrix Direct-Agent Prototype Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Replace `MockPlatformClient` with a real Matrix-only direct-agent prototype backend while keeping Matrix adapter logic stable and preserving a clean future migration path.
**Architecture:** Introduce a thin compatibility layer under `sdk/` that splits direct WebSocket agent messaging from local prototype-only control-plane state. Keep `core/` and Matrix handlers on the existing `PlatformClient` contract, and limit surface changes to runtime wiring and tests.
**Tech Stack:** Python 3.11, `aiohttp` WebSocket client, `matrix-nio`, `pydantic`, `pytest`, `pytest-asyncio`
---
## File Structure
- Create: `sdk/agent_session.py`
Purpose: Minimal direct WebSocket client for the real agent, including thread-key derivation, request/response parsing, and sync/stream helpers.
- Create: `sdk/prototype_state.py`
Purpose: Local prototype-only user mapping and settings store kept behind a small API.
- Create: `sdk/real.py`
Purpose: `PlatformClient` implementation that composes `AgentSessionClient` and `PrototypeStateStore`.
- Modify: `sdk/__init__.py`
Purpose: export `RealPlatformClient` if useful for runtime imports.
- Modify: `adapter/matrix/bot.py`
Purpose: runtime/backend selection and env-based configuration for mock vs real backend.
- Create: `tests/platform/test_agent_session.py`
Purpose: transport-level tests for direct agent communication.
- Create: `tests/platform/test_prototype_state.py`
Purpose: unit tests for local user/settings behavior.
- Create: `tests/platform/test_real.py`
Purpose: contract tests for `RealPlatformClient`.
- Modify: `tests/core/test_integration.py`
Purpose: prove the new platform implementation preserves core behavior.
- Modify: `README.md`
Purpose: document backend selection and prototype limitations after code is working.
---
### Task 1: Add Direct Agent Session Transport
**Files:**
- Create: `sdk/agent_session.py`
- Test: `tests/platform/test_agent_session.py`
- [ ] **Step 1: Write the failing transport tests**
```python
import pytest
from sdk.agent_session import AgentSessionClient, build_thread_key
def test_build_thread_key_uses_surface_user_and_chat_id():
assert build_thread_key("matrix", "@alice:example.org", "C1") == "matrix:@alice:example.org:C1"
@pytest.mark.asyncio
async def test_send_message_collects_text_chunks_and_tokens(aiohttp_server):
...
@pytest.mark.asyncio
async def test_stream_message_yields_incremental_chunks(aiohttp_server):
...
```
- [ ] **Step 2: Run test to verify it fails**
Run: `pytest tests/platform/test_agent_session.py -q`
Expected: FAIL with `ModuleNotFoundError: No module named 'sdk.agent_session'`
- [ ] **Step 3: Write minimal transport implementation**
```python
from __future__ import annotations
from dataclasses import dataclass
from typing import AsyncIterator
import aiohttp
from sdk.interface import MessageChunk, MessageResponse, PlatformError
def build_thread_key(platform: str, user_id: str, chat_id: str) -> str:
return f"{platform}:{user_id}:{chat_id}"
@dataclass
class AgentSessionConfig:
base_ws_url: str
timeout_seconds: float = 30.0
class AgentSessionClient:
def __init__(self, config: AgentSessionConfig) -> None:
self._config = config
async def send_message(self, *, thread_key: str, text: str) -> MessageResponse:
chunks = []
tokens_used = 0
async for chunk in self.stream_message(thread_key=thread_key, text=text):
chunks.append(chunk.delta)
tokens_used = chunk.tokens_used or tokens_used
return MessageResponse(
message_id=thread_key,
response="".join(chunks),
tokens_used=tokens_used,
finished=True,
)
async def stream_message(self, *, thread_key: str, text: str) -> AsyncIterator[MessageChunk]:
url = f"{self._config.base_ws_url}?thread_id={thread_key}"
async with aiohttp.ClientSession() as session:
async with session.ws_connect(url, heartbeat=30) as ws:
status_msg = await ws.receive_json(timeout=self._config.timeout_seconds)
if status_msg.get("type") != "STATUS":
raise PlatformError("Agent did not send STATUS", code="AGENT_PROTOCOL_ERROR")
await ws.send_json({"type": "USER_MESSAGE", "text": text})
while True:
payload = await ws.receive_json(timeout=self._config.timeout_seconds)
msg_type = payload.get("type")
if msg_type == "AGENT_EVENT_TEXT_CHUNK":
yield MessageChunk(message_id=thread_key, delta=payload["text"], finished=False)
elif msg_type == "AGENT_EVENT_END":
yield MessageChunk(
message_id=thread_key,
delta="",
finished=True,
tokens_used=payload.get("tokens_used", 0),
)
return
elif msg_type == "ERROR":
raise PlatformError(payload.get("details", "Agent error"), code=payload.get("code", "AGENT_ERROR"))
else:
raise PlatformError(f"Unexpected agent message: {payload}", code="AGENT_PROTOCOL_ERROR")
```
- [ ] **Step 4: Run test to verify it passes**
Run: `pytest tests/platform/test_agent_session.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add sdk/agent_session.py tests/platform/test_agent_session.py
git commit -m "feat: add direct agent session transport"
```
---
### Task 2: Add Local Prototype State For Users And Settings
**Files:**
- Create: `sdk/prototype_state.py`
- Test: `tests/platform/test_prototype_state.py`
- [ ] **Step 1: Write the failing state tests**
```python
import pytest
from core.protocol import SettingsAction
from sdk.prototype_state import PrototypeStateStore
@pytest.mark.asyncio
async def test_get_or_create_user_is_stable_per_surface_identity():
...
@pytest.mark.asyncio
async def test_settings_defaults_match_existing_mock_shape():
...
@pytest.mark.asyncio
async def test_update_settings_supports_toggle_skill_and_setters():
...
```
- [ ] **Step 2: Run test to verify it fails**
Run: `pytest tests/platform/test_prototype_state.py -q`
Expected: FAIL with `ModuleNotFoundError: No module named 'sdk.prototype_state'`
- [ ] **Step 3: Write minimal state implementation**
```python
from __future__ import annotations
from datetime import UTC, datetime
from sdk.interface import User, UserSettings
# Defaults are defined here, not imported from sdk.mock, to keep real backend
# isolated from the mock. Copy-paste intentional.
DEFAULT_SKILLS: dict[str, bool] = {
"web-search": True,
"fetch-url": True,
"email": False,
"browser": False,
"image-gen": False,
"files": True,
}
DEFAULT_SAFETY: dict[str, bool] = {"email-send": True, "file-delete": True, "social-post": True}
DEFAULT_SOUL: dict[str, str] = {"name": "Лямбда", "instructions": ""}
DEFAULT_PLAN: dict = {"name": "Beta", "tokens_used": 0, "tokens_limit": 1000}
class PrototypeStateStore:
def __init__(self) -> None:
self._users: dict[str, User] = {}
self._settings: dict[str, dict] = {}
async def get_or_create_user(
self,
*,
external_id: str,
platform: str,
display_name: str | None = None,
) -> User:
key = f"{platform}:{external_id}"
existing = self._users.get(key)
if existing is not None:
return existing.model_copy(update={"is_new": False})
user = User(
user_id=f"usr-{platform}-{external_id}",
external_id=external_id,
platform=platform,
display_name=display_name,
created_at=datetime.now(UTC),
is_new=True,
)
self._users[key] = user.model_copy(update={"is_new": False})
return user
async def get_settings(self, user_id: str) -> UserSettings:
stored = self._settings.get(user_id, {})
return UserSettings(
skills={**DEFAULT_SKILLS, **stored.get("skills", {})},
connectors=stored.get("connectors", {}),
soul={**DEFAULT_SOUL, **stored.get("soul", {})},
safety={**DEFAULT_SAFETY, **stored.get("safety", {})},
plan={**DEFAULT_PLAN, **stored.get("plan", {})},
)
async def update_settings(self, user_id: str, action) -> None:
settings = self._settings.setdefault(user_id, {})
if action.action == "toggle_skill":
skills = settings.setdefault("skills", DEFAULT_SKILLS.copy())
skills[action.payload["skill"]] = action.payload.get("enabled", True)
elif action.action == "set_soul":
soul = settings.setdefault("soul", DEFAULT_SOUL.copy())
soul[action.payload["field"]] = action.payload["value"]
elif action.action == "set_safety":
safety = settings.setdefault("safety", DEFAULT_SAFETY.copy())
safety[action.payload["trigger"]] = action.payload.get("enabled", True)
```
- [ ] **Step 4: Run test to verify it passes**
Run: `pytest tests/platform/test_prototype_state.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add sdk/prototype_state.py tests/platform/test_prototype_state.py
git commit -m "feat: add prototype local state store"
```
---
### Task 3: Implement RealPlatformClient Compatibility Layer
**Files:**
- Create: `sdk/real.py`
- Modify: `sdk/__init__.py`
- Test: `tests/platform/test_real.py`
- Test: `tests/core/test_integration.py`
- [ ] **Step 1: Write the failing compatibility tests**
```python
import pytest
from core.protocol import SettingsAction
from sdk.real import RealPlatformClient
@pytest.mark.asyncio
async def test_real_platform_client_get_or_create_user_uses_local_state():
...
@pytest.mark.asyncio
async def test_real_platform_client_send_message_uses_thread_key():
...
@pytest.mark.asyncio
async def test_real_platform_client_settings_are_local():
...
```
- [ ] **Step 2: Run test to verify it fails**
Run: `pytest tests/platform/test_real.py -q`
Expected: FAIL with `ModuleNotFoundError: No module named 'sdk.real'`
- [ ] **Step 3: Write minimal compatibility wrapper**
```python
from __future__ import annotations
from typing import AsyncIterator
from sdk.agent_session import AgentSessionClient, build_thread_key
from sdk.interface import Attachment, MessageChunk, MessageResponse, PlatformClient, User, UserSettings
from sdk.prototype_state import PrototypeStateStore
class RealPlatformClient(PlatformClient):
def __init__(
self,
agent_sessions: AgentSessionClient,
prototype_state: PrototypeStateStore,
platform: str = "matrix",
) -> None:
self._agent_sessions = agent_sessions
self._prototype_state = prototype_state
self._platform = platform # surface name used in thread key; pass explicitly for future surfaces
async def get_or_create_user(
self,
external_id: str,
platform: str,
display_name: str | None = None,
) -> User:
return await self._prototype_state.get_or_create_user(
external_id=external_id,
platform=platform,
display_name=display_name,
)
async def send_message(
self,
user_id: str,
chat_id: str,
text: str,
attachments: list[Attachment] | None = None,
) -> MessageResponse:
# user_id here is the internal id (e.g. "usr-matrix-@alice:example.org"), which is
# unique per user and stable — acceptable as thread identity for v1 prototype.
thread_key = build_thread_key(self._platform, user_id, chat_id)
return await self._agent_sessions.send_message(thread_key=thread_key, text=text)
async def stream_message(
self,
user_id: str,
chat_id: str,
text: str,
attachments: list[Attachment] | None = None,
) -> AsyncIterator[MessageChunk]:
thread_key = build_thread_key(self._platform, user_id, chat_id)
async for chunk in self._agent_sessions.stream_message(thread_key=thread_key, text=text):
yield chunk
async def get_settings(self, user_id: str) -> UserSettings:
return await self._prototype_state.get_settings(user_id)
async def update_settings(self, user_id: str, action) -> None:
await self._prototype_state.update_settings(user_id, action)
```
- [ ] **Step 4: Run tests to verify the contract holds**
Run: `pytest tests/platform/test_real.py tests/core/test_integration.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add sdk/real.py sdk/__init__.py tests/platform/test_real.py tests/core/test_integration.py
git commit -m "feat: add real platform compatibility layer"
```
---
### Task 4: Wire Matrix Runtime To Real Backend And Document Usage
**Files:**
- Modify: `adapter/matrix/bot.py`
- Modify: `README.md`
- Modify: `tests/adapter/matrix/test_dispatcher.py`
- [ ] **Step 1: Write the failing runtime wiring tests**
```python
import os
from adapter.matrix.bot import build_runtime
from sdk.real import RealPlatformClient
def test_build_runtime_uses_real_platform_when_matrix_backend_is_real(monkeypatch):
monkeypatch.setenv("MATRIX_PLATFORM_BACKEND", "real")
monkeypatch.setenv("AGENT_WS_URL", "ws://agent.example/agent_ws/")
runtime = build_runtime()
assert isinstance(runtime.platform, RealPlatformClient)
```
- [ ] **Step 2: Run test to verify it fails**
Run: `pytest tests/adapter/matrix/test_dispatcher.py -q`
Expected: FAIL because runtime still always constructs `MockPlatformClient`
- [ ] **Step 3: Implement backend selection and docs**
```python
# adapter/matrix/bot.py — add these imports at the top
from sdk.agent_session import AgentSessionClient, AgentSessionConfig
from sdk.interface import PlatformClient
from sdk.prototype_state import PrototypeStateStore
from sdk.real import RealPlatformClient
def _build_platform_from_env() -> PlatformClient:
backend = os.environ.get("MATRIX_PLATFORM_BACKEND", "mock")
if backend == "real":
ws_url = os.environ["AGENT_WS_URL"]
return RealPlatformClient(
agent_sessions=AgentSessionClient(AgentSessionConfig(base_ws_url=ws_url)),
prototype_state=PrototypeStateStore(),
platform="matrix",
)
return MockPlatformClient()
# Update build_runtime to use env-based selection when no platform is injected:
def build_runtime(
platform: PlatformClient | None = None, # was MockPlatformClient | None
store: StateStore | None = None,
client: AsyncClient | None = None,
) -> MatrixRuntime:
platform = platform or _build_platform_from_env()
... # rest unchanged
```
Also update the `MatrixRuntime` dataclass field type from `MockPlatformClient` to `PlatformClient` so the type annotation matches the runtime behavior.
```markdown
# README.md
Matrix prototype backend selection:
- `MATRIX_PLATFORM_BACKEND=mock` uses `sdk/mock.py`
- `MATRIX_PLATFORM_BACKEND=real` uses direct agent WebSocket integration
- `AGENT_WS_URL=ws://host:port/agent_ws/` is required for the real backend
Current real-backend limitations:
- text chat only
- local settings storage
- no attachments or async task callbacks yet
```
- [ ] **Step 4: Run targeted verification**
Run: `pytest tests/adapter/matrix/test_dispatcher.py tests/platform/test_agent_session.py tests/platform/test_prototype_state.py tests/platform/test_real.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add adapter/matrix/bot.py README.md tests/adapter/matrix/test_dispatcher.py
git commit -m "feat: wire matrix runtime to real backend"
```
---
## Self-Review
- Spec coverage:
- direct-agent transport: Task 1
- local settings/user state: Task 2
- stable `PlatformClient` wrapper: Task 3
- Matrix runtime wiring and docs: Task 4
- Placeholder scan: no `TBD`, `TODO`, or “implement later” steps remain in the plan.
- Type consistency:
- `build_thread_key(platform, user_id, chat_id)` is used consistently.
- `RealPlatformClient` remains the only bot-facing implementation.
- local settings stay in `PrototypeStateStore`.
## Execution Handoff
Plan complete and saved to `docs/superpowers/plans/2026-04-08-matrix-direct-agent-prototype.md`. Two execution options:
**1. Subagent-Driven (recommended)** - I dispatch a fresh subagent per task, review between tasks, fast iteration
**2. Inline Execution** - Execute tasks in this session using executing-plans, batch execution with checkpoints
**Which approach?**

View file

@ -0,0 +1,480 @@
# Matrix Per-Chat Context Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Move the Matrix surface from a shared agent context to true per-room platform contexts mapped through `platform_chat_id`, including `!new`, `!branch`, lazy migration, and per-room context commands.
**Architecture:** Matrix keeps owning UX chats (`C1`, `C2`, rooms, spaces). Each working room stores a `platform_chat_id` in `room_meta`, and platform-facing operations use that mapping. Legacy rooms without a mapping lazily create one on first context-aware use.
**Tech Stack:** Python 3.11, Matrix nio adapter, local state store, `lambda_agent_api`, pytest
---
### Task 1: Add `platform_chat_id` to Matrix metadata and tests
**Files:**
- Modify: `adapter/matrix/store.py`
- Test: `tests/adapter/matrix/test_store.py`
- [ ] **Step 1: Write the failing test**
```python
async def test_room_meta_roundtrip_with_platform_chat_id(store: InMemoryStore):
meta = {
"chat_id": "C1",
"matrix_user_id": "@alice:example.org",
"platform_chat_id": "chat-platform-1",
}
await set_room_meta(store, "!r:m.org", meta)
saved = await get_room_meta(store, "!r:m.org")
assert saved is not None
assert saved["platform_chat_id"] == "chat-platform-1"
```
- [ ] **Step 2: Run test to verify it fails or proves missing coverage**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_store.py -q`
Expected: either FAIL on missing assertion coverage or PASS after adding the new test with current generic storage behavior
- [ ] **Step 3: Write minimal implementation**
```python
# adapter/matrix/store.py
# No schema gate is required because room metadata is already stored as a dict.
# Keep helpers unchanged, but add focused helper functions if they reduce repeated logic:
async def get_platform_chat_id(store: StateStore, room_id: str) -> str | None:
meta = await get_room_meta(store, room_id)
return meta.get("platform_chat_id") if meta else None
async def set_platform_chat_id(store: StateStore, room_id: str, platform_chat_id: str) -> None:
meta = await get_room_meta(store, room_id) or {}
meta["platform_chat_id"] = platform_chat_id
await set_room_meta(store, room_id, meta)
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_store.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add adapter/matrix/store.py tests/adapter/matrix/test_store.py
git commit -m "feat: add platform chat id room metadata helpers"
```
### Task 2: Extend the platform wrapper to support context-aware API calls
**Files:**
- Modify: `sdk/agent_api_wrapper.py`
- Modify: `sdk/real.py`
- Test: `tests/platform/test_real.py`
- [ ] **Step 1: Write the failing tests**
```python
@pytest.mark.asyncio
async def test_real_client_send_message_uses_platform_chat_id():
api = FakeAgentApi()
client = RealPlatformClient(agent_api=api, prototype_state=PrototypeStateStore())
await client.send_message("@alice:example.org", "chat-platform-1", "hello")
assert api.sent == [("chat-platform-1", "hello")]
@pytest.mark.asyncio
async def test_real_client_create_and_branch_context_delegate_to_agent_api():
api = FakeAgentApi(create_ids=["chat-new", "chat-branch"])
client = RealPlatformClient(agent_api=api, prototype_state=PrototypeStateStore())
created = await client.create_chat_context("@alice:example.org")
branched = await client.branch_chat_context("@alice:example.org", "chat-source")
assert created == "chat-new"
assert branched == "chat-branch"
assert api.branch_calls == ["chat-source"]
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `PYTHONPATH=. uv run pytest tests/platform/test_real.py -q`
Expected: FAIL because `RealPlatformClient` does not yet expose context-aware methods or pass a platform context id through
- [ ] **Step 3: Write minimal implementation**
```python
# sdk/agent_api_wrapper.py
class AgentApiWrapper(AgentApi):
async def create_chat(self) -> str:
...
async def branch_chat(self, chat_id: str) -> str:
...
async def send_message(self, chat_id: str, text: str):
...
async def save_context(self, chat_id: str, name: str) -> None:
...
async def load_context(self, chat_id: str, name: str) -> None:
...
# sdk/real.py
class RealPlatformClient(PlatformClient):
async def create_chat_context(self, user_id: str) -> str:
return await self._agent_api.create_chat()
async def branch_chat_context(self, user_id: str, from_chat_id: str) -> str:
return await self._agent_api.branch_chat(from_chat_id)
async def save_chat_context(self, user_id: str, chat_id: str, name: str) -> None:
await self._agent_api.save_context(chat_id, name)
async def load_chat_context(self, user_id: str, chat_id: str, name: str) -> None:
await self._agent_api.load_context(chat_id, name)
async def stream_message(...):
async for event in self._agent_api.send_message(chat_id, text):
...
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `PYTHONPATH=. uv run pytest tests/platform/test_real.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add sdk/agent_api_wrapper.py sdk/real.py tests/platform/test_real.py
git commit -m "feat: add context-aware real platform client methods"
```
### Task 3: Create Matrix-side resolver for mapped or lazy-created platform contexts
**Files:**
- Modify: `adapter/matrix/bot.py`
- Modify: `adapter/matrix/store.py`
- Test: `tests/adapter/matrix/test_dispatcher.py`
- [ ] **Step 1: Write the failing tests**
```python
async def test_existing_room_without_platform_chat_id_gets_lazy_mapping_on_message():
runtime = build_runtime(platform=FakeRealPlatformClient(create_ids=["chat-platform-1"]))
await set_room_meta(runtime.store, "!room:example.org", {
"chat_id": "C1",
"matrix_user_id": "@alice:example.org",
})
client = SimpleNamespace(user_id="@bot:example.org", room_send=AsyncMock())
bot = MatrixBot(client, runtime)
room = SimpleNamespace(room_id="!room:example.org")
event = SimpleNamespace(sender="@alice:example.org", body="hello")
await bot.on_room_message(room, event)
meta = await get_room_meta(runtime.store, "!room:example.org")
assert meta["platform_chat_id"] == "chat-platform-1"
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_dispatcher.py -q`
Expected: FAIL because no lazy mapping exists
- [ ] **Step 3: Write minimal implementation**
```python
# adapter/matrix/bot.py
async def _ensure_platform_chat_id(self, room_id: str, user_id: str) -> str:
meta = await get_room_meta(self.runtime.store, room_id)
if meta is None:
raise ValueError("room metadata is required")
platform_chat_id = meta.get("platform_chat_id")
if platform_chat_id:
return platform_chat_id
if not hasattr(self.runtime.platform, "create_chat_context"):
raise ValueError("real platform backend required")
platform_chat_id = await self.runtime.platform.create_chat_context(user_id)
meta["platform_chat_id"] = platform_chat_id
await set_room_meta(self.runtime.store, room_id, meta)
return platform_chat_id
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_dispatcher.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add adapter/matrix/bot.py adapter/matrix/store.py tests/adapter/matrix/test_dispatcher.py
git commit -m "feat: lazily assign platform chat ids to matrix rooms"
```
### Task 4: Make `!new` and workspace bootstrap create independent platform contexts
**Files:**
- Modify: `adapter/matrix/handlers/chat.py`
- Modify: `adapter/matrix/handlers/auth.py`
- Modify: `adapter/matrix/bot.py`
- Test: `tests/adapter/matrix/test_chat_space.py`
- Test: `tests/adapter/matrix/test_invite_space.py`
- Test: `tests/adapter/matrix/test_dispatcher.py`
- [ ] **Step 1: Write the failing tests**
```python
async def test_new_chat_assigns_new_platform_chat_id():
client = SimpleNamespace(
room_create=AsyncMock(return_value=SimpleNamespace(room_id="!r2:example")),
room_put_state=AsyncMock(),
room_invite=AsyncMock(),
)
platform = FakeRealPlatformClient(create_ids=["chat-platform-7"])
runtime = build_runtime(platform=platform, client=client)
await set_user_meta(runtime.store, "u1", {"space_id": "!space:example", "next_chat_index": 7})
result = await runtime.dispatcher.dispatch(
IncomingCommand(user_id="u1", platform="matrix", chat_id="C3", command="new", args=["Research"])
)
meta = await get_room_meta(runtime.store, "!r2:example")
assert meta["platform_chat_id"] == "chat-platform-7"
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_chat_space.py tests/adapter/matrix/test_invite_space.py tests/adapter/matrix/test_dispatcher.py -q`
Expected: FAIL because new chats do not yet store a platform context id
- [ ] **Step 3: Write minimal implementation**
```python
# adapter/matrix/handlers/chat.py
# adapter/matrix/handlers/auth.py
platform_chat_id = None
if hasattr(platform, "create_chat_context"):
platform_chat_id = await platform.create_chat_context(event.user_id)
await set_room_meta(store, room_id, {
"chat_id": chat_id,
"matrix_user_id": event.user_id,
"space_id": space_id,
"platform_chat_id": platform_chat_id,
})
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_chat_space.py tests/adapter/matrix/test_invite_space.py tests/adapter/matrix/test_dispatcher.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add adapter/matrix/handlers/chat.py adapter/matrix/handlers/auth.py adapter/matrix/bot.py tests/adapter/matrix/test_chat_space.py tests/adapter/matrix/test_invite_space.py tests/adapter/matrix/test_dispatcher.py
git commit -m "feat: assign platform contexts when creating matrix chats"
```
### Task 5: Make per-room save, load, and context use the mapped platform context
**Files:**
- Modify: `adapter/matrix/handlers/context_commands.py`
- Modify: `adapter/matrix/bot.py`
- Modify: `sdk/prototype_state.py`
- Test: `tests/adapter/matrix/test_context_commands.py`
- [ ] **Step 1: Write the failing tests**
```python
@pytest.mark.asyncio
async def test_save_command_uses_room_platform_chat_id():
platform = MatrixCommandPlatform()
runtime = build_runtime(platform=platform)
await set_room_meta(runtime.store, "!room:example.org", {
"chat_id": "C1",
"matrix_user_id": "u1",
"platform_chat_id": "chat-platform-1",
})
event = IncomingCommand(user_id="u1", platform="matrix", chat_id="C1", command="save", args=["session-a"])
result = await make_handle_save(...)(event, runtime.auth_mgr, platform, runtime.chat_mgr, runtime.settings_mgr)
assert platform.saved_calls == [("chat-platform-1", "session-a")]
@pytest.mark.asyncio
async def test_context_command_reports_current_room_platform_chat_id():
...
assert "chat-platform-1" in result[0].text
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_context_commands.py -q`
Expected: FAIL because save/load/context do not currently use room-level platform mappings
- [ ] **Step 3: Write minimal implementation**
```python
# adapter/matrix/handlers/context_commands.py
room_id = await _resolve_room_id(event, chat_mgr)
meta = await get_room_meta(store, room_id)
platform_chat_id = meta.get("platform_chat_id")
await platform.save_chat_context(event.user_id, platform_chat_id, name)
await platform.load_chat_context(event.user_id, platform_chat_id, name)
# sdk/prototype_state.py
# store current loaded session per user+platform_chat_id instead of only per user when needed for Matrix `!context`
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_context_commands.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add adapter/matrix/handlers/context_commands.py adapter/matrix/bot.py sdk/prototype_state.py tests/adapter/matrix/test_context_commands.py
git commit -m "feat: bind matrix context commands to platform chat ids"
```
### Task 6: Add `!branch` and help-text updates
**Files:**
- Modify: `adapter/matrix/handlers/chat.py`
- Modify: `adapter/matrix/handlers/__init__.py`
- Modify: `adapter/matrix/handlers/settings.py`
- Modify: `adapter/matrix/handlers/auth.py`
- Modify: `adapter/matrix/bot.py`
- Test: `tests/adapter/matrix/test_chat_space.py`
- Test: `tests/adapter/matrix/test_dispatcher.py`
- [ ] **Step 1: Write the failing tests**
```python
async def test_branch_creates_new_room_with_branched_platform_chat_id():
client = SimpleNamespace(
room_create=AsyncMock(return_value=SimpleNamespace(room_id="!r3:example")),
room_put_state=AsyncMock(),
room_invite=AsyncMock(),
)
platform = FakeRealPlatformClient(branch_ids=["chat-platform-branch"])
runtime = build_runtime(platform=platform, client=client)
await set_room_meta(runtime.store, "!current:example", {
"chat_id": "C2",
"matrix_user_id": "u1",
"space_id": "!space:example",
"platform_chat_id": "chat-platform-source",
})
result = await runtime.dispatcher.dispatch(
IncomingCommand(user_id="u1", platform="matrix", chat_id="C2", command="branch", args=["Fork"])
)
meta = await get_room_meta(runtime.store, "!r3:example")
assert meta["platform_chat_id"] == "chat-platform-branch"
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_chat_space.py tests/adapter/matrix/test_dispatcher.py -q`
Expected: FAIL because `branch` is not implemented
- [ ] **Step 3: Write minimal implementation**
```python
# adapter/matrix/handlers/chat.py
def make_handle_branch(client, store):
async def handle_branch(event, auth_mgr, platform, chat_mgr, settings_mgr):
source_room_id = ...
source_meta = await get_room_meta(store, source_room_id)
platform_chat_id = await platform.branch_chat_context(event.user_id, source_meta["platform_chat_id"])
...
await set_room_meta(store, new_room_id, {
"chat_id": new_chat_id,
"matrix_user_id": event.user_id,
"space_id": space_id,
"platform_chat_id": platform_chat_id,
})
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_chat_space.py tests/adapter/matrix/test_dispatcher.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add adapter/matrix/handlers/chat.py adapter/matrix/handlers/__init__.py adapter/matrix/handlers/settings.py adapter/matrix/handlers/auth.py adapter/matrix/bot.py tests/adapter/matrix/test_chat_space.py tests/adapter/matrix/test_dispatcher.py
git commit -m "feat: add matrix branch command for platform contexts"
```
### Task 7: Verify the full Matrix flow and clean up legacy assumptions
**Files:**
- Modify: `tests/platform/test_real.py`
- Modify: `tests/adapter/matrix/test_dispatcher.py`
- Modify: `tests/adapter/matrix/test_context_commands.py`
- Modify: `tests/core/test_integration.py`
- [ ] **Step 1: Add integration coverage for independent room contexts**
```python
@pytest.mark.asyncio
async def test_two_rooms_send_messages_into_different_platform_contexts():
platform = FakeRealPlatformClient()
runtime = build_runtime(platform=platform)
await set_room_meta(runtime.store, "!r1:example", {"chat_id": "C1", "matrix_user_id": "u1", "platform_chat_id": "chat-1"})
await set_room_meta(runtime.store, "!r2:example", {"chat_id": "C2", "matrix_user_id": "u1", "platform_chat_id": "chat-2"})
...
assert platform.sent == [("chat-1", "hello"), ("chat-2", "world")]
```
- [ ] **Step 2: Run the focused verification suite**
Run: `PYTHONPATH=. uv run pytest tests/platform/test_real.py tests/adapter/matrix/test_store.py tests/adapter/matrix/test_chat_space.py tests/adapter/matrix/test_invite_space.py tests/adapter/matrix/test_dispatcher.py tests/adapter/matrix/test_context_commands.py tests/core/test_integration.py -q`
Expected: PASS
- [ ] **Step 3: Run the full Matrix suite**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix -q`
Expected: PASS
- [ ] **Step 4: Inspect help text and command visibility**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_dispatcher.py -q`
Expected: PASS with `!branch` present in help and hidden commands still absent
- [ ] **Step 5: Commit**
```bash
git add tests/platform/test_real.py tests/adapter/matrix/test_store.py tests/adapter/matrix/test_chat_space.py tests/adapter/matrix/test_invite_space.py tests/adapter/matrix/test_dispatcher.py tests/adapter/matrix/test_context_commands.py tests/core/test_integration.py
git commit -m "test: verify matrix per-chat platform context flow"
```
## Self-Review
- Spec coverage:
- `surface_chat -> platform_chat_id` mapping is covered by Tasks 1, 3, and 4.
- `!new` independent contexts are covered by Task 4.
- `!branch` snapshot flow is covered by Task 6.
- per-room `!save`, `!load`, and `!context` are covered by Task 5.
- lazy migration for legacy rooms is covered by Task 3.
- verification across rooms is covered by Task 7.
- Placeholder scan:
- No `TODO` or `TBD` placeholders remain.
- Commands and file paths are concrete.
- Type consistency:
- The plan consistently uses `platform_chat_id` for stored mapping and `create_chat_context` / `branch_chat_context` / `save_chat_context` / `load_chat_context` for platform-facing methods.

View file

@ -0,0 +1,624 @@
# Matrix Shared Workspace File Flow Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Move the Matrix surface to a prod-like shared-workspace file flow where incoming files are downloaded into `/workspace`, passed to `platform-agent` as relative attachment paths, and outbound `send_file` events are delivered back to the Matrix room.
**Architecture:** Keep the platform contract path-based and surface-agnostic. Extend the surface attachment model with a workspace-relative path, update the real SDK bridge to forward attachments and modern agent events, then add a Matrix-specific storage helper plus a shared-volume runtime. This preserves room/context semantics while aligning file flow with upstream `platform-agent`.
**Tech Stack:** Python 3.11, matrix-nio, aiohttp, Docker Compose, pytest, pytest-asyncio
---
## File Structure
- Modify: `core/protocol.py`
Purpose: add a workspace-relative attachment field that future surfaces can also use.
- Modify: `sdk/interface.py`
Purpose: keep the platform-side attachment shape aligned with the surface model.
- Modify: `core/handlers/message.py`
Purpose: stop dropping attachments before platform dispatch.
- Modify: `sdk/agent_api_wrapper.py`
Purpose: accept modern upstream agent events and modern WS route semantics.
- Modify: `sdk/real.py`
Purpose: convert attachment objects into workspace-relative paths and forward them to the agent API.
- Create: `adapter/matrix/files.py`
Purpose: Matrix-specific download/upload helper for shared `/workspace`.
- Modify: `adapter/matrix/bot.py`
Purpose: persist incoming Matrix files into shared workspace and render outbound file events back to Matrix.
- Modify: `tests/core/test_integration.py`
Purpose: prove message dispatch keeps attachments and platform send path receives them.
- Modify: `tests/platform/test_real.py`
Purpose: verify attachment forwarding and outbound file events.
- Create: `tests/adapter/matrix/test_files.py`
Purpose: unit coverage for Matrix workspace path building, download handling, and outbound upload behavior.
- Modify: `tests/adapter/matrix/test_dispatcher.py`
Purpose: verify Matrix bot file receive/send integration.
- Modify: `docker-compose.yml`
Purpose: define shared `/workspace` runtime between `matrix-bot` and `platform-agent`.
- Modify: `README.md`
Purpose: document the new default runtime and file flow.
- Modify: `.env.example`
Purpose: update real-backend defaults to in-compose service URLs and workspace-aware runtime.
### Task 1: Preserve Attachment Metadata Through Core Message Dispatch
**Files:**
- Modify: `core/protocol.py`
- Modify: `sdk/interface.py`
- Modify: `core/handlers/message.py`
- Test: `tests/core/test_dispatcher.py`
- Test: `tests/core/test_integration.py`
- [ ] **Step 1: Write the failing tests**
```python
# tests/core/test_integration.py
class RecordingAgentApi:
def __init__(self) -> None:
self.calls: list[tuple[str, list[str]]] = []
self.last_tokens_used = 0
async def send_message(self, text: str, attachments: list[str] | None = None):
self.calls.append((text, attachments or []))
yield type("Chunk", (), {"text": f"[REAL] {text}"})()
self.last_tokens_used = 5
async def test_full_flow_with_real_platform_forwards_workspace_attachment(real_dispatcher):
dispatcher, agent_api = real_dispatcher
start = IncomingCommand(user_id="u1", platform="matrix", chat_id="C1", command="start")
await dispatcher.dispatch(start)
msg = IncomingMessage(
user_id="u1",
platform="matrix",
chat_id="C1",
text="Посмотри файл",
attachments=[
Attachment(
type="document",
filename="report.pdf",
mime_type="application/pdf",
workspace_path="surfaces/matrix/u1/room/inbox/report.pdf",
)
],
)
await dispatcher.dispatch(msg)
assert agent_api.calls == [
("Посмотри файл", ["surfaces/matrix/u1/room/inbox/report.pdf"])
]
```
```python
# tests/core/test_dispatcher.py
async def test_dispatch_routes_document_before_catchall(dispatcher):
async def doc_handler(event, **kwargs):
return [OutgoingMessage(chat_id=event.chat_id, text="document")]
async def catch_all(event, **kwargs):
return [OutgoingMessage(chat_id=event.chat_id, text="text")]
dispatcher.register(IncomingMessage, "document", doc_handler)
dispatcher.register(IncomingMessage, "*", catch_all)
doc_msg = IncomingMessage(
user_id="u1",
platform="matrix",
chat_id="C1",
text="",
attachments=[Attachment(type="document", workspace_path="surfaces/matrix/u1/file.txt")],
)
assert (await dispatcher.dispatch(doc_msg))[0].text == "document"
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `PYTHONPATH=. uv run pytest tests/core/test_dispatcher.py tests/core/test_integration.py -q`
Expected:
- FAIL because `Attachment` has no `workspace_path`
- FAIL because `handle_message(...)` still sends `attachments=[]`
- [ ] **Step 3: Write minimal implementation**
```python
# core/protocol.py
@dataclass
class Attachment:
type: str
url: str | None = None
content: bytes | None = None
filename: str | None = None
mime_type: str | None = None
workspace_path: str | None = None
```
```python
# sdk/interface.py
class Attachment(BaseModel):
url: str | None = None
mime_type: str | None = None
size: int | None = None
filename: str | None = None
workspace_path: str | None = None
```
```python
# core/handlers/message.py
response = await platform.send_message(
user_id=event.user_id,
chat_id=event.chat_id,
text=event.text,
attachments=event.attachments,
)
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `PYTHONPATH=. uv run pytest tests/core/test_dispatcher.py tests/core/test_integration.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add core/protocol.py sdk/interface.py core/handlers/message.py tests/core/test_dispatcher.py tests/core/test_integration.py
git commit -m "feat: preserve workspace attachments through message dispatch"
```
### Task 2: Upgrade the Real SDK Bridge to Modern Agent File Events
**Files:**
- Modify: `sdk/agent_api_wrapper.py`
- Modify: `sdk/real.py`
- Test: `tests/platform/test_real.py`
- [ ] **Step 1: Write the failing tests**
```python
# tests/platform/test_real.py
class FakeSendFileEvent:
def __init__(self, path: str) -> None:
self.path = path
class FakeChatAgentApi:
...
async def send_message(self, text: str, attachments: list[str] | None = None):
self.calls.append((text, attachments or []))
midpoint = len(text) // 2
yield FakeChunk(text[:midpoint])
yield FakeChunk(text[midpoint:])
self.last_tokens_used = 3
@pytest.mark.asyncio
async def test_real_platform_client_send_message_forwards_workspace_paths():
agent_api = FakeAgentApiFactory()
client = RealPlatformClient(
agent_api=agent_api,
prototype_state=PrototypeStateStore(),
platform="matrix",
)
await client.send_message(
"@alice:example.org",
"chat-7",
"hello",
attachments=[
type("Attachment", (), {"workspace_path": "surfaces/matrix/alice/room/file.pdf"})()
],
)
assert agent_api.instances["chat-7"].calls == [
("hello", ["surfaces/matrix/alice/room/file.pdf"])
]
def test_agent_api_wrapper_treats_send_file_as_known_event(monkeypatch):
seen = []
class FakeSendFile:
type = "AGENT_EVENT_SEND_FILE"
path = "docs/result.pdf"
monkeypatch.setattr(
"sdk.agent_api_wrapper.ServerMessage.validate_json",
lambda raw: FakeSendFile(),
)
wrapper = AgentApiWrapper(agent_id="agent-1", base_url="https://agent.example.com", chat_id="7")
wrapper.callback = seen.append
wrapper._current_queue = None
# use the wrapper's dispatch branch directly inside _listen test harness
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `PYTHONPATH=. uv run pytest tests/platform/test_real.py -q`
Expected:
- FAIL because `RealPlatformClient` ignores attachments
- FAIL because `AgentApiWrapper` only recognizes text/end/error/disconnect events
- [ ] **Step 3: Write minimal implementation**
```python
# sdk/real.py
def _attachment_paths(self, attachments) -> list[str]:
if not attachments:
return []
paths = []
for attachment in attachments:
path = getattr(attachment, "workspace_path", None)
if path:
paths.append(path)
return paths
async def stream_message(...):
attachment_paths = self._attachment_paths(attachments)
...
async for event in chat_api.send_message(text, attachments=attachment_paths):
if hasattr(event, "path"):
yield MessageChunk(
message_id=user_id,
delta="",
finished=False,
)
continue
yield MessageChunk(...)
```
```python
# sdk/agent_api_wrapper.py
from lambda_agent_api.server import (
MsgError,
MsgEventCustomUpdate,
MsgEventEnd,
MsgEventSendFile,
MsgEventTextChunk,
MsgEventToolCallChunk,
MsgEventToolResult,
MsgGracefulDisconnect,
ServerMessage,
)
KNOWN_STREAM_EVENTS = (
MsgEventTextChunk,
MsgEventToolCallChunk,
MsgEventToolResult,
MsgEventCustomUpdate,
MsgEventSendFile,
MsgEventEnd,
)
if isinstance(outgoing_msg, KNOWN_STREAM_EVENTS):
if isinstance(outgoing_msg, MsgEventEnd):
self.last_tokens_used = outgoing_msg.tokens_used
if self._current_queue:
await self._current_queue.put(outgoing_msg)
elif self.callback:
self.callback(outgoing_msg)
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `PYTHONPATH=. uv run pytest tests/platform/test_real.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add sdk/agent_api_wrapper.py sdk/real.py tests/platform/test_real.py
git commit -m "feat: support attachment paths and file events in real sdk bridge"
```
### Task 3: Implement Matrix Shared-Workspace Receive/Send File Flow
**Files:**
- Create: `adapter/matrix/files.py`
- Modify: `adapter/matrix/bot.py`
- Test: `tests/adapter/matrix/test_files.py`
- Test: `tests/adapter/matrix/test_dispatcher.py`
- [ ] **Step 1: Write the failing tests**
```python
# tests/adapter/matrix/test_files.py
from pathlib import Path
import pytest
from adapter.matrix.files import build_workspace_attachment_path
def test_build_workspace_attachment_path_scopes_by_surface_user_and_room(tmp_path):
rel_path, abs_path = build_workspace_attachment_path(
workspace_root=tmp_path,
matrix_user_id="@alice:example.org",
room_id="!room:example.org",
filename="report.pdf",
timestamp="20260420-153000",
)
assert rel_path == "surfaces/matrix/alice_example.org/room_example.org/inbox/20260420-153000-report.pdf"
assert abs_path == tmp_path / rel_path
```
```python
# tests/adapter/matrix/test_dispatcher.py
async def test_bot_downloads_matrix_file_to_workspace_before_dispatch(tmp_path):
runtime = build_runtime(platform=MockPlatformClient())
await set_room_meta(
runtime.store,
"!chat1:example.org",
{
"chat_id": "C1",
"matrix_user_id": "@alice:example.org",
"platform_chat_id": "matrix:ctx-1",
},
)
client = SimpleNamespace(
user_id="@bot:example.org",
download=AsyncMock(return_value=SimpleNamespace(body=b"%PDF-1.7")),
)
bot = MatrixBot(client, runtime)
bot._send_all = AsyncMock()
runtime.dispatcher.dispatch = AsyncMock(return_value=[])
room = SimpleNamespace(room_id="!chat1:example.org")
event = SimpleNamespace(
sender="@alice:example.org",
body="Посмотри",
msgtype="m.file",
url="mxc://server/id",
mimetype="application/pdf",
replyto_event_id=None,
)
await bot.on_room_message(room, event)
dispatched = runtime.dispatcher.dispatch.await_args.args[0]
assert dispatched.attachments[0].workspace_path.endswith(".pdf")
```
```python
# tests/adapter/matrix/test_dispatcher.py
async def test_send_outgoing_uploads_file_attachment_to_matrix(tmp_path):
path = tmp_path / "result.txt"
path.write_text("ready")
client = SimpleNamespace(
upload=AsyncMock(return_value=(SimpleNamespace(content_uri="mxc://server/file"), {})),
room_send=AsyncMock(),
)
await send_outgoing(
client,
"!room:example.org",
OutgoingMessage(
chat_id="!room:example.org",
text="Файл готов",
attachments=[
Attachment(
type="document",
filename="result.txt",
mime_type="text/plain",
workspace_path=str(path),
)
],
),
)
client.upload.assert_awaited()
client.room_send.assert_awaited()
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_files.py tests/adapter/matrix/test_dispatcher.py -q`
Expected:
- FAIL because `adapter.matrix.files` does not exist
- FAIL because Matrix bot does not persist files before dispatch
- FAIL because `send_outgoing(...)` only sends text
- [ ] **Step 3: Write minimal implementation**
```python
# adapter/matrix/files.py
from __future__ import annotations
from pathlib import Path
from datetime import UTC, datetime
import re
from core.protocol import Attachment
def _sanitize_component(value: str) -> str:
stripped = re.sub(r"[^A-Za-z0-9._-]+", "_", value)
return stripped.strip("._-") or "unknown"
def build_workspace_attachment_path(
*,
workspace_root: Path,
matrix_user_id: str,
room_id: str,
filename: str,
timestamp: str | None = None,
) -> tuple[str, Path]:
stamp = timestamp or datetime.now(UTC).strftime("%Y%m%d-%H%M%S")
safe_user = _sanitize_component(matrix_user_id.lstrip("@"))
safe_room = _sanitize_component(room_id.lstrip("!"))
safe_name = _sanitize_component(filename) or "attachment.bin"
rel_path = Path("surfaces") / "matrix" / safe_user / safe_room / "inbox" / f"{stamp}-{safe_name}"
return rel_path.as_posix(), workspace_root / rel_path
```
```python
# adapter/matrix/bot.py
from adapter.matrix.files import download_matrix_attachments, load_workspace_attachment
...
incoming = from_room_event(event, room_id=room.room_id, chat_id=dispatch_chat_id)
if isinstance(incoming, IncomingMessage) and incoming.attachments:
incoming = await self._materialize_attachments(room.room_id, sender, incoming)
...
async def _materialize_attachments(...):
workspace_root = Path(os.environ.get("SURFACES_WORKSPACE_DIR", "/workspace"))
attachments = await download_matrix_attachments(...)
return IncomingMessage(..., attachments=attachments, ...)
```
```python
# adapter/matrix/bot.py
if isinstance(event, OutgoingMessage) and event.attachments:
for attachment in event.attachments:
if attachment.workspace_path:
await _send_matrix_file(client, room_id, attachment)
if event.text:
await client.room_send(...)
return
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_files.py tests/adapter/matrix/test_dispatcher.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add adapter/matrix/files.py adapter/matrix/bot.py tests/adapter/matrix/test_files.py tests/adapter/matrix/test_dispatcher.py
git commit -m "feat: add matrix shared-workspace file receive and send flow"
```
### Task 4: Make Shared Workspace the Default Local Runtime
**Files:**
- Modify: `docker-compose.yml`
- Modify: `README.md`
- Modify: `.env.example`
- [ ] **Step 1: Write the failing configuration checks**
```bash
python - <<'PY'
from pathlib import Path
text = Path("docker-compose.yml").read_text()
assert "platform-agent" in text
assert "/workspace" in text
assert "matrix-bot" in text
PY
```
```bash
python - <<'PY'
from pathlib import Path
readme = Path("README.md").read_text()
assert "docker compose up" in readme
assert "/workspace" in readme
assert "platform-agent" in readme
PY
```
- [ ] **Step 2: Run checks to verify they fail**
Run: `python - <<'PY' ... PY`
Expected:
- FAIL because root compose only defines `matrix-bot`
- FAIL because README still documents standalone `uvicorn` launch and old WS route
- [ ] **Step 3: Write minimal implementation**
```yaml
# docker-compose.yml
services:
platform-agent:
build:
context: ./external/platform-agent
target: development
additional_contexts:
agent_api: ./external/platform-agent_api
env_file:
- ./external/platform-agent/.env
volumes:
- workspace:/workspace
- ./external/platform-agent/src:/app/src
- ./external/platform-agent_api:/agent_api
ports:
- "8000:8000"
matrix-bot:
build: .
env_file: .env
depends_on:
- platform-agent
volumes:
- workspace:/workspace
restart: unless-stopped
volumes:
workspace:
```
```env
# .env.example
AGENT_WS_URL=ws://platform-agent:8000/v1/agent_ws/0/
AGENT_BASE_URL=http://platform-agent:8000
SURFACES_WORKSPACE_DIR=/workspace
MATRIX_PLATFORM_BACKEND=real
```
```md
# README.md
- make the root `docker compose up` path the primary local runtime
- describe shared `/workspace` as the file contract
- remove the statement that real backend is text-only and has no attachments
- replace the old standalone `uvicorn` instructions with compose-first instructions
```
- [ ] **Step 4: Run checks to verify they pass**
Run: `python - <<'PY' ... PY`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add docker-compose.yml README.md .env.example
git commit -m "chore: make shared workspace runtime the default local setup"
```
## Self-Review
- Spec coverage:
- shared `/workspace` runtime: Task 4
- incoming Matrix file persistence: Task 3
- attachment path propagation to agent API: Tasks 1-2
- outbound `send_file` flow: Tasks 2-3
- future-surface-friendly attachment contract: Task 1
- Placeholder scan:
- no `TODO`, `TBD`, or “similar to”
- each task has explicit test, run, implementation, verify, commit steps
- Type consistency:
- `workspace_path` is introduced in both attachment models and consumed consistently in Tasks 1-3
- path-based contract is always relative to `/workspace` until Matrix upload resolution step
## Execution Handoff
User already selected parallel subagent execution. Use subagent-driven development and split ownership like this:
- Worker A: `docker-compose.yml`, `README.md`, `.env.example`
- Worker B: `sdk/agent_api_wrapper.py`, `sdk/real.py`, `tests/platform/test_real.py`
- Main session / later worker: `core/protocol.py`, `sdk/interface.py`, `core/handlers/message.py`, `adapter/matrix/files.py`, `adapter/matrix/bot.py`, matrix/core tests

View file

@ -0,0 +1,555 @@
# Matrix Staged Attachments Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Add Matrix-side staged attachments so file-only events are stored per `(chat_id, user_id)`, acknowledged in chat, and committed to the agent on the next normal user message.
**Architecture:** Keep the existing shared-workspace file contract unchanged and add a thin Matrix-specific staging layer above it. The Matrix adapter will store staged attachment metadata in `adapter.matrix.store`, parse short staging commands in `adapter.matrix.converter`, and orchestrate commit/remove/list behavior in `adapter.matrix.bot` before calling the existing dispatcher.
**Tech Stack:** Python 3.11, matrix-nio, pytest, pytest-asyncio, in-memory state store, shared `/workspace`
---
## File Structure
- Modify: `adapter/matrix/store.py`
Purpose: store staged attachment state per `(room_id, user_id)`.
- Modify: `adapter/matrix/converter.py`
Purpose: parse `!list`, `!remove <n>`, `!remove all` into explicit Matrix-side commands.
- Modify: `adapter/matrix/bot.py`
Purpose: stage file-only events, emit service acknowledgments, process staging commands, and commit staged files with the next normal message.
- Modify: `tests/adapter/matrix/test_store.py`
Purpose: verify staged attachment persistence, ordering, and clear/remove helpers.
- Modify: `tests/adapter/matrix/test_converter.py`
Purpose: verify short staging commands parse correctly.
- Modify: `tests/adapter/matrix/test_dispatcher.py`
Purpose: verify Matrix bot staging, list/remove behavior, and commit semantics.
- Modify: `README.md`
Purpose: document the Matrix staging UX and short commands.
### Task 1: Add Per-Chat Staged Attachment Storage
**Files:**
- Modify: `adapter/matrix/store.py`
- Test: `tests/adapter/matrix/test_store.py`
- [ ] **Step 1: Write the failing tests**
```python
# tests/adapter/matrix/test_store.py
from adapter.matrix.store import (
add_staged_attachment,
clear_staged_attachments,
get_staged_attachments,
remove_staged_attachment_at,
)
async def test_staged_attachments_roundtrip(store: InMemoryStore):
await add_staged_attachment(
store,
room_id="!r1:example.org",
user_id="@alice:example.org",
attachment={
"filename": "report.pdf",
"workspace_path": "surfaces/matrix/alice/r1/inbox/report.pdf",
"mime_type": "application/pdf",
},
)
assert await get_staged_attachments(store, "!r1:example.org", "@alice:example.org") == [
{
"filename": "report.pdf",
"workspace_path": "surfaces/matrix/alice/r1/inbox/report.pdf",
"mime_type": "application/pdf",
}
]
async def test_staged_attachments_are_scoped_by_room_and_user(store: InMemoryStore):
await add_staged_attachment(
store,
room_id="!r1:example.org",
user_id="@alice:example.org",
attachment={"filename": "a.pdf", "workspace_path": "a.pdf"},
)
await add_staged_attachment(
store,
room_id="!r2:example.org",
user_id="@alice:example.org",
attachment={"filename": "b.pdf", "workspace_path": "b.pdf"},
)
await add_staged_attachment(
store,
room_id="!r1:example.org",
user_id="@bob:example.org",
attachment={"filename": "c.pdf", "workspace_path": "c.pdf"},
)
assert [item["filename"] for item in await get_staged_attachments(store, "!r1:example.org", "@alice:example.org")] == ["a.pdf"]
assert [item["filename"] for item in await get_staged_attachments(store, "!r2:example.org", "@alice:example.org")] == ["b.pdf"]
assert [item["filename"] for item in await get_staged_attachments(store, "!r1:example.org", "@bob:example.org")] == ["c.pdf"]
async def test_remove_staged_attachment_by_index(store: InMemoryStore):
await add_staged_attachment(store, "!r1:example.org", "@alice:example.org", {"filename": "a.pdf", "workspace_path": "a.pdf"})
await add_staged_attachment(store, "!r1:example.org", "@alice:example.org", {"filename": "b.pdf", "workspace_path": "b.pdf"})
removed = await remove_staged_attachment_at(store, "!r1:example.org", "@alice:example.org", 1)
assert removed["filename"] == "b.pdf"
assert [item["filename"] for item in await get_staged_attachments(store, "!r1:example.org", "@alice:example.org")] == ["a.pdf"]
async def test_clear_staged_attachments(store: InMemoryStore):
await add_staged_attachment(store, "!r1:example.org", "@alice:example.org", {"filename": "a.pdf", "workspace_path": "a.pdf"})
await clear_staged_attachments(store, "!r1:example.org", "@alice:example.org")
assert await get_staged_attachments(store, "!r1:example.org", "@alice:example.org") == []
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_store.py -q`
Expected:
- FAIL because staged attachment helper functions do not exist yet
- [ ] **Step 3: Write minimal implementation**
```python
# adapter/matrix/store.py
STAGED_ATTACHMENTS_PREFIX = "matrix_staged_attachments:"
def _staged_attachments_key(room_id: str, user_id: str) -> str:
return f"{STAGED_ATTACHMENTS_PREFIX}{room_id}:{user_id}"
async def get_staged_attachments(store: StateStore, room_id: str, user_id: str) -> list[dict]:
return list(await store.get(_staged_attachments_key(room_id, user_id)) or [])
async def add_staged_attachment(
store: StateStore,
room_id: str,
user_id: str,
attachment: dict,
) -> None:
items = await get_staged_attachments(store, room_id, user_id)
items.append(attachment)
await store.set(_staged_attachments_key(room_id, user_id), items)
async def remove_staged_attachment_at(
store: StateStore,
room_id: str,
user_id: str,
index: int,
) -> dict | None:
items = await get_staged_attachments(store, room_id, user_id)
if index < 0 or index >= len(items):
return None
removed = items.pop(index)
await store.set(_staged_attachments_key(room_id, user_id), items)
return removed
async def clear_staged_attachments(store: StateStore, room_id: str, user_id: str) -> None:
await store.delete(_staged_attachments_key(room_id, user_id))
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_store.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add adapter/matrix/store.py tests/adapter/matrix/test_store.py
git commit -m "feat: add matrix staged attachment state"
```
### Task 2: Parse Short Staging Commands
**Files:**
- Modify: `adapter/matrix/converter.py`
- Test: `tests/adapter/matrix/test_converter.py`
- [ ] **Step 1: Write the failing tests**
```python
# tests/adapter/matrix/test_converter.py
async def test_list_command_maps_to_matrix_staging_command():
result = from_room_event(text_event("!list"), room_id="!r:m.org", chat_id="C1")
assert isinstance(result, IncomingCommand)
assert result.command == "matrix_list_attachments"
assert result.args == []
async def test_remove_all_maps_to_matrix_staging_command():
result = from_room_event(text_event("!remove all"), room_id="!r:m.org", chat_id="C1")
assert isinstance(result, IncomingCommand)
assert result.command == "matrix_remove_attachment"
assert result.args == ["all"]
async def test_remove_index_maps_to_matrix_staging_command():
result = from_room_event(text_event("!remove 2"), room_id="!r:m.org", chat_id="C1")
assert isinstance(result, IncomingCommand)
assert result.command == "matrix_remove_attachment"
assert result.args == ["2"]
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_converter.py -q`
Expected:
- FAIL because `!list` and `!remove` still parse as generic unknown commands
- [ ] **Step 3: Write minimal implementation**
```python
# adapter/matrix/converter.py
def from_command(body: str, sender: str, chat_id: str, room_id: str | None = None) -> IncomingEvent:
raw = body.lstrip("!").strip()
parts = raw.split()
command = parts[0].lower() if parts else ""
args = parts[1:]
if command == "list":
return IncomingCommand(
user_id=sender,
platform=PLATFORM,
chat_id=chat_id,
command="matrix_list_attachments",
args=[],
)
if command == "remove":
return IncomingCommand(
user_id=sender,
platform=PLATFORM,
chat_id=chat_id,
command="matrix_remove_attachment",
args=args,
)
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_converter.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add adapter/matrix/converter.py tests/adapter/matrix/test_converter.py
git commit -m "feat: parse matrix staged attachment commands"
```
### Task 3: Stage File-Only Events and Handle List/Remove UX
**Files:**
- Modify: `adapter/matrix/bot.py`
- Modify: `adapter/matrix/store.py`
- Test: `tests/adapter/matrix/test_dispatcher.py`
- [ ] **Step 1: Write the failing tests**
```python
# tests/adapter/matrix/test_dispatcher.py
async def test_file_only_event_is_staged_and_does_not_dispatch():
runtime = build_runtime(platform=MockPlatformClient())
client = SimpleNamespace(user_id="@bot:example.org", room_send=AsyncMock())
bot = MatrixBot(client, runtime)
runtime.dispatcher.dispatch = AsyncMock(return_value=[])
bot._materialize_incoming_attachments = AsyncMock(
return_value=IncomingMessage(
user_id="@alice:example.org",
platform="matrix",
chat_id="matrix:!r:example.org",
text="",
attachments=[
Attachment(
type="document",
filename="report.pdf",
workspace_path="surfaces/matrix/alice/r/inbox/report.pdf",
mime_type="application/pdf",
)
],
)
)
room = SimpleNamespace(room_id="!r:example.org")
event = SimpleNamespace(
sender="@alice:example.org",
body="report.pdf",
msgtype="m.file",
url="mxc://hs/id",
mimetype="application/pdf",
replyto_event_id=None,
)
await bot.on_room_message(room, event)
runtime.dispatcher.dispatch.assert_not_awaited()
staged = await get_staged_attachments(runtime.store, "!r:example.org", "@alice:example.org")
assert [item["filename"] for item in staged] == ["report.pdf"]
client.room_send.assert_awaited_once()
assert "Следующее сообщение" in client.room_send.await_args.args[2]["body"]
async def test_list_command_returns_current_staged_attachments():
runtime = build_runtime(platform=MockPlatformClient())
await add_staged_attachment(runtime.store, "!r:example.org", "@alice:example.org", {"filename": "a.pdf", "workspace_path": "a.pdf"})
await add_staged_attachment(runtime.store, "!r:example.org", "@alice:example.org", {"filename": "b.pdf", "workspace_path": "b.pdf"})
client = SimpleNamespace(user_id="@bot:example.org", room_send=AsyncMock())
bot = MatrixBot(client, runtime)
room = SimpleNamespace(room_id="!r:example.org")
event = SimpleNamespace(sender="@alice:example.org", body="!list", msgtype="m.text", replyto_event_id=None)
await bot.on_room_message(room, event)
body = client.room_send.await_args.args[2]["body"]
assert "1. a.pdf" in body
assert "2. b.pdf" in body
async def test_remove_invalid_index_returns_short_error():
runtime = build_runtime(platform=MockPlatformClient())
await add_staged_attachment(runtime.store, "!r:example.org", "@alice:example.org", {"filename": "a.pdf", "workspace_path": "a.pdf"})
client = SimpleNamespace(user_id="@bot:example.org", room_send=AsyncMock())
bot = MatrixBot(client, runtime)
room = SimpleNamespace(room_id="!r:example.org")
event = SimpleNamespace(sender="@alice:example.org", body="!remove 9", msgtype="m.text", replyto_event_id=None)
await bot.on_room_message(room, event)
assert client.room_send.await_args.args[2]["body"] == "Нет такого вложения."
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_dispatcher.py -q`
Expected:
- FAIL because file-only events still go straight to dispatcher
- FAIL because `!list` and `!remove` have no Matrix-side handling in the bot
- [ ] **Step 3: Write minimal implementation**
```python
# adapter/matrix/bot.py
def _is_staging_command(self, incoming: IncomingEvent) -> bool:
return isinstance(incoming, IncomingCommand) and incoming.command in {
"matrix_list_attachments",
"matrix_remove_attachment",
}
async def _handle_staging_command(self, room_id: str, user_id: str, incoming: IncomingCommand) -> list[OutgoingMessage]:
if incoming.command == "matrix_list_attachments":
return [OutgoingMessage(chat_id=incoming.chat_id, text=self._format_staged_attachments(room_id, user_id))]
if incoming.command == "matrix_remove_attachment" and incoming.args == ["all"]:
await clear_staged_attachments(self.runtime.store, room_id, user_id)
return [OutgoingMessage(chat_id=incoming.chat_id, text="Вложения очищены.")]
```
```python
# adapter/matrix/bot.py
if isinstance(incoming, IncomingMessage) and incoming.attachments and not incoming.text:
incoming = await self._materialize_incoming_attachments(room.room_id, sender, incoming)
await self._stage_attachments(room.room_id, sender, incoming.attachments)
await self._send_all(room.room_id, [OutgoingMessage(chat_id=dispatch_chat_id, text=self._format_staged_attachments(room.room_id, sender, with_hint=True))])
return
if self._is_staging_command(incoming):
outgoing = await self._handle_staging_command(room.room_id, sender, incoming)
await self._send_all(room.room_id, outgoing)
return
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_dispatcher.py -q`
Expected: PASS for staging/list/remove behavior
- [ ] **Step 5: Commit**
```bash
git add adapter/matrix/bot.py adapter/matrix/store.py tests/adapter/matrix/test_dispatcher.py
git commit -m "feat: add matrix staging list and remove flow"
```
### Task 4: Commit Staged Files With the Next Normal Message
**Files:**
- Modify: `adapter/matrix/bot.py`
- Test: `tests/adapter/matrix/test_dispatcher.py`
- Modify: `README.md`
- [ ] **Step 1: Write the failing tests**
```python
# tests/adapter/matrix/test_dispatcher.py
async def test_next_normal_message_commits_staged_attachments():
runtime = build_runtime(platform=MockPlatformClient())
await add_staged_attachment(
runtime.store,
"!r:example.org",
"@alice:example.org",
{
"filename": "report.pdf",
"workspace_path": "surfaces/matrix/alice/r/inbox/report.pdf",
"mime_type": "application/pdf",
},
)
client = SimpleNamespace(user_id="@bot:example.org")
bot = MatrixBot(client, runtime)
bot._send_all = AsyncMock()
runtime.dispatcher.dispatch = AsyncMock(return_value=[])
room = SimpleNamespace(room_id="!r:example.org")
event = SimpleNamespace(sender="@alice:example.org", body="Проанализируй", msgtype="m.text", replyto_event_id=None)
await bot.on_room_message(room, event)
dispatched = runtime.dispatcher.dispatch.await_args.args[0]
assert isinstance(dispatched, IncomingMessage)
assert dispatched.text == "Проанализируй"
assert [a.workspace_path for a in dispatched.attachments] == [
"surfaces/matrix/alice/r/inbox/report.pdf"
]
assert await get_staged_attachments(runtime.store, "!r:example.org", "@alice:example.org") == []
async def test_failed_commit_preserves_staged_attachments():
runtime = build_runtime(platform=MockPlatformClient())
await add_staged_attachment(
runtime.store,
"!r:example.org",
"@alice:example.org",
{"filename": "report.pdf", "workspace_path": "surfaces/matrix/alice/r/inbox/report.pdf"},
)
client = SimpleNamespace(user_id="@bot:example.org", room_send=AsyncMock())
bot = MatrixBot(client, runtime)
runtime.dispatcher.dispatch = AsyncMock(side_effect=PlatformError("boom"))
room = SimpleNamespace(room_id="!r:example.org")
event = SimpleNamespace(sender="@alice:example.org", body="Проанализируй", msgtype="m.text", replyto_event_id=None)
await bot.on_room_message(room, event)
staged = await get_staged_attachments(runtime.store, "!r:example.org", "@alice:example.org")
assert [item["filename"] for item in staged] == ["report.pdf"]
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_dispatcher.py -q`
Expected:
- FAIL because normal text messages do not yet merge staged attachments
- FAIL because staged items are never preserved/cleared based on commit outcome
- [ ] **Step 3: Write minimal implementation**
```python
# adapter/matrix/bot.py
async def _merge_staged_attachments(
self,
room_id: str,
user_id: str,
incoming: IncomingMessage,
) -> IncomingMessage:
staged = await get_staged_attachments(self.runtime.store, room_id, user_id)
if not staged:
return incoming
return IncomingMessage(
user_id=incoming.user_id,
platform=incoming.platform,
chat_id=incoming.chat_id,
text=incoming.text,
reply_to=incoming.reply_to,
attachments=[
Attachment(
type="document",
filename=item.get("filename"),
mime_type=item.get("mime_type"),
workspace_path=item.get("workspace_path"),
)
for item in staged
],
)
```
```python
# adapter/matrix/bot.py
staged_before_dispatch = False
if isinstance(incoming, IncomingMessage) and incoming.text and not incoming.attachments:
staged = await get_staged_attachments(self.runtime.store, room.room_id, sender)
if staged:
incoming = await self._merge_staged_attachments(room.room_id, sender, incoming)
staged_before_dispatch = True
try:
outgoing = await self.runtime.dispatcher.dispatch(incoming)
except PlatformError:
...
else:
if staged_before_dispatch:
await clear_staged_attachments(self.runtime.store, room.room_id, sender)
```
- [ ] **Step 4: Run targeted tests to verify they pass**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_store.py tests/adapter/matrix/test_converter.py tests/adapter/matrix/test_dispatcher.py -q`
Expected: PASS
- [ ] **Step 5: Update docs**
Add to `README.md`:
```md
### Matrix staged attachments
If a Matrix user sends files without a text instruction, the bot stages them per chat and replies with the current list.
- `!list` shows staged files
- `!remove <n>` removes one staged file by index
- `!remove all` clears all staged files
The next normal user message is sent to the agent together with all staged files.
```
- [ ] **Step 6: Run broader verification**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_store.py tests/adapter/matrix/test_converter.py tests/adapter/matrix/test_dispatcher.py tests/adapter/matrix/test_files.py tests/adapter/matrix/test_send_outgoing.py tests/core/test_integration.py tests/platform/test_real.py -q`
Expected: PASS
- [ ] **Step 7: Commit**
```bash
git add adapter/matrix/bot.py README.md tests/adapter/matrix/test_store.py tests/adapter/matrix/test_converter.py tests/adapter/matrix/test_dispatcher.py
git commit -m "feat: commit staged matrix attachments on next message"
```
## Self-Review
- Spec coverage:
- staged per `(chat_id, user_id)`: Task 1
- short commands `!list`, `!remove <n>`, `!remove all`: Task 2 and Task 3
- file-only events do not invoke agent: Task 3
- next normal message commits staged attachments: Task 4
- failed commit preserves staged attachments: Task 4
- docs update: Task 4
- Placeholder scan:
- no `TODO`, `TBD`, or deferred behavior left in task steps
- Type consistency:
- staged storage uses `dict` records with `filename`, `workspace_path`, `mime_type`
- bot reconstructs `core.protocol.Attachment` from those same keys

View file

@ -0,0 +1,540 @@
# Transport Layer Thin Adapter Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Replace the current custom transport behavior with a thin upstream-compatible adapter so Matrix uses `platform-agent_api.AgentApi` almost as-is and keeps only integration concerns on the `surfaces` side.
**Architecture:** Keep a tiny `AgentApiWrapper` only as a construction/factory shim (`base_url` normalization + `for_chat(chat_id)`). Move all resilience logic that still belongs to `surfaces` into `RealPlatformClient`: per-chat client caching, per-chat send locks, disconnect-on-failure, and `PlatformError` mapping. Delete all custom stream semantics from the wrapper so bugs in streaming can be attributed cleanly to either upstream or our integration layer.
**Tech Stack:** Python 3.11, `aiohttp`, upstream `lambda_agent_api`, `pytest`, `pytest-asyncio`, `ruff`
---
## File Structure
- Modify: `sdk/agent_api_wrapper.py`
Purpose: shrink the wrapper to a thin constructor/factory shim with no custom `_listen()` or `send_message()` logic.
- Modify: `sdk/real.py`
Purpose: keep integration-only behavior: cached per-chat clients, chat locks, attachment forwarding, and failure cleanup.
- Modify: `adapter/matrix/bot.py`
Purpose: instantiate the wrapper with the modern `base_url` path instead of a raw `url`, matching the pinned upstream API.
- Modify: `tests/platform/test_real.py`
Purpose: remove tests that encode custom wrapper semantics and replace them with tests that prove only the intended integration guarantees.
- Modify: `README.md`
Purpose: document that the transport layer now uses the pinned upstream `platform-agent_api` client semantics directly, with only a thin local adapter.
### Task 1: Shrink `AgentApiWrapper` To A Thin Factory Shim
**Files:**
- Modify: `sdk/agent_api_wrapper.py`
- Test: `tests/platform/test_real.py`
- [ ] **Step 1: Replace wrapper-only behavior tests with thin-wrapper tests**
Update `tests/platform/test_real.py` so it no longer asserts any custom post-END drain or wrapper-owned timeout behavior. Replace those tests with the following:
```python
def test_agent_api_wrapper_normalizes_base_url_and_uses_modern_constructor(monkeypatch):
captured = {}
def fake_init(self, agent_id, base_url=None, chat_id=0, **kwargs):
captured["agent_id"] = agent_id
captured["base_url"] = base_url
captured["chat_id"] = chat_id
monkeypatch.setattr(agent_api_wrapper_module.AgentApi, "__init__", fake_init)
wrapper = AgentApiWrapper(
agent_id="agent-1",
base_url="ws://platform-agent:8000/v1/agent_ws/",
chat_id="41",
)
assert wrapper.chat_id == "41"
assert wrapper._base_url == "ws://platform-agent:8000"
assert captured == {
"agent_id": "agent-1",
"base_url": "ws://platform-agent:8000",
"chat_id": "41",
}
def test_agent_api_wrapper_for_chat_reuses_normalized_base_url(monkeypatch):
init_calls = []
def fake_init(self, agent_id, base_url=None, chat_id=0, **kwargs):
self.id = agent_id
self.chat_id = chat_id
self.url = base_url
init_calls.append((agent_id, base_url, chat_id))
monkeypatch.setattr(agent_api_wrapper_module.AgentApi, "__init__", fake_init)
root = AgentApiWrapper(
agent_id="agent-1",
base_url="http://platform-agent:8000/v1/agent_ws/",
chat_id="1",
)
child = root.for_chat("99")
assert child is not root
assert child.chat_id == "99"
assert child._base_url == "http://platform-agent:8000"
assert init_calls == [
("agent-1", "http://platform-agent:8000", "1"),
("agent-1", "http://platform-agent:8000", "99"),
]
```
- [ ] **Step 2: Run tests to verify old assumptions fail**
Run:
```bash
/bin/zsh -lc 'PYTHONPATH=.:external/platform-agent_api uv run pytest tests/platform/test_real.py -q -k "recovers_late_text_after_first_end or times_out_on_idle_stream or normalizes_base_url_and_uses_modern_constructor or for_chat_reuses_normalized_base_url"'
```
Expected:
- FAIL because the old wrapper-behavior tests still exist
- FAIL or SKIP for the new thin-wrapper tests until code/tests are aligned
- [ ] **Step 3: Replace `sdk/agent_api_wrapper.py` with a thin wrapper**
Rewrite `sdk/agent_api_wrapper.py` to the minimal implementation below:
```python
from __future__ import annotations
import inspect
import re
import sys
from pathlib import Path
from urllib.parse import urlsplit, urlunsplit
_api_root = Path(__file__).resolve().parents[1] / "external" / "platform-agent_api"
if str(_api_root) not in sys.path:
sys.path.insert(0, str(_api_root))
from lambda_agent_api.agent_api import AgentApi # noqa: E402
class AgentApiWrapper(AgentApi):
"""Thin construction/factory shim over the pinned upstream AgentApi."""
def __init__(
self,
agent_id: str,
base_url: str,
*,
chat_id: int | str = 0,
**kwargs,
) -> None:
self._base_url = self._normalize_base_url(base_url)
self._init_kwargs = dict(kwargs)
self.chat_id = chat_id
if not self._supports_modern_constructor():
raise RuntimeError("Pinned platform-agent_api is expected to support base_url + chat_id")
super().__init__(
agent_id=agent_id,
base_url=self._base_url,
chat_id=chat_id,
**kwargs,
)
@staticmethod
def _supports_modern_constructor() -> bool:
try:
parameters = inspect.signature(AgentApi.__init__).parameters
except (TypeError, ValueError):
return False
return "base_url" in parameters and "chat_id" in parameters
@staticmethod
def _normalize_base_url(base_url: str) -> str:
parsed = urlsplit(base_url)
path = re.sub(r"(?:/v1)?/agent_ws(?:/[^/]+)?/?$", "", parsed.path.rstrip("/"))
return urlunsplit((parsed.scheme, parsed.netloc, path, "", ""))
def for_chat(self, chat_id: int | str) -> "AgentApiWrapper":
return type(self)(
agent_id=self.id,
base_url=self._base_url,
chat_id=chat_id,
**self._init_kwargs,
)
```
- [ ] **Step 4: Run the wrapper-focused tests**
Run:
```bash
/bin/zsh -lc 'PYTHONPATH=.:external/platform-agent_api uv run pytest tests/platform/test_real.py -q -k "normalizes_base_url_and_uses_modern_constructor or for_chat_reuses_normalized_base_url"'
```
Expected:
- PASS
- [ ] **Step 5: Commit**
```bash
git add sdk/agent_api_wrapper.py tests/platform/test_real.py
git commit -m "refactor: shrink agent api wrapper to thin adapter"
```
### Task 2: Simplify `RealPlatformClient` To The Pinned Modern API
**Files:**
- Modify: `sdk/real.py`
- Modify: `adapter/matrix/bot.py`
- Test: `tests/platform/test_real.py`
- [ ] **Step 1: Add failing integration tests for the desired thin-adapter contract**
Extend `tests/platform/test_real.py` with these assertions:
```python
@pytest.mark.asyncio
async def test_real_platform_client_passes_attachments_to_modern_send_message():
agent_api = FakeAgentApiFactory()
client = RealPlatformClient(
agent_api=agent_api,
prototype_state=PrototypeStateStore(),
platform="matrix",
)
attachment = Attachment(
type="document",
filename="report.pdf",
mime_type="application/pdf",
workspace_path="surfaces/matrix/u1/r1/inbox/report.pdf",
)
result = await client.send_message(
"@alice:example.org",
"chat-1",
"read this",
attachments=[attachment],
)
assert result.response == "read this"
assert agent_api.instances["chat-1"].calls == [
("read this", ["surfaces/matrix/u1/r1/inbox/report.pdf"])
]
@pytest.mark.asyncio
async def test_real_platform_client_disconnects_chat_after_agent_exception():
class ErroringChatAgentApi:
def __init__(self, chat_id: str) -> None:
self.chat_id = chat_id
self.connect_calls = 0
self.close_calls = 0
async def connect(self) -> None:
self.connect_calls += 1
async def close(self) -> None:
self.close_calls += 1
async def send_message(self, text: str, attachments: list[str] | None = None):
raise agent_api_wrapper_module.AgentException("INTERNAL_ERROR", "boom")
yield
agent_api = FakeAgentApiFactory()
erroring = ErroringChatAgentApi("chat-1")
agent_api.for_chat = lambda chat_id: erroring
client = RealPlatformClient(
agent_api=agent_api,
prototype_state=PrototypeStateStore(),
platform="matrix",
)
with pytest.raises(PlatformError, match="boom") as exc_info:
await client.send_message("@alice:example.org", "chat-1", "hello")
assert exc_info.value.code == "INTERNAL_ERROR"
assert erroring.close_calls == 1
assert "chat-1" not in client._chat_apis
```
- [ ] **Step 2: Run tests to verify they fail before simplification**
Run:
```bash
/bin/zsh -lc 'PYTHONPATH=.:external/platform-agent_api uv run pytest tests/platform/test_real.py -q -k "passes_attachments_to_modern_send_message or disconnects_chat_after_agent_exception"'
```
Expected:
- FAIL until `sdk/real.py` and Matrix runtime wiring are aligned with the pinned modern API
- [ ] **Step 3: Simplify `sdk/real.py` and Matrix runtime construction**
Make these exact edits:
```python
# adapter/matrix/bot.py
def _build_platform_from_env() -> PlatformClient:
backend = os.environ.get("MATRIX_PLATFORM_BACKEND", "mock").strip().lower()
if backend == "real":
base_url = os.environ["AGENT_BASE_URL"]
return RealPlatformClient(
agent_api=AgentApiWrapper(agent_id="matrix-bot", base_url=base_url),
prototype_state=PrototypeStateStore(),
platform="matrix",
)
return MockPlatformClient()
```
```python
# sdk/real.py
from __future__ import annotations
import asyncio
from collections.abc import AsyncIterator
from pathlib import Path
from sdk.agent_api_wrapper import AgentApiWrapper
from sdk.interface import (
Attachment,
MessageChunk,
MessageResponse,
PlatformClient,
PlatformError,
User,
UserSettings,
)
from sdk.prototype_state import PrototypeStateStore
class RealPlatformClient(PlatformClient):
def __init__(
self,
agent_api: AgentApiWrapper,
prototype_state: PrototypeStateStore,
platform: str = "matrix",
) -> None:
self._agent_api = agent_api
self._prototype_state = prototype_state
self._platform = platform
self._chat_apis: dict[str, AgentApiWrapper] = {}
self._chat_api_lock = asyncio.Lock()
self._chat_send_locks: dict[str, asyncio.Lock] = {}
@property
def agent_api(self) -> AgentApiWrapper:
return self._agent_api
async def _get_chat_api(self, chat_id: str) -> AgentApiWrapper:
chat_key = str(chat_id)
chat_api = self._chat_apis.get(chat_key)
if chat_api is None:
async with self._chat_api_lock:
chat_api = self._chat_apis.get(chat_key)
if chat_api is None:
chat_api = self._agent_api.for_chat(chat_key)
await chat_api.connect()
self._chat_apis[chat_key] = chat_api
return chat_api
def _get_chat_send_lock(self, chat_id: str) -> asyncio.Lock:
chat_key = str(chat_id)
lock = self._chat_send_locks.get(chat_key)
if lock is None:
lock = asyncio.Lock()
self._chat_send_locks[chat_key] = lock
return lock
async def send_message(
self,
user_id: str,
chat_id: str,
text: str,
attachments: list[Attachment] | None = None,
) -> MessageResponse:
response_parts: list[str] = []
tokens_used = 0
sent_attachments: list[Attachment] = []
message_id = user_id
lock = self._get_chat_send_lock(chat_id)
async with lock:
chat_api = await self._get_chat_api(chat_id)
try:
async for event in chat_api.send_message(text, attachments=self._attachment_paths(attachments)):
if hasattr(event, "text"):
response_parts.append(event.text)
elif event.__class__.__name__ == "MsgEventEnd":
tokens_used = getattr(event, "tokens_used", 0)
elif "SEND_FILE" in getattr(getattr(event, "type", None), "value", str(getattr(event, "type", ""))):
attachment = self._attachment_from_send_file_event(event)
if attachment is not None:
sent_attachments.append(attachment)
except Exception as exc:
await self._handle_chat_api_failure(chat_id, exc)
await self._prototype_state.set_last_tokens_used(str(chat_id), tokens_used)
return MessageResponse(
message_id=message_id,
response="".join(response_parts),
tokens_used=tokens_used,
finished=True,
attachments=sent_attachments,
)
async def stream_message(
self,
user_id: str,
chat_id: str,
text: str,
attachments: list[Attachment] | None = None,
) -> AsyncIterator[MessageChunk]:
lock = self._get_chat_send_lock(chat_id)
async with lock:
chat_api = await self._get_chat_api(chat_id)
try:
async for event in chat_api.send_message(text, attachments=self._attachment_paths(attachments)):
if hasattr(event, "text"):
yield MessageChunk(
message_id=user_id,
delta=event.text,
finished=False,
)
elif event.__class__.__name__ == "MsgEventEnd":
tokens_used = getattr(event, "tokens_used", 0)
await self._prototype_state.set_last_tokens_used(str(chat_id), tokens_used)
yield MessageChunk(
message_id=user_id,
delta="",
finished=True,
tokens_used=tokens_used,
)
except Exception as exc:
await self._handle_chat_api_failure(chat_id, exc)
async def disconnect_chat(self, chat_id: str) -> None:
chat_key = str(chat_id)
chat_api = self._chat_apis.pop(chat_key, None)
self._chat_send_locks.pop(chat_key, None)
if chat_api is not None:
await chat_api.close()
async def close(self) -> None:
for chat_api in list(self._chat_apis.values()):
await chat_api.close()
self._chat_apis.clear()
self._chat_send_locks.clear()
async def _handle_chat_api_failure(self, chat_id: str, exc: Exception) -> None:
await self.disconnect_chat(chat_id)
code = getattr(exc, "code", None) or "PLATFORM_CONNECTION_ERROR"
raise PlatformError(str(exc), code=code) from exc
@staticmethod
def _attachment_paths(attachments: list[Attachment] | None) -> list[str]:
if not attachments:
return []
return [attachment.workspace_path for attachment in attachments if attachment.workspace_path]
```
- [ ] **Step 4: Run the focused transport tests**
Run:
```bash
/bin/zsh -lc 'PYTHONPATH=.:external/platform-agent_api uv run pytest tests/platform/test_real.py -q -k "passes_attachments_to_modern_send_message or disconnects_chat_after_agent_exception or wraps_connection_closed_as_platform_error or reconnects_after_closed_chat_api"'
```
Expected:
- PASS
- [ ] **Step 5: Commit**
```bash
git add adapter/matrix/bot.py sdk/real.py tests/platform/test_real.py
git commit -m "refactor: use upstream transport semantics in real client"
```
### Task 3: Remove Custom Transport Assumptions From Tests And Docs
**Files:**
- Modify: `tests/platform/test_real.py`
- Modify: `README.md`
- [ ] **Step 1: Delete tests that encode wrapper-owned stream semantics**
Remove any tests that assert:
- late text is recovered after the first `END`
- duplicate `END` is repaired inside our wrapper
- wrapper-owned idle timeout semantics
The file should keep only tests for:
- wrapper construction/factory behavior
- per-chat client reuse
- reconnect/disconnect after failure
- attachment forwarding
- per-chat send locking
- [ ] **Step 2: Update README transport description**
Add this text to the Matrix runtime/backend section in `README.md`:
```md
Transport layer note:
- `surfaces` now uses the pinned upstream `platform-agent_api.AgentApi` stream semantics directly
- local code keeps only a thin adapter for client construction and per-chat client factories
- request serialization, disconnect-on-failure, and `PlatformError` mapping remain in `sdk/real.py`
- `surfaces` no longer performs local post-END stream reconstruction
```
- [ ] **Step 3: Run the full verification set**
Run:
```bash
uv run ruff check adapter/matrix sdk tests/platform/test_real.py
/bin/zsh -lc 'PYTHONPATH=. uv run pytest tests/adapter/matrix -q'
/bin/zsh -lc 'PYTHONPATH=.:external/platform-agent_api uv run pytest tests/platform/test_real.py -q'
```
Expected:
- `ruff` reports `All checks passed!`
- Matrix adapter tests PASS
- `tests/platform/test_real.py` PASS
- [ ] **Step 4: Commit**
```bash
git add README.md tests/platform/test_real.py
git commit -m "test: remove custom transport semantics assumptions"
```
---
## Self-Review
- Spec coverage:
- thin adapter target: covered by Task 1
- integration-only `RealPlatformClient`: covered by Task 2
- removal of custom stream semantics assumptions: covered by Task 3
- re-verification after cleanup: covered by Task 3
- Placeholder scan:
- no `TODO`, `TBD`, or deferred implementation placeholders remain in task steps
- Type consistency:
- `AgentApiWrapper` remains the construction/factory type used by `RealPlatformClient`
- failure mapping still terminates in `PlatformError`
- attachment forwarding consistently uses `attachments: list[str]`

View file

@ -0,0 +1,855 @@
# Matrix Multi-Agent Routing And Restart State Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Add Matrix multi-agent routing with user agent selection, room-level agent binding, and durable surface state that survives normal restart.
**Architecture:** Keep the shared `PlatformClient` protocol unchanged. Add a Matrix-specific routing facade that translates local Matrix chat identity into `(agent_id, platform_chat_id)` and delegates to one `RealPlatformClient` per configured agent. Persist only durable routing state in the existing SQLite-backed surface store and deliberately drop temporary UX state on restart.
**Tech Stack:** Python 3.11, matrix-nio, structlog, PyYAML, pytest, pytest-asyncio
---
## File Structure
- Create: `adapter/matrix/agent_registry.py`
Purpose: load and validate the YAML agent registry used by Matrix runtime.
- Create: `adapter/matrix/routed_platform.py`
Purpose: implement a Matrix-specific `PlatformClient` facade that resolves room bindings and delegates to per-agent `RealPlatformClient` instances.
- Create: `adapter/matrix/handlers/agent.py`
Purpose: implement `!agent` listing and selection behavior.
- Create: `tests/adapter/matrix/test_agent_registry.py`
Purpose: cover YAML loading and registry validation.
- Create: `tests/adapter/matrix/test_routed_platform.py`
Purpose: cover room-target resolution and per-agent delegation without changing the shared protocol.
- Create: `tests/adapter/matrix/test_agent_handler.py`
Purpose: cover `!agent` UX and persistence of `selected_agent_id`.
- Create: `tests/adapter/matrix/test_restart_persistence.py`
Purpose: prove durable user/room state and `PLATFORM_CHAT_SEQ_KEY` survive runtime recreation with SQLite.
- Create: `config/matrix-agents.example.yaml`
Purpose: document the expected agent registry format.
- Modify: `pyproject.toml`
Purpose: add YAML parsing dependency required by the runtime registry loader.
- Modify: `.env.example`
Purpose: document the config path env var for the Matrix agent registry.
- Modify: `README.md`
Purpose: document the new config file, `!agent`, and restart persistence expectations.
- Modify: `adapter/matrix/store.py`
Purpose: add helpers for `selected_agent_id`, room `agent_id`, and explicit sequence persistence semantics.
- Modify: `adapter/matrix/bot.py`
Purpose: load the agent registry, construct the routed platform facade, keep local Matrix chat ids through dispatch, and enforce stale/unbound room behavior before dispatch.
- Modify: `adapter/matrix/handlers/__init__.py`
Purpose: register the new `!agent` command.
- Modify: `adapter/matrix/handlers/chat.py`
Purpose: require a selected agent for `!new` and bind new rooms to that agent.
- Modify: `adapter/matrix/handlers/context_commands.py`
Purpose: keep context commands compatible with local chat ids and routed platform delegation.
- Modify: `adapter/matrix/handlers/settings.py`
Purpose: expose `!agent` in help text.
- Modify: `tests/adapter/matrix/test_dispatcher.py`
Purpose: cover pre-dispatch gating, stale room behavior, and `!new` semantics.
- Modify: `tests/adapter/matrix/test_context_commands.py`
Purpose: keep load/reset/context flows aligned with the routed platform facade.
---
### Task 1: Add The Agent Registry And Configuration Wiring
**Files:**
- Create: `adapter/matrix/agent_registry.py`
- Create: `tests/adapter/matrix/test_agent_registry.py`
- Create: `config/matrix-agents.example.yaml`
- Modify: `pyproject.toml`
- Modify: `.env.example`
- Modify: `README.md`
- [ ] **Step 1: Write the failing registry tests**
```python
# tests/adapter/matrix/test_agent_registry.py
from pathlib import Path
import pytest
from adapter.matrix.agent_registry import AgentRegistryError, load_agent_registry
def test_load_agent_registry_reads_yaml_entries(tmp_path: Path):
path = tmp_path / "agents.yaml"
path.write_text(
"agents:\n"
" - id: agent-1\n"
" label: Analyst\n"
" - id: agent-2\n"
" label: Research\n",
encoding="utf-8",
)
registry = load_agent_registry(path)
assert [agent.agent_id for agent in registry.agents] == ["agent-1", "agent-2"]
assert registry.get("agent-1").label == "Analyst"
def test_load_agent_registry_rejects_duplicate_ids(tmp_path: Path):
path = tmp_path / "agents.yaml"
path.write_text(
"agents:\n"
" - id: agent-1\n"
" label: Analyst\n"
" - id: agent-1\n"
" label: Duplicate\n",
encoding="utf-8",
)
with pytest.raises(AgentRegistryError, match="duplicate agent id"):
load_agent_registry(path)
```
- [ ] **Step 2: Run the registry tests to verify they fail**
Run: `uv run pytest tests/adapter/matrix/test_agent_registry.py -q`
Expected: FAIL with `ModuleNotFoundError` or `ImportError` for `adapter.matrix.agent_registry`.
- [ ] **Step 3: Add the YAML dependency and implement the registry loader**
```toml
# pyproject.toml
dependencies = [
"aiogram>=3.4,<4",
"matrix-nio>=0.21",
"pydantic>=2.5",
"structlog>=24.1",
"python-dotenv>=1.0",
"httpx>=0.27",
"aiohttp>=3.9",
"PyYAML>=6.0",
]
```
```python
# adapter/matrix/agent_registry.py
from __future__ import annotations
from dataclasses import dataclass
from pathlib import Path
import yaml
class AgentRegistryError(ValueError):
pass
@dataclass(frozen=True)
class AgentDefinition:
agent_id: str
label: str
class AgentRegistry:
def __init__(self, agents: list[AgentDefinition]) -> None:
self.agents = agents
self._by_id = {agent.agent_id: agent for agent in agents}
def get(self, agent_id: str) -> AgentDefinition:
try:
return self._by_id[agent_id]
except KeyError as exc:
raise AgentRegistryError(f"unknown agent id: {agent_id}") from exc
def load_agent_registry(path: str | Path) -> AgentRegistry:
raw = yaml.safe_load(Path(path).read_text(encoding="utf-8")) or {}
entries = raw.get("agents")
if not isinstance(entries, list) or not entries:
raise AgentRegistryError("agents registry must contain a non-empty agents list")
agents: list[AgentDefinition] = []
seen: set[str] = set()
for entry in entries:
agent_id = str(entry.get("id", "")).strip()
label = str(entry.get("label", "")).strip()
if not agent_id or not label:
raise AgentRegistryError("each agent entry requires id and label")
if agent_id in seen:
raise AgentRegistryError(f"duplicate agent id: {agent_id}")
seen.add(agent_id)
agents.append(AgentDefinition(agent_id=agent_id, label=label))
return AgentRegistry(agents)
```
- [ ] **Step 4: Add the example config and runtime wiring docs**
```yaml
# config/matrix-agents.example.yaml
agents:
- id: agent-1
label: Analyst
- id: agent-2
label: Research
```
```env
# .env.example
MATRIX_AGENT_REGISTRY_PATH=config/matrix-agents.yaml
```
```markdown
# README.md
1. Copy `config/matrix-agents.example.yaml` to `config/matrix-agents.yaml`
2. Set `MATRIX_AGENT_REGISTRY_PATH=config/matrix-agents.yaml`
3. Use `!agent` in Matrix to select the active upstream agent
```
- [ ] **Step 5: Run the registry tests to verify they pass**
Run: `uv run pytest tests/adapter/matrix/test_agent_registry.py -q`
Expected: PASS
- [ ] **Step 6: Commit**
```bash
git add pyproject.toml .env.example README.md config/matrix-agents.example.yaml adapter/matrix/agent_registry.py tests/adapter/matrix/test_agent_registry.py
git commit -m "feat: add matrix agent registry loader"
```
---
### Task 2: Add A Matrix Routing Facade Without Changing `PlatformClient`
**Files:**
- Create: `adapter/matrix/routed_platform.py`
- Create: `tests/adapter/matrix/test_routed_platform.py`
- Modify: `adapter/matrix/bot.py`
- [ ] **Step 1: Write the failing routed-platform tests**
```python
# tests/adapter/matrix/test_routed_platform.py
import pytest
from adapter.matrix.routed_platform import RoutedPlatformClient
from adapter.matrix.store import set_room_meta
from core.chat import ChatManager
from core.store import InMemoryStore
from sdk.interface import MessageResponse
from sdk.prototype_state import PrototypeStateStore
class FakeDelegate:
def __init__(self, agent_id: str) -> None:
self.agent_id = agent_id
self.calls = []
async def send_message(self, user_id: str, chat_id: str, text: str, attachments=None):
self.calls.append((user_id, chat_id, text, attachments))
return MessageResponse(
message_id=user_id,
response=f"{self.agent_id}:{text}",
tokens_used=0,
finished=True,
)
async def get_or_create_user(self, external_id: str, platform: str, display_name=None):
return await PrototypeStateStore().get_or_create_user(external_id, platform, display_name)
async def get_settings(self, user_id: str):
return await PrototypeStateStore().get_settings(user_id)
async def update_settings(self, user_id: str, action):
return None
@pytest.mark.asyncio
async def test_routed_platform_delegates_using_room_agent_and_platform_chat_id():
store = InMemoryStore()
chat_mgr = ChatManager(None, store)
await chat_mgr.get_or_create("u1", "C1", "matrix", "!room:example.org", "Chat 1")
await set_room_meta(
store,
"!room:example.org",
{"chat_id": "C1", "matrix_user_id": "u1", "platform_chat_id": "41", "agent_id": "agent-2"},
)
delegates = {"agent-2": FakeDelegate("agent-2")}
platform = RoutedPlatformClient(store=store, chat_mgr=chat_mgr, delegates=delegates)
response = await platform.send_message("u1", "C1", "hello")
assert response.response == "agent-2:hello"
assert delegates["agent-2"].calls == [("u1", "41", "hello", None)]
```
- [ ] **Step 2: Run the routed-platform tests to verify they fail**
Run: `uv run pytest tests/adapter/matrix/test_routed_platform.py -q`
Expected: FAIL with `ImportError` for `RoutedPlatformClient`.
- [ ] **Step 3: Implement the routing facade and integrate runtime construction**
```python
# adapter/matrix/routed_platform.py
from __future__ import annotations
from sdk.interface import PlatformClient
class RoutedPlatformClient(PlatformClient):
def __init__(self, store, chat_mgr, delegates: dict[str, PlatformClient]) -> None:
self._store = store
self._chat_mgr = chat_mgr
self._delegates = delegates
async def _resolve_target(self, user_id: str, local_chat_id: str) -> tuple[PlatformClient, str]:
ctx = await self._chat_mgr.get(local_chat_id, user_id=user_id)
if ctx is None:
raise ValueError(f"Chat {local_chat_id} not found for {user_id}")
room_meta = await self._store.get(f"matrix_room:{ctx.surface_ref}")
if room_meta is None or not room_meta.get("agent_id") or not room_meta.get("platform_chat_id"):
raise ValueError(f"Room {ctx.surface_ref} is not bound to an agent target")
delegate = self._delegates[room_meta["agent_id"]]
return delegate, str(room_meta["platform_chat_id"])
async def send_message(self, user_id: str, chat_id: str, text: str, attachments=None):
delegate, platform_chat_id = await self._resolve_target(user_id, chat_id)
return await delegate.send_message(user_id, platform_chat_id, text, attachments)
async def stream_message(self, user_id: str, chat_id: str, text: str, attachments=None):
delegate, platform_chat_id = await self._resolve_target(user_id, chat_id)
async for chunk in delegate.stream_message(user_id, platform_chat_id, text, attachments):
yield chunk
async def get_or_create_user(self, external_id: str, platform: str, display_name=None):
first_delegate = next(iter(self._delegates.values()))
return await first_delegate.get_or_create_user(external_id, platform, display_name)
async def get_settings(self, user_id: str):
first_delegate = next(iter(self._delegates.values()))
return await first_delegate.get_settings(user_id)
async def update_settings(self, user_id: str, action):
first_delegate = next(iter(self._delegates.values()))
await first_delegate.update_settings(user_id, action)
```
```python
# adapter/matrix/bot.py
from adapter.matrix.agent_registry import load_agent_registry
from adapter.matrix.routed_platform import RoutedPlatformClient
def _build_platform_from_env(store: StateStore, chat_mgr: ChatManager) -> PlatformClient:
backend = os.environ.get("MATRIX_PLATFORM_BACKEND", "mock").strip().lower()
if backend != "real":
return MockPlatformClient()
registry = load_agent_registry(os.environ["MATRIX_AGENT_REGISTRY_PATH"])
delegates = {
agent.agent_id: RealPlatformClient(
agent_id=agent.agent_id,
agent_base_url=_agent_base_url_from_env(),
prototype_state=PrototypeStateStore(),
platform="matrix",
)
for agent in registry.agents
}
return RoutedPlatformClient(store=store, chat_mgr=chat_mgr, delegates=delegates)
def build_runtime(...):
store = store or InMemoryStore()
chat_mgr = ChatManager(None, store)
platform = platform or _build_platform_from_env(store, chat_mgr)
auth_mgr = AuthManager(platform, store)
settings_mgr = SettingsManager(platform, store)
dispatcher = EventDispatcher(
platform=platform,
chat_mgr=chat_mgr,
auth_mgr=auth_mgr,
settings_mgr=settings_mgr,
)
```
- [ ] **Step 4: Run the routed-platform tests to verify they pass**
Run: `uv run pytest tests/adapter/matrix/test_routed_platform.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add adapter/matrix/routed_platform.py adapter/matrix/bot.py tests/adapter/matrix/test_routed_platform.py
git commit -m "feat: add matrix routed platform facade"
```
---
### Task 3: Add `!agent` Selection And Durable User Agent State
**Files:**
- Create: `adapter/matrix/handlers/agent.py`
- Create: `tests/adapter/matrix/test_agent_handler.py`
- Modify: `adapter/matrix/store.py`
- Modify: `adapter/matrix/handlers/__init__.py`
- Modify: `adapter/matrix/handlers/settings.py`
- [ ] **Step 1: Write the failing agent-handler tests**
```python
# tests/adapter/matrix/test_agent_handler.py
import pytest
from adapter.matrix.handlers.agent import make_handle_agent
from adapter.matrix.store import get_room_meta, get_selected_agent_id, set_room_meta
from core.protocol import IncomingCommand
from core.store import InMemoryStore
class FakeRegistry:
def __init__(self) -> None:
self.agents = [
type("Agent", (), {"agent_id": "agent-1", "label": "Analyst"})(),
type("Agent", (), {"agent_id": "agent-2", "label": "Research"})(),
]
@pytest.mark.asyncio
async def test_agent_command_lists_available_agents():
handler = make_handle_agent(store=InMemoryStore(), registry=FakeRegistry())
result = await handler(
IncomingCommand(user_id="u1", platform="matrix", chat_id="C1", command="agent", args=[]),
None,
None,
None,
None,
)
assert "1. Analyst" in result[0].text
assert "2. Research" in result[0].text
@pytest.mark.asyncio
async def test_agent_command_persists_selected_agent_and_binds_unbound_room():
store = InMemoryStore()
await set_room_meta(store, "!room:example.org", {"chat_id": "C1", "matrix_user_id": "u1"})
handler = make_handle_agent(store=store, registry=FakeRegistry())
chat_mgr = type(
"ChatMgr",
(),
{"get": staticmethod(lambda chat_id, user_id=None: type("Ctx", (), {"surface_ref": "!room:example.org"})())},
)()
await handler(
IncomingCommand(user_id="u1", platform="matrix", chat_id="C1", command="agent", args=["2"]),
None,
None,
chat_mgr,
None,
)
assert await get_selected_agent_id(store, "u1") == "agent-2"
room_meta = await get_room_meta(store, "!room:example.org")
assert room_meta["agent_id"] == "agent-2"
```
- [ ] **Step 2: Run the agent-handler tests to verify they fail**
Run: `uv run pytest tests/adapter/matrix/test_agent_handler.py -q`
Expected: FAIL with missing handler or store helpers.
- [ ] **Step 3: Add durable store helpers and implement `!agent`**
```python
# adapter/matrix/store.py
async def get_selected_agent_id(store: StateStore, matrix_user_id: str) -> str | None:
meta = await get_user_meta(store, matrix_user_id) or {}
value = meta.get("selected_agent_id")
return str(value) if value else None
async def set_selected_agent_id(store: StateStore, matrix_user_id: str, agent_id: str) -> None:
meta = await get_user_meta(store, matrix_user_id) or {}
meta["selected_agent_id"] = agent_id
await set_user_meta(store, matrix_user_id, meta)
async def set_room_agent_id(store: StateStore, room_id: str, agent_id: str) -> None:
meta = dict(await get_room_meta(store, room_id) or {})
meta["agent_id"] = agent_id
await set_room_meta(store, room_id, meta)
```
```python
# adapter/matrix/handlers/agent.py
from __future__ import annotations
from adapter.matrix.store import (
get_room_meta,
get_selected_agent_id,
next_platform_chat_id,
set_platform_chat_id,
set_room_agent_id,
set_selected_agent_id,
)
from core.protocol import IncomingCommand, OutgoingMessage
def make_handle_agent(store, registry):
async def handle_agent(event: IncomingCommand, auth_mgr, platform, chat_mgr, settings_mgr):
if not event.args:
current = await get_selected_agent_id(store, event.user_id)
lines = ["Доступные агенты:"]
for index, agent in enumerate(registry.agents, start=1):
marker = " (текущий)" if agent.agent_id == current else ""
lines.append(f"{index}. {agent.label}{marker}")
lines.append("")
lines.append("Выбери агента: !agent <номер>")
return [OutgoingMessage(chat_id=event.chat_id, text="\n".join(lines))]
agent = registry.agents[int(event.args[0]) - 1]
await set_selected_agent_id(store, event.user_id, agent.agent_id)
ctx = await chat_mgr.get(event.chat_id, user_id=event.user_id) if chat_mgr else None
if ctx is not None:
room_meta = await get_room_meta(store, ctx.surface_ref)
if room_meta is not None and not room_meta.get("agent_id"):
await set_room_agent_id(store, ctx.surface_ref, agent.agent_id)
if not room_meta.get("platform_chat_id"):
await set_platform_chat_id(store, ctx.surface_ref, await next_platform_chat_id(store))
return [OutgoingMessage(chat_id=event.chat_id, text=f"Агент переключён на {agent.label}. Этот чат готов к работе.")]
return [OutgoingMessage(chat_id=event.chat_id, text=f"Агент переключён на {agent.label}. Для продолжения используй !new.")]
return handle_agent
```
- [ ] **Step 4: Register the command and update help text**
```python
# adapter/matrix/handlers/__init__.py
from adapter.matrix.handlers.agent import make_handle_agent
dispatcher.register(IncomingCommand, "agent", make_handle_agent(store, registry))
```
```python
# adapter/matrix/handlers/settings.py
HELP_TEXT = "\n".join(
[
"Команды",
"",
"!agent выбрать активного агента",
"!new [название] создать новый чат",
"!chats список активных чатов",
"!rename <название> переименовать текущий чат",
"!archive архивировать текущий чат",
"!context показать текущее состояние контекста",
"!save [имя] сохранить текущий контекст",
"!load показать сохранённые контексты",
]
)
```
- [ ] **Step 5: Run the agent-handler tests to verify they pass**
Run: `uv run pytest tests/adapter/matrix/test_agent_handler.py -q`
Expected: PASS
- [ ] **Step 6: Commit**
```bash
git add adapter/matrix/store.py adapter/matrix/handlers/agent.py adapter/matrix/handlers/__init__.py adapter/matrix/handlers/settings.py tests/adapter/matrix/test_agent_handler.py
git commit -m "feat: add matrix agent selection command"
```
---
### Task 4: Bind Rooms Correctly And Block Stale Chats
**Files:**
- Modify: `adapter/matrix/bot.py`
- Modify: `adapter/matrix/handlers/chat.py`
- Modify: `adapter/matrix/handlers/context_commands.py`
- Modify: `tests/adapter/matrix/test_dispatcher.py`
- Modify: `tests/adapter/matrix/test_context_commands.py`
- [ ] **Step 1: Write the failing dispatcher and context-command tests**
```python
# tests/adapter/matrix/test_dispatcher.py
@pytest.mark.asyncio
async def test_bot_replies_with_agent_prompt_when_user_has_no_selected_agent():
runtime = build_runtime(platform=MockPlatformClient())
client = SimpleNamespace(user_id="@bot:example.org", room_send=AsyncMock())
bot = MatrixBot(client, runtime)
await set_room_meta(runtime.store, "!room:example.org", {"chat_id": "C1", "matrix_user_id": "@alice:example.org"})
await bot.on_room_message(SimpleNamespace(room_id="!room:example.org"), SimpleNamespace(sender="@alice:example.org", body="hello"))
client.room_send.assert_awaited_once()
assert "выбери агента" in client.room_send.call_args.args[2]["body"].lower()
@pytest.mark.asyncio
async def test_new_chat_requires_selected_agent_and_binds_room_meta():
client = SimpleNamespace(
room_create=AsyncMock(return_value=SimpleNamespace(room_id="!r2:example")),
room_put_state=AsyncMock(),
)
runtime = build_runtime(platform=MockPlatformClient(), client=client)
await set_user_meta(runtime.store, "u1", {"space_id": "!space:example", "next_chat_index": 2, "selected_agent_id": "agent-2"})
result = await runtime.dispatcher.dispatch(
IncomingCommand(user_id="u1", platform="matrix", chat_id="C1", command="new", args=["Research"])
)
room_meta = await get_room_meta(runtime.store, "!r2:example")
assert room_meta["agent_id"] == "agent-2"
assert "Создан чат" in result[0].text
```
```python
# tests/adapter/matrix/test_context_commands.py
@pytest.mark.asyncio
async def test_load_selection_calls_platform_with_local_chat_id():
platform = MatrixCommandPlatform()
runtime = build_runtime(platform=platform)
await runtime.chat_mgr.get_or_create("u1", "C1", "matrix", "!room:example.org", "Chat 1")
await set_room_meta(runtime.store, "!room:example.org", {"chat_id": "C1", "matrix_user_id": "u1", "platform_chat_id": "41", "agent_id": "agent-2"})
client = SimpleNamespace(user_id="@bot:example.org", room_send=AsyncMock())
bot = MatrixBot(client, runtime)
await set_load_pending(runtime.store, "u1", "!room:example.org", {"saves": [{"name": "session-a", "created_at": "2026-04-17T00:00:00+00:00"}]})
await bot.on_room_message(SimpleNamespace(room_id="!room:example.org"), SimpleNamespace(sender="u1", body="1"))
platform.send_message.assert_awaited_once_with("u1", "C1", LOAD_PROMPT.format(name="session-a"))
```
- [ ] **Step 2: Run the dispatcher and context-command tests to verify they fail**
Run: `uv run pytest tests/adapter/matrix/test_dispatcher.py tests/adapter/matrix/test_context_commands.py -q`
Expected: FAIL because the current runtime still injects `platform_chat_id` into normal messages and `!new` does not require or persist `agent_id`.
- [ ] **Step 3: Implement room binding and stale-room checks in runtime**
```python
# adapter/matrix/bot.py
from adapter.matrix.store import (
get_selected_agent_id,
get_room_meta,
next_platform_chat_id,
set_platform_chat_id,
set_room_agent_id,
)
async def _ensure_active_room_target(self, room_id: str, user_id: str) -> tuple[dict | None, OutgoingMessage | None]:
room_meta = await get_room_meta(self.runtime.store, room_id)
selected_agent_id = await get_selected_agent_id(self.runtime.store, user_id)
if not selected_agent_id:
return room_meta, OutgoingMessage(chat_id=room_id, text="Сначала выбери агента через !agent.")
if room_meta is None:
return room_meta, None
if not room_meta.get("agent_id"):
await set_room_agent_id(self.runtime.store, room_id, selected_agent_id)
if not room_meta.get("platform_chat_id"):
await set_platform_chat_id(self.runtime.store, room_id, await next_platform_chat_id(self.runtime.store))
room_meta = await get_room_meta(self.runtime.store, room_id)
return room_meta, None
if room_meta["agent_id"] != selected_agent_id:
return room_meta, OutgoingMessage(chat_id=room_id, text="Этот чат привязан к старому агенту. Используй !new.")
return room_meta, None
```
```python
# adapter/matrix/bot.py
local_chat_id = await resolve_chat_id(self.runtime.store, room.room_id, sender)
dispatch_chat_id = local_chat_id
if not body.startswith("!"):
room_meta, blocking = await self._ensure_active_room_target(room.room_id, sender)
if blocking is not None:
await self._send_all(room.room_id, [blocking])
return
incoming = from_room_event(event, room_id=room.room_id, chat_id=dispatch_chat_id)
```
- [ ] **Step 4: Require selected agent for `!new` and persist room `agent_id`**
```python
# adapter/matrix/handlers/chat.py
from adapter.matrix.store import get_selected_agent_id
selected_agent_id = await get_selected_agent_id(store, event.user_id)
if not selected_agent_id:
return [OutgoingMessage(chat_id=event.chat_id, text="Сначала выбери агента через !agent.")]
await set_room_meta(
store,
room_id,
{
"room_type": "chat",
"chat_id": chat_id,
"display_name": room_name,
"matrix_user_id": event.user_id,
"space_id": space_id,
"platform_chat_id": platform_chat_id,
"agent_id": selected_agent_id,
},
)
```
```python
# adapter/matrix/bot.py
room_meta = await get_room_meta(self.runtime.store, room_id)
local_chat_id = room_meta.get("chat_id", room_id) if room_meta else room_id
await self.runtime.platform.send_message(
user_id,
local_chat_id,
LOAD_PROMPT.format(name=name),
)
```
- [ ] **Step 5: Run the dispatcher and context-command tests to verify they pass**
Run: `uv run pytest tests/adapter/matrix/test_dispatcher.py tests/adapter/matrix/test_context_commands.py -q`
Expected: PASS
- [ ] **Step 6: Commit**
```bash
git add adapter/matrix/bot.py adapter/matrix/handlers/chat.py adapter/matrix/handlers/context_commands.py tests/adapter/matrix/test_dispatcher.py tests/adapter/matrix/test_context_commands.py
git commit -m "feat: bind matrix rooms to selected agents"
```
---
### Task 5: Prove Durable Restart State And Sequence Persistence
**Files:**
- Create: `tests/adapter/matrix/test_restart_persistence.py`
- Modify: `adapter/matrix/store.py`
- Modify: `README.md`
- [ ] **Step 1: Write the failing restart-persistence tests**
```python
# tests/adapter/matrix/test_restart_persistence.py
import pytest
from adapter.matrix.store import (
get_selected_agent_id,
next_platform_chat_id,
set_room_meta,
set_selected_agent_id,
)
from core.store import SQLiteStore
@pytest.mark.asyncio
async def test_selected_agent_and_room_binding_survive_store_recreation(tmp_path):
db_path = tmp_path / "matrix.db"
store = SQLiteStore(str(db_path))
await set_selected_agent_id(store, "u1", "agent-2")
await set_room_meta(
store,
"!room:example.org",
{"chat_id": "C1", "matrix_user_id": "u1", "platform_chat_id": "41", "agent_id": "agent-2"},
)
reopened = SQLiteStore(str(db_path))
assert await get_selected_agent_id(reopened, "u1") == "agent-2"
assert (await reopened.get("matrix_room:!room:example.org"))["agent_id"] == "agent-2"
assert (await reopened.get("matrix_room:!room:example.org"))["platform_chat_id"] == "41"
@pytest.mark.asyncio
async def test_platform_chat_sequence_survives_store_recreation(tmp_path):
db_path = tmp_path / "matrix.db"
store = SQLiteStore(str(db_path))
assert await next_platform_chat_id(store) == "1"
assert await next_platform_chat_id(store) == "2"
reopened = SQLiteStore(str(db_path))
assert await next_platform_chat_id(reopened) == "3"
```
- [ ] **Step 2: Run the restart-persistence tests to verify they fail**
Run: `uv run pytest tests/adapter/matrix/test_restart_persistence.py -q`
Expected: FAIL because `selected_agent_id` helpers do not exist yet or sequence persistence behavior is not explicitly covered.
- [ ] **Step 3: Make sequence persistence explicit and document the restart boundary**
```python
# adapter/matrix/store.py
PLATFORM_CHAT_SEQ_KEY = "matrix_platform_chat_seq"
async def next_platform_chat_id(store: StateStore) -> str:
async with _PLATFORM_CHAT_SEQ_LOCK:
data = await store.get(PLATFORM_CHAT_SEQ_KEY)
index = int((data or {}).get("next_platform_chat_index", 1))
await store.set(PLATFORM_CHAT_SEQ_KEY, {"next_platform_chat_index": index + 1})
return str(index)
```
```markdown
# README.md
- Matrix durable state lives in `lambda_matrix.db` and `matrix_store`
- normal restart is supported only when those paths survive container recreation
- staged attachments and pending confirmations are intentionally not restored
```
- [ ] **Step 4: Run the restart-persistence tests to verify they pass**
Run: `uv run pytest tests/adapter/matrix/test_restart_persistence.py -q`
Expected: PASS
- [ ] **Step 5: Run the combined verification sweep**
Run: `uv run pytest tests/adapter/matrix/test_agent_registry.py tests/adapter/matrix/test_routed_platform.py tests/adapter/matrix/test_agent_handler.py tests/adapter/matrix/test_dispatcher.py tests/adapter/matrix/test_context_commands.py tests/adapter/matrix/test_restart_persistence.py tests/platform/test_real.py -q`
Expected: PASS
- [ ] **Step 6: Commit**
```bash
git add adapter/matrix/store.py README.md tests/adapter/matrix/test_restart_persistence.py
git commit -m "test: cover matrix restart state persistence"
```
---
## Self-Review
### Spec coverage
- Multi-agent agent registry: Task 1
- Shared `PlatformClient` preserved via routing facade: Task 2
- `!agent` UX and durable `selected_agent_id`: Task 3
- Unbound room activation, `!new`, stale room rejection: Task 4
- Restart durability for user state, room state, and `PLATFORM_CHAT_SEQ_KEY`: Task 5
### Placeholder scan
- No `TODO`, `TBD`, or “implement later” markers remain.
- Each task includes exact file paths, tests, commands, and minimal code snippets.
### Type consistency
- `selected_agent_id` lives in user metadata throughout the plan.
- `agent_id` and `platform_chat_id` live in room metadata throughout the plan.
- `RoutedPlatformClient` keeps the existing `PlatformClient` method names intact.

View file

@ -0,0 +1,243 @@
# Matrix Direct-Agent Prototype Design
## Goal
Ship a working Matrix prototype that talks to the real Lambda agent instead of `MockPlatformClient`, while preserving the current Matrix adapter logic and keeping the code expandable toward future platform versions.
## Scope
This design is for a Matrix-only prototype delivered from this repository on a dedicated branch. It is not a `main` branch rollout and it is not a separate prototype repo.
The design assumes a minimal companion fork of `platform/agent` may be used, but changes there must stay as small as possible.
## Constraints
- Preserve the current Matrix transport logic as much as possible.
- Keep `core/` unaware of platform immaturity.
- Avoid broad changes to platform repos.
- Prefer one narrow patch to `platform/agent` over changes to both `platform/agent` and `platform/agent_api`.
- Keep the backend boundary reusable for future Telegram or other surfaces.
- Do not pretend unsupported platform capabilities are real.
## Live Platform Findings
Based on the live repo analysis performed on April 7, 2026:
- `platform/master` is not yet a usable consumer-facing backend for surfaces.
- `platform/agent` exposes a working WebSocket endpoint for prompt/response exchange.
- `platform/agent_api` documents and implements text-oriented WebSocket messaging, but the bot does not need to depend on that package directly.
- `platform/agent` currently hardcodes a single shared backend memory thread via `thread_id = "default"`, which would cause all chats to share context.
## Architecture
The prototype remains in this repo and introduces a new real backend path behind the existing SDK boundary.
### New files
- `sdk/real.py`
- Exports `RealPlatformClient`
- Implements the existing `PlatformClient` contract from `sdk/interface.py`
- Composes the lower-level prototype pieces
- `sdk/agent_session.py`
- Owns direct WebSocket communication with the real agent
- Manages connection lifecycle, request/response handling, and thread identity
- `sdk/prototype_state.py`
- Owns local prototype-only state
- Stores user mapping, local settings, and lightweight metadata needed until a real control plane exists
### Responsibility split
- Matrix adapter remains transport-specific only.
- `core/` continues to depend only on `PlatformClient`.
- `RealPlatformClient` acts as the anti-corruption layer between the current bot contract and the platforms incomplete shape.
- Local control-plane behavior remains explicit and replaceable later.
## Message and Identity Model
Each Matrix chat gets a stable backend session identity.
### Surface identity
- Surface: `matrix`
- Surface user id: Matrix MXID, for example `@alice:example.org`
- Surface chat id: logical chat id from `ChatManager`, for example `C1`
- Surface ref: Matrix room id
### Backend thread identity
Use a deterministic thread key:
`matrix:{matrix_user_id}:{chat_id}`
Example:
`matrix:@alice:example.org:C1`
### Mapping rules
- One Matrix logical chat maps to one backend memory thread.
- `!new` creates a fresh logical chat and therefore a fresh backend thread.
- `!rename` only changes display metadata and does not change backend identity.
- `!archive` stops active use of the thread in the surface, but does not need to delete backend memory in v1.
## Runtime Flow
### Normal message flow
1. Matrix event arrives in an existing room.
2. Existing Matrix routing resolves room to logical `chat_id`.
3. `core/handlers/message.py` calls `platform.send_message(...)`.
4. `RealPlatformClient` derives the backend thread key from `(platform, user_id, chat_id)`.
5. `AgentSessionClient` sends the prompt to the agent WebSocket using that thread key.
6. The reply is converted into the existing `MessageResponse` or `MessageChunk` contract.
7. Matrix sends the final text back to the room.
### Settings flow
For v1, settings remain local:
- `get_settings()` reads from local prototype state
- `update_settings()` writes to local prototype state
This is intentional. The prototype must not claim settings are backed by the real platform when no such platform API exists yet.
## Feature Matrix
### Real in v1
- `!start`
- Plain text messaging with the real agent
- Matrix chat lifecycle already implemented in this repo:
- `!new`
- `!chats`
- `!rename`
- `!archive`
- Per-chat conversation memory, provided the agent accepts dynamic thread identity
### Local in v1
- `!settings`
- `!skills`
- `!soul`
- `!safety`
- `!status`
- user registration and local user mapping
### Deferred
- Attachments and file upload to the agent
- Voice input to the agent
- Image input to the agent
- Long-running task callbacks and webhook-style async completion
- Real control-plane integration through `platform/master`
## Minimal Upstream Change
To avoid shared memory across all conversations, make one narrow change in the forked `platform/agent` repo:
- stop hardcoding `thread_id = "default"`
- derive thread identity from WebSocket connection context
### Preferred mechanism
Read `thread_id` from WebSocket query parameters rather than changing the message payload format.
Example:
`ws://host:port/agent_ws/?thread_id=matrix:@alice:example.org:C1`
This is preferred because:
- it limits the platform patch to one repo
- it avoids changing both server and SDK protocol shape
- it keeps the client message body text-only
- it makes session identity explicit and easy to reason about
## Why Not Use `platform/agent_api` Directly
The bot should not depend on their client package for the prototype.
Reasons:
- the bot already has its own internal integration boundary in `sdk/interface.py`
- a tiny local WebSocket client is enough for this protocol
- avoiding a dependency on `platform/agent_api` keeps rebasing simpler
- if upstream stabilizes later, the bot can adopt their SDK without affecting Matrix handlers
## Repo Strategy
### This repo
Owns:
- Matrix surface logic
- SDK compatibility layer
- local prototype state
- backend selection and wiring
### Forked `platform/agent`
Owns only:
- minimal thread identity patch required for per-chat memory
### Explicitly not doing
- no separate prototype repo
- no changes to `platform/master` for v1
- no unnecessary changes to `platform/agent_api`
## Migration Path
This design is intentionally expandable.
When the platform develops further:
- `sdk/prototype_state.py` can be replaced or reduced by a real `MasterClient`
- `sdk/agent_session.py` can remain the direct session transport if still relevant
- `RealPlatformClient` can continue to present the stable bot-facing interface
- Telegram or another surface can reuse the same backend components without rethinking the integration model
## Risks
### Risk: hidden platform assumptions leak upward
Mitigation:
- keep all direct-agent logic below `RealPlatformClient`
- avoid changing `core/` contracts for prototype convenience
### Risk: settings semantics drift from future platform reality
Mitigation:
- make local settings behavior explicit in code and docs
- keep settings isolated in `sdk/prototype_state.py`
### Risk: upstream `agent` fork diverges
Mitigation:
- keep the patch minimal and narrowly scoped to thread identity
### Risk: thread identity source is unstable
Mitigation:
- derive thread key from existing stable bot-side identities: platform, surface user id, logical chat id
## Testing Strategy
- Unit tests for `sdk/agent_session.py` request/response behavior
- Unit tests for `sdk/prototype_state.py` local settings and user mapping
- Unit tests for `sdk/real.py` contract compliance with `PlatformClient`
- Matrix integration tests confirming:
- existing commands still work
- different logical chats map to different backend thread keys
- rename does not change thread identity
- archive stops reuse from the surface perspective
## Success Criteria
- Matrix can talk to the real agent without rewriting the Matrix adapter architecture
- Chats do not share backend memory accidentally
- Unsupported platform capabilities remain local or deferred rather than being faked as “real”
- The backend boundary remains suitable for later Telegram or other surfaces

View file

@ -0,0 +1,278 @@
# Matrix Per-Chat Context Design
## Goal
Move the Matrix surface from the current shared-agent-context MVP to true per-chat agent contexts using the new platform `chat_id`, while preserving the existing Matrix UX model built around rooms, spaces, and local chat labels such as `C1`, `C2`, and `C3`.
## Core Decision
The Matrix surface remains the owner of user-facing chat organization.
- Matrix rooms, spaces, chat names, and archive state remain surface concerns.
- The platform agent becomes the owner of actual conversation context.
- The integration layer stores an explicit mapping from each surface chat to one platform context.
This is the selected "Variant A" architecture:
`surface_chat -> platform_chat_id`
## Why This Decision
The current Matrix adapter already has a stable UX model:
- a user has a space
- each working room has a local chat id like `C1`
- commands such as `!new`, `!chats`, `!rename`, and `!archive` operate on that model
Replacing that entire model with platform-native chat ids would be a broader refactor than needed. The surface and the platform solve different problems:
- the surface organizes rooms and commands for users
- the platform persists and branches real conversation context
Keeping those responsibilities separate lets us add true per-chat context now without rebuilding the Matrix adapter around a new identity model.
## Scope
This design covers:
- true per-chat context for Matrix rooms
- a new `!branch` command
- real context-aware semantics for `!new`, `!context`, `!save`, and `!load`
- lazy migration of legacy Matrix rooms created before platform `chat_id` support
This design does not cover:
- end-to-end Matrix encryption support
- Telegram changes
- platform UI for browsing contexts
- a future unified cross-surface chat browser
## Data Model
### Surface chat identity
The Matrix surface keeps its existing identifiers:
- Matrix room id, for example `!room:example.org`
- local chat id, for example `C2`
- room name
- archive status
- owning space id
These remain the source of truth for Matrix UX.
### Platform context identity
Each working Matrix room gets a `platform_chat_id` stored in its room metadata.
Example `room_meta` shape:
```json
{
"chat_id": "C2",
"space_id": "!space:example.org",
"name": "Research",
"platform_chat_id": "chat_8f2c..."
}
```
Rules:
- one working Matrix room maps to exactly one current platform context
- two Matrix rooms must not share the same `platform_chat_id` unless we intentionally implement that later
- branching creates a new `platform_chat_id`, never reuses the old one
## Runtime Semantics
### Normal message flow
1. A Matrix message arrives in a working room.
2. The Matrix adapter resolves the room to local `room_meta`.
3. The integration layer reads `platform_chat_id` from that metadata.
4. `RealPlatformClient.send_message(...)` sends the message to the platform using that `platform_chat_id`.
5. The platform appends the exchange to that specific context and returns the reply.
6. The Matrix adapter sends the reply back to the room.
The key change is that the agent no longer treats all Matrix rooms as one shared context.
### `!new`
`!new` creates a new user-facing chat and a new empty platform context at the same time.
Flow:
1. Create a new Matrix room in the user space.
2. Ask the platform to create a new blank context and return its `platform_chat_id`.
3. Store that `platform_chat_id` in the new room metadata.
4. Invite the user into the room.
Result:
- the new room is immediately independent
- sending the first message does not share memory with the previous room
### `!branch`
`!branch` creates a new room whose starting point is a snapshot of the current room context.
Flow:
1. Resolve the current room's `platform_chat_id`.
2. Ask the platform to create a new context branched from that source.
3. Create a new Matrix room.
4. Store the new `platform_chat_id` in the new room metadata.
5. Invite the user into the new room.
Result:
- the new room starts with the current history and state
- later messages diverge independently
### `!save`
`!save [name]` saves a snapshot of the current room's platform context under the current user.
Semantics:
- saves are owned by the user, not by the room
- the saved snapshot originates from the current `platform_chat_id`
### `!load`
`!load` shows the user's saved contexts and loads the chosen snapshot into the current room's platform context.
Semantics:
- a saved context created in one room can be loaded into any other room owned by the same user
- loading does not replace the Matrix room identity
- loading affects only the current room's mapped `platform_chat_id`
### `!context`
`!context` reports the state of the current room context, not a global user session.
Minimum expected output:
- current room name or local chat id
- current `platform_chat_id` presence or status
- what saved context, if any, was last loaded here
- last token usage if the platform still returns it
## Legacy Room Migration
Existing Matrix rooms were created before real platform `chat_id` support and therefore may not have `platform_chat_id` in their metadata.
We need a non-destructive migration.
### Lazy migration strategy
For a room without `platform_chat_id`:
1. On the first operation that requires platform context, detect the missing mapping.
2. Create a new blank platform context for that room.
3. Persist the new `platform_chat_id` into room metadata.
4. Continue the requested operation normally.
This applies to:
- first normal message
- `!context`
- `!save`
- `!load`
- `!branch`
This avoids forcing users to recreate their rooms manually.
## Interface Changes
### Matrix metadata
Extend Matrix `room_meta` helpers to read and write `platform_chat_id`.
### Real platform client
`RealPlatformClient` must stop treating the current `chat_id` parameter as a purely local label when talking to the platform. The surface-facing call can still receive the local `chat_id`, but platform calls must use the resolved `platform_chat_id`.
Recommended integration direction:
- Matrix resolves the room mapping before calling the platform
- `RealPlatformClient` receives the platform context id it should use
This keeps the platform client simple and avoids giving it Matrix-specific storage responsibilities.
### Agent API wrapper
The wrapper must support platform calls that are explicitly context-aware:
- create new context
- branch context
- send message into a specific context
- save current context
- load saved context into a specific context
If upstream naming differs, the adapter layer should normalize those operations into stable local methods.
## Command Semantics in MVP
The MVP command set should evolve to this:
- `!new` creates a new room with a new empty platform context
- `!branch` creates a new room with a branched platform context
- `!context` reports the current room context
- `!save` saves the current room context for the user
- `!load` loads one of the user's saved contexts into the current room
Commands that do not have reliable backend support should remain hidden or explicitly marked unavailable.
## Error Handling
### Missing mapping
If `platform_chat_id` is missing:
- try lazy migration first
- only return an error if migration fails
### Platform create or branch failure
If the platform cannot create or branch a context:
- do not create partially-initialized room metadata
- return a user-facing error in the source room
- log enough detail to diagnose the backend failure
### Save and load failure
The surface must not claim success before the platform confirms success.
For MVP quality:
- user-facing text should say "request sent" only when confirmation is not available
- once platform confirmation exists, switch to real success or failure messages
## Testing
Add or update tests for:
- a new room gets a new `platform_chat_id`
- two rooms created with `!new` do not share context ids
- `!branch` creates a new room with a different `platform_chat_id` derived from the current one
- sending messages from two rooms uses different platform context ids
- saved contexts remain user-visible across rooms
- loading the same saved context into two different rooms affects those rooms independently afterward
- a legacy room without `platform_chat_id` lazily receives one on first use
- failures during create, branch, save, and load do not leave broken metadata behind
## Migration Path
This design preserves a clean future direction:
- Matrix continues to own its UX model
- Telegram can adopt the same `surface_chat -> platform_chat_id` mapping later
- when the platform matures further, more of the save/load/branch logic can move from prompts or local workarounds into real platform APIs
The key long-term boundary stays stable:
- surfaces own presentation and routing
- the platform owns context
- the integration layer owns the mapping

View file

@ -0,0 +1,252 @@
# Matrix Shared Workspace File Flow Design
## Goal
Bring the Matrix surface and `platform-agent` to a single file-handling model that matches the current platform runtime contract as closely as possible.
The result should be:
- Matrix receives user files and makes them visible to the agent through a shared `/workspace`
- `platform-agent` receives attachment paths, not ad hoc summaries or inline payloads
- the agent can send files back to the user through the surface via `send_file`
- local development and the default deployment path use the same storage contract
## Core Decision
The selected architecture is:
`Matrix surface <-> shared /workspace <-> platform-agent`
This means:
- the Matrix bot is responsible for downloading incoming Matrix media
- downloaded files are written into the same filesystem mounted into `platform-agent`
- the surface passes relative workspace paths to the agent as `attachments`
- the agent returns files to the user by emitting `MsgEventSendFile(path=...)`
This is the current platform-native direction and does not require new platform endpoints.
## Why This Decision
The current upstream platform changes already define the file contract:
- `MsgUserMessage.attachments` is `list[str]`
- each attachment is a path relative to `/workspace`
- the agent validates those paths against its configured backend root
- the agent can emit `send_file(path)` back to the client
That is not an upload API and not a remote blob contract. It is an explicit shared-workspace contract.
Trying to preserve the current separate-process launch model would force the surface to fake production behavior with inline text extraction, out-of-band path rewriting, or a future upload API that does not exist yet. That would increase the gap between our runtime and the platform runtime instead of reducing it.
## Scope
This design covers:
- shared workspace runtime for Matrix bot and `platform-agent`
- incoming Matrix file handling into shared storage
- attachment path propagation to `RealPlatformClient` and `AgentApi`
- outbound file delivery from agent to Matrix user
- local compose/dev workflow and README updates
This design does not cover:
- Telegram file flow
- encrypted Matrix media handling
- upload APIs on the platform side
- OCR, PDF parsing, or content extraction pipelines
- long-term object storage or file lifecycle policies beyond basic cleanup boundaries
## Runtime Contract
### Shared filesystem
Both containers must mount the same directory at `/workspace`.
Requirements:
- the Matrix bot can create files under `/workspace`
- `platform-agent` sees the same files at the same relative paths
- agent-originated files written under `/workspace` are readable by the Matrix bot
The contract is path-based, not URL-based.
### Attachment path format
The surface sends attachments to the agent as relative workspace paths, for example:
- `surfaces/matrix/<matrix_user_id>/<room_id>/inbox/20260420-153000-report.pdf`
- `surfaces/matrix/<matrix_user_id>/<room_id>/inbox/20260420-153200-photo.jpg`
Rules:
- paths must be relative to `/workspace`
- paths must be normalized before sending to the agent
- surface-owned uploads must live under a dedicated namespace to avoid collisions with agent-created files
## Data Flow
### Incoming file from Matrix user
1. Matrix receives `m.file`, `m.image`, `m.audio`, or `m.video`.
2. The Matrix bot resolves the target room and platform chat context as usual.
3. The Matrix bot downloads the media from Matrix.
4. The file is stored under `/workspace/surfaces/matrix/.../inbox/...`.
5. The outgoing platform call includes:
- original user text
- `attachments=[relative_path_1, ...]`
6. `platform-agent` validates that those files exist and exposes them to the agent through the upstream attachment mechanism.
Important detail:
- the surface should not rewrite the user message into a synthetic file summary unless the message body is empty
- when body is empty, the surface may send a minimal synthetic text such as `User sent one or more attachments.`
### Outbound file from agent to Matrix user
1. The agent uses `send_file(path)`.
2. `platform-agent` emits `MsgEventSendFile(path=...)`.
3. The Matrix integration catches that event.
4. The Matrix bot resolves the file inside shared `/workspace`.
5. The Matrix bot uploads the file to Matrix and sends the appropriate media message to the room.
Surface behavior:
- if MIME type and extension are known, send the closest native Matrix media type
- otherwise send as `m.file`
- user-visible failures must be explicit if the referenced file does not exist or cannot be uploaded
## Filesystem Layout
The Matrix surface owns a dedicated subtree:
```text
/workspace/
surfaces/
matrix/
<sanitized-user-id>/
<sanitized-room-id>/
inbox/
20260420-153000-report.pdf
```
Design constraints:
- sanitize user ids and room ids before using them as path components
- preserve the original filename in the final basename where possible
- prefix filenames with a timestamp or unique id to avoid collisions
This layout is intentionally surface-scoped. The agent may read these files, but the surface remains the owner of how inbound messenger files are organized.
## Components
### Matrix attachment storage helper
Add a focused helper module responsible for:
- building stable workspace-relative paths
- sanitizing path components
- downloading Matrix media into `/workspace`
- returning attachment metadata needed by the platform layer
This helper should not know about agent transport details beyond the final relative path output.
### Real platform client
`RealPlatformClient` must pass attachment relative paths through to `AgentApi.send_message(...)`.
It must also surface non-text agent events needed by the Matrix adapter, especially `MsgEventSendFile`.
### Agent API wrapper
`AgentApiWrapper` must be compatible with the modern upstream protocol:
- `/v1/agent_ws/{chat_id}/`
- `attachments` on outgoing user messages
- `MsgEventToolCallChunk`
- `MsgEventToolResult`
- `MsgEventCustomUpdate`
- `MsgEventSendFile`
- `MsgEventEnd`
### Matrix bot outbound renderer
The Matrix adapter must support sending files back to the room.
At minimum it needs:
- path resolution inside shared workspace
- Matrix upload of the local file
- send of an `m.file` or native media event with filename and MIME type
## Deployment Changes
### Compose
The repository root `docker-compose.yml` becomes the primary prod-like local runtime.
It should define at least:
- `matrix-bot`
- `platform-agent`
- one shared volume mounted as `/workspace` into both services
The default developer workflow should stop describing `platform-agent` as a separately started side process.
### Environment
The Matrix bot must connect to the in-compose `platform-agent` service by service name, not by assuming a separately launched localhost process.
The agent WebSocket configuration in docs and examples must match the modern upstream route.
## Error Handling
### Incoming files
If the Matrix bot cannot download or persist the file:
- do not send a broken attachment path to the agent
- return a user-visible error in the room
- log the Matrix event id, room id, and failure reason
### Outbound files
If the agent asks to send a missing file:
- log a structured warning with the requested path
- send a user-visible message that the file could not be delivered
### Shared workspace mismatch
If the runtime is misconfigured and `/workspace` is not actually shared:
- inbound attachments will fail agent-side path validation
- outbound `send_file` will fail surface-side file resolution
The implementation should make such failures obvious in logs rather than silently degrading to text-only behavior.
## Testing
The implementation must cover:
- Matrix media download writes into the expected workspace-relative path
- `RealPlatformClient` forwards attachment relative paths to the agent API
- Matrix plain messages with attachments preserve the original text while adding attachment paths
- empty-body attachment-only messages produce the synthetic text fallback
- `AgentApiWrapper` accepts `MsgEventSendFile` without treating it as unknown
- Matrix outbound file handling converts `MsgEventSendFile` into a Matrix upload/send call
- compose configuration mounts the same workspace into both containers
## Non-Goals
- no inline text extraction MVP
- no temporary URL-passing contract to the agent
- no fake “prod” mode with separate local filesystems
- no platform API additions in this phase
## Success Criteria
- the default local runtime uses a shared `/workspace`
- a user can send a file in Matrix and the agent receives it through upstream `attachments`
- the agent can emit `send_file(path)` and the Matrix user receives the file in the same room
- our runtime behavior matches the current platform contract closely enough that moving from local compose to production does not require redesigning file flow

View file

@ -0,0 +1,262 @@
# Matrix Staged Attachments Design
## Goal
Make file sending in the Matrix surface usable for an AI agent despite current Matrix client behavior, especially in Element where media is often sent immediately as separate events without a shared text composer.
The result should be:
- files can arrive before the user writes the actual instruction
- the surface stages those files instead of immediately sending them to the agent
- the next normal user message in the same chat commits all staged files as one agent turn
- the user can inspect and remove staged files with short chat commands
## Core Decision
The selected UX model is:
`incoming Matrix media -> staged attachments for (chat_id, user_id) -> next normal message commits them`
This means:
- attachment-only events do not immediately invoke the agent
- the bot acknowledges staged files with a service message
- the next normal user message sends text plus all currently staged files to the agent
- staged files are then cleared
## Why This Decision
Matrix natively models messages as separate events, and common clients do not provide a reliable "one text message with many attachments" composer flow.
In practice this causes two UX failures for an AI bot:
- users may send files first and only then write the task
- users may send multiple files as multiple independent Matrix events
If the surface treats each incoming file as a full agent turn, the bot becomes noisy and context-fragmented. If it ignores file-only messages, file handling feels broken.
Staging is the smallest surface-side abstraction that fixes both problems without fighting the Matrix event model.
## Scope
This design covers:
- staging inbound Matrix attachments before agent submission
- per-chat attachment state for a specific user
- user-facing service messages for staged attachments
- short commands for listing and removing staged files
- commit behavior on the next normal message
This design does not cover:
- edits or redactions of original Matrix media events as attachment controls
- cross-surface shared staging
- thread-aware staging beyond the existing `chat_id` boundary
- changes to the platform attachment contract
## State Model
### Staging key
Staged attachments are isolated by:
- `chat_id`
- `user_id`
This means:
- files staged by a user in one chat never appear in another chat
- files staged by one user do not mix with another user's files in the same room
### Staged attachment record
Each staged attachment must track at least:
- stable internal id
- display filename
- workspace-relative path
- MIME type if known
- created timestamp
User-visible commands operate on the current ordered list, not on internal ids.
### Lifecycle
A staged attachment is in exactly one of these states:
1. `staged`
2. `committed`
3. `removed`
Rules:
- only `staged` attachments appear in `!list`
- `committed` attachments are no longer user-removable
- `removed` attachments are excluded from future commits
## Inbound Behavior
### Attachment-only event
If the Matrix surface receives one or more file/media events from a user without a normal text message to commit them:
1. download each file into shared `/workspace`
2. add each file to the staged set for `(chat_id, user_id)`
3. do not call the agent yet
4. send a service acknowledgment message
### Service acknowledgment
The service message must communicate:
- the current staged attachment list with indices
- that the next normal message will be sent to the agent together with those files
- available commands: `!list`, `!remove <n>`, `!remove all`
Example shape:
```text
Staged attachments:
1. screenshot.png
2. invoice.pdf
Your next message will be sent to the agent with these files.
Commands: !list, !remove <n>, !remove all
```
### Burst handling
Matrix clients may send multiple files as separate consecutive events.
To avoid bot spam, service acknowledgments should be debounced over a short window and aggregated into one reply where feasible.
The acknowledgment must reflect the full current staged set, not only the most recently received file.
## Commit Behavior
### Commit trigger
The commit trigger is:
- the next normal user message in the same `(chat_id, user_id)` scope
Normal user message means:
- not a staging control command
- not a pure attachment event being staged
### Commit action
When a commit-triggering message arrives:
1. collect all currently staged attachments for `(chat_id, user_id)`
2. send the user text plus those attachments to the agent as one turn
3. mark all included staged attachments as `committed`
4. clear the staged set
After commit:
- the just-sent attachments must no longer appear in `!list`
- a later file upload starts a new staged set
## Commands
### `!list`
Shows the current staged attachment list for the user in the current chat.
If the list is empty, the response should be short and explicit.
### `!remove <n>`
Removes the staged attachment at the current 1-based index.
Behavior:
- if the index is valid, remove that staged attachment and return the updated staged list
- if the index is invalid, return a short error without repeating the list
### `!remove all`
Clears the entire staged set for the user in the current chat.
The response should be short and explicit.
## Ordering Rules
The staged list is ordered by staging time.
User-facing indices:
- are 1-based
- are recalculated from the current staged set
- may change after removals
Therefore:
- `!list` always shows the current authoritative numbering
- after a successful `!remove <n>`, the bot should reply with the refreshed list
## Error Handling
### Download failure
If a file cannot be downloaded or stored:
- do not add it to the staged set
- do not pretend it will be sent later
- send a short user-visible failure message
### Invalid command
If the command is malformed or uses an invalid index:
- return a short error
- do not commit staged attachments
- do not clear the staged set
### Agent submission failure
If commit fails when sending the text plus staged files to the agent:
- staged attachments must remain available for retry unless the failure is known to be irreversible
- the user-visible error should make it clear that the files were not consumed
This prevents silent loss of staged context.
## Interaction with Shared Workspace Design
This design assumes the shared-workspace contract defined in
[2026-04-20-matrix-shared-workspace-file-flow-design.md](/Users/a/MAI/sem2/lambda/surfaces-bot/docs/superpowers/specs/2026-04-20-matrix-shared-workspace-file-flow-design.md).
Specifically:
- staged files are stored in shared `/workspace`
- the final commit still passes workspace-relative paths to `platform-agent`
- staging changes only when the surface chooses to invoke the agent, not how attachments are represented
## Testing
The implementation must cover:
- file-only Matrix events are staged and do not immediately invoke the agent
- service acknowledgment includes staged filenames and command hints
- `!list` returns the current staged set for the correct `(chat_id, user_id)`
- `!remove <n>` removes the correct staged attachment and refreshes numbering
- `!remove all` clears the staged set
- invalid `!remove <n>` returns a short error and keeps state unchanged
- the next normal message commits all staged attachments with the text as one agent turn
- committed attachments disappear from staging after success
- failed commits preserve staged attachments
- staging in one chat does not leak into another chat
- staging for one user does not leak to another user in the same room
## Non-Goals
This design intentionally does not attempt to:
- emulate Telegram-style albums in Matrix
- rely on special support from Element or other Matrix clients
- introduce a rich interactive attachment management UI
The goal is a reliable chat-native workflow that works within Matrix's actual event model.

View file

@ -0,0 +1,318 @@
# Transport Layer Thin Adapter Design
## Цель
Упростить transport layer между Matrix surface и `platform-agent` до максимально production-like вида:
- использовать upstream `platform-agent_api.AgentApi` почти как есть
- убрать из surface-side клиента собственную интерпретацию stream semantics
- оставить в нашем коде только integration concerns:
- per-chat lifecycle
- per-chat serialization
- attachment path forwarding
- exception mapping в `PlatformError`
Это нужно, чтобы:
- восстановить чёткую границу ответственности между `surfaces` и платформой
- убрать из диагностики ложные факторы, внесённые нашей кастомной обёрткой
- получить честную картину реальных platform bugs до добавления любых policy-надстроек
## Контекст
Сейчас transport path состоит из:
- [sdk/agent_api_wrapper.py](/Users/a/MAI/sem2/lambda/surfaces-bot/sdk/agent_api_wrapper.py)
- [sdk/real.py](/Users/a/MAI/sem2/lambda/surfaces-bot/sdk/real.py)
Изначально `AgentApiWrapper` был создан по разумным причинам:
- поддержка переходного периода между разными версиями `platform-agent_api`
- унификация `base_url/url`
- создание per-chat client instances через `for_chat()`
- локальный учёт `tokens_used`
Позже в этот слой были добавлены уже не compatibility-функции, а собственные transport semantics:
- custom `_listen()`
- custom `send_message()`
- post-END drain window
- custom idle timeout
- event-kind reclassification
После этого `surfaces` перестал быть тонким клиентом платформы и начал вести себя как альтернативная реализация transport layer. Это делает диагностику platform bugs нечистой.
## Принципы дизайна
### 1. Transport должен быть скучным
Transport layer не должен:
- спасать поздние chunks
- лечить duplicate `END`
- придумывать собственные правила границы ответа
- по-своему классифицировать stream events сверх upstream client behavior
Если upstream stream protocol повреждён, мы должны видеть это как platform issue, а не скрывать его кастомной очередью.
### 2. Policy и transport разделяются
Transport:
- говорит с upstream API
- доставляет события
- закрывает соединение
Policy:
- решает, что считать recoverable failure
- нужна ли повторная попытка
- как сообщать ошибку пользователю
- нужно ли сбрасывать chat session
На первом этапе policy не расширяется. Мы сначала приводим transport к тонкому адаптеру, потом заново оцениваем реальные проблемы.
### 3. Session lifecycle остаётся на нашей стороне
Даже в thin-adapter модели `surfaces` по-прежнему отвечает за:
- кеширование client per chat
- один send lock на chat
- сброс мёртвой chat session после failure
- mapping upstream exceptions в `PlatformError`
Это не transport semantics, а integration lifecycle.
## Варианты
### Вариант A. Оставить текущий кастомный wrapper
Плюсы:
- уже работает на части сценариев
- содержит built-in mitigations против observed failures
Минусы:
- нарушает границу ответственности
- усложняет диагностику
- делает platform bug reports спорными
- содержит symptom-fix логику в transport layer
Вердикт: не подходит как production-like target.
### Вариант B. Thin upstream adapter
Плюсы:
- чистая архитектура
- честная диагностика upstream проблем
- минимальная собственная магия
Минусы:
- локальные mitigations исчезают
- если upstream client несовершенен, это сразу проявится
Вердикт: правильный первый этап.
### Вариант C. Thin adapter сейчас, outer policy layer потом
Плюсы:
- даёт production-like эволюцию
- не смешивает transport и resilience policy
- позволяет сначала увидеть реальные проблемы, потом адресовать только нужные
Минусы:
- требует двух фаз вместо одной
Вердикт: рекомендуемый путь.
## Рекомендуемая архитектура
### Слой 1. Upstream client
Источник истины:
- [external/platform-agent_api/lambda_agent_api/agent_api.py](/Users/a/MAI/sem2/lambda/surfaces-bot/external/platform-agent_api/lambda_agent_api/agent_api.py)
Мы принимаем его stream semantics как authoritative behavior.
### Слой 2. Thin adapter
Файл:
- [sdk/agent_api_wrapper.py](/Users/a/MAI/sem2/lambda/surfaces-bot/sdk/agent_api_wrapper.py)
После cleanup он должен содержать только:
- создание клиента через modern constructor
- `base_url` normalization, если это действительно нужно для наших env
- `for_chat(chat_id)` как factory convenience
- опционально thin storage для `last_tokens_used`, если это можно сделать без переопределения stream semantics
Он не должен переопределять:
- `_listen()`
- `send_message()`
- queue lifecycle
- post-END behavior
- timeout behavior
### Слой 3. Integration/session layer
Файл:
- [sdk/real.py](/Users/a/MAI/sem2/lambda/surfaces-bot/sdk/real.py)
Ответственность:
- кешировать chat client instances
- сериализовать sends по chat lock
- вызывать `disconnect_chat(chat_id)` после transport failure
- превращать upstream exceptions в `PlatformError`
- форвардить `attachments` как relative workspace paths
- собирать `MessageResponse` / `MessageChunk` для остального приложения
Этот слой не должен заниматься:
- исправлением broken stream boundaries
- custom post-END reconstruction
- поздним дренированием очереди
## Что удаляем
Из [sdk/agent_api_wrapper.py](/Users/a/MAI/sem2/lambda/surfaces-bot/sdk/agent_api_wrapper.py):
- custom `_listen()`
- custom `send_message()`
- `_drain_post_end_events()`
- `_event_kind()`
- `_is_kind()`
- `_is_text_event()`
- `_is_end_event()`
- `_is_send_file_event()`
- `_POST_END_DRAIN_MS`
- `_STREAM_IDLE_TIMEOUT_MS`
- debug logging, завязанное на наш собственный queue lifecycle
## Что оставляем
В thin adapter:
- `__init__()` для modern `base_url/chat_id`
- `_normalize_base_url()` только если нужен стабильный env input
- `for_chat(chat_id)`
В [sdk/real.py](/Users/a/MAI/sem2/lambda/surfaces-bot/sdk/real.py):
- `_get_chat_api()`
- `_get_chat_send_lock()`
- `_attachment_paths()`
- `disconnect_chat()`
- `_handle_chat_api_failure()`
- `send_message()`
- `stream_message()`
## Дополнительное упрощение
Если после cleanup мы считаем pinned upstream API обязательным, то из [sdk/real.py](/Users/a/MAI/sem2/lambda/surfaces-bot/sdk/real.py) можно убрать legacy-minded probing:
- `inspect.signature(send_message)`
- conditional fallback на старый `send_message(text)` без `attachments`
В этом случае `RealPlatformClient` всегда использует современный контракт:
- `send_message(text, attachments=...)`
Это ещё сильнее уменьшит ambiguity.
## Этапы миграции
### Этап 1. Cleanup до thin adapter
Делаем:
- сжимаем `sdk/agent_api_wrapper.py` до thin shim
- переносим всю допустимую resilience logic только в `sdk/real.py`
- удаляем тесты, которые закрепляют наши кастомные transport semantics
### Этап 2. Повторная верификация
Заново прогоняем:
- text-only flow
- staged attachments flow
- large image failure
- duplicate `END` behavior
- behavior after transport disconnect
На этом этапе мы честно увидим, что реально делает upstream transport.
### Этап 3. Опциональный outer policy layer
Только если после Этапа 2 это действительно нужно, добавляем policy поверх transport:
- request timeout целиком
- retry policy
- circuit-breaker-like behavior
Но это должно жить не в client wrapper, а выше, в integration layer.
## Тестовая стратегия
### Удаляем как нецелевые тесты
Больше не считаем нормой:
- post-END drain behavior
- recovery late chunks после `END`
- idle timeout внутри wrapper как часть client contract
### Оставляем и добавляем
Нужные guarantees:
1. создаётся отдельный client per chat
2. один chat сериализуется через lock
3. разные чаты не делят client instance
4. attachment paths уходят в `send_message(..., attachments=...)`
5. transport failure приводит к `disconnect_chat(chat_id)`
6. следующий запрос после failure открывает новую chat session
7. upstream exception превращается в `PlatformError`
## Риски
### 1. Может снова проявиться реальный upstream bug
Это не regression дизайна, а полезный результат cleanup.
### 2. Может исчезнуть локальная защита от зависших стримов
Это допустимо на первом этапе.
Если она действительно нужна, она должна вернуться как outer policy, а не как переписанный client transport.
### 3. Может выясниться, что даже thin wrapper не нужен
Если modern upstream `AgentApi` уже полностью покрывает наш use case, файл `sdk/agent_api_wrapper.py` можно будет заменить на маленький factory helper или убрать совсем.
## Критерии успеха
Результат считается успешным, если:
- transport layer в `surfaces` перестаёт иметь собственную stream semantics
- platform bug reports снова можно формулировать без сильного caveat про кастомный клиент
- Matrix real backend продолжает работать на text-only и attachments scenarios
- failure handling остаётся контролируемым, но больше не маскирует transport behavior платформы
## Решение
Принять путь:
- `Thin upstream adapter now`
- `Observe real behavior`
- `Add outer policy later only if needed`
Это наиболее близкий к production best practice вариант для текущего состояния проекта.

View file

@ -0,0 +1,336 @@
# Matrix Multi-Agent Routing Design
## Goal
Move the Matrix surface from a single hardcoded upstream agent to a user-selectable multi-agent model, while preserving the existing room-based UX and the current `PlatformClient` boundary.
The result should be:
- one Matrix bot can work with multiple upstream agents
- users can choose an agent from the full configured list
- each chat is bound to exactly one agent
- switching the selected agent does not silently retarget an existing chat
## Core Decision
The selected routing model is:
`user.selected_agent_id + room.agent_id + room.platform_chat_id`
This means:
- the user has one current selected agent
- each Matrix working room stores the agent it is bound to
- each Matrix working room stores its own `platform_chat_id`
- a room never changes agent implicitly
- the shared `PlatformClient` protocol remains unchanged
- Matrix multi-agent routing is implemented by a single routing facade that delegates to per-agent real clients
## Why This Decision
The current Matrix adapter already separates:
- user-facing room organization
- local chat labels such as `C1`, `C2`, `C3`
- platform-facing conversation identity via `platform_chat_id`
Adding multi-agent support should preserve that shape instead of replacing it.
If routing depended only on the current user selection, then an old room could start talking to a different agent after a switch. That would make room history and backend context hard to reason about. Binding an agent to the room keeps the conversation model explicit.
## Scope
This design covers:
- agent selection by the user inside the Matrix surface
- durable storage of the selected agent
- durable storage of the room-bound agent
- routing normal messages and context commands to the correct upstream agent
- behavior when a room becomes stale after an agent switch
This design does not cover:
- per-agent workspace isolation
- platform-side agent lifecycle or memory persistence
- per-user allowlists for available agents
- Telegram or other surfaces
## Configuration Model
### Agent registry
Available agents are defined in a local config file loaded once at bot startup.
Example:
```yaml
agents:
- id: agent-1
label: Analyst
- id: agent-2
label: Research
- id: agent-3
label: Ops
```
Rules:
- every entry must have a stable `id`
- every entry must have a user-visible `label`
- all configured agents are selectable by all users
- config changes apply only after bot restart
### Startup validation
If the agent config is missing, empty, or invalid, the Matrix bot must fail fast on startup with a clear operator error.
## Durable State Model
### User-level state
User metadata keeps the current selected agent.
Example `matrix_user:*` shape:
```json
{
"space_id": "!space:example.org",
"next_chat_index": 4,
"selected_agent_id": "agent-2"
}
```
Meaning:
- `selected_agent_id` controls future chat creation and activation of an unbound room
- `selected_agent_id` does not rewrite already bound rooms
### Room-level state
Room metadata stores the agent bound to that chat.
Example `matrix_room:*` shape:
```json
{
"room_type": "chat",
"chat_id": "C3",
"display_name": "Чат 3",
"matrix_user_id": "@alice:example.org",
"space_id": "!space:example.org",
"platform_chat_id": "42",
"agent_id": "agent-2"
}
```
Rules:
- one room binds to exactly one `agent_id`
- one room binds to exactly one current `platform_chat_id`
- once a room becomes stale after an agent switch, it never becomes active again
## Runtime Semantics
### `!start`
`!start` remains lightweight:
- if no agent is selected, the bot explains that an agent must be selected before normal messaging
- if an agent is already selected, the bot reports the current selection and reminds the user that `!new` creates a new room under that agent
### `!agent`
Introduce an agent-selection command.
Behavior:
- `!agent` shows the available agent list
- agent selection stores `selected_agent_id` in user metadata
- after a successful switch, the bot tells the user that existing chats bound to another agent are stale and that `!new` is required for continued work
The exact UI can be text-first for MVP. A richer UI can be added later without changing the state model.
### Normal message without selected agent
If the user has not selected an agent yet:
- do not call the platform
- return the available agent list
- ask the user to choose one first
This is an intentional one-time routing handshake, not an accidental fallback.
In a multi-agent deployment, the surface must not silently guess which agent an unbound user should talk to.
### Selecting an agent inside an unbound chat
If the current room has never been bound to any agent:
- store the new `selected_agent_id` for the user
- bind the current room to that same `agent_id`
- allow the room to become the active working chat immediately
This avoids forcing `!new` for the user's first usable chat.
### `!new`
`!new` creates a new working room under the current selected agent.
Behavior:
1. require `selected_agent_id`
2. create the new Matrix room
3. allocate a new `platform_chat_id`
4. store `agent_id = selected_agent_id` in the new room metadata
### Normal message in an unbound room with selected agent
If a room exists but has no `agent_id` yet and the user already has `selected_agent_id`:
- bind the room to `selected_agent_id`
- ensure it has `platform_chat_id`
- continue normal message dispatch
### Normal message in a bound room
If the room already has `agent_id` and it matches the current selected agent:
- route the message to that `agent_id`
- use the room's `platform_chat_id`
### Stale room after agent switch
If the room's bound `agent_id` differs from the user's current `selected_agent_id`:
- do not call the platform
- treat the room as stale
- return a short message telling the user that this chat belongs to the old agent and that they must use `!new`
### Returning to a previously selected agent
If the user later selects an old agent again:
- previously stale rooms do not become valid again
- the user must still create a fresh room via `!new`
## Routing and Component Changes
### Agent registry loader
Add a small loader responsible for:
- reading `agents.yaml`
- validating ids and labels
- exposing a read-only registry to runtime code
The runtime should not parse YAML ad hoc during message handling.
### Matrix runtime pre-check
Before dispatching a normal message, the Matrix runtime must resolve:
- whether the user has `selected_agent_id`
- whether the current room already has `agent_id`
- whether the room can be bound now
- whether the room is stale
This pre-check happens before handing the message to the existing dispatcher path.
### Routed platform client
The selected implementation keeps the shared `PlatformClient` protocol unchanged.
The Matrix runtime owns one routing-aware facade, for example `RoutedPlatformClient`, that implements `PlatformClient` and delegates to agent-specific real clients.
Responsibilities:
- resolve the current room binding from local Matrix metadata
- translate a local Matrix logical chat id into the room's `platform_chat_id`
- choose the correct per-agent delegate for the room's bound `agent_id`
- keep `get_or_create_user`, `get_settings`, and `update_settings` behavior stable for the rest of the runtime
This keeps the multi-agent logic inside the Matrix integration boundary instead of pushing agent selection into the shared protocol.
### Real platform bridge delegates
The current real backend path hardcodes a single runtime-level `agent_id`.
That must be replaced with per-agent delegates hidden behind the routing facade.
The selected design is:
- `RealPlatformClient` remains the low-level direct-agent delegate for one configured `agent_id`
- the routing facade holds or creates one `RealPlatformClient` delegate per configured agent
- `send_message(...)` and `stream_message(...)` on the facade resolve the room target and forward the call to the matching delegate
- the delegate creates a fresh upstream `AgentApi` for its configured `agent_id`
- no long-lived `AgentApi` instances are cached by user
This preserves the current fresh-connection-per-request behavior while avoiding a protocol break for Telegram or other surfaces.
## Error Handling
### Missing or invalid selected agent
If `selected_agent_id` is absent:
- ask the user to select an agent
If `selected_agent_id` points to an agent that no longer exists in config:
- treat the selection as invalid
- ask the user to select again
### Missing room binding
If the room has no `agent_id`:
- bind it only when the user has a valid current selection
- otherwise return the selection prompt
### Stale room
If the room is stale:
- do not attempt fallback routing
- do not silently rewrite room metadata
- instruct the user to run `!new`
### Invalid config
If the bot cannot load a valid agent registry:
- fail at startup
- do not start in degraded single-agent mode
## Testing Expectations
Tests for this design should prove:
- config parsing and startup validation
- selecting an agent persists `selected_agent_id`
- selecting an agent inside an unbound room activates that room
- `!new` binds the new room to the selected agent
- messages in a bound room use that room's `agent_id`
- stale rooms reject normal messaging with a clear `!new` instruction
- returning to the same agent later does not revive stale rooms
## Migration Notes
Existing rooms may have `platform_chat_id` but no `agent_id`.
For this MVP, treat those rooms as legacy-unbound rooms:
- if the user has a valid selected agent, the room may be bound on first use
- if no agent is selected, the room prompts for selection first
No automatic migration across agents is introduced.
### Existing users without `selected_agent_id`
Existing users upgraded from the single-agent model may have working rooms but no stored `selected_agent_id`.
For this MVP, that is handled explicitly:
- normal messaging is paused until the user selects an agent
- the first valid selection can bind an unbound room immediately
- the surface does not auto-assign a default agent in a multi-agent config
This is intentional. Once more than one agent exists, silent migration would be ambiguous and could route a user to the wrong backend target.

View file

@ -0,0 +1,258 @@
# Matrix Surface Restart State Persistence Design
## Goal
Make the Matrix surface survive a normal restart or container recreate without losing the minimal state required to keep working as a bot.
The result should be:
- after restart, the bot can still answer messages and execute commands
- the bot remembers the selected agent for each user
- the bot remembers which agent and `platform_chat_id` each room is bound to
- temporary UX flows may be lost without being treated as a bug
## Core Decision
The selected persistence model is:
`durable surface state only`
This means:
- persist only the state needed for routing and normal command handling
- do not persist temporary UI and wizard state
- require persistent local storage for the surface
- do not attempt recovery if those volumes are lost
## Why This Decision
The Matrix surface already has two different classes of state:
- stable local state that defines how rooms and users are routed
- temporary UX state that exists only to complete short-lived interactions
Trying to make all temporary UX state survive restart would add complexity and edge cases without improving the core requirement: the bot should still function normally after restart.
The chosen design keeps persistence aligned with what the surface actually owns:
- Matrix-side metadata and routing state are durable
- agent conversation memory is the platform's responsibility
- lost local volumes are treated as environment reset, not as an auto-recovery scenario
## Scope
This design covers:
- which Matrix surface data must persist across restart
- where that data lives
- how restart behavior interacts with multi-agent routing
- what state is intentionally non-durable
This design does not cover:
- platform-side persistence of agent memory
- workspace isolation between multiple agents
- automatic reconstruction after total local volume loss
- persistence of temporary UX flows
## Persistence Boundary
### Durable state
The Matrix surface must persist:
- `matrix_user:*`
- `matrix_room:*`
- `chat:*`
- `PLATFORM_CHAT_SEQ_KEY`
- `selected_agent_id`
- room-bound `agent_id`
- room-bound `platform_chat_id`
This is the minimal state required so that, after restart, the surface can:
- identify the user
- identify the room
- determine which agent should receive a message
- determine which `platform_chat_id` should be used
- continue allocating new `platform_chat_id` values without reusing an already issued sequence number
### Non-durable state
The Matrix surface does not need to persist:
- staged attachments
- pending `!load` selection
- pending `!yes/!no` confirmation
- any temporary service UI step
- live `AgentApi` instances or connection objects
After restart, those flows may be lost. The bot only needs to remain operational.
## Storage Model
### Surface durable storage
The Matrix surface must use persistent storage for:
- `lambda_matrix.db`
- `matrix_store`
`lambda_matrix.db` stores the local key-value state used by the surface.
`matrix_store` stores Matrix client state needed by `nio`.
These paths must be backed by persistent container storage in normal deployments.
### Shared `/workspace`
The current local runtime also uses `/workspace`, but workspace behavior is outside the scope of this design.
For this document, the only requirement is:
- do not make restart persistence depend on solving per-agent workspace isolation first
## Restart Assumptions
This design assumes:
- normal restart or redeploy with persistent local volumes still present
This design does not assume:
- automatic recovery after deleting or losing those volumes
If the relevant volumes are lost, the environment is treated as reset.
## Data Model Requirements
### User metadata
User metadata remains the durable location for user-level routing state.
Example:
```json
{
"space_id": "!space:example.org",
"next_chat_index": 4,
"selected_agent_id": "agent-2"
}
```
### Room metadata
Room metadata remains the durable location for room-level routing state.
Example:
```json
{
"room_type": "chat",
"chat_id": "C3",
"display_name": "Чат 3",
"matrix_user_id": "@alice:example.org",
"space_id": "!space:example.org",
"platform_chat_id": "42",
"agent_id": "agent-2"
}
```
### Platform chat sequence
The global `PLATFORM_CHAT_SEQ_KEY` remains part of durable surface state.
Its purpose is:
- allocate monotonically increasing `platform_chat_id` values
- avoid reusing a previously issued platform chat identifier during normal restart or redeploy
This sequence must be stored in the same durable surface store as the room and user metadata.
## Runtime Semantics After Restart
After restart, the Matrix surface must:
1. load the durable Matrix store
2. load the durable surface key-value state
3. load the agent registry config
4. resume normal room routing using persisted `selected_agent_id`, `agent_id`, and `platform_chat_id`
Expected behavior:
- a user with a valid previously selected agent does not need to reselect it
- a room previously bound to an agent remains bound to that agent
- normal messages and commands continue to work
### Lost temporary UX state
If the bot restarts during a transient UX flow:
- staged attachments may disappear
- pending `!load` selections may disappear
- pending confirmations may disappear
This is acceptable and should not block normal operation after restart.
## Interaction With Multi-Agent Routing
The multi-agent design introduces new durable state that must survive restart:
- `selected_agent_id` on the user
- `agent_id` on the room
- `PLATFORM_CHAT_SEQ_KEY` in the surface store
Restart persistence and multi-agent routing therefore belong together.
Without durable storage for those fields, a restart would make room routing ambiguous.
## Failure Handling
### Missing durable surface store
If the durable store paths are missing because the environment was reset:
- do not attempt to reconstruct a full working state from scratch in this design
- treat startup as a clean environment
- allow normal onboarding flows to begin again
### Invalid durable references
If persisted `selected_agent_id` or room `agent_id` references an agent no longer present in config:
- do not crash
- treat the selection or room binding as invalid
- ask the user to select a valid agent again
### Platform conversation memory
If the upstream platform loses agent memory across restart:
- that is outside the surface persistence boundary
- the surface must still route correctly
- platform memory persistence remains a platform responsibility
## Testing Expectations
Tests for this design should prove:
- `selected_agent_id` survives restart through durable local storage
- room `agent_id` and `platform_chat_id` survive restart through durable local storage
- the bot can route messages correctly after restart without user reconfiguration
- missing temporary UX state does not break normal messaging and command handling
- invalid persisted agent references degrade into reselection prompts rather than crashes
## Operational Notes
For the Matrix surface to survive restart in the intended way, deployment must persist:
- `lambda_matrix.db`
- `matrix_store`
This is a deployment requirement, not an optional optimization.
The design intentionally stops there. It does not require:
- hot reload of agent config
- recovery after total local state loss
- persistence of temporary UX flows
- a solved multi-agent workspace story

View file

@ -38,9 +38,10 @@ surfaces-bot/
converter.py — matrix-nio Event → IncomingEvent, OutgoingEvent → Matrix API
bot.py — точка входа, клиент
platform/
interface.py — Protocol: PlatformClient
mock.py — MockPlatformClient
sdk/
interface.py — Protocol: PlatformClient (контракт к SDK)
real.py — RealPlatformClient (через AgentApi)
mock.py — MockPlatformClient (для локальных тестов)
```
---
@ -140,7 +141,7 @@ class UIButton:
```
Telegram рендерит это как InlineKeyboard.
Matrix рендерит как текст с описанием реакций или HTML-кнопки.
Matrix рендерит как текст (в MVP).
### OutgoingNotification
Асинхронное уведомление — агент закончил долгую задачу.
@ -209,7 +210,7 @@ class ConfirmationRequest:
```
Telegram показывает как Inline-кнопки.
Matrix показывает как реакции 👍 / ❌.
Matrix показывает как запрос для `!yes` / `!no`.
Ядро не знает как именно — только получает `IncomingCallback` с `action: "confirm"`.
---
@ -304,9 +305,9 @@ class PlatformClient(Protocol):
async def update_settings(self, user_id: str, action: Any) -> None: ...
```
Бот **не управляет lifecycle контейнеров** — это делает Master (платформа).
Бот передаёт `user_id` + `chat_id` + текст; Master сам решает нужно ли поднять контейнер, смонтировать `C1/`/`C2/`, запустить агента.
Бот **не управляет lifecycle контейнеров** агентов. Запуск/перезапуск агентов — ответственность платформы.
Бот передаёт `user_id` + `chat_id` + текст.
`MockPlatformClient` реализует этот протокол сейчас.
Реальный SDK — тоже реализует этот протокол, заменяя один файл.
Адаптеры поверхностей и ядро не меняются вообще.
`MockPlatformClient` реализует этот протокол для локальных тестов.
Реальный SDK используется через `RealPlatformClient` (`sdk/real.py`), который подключается к `AgentApi` по WebSocket.
Адаптеры поверхностей и ядро не меняются вообще, привязка идёт через `config/matrix-agents.yaml`.

View file

@ -1,5 +1,8 @@
# Telegram — описание прототипа
> **ВНИМАНИЕ: Telegram-адаптер не является частью текущего MVP-деплоя.**
> Код Telegram-поверхности находится в отдельной ветке `feat/telegram-adapter`. Данный документ описывает возможности этого адаптера, но многие концепции (например, AuthFlow и MockPlatformClient) устарели по отношению к актуальной архитектуре `main`.
## Концепция
Один бот, несколько чатов через Topics в Forum-группе.

View file

@ -1,65 +0,0 @@
# User Flow — Lambda Bot
> **Статус:** ШАБЛОН — заполняет @architect после исследований
> **Зависит от:** docs/research/telegram-flows.md, docs/research/competitor-ux.md
---
## Основной сценарий (happy path)
```mermaid
sequenceDiagram
actor User
participant Bot as Telegram/Matrix Bot
participant Platform as Lambda Platform (Master)
User->>Bot: /start
Bot->>Platform: GET /users/{tg_id}?platform=telegram
Platform-->>Bot: {user_id, is_new}
alt Новый пользователь
Bot->>User: Приветствие + инструкция
else Существующий пользователь
Bot->>User: Добро пожаловать обратно
end
loop Диалог (бот не управляет сессиями — Master делает это автоматически)
User->>Bot: Сообщение в чат C1/C2/...
Bot->>Platform: POST /users/{user_id}/chats/{chat_id}/messages
Note over Platform: Master поднимает контейнер,<br/>монтирует нужный чат, запускает агента
Platform-->>Bot: {message_id, response, tokens_used}
Bot->>User: Ответ агента
end
```
---
## Состояния FSM (Telegram)
```mermaid
stateDiagram-v2
[*] --> Unauthenticated: первый контакт
Unauthenticated --> Idle: /start (auth confirmed)
Idle --> WaitingResponse: сообщение пользователя
WaitingResponse --> Idle: ответ получен
WaitingResponse --> Error: ошибка платформы
Idle --> Idle: /new (создан новый чат)
Idle --> ConfirmAction: агент запрашивает подтверждение
ConfirmAction --> Idle: подтверждено / отменено
Error --> Idle: /start
```
---
## Открытые вопросы
> Заполняет @researcher и @architect после исследований
- [ ] Как выглядит онбординг новых пользователей у конкурентов?
- [ ] Нужна ли кнопка "Новая сессия" или сессия стартует автоматически?
- [ ] Что показываем пока агент думает (typing indicator)?
- [ ] Как обрабатываем timeout ответа от платформы?

View file

@ -1,174 +0,0 @@
# Surfaces team — Lambda Lab 3.0
Telegram и Matrix боты для взаимодействия пользователя с AI-агентом Lambda.
## Правило №1: не быть ждуном
Платформа (SDK от Азамата) ещё не готова. Это **не блокер**.
- Все вызовы платформы — через `platform/interface.py` (Protocol)
- Реализация сейчас — `platform/mock.py` (MockPlatformClient)
- При подключении реального SDK — меняем только `platform/mock.py`
- Архитектурные решения принимаем сами, фиксируем в `docs/api-contract.md`
---
## Архитектура
```
surfaces-bot/
core/
protocol.py — унифицированные структуры (IncomingMessage, OutgoingUI, ...)
handler.py — EventDispatcher: IncomingEvent → OutgoingEvent (общее для всех ботов)
handlers/ — обработчики по типам событий (start, message, chat, settings, callback)
store.py — StateStore Protocol + InMemoryStore + SQLiteStore
chat.py — ChatManager: метаданные чатов C1/C2/C3
auth.py — AuthManager: AuthFlow
settings.py — SettingsManager: SettingsAction
adapter/
telegram/ — aiogram адаптер
converter.py — aiogram Event → IncomingEvent и обратно
bot.py — точка входа
handlers/ — aiogram роутеры
keyboards/ — инлайн-клавиатуры
states.py — FSM состояния
matrix/ — matrix-nio адаптер
converter.py — matrix-nio Event → IncomingEvent и обратно
bot.py — точка входа
handlers/ — обработчики событий
platform/
interface.py — Protocol: PlatformClient (контракт к SDK)
mock.py — MockPlatformClient (заглушка)
docs/ — вся документация
tests/ — pytest тесты
.claude/agents/ — конфиги агентов
```
Подробно об унификации: `docs/surface-protocol.md`
Telegram функционал: `docs/telegram-prototype.md`
Matrix функционал: `docs/matrix-prototype.md`
---
## Агенты
| Агент | Когда запускать | Модель | Токены |
|-------|----------------|--------|--------|
| `@researcher` | Изучить API, найти примеры | Haiku | ~дёшево |
| `@architect` | Спроектировать решение | Sonnet | ~средне |
| `@tg-developer` | Писать код Telegram-адаптера | Sonnet | ~средне |
| `@matrix-developer` | Писать код Matrix-адаптера | Sonnet | ~средне |
| `@core-developer` | Писать core/ и platform/ | Sonnet | ~средне |
| `@reviewer` | Проверить код перед PR | Sonnet | ~средне |
**Важно (Pro-лимиты):** не запускай больше двух Sonnet-агентов одновременно.
Haiku можно запускать параллельно сколько угодно.
---
## Стратегия параллельной разработки
Два бота разрабатываются параллельно, но через общее ядро.
### Порядок работы
```
1. core/ — сначала (однократно, все ждут)
@core-developer пишет protocol.py, handler.py, session.py, auth.py, settings.py
2. platform/ — сразу после core/
@core-developer пишет interface.py и mock.py
3. adapter/telegram/ и adapter/matrix/ — параллельно
@tg-developer → adapter/telegram/
@matrix-developer → adapter/matrix/
Не пересекаются по файлам — можно одновременно в разных терминалах.
```
### Что можно делать одновременно (разные терминалы)
```bash
# Терминал 1 — Telegram адаптер
claude "Use @tg-developer to implement adapter/telegram/handlers/start.py"
# Терминал 2 — Matrix адаптер (параллельно)
claude "Use @matrix-developer to implement adapter/matrix/handlers/start.py"
```
### Что нельзя делать одновременно
- Два агента в одном файле
- @core-developer параллельно с @tg-developer или @matrix-developer
(core/ должен быть готов до адаптеров)
- Больше двух Sonnet-агентов одновременно (Pro-лимит)
---
## Git worktree workflow
Каждая фича в отдельном worktree — адаптеры не мешают друг другу:
```bash
# Создать worktrees для параллельной работы
git worktree add .worktrees/telegram -b feat/telegram-adapter
git worktree add .worktrees/matrix -b feat/matrix-adapter
# Работать в каждом независимо
cd .worktrees/telegram && claude "Use @tg-developer to ..."
cd .worktrees/matrix && claude "Use @matrix-developer to ..."
# Смержить когда готово
git checkout main
git merge feat/telegram-adapter
git merge feat/matrix-adapter
```
---
## Команды запуска
```bash
# Установить зависимости
uv sync
# Запустить тесты
pytest tests/ -v
# Запустить только тесты Telegram
pytest tests/adapter/telegram/ -v
# Запустить только тесты Matrix
pytest tests/adapter/matrix/ -v
# Запустить только тесты ядра
pytest tests/core/ -v
# Запустить Telegram бота
python -m adapter.telegram.bot
# Запустить Matrix бота
python -m adapter.matrix.bot
```
---
## Переменные окружения
```bash
cp .env.example .env
```
Никогда не коммить `.env`.
---
## Экономия токенов (Pro-лимиты)
- Исследования → всегда `@researcher` (Haiku), не Sonnet
- Точечные правки в одном файле → напрямую без агента
- Ревью → только перед PR, не после каждого коммита
- Длинный контекст → дай агенту конкретный файл, не весь проект
- Если агент "завис" в рассуждениях → прерви, переформулируй задачу точнее

View file

@ -15,12 +15,15 @@ dependencies = [
"structlog>=24.1",
"python-dotenv>=1.0",
"httpx>=0.27",
"aiohttp>=3.9",
"pyyaml>=6.0",
]
[project.optional-dependencies]
dev = [
"pytest>=8.0",
"pytest-asyncio>=0.23",
"pytest-aiohttp>=1.0",
"pytest-cov>=4.1",
"ruff>=0.3",
"mypy>=1.8",

View file

@ -0,0 +1,9 @@
__all__ = ["RealPlatformClient"]
def __getattr__(name: str):
if name == "RealPlatformClient":
from sdk.real import RealPlatformClient
return RealPlatformClient
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

1
sdk/agent_session.py Normal file
View file

@ -0,0 +1 @@
"""Compatibility stub: AgentSessionClient was replaced by direct AgentApi usage in Phase 4."""

View file

@ -1,10 +1,11 @@
# platform/interface.py
from __future__ import annotations
from collections.abc import AsyncIterator
from datetime import datetime
from typing import Any, AsyncIterator, Literal, Protocol
from typing import Any, Literal, Protocol
from pydantic import BaseModel
from pydantic import BaseModel, Field
class User(BaseModel):
@ -17,10 +18,11 @@ class User(BaseModel):
class Attachment(BaseModel):
url: str
mime_type: str
url: str | None = None
mime_type: str | None = None
size: int | None = None
filename: str | None = None
workspace_path: str | None = None
class MessageResponse(BaseModel):
@ -28,10 +30,12 @@ class MessageResponse(BaseModel):
response: str
tokens_used: int
finished: bool
attachments: list[Attachment] = Field(default_factory=list)
class MessageChunk(BaseModel):
"""Один кусок стримингового ответа. При sync-режиме — единственный чанк с finished=True."""
message_id: str
delta: str
finished: bool
@ -48,6 +52,7 @@ class UserSettings(BaseModel):
class AgentEvent(BaseModel):
"""Webhook-уведомление от платформы — агент закончил долгую задачу."""
event_id: str
user_id: str
chat_id: str
@ -94,4 +99,5 @@ class PlatformClient(Protocol):
class WebhookReceiver(Protocol):
"""Регистрируется в боте. Платформа зовёт нас когда агент закончил долгую задачу."""
async def on_agent_event(self, event: AgentEvent) -> None: ...

View file

@ -4,8 +4,9 @@ from __future__ import annotations
import asyncio
import random
import uuid
from collections.abc import AsyncIterator
from datetime import UTC, datetime
from typing import Any, AsyncIterator, Literal
from typing import Any, Literal
import structlog
@ -222,14 +223,16 @@ class MockPlatformClient:
response = f"[MOCK] Ответ на: «{preview}»{attachment_note}"
tokens = len(text.split()) * 2
self._messages[key].append({
"message_id": message_id,
"user_text": text,
"response": response,
"tokens_used": tokens,
"finished": True,
"created_at": datetime.now(UTC).isoformat(),
})
self._messages[key].append(
{
"message_id": message_id,
"user_text": text,
"response": response,
"tokens_used": tokens,
"finished": True,
"created_at": datetime.now(UTC).isoformat(),
}
)
return message_id, response, tokens
async def _latency(self, min_ms: int = 10, max_ms: int = 80) -> None:

129
sdk/prototype_state.py Normal file
View file

@ -0,0 +1,129 @@
from __future__ import annotations
from datetime import UTC, datetime
from typing import Any
from sdk.interface import User, UserSettings
# Keep the prototype backend self-contained; do not import these from sdk.mock.
DEFAULT_SKILLS: dict[str, bool] = {
"web-search": True,
"fetch-url": True,
"email": False,
"browser": False,
"image-gen": False,
"files": True,
}
DEFAULT_SAFETY: dict[str, bool] = {
"email-send": True,
"file-delete": True,
"social-post": True,
}
DEFAULT_SOUL: dict[str, str] = {"name": "Лямбда", "instructions": ""}
DEFAULT_PLAN: dict[str, Any] = {
"name": "Beta",
"tokens_used": 0,
"tokens_limit": 1000,
}
class PrototypeStateStore:
def __init__(self) -> None:
self._users: dict[str, User] = {}
self._settings: dict[str, dict[str, Any]] = {}
self._saved_sessions: dict[str, list[dict[str, str]]] = {}
self._context_last_tokens_used: dict[str, int] = {}
self._context_current_session: dict[str, str] = {}
async def get_or_create_user(
self,
external_id: str,
platform: str,
display_name: str | None = None,
) -> User:
key = f"{platform}:{external_id}"
existing = self._users.get(key)
if existing is not None:
stored = existing.model_copy(update={"is_new": False})
self._users[key] = stored
return stored.model_copy()
user = User(
user_id=f"usr-{platform}-{external_id}",
external_id=external_id,
platform=platform,
display_name=display_name,
created_at=datetime.now(UTC),
is_new=True,
)
self._users[key] = user.model_copy(update={"is_new": False})
return user.model_copy()
async def get_settings(self, user_id: str) -> UserSettings:
stored = self._settings.get(user_id, {})
return UserSettings(
skills={**DEFAULT_SKILLS, **stored.get("skills", {})},
connectors=dict(stored.get("connectors", {})),
soul={**DEFAULT_SOUL, **stored.get("soul", {})},
safety={**DEFAULT_SAFETY, **stored.get("safety", {})},
plan={**DEFAULT_PLAN, **stored.get("plan", {})},
)
async def update_settings(self, user_id: str, action: Any) -> None:
settings = self._settings.setdefault(user_id, {})
if action.action == "toggle_skill":
skills = settings.setdefault("skills", DEFAULT_SKILLS.copy())
skills[action.payload["skill"]] = action.payload.get("enabled", True)
elif action.action == "set_soul":
soul = settings.setdefault("soul", DEFAULT_SOUL.copy())
soul[action.payload["field"]] = action.payload["value"]
elif action.action == "set_safety":
safety = settings.setdefault("safety", DEFAULT_SAFETY.copy())
safety[action.payload["trigger"]] = action.payload.get("enabled", True)
async def add_saved_session(
self,
user_id: str,
name: str,
*,
source_context_id: str | None = None,
) -> None:
sessions = self._saved_sessions.setdefault(user_id, [])
session = {"name": name, "created_at": datetime.now(UTC).isoformat()}
if source_context_id is not None:
session["source_context_id"] = source_context_id
sessions.append(session)
async def list_saved_sessions(self, user_id: str) -> list[dict[str, str]]:
return [dict(session) for session in self._saved_sessions.get(user_id, [])]
async def get_last_tokens_used_for_context(self, context_id: str) -> int:
return self._context_last_tokens_used.get(context_id, 0)
async def set_last_tokens_used_for_context(self, context_id: str, tokens: int) -> None:
self._context_last_tokens_used[context_id] = tokens
async def get_current_session_for_context(self, context_id: str) -> str | None:
return self._context_current_session.get(context_id)
async def set_current_session_for_context(self, context_id: str, name: str) -> None:
self._context_current_session[context_id] = name
async def clear_current_session_for_context(self, context_id: str) -> None:
self._context_current_session.pop(context_id, None)
async def get_last_tokens_used(self, context_id: str) -> int:
return await self.get_last_tokens_used_for_context(context_id)
async def set_last_tokens_used(self, context_id: str, tokens: int) -> None:
await self.set_last_tokens_used_for_context(context_id, tokens)
async def get_current_session(self, context_id: str) -> str | None:
return await self.get_current_session_for_context(context_id)
async def set_current_session(self, context_id: str, name: str) -> None:
await self.set_current_session_for_context(context_id, name)
async def clear_current_session(self, context_id: str) -> None:
await self.clear_current_session_for_context(context_id)

273
sdk/real.py Normal file
View file

@ -0,0 +1,273 @@
from __future__ import annotations
import asyncio
import os
import re
from collections.abc import AsyncIterator
from pathlib import Path
from urllib.parse import urljoin, urlsplit, urlunsplit
import structlog
from sdk.interface import (
Attachment,
MessageChunk,
MessageResponse,
PlatformClient,
PlatformError,
User,
UserSettings,
)
from sdk.prototype_state import PrototypeStateStore
from sdk.upstream_agent_api import AgentApi, MsgEventSendFile, MsgEventTextChunk
logger = structlog.get_logger(__name__)
def _ws_debug_enabled() -> bool:
value = os.environ.get("SURFACES_DEBUG_WS", "")
return value.strip().lower() in {"1", "true", "yes", "on"}
class RealPlatformClient(PlatformClient):
def __init__(
self,
agent_id: str,
agent_base_url: str,
prototype_state: PrototypeStateStore,
platform: str = "matrix",
agent_api_cls=AgentApi,
) -> None:
self._agent_id = agent_id
self._raw_agent_base_url = agent_base_url
self._agent_base_url = self._normalize_agent_base_url(agent_base_url)
self._agent_api_cls = agent_api_cls
self._prototype_state = prototype_state
self._platform = platform
self._chat_send_locks: dict[str, asyncio.Lock] = {}
if _ws_debug_enabled():
logger.warning(
"agent_client_initialized",
agent_id=self._agent_id,
platform=self._platform,
raw_base_url=self._raw_agent_base_url,
normalized_base_url=self._agent_base_url,
)
@property
def agent_id(self) -> str:
return self._agent_id
@property
def agent_base_url(self) -> str:
return self._agent_base_url
def _get_chat_send_lock(self, chat_id: str) -> asyncio.Lock:
chat_key = str(chat_id)
lock = self._chat_send_locks.get(chat_key)
if lock is None:
lock = asyncio.Lock()
self._chat_send_locks[chat_key] = lock
return lock
async def get_or_create_user(
self,
external_id: str,
platform: str,
display_name: str | None = None,
) -> User:
return await self._prototype_state.get_or_create_user(
external_id=external_id,
platform=platform,
display_name=display_name,
)
async def send_message(
self,
user_id: str,
chat_id: str,
text: str,
attachments: list[Attachment] | None = None,
) -> MessageResponse:
response_parts: list[str] = []
sent_attachments: list[Attachment] = []
message_id = user_id
lock = self._get_chat_send_lock(chat_id)
async with lock:
chat_api = self._build_chat_api(chat_id)
try:
await chat_api.connect()
async for event in self._stream_agent_events(
chat_api, text, attachments=attachments
):
message_id = user_id
if isinstance(event, MsgEventTextChunk) and event.text:
response_parts.append(event.text)
elif isinstance(event, MsgEventSendFile):
attachment = self._attachment_from_send_file_event(event)
if attachment is not None:
sent_attachments.append(attachment)
except Exception as exc:
raise self._to_platform_error(exc) from exc
finally:
await self._close_chat_api(chat_api)
await self._prototype_state.set_last_tokens_used(str(chat_id), 0)
response_kwargs = {
"message_id": message_id,
"response": "".join(response_parts),
"tokens_used": 0,
"finished": True,
"attachments": sent_attachments,
}
return MessageResponse(**response_kwargs)
async def stream_message(
self,
user_id: str,
chat_id: str,
text: str,
attachments: list[Attachment] | None = None,
) -> AsyncIterator[MessageChunk]:
lock = self._get_chat_send_lock(chat_id)
async with lock:
chat_api = self._build_chat_api(chat_id)
try:
await chat_api.connect()
async for event in self._stream_agent_events(
chat_api, text, attachments=attachments
):
if isinstance(event, MsgEventTextChunk):
yield MessageChunk(
message_id=user_id,
delta=event.text,
finished=False,
)
elif isinstance(event, MsgEventSendFile):
continue
except Exception as exc:
raise self._to_platform_error(exc) from exc
finally:
await self._close_chat_api(chat_api)
await self._prototype_state.set_last_tokens_used(str(chat_id), 0)
yield MessageChunk(
message_id=user_id,
delta="",
finished=True,
tokens_used=0,
)
async def get_settings(self, user_id: str) -> UserSettings:
return await self._prototype_state.get_settings(user_id)
async def update_settings(self, user_id: str, action) -> None:
await self._prototype_state.update_settings(user_id, action)
async def disconnect_chat(self, chat_id: str) -> None:
self._chat_send_locks.pop(str(chat_id), None)
async def close(self) -> None:
self._chat_send_locks.clear()
async def _stream_agent_events(
self,
chat_api,
text: str,
attachments: list[Attachment] | None = None,
) -> AsyncIterator[object]:
attachment_paths = self._attachment_paths(attachments)
event_stream = chat_api.send_message(text, attachments=attachment_paths or None)
chunk_index = 0
async for event in event_stream:
if isinstance(event, MsgEventTextChunk):
logger.debug("agent_chunk", index=chunk_index, text=repr(event.text[:40]))
chunk_index += 1
else:
logger.debug("agent_event", index=chunk_index, type=type(event).__name__)
yield event
def _build_chat_api(self, chat_id: str):
if _ws_debug_enabled():
logger.warning(
"agent_chat_api_build",
agent_id=self._agent_id,
chat_id=str(chat_id),
normalized_base_url=self._agent_base_url,
ws_url=urljoin(self._agent_base_url, f"v1/agent_ws/{chat_id}/"),
)
return self._agent_api_cls(
agent_id=self._agent_id,
base_url=self._agent_base_url,
chat_id=str(chat_id),
)
@staticmethod
def _normalize_agent_base_url(base_url: str) -> str:
parsed = urlsplit(base_url)
path = re.sub(r"(?:/v1)?/agent_ws(?:/[^/]+)?/?$", "", parsed.path.rstrip("/"))
if path:
path = f"{path}/"
return urlunsplit((parsed.scheme, parsed.netloc, path, "", ""))
@staticmethod
async def _close_chat_api(chat_api) -> None:
close = getattr(chat_api, "close", None)
if callable(close):
try:
await close()
except Exception:
pass
@staticmethod
def _to_platform_error(exc: Exception) -> PlatformError:
code = getattr(exc, "code", None) or "PLATFORM_CONNECTION_ERROR"
return PlatformError(str(exc), code=code)
@staticmethod
def _normalize_workspace_path(location: str) -> str | None:
if not location:
return None
path = Path(location)
if not path.is_absolute():
normalized = path.as_posix()
return normalized or None
parts = path.parts
if len(parts) >= 2 and parts[1] == "workspace":
relative = Path(*parts[2:]).as_posix()
return relative or None
if len(parts) >= 3 and parts[1] == "agents":
relative = Path(*parts[3:]).as_posix()
return relative or None
relative = path.as_posix().lstrip("/")
return relative or None
@staticmethod
def _attachment_paths(attachments: list[Attachment] | None) -> list[str]:
if not attachments:
return []
paths = []
for attachment in attachments:
if attachment.workspace_path:
normalized = RealPlatformClient._normalize_workspace_path(
attachment.workspace_path
)
if normalized:
paths.append(normalized)
return paths
@staticmethod
def _attachment_from_send_file_event(event: MsgEventSendFile) -> Attachment:
location = str(event.path)
filename = Path(location).name or None
workspace_path = RealPlatformClient._normalize_workspace_path(location)
return Attachment(
url=location,
mime_type="application/octet-stream",
size=None,
filename=filename,
workspace_path=workspace_path or None,
)

19
sdk/upstream_agent_api.py Normal file
View file

@ -0,0 +1,19 @@
from __future__ import annotations
import sys
from pathlib import Path
_api_root = Path(__file__).resolve().parents[1] / "external" / "platform-agent_api"
if str(_api_root) not in sys.path:
sys.path.insert(0, str(_api_root))
from lambda_agent_api.agent_api import AgentApi, AgentBusyException, AgentException # noqa: E402
from lambda_agent_api.server import MsgEventSendFile, MsgEventTextChunk # noqa: E402
__all__ = [
"AgentApi",
"AgentBusyException",
"AgentException",
"MsgEventSendFile",
"MsgEventTextChunk",
]

Binary file not shown.

After

Width:  |  Height:  |  Size: 48 KiB

View file

@ -0,0 +1,199 @@
from pathlib import Path
import pytest
from adapter.matrix.agent_registry import AgentRegistryError, load_agent_registry
def test_load_agent_registry_reads_yaml_entries(tmp_path: Path):
path = tmp_path / "agents.yaml"
path.write_text(
"agents:\n"
" - id: agent-1\n"
" label: Analyst\n"
" - id: agent-2\n"
" label: Research\n",
encoding="utf-8",
)
registry = load_agent_registry(path)
assert [agent.agent_id for agent in registry.agents] == ["agent-1", "agent-2"]
assert registry.get("agent-1").label == "Analyst"
def test_agent_registry_agents_sequence_is_immutable(tmp_path: Path):
path = tmp_path / "agents.yaml"
path.write_text(
"agents:\n"
" - id: agent-1\n"
" label: Analyst\n",
encoding="utf-8",
)
registry = load_agent_registry(path)
with pytest.raises(AttributeError):
registry.agents.append( # type: ignore[attr-defined]
registry.agents[0]
)
def test_load_agent_registry_rejects_duplicate_ids(tmp_path: Path):
path = tmp_path / "agents.yaml"
path.write_text(
"agents:\n"
" - id: agent-1\n"
" label: Analyst\n"
" - id: agent-1\n"
" label: Duplicate\n",
encoding="utf-8",
)
with pytest.raises(AgentRegistryError, match="duplicate agent id"):
load_agent_registry(path)
def test_load_agent_registry_rejects_non_mapping_entries(tmp_path: Path):
path = tmp_path / "agents.yaml"
path.write_text(
"agents:\n"
" - agent-1\n",
encoding="utf-8",
)
with pytest.raises(AgentRegistryError, match="each agent entry requires id and label"):
load_agent_registry(path)
@pytest.mark.parametrize(
"content",
[
"",
"agents: []\n",
"agents: agent-1\n",
"foo: bar\n",
],
)
def test_load_agent_registry_rejects_missing_non_list_and_empty_agents(
tmp_path: Path, content: str
):
path = tmp_path / "agents.yaml"
path.write_text(content, encoding="utf-8")
with pytest.raises(AgentRegistryError, match="agents registry must contain a non-empty agents list"):
load_agent_registry(path)
@pytest.mark.parametrize(
"content, expected",
[
(
"agents:\n"
" - label: Analyst\n",
"each agent entry requires id and label",
),
(
"agents:\n"
" - id: agent-1\n",
"each agent entry requires id and label",
),
],
)
def test_load_agent_registry_rejects_missing_id_or_label(tmp_path: Path, content: str, expected: str):
path = tmp_path / "agents.yaml"
path.write_text(content, encoding="utf-8")
with pytest.raises(AgentRegistryError, match=expected):
load_agent_registry(path)
def test_load_agent_registry_rejects_invalid_top_level_yaml(tmp_path: Path):
path = tmp_path / "agents.yaml"
path.write_text(
"- id: agent-1\n"
" label: Analyst\n",
encoding="utf-8",
)
with pytest.raises(AgentRegistryError, match="agent registry must be a mapping with an agents list"):
load_agent_registry(path)
def test_load_agent_registry_rejects_malformed_yaml(tmp_path: Path):
path = tmp_path / "agents.yaml"
path.write_text(
"agents:\n"
" - id: agent-1\n"
" label: Analyst\n"
" - id: agent-2\n"
" label: Research\n"
" - [\n",
encoding="utf-8",
)
with pytest.raises(AgentRegistryError, match="invalid agent registry YAML"):
load_agent_registry(path)
@pytest.mark.parametrize(
"content",
[
"agents:\n"
" - id: null\n"
" label: Analyst\n",
"agents:\n"
" - id: agent-1\n"
" label: null\n",
],
)
def test_load_agent_registry_rejects_null_id_or_label(tmp_path: Path, content: str):
path = tmp_path / "agents.yaml"
path.write_text(content, encoding="utf-8")
with pytest.raises(AgentRegistryError, match="each agent entry requires id and label"):
load_agent_registry(path)
@pytest.mark.parametrize(
"content",
[
"agents:\n"
" - id: ' '\n"
" label: Analyst\n",
"agents:\n"
" - id: agent-1\n"
" label: ' '\n",
],
)
def test_load_agent_registry_rejects_blank_id_or_label(tmp_path: Path, content: str):
path = tmp_path / "agents.yaml"
path.write_text(content, encoding="utf-8")
with pytest.raises(AgentRegistryError, match="each agent entry requires id and label"):
load_agent_registry(path)
@pytest.mark.parametrize(
"content",
[
"agents:\n"
" - id: 123\n"
" label: Analyst\n",
"agents:\n"
" - id: agent-1\n"
" label: 456\n",
"agents:\n"
" - id: true\n"
" label: Analyst\n",
"agents:\n"
" - id: agent-1\n"
" label: false\n",
],
)
def test_load_agent_registry_rejects_non_string_id_or_label(tmp_path: Path, content: str):
path = tmp_path / "agents.yaml"
path.write_text(content, encoding="utf-8")
with pytest.raises(AgentRegistryError, match="each agent entry requires id and label"):
load_agent_registry(path)

Some files were not shown because too many files have changed in this diff Show more