feat: finalize matrix platform audit and docs

This commit is contained in:
Mikhail Putilovskij 2026-04-21 15:35:03 +03:00
parent 6422c7db58
commit 4524a6abc8
30 changed files with 3093 additions and 176 deletions

View file

@ -0,0 +1,515 @@
# Matrix Direct-Agent Prototype Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Replace `MockPlatformClient` with a real Matrix-only direct-agent prototype backend while keeping Matrix adapter logic stable and preserving a clean future migration path.
**Architecture:** Introduce a thin compatibility layer under `sdk/` that splits direct WebSocket agent messaging from local prototype-only control-plane state. Keep `core/` and Matrix handlers on the existing `PlatformClient` contract, and limit surface changes to runtime wiring and tests.
**Tech Stack:** Python 3.11, `aiohttp` WebSocket client, `matrix-nio`, `pydantic`, `pytest`, `pytest-asyncio`
---
## File Structure
- Create: `sdk/agent_session.py`
Purpose: Minimal direct WebSocket client for the real agent, including thread-key derivation, request/response parsing, and sync/stream helpers.
- Create: `sdk/prototype_state.py`
Purpose: Local prototype-only user mapping and settings store kept behind a small API.
- Create: `sdk/real.py`
Purpose: `PlatformClient` implementation that composes `AgentSessionClient` and `PrototypeStateStore`.
- Modify: `sdk/__init__.py`
Purpose: export `RealPlatformClient` if useful for runtime imports.
- Modify: `adapter/matrix/bot.py`
Purpose: runtime/backend selection and env-based configuration for mock vs real backend.
- Create: `tests/platform/test_agent_session.py`
Purpose: transport-level tests for direct agent communication.
- Create: `tests/platform/test_prototype_state.py`
Purpose: unit tests for local user/settings behavior.
- Create: `tests/platform/test_real.py`
Purpose: contract tests for `RealPlatformClient`.
- Modify: `tests/core/test_integration.py`
Purpose: prove the new platform implementation preserves core behavior.
- Modify: `README.md`
Purpose: document backend selection and prototype limitations after code is working.
---
### Task 1: Add Direct Agent Session Transport
**Files:**
- Create: `sdk/agent_session.py`
- Test: `tests/platform/test_agent_session.py`
- [ ] **Step 1: Write the failing transport tests**
```python
import pytest
from sdk.agent_session import AgentSessionClient, build_thread_key
def test_build_thread_key_uses_surface_user_and_chat_id():
assert build_thread_key("matrix", "@alice:example.org", "C1") == "matrix:@alice:example.org:C1"
@pytest.mark.asyncio
async def test_send_message_collects_text_chunks_and_tokens(aiohttp_server):
...
@pytest.mark.asyncio
async def test_stream_message_yields_incremental_chunks(aiohttp_server):
...
```
- [ ] **Step 2: Run test to verify it fails**
Run: `pytest tests/platform/test_agent_session.py -q`
Expected: FAIL with `ModuleNotFoundError: No module named 'sdk.agent_session'`
- [ ] **Step 3: Write minimal transport implementation**
```python
from __future__ import annotations
from dataclasses import dataclass
from typing import AsyncIterator
import aiohttp
from sdk.interface import MessageChunk, MessageResponse, PlatformError
def build_thread_key(platform: str, user_id: str, chat_id: str) -> str:
return f"{platform}:{user_id}:{chat_id}"
@dataclass
class AgentSessionConfig:
base_ws_url: str
timeout_seconds: float = 30.0
class AgentSessionClient:
def __init__(self, config: AgentSessionConfig) -> None:
self._config = config
async def send_message(self, *, thread_key: str, text: str) -> MessageResponse:
chunks = []
tokens_used = 0
async for chunk in self.stream_message(thread_key=thread_key, text=text):
chunks.append(chunk.delta)
tokens_used = chunk.tokens_used or tokens_used
return MessageResponse(
message_id=thread_key,
response="".join(chunks),
tokens_used=tokens_used,
finished=True,
)
async def stream_message(self, *, thread_key: str, text: str) -> AsyncIterator[MessageChunk]:
url = f"{self._config.base_ws_url}?thread_id={thread_key}"
async with aiohttp.ClientSession() as session:
async with session.ws_connect(url, heartbeat=30) as ws:
status_msg = await ws.receive_json(timeout=self._config.timeout_seconds)
if status_msg.get("type") != "STATUS":
raise PlatformError("Agent did not send STATUS", code="AGENT_PROTOCOL_ERROR")
await ws.send_json({"type": "USER_MESSAGE", "text": text})
while True:
payload = await ws.receive_json(timeout=self._config.timeout_seconds)
msg_type = payload.get("type")
if msg_type == "AGENT_EVENT_TEXT_CHUNK":
yield MessageChunk(message_id=thread_key, delta=payload["text"], finished=False)
elif msg_type == "AGENT_EVENT_END":
yield MessageChunk(
message_id=thread_key,
delta="",
finished=True,
tokens_used=payload.get("tokens_used", 0),
)
return
elif msg_type == "ERROR":
raise PlatformError(payload.get("details", "Agent error"), code=payload.get("code", "AGENT_ERROR"))
else:
raise PlatformError(f"Unexpected agent message: {payload}", code="AGENT_PROTOCOL_ERROR")
```
- [ ] **Step 4: Run test to verify it passes**
Run: `pytest tests/platform/test_agent_session.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add sdk/agent_session.py tests/platform/test_agent_session.py
git commit -m "feat: add direct agent session transport"
```
---
### Task 2: Add Local Prototype State For Users And Settings
**Files:**
- Create: `sdk/prototype_state.py`
- Test: `tests/platform/test_prototype_state.py`
- [ ] **Step 1: Write the failing state tests**
```python
import pytest
from core.protocol import SettingsAction
from sdk.prototype_state import PrototypeStateStore
@pytest.mark.asyncio
async def test_get_or_create_user_is_stable_per_surface_identity():
...
@pytest.mark.asyncio
async def test_settings_defaults_match_existing_mock_shape():
...
@pytest.mark.asyncio
async def test_update_settings_supports_toggle_skill_and_setters():
...
```
- [ ] **Step 2: Run test to verify it fails**
Run: `pytest tests/platform/test_prototype_state.py -q`
Expected: FAIL with `ModuleNotFoundError: No module named 'sdk.prototype_state'`
- [ ] **Step 3: Write minimal state implementation**
```python
from __future__ import annotations
from datetime import UTC, datetime
from sdk.interface import User, UserSettings
# Defaults are defined here, not imported from sdk.mock, to keep real backend
# isolated from the mock. Copy-paste intentional.
DEFAULT_SKILLS: dict[str, bool] = {
"web-search": True,
"fetch-url": True,
"email": False,
"browser": False,
"image-gen": False,
"files": True,
}
DEFAULT_SAFETY: dict[str, bool] = {"email-send": True, "file-delete": True, "social-post": True}
DEFAULT_SOUL: dict[str, str] = {"name": "Лямбда", "instructions": ""}
DEFAULT_PLAN: dict = {"name": "Beta", "tokens_used": 0, "tokens_limit": 1000}
class PrototypeStateStore:
def __init__(self) -> None:
self._users: dict[str, User] = {}
self._settings: dict[str, dict] = {}
async def get_or_create_user(
self,
*,
external_id: str,
platform: str,
display_name: str | None = None,
) -> User:
key = f"{platform}:{external_id}"
existing = self._users.get(key)
if existing is not None:
return existing.model_copy(update={"is_new": False})
user = User(
user_id=f"usr-{platform}-{external_id}",
external_id=external_id,
platform=platform,
display_name=display_name,
created_at=datetime.now(UTC),
is_new=True,
)
self._users[key] = user.model_copy(update={"is_new": False})
return user
async def get_settings(self, user_id: str) -> UserSettings:
stored = self._settings.get(user_id, {})
return UserSettings(
skills={**DEFAULT_SKILLS, **stored.get("skills", {})},
connectors=stored.get("connectors", {}),
soul={**DEFAULT_SOUL, **stored.get("soul", {})},
safety={**DEFAULT_SAFETY, **stored.get("safety", {})},
plan={**DEFAULT_PLAN, **stored.get("plan", {})},
)
async def update_settings(self, user_id: str, action) -> None:
settings = self._settings.setdefault(user_id, {})
if action.action == "toggle_skill":
skills = settings.setdefault("skills", DEFAULT_SKILLS.copy())
skills[action.payload["skill"]] = action.payload.get("enabled", True)
elif action.action == "set_soul":
soul = settings.setdefault("soul", DEFAULT_SOUL.copy())
soul[action.payload["field"]] = action.payload["value"]
elif action.action == "set_safety":
safety = settings.setdefault("safety", DEFAULT_SAFETY.copy())
safety[action.payload["trigger"]] = action.payload.get("enabled", True)
```
- [ ] **Step 4: Run test to verify it passes**
Run: `pytest tests/platform/test_prototype_state.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add sdk/prototype_state.py tests/platform/test_prototype_state.py
git commit -m "feat: add prototype local state store"
```
---
### Task 3: Implement RealPlatformClient Compatibility Layer
**Files:**
- Create: `sdk/real.py`
- Modify: `sdk/__init__.py`
- Test: `tests/platform/test_real.py`
- Test: `tests/core/test_integration.py`
- [ ] **Step 1: Write the failing compatibility tests**
```python
import pytest
from core.protocol import SettingsAction
from sdk.real import RealPlatformClient
@pytest.mark.asyncio
async def test_real_platform_client_get_or_create_user_uses_local_state():
...
@pytest.mark.asyncio
async def test_real_platform_client_send_message_uses_thread_key():
...
@pytest.mark.asyncio
async def test_real_platform_client_settings_are_local():
...
```
- [ ] **Step 2: Run test to verify it fails**
Run: `pytest tests/platform/test_real.py -q`
Expected: FAIL with `ModuleNotFoundError: No module named 'sdk.real'`
- [ ] **Step 3: Write minimal compatibility wrapper**
```python
from __future__ import annotations
from typing import AsyncIterator
from sdk.agent_session import AgentSessionClient, build_thread_key
from sdk.interface import Attachment, MessageChunk, MessageResponse, PlatformClient, User, UserSettings
from sdk.prototype_state import PrototypeStateStore
class RealPlatformClient(PlatformClient):
def __init__(
self,
agent_sessions: AgentSessionClient,
prototype_state: PrototypeStateStore,
platform: str = "matrix",
) -> None:
self._agent_sessions = agent_sessions
self._prototype_state = prototype_state
self._platform = platform # surface name used in thread key; pass explicitly for future surfaces
async def get_or_create_user(
self,
external_id: str,
platform: str,
display_name: str | None = None,
) -> User:
return await self._prototype_state.get_or_create_user(
external_id=external_id,
platform=platform,
display_name=display_name,
)
async def send_message(
self,
user_id: str,
chat_id: str,
text: str,
attachments: list[Attachment] | None = None,
) -> MessageResponse:
# user_id here is the internal id (e.g. "usr-matrix-@alice:example.org"), which is
# unique per user and stable — acceptable as thread identity for v1 prototype.
thread_key = build_thread_key(self._platform, user_id, chat_id)
return await self._agent_sessions.send_message(thread_key=thread_key, text=text)
async def stream_message(
self,
user_id: str,
chat_id: str,
text: str,
attachments: list[Attachment] | None = None,
) -> AsyncIterator[MessageChunk]:
thread_key = build_thread_key(self._platform, user_id, chat_id)
async for chunk in self._agent_sessions.stream_message(thread_key=thread_key, text=text):
yield chunk
async def get_settings(self, user_id: str) -> UserSettings:
return await self._prototype_state.get_settings(user_id)
async def update_settings(self, user_id: str, action) -> None:
await self._prototype_state.update_settings(user_id, action)
```
- [ ] **Step 4: Run tests to verify the contract holds**
Run: `pytest tests/platform/test_real.py tests/core/test_integration.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add sdk/real.py sdk/__init__.py tests/platform/test_real.py tests/core/test_integration.py
git commit -m "feat: add real platform compatibility layer"
```
---
### Task 4: Wire Matrix Runtime To Real Backend And Document Usage
**Files:**
- Modify: `adapter/matrix/bot.py`
- Modify: `README.md`
- Modify: `tests/adapter/matrix/test_dispatcher.py`
- [ ] **Step 1: Write the failing runtime wiring tests**
```python
import os
from adapter.matrix.bot import build_runtime
from sdk.real import RealPlatformClient
def test_build_runtime_uses_real_platform_when_matrix_backend_is_real(monkeypatch):
monkeypatch.setenv("MATRIX_PLATFORM_BACKEND", "real")
monkeypatch.setenv("AGENT_WS_URL", "ws://agent.example/agent_ws/")
runtime = build_runtime()
assert isinstance(runtime.platform, RealPlatformClient)
```
- [ ] **Step 2: Run test to verify it fails**
Run: `pytest tests/adapter/matrix/test_dispatcher.py -q`
Expected: FAIL because runtime still always constructs `MockPlatformClient`
- [ ] **Step 3: Implement backend selection and docs**
```python
# adapter/matrix/bot.py — add these imports at the top
from sdk.agent_session import AgentSessionClient, AgentSessionConfig
from sdk.interface import PlatformClient
from sdk.prototype_state import PrototypeStateStore
from sdk.real import RealPlatformClient
def _build_platform_from_env() -> PlatformClient:
backend = os.environ.get("MATRIX_PLATFORM_BACKEND", "mock")
if backend == "real":
ws_url = os.environ["AGENT_WS_URL"]
return RealPlatformClient(
agent_sessions=AgentSessionClient(AgentSessionConfig(base_ws_url=ws_url)),
prototype_state=PrototypeStateStore(),
platform="matrix",
)
return MockPlatformClient()
# Update build_runtime to use env-based selection when no platform is injected:
def build_runtime(
platform: PlatformClient | None = None, # was MockPlatformClient | None
store: StateStore | None = None,
client: AsyncClient | None = None,
) -> MatrixRuntime:
platform = platform or _build_platform_from_env()
... # rest unchanged
```
Also update the `MatrixRuntime` dataclass field type from `MockPlatformClient` to `PlatformClient` so the type annotation matches the runtime behavior.
```markdown
# README.md
Matrix prototype backend selection:
- `MATRIX_PLATFORM_BACKEND=mock` uses `sdk/mock.py`
- `MATRIX_PLATFORM_BACKEND=real` uses direct agent WebSocket integration
- `AGENT_WS_URL=ws://host:port/agent_ws/` is required for the real backend
Current real-backend limitations:
- text chat only
- local settings storage
- no attachments or async task callbacks yet
```
- [ ] **Step 4: Run targeted verification**
Run: `pytest tests/adapter/matrix/test_dispatcher.py tests/platform/test_agent_session.py tests/platform/test_prototype_state.py tests/platform/test_real.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add adapter/matrix/bot.py README.md tests/adapter/matrix/test_dispatcher.py
git commit -m "feat: wire matrix runtime to real backend"
```
---
## Self-Review
- Spec coverage:
- direct-agent transport: Task 1
- local settings/user state: Task 2
- stable `PlatformClient` wrapper: Task 3
- Matrix runtime wiring and docs: Task 4
- Placeholder scan: no `TBD`, `TODO`, or “implement later” steps remain in the plan.
- Type consistency:
- `build_thread_key(platform, user_id, chat_id)` is used consistently.
- `RealPlatformClient` remains the only bot-facing implementation.
- local settings stay in `PrototypeStateStore`.
## Execution Handoff
Plan complete and saved to `docs/superpowers/plans/2026-04-08-matrix-direct-agent-prototype.md`. Two execution options:
**1. Subagent-Driven (recommended)** - I dispatch a fresh subagent per task, review between tasks, fast iteration
**2. Inline Execution** - Execute tasks in this session using executing-plans, batch execution with checkpoints
**Which approach?**

View file

@ -0,0 +1,480 @@
# Matrix Per-Chat Context Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Move the Matrix surface from a shared agent context to true per-room platform contexts mapped through `platform_chat_id`, including `!new`, `!branch`, lazy migration, and per-room context commands.
**Architecture:** Matrix keeps owning UX chats (`C1`, `C2`, rooms, spaces). Each working room stores a `platform_chat_id` in `room_meta`, and platform-facing operations use that mapping. Legacy rooms without a mapping lazily create one on first context-aware use.
**Tech Stack:** Python 3.11, Matrix nio adapter, local state store, `lambda_agent_api`, pytest
---
### Task 1: Add `platform_chat_id` to Matrix metadata and tests
**Files:**
- Modify: `adapter/matrix/store.py`
- Test: `tests/adapter/matrix/test_store.py`
- [ ] **Step 1: Write the failing test**
```python
async def test_room_meta_roundtrip_with_platform_chat_id(store: InMemoryStore):
meta = {
"chat_id": "C1",
"matrix_user_id": "@alice:example.org",
"platform_chat_id": "chat-platform-1",
}
await set_room_meta(store, "!r:m.org", meta)
saved = await get_room_meta(store, "!r:m.org")
assert saved is not None
assert saved["platform_chat_id"] == "chat-platform-1"
```
- [ ] **Step 2: Run test to verify it fails or proves missing coverage**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_store.py -q`
Expected: either FAIL on missing assertion coverage or PASS after adding the new test with current generic storage behavior
- [ ] **Step 3: Write minimal implementation**
```python
# adapter/matrix/store.py
# No schema gate is required because room metadata is already stored as a dict.
# Keep helpers unchanged, but add focused helper functions if they reduce repeated logic:
async def get_platform_chat_id(store: StateStore, room_id: str) -> str | None:
meta = await get_room_meta(store, room_id)
return meta.get("platform_chat_id") if meta else None
async def set_platform_chat_id(store: StateStore, room_id: str, platform_chat_id: str) -> None:
meta = await get_room_meta(store, room_id) or {}
meta["platform_chat_id"] = platform_chat_id
await set_room_meta(store, room_id, meta)
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_store.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add adapter/matrix/store.py tests/adapter/matrix/test_store.py
git commit -m "feat: add platform chat id room metadata helpers"
```
### Task 2: Extend the platform wrapper to support context-aware API calls
**Files:**
- Modify: `sdk/agent_api_wrapper.py`
- Modify: `sdk/real.py`
- Test: `tests/platform/test_real.py`
- [ ] **Step 1: Write the failing tests**
```python
@pytest.mark.asyncio
async def test_real_client_send_message_uses_platform_chat_id():
api = FakeAgentApi()
client = RealPlatformClient(agent_api=api, prototype_state=PrototypeStateStore())
await client.send_message("@alice:example.org", "chat-platform-1", "hello")
assert api.sent == [("chat-platform-1", "hello")]
@pytest.mark.asyncio
async def test_real_client_create_and_branch_context_delegate_to_agent_api():
api = FakeAgentApi(create_ids=["chat-new", "chat-branch"])
client = RealPlatformClient(agent_api=api, prototype_state=PrototypeStateStore())
created = await client.create_chat_context("@alice:example.org")
branched = await client.branch_chat_context("@alice:example.org", "chat-source")
assert created == "chat-new"
assert branched == "chat-branch"
assert api.branch_calls == ["chat-source"]
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `PYTHONPATH=. uv run pytest tests/platform/test_real.py -q`
Expected: FAIL because `RealPlatformClient` does not yet expose context-aware methods or pass a platform context id through
- [ ] **Step 3: Write minimal implementation**
```python
# sdk/agent_api_wrapper.py
class AgentApiWrapper(AgentApi):
async def create_chat(self) -> str:
...
async def branch_chat(self, chat_id: str) -> str:
...
async def send_message(self, chat_id: str, text: str):
...
async def save_context(self, chat_id: str, name: str) -> None:
...
async def load_context(self, chat_id: str, name: str) -> None:
...
# sdk/real.py
class RealPlatformClient(PlatformClient):
async def create_chat_context(self, user_id: str) -> str:
return await self._agent_api.create_chat()
async def branch_chat_context(self, user_id: str, from_chat_id: str) -> str:
return await self._agent_api.branch_chat(from_chat_id)
async def save_chat_context(self, user_id: str, chat_id: str, name: str) -> None:
await self._agent_api.save_context(chat_id, name)
async def load_chat_context(self, user_id: str, chat_id: str, name: str) -> None:
await self._agent_api.load_context(chat_id, name)
async def stream_message(...):
async for event in self._agent_api.send_message(chat_id, text):
...
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `PYTHONPATH=. uv run pytest tests/platform/test_real.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add sdk/agent_api_wrapper.py sdk/real.py tests/platform/test_real.py
git commit -m "feat: add context-aware real platform client methods"
```
### Task 3: Create Matrix-side resolver for mapped or lazy-created platform contexts
**Files:**
- Modify: `adapter/matrix/bot.py`
- Modify: `adapter/matrix/store.py`
- Test: `tests/adapter/matrix/test_dispatcher.py`
- [ ] **Step 1: Write the failing tests**
```python
async def test_existing_room_without_platform_chat_id_gets_lazy_mapping_on_message():
runtime = build_runtime(platform=FakeRealPlatformClient(create_ids=["chat-platform-1"]))
await set_room_meta(runtime.store, "!room:example.org", {
"chat_id": "C1",
"matrix_user_id": "@alice:example.org",
})
client = SimpleNamespace(user_id="@bot:example.org", room_send=AsyncMock())
bot = MatrixBot(client, runtime)
room = SimpleNamespace(room_id="!room:example.org")
event = SimpleNamespace(sender="@alice:example.org", body="hello")
await bot.on_room_message(room, event)
meta = await get_room_meta(runtime.store, "!room:example.org")
assert meta["platform_chat_id"] == "chat-platform-1"
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_dispatcher.py -q`
Expected: FAIL because no lazy mapping exists
- [ ] **Step 3: Write minimal implementation**
```python
# adapter/matrix/bot.py
async def _ensure_platform_chat_id(self, room_id: str, user_id: str) -> str:
meta = await get_room_meta(self.runtime.store, room_id)
if meta is None:
raise ValueError("room metadata is required")
platform_chat_id = meta.get("platform_chat_id")
if platform_chat_id:
return platform_chat_id
if not hasattr(self.runtime.platform, "create_chat_context"):
raise ValueError("real platform backend required")
platform_chat_id = await self.runtime.platform.create_chat_context(user_id)
meta["platform_chat_id"] = platform_chat_id
await set_room_meta(self.runtime.store, room_id, meta)
return platform_chat_id
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_dispatcher.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add adapter/matrix/bot.py adapter/matrix/store.py tests/adapter/matrix/test_dispatcher.py
git commit -m "feat: lazily assign platform chat ids to matrix rooms"
```
### Task 4: Make `!new` and workspace bootstrap create independent platform contexts
**Files:**
- Modify: `adapter/matrix/handlers/chat.py`
- Modify: `adapter/matrix/handlers/auth.py`
- Modify: `adapter/matrix/bot.py`
- Test: `tests/adapter/matrix/test_chat_space.py`
- Test: `tests/adapter/matrix/test_invite_space.py`
- Test: `tests/adapter/matrix/test_dispatcher.py`
- [ ] **Step 1: Write the failing tests**
```python
async def test_new_chat_assigns_new_platform_chat_id():
client = SimpleNamespace(
room_create=AsyncMock(return_value=SimpleNamespace(room_id="!r2:example")),
room_put_state=AsyncMock(),
room_invite=AsyncMock(),
)
platform = FakeRealPlatformClient(create_ids=["chat-platform-7"])
runtime = build_runtime(platform=platform, client=client)
await set_user_meta(runtime.store, "u1", {"space_id": "!space:example", "next_chat_index": 7})
result = await runtime.dispatcher.dispatch(
IncomingCommand(user_id="u1", platform="matrix", chat_id="C3", command="new", args=["Research"])
)
meta = await get_room_meta(runtime.store, "!r2:example")
assert meta["platform_chat_id"] == "chat-platform-7"
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_chat_space.py tests/adapter/matrix/test_invite_space.py tests/adapter/matrix/test_dispatcher.py -q`
Expected: FAIL because new chats do not yet store a platform context id
- [ ] **Step 3: Write minimal implementation**
```python
# adapter/matrix/handlers/chat.py
# adapter/matrix/handlers/auth.py
platform_chat_id = None
if hasattr(platform, "create_chat_context"):
platform_chat_id = await platform.create_chat_context(event.user_id)
await set_room_meta(store, room_id, {
"chat_id": chat_id,
"matrix_user_id": event.user_id,
"space_id": space_id,
"platform_chat_id": platform_chat_id,
})
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_chat_space.py tests/adapter/matrix/test_invite_space.py tests/adapter/matrix/test_dispatcher.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add adapter/matrix/handlers/chat.py adapter/matrix/handlers/auth.py adapter/matrix/bot.py tests/adapter/matrix/test_chat_space.py tests/adapter/matrix/test_invite_space.py tests/adapter/matrix/test_dispatcher.py
git commit -m "feat: assign platform contexts when creating matrix chats"
```
### Task 5: Make per-room save, load, and context use the mapped platform context
**Files:**
- Modify: `adapter/matrix/handlers/context_commands.py`
- Modify: `adapter/matrix/bot.py`
- Modify: `sdk/prototype_state.py`
- Test: `tests/adapter/matrix/test_context_commands.py`
- [ ] **Step 1: Write the failing tests**
```python
@pytest.mark.asyncio
async def test_save_command_uses_room_platform_chat_id():
platform = MatrixCommandPlatform()
runtime = build_runtime(platform=platform)
await set_room_meta(runtime.store, "!room:example.org", {
"chat_id": "C1",
"matrix_user_id": "u1",
"platform_chat_id": "chat-platform-1",
})
event = IncomingCommand(user_id="u1", platform="matrix", chat_id="C1", command="save", args=["session-a"])
result = await make_handle_save(...)(event, runtime.auth_mgr, platform, runtime.chat_mgr, runtime.settings_mgr)
assert platform.saved_calls == [("chat-platform-1", "session-a")]
@pytest.mark.asyncio
async def test_context_command_reports_current_room_platform_chat_id():
...
assert "chat-platform-1" in result[0].text
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_context_commands.py -q`
Expected: FAIL because save/load/context do not currently use room-level platform mappings
- [ ] **Step 3: Write minimal implementation**
```python
# adapter/matrix/handlers/context_commands.py
room_id = await _resolve_room_id(event, chat_mgr)
meta = await get_room_meta(store, room_id)
platform_chat_id = meta.get("platform_chat_id")
await platform.save_chat_context(event.user_id, platform_chat_id, name)
await platform.load_chat_context(event.user_id, platform_chat_id, name)
# sdk/prototype_state.py
# store current loaded session per user+platform_chat_id instead of only per user when needed for Matrix `!context`
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_context_commands.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add adapter/matrix/handlers/context_commands.py adapter/matrix/bot.py sdk/prototype_state.py tests/adapter/matrix/test_context_commands.py
git commit -m "feat: bind matrix context commands to platform chat ids"
```
### Task 6: Add `!branch` and help-text updates
**Files:**
- Modify: `adapter/matrix/handlers/chat.py`
- Modify: `adapter/matrix/handlers/__init__.py`
- Modify: `adapter/matrix/handlers/settings.py`
- Modify: `adapter/matrix/handlers/auth.py`
- Modify: `adapter/matrix/bot.py`
- Test: `tests/adapter/matrix/test_chat_space.py`
- Test: `tests/adapter/matrix/test_dispatcher.py`
- [ ] **Step 1: Write the failing tests**
```python
async def test_branch_creates_new_room_with_branched_platform_chat_id():
client = SimpleNamespace(
room_create=AsyncMock(return_value=SimpleNamespace(room_id="!r3:example")),
room_put_state=AsyncMock(),
room_invite=AsyncMock(),
)
platform = FakeRealPlatformClient(branch_ids=["chat-platform-branch"])
runtime = build_runtime(platform=platform, client=client)
await set_room_meta(runtime.store, "!current:example", {
"chat_id": "C2",
"matrix_user_id": "u1",
"space_id": "!space:example",
"platform_chat_id": "chat-platform-source",
})
result = await runtime.dispatcher.dispatch(
IncomingCommand(user_id="u1", platform="matrix", chat_id="C2", command="branch", args=["Fork"])
)
meta = await get_room_meta(runtime.store, "!r3:example")
assert meta["platform_chat_id"] == "chat-platform-branch"
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_chat_space.py tests/adapter/matrix/test_dispatcher.py -q`
Expected: FAIL because `branch` is not implemented
- [ ] **Step 3: Write minimal implementation**
```python
# adapter/matrix/handlers/chat.py
def make_handle_branch(client, store):
async def handle_branch(event, auth_mgr, platform, chat_mgr, settings_mgr):
source_room_id = ...
source_meta = await get_room_meta(store, source_room_id)
platform_chat_id = await platform.branch_chat_context(event.user_id, source_meta["platform_chat_id"])
...
await set_room_meta(store, new_room_id, {
"chat_id": new_chat_id,
"matrix_user_id": event.user_id,
"space_id": space_id,
"platform_chat_id": platform_chat_id,
})
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_chat_space.py tests/adapter/matrix/test_dispatcher.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add adapter/matrix/handlers/chat.py adapter/matrix/handlers/__init__.py adapter/matrix/handlers/settings.py adapter/matrix/handlers/auth.py adapter/matrix/bot.py tests/adapter/matrix/test_chat_space.py tests/adapter/matrix/test_dispatcher.py
git commit -m "feat: add matrix branch command for platform contexts"
```
### Task 7: Verify the full Matrix flow and clean up legacy assumptions
**Files:**
- Modify: `tests/platform/test_real.py`
- Modify: `tests/adapter/matrix/test_dispatcher.py`
- Modify: `tests/adapter/matrix/test_context_commands.py`
- Modify: `tests/core/test_integration.py`
- [ ] **Step 1: Add integration coverage for independent room contexts**
```python
@pytest.mark.asyncio
async def test_two_rooms_send_messages_into_different_platform_contexts():
platform = FakeRealPlatformClient()
runtime = build_runtime(platform=platform)
await set_room_meta(runtime.store, "!r1:example", {"chat_id": "C1", "matrix_user_id": "u1", "platform_chat_id": "chat-1"})
await set_room_meta(runtime.store, "!r2:example", {"chat_id": "C2", "matrix_user_id": "u1", "platform_chat_id": "chat-2"})
...
assert platform.sent == [("chat-1", "hello"), ("chat-2", "world")]
```
- [ ] **Step 2: Run the focused verification suite**
Run: `PYTHONPATH=. uv run pytest tests/platform/test_real.py tests/adapter/matrix/test_store.py tests/adapter/matrix/test_chat_space.py tests/adapter/matrix/test_invite_space.py tests/adapter/matrix/test_dispatcher.py tests/adapter/matrix/test_context_commands.py tests/core/test_integration.py -q`
Expected: PASS
- [ ] **Step 3: Run the full Matrix suite**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix -q`
Expected: PASS
- [ ] **Step 4: Inspect help text and command visibility**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_dispatcher.py -q`
Expected: PASS with `!branch` present in help and hidden commands still absent
- [ ] **Step 5: Commit**
```bash
git add tests/platform/test_real.py tests/adapter/matrix/test_store.py tests/adapter/matrix/test_chat_space.py tests/adapter/matrix/test_invite_space.py tests/adapter/matrix/test_dispatcher.py tests/adapter/matrix/test_context_commands.py tests/core/test_integration.py
git commit -m "test: verify matrix per-chat platform context flow"
```
## Self-Review
- Spec coverage:
- `surface_chat -> platform_chat_id` mapping is covered by Tasks 1, 3, and 4.
- `!new` independent contexts are covered by Task 4.
- `!branch` snapshot flow is covered by Task 6.
- per-room `!save`, `!load`, and `!context` are covered by Task 5.
- lazy migration for legacy rooms is covered by Task 3.
- verification across rooms is covered by Task 7.
- Placeholder scan:
- No `TODO` or `TBD` placeholders remain.
- Commands and file paths are concrete.
- Type consistency:
- The plan consistently uses `platform_chat_id` for stored mapping and `create_chat_context` / `branch_chat_context` / `save_chat_context` / `load_chat_context` for platform-facing methods.

View file

@ -0,0 +1,624 @@
# Matrix Shared Workspace File Flow Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Move the Matrix surface to a prod-like shared-workspace file flow where incoming files are downloaded into `/workspace`, passed to `platform-agent` as relative attachment paths, and outbound `send_file` events are delivered back to the Matrix room.
**Architecture:** Keep the platform contract path-based and surface-agnostic. Extend the surface attachment model with a workspace-relative path, update the real SDK bridge to forward attachments and modern agent events, then add a Matrix-specific storage helper plus a shared-volume runtime. This preserves room/context semantics while aligning file flow with upstream `platform-agent`.
**Tech Stack:** Python 3.11, matrix-nio, aiohttp, Docker Compose, pytest, pytest-asyncio
---
## File Structure
- Modify: `core/protocol.py`
Purpose: add a workspace-relative attachment field that future surfaces can also use.
- Modify: `sdk/interface.py`
Purpose: keep the platform-side attachment shape aligned with the surface model.
- Modify: `core/handlers/message.py`
Purpose: stop dropping attachments before platform dispatch.
- Modify: `sdk/agent_api_wrapper.py`
Purpose: accept modern upstream agent events and modern WS route semantics.
- Modify: `sdk/real.py`
Purpose: convert attachment objects into workspace-relative paths and forward them to the agent API.
- Create: `adapter/matrix/files.py`
Purpose: Matrix-specific download/upload helper for shared `/workspace`.
- Modify: `adapter/matrix/bot.py`
Purpose: persist incoming Matrix files into shared workspace and render outbound file events back to Matrix.
- Modify: `tests/core/test_integration.py`
Purpose: prove message dispatch keeps attachments and platform send path receives them.
- Modify: `tests/platform/test_real.py`
Purpose: verify attachment forwarding and outbound file events.
- Create: `tests/adapter/matrix/test_files.py`
Purpose: unit coverage for Matrix workspace path building, download handling, and outbound upload behavior.
- Modify: `tests/adapter/matrix/test_dispatcher.py`
Purpose: verify Matrix bot file receive/send integration.
- Modify: `docker-compose.yml`
Purpose: define shared `/workspace` runtime between `matrix-bot` and `platform-agent`.
- Modify: `README.md`
Purpose: document the new default runtime and file flow.
- Modify: `.env.example`
Purpose: update real-backend defaults to in-compose service URLs and workspace-aware runtime.
### Task 1: Preserve Attachment Metadata Through Core Message Dispatch
**Files:**
- Modify: `core/protocol.py`
- Modify: `sdk/interface.py`
- Modify: `core/handlers/message.py`
- Test: `tests/core/test_dispatcher.py`
- Test: `tests/core/test_integration.py`
- [ ] **Step 1: Write the failing tests**
```python
# tests/core/test_integration.py
class RecordingAgentApi:
def __init__(self) -> None:
self.calls: list[tuple[str, list[str]]] = []
self.last_tokens_used = 0
async def send_message(self, text: str, attachments: list[str] | None = None):
self.calls.append((text, attachments or []))
yield type("Chunk", (), {"text": f"[REAL] {text}"})()
self.last_tokens_used = 5
async def test_full_flow_with_real_platform_forwards_workspace_attachment(real_dispatcher):
dispatcher, agent_api = real_dispatcher
start = IncomingCommand(user_id="u1", platform="matrix", chat_id="C1", command="start")
await dispatcher.dispatch(start)
msg = IncomingMessage(
user_id="u1",
platform="matrix",
chat_id="C1",
text="Посмотри файл",
attachments=[
Attachment(
type="document",
filename="report.pdf",
mime_type="application/pdf",
workspace_path="surfaces/matrix/u1/room/inbox/report.pdf",
)
],
)
await dispatcher.dispatch(msg)
assert agent_api.calls == [
("Посмотри файл", ["surfaces/matrix/u1/room/inbox/report.pdf"])
]
```
```python
# tests/core/test_dispatcher.py
async def test_dispatch_routes_document_before_catchall(dispatcher):
async def doc_handler(event, **kwargs):
return [OutgoingMessage(chat_id=event.chat_id, text="document")]
async def catch_all(event, **kwargs):
return [OutgoingMessage(chat_id=event.chat_id, text="text")]
dispatcher.register(IncomingMessage, "document", doc_handler)
dispatcher.register(IncomingMessage, "*", catch_all)
doc_msg = IncomingMessage(
user_id="u1",
platform="matrix",
chat_id="C1",
text="",
attachments=[Attachment(type="document", workspace_path="surfaces/matrix/u1/file.txt")],
)
assert (await dispatcher.dispatch(doc_msg))[0].text == "document"
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `PYTHONPATH=. uv run pytest tests/core/test_dispatcher.py tests/core/test_integration.py -q`
Expected:
- FAIL because `Attachment` has no `workspace_path`
- FAIL because `handle_message(...)` still sends `attachments=[]`
- [ ] **Step 3: Write minimal implementation**
```python
# core/protocol.py
@dataclass
class Attachment:
type: str
url: str | None = None
content: bytes | None = None
filename: str | None = None
mime_type: str | None = None
workspace_path: str | None = None
```
```python
# sdk/interface.py
class Attachment(BaseModel):
url: str | None = None
mime_type: str | None = None
size: int | None = None
filename: str | None = None
workspace_path: str | None = None
```
```python
# core/handlers/message.py
response = await platform.send_message(
user_id=event.user_id,
chat_id=event.chat_id,
text=event.text,
attachments=event.attachments,
)
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `PYTHONPATH=. uv run pytest tests/core/test_dispatcher.py tests/core/test_integration.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add core/protocol.py sdk/interface.py core/handlers/message.py tests/core/test_dispatcher.py tests/core/test_integration.py
git commit -m "feat: preserve workspace attachments through message dispatch"
```
### Task 2: Upgrade the Real SDK Bridge to Modern Agent File Events
**Files:**
- Modify: `sdk/agent_api_wrapper.py`
- Modify: `sdk/real.py`
- Test: `tests/platform/test_real.py`
- [ ] **Step 1: Write the failing tests**
```python
# tests/platform/test_real.py
class FakeSendFileEvent:
def __init__(self, path: str) -> None:
self.path = path
class FakeChatAgentApi:
...
async def send_message(self, text: str, attachments: list[str] | None = None):
self.calls.append((text, attachments or []))
midpoint = len(text) // 2
yield FakeChunk(text[:midpoint])
yield FakeChunk(text[midpoint:])
self.last_tokens_used = 3
@pytest.mark.asyncio
async def test_real_platform_client_send_message_forwards_workspace_paths():
agent_api = FakeAgentApiFactory()
client = RealPlatformClient(
agent_api=agent_api,
prototype_state=PrototypeStateStore(),
platform="matrix",
)
await client.send_message(
"@alice:example.org",
"chat-7",
"hello",
attachments=[
type("Attachment", (), {"workspace_path": "surfaces/matrix/alice/room/file.pdf"})()
],
)
assert agent_api.instances["chat-7"].calls == [
("hello", ["surfaces/matrix/alice/room/file.pdf"])
]
def test_agent_api_wrapper_treats_send_file_as_known_event(monkeypatch):
seen = []
class FakeSendFile:
type = "AGENT_EVENT_SEND_FILE"
path = "docs/result.pdf"
monkeypatch.setattr(
"sdk.agent_api_wrapper.ServerMessage.validate_json",
lambda raw: FakeSendFile(),
)
wrapper = AgentApiWrapper(agent_id="agent-1", base_url="https://agent.example.com", chat_id="7")
wrapper.callback = seen.append
wrapper._current_queue = None
# use the wrapper's dispatch branch directly inside _listen test harness
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `PYTHONPATH=. uv run pytest tests/platform/test_real.py -q`
Expected:
- FAIL because `RealPlatformClient` ignores attachments
- FAIL because `AgentApiWrapper` only recognizes text/end/error/disconnect events
- [ ] **Step 3: Write minimal implementation**
```python
# sdk/real.py
def _attachment_paths(self, attachments) -> list[str]:
if not attachments:
return []
paths = []
for attachment in attachments:
path = getattr(attachment, "workspace_path", None)
if path:
paths.append(path)
return paths
async def stream_message(...):
attachment_paths = self._attachment_paths(attachments)
...
async for event in chat_api.send_message(text, attachments=attachment_paths):
if hasattr(event, "path"):
yield MessageChunk(
message_id=user_id,
delta="",
finished=False,
)
continue
yield MessageChunk(...)
```
```python
# sdk/agent_api_wrapper.py
from lambda_agent_api.server import (
MsgError,
MsgEventCustomUpdate,
MsgEventEnd,
MsgEventSendFile,
MsgEventTextChunk,
MsgEventToolCallChunk,
MsgEventToolResult,
MsgGracefulDisconnect,
ServerMessage,
)
KNOWN_STREAM_EVENTS = (
MsgEventTextChunk,
MsgEventToolCallChunk,
MsgEventToolResult,
MsgEventCustomUpdate,
MsgEventSendFile,
MsgEventEnd,
)
if isinstance(outgoing_msg, KNOWN_STREAM_EVENTS):
if isinstance(outgoing_msg, MsgEventEnd):
self.last_tokens_used = outgoing_msg.tokens_used
if self._current_queue:
await self._current_queue.put(outgoing_msg)
elif self.callback:
self.callback(outgoing_msg)
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `PYTHONPATH=. uv run pytest tests/platform/test_real.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add sdk/agent_api_wrapper.py sdk/real.py tests/platform/test_real.py
git commit -m "feat: support attachment paths and file events in real sdk bridge"
```
### Task 3: Implement Matrix Shared-Workspace Receive/Send File Flow
**Files:**
- Create: `adapter/matrix/files.py`
- Modify: `adapter/matrix/bot.py`
- Test: `tests/adapter/matrix/test_files.py`
- Test: `tests/adapter/matrix/test_dispatcher.py`
- [ ] **Step 1: Write the failing tests**
```python
# tests/adapter/matrix/test_files.py
from pathlib import Path
import pytest
from adapter.matrix.files import build_workspace_attachment_path
def test_build_workspace_attachment_path_scopes_by_surface_user_and_room(tmp_path):
rel_path, abs_path = build_workspace_attachment_path(
workspace_root=tmp_path,
matrix_user_id="@alice:example.org",
room_id="!room:example.org",
filename="report.pdf",
timestamp="20260420-153000",
)
assert rel_path == "surfaces/matrix/alice_example.org/room_example.org/inbox/20260420-153000-report.pdf"
assert abs_path == tmp_path / rel_path
```
```python
# tests/adapter/matrix/test_dispatcher.py
async def test_bot_downloads_matrix_file_to_workspace_before_dispatch(tmp_path):
runtime = build_runtime(platform=MockPlatformClient())
await set_room_meta(
runtime.store,
"!chat1:example.org",
{
"chat_id": "C1",
"matrix_user_id": "@alice:example.org",
"platform_chat_id": "matrix:ctx-1",
},
)
client = SimpleNamespace(
user_id="@bot:example.org",
download=AsyncMock(return_value=SimpleNamespace(body=b"%PDF-1.7")),
)
bot = MatrixBot(client, runtime)
bot._send_all = AsyncMock()
runtime.dispatcher.dispatch = AsyncMock(return_value=[])
room = SimpleNamespace(room_id="!chat1:example.org")
event = SimpleNamespace(
sender="@alice:example.org",
body="Посмотри",
msgtype="m.file",
url="mxc://server/id",
mimetype="application/pdf",
replyto_event_id=None,
)
await bot.on_room_message(room, event)
dispatched = runtime.dispatcher.dispatch.await_args.args[0]
assert dispatched.attachments[0].workspace_path.endswith(".pdf")
```
```python
# tests/adapter/matrix/test_dispatcher.py
async def test_send_outgoing_uploads_file_attachment_to_matrix(tmp_path):
path = tmp_path / "result.txt"
path.write_text("ready")
client = SimpleNamespace(
upload=AsyncMock(return_value=(SimpleNamespace(content_uri="mxc://server/file"), {})),
room_send=AsyncMock(),
)
await send_outgoing(
client,
"!room:example.org",
OutgoingMessage(
chat_id="!room:example.org",
text="Файл готов",
attachments=[
Attachment(
type="document",
filename="result.txt",
mime_type="text/plain",
workspace_path=str(path),
)
],
),
)
client.upload.assert_awaited()
client.room_send.assert_awaited()
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_files.py tests/adapter/matrix/test_dispatcher.py -q`
Expected:
- FAIL because `adapter.matrix.files` does not exist
- FAIL because Matrix bot does not persist files before dispatch
- FAIL because `send_outgoing(...)` only sends text
- [ ] **Step 3: Write minimal implementation**
```python
# adapter/matrix/files.py
from __future__ import annotations
from pathlib import Path
from datetime import UTC, datetime
import re
from core.protocol import Attachment
def _sanitize_component(value: str) -> str:
stripped = re.sub(r"[^A-Za-z0-9._-]+", "_", value)
return stripped.strip("._-") or "unknown"
def build_workspace_attachment_path(
*,
workspace_root: Path,
matrix_user_id: str,
room_id: str,
filename: str,
timestamp: str | None = None,
) -> tuple[str, Path]:
stamp = timestamp or datetime.now(UTC).strftime("%Y%m%d-%H%M%S")
safe_user = _sanitize_component(matrix_user_id.lstrip("@"))
safe_room = _sanitize_component(room_id.lstrip("!"))
safe_name = _sanitize_component(filename) or "attachment.bin"
rel_path = Path("surfaces") / "matrix" / safe_user / safe_room / "inbox" / f"{stamp}-{safe_name}"
return rel_path.as_posix(), workspace_root / rel_path
```
```python
# adapter/matrix/bot.py
from adapter.matrix.files import download_matrix_attachments, load_workspace_attachment
...
incoming = from_room_event(event, room_id=room.room_id, chat_id=dispatch_chat_id)
if isinstance(incoming, IncomingMessage) and incoming.attachments:
incoming = await self._materialize_attachments(room.room_id, sender, incoming)
...
async def _materialize_attachments(...):
workspace_root = Path(os.environ.get("SURFACES_WORKSPACE_DIR", "/workspace"))
attachments = await download_matrix_attachments(...)
return IncomingMessage(..., attachments=attachments, ...)
```
```python
# adapter/matrix/bot.py
if isinstance(event, OutgoingMessage) and event.attachments:
for attachment in event.attachments:
if attachment.workspace_path:
await _send_matrix_file(client, room_id, attachment)
if event.text:
await client.room_send(...)
return
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_files.py tests/adapter/matrix/test_dispatcher.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add adapter/matrix/files.py adapter/matrix/bot.py tests/adapter/matrix/test_files.py tests/adapter/matrix/test_dispatcher.py
git commit -m "feat: add matrix shared-workspace file receive and send flow"
```
### Task 4: Make Shared Workspace the Default Local Runtime
**Files:**
- Modify: `docker-compose.yml`
- Modify: `README.md`
- Modify: `.env.example`
- [ ] **Step 1: Write the failing configuration checks**
```bash
python - <<'PY'
from pathlib import Path
text = Path("docker-compose.yml").read_text()
assert "platform-agent" in text
assert "/workspace" in text
assert "matrix-bot" in text
PY
```
```bash
python - <<'PY'
from pathlib import Path
readme = Path("README.md").read_text()
assert "docker compose up" in readme
assert "/workspace" in readme
assert "platform-agent" in readme
PY
```
- [ ] **Step 2: Run checks to verify they fail**
Run: `python - <<'PY' ... PY`
Expected:
- FAIL because root compose only defines `matrix-bot`
- FAIL because README still documents standalone `uvicorn` launch and old WS route
- [ ] **Step 3: Write minimal implementation**
```yaml
# docker-compose.yml
services:
platform-agent:
build:
context: ./external/platform-agent
target: development
additional_contexts:
agent_api: ./external/platform-agent_api
env_file:
- ./external/platform-agent/.env
volumes:
- workspace:/workspace
- ./external/platform-agent/src:/app/src
- ./external/platform-agent_api:/agent_api
ports:
- "8000:8000"
matrix-bot:
build: .
env_file: .env
depends_on:
- platform-agent
volumes:
- workspace:/workspace
restart: unless-stopped
volumes:
workspace:
```
```env
# .env.example
AGENT_WS_URL=ws://platform-agent:8000/v1/agent_ws/0/
AGENT_BASE_URL=http://platform-agent:8000
SURFACES_WORKSPACE_DIR=/workspace
MATRIX_PLATFORM_BACKEND=real
```
```md
# README.md
- make the root `docker compose up` path the primary local runtime
- describe shared `/workspace` as the file contract
- remove the statement that real backend is text-only and has no attachments
- replace the old standalone `uvicorn` instructions with compose-first instructions
```
- [ ] **Step 4: Run checks to verify they pass**
Run: `python - <<'PY' ... PY`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add docker-compose.yml README.md .env.example
git commit -m "chore: make shared workspace runtime the default local setup"
```
## Self-Review
- Spec coverage:
- shared `/workspace` runtime: Task 4
- incoming Matrix file persistence: Task 3
- attachment path propagation to agent API: Tasks 1-2
- outbound `send_file` flow: Tasks 2-3
- future-surface-friendly attachment contract: Task 1
- Placeholder scan:
- no `TODO`, `TBD`, or “similar to”
- each task has explicit test, run, implementation, verify, commit steps
- Type consistency:
- `workspace_path` is introduced in both attachment models and consumed consistently in Tasks 1-3
- path-based contract is always relative to `/workspace` until Matrix upload resolution step
## Execution Handoff
User already selected parallel subagent execution. Use subagent-driven development and split ownership like this:
- Worker A: `docker-compose.yml`, `README.md`, `.env.example`
- Worker B: `sdk/agent_api_wrapper.py`, `sdk/real.py`, `tests/platform/test_real.py`
- Main session / later worker: `core/protocol.py`, `sdk/interface.py`, `core/handlers/message.py`, `adapter/matrix/files.py`, `adapter/matrix/bot.py`, matrix/core tests

View file

@ -0,0 +1,555 @@
# Matrix Staged Attachments Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Add Matrix-side staged attachments so file-only events are stored per `(chat_id, user_id)`, acknowledged in chat, and committed to the agent on the next normal user message.
**Architecture:** Keep the existing shared-workspace file contract unchanged and add a thin Matrix-specific staging layer above it. The Matrix adapter will store staged attachment metadata in `adapter.matrix.store`, parse short staging commands in `adapter.matrix.converter`, and orchestrate commit/remove/list behavior in `adapter.matrix.bot` before calling the existing dispatcher.
**Tech Stack:** Python 3.11, matrix-nio, pytest, pytest-asyncio, in-memory state store, shared `/workspace`
---
## File Structure
- Modify: `adapter/matrix/store.py`
Purpose: store staged attachment state per `(room_id, user_id)`.
- Modify: `adapter/matrix/converter.py`
Purpose: parse `!list`, `!remove <n>`, `!remove all` into explicit Matrix-side commands.
- Modify: `adapter/matrix/bot.py`
Purpose: stage file-only events, emit service acknowledgments, process staging commands, and commit staged files with the next normal message.
- Modify: `tests/adapter/matrix/test_store.py`
Purpose: verify staged attachment persistence, ordering, and clear/remove helpers.
- Modify: `tests/adapter/matrix/test_converter.py`
Purpose: verify short staging commands parse correctly.
- Modify: `tests/adapter/matrix/test_dispatcher.py`
Purpose: verify Matrix bot staging, list/remove behavior, and commit semantics.
- Modify: `README.md`
Purpose: document the Matrix staging UX and short commands.
### Task 1: Add Per-Chat Staged Attachment Storage
**Files:**
- Modify: `adapter/matrix/store.py`
- Test: `tests/adapter/matrix/test_store.py`
- [ ] **Step 1: Write the failing tests**
```python
# tests/adapter/matrix/test_store.py
from adapter.matrix.store import (
add_staged_attachment,
clear_staged_attachments,
get_staged_attachments,
remove_staged_attachment_at,
)
async def test_staged_attachments_roundtrip(store: InMemoryStore):
await add_staged_attachment(
store,
room_id="!r1:example.org",
user_id="@alice:example.org",
attachment={
"filename": "report.pdf",
"workspace_path": "surfaces/matrix/alice/r1/inbox/report.pdf",
"mime_type": "application/pdf",
},
)
assert await get_staged_attachments(store, "!r1:example.org", "@alice:example.org") == [
{
"filename": "report.pdf",
"workspace_path": "surfaces/matrix/alice/r1/inbox/report.pdf",
"mime_type": "application/pdf",
}
]
async def test_staged_attachments_are_scoped_by_room_and_user(store: InMemoryStore):
await add_staged_attachment(
store,
room_id="!r1:example.org",
user_id="@alice:example.org",
attachment={"filename": "a.pdf", "workspace_path": "a.pdf"},
)
await add_staged_attachment(
store,
room_id="!r2:example.org",
user_id="@alice:example.org",
attachment={"filename": "b.pdf", "workspace_path": "b.pdf"},
)
await add_staged_attachment(
store,
room_id="!r1:example.org",
user_id="@bob:example.org",
attachment={"filename": "c.pdf", "workspace_path": "c.pdf"},
)
assert [item["filename"] for item in await get_staged_attachments(store, "!r1:example.org", "@alice:example.org")] == ["a.pdf"]
assert [item["filename"] for item in await get_staged_attachments(store, "!r2:example.org", "@alice:example.org")] == ["b.pdf"]
assert [item["filename"] for item in await get_staged_attachments(store, "!r1:example.org", "@bob:example.org")] == ["c.pdf"]
async def test_remove_staged_attachment_by_index(store: InMemoryStore):
await add_staged_attachment(store, "!r1:example.org", "@alice:example.org", {"filename": "a.pdf", "workspace_path": "a.pdf"})
await add_staged_attachment(store, "!r1:example.org", "@alice:example.org", {"filename": "b.pdf", "workspace_path": "b.pdf"})
removed = await remove_staged_attachment_at(store, "!r1:example.org", "@alice:example.org", 1)
assert removed["filename"] == "b.pdf"
assert [item["filename"] for item in await get_staged_attachments(store, "!r1:example.org", "@alice:example.org")] == ["a.pdf"]
async def test_clear_staged_attachments(store: InMemoryStore):
await add_staged_attachment(store, "!r1:example.org", "@alice:example.org", {"filename": "a.pdf", "workspace_path": "a.pdf"})
await clear_staged_attachments(store, "!r1:example.org", "@alice:example.org")
assert await get_staged_attachments(store, "!r1:example.org", "@alice:example.org") == []
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_store.py -q`
Expected:
- FAIL because staged attachment helper functions do not exist yet
- [ ] **Step 3: Write minimal implementation**
```python
# adapter/matrix/store.py
STAGED_ATTACHMENTS_PREFIX = "matrix_staged_attachments:"
def _staged_attachments_key(room_id: str, user_id: str) -> str:
return f"{STAGED_ATTACHMENTS_PREFIX}{room_id}:{user_id}"
async def get_staged_attachments(store: StateStore, room_id: str, user_id: str) -> list[dict]:
return list(await store.get(_staged_attachments_key(room_id, user_id)) or [])
async def add_staged_attachment(
store: StateStore,
room_id: str,
user_id: str,
attachment: dict,
) -> None:
items = await get_staged_attachments(store, room_id, user_id)
items.append(attachment)
await store.set(_staged_attachments_key(room_id, user_id), items)
async def remove_staged_attachment_at(
store: StateStore,
room_id: str,
user_id: str,
index: int,
) -> dict | None:
items = await get_staged_attachments(store, room_id, user_id)
if index < 0 or index >= len(items):
return None
removed = items.pop(index)
await store.set(_staged_attachments_key(room_id, user_id), items)
return removed
async def clear_staged_attachments(store: StateStore, room_id: str, user_id: str) -> None:
await store.delete(_staged_attachments_key(room_id, user_id))
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_store.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add adapter/matrix/store.py tests/adapter/matrix/test_store.py
git commit -m "feat: add matrix staged attachment state"
```
### Task 2: Parse Short Staging Commands
**Files:**
- Modify: `adapter/matrix/converter.py`
- Test: `tests/adapter/matrix/test_converter.py`
- [ ] **Step 1: Write the failing tests**
```python
# tests/adapter/matrix/test_converter.py
async def test_list_command_maps_to_matrix_staging_command():
result = from_room_event(text_event("!list"), room_id="!r:m.org", chat_id="C1")
assert isinstance(result, IncomingCommand)
assert result.command == "matrix_list_attachments"
assert result.args == []
async def test_remove_all_maps_to_matrix_staging_command():
result = from_room_event(text_event("!remove all"), room_id="!r:m.org", chat_id="C1")
assert isinstance(result, IncomingCommand)
assert result.command == "matrix_remove_attachment"
assert result.args == ["all"]
async def test_remove_index_maps_to_matrix_staging_command():
result = from_room_event(text_event("!remove 2"), room_id="!r:m.org", chat_id="C1")
assert isinstance(result, IncomingCommand)
assert result.command == "matrix_remove_attachment"
assert result.args == ["2"]
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_converter.py -q`
Expected:
- FAIL because `!list` and `!remove` still parse as generic unknown commands
- [ ] **Step 3: Write minimal implementation**
```python
# adapter/matrix/converter.py
def from_command(body: str, sender: str, chat_id: str, room_id: str | None = None) -> IncomingEvent:
raw = body.lstrip("!").strip()
parts = raw.split()
command = parts[0].lower() if parts else ""
args = parts[1:]
if command == "list":
return IncomingCommand(
user_id=sender,
platform=PLATFORM,
chat_id=chat_id,
command="matrix_list_attachments",
args=[],
)
if command == "remove":
return IncomingCommand(
user_id=sender,
platform=PLATFORM,
chat_id=chat_id,
command="matrix_remove_attachment",
args=args,
)
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_converter.py -q`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add adapter/matrix/converter.py tests/adapter/matrix/test_converter.py
git commit -m "feat: parse matrix staged attachment commands"
```
### Task 3: Stage File-Only Events and Handle List/Remove UX
**Files:**
- Modify: `adapter/matrix/bot.py`
- Modify: `adapter/matrix/store.py`
- Test: `tests/adapter/matrix/test_dispatcher.py`
- [ ] **Step 1: Write the failing tests**
```python
# tests/adapter/matrix/test_dispatcher.py
async def test_file_only_event_is_staged_and_does_not_dispatch():
runtime = build_runtime(platform=MockPlatformClient())
client = SimpleNamespace(user_id="@bot:example.org", room_send=AsyncMock())
bot = MatrixBot(client, runtime)
runtime.dispatcher.dispatch = AsyncMock(return_value=[])
bot._materialize_incoming_attachments = AsyncMock(
return_value=IncomingMessage(
user_id="@alice:example.org",
platform="matrix",
chat_id="matrix:!r:example.org",
text="",
attachments=[
Attachment(
type="document",
filename="report.pdf",
workspace_path="surfaces/matrix/alice/r/inbox/report.pdf",
mime_type="application/pdf",
)
],
)
)
room = SimpleNamespace(room_id="!r:example.org")
event = SimpleNamespace(
sender="@alice:example.org",
body="report.pdf",
msgtype="m.file",
url="mxc://hs/id",
mimetype="application/pdf",
replyto_event_id=None,
)
await bot.on_room_message(room, event)
runtime.dispatcher.dispatch.assert_not_awaited()
staged = await get_staged_attachments(runtime.store, "!r:example.org", "@alice:example.org")
assert [item["filename"] for item in staged] == ["report.pdf"]
client.room_send.assert_awaited_once()
assert "Следующее сообщение" in client.room_send.await_args.args[2]["body"]
async def test_list_command_returns_current_staged_attachments():
runtime = build_runtime(platform=MockPlatformClient())
await add_staged_attachment(runtime.store, "!r:example.org", "@alice:example.org", {"filename": "a.pdf", "workspace_path": "a.pdf"})
await add_staged_attachment(runtime.store, "!r:example.org", "@alice:example.org", {"filename": "b.pdf", "workspace_path": "b.pdf"})
client = SimpleNamespace(user_id="@bot:example.org", room_send=AsyncMock())
bot = MatrixBot(client, runtime)
room = SimpleNamespace(room_id="!r:example.org")
event = SimpleNamespace(sender="@alice:example.org", body="!list", msgtype="m.text", replyto_event_id=None)
await bot.on_room_message(room, event)
body = client.room_send.await_args.args[2]["body"]
assert "1. a.pdf" in body
assert "2. b.pdf" in body
async def test_remove_invalid_index_returns_short_error():
runtime = build_runtime(platform=MockPlatformClient())
await add_staged_attachment(runtime.store, "!r:example.org", "@alice:example.org", {"filename": "a.pdf", "workspace_path": "a.pdf"})
client = SimpleNamespace(user_id="@bot:example.org", room_send=AsyncMock())
bot = MatrixBot(client, runtime)
room = SimpleNamespace(room_id="!r:example.org")
event = SimpleNamespace(sender="@alice:example.org", body="!remove 9", msgtype="m.text", replyto_event_id=None)
await bot.on_room_message(room, event)
assert client.room_send.await_args.args[2]["body"] == "Нет такого вложения."
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_dispatcher.py -q`
Expected:
- FAIL because file-only events still go straight to dispatcher
- FAIL because `!list` and `!remove` have no Matrix-side handling in the bot
- [ ] **Step 3: Write minimal implementation**
```python
# adapter/matrix/bot.py
def _is_staging_command(self, incoming: IncomingEvent) -> bool:
return isinstance(incoming, IncomingCommand) and incoming.command in {
"matrix_list_attachments",
"matrix_remove_attachment",
}
async def _handle_staging_command(self, room_id: str, user_id: str, incoming: IncomingCommand) -> list[OutgoingMessage]:
if incoming.command == "matrix_list_attachments":
return [OutgoingMessage(chat_id=incoming.chat_id, text=self._format_staged_attachments(room_id, user_id))]
if incoming.command == "matrix_remove_attachment" and incoming.args == ["all"]:
await clear_staged_attachments(self.runtime.store, room_id, user_id)
return [OutgoingMessage(chat_id=incoming.chat_id, text="Вложения очищены.")]
```
```python
# adapter/matrix/bot.py
if isinstance(incoming, IncomingMessage) and incoming.attachments and not incoming.text:
incoming = await self._materialize_incoming_attachments(room.room_id, sender, incoming)
await self._stage_attachments(room.room_id, sender, incoming.attachments)
await self._send_all(room.room_id, [OutgoingMessage(chat_id=dispatch_chat_id, text=self._format_staged_attachments(room.room_id, sender, with_hint=True))])
return
if self._is_staging_command(incoming):
outgoing = await self._handle_staging_command(room.room_id, sender, incoming)
await self._send_all(room.room_id, outgoing)
return
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_dispatcher.py -q`
Expected: PASS for staging/list/remove behavior
- [ ] **Step 5: Commit**
```bash
git add adapter/matrix/bot.py adapter/matrix/store.py tests/adapter/matrix/test_dispatcher.py
git commit -m "feat: add matrix staging list and remove flow"
```
### Task 4: Commit Staged Files With the Next Normal Message
**Files:**
- Modify: `adapter/matrix/bot.py`
- Test: `tests/adapter/matrix/test_dispatcher.py`
- Modify: `README.md`
- [ ] **Step 1: Write the failing tests**
```python
# tests/adapter/matrix/test_dispatcher.py
async def test_next_normal_message_commits_staged_attachments():
runtime = build_runtime(platform=MockPlatformClient())
await add_staged_attachment(
runtime.store,
"!r:example.org",
"@alice:example.org",
{
"filename": "report.pdf",
"workspace_path": "surfaces/matrix/alice/r/inbox/report.pdf",
"mime_type": "application/pdf",
},
)
client = SimpleNamespace(user_id="@bot:example.org")
bot = MatrixBot(client, runtime)
bot._send_all = AsyncMock()
runtime.dispatcher.dispatch = AsyncMock(return_value=[])
room = SimpleNamespace(room_id="!r:example.org")
event = SimpleNamespace(sender="@alice:example.org", body="Проанализируй", msgtype="m.text", replyto_event_id=None)
await bot.on_room_message(room, event)
dispatched = runtime.dispatcher.dispatch.await_args.args[0]
assert isinstance(dispatched, IncomingMessage)
assert dispatched.text == "Проанализируй"
assert [a.workspace_path for a in dispatched.attachments] == [
"surfaces/matrix/alice/r/inbox/report.pdf"
]
assert await get_staged_attachments(runtime.store, "!r:example.org", "@alice:example.org") == []
async def test_failed_commit_preserves_staged_attachments():
runtime = build_runtime(platform=MockPlatformClient())
await add_staged_attachment(
runtime.store,
"!r:example.org",
"@alice:example.org",
{"filename": "report.pdf", "workspace_path": "surfaces/matrix/alice/r/inbox/report.pdf"},
)
client = SimpleNamespace(user_id="@bot:example.org", room_send=AsyncMock())
bot = MatrixBot(client, runtime)
runtime.dispatcher.dispatch = AsyncMock(side_effect=PlatformError("boom"))
room = SimpleNamespace(room_id="!r:example.org")
event = SimpleNamespace(sender="@alice:example.org", body="Проанализируй", msgtype="m.text", replyto_event_id=None)
await bot.on_room_message(room, event)
staged = await get_staged_attachments(runtime.store, "!r:example.org", "@alice:example.org")
assert [item["filename"] for item in staged] == ["report.pdf"]
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_dispatcher.py -q`
Expected:
- FAIL because normal text messages do not yet merge staged attachments
- FAIL because staged items are never preserved/cleared based on commit outcome
- [ ] **Step 3: Write minimal implementation**
```python
# adapter/matrix/bot.py
async def _merge_staged_attachments(
self,
room_id: str,
user_id: str,
incoming: IncomingMessage,
) -> IncomingMessage:
staged = await get_staged_attachments(self.runtime.store, room_id, user_id)
if not staged:
return incoming
return IncomingMessage(
user_id=incoming.user_id,
platform=incoming.platform,
chat_id=incoming.chat_id,
text=incoming.text,
reply_to=incoming.reply_to,
attachments=[
Attachment(
type="document",
filename=item.get("filename"),
mime_type=item.get("mime_type"),
workspace_path=item.get("workspace_path"),
)
for item in staged
],
)
```
```python
# adapter/matrix/bot.py
staged_before_dispatch = False
if isinstance(incoming, IncomingMessage) and incoming.text and not incoming.attachments:
staged = await get_staged_attachments(self.runtime.store, room.room_id, sender)
if staged:
incoming = await self._merge_staged_attachments(room.room_id, sender, incoming)
staged_before_dispatch = True
try:
outgoing = await self.runtime.dispatcher.dispatch(incoming)
except PlatformError:
...
else:
if staged_before_dispatch:
await clear_staged_attachments(self.runtime.store, room.room_id, sender)
```
- [ ] **Step 4: Run targeted tests to verify they pass**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_store.py tests/adapter/matrix/test_converter.py tests/adapter/matrix/test_dispatcher.py -q`
Expected: PASS
- [ ] **Step 5: Update docs**
Add to `README.md`:
```md
### Matrix staged attachments
If a Matrix user sends files without a text instruction, the bot stages them per chat and replies with the current list.
- `!list` shows staged files
- `!remove <n>` removes one staged file by index
- `!remove all` clears all staged files
The next normal user message is sent to the agent together with all staged files.
```
- [ ] **Step 6: Run broader verification**
Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_store.py tests/adapter/matrix/test_converter.py tests/adapter/matrix/test_dispatcher.py tests/adapter/matrix/test_files.py tests/adapter/matrix/test_send_outgoing.py tests/core/test_integration.py tests/platform/test_real.py -q`
Expected: PASS
- [ ] **Step 7: Commit**
```bash
git add adapter/matrix/bot.py README.md tests/adapter/matrix/test_store.py tests/adapter/matrix/test_converter.py tests/adapter/matrix/test_dispatcher.py
git commit -m "feat: commit staged matrix attachments on next message"
```
## Self-Review
- Spec coverage:
- staged per `(chat_id, user_id)`: Task 1
- short commands `!list`, `!remove <n>`, `!remove all`: Task 2 and Task 3
- file-only events do not invoke agent: Task 3
- next normal message commits staged attachments: Task 4
- failed commit preserves staged attachments: Task 4
- docs update: Task 4
- Placeholder scan:
- no `TODO`, `TBD`, or deferred behavior left in task steps
- Type consistency:
- staged storage uses `dict` records with `filename`, `workspace_path`, `mime_type`
- bot reconstructs `core.protocol.Attachment` from those same keys