surfaces/docs/superpowers/plans/2026-04-20-matrix-shared-workspace-file-flow.md

Matrix Shared Workspace File Flow Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Move the Matrix surface to a prod-like shared-workspace file flow where incoming files are downloaded into /workspace, passed to platform-agent as relative attachment paths, and outbound send_file events are delivered back to the Matrix room.

Architecture: Keep the platform contract path-based and surface-agnostic. Extend the surface attachment model with a workspace-relative path, update the real SDK bridge to forward attachments and modern agent events, then add a Matrix-specific storage helper plus a shared-volume runtime. This preserves room/context semantics while aligning file flow with upstream platform-agent.

Tech Stack: Python 3.11, matrix-nio, aiohttp, Docker Compose, pytest, pytest-asyncio

---

File Structure

• Modify: core/protocol.py Purpose: add a workspace-relative attachment field that future surfaces can also use.
• Modify: sdk/interface.py Purpose: keep the platform-side attachment shape aligned with the surface model.
• Modify: core/handlers/message.py Purpose: stop dropping attachments before platform dispatch.
• Modify: sdk/agent_api_wrapper.py Purpose: accept modern upstream agent events and modern WS route semantics.
• Modify: sdk/real.py Purpose: convert attachment objects into workspace-relative paths and forward them to the agent API.
• Create: adapter/matrix/files.py Purpose: Matrix-specific download/upload helper for shared /workspace.
• Modify: adapter/matrix/bot.py Purpose: persist incoming Matrix files into shared workspace and render outbound file events back to Matrix.
• Modify: tests/core/test_integration.py Purpose: prove message dispatch keeps attachments and platform send path receives them.
• Modify: tests/platform/test_real.py Purpose: verify attachment forwarding and outbound file events.
• Create: tests/adapter/matrix/test_files.py Purpose: unit coverage for Matrix workspace path building, download handling, and outbound upload behavior.
• Modify: tests/adapter/matrix/test_dispatcher.py Purpose: verify Matrix bot file receive/send integration.
• Modify: docker-compose.yml Purpose: define shared /workspace runtime between matrix-bot and platform-agent.
• Modify: README.md Purpose: document the new default runtime and file flow.
• Modify: .env.example Purpose: update real-backend defaults to in-compose service URLs and workspace-aware runtime.

Task 1: Preserve Attachment Metadata Through Core Message Dispatch

Files:

• Modify: core/protocol.py

• Modify: sdk/interface.py

• Modify: core/handlers/message.py

• Test: tests/core/test_dispatcher.py

• Test: tests/core/test_integration.py

• Step 1: Write the failing tests

# tests/core/test_integration.py
class RecordingAgentApi:
    def __init__(self) -> None:
        self.calls: list[tuple[str, list[str]]] = []
        self.last_tokens_used = 0

    async def send_message(self, text: str, attachments: list[str] | None = None):
        self.calls.append((text, attachments or []))
        yield type("Chunk", (), {"text": f"[REAL] {text}"})()
        self.last_tokens_used = 5


async def test_full_flow_with_real_platform_forwards_workspace_attachment(real_dispatcher):
    dispatcher, agent_api = real_dispatcher

    start = IncomingCommand(user_id="u1", platform="matrix", chat_id="C1", command="start")
    await dispatcher.dispatch(start)

    msg = IncomingMessage(
        user_id="u1",
        platform="matrix",
        chat_id="C1",
        text="Посмотри файл",
        attachments=[
            Attachment(
                type="document",
                filename="report.pdf",
                mime_type="application/pdf",
                workspace_path="surfaces/matrix/u1/room/inbox/report.pdf",
            )
        ],
    )
    await dispatcher.dispatch(msg)

    assert agent_api.calls == [
        ("Посмотри файл", ["surfaces/matrix/u1/room/inbox/report.pdf"])
    ]

# tests/core/test_dispatcher.py
async def test_dispatch_routes_document_before_catchall(dispatcher):
    async def doc_handler(event, **kwargs):
        return [OutgoingMessage(chat_id=event.chat_id, text="document")]

    async def catch_all(event, **kwargs):
        return [OutgoingMessage(chat_id=event.chat_id, text="text")]

    dispatcher.register(IncomingMessage, "document", doc_handler)
    dispatcher.register(IncomingMessage, "*", catch_all)

    doc_msg = IncomingMessage(
        user_id="u1",
        platform="matrix",
        chat_id="C1",
        text="",
        attachments=[Attachment(type="document", workspace_path="surfaces/matrix/u1/file.txt")],
    )

    assert (await dispatcher.dispatch(doc_msg))[0].text == "document"

• Step 2: Run tests to verify they fail

Run: PYTHONPATH=. uv run pytest tests/core/test_dispatcher.py tests/core/test_integration.py -q

Expected:

• FAIL because Attachment has no workspace_path

• FAIL because handle_message(...) still sends attachments=[]

• Step 3: Write minimal implementation

# core/protocol.py
@dataclass
class Attachment:
    type: str
    url: str | None = None
    content: bytes | None = None
    filename: str | None = None
    mime_type: str | None = None
    workspace_path: str | None = None

# sdk/interface.py
class Attachment(BaseModel):
    url: str | None = None
    mime_type: str | None = None
    size: int | None = None
    filename: str | None = None
    workspace_path: str | None = None

# core/handlers/message.py
response = await platform.send_message(
    user_id=event.user_id,
    chat_id=event.chat_id,
    text=event.text,
    attachments=event.attachments,
)

• Step 4: Run tests to verify they pass

Run: PYTHONPATH=. uv run pytest tests/core/test_dispatcher.py tests/core/test_integration.py -q

Expected: PASS

• Step 5: Commit

git add core/protocol.py sdk/interface.py core/handlers/message.py tests/core/test_dispatcher.py tests/core/test_integration.py
git commit -m "feat: preserve workspace attachments through message dispatch"

Task 2: Upgrade the Real SDK Bridge to Modern Agent File Events

Files:

• Modify: sdk/agent_api_wrapper.py

• Modify: sdk/real.py

• Test: tests/platform/test_real.py

• Step 1: Write the failing tests

# tests/platform/test_real.py
class FakeSendFileEvent:
    def __init__(self, path: str) -> None:
        self.path = path


class FakeChatAgentApi:
    ...
    async def send_message(self, text: str, attachments: list[str] | None = None):
        self.calls.append((text, attachments or []))
        midpoint = len(text) // 2
        yield FakeChunk(text[:midpoint])
        yield FakeChunk(text[midpoint:])
        self.last_tokens_used = 3


@pytest.mark.asyncio
async def test_real_platform_client_send_message_forwards_workspace_paths():
    agent_api = FakeAgentApiFactory()
    client = RealPlatformClient(
        agent_api=agent_api,
        prototype_state=PrototypeStateStore(),
        platform="matrix",
    )

    await client.send_message(
        "@alice:example.org",
        "chat-7",
        "hello",
        attachments=[
            type("Attachment", (), {"workspace_path": "surfaces/matrix/alice/room/file.pdf"})()
        ],
    )

    assert agent_api.instances["chat-7"].calls == [
        ("hello", ["surfaces/matrix/alice/room/file.pdf"])
    ]


def test_agent_api_wrapper_treats_send_file_as_known_event(monkeypatch):
    seen = []

    class FakeSendFile:
        type = "AGENT_EVENT_SEND_FILE"
        path = "docs/result.pdf"

    monkeypatch.setattr(
        "sdk.agent_api_wrapper.ServerMessage.validate_json",
        lambda raw: FakeSendFile(),
    )

    wrapper = AgentApiWrapper(agent_id="agent-1", base_url="https://agent.example.com", chat_id="7")
    wrapper.callback = seen.append
    wrapper._current_queue = None

    # use the wrapper's dispatch branch directly inside _listen test harness

• Step 2: Run tests to verify they fail

Run: PYTHONPATH=. uv run pytest tests/platform/test_real.py -q

Expected:

• FAIL because RealPlatformClient ignores attachments

• FAIL because AgentApiWrapper only recognizes text/end/error/disconnect events

• Step 3: Write minimal implementation

# sdk/real.py
def _attachment_paths(self, attachments) -> list[str]:
    if not attachments:
        return []
    paths = []
    for attachment in attachments:
        path = getattr(attachment, "workspace_path", None)
        if path:
            paths.append(path)
    return paths

async def stream_message(...):
    attachment_paths = self._attachment_paths(attachments)
    ...
    async for event in chat_api.send_message(text, attachments=attachment_paths):
        if hasattr(event, "path"):
            yield MessageChunk(
                message_id=user_id,
                delta="",
                finished=False,
            )
            continue
        yield MessageChunk(...)

# sdk/agent_api_wrapper.py
from lambda_agent_api.server import (
    MsgError,
    MsgEventCustomUpdate,
    MsgEventEnd,
    MsgEventSendFile,
    MsgEventTextChunk,
    MsgEventToolCallChunk,
    MsgEventToolResult,
    MsgGracefulDisconnect,
    ServerMessage,
)

KNOWN_STREAM_EVENTS = (
    MsgEventTextChunk,
    MsgEventToolCallChunk,
    MsgEventToolResult,
    MsgEventCustomUpdate,
    MsgEventSendFile,
    MsgEventEnd,
)

if isinstance(outgoing_msg, KNOWN_STREAM_EVENTS):
    if isinstance(outgoing_msg, MsgEventEnd):
        self.last_tokens_used = outgoing_msg.tokens_used
    if self._current_queue:
        await self._current_queue.put(outgoing_msg)
    elif self.callback:
        self.callback(outgoing_msg)

• Step 4: Run tests to verify they pass

Run: PYTHONPATH=. uv run pytest tests/platform/test_real.py -q

Expected: PASS

• Step 5: Commit

git add sdk/agent_api_wrapper.py sdk/real.py tests/platform/test_real.py
git commit -m "feat: support attachment paths and file events in real sdk bridge"

Task 3: Implement Matrix Shared-Workspace Receive/Send File Flow

Files:

• Create: adapter/matrix/files.py

• Modify: adapter/matrix/bot.py

• Test: tests/adapter/matrix/test_files.py

• Test: tests/adapter/matrix/test_dispatcher.py

• Step 1: Write the failing tests

# tests/adapter/matrix/test_files.py
from pathlib import Path

import pytest

from adapter.matrix.files import build_workspace_attachment_path


def test_build_workspace_attachment_path_scopes_by_surface_user_and_room(tmp_path):
    rel_path, abs_path = build_workspace_attachment_path(
        workspace_root=tmp_path,
        matrix_user_id="@alice:example.org",
        room_id="!room:example.org",
        filename="report.pdf",
        timestamp="20260420-153000",
    )

    assert rel_path == "surfaces/matrix/alice_example.org/room_example.org/inbox/20260420-153000-report.pdf"
    assert abs_path == tmp_path / rel_path

# tests/adapter/matrix/test_dispatcher.py
async def test_bot_downloads_matrix_file_to_workspace_before_dispatch(tmp_path):
    runtime = build_runtime(platform=MockPlatformClient())
    await set_room_meta(
        runtime.store,
        "!chat1:example.org",
        {
            "chat_id": "C1",
            "matrix_user_id": "@alice:example.org",
            "platform_chat_id": "matrix:ctx-1",
        },
    )
    client = SimpleNamespace(
        user_id="@bot:example.org",
        download=AsyncMock(return_value=SimpleNamespace(body=b"%PDF-1.7")),
    )
    bot = MatrixBot(client, runtime)
    bot._send_all = AsyncMock()
    runtime.dispatcher.dispatch = AsyncMock(return_value=[])
    room = SimpleNamespace(room_id="!chat1:example.org")
    event = SimpleNamespace(
        sender="@alice:example.org",
        body="Посмотри",
        msgtype="m.file",
        url="mxc://server/id",
        mimetype="application/pdf",
        replyto_event_id=None,
    )

    await bot.on_room_message(room, event)

    dispatched = runtime.dispatcher.dispatch.await_args.args[0]
    assert dispatched.attachments[0].workspace_path.endswith(".pdf")

# tests/adapter/matrix/test_dispatcher.py
async def test_send_outgoing_uploads_file_attachment_to_matrix(tmp_path):
    path = tmp_path / "result.txt"
    path.write_text("ready")
    client = SimpleNamespace(
        upload=AsyncMock(return_value=(SimpleNamespace(content_uri="mxc://server/file"), {})),
        room_send=AsyncMock(),
    )

    await send_outgoing(
        client,
        "!room:example.org",
        OutgoingMessage(
            chat_id="!room:example.org",
            text="Файл готов",
            attachments=[
                Attachment(
                    type="document",
                    filename="result.txt",
                    mime_type="text/plain",
                    workspace_path=str(path),
                )
            ],
        ),
    )

    client.upload.assert_awaited()
    client.room_send.assert_awaited()

• Step 2: Run tests to verify they fail

Run: PYTHONPATH=. uv run pytest tests/adapter/matrix/test_files.py tests/adapter/matrix/test_dispatcher.py -q

Expected:

• FAIL because adapter.matrix.files does not exist

• FAIL because Matrix bot does not persist files before dispatch

• FAIL because send_outgoing(...) only sends text

• Step 3: Write minimal implementation

# adapter/matrix/files.py
from __future__ import annotations

from pathlib import Path
from datetime import UTC, datetime
import re

from core.protocol import Attachment


def _sanitize_component(value: str) -> str:
    stripped = re.sub(r"[^A-Za-z0-9._-]+", "_", value)
    return stripped.strip("._-") or "unknown"


def build_workspace_attachment_path(
    *,
    workspace_root: Path,
    matrix_user_id: str,
    room_id: str,
    filename: str,
    timestamp: str | None = None,
) -> tuple[str, Path]:
    stamp = timestamp or datetime.now(UTC).strftime("%Y%m%d-%H%M%S")
    safe_user = _sanitize_component(matrix_user_id.lstrip("@"))
    safe_room = _sanitize_component(room_id.lstrip("!"))
    safe_name = _sanitize_component(filename) or "attachment.bin"
    rel_path = Path("surfaces") / "matrix" / safe_user / safe_room / "inbox" / f"{stamp}-{safe_name}"
    return rel_path.as_posix(), workspace_root / rel_path

# adapter/matrix/bot.py
from adapter.matrix.files import download_matrix_attachments, load_workspace_attachment

...
incoming = from_room_event(event, room_id=room.room_id, chat_id=dispatch_chat_id)
if isinstance(incoming, IncomingMessage) and incoming.attachments:
    incoming = await self._materialize_attachments(room.room_id, sender, incoming)
...

async def _materialize_attachments(...):
    workspace_root = Path(os.environ.get("SURFACES_WORKSPACE_DIR", "/workspace"))
    attachments = await download_matrix_attachments(...)
    return IncomingMessage(..., attachments=attachments, ...)

# adapter/matrix/bot.py
if isinstance(event, OutgoingMessage) and event.attachments:
    for attachment in event.attachments:
        if attachment.workspace_path:
            await _send_matrix_file(client, room_id, attachment)
    if event.text:
        await client.room_send(...)
    return

• Step 4: Run tests to verify they pass

Run: PYTHONPATH=. uv run pytest tests/adapter/matrix/test_files.py tests/adapter/matrix/test_dispatcher.py -q

Expected: PASS

• Step 5: Commit

git add adapter/matrix/files.py adapter/matrix/bot.py tests/adapter/matrix/test_files.py tests/adapter/matrix/test_dispatcher.py
git commit -m "feat: add matrix shared-workspace file receive and send flow"

Task 4: Make Shared Workspace the Default Local Runtime

Files:

• Modify: docker-compose.yml

• Modify: README.md

• Modify: .env.example

• Step 1: Write the failing configuration checks

python - <<'PY'
from pathlib import Path
text = Path("docker-compose.yml").read_text()
assert "platform-agent" in text
assert "/workspace" in text
assert "matrix-bot" in text
PY

python - <<'PY'
from pathlib import Path
readme = Path("README.md").read_text()
assert "docker compose up" in readme
assert "/workspace" in readme
assert "platform-agent" in readme
PY

• Step 2: Run checks to verify they fail

Run: python - <<'PY' ... PY

Expected:

• FAIL because root compose only defines matrix-bot

• FAIL because README still documents standalone uvicorn launch and old WS route

• Step 3: Write minimal implementation

# docker-compose.yml
services:
  platform-agent:
    build:
      context: ./external/platform-agent
      target: development
      additional_contexts:
        agent_api: ./external/platform-agent_api
    env_file:
      - ./external/platform-agent/.env
    volumes:
      - workspace:/workspace
      - ./external/platform-agent/src:/app/src
      - ./external/platform-agent_api:/agent_api
    ports:
      - "8000:8000"

  matrix-bot:
    build: .
    env_file: .env
    depends_on:
      - platform-agent
    volumes:
      - workspace:/workspace
    restart: unless-stopped

volumes:
  workspace:

# .env.example
AGENT_WS_URL=ws://platform-agent:8000/v1/agent_ws/0/
AGENT_BASE_URL=http://platform-agent:8000
SURFACES_WORKSPACE_DIR=/workspace
MATRIX_PLATFORM_BACKEND=real

# README.md
- make the root `docker compose up` path the primary local runtime
- describe shared `/workspace` as the file contract
- remove the statement that real backend is text-only and has no attachments
- replace the old standalone `uvicorn` instructions with compose-first instructions

• Step 4: Run checks to verify they pass

Run: python - <<'PY' ... PY

Expected: PASS

• Step 5: Commit

git add docker-compose.yml README.md .env.example
git commit -m "chore: make shared workspace runtime the default local setup"

Self-Review

• Spec coverage:
  • shared /workspace runtime: Task 4
  • incoming Matrix file persistence: Task 3
  • attachment path propagation to agent API: Tasks 1-2
  • outbound send_file flow: Tasks 2-3
  • future-surface-friendly attachment contract: Task 1
• Placeholder scan:
  • no TODO, TBD, or “similar to”
  • each task has explicit test, run, implementation, verify, commit steps
• Type consistency:
  • workspace_path is introduced in both attachment models and consumed consistently in Tasks 1-3
  • path-based contract is always relative to /workspace until Matrix upload resolution step

Execution Handoff

User already selected parallel subagent execution. Use subagent-driven development and split ownership like this:

• Worker A: docker-compose.yml, README.md, .env.example
• Worker B: sdk/agent_api_wrapper.py, sdk/real.py, tests/platform/test_real.py
• Main session / later worker: core/protocol.py, sdk/interface.py, core/handlers/message.py, adapter/matrix/files.py, adapter/matrix/bot.py, matrix/core tests