# Matrix Shared Workspace File Flow Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Move the Matrix surface to a prod-like shared-workspace file flow where incoming files are downloaded into `/workspace`, passed to `platform-agent` as relative attachment paths, and outbound `send_file` events are delivered back to the Matrix room.

**Architecture:** Keep the platform contract path-based and surface-agnostic. Extend the surface attachment model with a workspace-relative path, update the real SDK bridge to forward attachments and modern agent events, then add a Matrix-specific storage helper plus a shared-volume runtime. This preserves room/context semantics while aligning file flow with upstream `platform-agent`.

**Tech Stack:** Python 3.11, matrix-nio, aiohttp, Docker Compose, pytest, pytest-asyncio

---

## File Structure

- Modify: `core/protocol.py`
  Purpose: add a workspace-relative attachment field that future surfaces can also use.
- Modify: `sdk/interface.py`
  Purpose: keep the platform-side attachment shape aligned with the surface model.
- Modify: `core/handlers/message.py`
  Purpose: stop dropping attachments before platform dispatch.
- Modify: `sdk/agent_api_wrapper.py`
  Purpose: accept modern upstream agent events and modern WS route semantics.
- Modify: `sdk/real.py`
  Purpose: convert attachment objects into workspace-relative paths and forward them to the agent API.
- Create: `adapter/matrix/files.py`
  Purpose: Matrix-specific download/upload helper for shared `/workspace`.
- Modify: `adapter/matrix/bot.py`
  Purpose: persist incoming Matrix files into shared workspace and render outbound file events back to Matrix.
- Modify: `tests/core/test_integration.py`
  Purpose: prove message dispatch keeps attachments and platform send path receives them.
- Modify: `tests/platform/test_real.py`
  Purpose: verify attachment forwarding and outbound file events.
- Create: `tests/adapter/matrix/test_files.py`
  Purpose: unit coverage for Matrix workspace path building, download handling, and outbound upload behavior.
- Modify: `tests/adapter/matrix/test_dispatcher.py`
  Purpose: verify Matrix bot file receive/send integration.
- Modify: `docker-compose.yml`
  Purpose: define shared `/workspace` runtime between `matrix-bot` and `platform-agent`.
- Modify: `README.md`
  Purpose: document the new default runtime and file flow.
- Modify: `.env.example`
  Purpose: update real-backend defaults to in-compose service URLs and workspace-aware runtime.

### Task 1: Preserve Attachment Metadata Through Core Message Dispatch

**Files:**
- Modify: `core/protocol.py`
- Modify: `sdk/interface.py`
- Modify: `core/handlers/message.py`
- Test: `tests/core/test_dispatcher.py`
- Test: `tests/core/test_integration.py`

- [ ] **Step 1: Write the failing tests**

```python
# tests/core/test_integration.py
class RecordingAgentApi:
    def __init__(self) -> None:
        self.calls: list[tuple[str, list[str]]] = []
        self.last_tokens_used = 0

    async def send_message(self, text: str, attachments: list[str] | None = None):
        self.calls.append((text, attachments or []))
        yield type("Chunk", (), {"text": f"[REAL] {text}"})()
        self.last_tokens_used = 5


async def test_full_flow_with_real_platform_forwards_workspace_attachment(real_dispatcher):
    dispatcher, agent_api = real_dispatcher

    start = IncomingCommand(user_id="u1", platform="matrix", chat_id="C1", command="start")
    await dispatcher.dispatch(start)

    msg = IncomingMessage(
        user_id="u1",
        platform="matrix",
        chat_id="C1",
        text="Посмотри файл",
        attachments=[
            Attachment(
                type="document",
                filename="report.pdf",
                mime_type="application/pdf",
                workspace_path="surfaces/matrix/u1/room/inbox/report.pdf",
            )
        ],
    )
    await dispatcher.dispatch(msg)

    assert agent_api.calls == [
        ("Посмотри файл", ["surfaces/matrix/u1/room/inbox/report.pdf"])
    ]
```

```python
# tests/core/test_dispatcher.py
async def test_dispatch_routes_document_before_catchall(dispatcher):
    async def doc_handler(event, **kwargs):
        return [OutgoingMessage(chat_id=event.chat_id, text="document")]

    async def catch_all(event, **kwargs):
        return [OutgoingMessage(chat_id=event.chat_id, text="text")]

    dispatcher.register(IncomingMessage, "document", doc_handler)
    dispatcher.register(IncomingMessage, "*", catch_all)

    doc_msg = IncomingMessage(
        user_id="u1",
        platform="matrix",
        chat_id="C1",
        text="",
        attachments=[Attachment(type="document", workspace_path="surfaces/matrix/u1/file.txt")],
    )

    assert (await dispatcher.dispatch(doc_msg))[0].text == "document"
```

- [ ] **Step 2: Run tests to verify they fail**

Run: `PYTHONPATH=. uv run pytest tests/core/test_dispatcher.py tests/core/test_integration.py -q`

Expected:
- FAIL because `Attachment` has no `workspace_path`
- FAIL because `handle_message(...)` still sends `attachments=[]`

- [ ] **Step 3: Write minimal implementation**

```python
# core/protocol.py
@dataclass
class Attachment:
    type: str
    url: str | None = None
    content: bytes | None = None
    filename: str | None = None
    mime_type: str | None = None
    workspace_path: str | None = None
```

```python
# sdk/interface.py
class Attachment(BaseModel):
    url: str | None = None
    mime_type: str | None = None
    size: int | None = None
    filename: str | None = None
    workspace_path: str | None = None
```

```python
# core/handlers/message.py
response = await platform.send_message(
    user_id=event.user_id,
    chat_id=event.chat_id,
    text=event.text,
    attachments=event.attachments,
)
```

- [ ] **Step 4: Run tests to verify they pass**

Run: `PYTHONPATH=. uv run pytest tests/core/test_dispatcher.py tests/core/test_integration.py -q`

Expected: PASS

- [ ] **Step 5: Commit**

```bash
git add core/protocol.py sdk/interface.py core/handlers/message.py tests/core/test_dispatcher.py tests/core/test_integration.py
git commit -m "feat: preserve workspace attachments through message dispatch"
```

### Task 2: Upgrade the Real SDK Bridge to Modern Agent File Events

**Files:**
- Modify: `sdk/agent_api_wrapper.py`
- Modify: `sdk/real.py`
- Test: `tests/platform/test_real.py`

- [ ] **Step 1: Write the failing tests**

```python
# tests/platform/test_real.py
class FakeSendFileEvent:
    def __init__(self, path: str) -> None:
        self.path = path


class FakeChatAgentApi:
    ...
    async def send_message(self, text: str, attachments: list[str] | None = None):
        self.calls.append((text, attachments or []))
        midpoint = len(text) // 2
        yield FakeChunk(text[:midpoint])
        yield FakeChunk(text[midpoint:])
        self.last_tokens_used = 3


@pytest.mark.asyncio
async def test_real_platform_client_send_message_forwards_workspace_paths():
    agent_api = FakeAgentApiFactory()
    client = RealPlatformClient(
        agent_api=agent_api,
        prototype_state=PrototypeStateStore(),
        platform="matrix",
    )

    await client.send_message(
        "@alice:example.org",
        "chat-7",
        "hello",
        attachments=[
            type("Attachment", (), {"workspace_path": "surfaces/matrix/alice/room/file.pdf"})()
        ],
    )

    assert agent_api.instances["chat-7"].calls == [
        ("hello", ["surfaces/matrix/alice/room/file.pdf"])
    ]


def test_agent_api_wrapper_treats_send_file_as_known_event(monkeypatch):
    seen = []

    class FakeSendFile:
        type = "AGENT_EVENT_SEND_FILE"
        path = "docs/result.pdf"

    monkeypatch.setattr(
        "sdk.agent_api_wrapper.ServerMessage.validate_json",
        lambda raw: FakeSendFile(),
    )

    wrapper = AgentApiWrapper(agent_id="agent-1", base_url="https://agent.example.com", chat_id="7")
    wrapper.callback = seen.append
    wrapper._current_queue = None

    # use the wrapper's dispatch branch directly inside _listen test harness
```

- [ ] **Step 2: Run tests to verify they fail**

Run: `PYTHONPATH=. uv run pytest tests/platform/test_real.py -q`

Expected:
- FAIL because `RealPlatformClient` ignores attachments
- FAIL because `AgentApiWrapper` only recognizes text/end/error/disconnect events

- [ ] **Step 3: Write minimal implementation**

```python
# sdk/real.py
def _attachment_paths(self, attachments) -> list[str]:
    if not attachments:
        return []
    paths = []
    for attachment in attachments:
        path = getattr(attachment, "workspace_path", None)
        if path:
            paths.append(path)
    return paths

async def stream_message(...):
    attachment_paths = self._attachment_paths(attachments)
    ...
    async for event in chat_api.send_message(text, attachments=attachment_paths):
        if hasattr(event, "path"):
            yield MessageChunk(
                message_id=user_id,
                delta="",
                finished=False,
            )
            continue
        yield MessageChunk(...)
```

```python
# sdk/agent_api_wrapper.py
from lambda_agent_api.server import (
    MsgError,
    MsgEventCustomUpdate,
    MsgEventEnd,
    MsgEventSendFile,
    MsgEventTextChunk,
    MsgEventToolCallChunk,
    MsgEventToolResult,
    MsgGracefulDisconnect,
    ServerMessage,
)

KNOWN_STREAM_EVENTS = (
    MsgEventTextChunk,
    MsgEventToolCallChunk,
    MsgEventToolResult,
    MsgEventCustomUpdate,
    MsgEventSendFile,
    MsgEventEnd,
)

if isinstance(outgoing_msg, KNOWN_STREAM_EVENTS):
    if isinstance(outgoing_msg, MsgEventEnd):
        self.last_tokens_used = outgoing_msg.tokens_used
    if self._current_queue:
        await self._current_queue.put(outgoing_msg)
    elif self.callback:
        self.callback(outgoing_msg)
```

- [ ] **Step 4: Run tests to verify they pass**

Run: `PYTHONPATH=. uv run pytest tests/platform/test_real.py -q`

Expected: PASS

- [ ] **Step 5: Commit**

```bash
git add sdk/agent_api_wrapper.py sdk/real.py tests/platform/test_real.py
git commit -m "feat: support attachment paths and file events in real sdk bridge"
```

### Task 3: Implement Matrix Shared-Workspace Receive/Send File Flow

**Files:**
- Create: `adapter/matrix/files.py`
- Modify: `adapter/matrix/bot.py`
- Test: `tests/adapter/matrix/test_files.py`
- Test: `tests/adapter/matrix/test_dispatcher.py`

- [ ] **Step 1: Write the failing tests**

```python
# tests/adapter/matrix/test_files.py
from pathlib import Path

import pytest

from adapter.matrix.files import build_workspace_attachment_path


def test_build_workspace_attachment_path_scopes_by_surface_user_and_room(tmp_path):
    rel_path, abs_path = build_workspace_attachment_path(
        workspace_root=tmp_path,
        matrix_user_id="@alice:example.org",
        room_id="!room:example.org",
        filename="report.pdf",
        timestamp="20260420-153000",
    )

    assert rel_path == "surfaces/matrix/alice_example.org/room_example.org/inbox/20260420-153000-report.pdf"
    assert abs_path == tmp_path / rel_path
```

```python
# tests/adapter/matrix/test_dispatcher.py
async def test_bot_downloads_matrix_file_to_workspace_before_dispatch(tmp_path):
    runtime = build_runtime(platform=MockPlatformClient())
    await set_room_meta(
        runtime.store,
        "!chat1:example.org",
        {
            "chat_id": "C1",
            "matrix_user_id": "@alice:example.org",
            "platform_chat_id": "matrix:ctx-1",
        },
    )
    client = SimpleNamespace(
        user_id="@bot:example.org",
        download=AsyncMock(return_value=SimpleNamespace(body=b"%PDF-1.7")),
    )
    bot = MatrixBot(client, runtime)
    bot._send_all = AsyncMock()
    runtime.dispatcher.dispatch = AsyncMock(return_value=[])
    room = SimpleNamespace(room_id="!chat1:example.org")
    event = SimpleNamespace(
        sender="@alice:example.org",
        body="Посмотри",
        msgtype="m.file",
        url="mxc://server/id",
        mimetype="application/pdf",
        replyto_event_id=None,
    )

    await bot.on_room_message(room, event)

    dispatched = runtime.dispatcher.dispatch.await_args.args[0]
    assert dispatched.attachments[0].workspace_path.endswith(".pdf")
```

```python
# tests/adapter/matrix/test_dispatcher.py
async def test_send_outgoing_uploads_file_attachment_to_matrix(tmp_path):
    path = tmp_path / "result.txt"
    path.write_text("ready")
    client = SimpleNamespace(
        upload=AsyncMock(return_value=(SimpleNamespace(content_uri="mxc://server/file"), {})),
        room_send=AsyncMock(),
    )

    await send_outgoing(
        client,
        "!room:example.org",
        OutgoingMessage(
            chat_id="!room:example.org",
            text="Файл готов",
            attachments=[
                Attachment(
                    type="document",
                    filename="result.txt",
                    mime_type="text/plain",
                    workspace_path=str(path),
                )
            ],
        ),
    )

    client.upload.assert_awaited()
    client.room_send.assert_awaited()
```

- [ ] **Step 2: Run tests to verify they fail**

Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_files.py tests/adapter/matrix/test_dispatcher.py -q`

Expected:
- FAIL because `adapter.matrix.files` does not exist
- FAIL because Matrix bot does not persist files before dispatch
- FAIL because `send_outgoing(...)` only sends text

- [ ] **Step 3: Write minimal implementation**

```python
# adapter/matrix/files.py
from __future__ import annotations

from pathlib import Path
from datetime import UTC, datetime
import re

from core.protocol import Attachment


def _sanitize_component(value: str) -> str:
    stripped = re.sub(r"[^A-Za-z0-9._-]+", "_", value)
    return stripped.strip("._-") or "unknown"


def build_workspace_attachment_path(
    *,
    workspace_root: Path,
    matrix_user_id: str,
    room_id: str,
    filename: str,
    timestamp: str | None = None,
) -> tuple[str, Path]:
    stamp = timestamp or datetime.now(UTC).strftime("%Y%m%d-%H%M%S")
    safe_user = _sanitize_component(matrix_user_id.lstrip("@"))
    safe_room = _sanitize_component(room_id.lstrip("!"))
    safe_name = _sanitize_component(filename) or "attachment.bin"
    rel_path = Path("surfaces") / "matrix" / safe_user / safe_room / "inbox" / f"{stamp}-{safe_name}"
    return rel_path.as_posix(), workspace_root / rel_path
```

```python
# adapter/matrix/bot.py
from adapter.matrix.files import download_matrix_attachments, load_workspace_attachment

...
incoming = from_room_event(event, room_id=room.room_id, chat_id=dispatch_chat_id)
if isinstance(incoming, IncomingMessage) and incoming.attachments:
    incoming = await self._materialize_attachments(room.room_id, sender, incoming)
...

async def _materialize_attachments(...):
    workspace_root = Path(os.environ.get("SURFACES_WORKSPACE_DIR", "/workspace"))
    attachments = await download_matrix_attachments(...)
    return IncomingMessage(..., attachments=attachments, ...)
```

```python
# adapter/matrix/bot.py
if isinstance(event, OutgoingMessage) and event.attachments:
    for attachment in event.attachments:
        if attachment.workspace_path:
            await _send_matrix_file(client, room_id, attachment)
    if event.text:
        await client.room_send(...)
    return
```

- [ ] **Step 4: Run tests to verify they pass**

Run: `PYTHONPATH=. uv run pytest tests/adapter/matrix/test_files.py tests/adapter/matrix/test_dispatcher.py -q`

Expected: PASS

- [ ] **Step 5: Commit**

```bash
git add adapter/matrix/files.py adapter/matrix/bot.py tests/adapter/matrix/test_files.py tests/adapter/matrix/test_dispatcher.py
git commit -m "feat: add matrix shared-workspace file receive and send flow"
```

### Task 4: Make Shared Workspace the Default Local Runtime

**Files:**
- Modify: `docker-compose.yml`
- Modify: `README.md`
- Modify: `.env.example`

- [ ] **Step 1: Write the failing configuration checks**

```bash
python - <<'PY'
from pathlib import Path
text = Path("docker-compose.yml").read_text()
assert "platform-agent" in text
assert "/workspace" in text
assert "matrix-bot" in text
PY
```

```bash
python - <<'PY'
from pathlib import Path
readme = Path("README.md").read_text()
assert "docker compose up" in readme
assert "/workspace" in readme
assert "platform-agent" in readme
PY
```

- [ ] **Step 2: Run checks to verify they fail**

Run: `python - <<'PY' ... PY`

Expected:
- FAIL because root compose only defines `matrix-bot`
- FAIL because README still documents standalone `uvicorn` launch and old WS route

- [ ] **Step 3: Write minimal implementation**

```yaml
# docker-compose.yml
services:
  platform-agent:
    build:
      context: ./external/platform-agent
      target: development
      additional_contexts:
        agent_api: ./external/platform-agent_api
    env_file:
      - ./external/platform-agent/.env
    volumes:
      - workspace:/workspace
      - ./external/platform-agent/src:/app/src
      - ./external/platform-agent_api:/agent_api
    ports:
      - "8000:8000"

  matrix-bot:
    build: .
    env_file: .env
    depends_on:
      - platform-agent
    volumes:
      - workspace:/workspace
    restart: unless-stopped

volumes:
  workspace:
```

```env
# .env.example
AGENT_WS_URL=ws://platform-agent:8000/v1/agent_ws/0/
AGENT_BASE_URL=http://platform-agent:8000
SURFACES_WORKSPACE_DIR=/workspace
MATRIX_PLATFORM_BACKEND=real
```

```md
# README.md
- make the root `docker compose up` path the primary local runtime
- describe shared `/workspace` as the file contract
- remove the statement that real backend is text-only and has no attachments
- replace the old standalone `uvicorn` instructions with compose-first instructions
```

- [ ] **Step 4: Run checks to verify they pass**

Run: `python - <<'PY' ... PY`

Expected: PASS

- [ ] **Step 5: Commit**

```bash
git add docker-compose.yml README.md .env.example
git commit -m "chore: make shared workspace runtime the default local setup"
```

## Self-Review

- Spec coverage:
  - shared `/workspace` runtime: Task 4
  - incoming Matrix file persistence: Task 3
  - attachment path propagation to agent API: Tasks 1-2
  - outbound `send_file` flow: Tasks 2-3
  - future-surface-friendly attachment contract: Task 1
- Placeholder scan:
  - no `TODO`, `TBD`, or “similar to”
  - each task has explicit test, run, implementation, verify, commit steps
- Type consistency:
  - `workspace_path` is introduced in both attachment models and consumed consistently in Tasks 1-3
  - path-based contract is always relative to `/workspace` until Matrix upload resolution step

## Execution Handoff

User already selected parallel subagent execution. Use subagent-driven development and split ownership like this:

- Worker A: `docker-compose.yml`, `README.md`, `.env.example`
- Worker B: `sdk/agent_api_wrapper.py`, `sdk/real.py`, `tests/platform/test_real.py`
- Main session / later worker: `core/protocol.py`, `sdk/interface.py`, `core/handlers/message.py`, `adapter/matrix/files.py`, `adapter/matrix/bot.py`, matrix/core tests