docs: add matrix shared workspace file flow design
This commit is contained in:
parent
e6a42d9297
commit
8b04fcaf77
1 changed files with 252 additions and 0 deletions
|
|
@ -0,0 +1,252 @@
|
||||||
|
# Matrix Shared Workspace File Flow Design
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Bring the Matrix surface and `platform-agent` to a single file-handling model that matches the current platform runtime contract as closely as possible.
|
||||||
|
|
||||||
|
The result should be:
|
||||||
|
|
||||||
|
- Matrix receives user files and makes them visible to the agent through a shared `/workspace`
|
||||||
|
- `platform-agent` receives attachment paths, not ad hoc summaries or inline payloads
|
||||||
|
- the agent can send files back to the user through the surface via `send_file`
|
||||||
|
- local development and the default deployment path use the same storage contract
|
||||||
|
|
||||||
|
## Core Decision
|
||||||
|
|
||||||
|
The selected architecture is:
|
||||||
|
|
||||||
|
`Matrix surface <-> shared /workspace <-> platform-agent`
|
||||||
|
|
||||||
|
This means:
|
||||||
|
|
||||||
|
- the Matrix bot is responsible for downloading incoming Matrix media
|
||||||
|
- downloaded files are written into the same filesystem mounted into `platform-agent`
|
||||||
|
- the surface passes relative workspace paths to the agent as `attachments`
|
||||||
|
- the agent returns files to the user by emitting `MsgEventSendFile(path=...)`
|
||||||
|
|
||||||
|
This is the current platform-native direction and does not require new platform endpoints.
|
||||||
|
|
||||||
|
## Why This Decision
|
||||||
|
|
||||||
|
The current upstream platform changes already define the file contract:
|
||||||
|
|
||||||
|
- `MsgUserMessage.attachments` is `list[str]`
|
||||||
|
- each attachment is a path relative to `/workspace`
|
||||||
|
- the agent validates those paths against its configured backend root
|
||||||
|
- the agent can emit `send_file(path)` back to the client
|
||||||
|
|
||||||
|
That is not an upload API and not a remote blob contract. It is an explicit shared-workspace contract.
|
||||||
|
|
||||||
|
Trying to preserve the current separate-process launch model would force the surface to fake production behavior with inline text extraction, out-of-band path rewriting, or a future upload API that does not exist yet. That would increase the gap between our runtime and the platform runtime instead of reducing it.
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
This design covers:
|
||||||
|
|
||||||
|
- shared workspace runtime for Matrix bot and `platform-agent`
|
||||||
|
- incoming Matrix file handling into shared storage
|
||||||
|
- attachment path propagation to `RealPlatformClient` and `AgentApi`
|
||||||
|
- outbound file delivery from agent to Matrix user
|
||||||
|
- local compose/dev workflow and README updates
|
||||||
|
|
||||||
|
This design does not cover:
|
||||||
|
|
||||||
|
- Telegram file flow
|
||||||
|
- encrypted Matrix media handling
|
||||||
|
- upload APIs on the platform side
|
||||||
|
- OCR, PDF parsing, or content extraction pipelines
|
||||||
|
- long-term object storage or file lifecycle policies beyond basic cleanup boundaries
|
||||||
|
|
||||||
|
## Runtime Contract
|
||||||
|
|
||||||
|
### Shared filesystem
|
||||||
|
|
||||||
|
Both containers must mount the same directory at `/workspace`.
|
||||||
|
|
||||||
|
Requirements:
|
||||||
|
|
||||||
|
- the Matrix bot can create files under `/workspace`
|
||||||
|
- `platform-agent` sees the same files at the same relative paths
|
||||||
|
- agent-originated files written under `/workspace` are readable by the Matrix bot
|
||||||
|
|
||||||
|
The contract is path-based, not URL-based.
|
||||||
|
|
||||||
|
### Attachment path format
|
||||||
|
|
||||||
|
The surface sends attachments to the agent as relative workspace paths, for example:
|
||||||
|
|
||||||
|
- `surfaces/matrix/<matrix_user_id>/<room_id>/inbox/20260420-153000-report.pdf`
|
||||||
|
- `surfaces/matrix/<matrix_user_id>/<room_id>/inbox/20260420-153200-photo.jpg`
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
|
||||||
|
- paths must be relative to `/workspace`
|
||||||
|
- paths must be normalized before sending to the agent
|
||||||
|
- surface-owned uploads must live under a dedicated namespace to avoid collisions with agent-created files
|
||||||
|
|
||||||
|
## Data Flow
|
||||||
|
|
||||||
|
### Incoming file from Matrix user
|
||||||
|
|
||||||
|
1. Matrix receives `m.file`, `m.image`, `m.audio`, or `m.video`.
|
||||||
|
2. The Matrix bot resolves the target room and platform chat context as usual.
|
||||||
|
3. The Matrix bot downloads the media from Matrix.
|
||||||
|
4. The file is stored under `/workspace/surfaces/matrix/.../inbox/...`.
|
||||||
|
5. The outgoing platform call includes:
|
||||||
|
- original user text
|
||||||
|
- `attachments=[relative_path_1, ...]`
|
||||||
|
6. `platform-agent` validates that those files exist and exposes them to the agent through the upstream attachment mechanism.
|
||||||
|
|
||||||
|
Important detail:
|
||||||
|
|
||||||
|
- the surface should not rewrite the user message into a synthetic file summary unless the message body is empty
|
||||||
|
- when body is empty, the surface may send a minimal synthetic text such as `User sent one or more attachments.`
|
||||||
|
|
||||||
|
### Outbound file from agent to Matrix user
|
||||||
|
|
||||||
|
1. The agent uses `send_file(path)`.
|
||||||
|
2. `platform-agent` emits `MsgEventSendFile(path=...)`.
|
||||||
|
3. The Matrix integration catches that event.
|
||||||
|
4. The Matrix bot resolves the file inside shared `/workspace`.
|
||||||
|
5. The Matrix bot uploads the file to Matrix and sends the appropriate media message to the room.
|
||||||
|
|
||||||
|
Surface behavior:
|
||||||
|
|
||||||
|
- if MIME type and extension are known, send the closest native Matrix media type
|
||||||
|
- otherwise send as `m.file`
|
||||||
|
- user-visible failures must be explicit if the referenced file does not exist or cannot be uploaded
|
||||||
|
|
||||||
|
## Filesystem Layout
|
||||||
|
|
||||||
|
The Matrix surface owns a dedicated subtree:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/workspace/
|
||||||
|
surfaces/
|
||||||
|
matrix/
|
||||||
|
<sanitized-user-id>/
|
||||||
|
<sanitized-room-id>/
|
||||||
|
inbox/
|
||||||
|
20260420-153000-report.pdf
|
||||||
|
```
|
||||||
|
|
||||||
|
Design constraints:
|
||||||
|
|
||||||
|
- sanitize user ids and room ids before using them as path components
|
||||||
|
- preserve the original filename in the final basename where possible
|
||||||
|
- prefix filenames with a timestamp or unique id to avoid collisions
|
||||||
|
|
||||||
|
This layout is intentionally surface-scoped. The agent may read these files, but the surface remains the owner of how inbound messenger files are organized.
|
||||||
|
|
||||||
|
## Components
|
||||||
|
|
||||||
|
### Matrix attachment storage helper
|
||||||
|
|
||||||
|
Add a focused helper module responsible for:
|
||||||
|
|
||||||
|
- building stable workspace-relative paths
|
||||||
|
- sanitizing path components
|
||||||
|
- downloading Matrix media into `/workspace`
|
||||||
|
- returning attachment metadata needed by the platform layer
|
||||||
|
|
||||||
|
This helper should not know about agent transport details beyond the final relative path output.
|
||||||
|
|
||||||
|
### Real platform client
|
||||||
|
|
||||||
|
`RealPlatformClient` must pass attachment relative paths through to `AgentApi.send_message(...)`.
|
||||||
|
|
||||||
|
It must also surface non-text agent events needed by the Matrix adapter, especially `MsgEventSendFile`.
|
||||||
|
|
||||||
|
### Agent API wrapper
|
||||||
|
|
||||||
|
`AgentApiWrapper` must be compatible with the modern upstream protocol:
|
||||||
|
|
||||||
|
- `/v1/agent_ws/{chat_id}/`
|
||||||
|
- `attachments` on outgoing user messages
|
||||||
|
- `MsgEventToolCallChunk`
|
||||||
|
- `MsgEventToolResult`
|
||||||
|
- `MsgEventCustomUpdate`
|
||||||
|
- `MsgEventSendFile`
|
||||||
|
- `MsgEventEnd`
|
||||||
|
|
||||||
|
### Matrix bot outbound renderer
|
||||||
|
|
||||||
|
The Matrix adapter must support sending files back to the room.
|
||||||
|
|
||||||
|
At minimum it needs:
|
||||||
|
|
||||||
|
- path resolution inside shared workspace
|
||||||
|
- Matrix upload of the local file
|
||||||
|
- send of an `m.file` or native media event with filename and MIME type
|
||||||
|
|
||||||
|
## Deployment Changes
|
||||||
|
|
||||||
|
### Compose
|
||||||
|
|
||||||
|
The repository root `docker-compose.yml` becomes the primary prod-like local runtime.
|
||||||
|
|
||||||
|
It should define at least:
|
||||||
|
|
||||||
|
- `matrix-bot`
|
||||||
|
- `platform-agent`
|
||||||
|
- one shared volume mounted as `/workspace` into both services
|
||||||
|
|
||||||
|
The default developer workflow should stop describing `platform-agent` as a separately started side process.
|
||||||
|
|
||||||
|
### Environment
|
||||||
|
|
||||||
|
The Matrix bot must connect to the in-compose `platform-agent` service by service name, not by assuming a separately launched localhost process.
|
||||||
|
|
||||||
|
The agent WebSocket configuration in docs and examples must match the modern upstream route.
|
||||||
|
|
||||||
|
## Error Handling
|
||||||
|
|
||||||
|
### Incoming files
|
||||||
|
|
||||||
|
If the Matrix bot cannot download or persist the file:
|
||||||
|
|
||||||
|
- do not send a broken attachment path to the agent
|
||||||
|
- return a user-visible error in the room
|
||||||
|
- log the Matrix event id, room id, and failure reason
|
||||||
|
|
||||||
|
### Outbound files
|
||||||
|
|
||||||
|
If the agent asks to send a missing file:
|
||||||
|
|
||||||
|
- log a structured warning with the requested path
|
||||||
|
- send a user-visible message that the file could not be delivered
|
||||||
|
|
||||||
|
### Shared workspace mismatch
|
||||||
|
|
||||||
|
If the runtime is misconfigured and `/workspace` is not actually shared:
|
||||||
|
|
||||||
|
- inbound attachments will fail agent-side path validation
|
||||||
|
- outbound `send_file` will fail surface-side file resolution
|
||||||
|
|
||||||
|
The implementation should make such failures obvious in logs rather than silently degrading to text-only behavior.
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
The implementation must cover:
|
||||||
|
|
||||||
|
- Matrix media download writes into the expected workspace-relative path
|
||||||
|
- `RealPlatformClient` forwards attachment relative paths to the agent API
|
||||||
|
- Matrix plain messages with attachments preserve the original text while adding attachment paths
|
||||||
|
- empty-body attachment-only messages produce the synthetic text fallback
|
||||||
|
- `AgentApiWrapper` accepts `MsgEventSendFile` without treating it as unknown
|
||||||
|
- Matrix outbound file handling converts `MsgEventSendFile` into a Matrix upload/send call
|
||||||
|
- compose configuration mounts the same workspace into both containers
|
||||||
|
|
||||||
|
## Non-Goals
|
||||||
|
|
||||||
|
- no inline text extraction MVP
|
||||||
|
- no temporary URL-passing contract to the agent
|
||||||
|
- no fake “prod” mode with separate local filesystems
|
||||||
|
- no platform API additions in this phase
|
||||||
|
|
||||||
|
## Success Criteria
|
||||||
|
|
||||||
|
- the default local runtime uses a shared `/workspace`
|
||||||
|
- a user can send a file in Matrix and the agent receives it through upstream `attachments`
|
||||||
|
- the agent can emit `send_file(path)` and the Matrix user receives the file in the same room
|
||||||
|
- our runtime behavior matches the current platform contract closely enough that moving from local compose to production does not require redesigning file flow
|
||||||
Loading…
Add table
Add a link
Reference in a new issue