docs: add matrix shared workspace file flow design
This commit is contained in:
parent
e6a42d9297
commit
8b04fcaf77
1 changed files with 252 additions and 0 deletions
|
|
@ -0,0 +1,252 @@
|
|||
# Matrix Shared Workspace File Flow Design
|
||||
|
||||
## Goal
|
||||
|
||||
Bring the Matrix surface and `platform-agent` to a single file-handling model that matches the current platform runtime contract as closely as possible.
|
||||
|
||||
The result should be:
|
||||
|
||||
- Matrix receives user files and makes them visible to the agent through a shared `/workspace`
|
||||
- `platform-agent` receives attachment paths, not ad hoc summaries or inline payloads
|
||||
- the agent can send files back to the user through the surface via `send_file`
|
||||
- local development and the default deployment path use the same storage contract
|
||||
|
||||
## Core Decision
|
||||
|
||||
The selected architecture is:
|
||||
|
||||
`Matrix surface <-> shared /workspace <-> platform-agent`
|
||||
|
||||
This means:
|
||||
|
||||
- the Matrix bot is responsible for downloading incoming Matrix media
|
||||
- downloaded files are written into the same filesystem mounted into `platform-agent`
|
||||
- the surface passes relative workspace paths to the agent as `attachments`
|
||||
- the agent returns files to the user by emitting `MsgEventSendFile(path=...)`
|
||||
|
||||
This is the current platform-native direction and does not require new platform endpoints.
|
||||
|
||||
## Why This Decision
|
||||
|
||||
The current upstream platform changes already define the file contract:
|
||||
|
||||
- `MsgUserMessage.attachments` is `list[str]`
|
||||
- each attachment is a path relative to `/workspace`
|
||||
- the agent validates those paths against its configured backend root
|
||||
- the agent can emit `send_file(path)` back to the client
|
||||
|
||||
That is not an upload API and not a remote blob contract. It is an explicit shared-workspace contract.
|
||||
|
||||
Trying to preserve the current separate-process launch model would force the surface to fake production behavior with inline text extraction, out-of-band path rewriting, or a future upload API that does not exist yet. That would increase the gap between our runtime and the platform runtime instead of reducing it.
|
||||
|
||||
## Scope
|
||||
|
||||
This design covers:
|
||||
|
||||
- shared workspace runtime for Matrix bot and `platform-agent`
|
||||
- incoming Matrix file handling into shared storage
|
||||
- attachment path propagation to `RealPlatformClient` and `AgentApi`
|
||||
- outbound file delivery from agent to Matrix user
|
||||
- local compose/dev workflow and README updates
|
||||
|
||||
This design does not cover:
|
||||
|
||||
- Telegram file flow
|
||||
- encrypted Matrix media handling
|
||||
- upload APIs on the platform side
|
||||
- OCR, PDF parsing, or content extraction pipelines
|
||||
- long-term object storage or file lifecycle policies beyond basic cleanup boundaries
|
||||
|
||||
## Runtime Contract
|
||||
|
||||
### Shared filesystem
|
||||
|
||||
Both containers must mount the same directory at `/workspace`.
|
||||
|
||||
Requirements:
|
||||
|
||||
- the Matrix bot can create files under `/workspace`
|
||||
- `platform-agent` sees the same files at the same relative paths
|
||||
- agent-originated files written under `/workspace` are readable by the Matrix bot
|
||||
|
||||
The contract is path-based, not URL-based.
|
||||
|
||||
### Attachment path format
|
||||
|
||||
The surface sends attachments to the agent as relative workspace paths, for example:
|
||||
|
||||
- `surfaces/matrix/<matrix_user_id>/<room_id>/inbox/20260420-153000-report.pdf`
|
||||
- `surfaces/matrix/<matrix_user_id>/<room_id>/inbox/20260420-153200-photo.jpg`
|
||||
|
||||
Rules:
|
||||
|
||||
- paths must be relative to `/workspace`
|
||||
- paths must be normalized before sending to the agent
|
||||
- surface-owned uploads must live under a dedicated namespace to avoid collisions with agent-created files
|
||||
|
||||
## Data Flow
|
||||
|
||||
### Incoming file from Matrix user
|
||||
|
||||
1. Matrix receives `m.file`, `m.image`, `m.audio`, or `m.video`.
|
||||
2. The Matrix bot resolves the target room and platform chat context as usual.
|
||||
3. The Matrix bot downloads the media from Matrix.
|
||||
4. The file is stored under `/workspace/surfaces/matrix/.../inbox/...`.
|
||||
5. The outgoing platform call includes:
|
||||
- original user text
|
||||
- `attachments=[relative_path_1, ...]`
|
||||
6. `platform-agent` validates that those files exist and exposes them to the agent through the upstream attachment mechanism.
|
||||
|
||||
Important detail:
|
||||
|
||||
- the surface should not rewrite the user message into a synthetic file summary unless the message body is empty
|
||||
- when body is empty, the surface may send a minimal synthetic text such as `User sent one or more attachments.`
|
||||
|
||||
### Outbound file from agent to Matrix user
|
||||
|
||||
1. The agent uses `send_file(path)`.
|
||||
2. `platform-agent` emits `MsgEventSendFile(path=...)`.
|
||||
3. The Matrix integration catches that event.
|
||||
4. The Matrix bot resolves the file inside shared `/workspace`.
|
||||
5. The Matrix bot uploads the file to Matrix and sends the appropriate media message to the room.
|
||||
|
||||
Surface behavior:
|
||||
|
||||
- if MIME type and extension are known, send the closest native Matrix media type
|
||||
- otherwise send as `m.file`
|
||||
- user-visible failures must be explicit if the referenced file does not exist or cannot be uploaded
|
||||
|
||||
## Filesystem Layout
|
||||
|
||||
The Matrix surface owns a dedicated subtree:
|
||||
|
||||
```text
|
||||
/workspace/
|
||||
surfaces/
|
||||
matrix/
|
||||
<sanitized-user-id>/
|
||||
<sanitized-room-id>/
|
||||
inbox/
|
||||
20260420-153000-report.pdf
|
||||
```
|
||||
|
||||
Design constraints:
|
||||
|
||||
- sanitize user ids and room ids before using them as path components
|
||||
- preserve the original filename in the final basename where possible
|
||||
- prefix filenames with a timestamp or unique id to avoid collisions
|
||||
|
||||
This layout is intentionally surface-scoped. The agent may read these files, but the surface remains the owner of how inbound messenger files are organized.
|
||||
|
||||
## Components
|
||||
|
||||
### Matrix attachment storage helper
|
||||
|
||||
Add a focused helper module responsible for:
|
||||
|
||||
- building stable workspace-relative paths
|
||||
- sanitizing path components
|
||||
- downloading Matrix media into `/workspace`
|
||||
- returning attachment metadata needed by the platform layer
|
||||
|
||||
This helper should not know about agent transport details beyond the final relative path output.
|
||||
|
||||
### Real platform client
|
||||
|
||||
`RealPlatformClient` must pass attachment relative paths through to `AgentApi.send_message(...)`.
|
||||
|
||||
It must also surface non-text agent events needed by the Matrix adapter, especially `MsgEventSendFile`.
|
||||
|
||||
### Agent API wrapper
|
||||
|
||||
`AgentApiWrapper` must be compatible with the modern upstream protocol:
|
||||
|
||||
- `/v1/agent_ws/{chat_id}/`
|
||||
- `attachments` on outgoing user messages
|
||||
- `MsgEventToolCallChunk`
|
||||
- `MsgEventToolResult`
|
||||
- `MsgEventCustomUpdate`
|
||||
- `MsgEventSendFile`
|
||||
- `MsgEventEnd`
|
||||
|
||||
### Matrix bot outbound renderer
|
||||
|
||||
The Matrix adapter must support sending files back to the room.
|
||||
|
||||
At minimum it needs:
|
||||
|
||||
- path resolution inside shared workspace
|
||||
- Matrix upload of the local file
|
||||
- send of an `m.file` or native media event with filename and MIME type
|
||||
|
||||
## Deployment Changes
|
||||
|
||||
### Compose
|
||||
|
||||
The repository root `docker-compose.yml` becomes the primary prod-like local runtime.
|
||||
|
||||
It should define at least:
|
||||
|
||||
- `matrix-bot`
|
||||
- `platform-agent`
|
||||
- one shared volume mounted as `/workspace` into both services
|
||||
|
||||
The default developer workflow should stop describing `platform-agent` as a separately started side process.
|
||||
|
||||
### Environment
|
||||
|
||||
The Matrix bot must connect to the in-compose `platform-agent` service by service name, not by assuming a separately launched localhost process.
|
||||
|
||||
The agent WebSocket configuration in docs and examples must match the modern upstream route.
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Incoming files
|
||||
|
||||
If the Matrix bot cannot download or persist the file:
|
||||
|
||||
- do not send a broken attachment path to the agent
|
||||
- return a user-visible error in the room
|
||||
- log the Matrix event id, room id, and failure reason
|
||||
|
||||
### Outbound files
|
||||
|
||||
If the agent asks to send a missing file:
|
||||
|
||||
- log a structured warning with the requested path
|
||||
- send a user-visible message that the file could not be delivered
|
||||
|
||||
### Shared workspace mismatch
|
||||
|
||||
If the runtime is misconfigured and `/workspace` is not actually shared:
|
||||
|
||||
- inbound attachments will fail agent-side path validation
|
||||
- outbound `send_file` will fail surface-side file resolution
|
||||
|
||||
The implementation should make such failures obvious in logs rather than silently degrading to text-only behavior.
|
||||
|
||||
## Testing
|
||||
|
||||
The implementation must cover:
|
||||
|
||||
- Matrix media download writes into the expected workspace-relative path
|
||||
- `RealPlatformClient` forwards attachment relative paths to the agent API
|
||||
- Matrix plain messages with attachments preserve the original text while adding attachment paths
|
||||
- empty-body attachment-only messages produce the synthetic text fallback
|
||||
- `AgentApiWrapper` accepts `MsgEventSendFile` without treating it as unknown
|
||||
- Matrix outbound file handling converts `MsgEventSendFile` into a Matrix upload/send call
|
||||
- compose configuration mounts the same workspace into both containers
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- no inline text extraction MVP
|
||||
- no temporary URL-passing contract to the agent
|
||||
- no fake “prod” mode with separate local filesystems
|
||||
- no platform API additions in this phase
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- the default local runtime uses a shared `/workspace`
|
||||
- a user can send a file in Matrix and the agent receives it through upstream `attachments`
|
||||
- the agent can emit `send_file(path)` and the Matrix user receives the file in the same room
|
||||
- our runtime behavior matches the current platform contract closely enough that moving from local compose to production does not require redesigning file flow
|
||||
Loading…
Add table
Add a link
Reference in a new issue