Architecture
One application image plays three roles — API, worker and migrator — beside PostgreSQL, Valkey, object storage and a TURN server. Seven containers, about 170 MiB of RAM at rest.
Components #
| Component | Role | If it dies |
|---|---|---|
| API | HTTP + both WebSocket endpoints; the SFU media loops run in-process. | Clients reconnect; nothing is lost (state is in Postgres). |
| Worker | Durable jobs: attachment verification, thumbnails, push fan-out, webhook delivery, retention and erasure sweeps. | Jobs resume — they are rows, claimed with FOR UPDATE SKIP LOCKED. |
| PostgreSQL | Source of truth: rooms, members, messages, sequences, the outbox, jobs, keys, usage. | The deployment is down. This is the one stateful dependency that matters. |
| Valkey | Presence, rate limits, fan-out pub/sub, ephemeral captures. Regenerable only. | Degrades gracefully — it can be flushed without data loss. |
| Object storage | Attachment blobs and thumbnails (S3-compatible, RustFS by default). | Media unavailable; chat unaffected. |
| coturn | STUN/TURN for NAT traversal, with per-session time-limited credentials. | Calls fail for users on restrictive networks. |
How an event reaches every device #
This is the single most important path to understand, because every guarantee the platform makes follows from it.
- Write, then sequence — in one transaction
The message row and an outbox row are inserted together and the transaction commits.
The message gets a monotonically increasing per-room
event_seqat that moment. - Publish strictly after commit A single leased publisher claims outbox rows in order and pushes them to Valkey pub/sub. Because this happens after the commit, a cache or broker failure can never lose a message that a client was told was sent.
- Fan out to live sockets
Every chat socket subscribed to that room receives the server envelope verbatim, carrying the
event_idthe client uses as its resume point. - Fan out to everyone else The same published row drives durable jobs: a push to members with no live socket, and a signed webhook delivery to each subscribed endpoint. Retries and backoff are job-table mechanics.
Media takes a different path #
Two things deliberately bypass the flow above:
- Call media. RTP flows over UDP directly between each client and the SFU. Only signalling (SDP, ICE candidates) rides the WebSocket. The SFU forwards per subscriber, so a participant uploads one stream no matter how many people are in the room.
- Attachment bytes. Clients upload and download through presigned URLs straight to object storage; the bytes never pass through the API. Upload throughput is therefore independent of API load.
Multi-tenancy #
Every record and every query is tenant-scoped, and cross-tenant access is refused at the
authorisation layer rather than filtered afterwards. A customer API key is pinned to its tenant —
passing a different tenant_id in a body does not change what it can reach.
Keys additionally carry an environment. A test key operates in a
separate data namespace from a live key for the same tenant, so a sandbox integration
can never read or write production rooms, and every token it mints inherits that isolation.
Deployment shape #
One compose file, one image, forward-only migrations applied automatically on start. Releases are built and gated on a test machine, then the built image is shipped to production — production never compiles. A 13-check smoke suite runs against production on every deploy and the release is refused if it fails.