> ## Documentation Index
> Fetch the complete documentation index at: https://docs.opendot.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Platform architecture
> Understand the current OpenDot control plane, voice runtime, database, deployment, and device architecture.
OpenDot is a modular monolith plus a separate realtime voice runtime. The
platform console owns configuration and device binding; the runtime owns live
audio sessions, OpenAI Realtime client-secret exchange, and device Realtime
bridging; PostgreSQL stores
durable control-plane state; firmware connects over OTA/bootstrap and WebSocket
paths.
## Current system
| Layer | Current implementation | Responsibility |
| ------------- | ------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------- |
| Docs | Mintlify in `docs/` | Product, setup, contribution, and architecture guidance |
| Web console | React, TypeScript, Vite, TanStack Query | Agent creation, pipeline tuning, browser testing, device binding, settings |
| Platform API | Fastify, TypeScript, Drizzle | Auth, agents, pipelines, devices, deployments, runtime/session tokens |
| Database | PostgreSQL | OpenDot-owned product state and Supabase-compatible auth bridge |
| Voice runtime | Node, WebSocket, browser Realtime client-secret exchange, Deepgram, OpenAI-compatible calls | Sandwich sessions plus browser and device Speech-to-speech runtime paths |
| Firmware | ESP-IDF for [Waveshare ESP32-S3-AUDIO-Board](https://www.waveshare.com/esp32-s3-audio-board.htm) | Wi-Fi provisioning, activation, wake word, audio I/O, display, WebSocket client |
The control boundary is intentional: browser and device clients do not push raw
identity configuration directly into the runtime. They ask the platform API for
an authorized session or credential, and the runtime verifies that token with
the platform API before loading the identity config.
For contributor orientation, see [Contribution areas](/contribution-areas). For
sequencing across current, next, and later work, see the [Roadmap](/roadmap).
## Platform stack
Mintlify in `docs/` publishes product, setup, architecture, and contributor guidance.
React, TypeScript, Vite, TanStack Query, and lucide-react power the browser workflow.
Fastify handles auth, agents, pipelines, device inventory, deployments, and
runtime/session tokens.
PostgreSQL stores OpenDot-owned product state through Drizzle schema and SQL
migrations.
Node.js orchestrates `/voice` and `/ws` WebSocket sessions, the
`/realtime/client-secret` boundary for browser Speech-to-speech tests, and the
server-side Realtime bridge for Speech-to-speech Dot turns.
Deepgram backs the starter VAD, STT, and TTS paths. OpenAI-compatible Responses or
Chat Completions calls back the starter LLM stage. OpenAI Realtime backs the
Speech-to-speech browser and device paths.
ESP-IDF firmware targets the [Waveshare
ESP32-S3-AUDIO-Board](https://www.waveshare.com/esp32-s3-audio-board.htm) reference
device today.
Docker Compose runs local services, while the Render Blueprint maps the same
containers to hosted preview services.
## Local development topology
Docker Compose runs PostgreSQL, migrations, the platform API, the voice runtime,
and the web console. The pnpm workflow runs the same application surfaces as
separate local processes. The docs site is separate and previews from `docs/`
with `mint dev`.
Use [Runtime configuration](/runtime-configuration) when changing environment
variables or service URLs.
## Voice turn flow
The default Sandwich pipeline remains explicit even when a provider combines
lower-level details:
```text theme={null}
audio input -> VAD -> STT -> LLM -> TTS -> audio output
```
Current media paths are:
* **Sandwich Browser Test:** the platform mints a short-lived voice-session
token, then the runtime verifies it before accepting `/voice`.
* **Dot device:** activated devices use stored device credentials, then the
runtime verifies them before accepting `/ws`. Sandwich agents use the
Deepgram + LLM + TTS path; Speech-to-speech agents use a server-side OpenAI
Realtime WebSocket bridge while firmware keeps sending and receiving Opus.
* **Speech-to-speech Browser Test:** the platform mints a realtime
browser-session token at `POST /api/runtime/realtime-browser-sessions`, the
runtime exchanges it at `/realtime/client-secret` for an OpenAI Realtime
client secret, and the browser connects with native WebRTC.
`OPENAI_API_KEY` stays inside the runtime. `OPENAI_BASE_URL` only affects the
Sandwich LLM stage; OpenAI Realtime browser secrets and device bridges call
OpenAI's Realtime API from the runtime.
See [Configuration](/configuration) for stage settings and
[Browser Test](/browser-test) for the live test loop.
## Device activation and binding
Firmware reaches the runtime OTA endpoint first. The runtime proxies activation
requests to the platform API, the platform creates or verifies a device
activation request, and the user claims the spoken code in **Dot Device**. After
claiming, the platform stores an active device credential. Authenticated `/ws`
connections use that credential, and the runtime loads the currently bound agent
configuration from the platform API before choosing Sandwich or
Speech-to-speech.
See [Dot Device console](/dot-device) for the product flow and the firmware
README for flashing and LAN setup details. See [Dot device](/device-firmware-hardware)
for the firmware and hardware reference target.
## Data ownership
PostgreSQL stores OpenDot-owned product data: app users, local auth credentials,
preferences, SDK API keys, versioned agents and pipelines, devices, activation
requests, device credentials, runtime session tokens, device state, deployments,
and deployment device targets. Supabase Auth can provide identity, but OpenDot
keeps durable product state in its own tables.
When changing persisted platform data, update the Drizzle schema, SQL
migrations, API types, environment examples, and docs together.