Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.opendot.ai/llms.txt

Use this file to discover all available pages before exploring further.

OpenDot is a modular monolith plus a separate realtime voice runtime. The platform console owns configuration and device binding; the runtime owns live audio sessions; PostgreSQL stores durable control-plane state; firmware connects over OTA/bootstrap and WebSocket paths.
OpenDot system architecture diagram

​
Current system

LayerCurrent implementationResponsibility
DocsMintlify in docs/Product, setup, contribution, and architecture guidance
Web consoleReact, TypeScript, Vite, TanStack QueryAgent creation, pipeline tuning, browser testing, device binding, settings
Platform APIFastify, TypeScript, DrizzleAuth, agents, pipelines, devices, deployments, runtime/session tokens
DatabasePostgreSQLOpenDot-owned product state and Supabase-compatible auth bridge
Voice runtimeNode, WebSocket, Deepgram, OpenAI-compatible LLM callsBrowser and device voice sessions through VAD, STT, LLM, and TTS
FirmwareESP-IDF for Waveshare ESP32-S3-AUDIO-BoardWi-Fi provisioning, activation, wake word, audio I/O, display, WebSocket client
The control boundary is intentional: browser and device clients do not push raw agent configuration directly into the runtime. They ask the platform API for an authorized session or credential, and the runtime verifies that token with the platform API before loading the agent config.

​
Platform stack

Docs

Mintlify in docs/ publishes product, setup, architecture, and contributor guidance.

Console

React, TypeScript, Vite, TanStack Query, and lucide-react power the browser workflow.

Platform API

Fastify handles auth, agents, pipelines, device inventory, deployments, and runtime/session tokens.

Database

PostgreSQL stores OpenDot-owned product state through Drizzle schema and SQL migrations.

Voice runtime

Node.js and WebSocket sessions orchestrate VAD, STT, LLM, TTS, browser audio, and device audio.

Providers

Deepgram backs the starter VAD, STT, and TTS paths. OpenAI-compatible Responses calls back the starter LLM stage.

Device firmware

ESP-IDF firmware targets the Waveshare ESP32-S3-AUDIO-Board reference device today.

Packaging

Docker Compose runs local services, while the Render Blueprint maps the same containers to hosted preview services.

​
Local development topology

OpenDot local development topology diagram
Docker Compose runs PostgreSQL, migrations, the platform API, the voice runtime, and the web console. The pnpm workflow runs the same application surfaces as separate local processes. The docs site is separate and previews from docs/ with mint dev. Use Runtime configuration when changing environment variables or service URLs.

​
Voice turn flow

OpenDot voice turn flow diagram
The pipeline remains explicit even when a provider combines lower-level details: 
audio input -> VAD -> STT -> LLM -> TTS -> audio output
Browser sessions use a short-lived voice-session token minted by the platform API. Device sessions use device credentials created during activation. In both cases, the runtime verifies the token with the platform API before accepting the WebSocket connection. See Configuration for stage settings and Browser Test for the live test loop.

​
Device activation and binding

OpenDot device activation and binding diagram
Firmware reaches the runtime OTA endpoint first. The runtime proxies activation requests to the platform API, the platform creates or verifies a device activation request, and the user claims the spoken code in Dot Device. After claiming, the platform stores an active device credential. Future /ws connections use that credential, and the runtime loads the currently bound agent configuration from the platform API. See Dot Device console for the product flow and the firmware README for flashing and LAN setup details. See Dot device for the firmware and hardware reference target.

​
Data ownership

OpenDot data model map diagram
PostgreSQL stores OpenDot-owned product data: app users, local auth credentials, preferences, SDK API keys, versioned agents and pipelines, devices, activation requests, device credentials, runtime session tokens, device state, deployments, and deployment device targets. Supabase Auth can provide identity, but OpenDot keeps durable product state in its own tables. When changing persisted platform data, update the Drizzle schema, SQL migrations, API types, environment examples, and docs together.