Contribution tracks wrap the same build, tune, bind, run, and review loop.
The docs are the canonical contributor map. Root files such as
README.md,
CONTRIBUTING.md, and ROADMAP.md summarize this page and point back here.Track map
Voice Pipeline & Providers
Purpose: Keep VAD, STT/ASR, LLM, and TTS visible as separate stages while making hosted providers, self-hosted services, and local models easier to swap. Current state: The starter runtime uses Deepgram for VAD, STT, and TTS, and OpenAI-compatible calls for the LLM stage. The console exposes the stage model through Configuration, and the runtime keeps provider calls inside the live voice session path. Good first directions: Improve stage validation, document provider keys, clarify provider-neutral settings, add test fixtures for pipeline defaults, or make error states easier to diagnose in Browser Test.
Start in: Configuration, Browser Test,
platform/src/lib/pipeline.ts, platform/src/types.ts, and
platform/src/server/runtime.js.
Voice Agents & Harnesses
Purpose: Make agent behavior portable and testable through prompts, knowledge, tools, model settings, local model harnesses, evals, and agent-framework integration categories. Current state: Agent Studio and Configuration provide the first agent identity workflow. Knowledge, tools, local AI/ML harnesses, and formal evals are future surfaces rather than mature subsystems. Good first directions: Improve agent copy and empty states, document prompt and model expectations, propose eval fixtures, or sketch integration boundaries that do not bind OpenDot to one framework.
Start in: Agent Studio, Configuration,
Platform architecture, and
platform/src/pages.
Platform Control Plane
Purpose: Make the web console the place where contributors and operators can create agents, tune pipelines, test sessions, bind devices, and see what is happening. Current state: The React/Vite console covers login, Agent Studio, Configuration, Browser Test, Dot Device, and Settings. The current opportunity is to make these workflows more durable, inspectable, and helpful. Good first directions: Improve loading and error states, add clearer pipeline summaries, polish device binding feedback, or make Browser Test timing and replay more useful.
Start in: Agent Studio, Browser Test,
Dot Device, and
platform/src/pages.
Platform Backend & Data
Purpose: Keep the control API, auth, database schema, runtime credentials, settings, deployments, and API contracts boring, explicit, and safe to evolve. Current state: Fastify, Drizzle, and PostgreSQL own durable product state. The runtime asks the platform API to verify browser voice-session tokens and device credentials before accepting live connections. Good first directions: Add request validation, improve auth/device errors, document internal trust boundaries, or keep migrations and env examples aligned when data contracts change.
Start in: Authentication, Database inspection,
platform/src/server/api.ts, platform/src/server/db/schema.ts, and
platform/drizzle.
Media Transport
Purpose: Carry realtime audio, text, and session events between browsers, devices, and the runtime while preserving the voice pipeline product model. Current state: Sandwich Browser Test uses/voice WebSocket connections.
Dot devices use /ws WebSocket connections. Speech-to-speech Browser Test uses
native browser WebRTC through OpenAI Realtime after the platform and runtime
complete the realtime browser-session token and client-secret exchange.
Speech-to-speech Dot sessions keep /ws, while the runtime opens a server-side
OpenAI Realtime WebSocket bridge. SFU-style media infrastructure and fleet media
paths are future transport adapters, not rewrites of the control plane.
Good first directions: Document current message flows, clarify the
realtime client-secret exchange, improve connection diagnostics, add structured
runtime events, or benchmark WebSocket audio limits.
Start in: Platform architecture, Runtime configuration,
platform/src/server/runtime.js, and platform/platform-architecture.md.
Device Communication & Fleet
Purpose: Move beyond live session sockets toward reliable device presence, desired/reported state, commands, telemetry, diagnostics, OTA metadata, and multi-device coordination. Current state: Device communication is currently tied to activation, OTA bootstrap, runtime WebSocket sessions, platform device state, and console binding. MQTT-style fleet communication is a later track once the device lifecycle needs it. Good first directions: Make current device state easier to inspect, document activation failure modes, or specify the future state/command contract before introducing a broker.
Start in: Dot Device, Dot device,
Platform architecture, and
platform/src/server/api.ts.
Dot Hardware
Purpose: Turn real-device constraints into open hardware work: CAD, enclosure, acoustics, microphone and speaker placement, PCB, BOM, fixtures, and manufacturability. Current state: OpenDot currently targets the Waveshare ESP32-S3-AUDIO-Board as the tested reference board. Purpose-built Dot hardware is a future open hardware direction. Good first directions: Capture reference-board constraints, document acoustic tradeoffs, create hardware requirement notes, or prepare contribution criteria for future CAD and PCB files.
Start in: Dot device and
dot-device/firmware/main/boards/waveshare/esp32-s3-audio-board.
Dot Firmware & Edge
Purpose: Improve the ESP-IDF firmware that runs on current hardware while keeping a later path open for MicroPython runtime exploration and on-device inference research. Current state: Firmware is an ESP-IDF project for the Waveshare ESP32-S3 audio board. It handles Wi-Fi provisioning, OTA bootstrap, activation, wake-word flow, audio I/O, display, LEDs, and WebSocket runtime connection. MicroPython and on-device inference are exploratory later tracks, not current architecture. Good first directions: Improve serial diagnostics, harden provisioning, document board configuration, fix driver assumptions, or make runtime endpoint setup less error-prone.
Start in: Dot device,
dot-device/firmware/README.md, and dot-device/firmware/main.
Docs, Tooling & Developer Experience
Purpose: Make it easy for contributors to understand the product loop, run the stack, choose a track, and open focused pull requests. Current state: Docs live in Mintlify, root docs introduce the project, CI runs formatting/lint/test/build for the platform, and GitHub templates route bugs and feature requests. Good first directions: Improve quickstart, troubleshooting, diagrams, issue templates, examples, local scripts, or verification guidance.
Start in: Contributing, Repo map,
docs/,
.github/, and root docs.
Picking a first pull request
- Choose one track and one user-visible outcome.
- Prefer a change that can be reviewed with one primary command or one manual workflow.
- Link the affected docs page when your change alters setup, configuration, runtime behavior, device behavior, or architecture.
- Keep provider, protocol, and hardware proposals category-level unless the implementation already exists in the repository.