fayad
Team led by a Confluent Senior Software Engineer and Queen's University graduate with deep expertise in Rust, distributed systems, and low-level LSM-tree storage engines.
YouTube Video
Project Description
Duet is a voice-driven IDE for hands-off programming. You talk; the agent builds in real time, opening files, writing code, and refreshing a live preview while you watch.
What makes it different is emotional awareness. Duet reads how you’re doing from your voice and face, not just what you say. Frustrated? It stops explaining and offers to take over. Confused? It slows down. In flow? It stays quiet.
The architecture uses two agents:
Voice agent (Tavus) — the emotional layer. The avatar you talk to. It hears you, reads emotion, decides when to speak, and delegates work through a single request(instruction) call. It never touches code.
Coding agent (OpenRouter) — the silent builder. Runs its own tool loop in the background: writes files, reveals code, highlights lines, searches the project, refreshes the preview. No audio.
An emotion HUD at the top shows the detected state live (annoyed, confused, flow, stuck, satisfied), so judges and users can see the system actually hearing.
Built for the AI Tinkerers NYC × ElevenLabs × Tavus Emotionally Intelligent AI Hackathon. Stack: Tavus CVI, ElevenLabs TTS, Redis (emotion timeline + vector search), React/Vite/CodeMirror, Express.
Tagline: Hear how you’re doing. Build accordingly.