The Informant

The Informant is an interactive noir interrogation experience where the player speaks to an AI narrator/suspect who can hear emotional tone, adapt the story in real time, and perform the resulting emotion through a lifelike video avatar. The core creative idea is a closed emotional loop: the AI listens to how the player sounds, interprets the emotional signal, changes its loyalty and narrative choices, then responds with matching facial expression, voice delivery, and story consequences.

For emotional accuracy, we use Tavus Raven-1 perception to detect player tone during conversation, then map those signals into story-state variables such as trust, suspicion, hostility, cooperation, and betrayal. The system does not rely only on raw transcript text; it considers how the user sounds, allowing the narrator to distinguish between calm cooperation, pressure, anger, hesitation, or emotional betrayal.

For real-time adaptation, the experience uses a scripted narrative backbone with a reactive emotional overlay. This means the demo always progresses reliably, but each player’s emotional delivery can alter branches, narrator mood, and the final case-file outcome. If the player sounds hostile, the narrator becomes guarded. If they sound empathetic, he reveals more. If they betray him, the final beat shifts into a visibly hurt and dejected performance.

For multilingual capability, the architecture supports multilingual speech interaction through Tavus CVI’s conversational interface and ElevenLabs voice generation, allowing the emotional storytelling loop to extend beyond English-language text prompts. The design is built so emotion detection and response logic can operate on tone and conversational context, not just fixed English keywords.

For creative expression, the project combines noir writing, live video performance, emotional state machines, and generative conversation into a dramatic interactive scene. The AI is not just answering questions; it is acting. It changes its face, voice, pacing, loyalty, and narrative role based on the player’s emotional choices, culminating in a betrayal moment designed to feel personal and cinematic.

Technologies Used

Next.js App Router
TypeScript
React
Tavus Conversational Video Interface API
Tavus Raven-1 for emotional perception
Tavus Phoenix-4 for expressive video replica performance
Tavus Echo-mode emotion tags for emotional delivery control
@tavus/cvi-ui for the embedded call interface
WebRTC / Daily-powered Tavus CVI session
ElevenLabs API for expressive text-to-speech
Server-side Next.js Route Handlers for secure conversation creation
Story state machine for branch selection and emotional progression
Hand-authored noir story content with emotion-tagged narrative beats
Debug emotion overlay showing live player tone and state changes
Case-file transcript and emotion scorecard generated as the final artifact

N/A