Soothify

Soothify is a multimodal mental wellness companion built to help users move from self-reflection into immediate, emotionally aware support. It combines assessments, support resources, Redis-backed companion memory, and real-time audio/video interactions so the system can respond differently based on what the user is experiencing in the moment.

For emotional accuracy, Soothify detects distress through two layers: agent-level emotional interpretation inside the ElevenLabs voice companion, and app-level panic keyword detection on live transcript content. When a user says phrases such as “I’m freaking out” or “I can’t breathe,” the system classifies that as an acute panic signal rather than ordinary stress. That distinction matters because it changes the response path immediately.

For real-time adaptation, Soothify shifts behavior during a live session instead of waiting until afterward. In the normal audio companion, if panic is detected, the UI immediately surfaces a grounding support block with breathing cues, nearby-help links, a 988 action, and a one-click transfer into a dedicated panic-support voice agent. The active companion is also steered with contextual updates so its spoken responses become shorter, calmer, and more regulation-focused in that same session. On the video side, Tavus sessions provide a more expressive face-to-face support experience, and session context can be persisted into memory for continuity.

For multilingual capability, the architecture is designed to support language-aware model routing and multilingual agent behavior through configurable voice agents and model-backed memory infrastructure. The companion stack can be extended to support localized prompts, multilingual transcript handling, and language-specific emotional support flows without changing the core product design.

For creative expression, Soothify is not just a chatbot wrapper. It blends voice, video, memory, grounding UX, and emotional escalation design into a product-shaped experience. The companion can remember prior session themes, surface recurring stressors on the dashboard, and connect immediate emotional support with longer-term reflection. The result is a system that does not just “talk back,” but reads the room, adapts in real time, and guides users toward calmer and safer next steps.

Soothify processes human emotion by combining live transcript signals, agent behavior design, contextual memory, and mode switching. Instead of treating all distress the same way, it distinguishes between normal support and acute panic, then changes both the interface and the agent’s conversational behavior to match the user’s emotional state.

Technologies, Frameworks, Libraries, and APIs Used

Next.js for the application framework
React for frontend UI
Node.js for server/runtime logic
TypeScript for typed application code
Tailwind CSS for styling
MongoDB for legacy app data
Redis for companion memory, session state, and dashboard insights
Redis Agent Memory Server and agent-memory-client for structured memory workflows
ElevenLabs voice agent platform for real-time audio companion sessions
@elevenlabs/react for browser-based voice session integration
Tavus API for video companion sessions
OpenRouter for model-backed proxy and memory-related generation flows
OpenAI-compatible APIs for legacy chat/STT/TTS routes and embedding-compatible integrations
Jest and Testing Library for testing
Zod for request/env validation
Lottie for animated voice companion visuals

NA