Vibe✓ : Agentic Street-Level Stress Detection with Gemini and Vertex AI

This talk walks through the development of a real-time, cloud-native system for analyzing stress conditions across New York City using live CCTV footage. The system built using Angular and Google Cloud’s AI stack: Vertex AI, Gemini, BigQuery ML, together with Firebase app hosting, captures 940 concurrent camera streams and performs context-aware analysis through a Gemini-powered multimodal agent. Each camera is mapped to dynamically generated borough zones based on Voronoi tessellation, enabling location-specific metrics.

The project spans five phases:

1. Multimodal vision analysis using Gemini for stress detection, with adaptive confidence thresholds.
2. A hybrid ML pipeline using Vertex AI and rule-based logic to classify high-risk conditions with 85%+ accuracy.
3. Time-series forecasting with BigQuery ML (ARIMA_PLUS) to detect seasonal patterns and enable pedestrian-aware predictive routing.
4. Event-driven, serverless infrastructure using Firebase Functions to support six-tier adaptive monitoring windows.
5. Live deployment with Firebase Hosting and GitHub Pages to ensure demo reliability and real-time updates.

The demo includes a live analytics dashboard showing violation forecasts and real-time confidence scores for each camera zone. Each camera stream is sampled adaptively based on stress levels, and the backend is built for continuous training and edge-case reinforcement.