Bellhop - Tools for luxury resort staff

Bellhop is a background audio assistant for front desk and customer support staff at luxury resorts. Services providers at this price level will never replace human support reps, so how do we make their lives easier? The idea of this prototype is to envision what a “suggestions” tool would look like which listened to existing phone calls with customers, or while they are standing at the front desk.

I used both the realtime audio API, and the reasoning API, to robustly listen to the conversation and make changes to existing bookings, dinner reservations, activities, and so on. Creating new entries is also supported. While the 4o-realtime model is fast and can respond with tool calls, it’s not always perfect. This is where the o1 reasoning model was used, to double-check the work (suggested changes) of the first pass, and make any updates / corrections where necessary.

The underlying idea of this prototype could be used across a wide variety of use cases where customer calls will continue to come in. Since the tool calling logic is entirely happening on the frontend, this could theoretically be built into a browser extension and specalized for an existing saas tool that businesses already use (think salesforce, pipedrive, etc).

The idea came out of one of my active clients in Tokyo, where we are currently building out a network of 5 star luxury resorts in the countryside. We are in charge of the entire property management system build out, and have been on the project for 2 years, so there’s a chance this could actually see real world use. 🤙

The first experiments with WebRTC + Realtime API used some sample code found here:
https://github.com/fuwei007/OpenAIRealTimeAPIWebRTC/