Twilio Announces Enhanced Partnership with OpenAI to Improve Speech-to-Speech

In the past, customers have found that speaking with AI-driven chatbots has felt very much less like “chat” and more like “bot.” Responses, even if they were correct, rarely sounded particularly natural.

Now. technology has grown to ideally fix this problem; namely, through speech-to-speech. This is an emerging solution that allows for voice conversations by AI virtual agents to feel much more like real human dialogue.

In this vein, many contact center companies are turning to OpenAI’s Realtime API to reduce latency and improve key components like conversation pacing, interruption handling, tone and balance between speaking and listening – all critical user experience elements that make conversation with a virtual agent more human-like.

And so, customer engagement solutions provider Twilio recently announced an integration with OpenAI to bring the latter company’s new Realtime API to the Twilio platform. The integration of streaming speech-to-speech capabilities, which are part of the Realtime API, will enable 300,000+ Twilio customers and more than 10 million developers to build conversational AI virtual agents leveraging OpenAI’s flagship multilingual and multimodal GPT-4o model. The new integration builds on existing OpenAI and Twilio product integrations announced last year to bring the power of large language models (LLMs) to the customer engagement platform.

In the announcement, the companies noted that the combined technology "is especially relevant for customer service and sales, delivering both operational efficiency and exceptional customer outcomes." Speech-to-speech is also set to support social impact at scale, empowering nonprofit and public sector organizations to deploy novel use cases like voice translation in real time between constituents and staff members who speak different languages.

“Integrating OpenAI’s Realtime API with Twilio’s platform enables businesses to offer more natural, real-time AI voice interactions at scale,” said Inbal Shani, Chief Product Officer, Twilio Communications. “Businesses can use this to create voice experiences that feel more human and can reduce operational costs and drive higher customer satisfaction.”