6 Best Practices to Implement Real-Time Voice Translation

Today's globalized world has blurred not only the physical borders but also the cultural and communication barriers. In customer service, AI and Real-Time Translation (RTT) technology enables not only multilingual self-service but also empowers contact center agents with linguistic superpowers to serve customers in the language of their choice.

While RTT on messaging channels is straightforward, its application on the phone line, especially for agent assistance, is plagued by the inevitable latency issues. In this blog post, we explore this inherent challenge and the relevant mitigation strategies when harnessing AI-powered customer service.

Why Is Voice RTT for Agent Assistance so Hard?

Compared to chat, the challenge of real-time translation for voice use cases is two-fold. First, our tolerance for latency is much lower in a spoken conversation. Second, more processing steps are required system-wise, each contributing to response delays.

A typical voice-to-voice RTT system for live translation between a customer and a service agent incorporates the following steps:

• Customer Speech Recognition: Converting the customer’s spoken customers language into text.
• Translation: Machine translation systems process and generate translations.
• Replay of Translated Text: The AI relays translated text to the agent in audio form.
• Information Retrieval: The agent retrieves relevant responses from backend systems.
• Agent Spoken Utterances: The agent communicates their response in audio.
• Agent Speech Recognition: The agent’s spoken utterance is converted into text.
• Translation: This text is translated back into the customer’s language.
• Text-to-Speech: The translated text is converted into spoken language for the customer

Each of the above steps introduces latency, and their cumulative effect can result in noticeable delays. As such, while voice RTT is technically feasible, its practical application in customer service that demands real-time or near-real-time communication is substantially hindered.

What Can You Do to Mitigate Latency?

Today’s technology cannot eliminate inherent latency. However, there are multiple strategies to optimize system design and reduce its perceptible impact.

1. Set Customer Expectations Upfront

Even the most well-thought-out RTT setup isn’t immune to errors. Informing customers upfront that they are engaging with an AI-powered RTT system sets realistic expectations to avoid frustration. Rather than wasting their time on hold, customers might most likely appreciate an alternative that, despite technical imperfections, accelerates issue resolution.

2. Implement Voice-to-Chat RTT

In this approach, customers speak to a Voice AI Agent in their native language, while the service agent engages via chat in their preferred language, with AI managing real-time translation over the phone. Compared to a voice-to-voice approach, this bypasses multiple latency-inducing steps, i.e., replaying translated customer input, agents responding in speech, and speech-to-text of the agent output.

Combined with advanced Agent Assist, Voice-to-Chat RTT streamlines system execution significantly, as immediate text processing is much more efficient than repeated voice processing.

3. Harness Advanced Agent Assist

In a voice interaction where every second counts, advanced agent assist tools that equip agents with immediate access to necessary information can make a world of difference. Instead of agents having to navigate numerous backend applications, AI-powered real-time knowledge lookups and proactive suggestions can cut response times from tens of seconds or even minutes to an instant.

4. Use Atmosphere Sounds and Silence Overlay

Adding background noises that simulate a contact center ambiance throughout and/or during extended delays is another effective way to make perceived latency less jarring. Contextual, unobtrusive soundscapes, like distant chatters and keyboard typing, signal ongoing interactions and help to distract users from lagging responses. Plus, well-designed ambient sounds can make the overall voice experience more engaging and realistic.

5. Leverage Filler Responses

Phrases like “I see”, “Just a moment”, “Let me check that for you” or “I’m looking up right now” acknowledge customer input and mask the processing delays. These fillers can be incorporated as canned or pre-rendered responses to provide customers with immediate feedback, ensuring responsiveness while creating a natural buffer time.

6. Experiment with Multiple Speech and Translation Solutions

Robust call analytics tools that visualize the audio soundwaves allow developers to experiment with different speech and translation solutions by effectively measuring their processing time. As such, you can identify the most efficient options and configurations to minimize latency.

Interested in learning how Cognigy enables real-time voice translation for contact centers? Watch the video below and see it for yourself!

Conclusion

With labor shortage and ever-growing agent attrition rates, real-time translation is a viable solution to bridge the service gaps, especially for peripheral markets and languages spoken by a smaller population. Voice RTT for contact centers is technically feasible, but it doesn’t come without challenges. These strategies can help to address its evergreen latency issue and contribute to more frictionless, efficient, and satisfactory multilingual customer interactions.