Meta Launches AI-Powered Audio Translations for Video

Meta is launching an impressive AI-powered Audio translations for video that allows seamless cross-language communication as well as content access.

Meta expands accessibility to content through the introduction of AI-driven audio translations of content. This advanced technology is able to synchronize lip movements to translate speech. This technology enhances the user experience by creating the illusion that speakers are speaking in a different language.

As you can see in this example, Facebook users can now view some videos that have been translated into a different spoken language with Meta AI’s ever-evolving translation tool for translating. 

As Meta explains:

“Meta AI Translations uses the sound and tone of your own voice so it feels authentically you. And the lip syncing feature matches your mouth’s movements to the translated audio, so that it looks like you’re really speaking the language in the reel.”

This feature is a major increase in accessibility and engagement, allowing creators to reach out to a wider audience with multilingual support.

Initial Launch Focus

Meta has been developing this for some time and is now beginning a large-scale rollout, beginning with automated translators that can switch to English as well as Spanish. 

Creators can enable these translations using a new switch in the Reels composer that is labeled “Translate your voice with Meta AI.”

As you can see in this example that users can also choose to allow automatic lip syncing or select to only translate audio. 

“If you want to review the translation before it’s published, you can select the toggle to enable this option, you will then receive a notification (or you can go to Professional Dashboard) to review and approve the translation. Accepting or rejecting the translation will not impact your original, non-translated reel.”

The translated reels are shown to viewers in their preferred languages, accompanied by notifications indicating that content was translated by Meta AI. 

Viewers have control over the versions of the content they view, which makes the experience flexible and a pleasure for users. Starting with Spanish, Meta will rapidly expand support to include a wide range of global languages, unlocking audiences worldwide

Practical Advice for Creators

Although the system for translation supports as many as two languages, the creators should:

• The system supports translation for up to two speakers; therefore, it is advised not to speak over one another so as to improve the accuracy of translation.
• Limit background noise and music on reels intended to be used for translation
• Meta acknowledges that reaching the audience of the language of other languages may take the time and effort required, similar to the process of starting over with a completely new segment.

Even with some limitations, this technology can allow you to reach a greater international audience by allowing more people to be aware of and engaged in your work.

Expanding Opportunities

Together with AI Translations, Meta lets Facebook Page managers add up to 20 audio tracks that are custom-dubbed to one Reel. 

Meta says:

“Like Meta AI translations, viewers will hear your reel in their own language so you can continue to build deeper connections with diverse audiences without language barriers.”

While these tracks, manually added, are not synchronized with AI-generated lips and synchronization, they are a great method of reaching people across languages.

Eligibility and Availability

Facebook creators who have Professional Mode enabled and over 1,000 followers will be able to access this feature via the latest app update.

Instagram creators who have public accounts in areas in which Meta AI is available will also be able to use these AI translators.

Final Thoughts

Meta’s AI-powered, audio-powered translations with lip syncing are a thrilling development for content creators who want to increase their reach across the globe. Although occasional errors in translation are normal, the technology allows creators to connect authentically to a variety of audiences and broaden their reach across multiple languages.

Learn more about Meta’s leading-edge AI translation capabilities here.