A startup named Panjaya is revolutionizing the video translation industry. After three years of secret development, the company has launched a groundbreaking product called BodyTalk, an AI tool that accurately translates the voices in videos into other languages. Unlike traditional dubbing, BodyTalk not only perfectly replicates the original vocal characteristics of the speakers but also automatically adjusts the facial expressions and body movements of the characters in the video to naturally match the new language's speech patterns.
Panjaya was founded by two deep learning experts from the Israeli government, Hilik Shani and Ariel Shalom. In 2021, the duo left the government sector to start their own business and later welcomed Guy Piekarz, a seasoned industry professional, as CEO. Piekarz previously founded Matcha, a streaming discovery and recommendation platform, which was acquired by Apple in 2013.
Image source: This image was generated by AI, provided by the image authorization service Midjourney
Currently, BodyTalk supports translation into 29 languages. Its workflow begins with audio translation, then generates a new voice that mimics the original speaker, and finally automatically adjusts the speaker's mouth movements and actions in the video to match the new language. Although the processing time is close to real-time, it currently still takes a few minutes to process a video.
Technically, Panjaya employs a hybrid strategy, utilizing both third-party large language models and proprietary core technologies. According to Piekarz, the company's lip-sync engine was entirely developed by its internal AI research team, as there were no existing solutions on the market that could meet its complex commercial needs for multi-angle and multi-speaker scenarios.
The company is currently focused on the B2B market and has partnered with institutions such as JFrog and TED. TED reports that lecture videos dubbed using Panjaya's tools have seen a 115% increase in views and a doubling of complete watch rates. Panjaya plans to further expand its applications in sports, education, marketing, and healthcare.
To prevent technology misuse, Panjaya strictly controls tool access and plans to develop watermarking features to identify synthesized video content. Although subtitles have become a standard for video content—over half of American viewers turn on subtitles according to a CBS survey—there is still a huge demand for dubbed content internationally. Research by CSA shows that, especially in the B2B sector, native language content can lead to higher user engagement.
This round of $9.5 million in funding was participated by multiple investment institutions and individuals, including Viola Ventures, R-Squared Ventures, and Shlomi Ben Haim, co-founder and CEO of JFrog. In the future, Panjaya plans to launch an API interface and further improve processing speed, aiming for real-time processing.
