At this year's Mobile World Congress (MWC), Google announced new real-time video analysis and screen sharing capabilities for its Gemini assistant. Google One AI Premium subscribers with Gemini Advanced will be the first to experience these features later this month.
This update gives Gemini Live two core abilities: real-time analysis of video content via an external camera, and screen sharing, allowing the AI assistant to directly interpret information on the user's phone and provide feedback. These features enable more interactive visual communication with the AI, such as object recognition, screen content interpretation, or real-time suggestions.
The new features will initially launch on Android devices and support multiple languages. Google demonstrated the integration of these features across various Android devices at MWC, further solidifying its competitiveness in the AI assistant space.
This update also marks a key step towards real-world interaction for AI assistants. Google's long-term goal is Project Astra – a general-purpose multimodal AI assistant capable of processing text, video, and audio in real-time with short-term memory. Astra is expected to integrate deeply with Google Search, Lens, and Maps in the future.
With the launch of Gemini Live, the competition between Google and OpenAI is intensifying. Since December last year, ChatGPT has supported real-time video and screen sharing with its advanced voice mode, and Google's update is undoubtedly a direct response. Whether Gemini can leverage this new functionality to further solidify its AI leadership remains to be seen.
