At the recently concluded MWC, Google officially launched new features for Gemini Live, marking the integration of Project Astra and Gemini Live. These new features allow users to share their phone screens and engage in real-time video interactions using their smartphone's camera, representing a significant leap in enhancing the user experience of AI assistants.
Google spokesperson Alex Joseph confirmed this news via email, stating that the new features are being rolled out to users. A new button, "Share screen with Live," allows users to easily share their phone screens. A Reddit user even showcased the feature in a video, demonstrating Gemini accurately answering questions like the current date and temperature based on the shared screen information.
Beyond screen sharing, Gemini Live introduces real-time video interpretation. Users can simply open their phone's camera, and Gemini analyzes the captured image and answers related questions. In a demonstration video released by Google, a user asked Gemini which paint was best for glazing pottery. With minimal delay, Gemini provided detailed recommendations, showcasing its powerful real-time processing capabilities.
Project Astra, Google's AI agent project launched last year, aims to enable real-time visual and voice interaction with large language models. Key features include low-latency audio and video conversations, multi-language support, the ability to remember key details from past conversations, and even the use of tools like Google Search and Maps within the conversation. Notably, Project Astra is cross-device compatible, allowing users to enjoy this functionality on Android phones or specialized glasses.
In stark contrast to Google's rapid progress, Apple's Siri upgrades have stalled. According to Bloomberg, Apple executives admitted in internal meetings that Siri's AI upgrades are far from satisfactory, and the promised Apple Intelligence remains elusive. This slow pace puts Apple behind in the AI race.
Google's new features demonstrate its strength in the AI assistant field and generate excitement for the future of AI interaction. While Apple faces challenges, increased competition will drive more innovation and improvements, ultimately benefiting users.
