Google is Enhancing Robot Navigation and Task Completion Capabilities with Gemini AI Training.

AIbase

Published inAI News · 3 min read · Jul 12, 2024

100

Google is training its robots through Gemini AI to enhance their navigation and task completion abilities.

In a new research paper, the DeepMind Robotics Team explains in detail how to utilize the long context window of Gemini1.5Pro to make it easier for users to interact with the RT-2 robot using natural language commands. By filming video tours of specified areas, researchers used Gemini1.5Pro to have the robot "watch" the videos to understand the environment, allowing the robot to execute commands based on what it observes, such as guiding users to the power outlet for charging.

DeepMind states that robots equipped with Gemini successfully executed over 50 user commands within a 9000 square feet operational area, with a success rate of 90%.

Additionally, researchers found that Gemini1.5Pro enables robots to plan how to complete commands, not limited to navigation. For example, when a user asks the robot if they have their favorite drink on a table full of cola cans, Gemini informs the robot to check the fridge and then report the result to the user. DeepMind says they will further investigate these findings.

According to the research paper, although the video demonstration provided by Google is impressive, as shown in the paper, it takes the robot 10-30 seconds to process these commands. Although we may need some time before we can share our homes with more advanced environment mapping robots, at least these robots might be able to help us find lost keys or wallets.

Highlight:

🤖 Gemini AI trains robots to improve navigation and task completion capabilities

🧠 Gemini1.5Pro enables robots to execute natural language commands

🔍 Research shows Gemini allows robots to plan commands beyond navigation

Japanese Supermarkets Use AI to Monitor Employees' Smiles, Netizens Say It's Unnecessary

In today's pursuit of efficient and humanized management, Aeon, a large Japanese supermarket chain, has sparked widespread controversy by using AI technology to monitor employee smiles.On July 16, Yahoo Japan reported that Aeon has rolled out an AI system called "Smile-Kun" in 240 stores since July 1. This system analyzes employees' smiles and volume, providing real-time feedback to enhance customer service skills.The Smile-Kun system requires employees to greet it daily, then scores them based

Musk's xAI to Launch Grok 2 Next Month, Grok 3 Expected in December

Elon Musk recently announced in an interview with psychologist and author Jordan Peterson that xAI will release its latest artificial intelligence model, Grok2, next month.Musk stated that Grok2's performance is expected to be on par with GPT-4, having been trained on approximately 15,000 H100 GPUs. Meanwhile, the release of Grok3 is scheduled for December, which Musk claims will be the most powerful AI model.During the interview, Musk further revealed that Grok3 is currently being trained at th

Xiaohongshu Launches First 'AI Convenience Store' to Support Quality AI Notes with Traffic Boosts of 30,000 to 500,000

Recently, Xiaohongshu officially announced the launch of an intriguing tech account named "TechPotato," introducing its entrepreneurial project: the world's first AI-powered convenience store.According to official information, the "AI Convenience Store" is a collaboration between TechPotato and top authors across the web, along with renowned tech companies, to curate the most authentic and visually appealing AI-related content, which is then offered as "limited edition products" for continuous o

Microsoft Research Introduces AI Framework E5-V: Simplifying Multimodal Learning with Text Pair Unimodal Training to Reduce Costs

Recently, a research team from Microsoft Research and Beihang University has jointly introduced a novel framework called E5-V, aimed at providing a more efficient solution for multi-modal embeddings. With the continuous advancement of artificial intelligence, multi-modal large language models (MLLMs) have become a focal point of research, as they are capable of understanding both textual and visual information simultaneously, thereby better handling complex data relationships. However, effective