Who's the true king of gaming? AI challenges the classic game Super Mario Bros.! The Hao AI Lab at the University of California, San Diego, has released surprising news: in a thrilling AI "Mario" showdown, Anthropic's Claude 3.7 model emerged victorious, taking the crown of "Strongest AI Mario"! Close behind was its sibling, Claude 3.5, while Google's Gemini 1.5 Pro and OpenAI's GPT-4o surprisingly underperformed, leading to a shocking outcome. What happened?
This AI "Mario" competition didn't take place on an old NES console, but in a sophisticated simulator. Researchers created a framework called GamingAgent as a bridge between AI and the gaming world. In this virtual world, the AI embodies Mario, wielding a virtual controller and receiving commands from the system: "Obstacle ahead! Jump!", "Enemy approaching! Dodge!" The instructions are simple yet challenging. The system also provides screenshots, allowing the "AI Mario" to gain a comprehensive view of the game. Even cooler, the AI can write Python code on the fly to control Mario's actions, performing impressive jumps and dodges.
However, the results were unexpected. Experienced AI models known for their reasoning abilities, such as OpenAI's, stumbled, performing worse than non-reasoning models! Why? It turns out that even "reasoning masters" have a fatal weakness – slow reaction times! In a fast-paced game like Super Mario Bros., reasoning models take several seconds to make decisions, but opportunities vanish quickly. A single second of hesitation can be fatal for Mario. In the dynamic world of gaming, speed is key!
While gaming has become a significant arena for AI competition, some experts remain skeptical. They argue that the gaming world is a virtual environment, vastly different from the real world. The simplified and abstract nature of the game allows AI to accumulate theoretical data, but this "book learning" doesn't fully reflect real-world capabilities. OpenAI research scientist Andrej Karpathy even raised concerns about the validity of these evaluations, prompting reflection.
Despite the skepticism, watching AI master Super Mario is a captivating display of technological advancement. It showcases the rapid progress of AI and offers a glimpse into the future. Who would have thought that AI, once confined to chessboards, could now excel in the gaming world? Perhaps in the near future, AI will truly dominate gaming, surpassing human players and becoming the true king! We'll just have to wait and see!
