Recently, a unique AI capability assessment took place on the Minecraft platform, attracting a lot of attention. The old and new versions of Claude3.5Sonnet competed in building challenges, highlighting significant differences in their abilities, with the new version (tentatively called Sonnet3.6) performing especially well. This test, initiated by developer adi, has been humorously dubbed the only reliable assessment benchmark. Assessment researcher Aidan McLau believes this method precisely meets the current needs for AI evaluation and points out the relationship with aesthetic abilities.

Beijing Academy of Artificial Intelligence (BAAI) has recently launched the world’s first Chinese large model debate platform, FlagEval Debate. This new platform aims to provide a novel measurement approach for evaluating the capabilities of large language models through a competitive mechanism of model debates. It is an extension of the FlagEval arena for large model combat evaluation services, designed to identify capability differences among large language models.

OpenAI has released SWE-bench Verified, aiming to more accurately assess AI performance in software engineering tasks and address the limitations of the original SWE-bench, such as overly strict unit tests, ambiguous problem descriptions, and challenging development environment setups. The new benchmark improves assessment consistency and reliability by introducing a containerized Docker environment, significantly enhancing the performance scoring of AI models. GPT-4o solved 33.2% of the samples under the new benchmark, while the best open-source agent framework has...

OpenAI recently released the GPT-4o System Card, detailing the safety measures and risk assessments prior to the launch of the new model. GPT-4o was officially launched in May, and the assessment showed an overall risk rating of 'medium', with major risks concentrated in cybersecurity, biological threats, persuasive capability, and model autonomy. Researchers found that while GPT-4o may be more persuasive in influencing reader opinions, it did not surpass humans overall. At the time of releasing the system card, OpenAI faced criticism from internal employees and state senators questioning its decision-making.

With the rapid development of artificial intelligence, educators are exploring its applications in the new school year, especially after experiencing learning losses due to the COVID-19 pandemic. Strategic assessments, often overlooked, have been found to significantly boost student performance. By utilizing AI to generate frequent, low-risk evaluations, teachers can effectively leverage the 'testing effect', meaning that students learn better through testing compared to traditional methods, with retention improvements of up to 50%. This type of assessment is suitable for various subjects and age groups, even for those with memory impairments.

Ant Group, in collaboration with Xin京Bao and Beike Finance, has developed and launched a generative large-scale model product named "Beike Finance · Smart Guidance for ESG." This product aims to provide users with more accurate, comprehensive, and valuable analysis and interpretation of ESG (Environmental, Social, and Corporate Governance) reports of A-share listed companies. It can answer various user queries in real-time and offer in-depth analysis and summaries."Beike Finance · Smart Guida

Hugging Face has updated its Open LLM Leaderboard, a move that will have a significant impact on the landscape of open-source AI development. This improvement comes at a crucial moment in AI development as researchers and companies are facing a plateau in the performance enhancement of large language models (LLMs).

In the world of artificial intelligence, the Gaokao is no longer just a human stage. Recently, the Shanghai Artificial Intelligence Lab hosted a unique 'Gaokao' that showcased the academic capabilities of AI. They utilized the OpenCompass assessment system, testing seven AI models, including GPT-4o, on their comprehensive abilities in Chinese, Mathematics, and English.

AI researchers generally believe that the development of superintelligent AI poses an insignificant yet very small risk of human extinction. The largest survey of AI researchers shows that about 58% of them believe there is a 5% chance of leading to human extinction or other extreme negative AI-related outcomes. Researchers have a wide range of disagreements over the timeline for future AI technological milestones and feel uncertain about the social consequences that AI may bring. The survey also reveals the urgent concerns AI researchers have regarding AI-driven scenarios such as deepfakes, manipulation of public opinion, and weaponization.

Recent research shows that AI outperforms biopsies in assessing the invasiveness of certain cancers, potentially saving thousands of patients' lives. A study focused on sarcomas indicates that AI algorithms are more accurate than tissue biopsies, providing better treatment outcomes for patients. This technology is expected to expand to other cancer types, potentially benefiting many more individuals.

Research shows that large pre-trained language models (LLMs), such as GPT-3, can improve themselves through the RAIN method to meet human needs without additional data and fine-tuning. The RAIN method is widely applicable to various language generation tasks, requiring no additional models or data storage, nor does it rely on labeled data or training. RAIN enhances the performance of LLMs through self-assessment, reducing the success rate of adversarial attacks, and enabling AI to generate more coherent and secure responses. This study introduces RAIN as a technique for adjusting LLMs to align with human preferences.

Scientists have proposed a checklist of standards to evaluate whether artificial intelligence possesses autonomous consciousness. The checklist includes multiple theories to assess consciousness by matching the capabilities of AI systems. AI companies need to put in more effort to evaluate consciousness and develop corresponding plans.