Anthropic has launched a program to fund the development of new types of benchmark tests to evaluate the performance and impact of AI models, including generative models like its own Claude.
The program was announced by Anthropic on Monday, which will provide funding to third-party organizations that can "effectively measure the advanced capabilities of AI models," as the company stated in a blog post. Interested parties can submit applications for rolling evaluations.
"Our investment in these evaluations is intended to elevate the entire AI safety field, providing valuable tools that benefit the entire ecosystem," Anthropic wrote on its official blog. "Developing high-quality, safety-related evaluations remains challenging, with demand outpacing supply."
As we have emphasized before, AI faces benchmarking issues. The benchmark tests most commonly cited by AI today often do not capture how ordinary people actually use the tested systems. Moreover, some benchmark tests, especially those released before the advent of modern generative AI, may even fail to measure what they claim to measure due to their outdated nature.
Anthropic's very high-level, seemingly difficult solution is to create challenging benchmark tests through new tools, infrastructure, and methods, focusing on AI safety and social impact.
The company specifically calls for tests to assess the models' abilities to complete tasks such as executing cyber attacks, "enhancing" weapons of mass destruction (such as nuclear weapons), and manipulating or deceiving people (such as through Deepfakes or misinformation). Anthropic says it is committed to developing a "early warning system" for AI risks involving national security and defense, although the blog post did not disclose what such a system might include.
Anthropic also stated that it plans to support the research of benchmark tests and "end-to-end" tasks through the new program, exploring the potential of AI in scientific research, multilingual communication, and reducing entrenched biases and self-censorship toxicity.
To achieve this goal, Anthropic envisions a new platform that allows discipline experts to develop their own evaluations and involves "thousands" of user model large-scale trials. The company said it has hired a full-time coordinator for the program and may purchase or expand promising projects.
Anthropic's efforts to support new AI benchmark tests are commendable — certainly, assuming sufficient funding and human resources. However, considering the company's commercial ambitions in AI competitions, complete trust in it may be difficult.
Anthropic also said it hopes its program will become a "catalyst for progress" to make comprehensive AI evaluation an industry standard in the future. This is a mission that many open, company-independent efforts can agree with. However, whether these efforts are willing to cooperate with AI suppliers whose loyalty ultimately belongs to shareholders is yet to be seen.
Key Points:
- 📌 Anthropic launches a program to fund new types of benchmark tests to evaluate the performance and impact of AI models.
- 📌 The program aims to create challenging benchmark tests focusing on AI safety and social impact.
- 📌 Anthropic hopes its program will become a "catalyst for progress" to make comprehensive AI evaluation an industry standard in the future.