The OpenCompass team from Shanghai Artificial Intelligence Laboratory and ModelScope have jointly launched an upgrade for the large model evaluation platform CompassArena. This upgrade aims to provide users with a more scientific and comprehensive model evaluation experience. Since its launch, the platform has attracted a large number of community users to participate and contribute data. Based on this data, CompassArena continues to optimize, and this upgrade includes the new Judge Copilot feature and improvements to the ranking algorithm.

Alibaba DAMO Academy, in collaboration with the ModelScope community, recently announced the open sourcing of a new multilingual benchmark testing set, P-MMEval, aimed at comprehensively evaluating the multilingual capabilities of Large Language Models (LLMs) and conducting comparative analysis of cross-language transfer abilities. This testing set covers efficient datasets for both basic and specialized capabilities, ensuring consistency in multilingual coverage across all selected datasets, and provides parallel samples across multiple languages, supporting up to 10 languages from 8 different language families, including English, Chinese, and Arabic.

On December 5th, the Byte Bean Bag model team launched the latest code model evaluation benchmark - FullStack Bench, covering over 11 real-world scenarios, supporting 16 programming languages, and including 3374 questions. Compared to previous evaluation standards, this benchmark can more accurately assess the code development capabilities of large models across a broader programming domain, facilitating the optimization of models in real-world programming tasks. Current mainstream code evaluation benchmarks, such as HumanEval and MBPP, typically focus on basic and advanced.

In the vast universe of artificial intelligence, mathematics has long been considered the last bastion of machine intelligence. Now, a new benchmarking test named FrontierMath has emerged, pushing AI's mathematical reasoning capabilities to unprecedented limits. Epoch AI has collaborated with over 60 top minds in mathematics to create this challenge, which can be described as the 'Olympics of Mathematics' for AI. This is not just a technology test; it is the ultimate interrogation of artificial intelligence's mathematical wisdom. Imagine a laboratory filled with the world's top mathematicians, meticulously designing...

The rapid development of generative AI has raised the challenge of comprehensively evaluating its performance. A plethora of models have emerged, each showcasing increasingly impressive results. However, the pressing question remains: how do we assess the effectiveness of these text-to-image models? Traditional evaluation methods often rely on subjective visual assessments or simple metrics like CLIPScore, which frequently fail to capture the intricate details found in complex text prompts, such as relationships between objects and logical reasoning. This leads to inaccurate evaluation outcomes for many text-to-image models.

During a recent visit to China, Apple CEO Tim Cook was asked about when the Apple Intelligence service would enter the Chinese market during a visit to Weibo. In response, Cook stated that the company is actively promoting the matter, but they need to complete a series of specific regulatory procedures before bringing the service to Chinese consumers. He added that Apple is very much looking forward to providing this service to Chinese users as soon as possible.

Following the New York Times, the New York Post (NYP Holdings) and the Wall Street Journal (Dow Jones) have recently jointly filed a lawsuit against the AI company Perplexity AI in federal court in Manhattan, accusing it of 'mass illegal copying' of copyright works from these two media outlets. This AI startup, backed by Amazon founder Jeff Bezos, is facing serious copyright infringement charges. The lawsuit demands that Perplexity cease using these news articles to respond to queries.