At the recent Gemma Developer Day held in Tokyo, Google officially launched its new Japanese version of the Gemma AI model. This model's performance rivals that of GPT-3.5, but it has a parameter count of only 2 billion, making it very compact and suitable for operation on mobile devices. The newly released Gemma model excels in Japanese processing while also maintaining its capabilities in English. This is particularly important for small models, as they may face 'catastrophic forgetting' when fine-tuning for new languages.

In a recent study, the OpenAI research team launched a new benchmark called MLE-bench, aimed at assessing the performance of AI agents in machine learning engineering. This research particularly focuses on 75 machine learning engineering-related competitions from Kaggle, intending to test agents' abilities in various skills required in the real world, including model training, dataset preparation, and experiment execution. To facilitate better evaluation, the research team utilized foundational data from Kaggle's public leaderboard to establish performance metrics for each competition.