At the recent Gemma Developer Day held in Tokyo, Google officially launched its new Japanese version of the Gemma AI model. This model's performance rivals that of GPT-3.5, but it has a parameter count of only 2 billion, making it very compact and suitable for operation on mobile devices. The newly released Gemma model excels in Japanese processing while also maintaining its capabilities in English. This is particularly important for small models, as they may face 'catastrophic forgetting' when fine-tuning for new languages.

Recently, Zyphra officially launched Zamba2-7B, a small language model with unprecedented performance, featuring 7B parameters. This model claims to surpass its current competitors in quality and speed, including Mistral-7B, Google's Gemma-7B, and Meta's Llama3-8B. The design goal of Zamba2-7B is to meet the needs of environments that require strong language processing capabilities but are constrained by hardware conditions, such as processing or using it on devices.

Recently, Google announced the launch of a new open-source AI model, DataGemma, aimed at addressing the 'hallucination' problem that large language models (LLMs) often encounter when handling statistical data. This hallucination phenomenon causes the model to potentially provide inaccurate answers to questions related to numbers and statistics. The release of the DataGemma model marks a significant advancement for Google in the field of AI. Image source note: Image generated by AI, image licensed by service provider Midjourney to reduce hallucinations during statistical queries.

Google DeepMind's latest research, Gemma Scope, unveils the secrets of the 'black box' of language models by using Sparse Autoencoders (SAEs) to decompose and reconstruct the activations of language models, aiming to reveal meaningful features. Gemma Scope employs JumpReLU SAEs, which optimize reconstruction loss and regularize the number of active latent features by controlling activations to reveal the internal mechanisms of language models. The research found that the performance of residual flow SAEs is generally low, and sequence length affects the performance of SAEs.

Google launched the Gemma22B model with 2 billion parameters, demonstrating exceptional performance across various hardware environments, particularly outperforming models like GPT-3.5-Turbo-0613 and Mixtral-8x7b in the chatbot arena. The model is optimized with the NVIDIA TensorRT-LLM library for efficient deployment in diverse scenarios, from edge devices to powerful cloud environments. To enhance safety, Google introduced the ShieldGemma safet....

Recently at the Google I/O Berlin conference, Google announced the open-sourcing of its latest language model Gemma2, which has made significant breakthroughs in performance and efficiency. Gemma2 offers two parameter sizes: 9B and 27B, with the 27B version's performance approaching that of the 70B Llama3 model, but with a model size that is only about 40% of the latter.

Google has released the open source AI large model Gemma, with the domestic open source large model InternLM2 outperforming Gemma in all aspects. The competition in the open source large model field is intense, making the large model landscape increasingly lively.

Google has launched a new series of open-source models called 'Gemma'. A comparison between Gemma, Llama 2, and Mistral. The design principles and features of Gemma. Characteristics of the GeGLU activation function used in Gemma.