Alibaba's DAMO Academy has open-sourced a multilingual large language model, Babel, aiming to bridge the language gap and enable AI to communicate in languages spoken by over 90% of the global population.
Many current large language models favor resource-rich languages like English, French, and German. However, similar to how speakers of less-represented languages are often overlooked in global conferences, languages with vast user bases such as Hindi, Bengali, and Urdu are frequently neglected in the AI field.
Alibaba's Babel aims to change this. It supports the top 25 most spoken languages globally, covering over 90% of the world's population. Even more commendable is its inclusion of Swahili, Javanese, Burmese, and other languages rarely explored in open-source LLMs. This initiative will undoubtedly bring more convenient and higher-quality AI language services to billions of people.
Unlike traditional continuous pre-training methods, Babel employs a unique layer expansion technique to enhance its capabilities. This method can be understood as adding "knowledge reserves" to the model's foundation in a more sophisticated way, improving performance while maintaining computational efficiency. The research team has released two distinct models: Babel-9B, optimized for efficient single-GPU inference and fine-tuning; and Babel-83B, an 83-billion-parameter model aiming to set a new benchmark for open-source multilingual LLMs.
To validate Babel's capabilities, the research team conducted rigorous evaluations across multiple multilingual tasks. The results are impressive: both the 9-billion-parameter Babel-9B and the 83-billion-parameter Babel-83B outperformed other open-source models of comparable size in several benchmark tests. For example, Babel excelled in tasks such as world knowledge (MMMLU, M3Exam), reasoning (MGSM, XCOPA), understanding (XNLI), and translation (Flores-200).
Particularly noteworthy is that Babel achieved a 5% to 10% accuracy improvement over previous multilingual LLMs when handling low-resource languages. This demonstrates Babel's focus on performance across various languages while expanding language coverage.
Even more excitingly, after supervised fine-tuning (SFT) on over one million conversational datasets, Babel's chat versions, Babel-9B-Chat and Babel-83B-Chat, demonstrated powerful conversational abilities. Their performance is comparable to some top commercial AI models, with Babel-83B-Chat even rivaling GPT-4 on certain tasks. This injects new vitality into the open-source community, proving that open-source models can achieve leading performance in multilingual capabilities.
