The Technology Innovation Institute (TII) supported by the UAE government recently announced the launch of its next-generation open-source Small Language Model (SLM) - the Falcon3 series. This series includes four models of different sizes: 1B, 3B, 7B, and 10B, offering both base and instruction variants, aimed at providing developers, researchers, and businesses with an efficient and cost-effective AI solution. The launch of these models marks a further democratization of AI capabilities, allowing them to run on lightweight single GPU infrastructures, catering to devices and applications with limited computing resources.
Image Source Note: Image generated by AI, image licensed from Midjourney
Falcon3 has already stood out on the Hugging Face leaderboard, surpassing open-source models of similar sizes, such as Meta's Llama and Qwen-2.5. Notably, the 7B and 10B versions demonstrate leading technical advantages in inference speed, language understanding, instruction execution, as well as in coding and mathematical tasks, even outperforming competitors like Google, Meta, and Alibaba in multiple benchmark tests.
Compared to traditional large language models (LLMs), SLM models offer efficiency and cost advantages due to their fewer parameters and simpler designs, making them particularly suitable for applications in customer service, healthcare, and the Internet of Things (IoT). According to market research firm Valuates Reports, the SLM market is expected to achieve an 18% annual growth rate over the next five years.
The training data size for the Falcon3 series has reached 14 trillion tokens, more than double that of its predecessor Falcon2. This series employs a decoder-only architecture and a grouped query attention mechanism, minimizing memory usage while enhancing inference efficiency. Falcon3 supports four languages: English, French, Spanish, and Portuguese, and is equipped with a 32K context window, capable of handling long input texts to meet various industry needs.
TII stated that the base model of Falcon3 is suitable for general tasks, while the instruction version is optimized for conversational tasks such as customer service and virtual assistants. The launch of this series will further promote the development of edge computing and privacy-sensitive applications, supporting scenarios like personalized recommendations, data analysis, medical diagnosis, and supply chain optimization.
All Falcon3 models are released under the TII Falcon License 2.0, a permissive license based on Apache 2.0, supporting responsible AI development and deployment. To assist developers and researchers in getting started, TII has also launched the Falcon Playground testing environment, allowing users to try out these models before integration.
