AI2 recently released an open source dataset named Dolma, which contains 3 trillion tokens. Dolma's data will serve as the foundation for AI2's developing Open Language Model OLMo, expected to launch in early 2024. The Dolma dataset comes from a wide range of sources, including web content, academic publications, code, and books, making it the largest publicly available dataset of its kind.

["The Allen Institute for Artificial Intelligence has released the open text dataset Dolma, aimed at promoting transparency and innovation in AI language models.", "Dolma, as the core of AI2's open language model initiative, provides researchers and developers with free data resources.", "The Dolma dataset boasts a scale of 3 billion tokens, operates under the mid-risk ImpACT license, and encourages users to provide contact information and usage feedback."]