DCLM-baseline

High-performance language model benchmark dataset

CommonProductProgrammingNatural language processingLanguage model

DCLM-baseline is a pretraining dataset for language model benchmarking, containing 4T tokens and 3B documents. It is curated from the Common Crawl dataset after a careful planning of data cleaning, filtering, and deduplication steps, aiming to demonstrate the importance of data curation in training efficient language models. The dataset is only for research purposes and should not be used in production environments or for training domain-specific models, such as those for code and mathematics.

Best AI Websites & Tools

DCLM-baseline

DCLM-baseline Visit Over Time

DCLM-baseline Visit Trend

DCLM-baseline Visit Geography

DCLM-baseline Traffic Sources

DCLM-baseline Alternatives

MobileLLM-1B — A sub-billion parameter language model developed by Meta, suitable for device-side applications.

PARTNR — Benchmarking for Multi-Agent Task Planning and Reasoning

MobileLLM-350M — An efficiently optimized language model with sub-billion parameters, specifically designed for device-side applications.

Zamba2-7B — High-performance small language model

Cheating LLM Benchmarks — A research project that explores cheating behaviors in automated language model benchmarking.

MedTrinity-25M — A large-scale multimodal medical dataset

Meta Llama 3.1-405B — Large multilingual pre-trained language model

Arcee Spark — A highly efficient and compact 7B parameter language model

Multi-Token Prediction — A multi-token prediction model designed to boost the efficiency and performance of language models

MDLM — An efficient masked diffusion language model.

Samba — Official implementation of an efficient infinite context language model

emo-visual-data — Emoji Visual Annotation Dataset

FlashRAG — Efficient Toolkit for Retrieval-Augmented Generation Research

MAP-NEO — MAP-NEO is an entirely open-source large language model offering advanced natural language processing capabilities.

ImageInWords — A model for generating highly detailed image descriptions, designed for training visual language models.

Trustworthy Language Model (TLM) Playground — Try Cleanlab's Trustworthy Language Model (TLM) in your browser

WildChat — A corpus composed of real-world user interactions with ChatGPT.

LLaVA++ — LLaVA++ extends the LLaVA model by integrating Phi-3 and LLaMA-3, enhancing the interaction capability between visual and language models.

OpenELM — OpenELM is a family of efficient language models equipped with an open-source training and inference framework.

FineWeb — High-quality English webpage dataset

Cappy — A lightweight scoring model that enhances the performance of large, multi-task language models.

MNBVC — MNBVC is a massive Chinese corpus comparable to the 40T data used to train ChatGPT.

H2O-Danube-1.8B — A 1.8B parameter language model, open-source and free.

Baichuan 3 — A large language model with over trillion parameters

Lepton Search — Lepton is an open-source language model search platform

MaLA-500 — A large language model covering 534 languages

Wiseses AI — Intelligent Content Creation Platform

TinyGPT-V — Efficient multimodal large language model

intfloat/e5-mistral-7b-instruct — A text embedding model improved by a large language model for better text representation.