llm.c

Utilizes simple C/CUDA for LLM training.

CommonProductProgrammingLLMC

karpathy/llm.c is a project that implements LLM training using a simple C/CUDA implementation. It aims to provide a clean and straightforward reference implementation while also including optimized versions that can approach the performance of PyTorch with drastically reduced code and dependencies. It is currently under development for a direct CUDA implementation, CPU version optimization using SIMD instructions, and support for more modern architectures such as Llama2 and Gemma.

Uses simple C/CUDA to implement LLM training
Provides a clean and simple reference implementation
Includes optimized versions with performance close to PyTorch
Supports various modern LLM architectures

Serves as a reference implementation for LLM training and a foundation for further optimization and expansion of LLM models.

Implemented a small-scale language model based on GPT-2 using karpathy/llm.c and achieved significant speed improvements through the optimized version.
Developed an LLM training tool that supports the Llama2 architecture based on karpathy/llm.c.
Referred to the code structure and implementation of karpathy/llm.c to develop a custom LLM model tailored to a specific application scenario.

Visit

llm.c Visit Over Time

Monthly Visits

490881889

Bounce Rate

37.92%

Page per Visit

5.6

Visit Duration

00:06:18

llm.c Visit Trend

llm.c Visit Geography

llm.c Traffic Sources

llm.c Alternatives

Prompt Engineering — A comprehensive resource library for Prompt Engineering techniques.

Productivity•Prompt Engineering•Machine Learning

RAG_Techniques — A revolutionary collection of Retrieval-Augmented Generation system technologies.

Programming•RAG•NLP

180

Imitate Before Detect — An advanced approach for detecting machine-revised text, improving detection accuracy by mimicking machine styles.

Programming•Text Detection•Machine Learning

Sonus-1 — Sonus-1: A New Era of Large Language Models (LLMs)

Productivity•Artificial Intelligence•Large Language Models

300

Text-to-CAD UI — Creates B-Rep CAD files and meshes from natural language prompts.

Design•CAD•3D Modeling

210

InternVL2_5-8B-MPO — A large multimodal language model showcasing exceptional overall performance.

Image•Multimodal•Large Language Model

120

Llama-3.1-70B-Instruct-AWQ-INT4 — Text generation model with 70 billion parameters

Productivity•Text Generation•Natural Language Processing

ModernBERT — ModernBERT is a next-generation encoder model with outstanding performance.

Programming•BERT•Transformer

Smolagents — A lightweight library for building high-performance intelligent agents.

Programming•Intelligent Agents•Machine Learning

234

Llama-lynx-70b-4bitAWQ — A 70 billion parameter text generation model.

Programming•Text Generation•Natural Language Processing

PromptWizard — Task-aware prompt optimization framework

Programming•Microsoft•LLM

408

Gemini 2.0 Flash Experimental — A high-performance AI model developed by Google DeepMind

InternationalSelection•Machine Learning•Natural Language Processing

372

Q-RWKV-6 32B Instruct Preview — The most powerful variant of the RWKV model, breaking multiple English benchmark tests.

Programming•Machine Learning•Natural Language Processing

CosyVoice Speech Generation Model 2.0-0.5B — Efficient, multilingual speech synthesis model

Music•Speech Synthesis•Artificial Intelligence

336

tulu-3-sft-olmo-2-mixture — A large-scale multilingual text dataset.

Others•Multilingual•Text Dataset

OLMo-2-1124-7B-RM — A large language model for text generation and classification.

Productivity•Artificial Intelligence•Natural Language Processing

ChatGPT Pro — Scalable access plan for cutting-edge AI models.

InternationalSelection•Machine Learning•Data Science

312

PaliGemma 2 — PaliGemma 2 is a powerful visual language model that is easy to fine-tune.

Productivity•Visual Language Model•Machine Learning

OLMo 2 1124 7B Preference Mixture — A large-scale textual dataset for preference mixture research.

Others•Natural Language Processing•Text Dataset

OLMo-2-1124-7B-SFT — High-performance English text generation model

Productivity•Text Generation•Natural Language Processing

OLMo-2-1124-7B-DPO — An advanced text generation model supporting diverse task handling.

Productivity•Text Generation•Natural Language Processing

OLMo-2-1124-13B-DPO — High-performance English language model suitable for diverse tasks.

Programming•Language Model•Natural Language Processing

ProactiveAgent — A proactive agent based on large language models that predicts user needs and offers assistance.

Programming•Artificial Intelligence•Machine Learning

OpenScholar — A retrieval-augmented language model for synthesizing scientific literature.

Education•Scientific Literature•Retrieval Augmentation

dolmino-mix-1124 — A high-quality dataset for the second phase of OLMo2 training.

Programming•Dataset•Natural Language Processing

OLMo-2-1124-7B-Instruct — High-performance English dialogue generation model

chatting•Natural Language Processing•Dialogue Generation

Skywork-o1-Open-PRM-Qwen-2.5-7B — High-performance AI model that enhances reasoning task capabilities.

Programming•AI Reasoning•Machine Learning

150

OLMo 2 — State-of-the-art fully open language model

Programming•Language Model•Natural Language Processing

ZipPy — A tool for rapid detection of AI-generated text using compression ratios.

Programming•AI Detection•Text Analysis

Best AI Websites & Tools

llm.c

llm.c Visit Over Time

llm.c Visit Trend

llm.c Visit Geography

llm.c Traffic Sources

llm.c Alternatives

Prompt Engineering — A comprehensive resource library for Prompt Engineering techniques.

RAG_Techniques — A revolutionary collection of Retrieval-Augmented Generation system technologies.

Imitate Before Detect — An advanced approach for detecting machine-revised text, improving detection accuracy by mimicking machine styles.

Sonus-1 — Sonus-1: A New Era of Large Language Models (LLMs)

Text-to-CAD UI — Creates B-Rep CAD files and meshes from natural language prompts.

InternVL2_5-8B-MPO — A large multimodal language model showcasing exceptional overall performance.

Llama-3.1-70B-Instruct-AWQ-INT4 — Text generation model with 70 billion parameters

ModernBERT — ModernBERT is a next-generation encoder model with outstanding performance.

Smolagents — A lightweight library for building high-performance intelligent agents.

Llama-lynx-70b-4bitAWQ — A 70 billion parameter text generation model.

PromptWizard — Task-aware prompt optimization framework

Gemini 2.0 Flash Experimental — A high-performance AI model developed by Google DeepMind

Q-RWKV-6 32B Instruct Preview — The most powerful variant of the RWKV model, breaking multiple English benchmark tests.

CosyVoice Speech Generation Model 2.0-0.5B — Efficient, multilingual speech synthesis model

tulu-3-sft-olmo-2-mixture — A large-scale multilingual text dataset.

OLMo-2-1124-7B-RM — A large language model for text generation and classification.

ChatGPT Pro — Scalable access plan for cutting-edge AI models.

PaliGemma 2 — PaliGemma 2 is a powerful visual language model that is easy to fine-tune.

OLMo 2 1124 7B Preference Mixture — A large-scale textual dataset for preference mixture research.

OLMo-2-1124-7B-SFT — High-performance English text generation model

OLMo-2-1124-7B-DPO — An advanced text generation model supporting diverse task handling.

OLMo-2-1124-13B-DPO — High-performance English language model suitable for diverse tasks.

ProactiveAgent — A proactive agent based on large language models that predicts user needs and offers assistance.

OpenScholar — A retrieval-augmented language model for synthesizing scientific literature.

dolmino-mix-1124 — A high-quality dataset for the second phase of OLMo2 training.

OLMo-2-1124-7B-Instruct — High-performance English dialogue generation model

Skywork-o1-Open-PRM-Qwen-2.5-7B — High-performance AI model that enhances reasoning task capabilities.

OLMo 2 — State-of-the-art fully open language model

ZipPy — A tool for rapid detection of AI-generated text using compression ratios.