llm.c

Utilizes simple C/CUDA for LLM training.

CommonProductProgrammingLLMC

karpathy/llm.c is a project that implements LLM training using a simple C/CUDA implementation. It aims to provide a clean and straightforward reference implementation while also including optimized versions that can approach the performance of PyTorch with drastically reduced code and dependencies. It is currently under development for a direct CUDA implementation, CPU version optimization using SIMD instructions, and support for more modern architectures such as Llama2 and Gemma.

Uses simple C/CUDA to implement LLM training
Provides a clean and simple reference implementation
Includes optimized versions with performance close to PyTorch
Supports various modern LLM architectures

Serves as a reference implementation for LLM training and a foundation for further optimization and expansion of LLM models.

Implemented a small-scale language model based on GPT-2 using karpathy/llm.c and achieved significant speed improvements through the optimized version.
Developed an LLM training tool that supports the Llama2 architecture based on karpathy/llm.c.
Referred to the code structure and implementation of karpathy/llm.c to develop a custom LLM model tailored to a specific application scenario.

Visit

llm.c Visit Over Time

Monthly Visits

499904316

Bounce Rate

37.31%

Page per Visit

5.8

Visit Duration

00:06:52

llm.c Visit Trend

llm.c Visit Geography

llm.c Traffic Sources

llm.c Alternatives

RAG_Techniques — A revolutionary collection of Retrieval-Augmented Generation system technologies.

Programming•RAG•NLP

132

Copilot in Excel — Copilot in Excel: Unlocking data-driven decision-making.

Productivity•Data Analysis•Automation

Mistral-Small-Instruct-2409 — High-performance instruct-tuned AI model

Programming•Artificial Intelligence•Natural Language Processing

Qwen2.5-LLM — An open-source high-performance language model that supports multi-platform applications.

Productivity•Natural Language Processing•Machine Learning

xAI — AI to Accelerate Human Scientific Discovery

Skywork-Reward-Llama-3.1-8B — An advanced reward model for text classification and preference assessment.

Programming•Machine Learning•Natural Language Processing

pixtral-12b-240910 — A multimodal large language model that supports understanding of both images and text.

Image•Multimodal•Image Processing

MemoRAG — A memory-based RAG framework for general-purpose applications

Programming•Natural Language Processing•Machine Learning

Qihoo-T2X — Qihoo-T2X, an efficient diffusion transformer model aimed at text-to-any-task processing.

Programming•Text Processing•Diffusion Transformer

rStar — Enhances problem-solving capabilities of small language models through self-play mutual reasoning.

Programming•Machine Learning•Natural Language Processing

RAGLAB — A modular, research-oriented unified framework for retrieval-augmented generation

Programming•Natural Language Processing•Machine Learning

DeepSeek-V2.5 — An AI model integrating general and programming capabilities.

Productivity•Artificial Intelligence•Machine Learning

126

West Lake AI Model — A multimodal model with high emotional and intellectual intelligence

ChineseSelection•Artificial Intelligence•Multimodal

120

RWKV v6 Finch 14B — RWKV v6 Finch 14B is an open-source large model that efficiently processes long texts.

Productivity•Artificial Intelligence•Natural Language Processing

162

C4AI Command R+ 08-2024 — A multilingual advanced dialogue generation model with 104 billion parameters.

Productivity•Artificial Intelligence•Machine Learning

LLM Playground — An interactive platform for loading and testing large language models.

Productivity•Artificial Intelligence•Machine Learning

Llama3-s v0.2 — Latest multimodal checkpoint to enhance speech comprehension capabilities.

Programming•Speech Recognition•Natural Language Processing

150

1.5-Pints — A compact large language model pre-trained in 9 days

Programming•Open Source•Machine Learning

150

MedTrinity-25M — A large-scale multimodal medical dataset

Others•Medical Imaging•Multimodal

252

Data-Juicer — A one-stop data processing system that provides high-quality data for large language models.

Productivity•Machine Learning•Data Science

174

llama3-s — An open-source language model currently being trained, equipped with 'hearing' capabilities.

Programming•Natural Language Processing•Machine Learning

WeST — Implement speech transcription based on LLM in just 300 lines of code.

Programming•Speech Recognition•Natural Language Processing

Gemma 2 2B — Lightweight, advanced 2B parameter text generation model.

Productivity•Text Generation•Natural Language Processing

126

Agents 2.0 — Open-source framework supporting data-driven adaptive language agents.

Programming•Open Source•Natural Language Processing

IncarnaMind — Engage in conversations with your documents using large language models like GPT.

Productivity•Machine Learning•PDF

120

Korvus — One-stop RAG Search SDK

Programming•Postgres•RAG

108

Best AI Websites & Tools

llm.c

llm.c Visit Over Time

llm.c Visit Trend

llm.c Visit Geography

llm.c Traffic Sources

llm.c Alternatives

RAG_Techniques — A revolutionary collection of Retrieval-Augmented Generation system technologies.

Copilot in Excel — Copilot in Excel: Unlocking data-driven decision-making.

Mistral-Small-Instruct-2409 — High-performance instruct-tuned AI model

Qwen2.5-LLM — An open-source high-performance language model that supports multi-platform applications.

xAI — AI to Accelerate Human Scientific Discovery

Skywork-Reward-Llama-3.1-8B — An advanced reward model for text classification and preference assessment.

pixtral-12b-240910 — A multimodal large language model that supports understanding of both images and text.

MemoRAG — A memory-based RAG framework for general-purpose applications

Qihoo-T2X — Qihoo-T2X, an efficient diffusion transformer model aimed at text-to-any-task processing.

rStar — Enhances problem-solving capabilities of small language models through self-play mutual reasoning.

RAGLAB — A modular, research-oriented unified framework for retrieval-augmented generation

DeepSeek-V2.5 — An AI model integrating general and programming capabilities.

West Lake AI Model — A multimodal model with high emotional and intellectual intelligence

Reflection Llama-3.1 70B — The world's leading open-source large language model

OLMoE-1B-7B — An efficient open-source large language model.

Aixploria — AI tools directory for discovering the best AI tools

RWKV v6 Finch 14B — RWKV v6 Finch 14B is an open-source large model that efficiently processes long texts.

C4AI Command R+ 08-2024 — A multilingual advanced dialogue generation model with 104 billion parameters.

LLM Playground — An interactive platform for loading and testing large language models.

Llama3-s v0.2 — Latest multimodal checkpoint to enhance speech comprehension capabilities.

1.5-Pints — A compact large language model pre-trained in 9 days

MedTrinity-25M — A large-scale multimodal medical dataset

Data-Juicer — A one-stop data processing system that provides high-quality data for large language models.

llama3-s — An open-source language model currently being trained, equipped with 'hearing' capabilities.

WeST — Implement speech transcription based on LLM in just 300 lines of code.

Gemma 2 2B — Lightweight, advanced 2B parameter text generation model.

Agents 2.0 — Open-source framework supporting data-driven adaptive language agents.

IncarnaMind — Engage in conversations with your documents using large language models like GPT.

Korvus — One-stop RAG Search SDK