LLMs-from-scratch

Deep dive into the inner workings of large language models.

CommonProductProgrammingLanguage ModelsDeep Learning

LLMs-from-scratch takes you on a journey to understand how LLMs work. This book guides you step-by-step in creating your own LLM, explaining each stage with clear text, diagrams, and examples. The described methods for training and developing small but functional models for educational purposes share similarities with techniques used to create large-scale foundation models like ChatGPT.

Deep understanding of large language models
Processing text data
Implementing attention mechanisms
Building a GPT model from scratch
Pre-training with unlabeled data
Fine-tuning for text classification
Fine-tuning with human feedback
Practical applications of large language models

Education & Research

Textbook for university deep learning courses
Reference guide for researchers
Self-learning resource for AI engineers

Visit

LLMs-from-scratch Visit Over Time

Monthly Visits

515580771

Bounce Rate

37.20%

Page per Visit

5.8

Visit Duration

00:06:42

LLMs-from-scratch Visit Trend

LLMs-from-scratch Visit Geography

LLMs-from-scratch Traffic Sources

LLMs-from-scratch Alternatives

LLaMA-Mesh — Unified 3D Mesh Generation with Language Models

Productivity•3D Modeling•Artificial Intelligence

MaskGCT TTS Demo — Text-to-speech demonstration based on the MaskGCT model.

Others•Text-to-Speech•Deep Learning

582

mPLUG-DocOwl 1.5 — Unified Structural Learning Model for OCR-free Document Understanding

Productivity•Document Understanding•Deep Learning

Prompt Engineering — A comprehensive resource library for Prompt Engineering techniques.

Productivity•Prompt Engineering•Machine Learning

F5-TTS — A high-quality text-to-speech synthesis model based on deep learning.

Productivity•text-to-speech•deep learning

936

Entropy-based Sampling — Entropy-based sampling technique that optimizes the diversity and accuracy of model outputs

Programming•Machine Learning•Natural Language Processing

Llama 3.2 3b Voice — Voice synthesis tool using the Llama model

Productivity•Speech Synthesis•Natural Language Processing

642

rStar — Enhances problem-solving capabilities of small language models through self-play mutual reasoning.

Programming•Machine Learning•Natural Language Processing

Aixploria — AI tools directory for discovering the best AI tools

Productivity•AI tools•AI navigation

258

RWKV — The new generation of large-scale model architecture, surpassing transformer.

Productivity•Open Source•Deep Learning

114

HippoRAG — A novel RAG framework inspired by human long-term memory

Programming•Natural Language Processing•Knowledge Integration

198

llama3-from-scratch — Implement the Llama3 model from scratch

Programming•Deep Learning•Natural Language Processing

822

RAGFlow — An open-source Retrieval-Augmented Generation (RAG) engine based on deep document understanding

Productivity•Natural language processing•Machine learning

576

llava-llama-3-8b-v1_1 — A LLaVA model optimized by XTuner, which combines image and text processing capabilities.

Programming•Artificial Intelligence•Multimodal Learning

420

EasyContext — EasyContext demonstrates how to leverage existing technologies to train language models with 700K and 1M context lengths.

Programming•Language Models•Context Length

132

nasa-smd-ibm-st — Enhances natural language technologies for information retrieval and intelligent search in the NASA Science Mission Directorate (SMD).

Programming•NASA•Deep Learning

216

LaVi-Bridge — Connects different language models and generative visual models for text-to-image generation

Image•Text-to-Image Generation•Language Models

600

MovieLLM — MovieLLM is an AI-powered film framework designed to enhance long video understanding.

Video•Image Processing•Video Creation

2022

KarpathyLLMChallenge — Deep dive into the tokenization process within language models

Education•Language Models•Tokenization

BASE TTS — Amazon's Large-scale Voice Synthesis Model

Others•Voice Synthesis•Natural Language Processing

1236

ASPIRE — A framework to enhance the selective prediction capability of large language models

Productivity•Artificial Intelligence•Natural Language Processing

DenserBot — Transform data into knowledge

chatting•Chatbot•Large Language Model

150

PromptPoint — A tool for efficiently designing, testing, and deploying prompts.

Productivity•Language Models•Natural Language Processing

210

newsanalyst — An AI news analyst that updates global political, technological, and business news every hour

Productivity•News Analysis•Global Events

126

Qwen Turbo 1M Demo — The Qwen Turbo 1M Demo is a Hugging Face space provided by Qwen.

Productivity•Natural Language Processing•Machine Learning

Best AI Websites & Tools