LLMs-from-scratch

Deep dive into the inner workings of large language models.

CommonProductProgrammingLanguage ModelsDeep Learning

LLMs-from-scratch takes you on a journey to understand how LLMs work. This book guides you step-by-step in creating your own LLM, explaining each stage with clear text, diagrams, and examples. The described methods for training and developing small but functional models for educational purposes share similarities with techniques used to create large-scale foundation models like ChatGPT.

Best AI Websites & Tools

LLMs-from-scratch

LLMs-from-scratch Visit Over Time

LLMs-from-scratch Visit Trend

LLMs-from-scratch Visit Geography

LLMs-from-scratch Traffic Sources

LLMs-from-scratch Alternatives

mwp_ReFT — A deep reinforcement learning-based model fine-tuning framework

Florence-VL — Enhancement tool for visual language models, combining generative visual encoders and deep breadth fusion technology.

PaliGemma 2 — PaliGemma 2 is a powerful visual language model that is easy to fine-tune.

LLaMA-Mesh — Unified 3D Mesh Generation with Language Models

MaskGCT TTS Demo — Text-to-speech demonstration based on the MaskGCT model.

mPLUG-DocOwl 1.5 — Unified Structural Learning Model for OCR-free Document Understanding

Prompt Engineering — A comprehensive resource library for Prompt Engineering techniques.

F5-TTS — A high-quality text-to-speech synthesis model based on deep learning.

Entropy-based Sampling — Entropy-based sampling technique that optimizes the diversity and accuracy of model outputs

Llama 3.2 3b Voice — Voice synthesis tool using the Llama model

rStar — Enhances problem-solving capabilities of small language models through self-play mutual reasoning.

Aixploria — AI tools directory for discovering the best AI tools

RWKV — The new generation of large-scale model architecture, surpassing transformer.

HippoRAG — A novel RAG framework inspired by human long-term memory

llama3-from-scratch — Implement the Llama3 model from scratch

RAGFlow — An open-source Retrieval-Augmented Generation (RAG) engine based on deep document understanding

llava-llama-3-8b-v1_1 — A LLaVA model optimized by XTuner, which combines image and text processing capabilities.

EasyContext — EasyContext demonstrates how to leverage existing technologies to train language models with 700K and 1M context lengths.

nasa-smd-ibm-st — Enhances natural language technologies for information retrieval and intelligent search in the NASA Science Mission Directorate (SMD).

LaVi-Bridge — Connects different language models and generative visual models for text-to-image generation

MovieLLM — MovieLLM is an AI-powered film framework designed to enhance long video understanding.

KarpathyLLMChallenge — Deep dive into the tokenization process within language models

BASE TTS — Amazon's Large-scale Voice Synthesis Model

ASPIRE — A framework to enhance the selective prediction capability of large language models

DenserBot — Transform data into knowledge

PromptPoint — A tool for efficiently designing, testing, and deploying prompts.

newsanalyst — An AI news analyst that updates global political, technological, and business news every hour

Claude — Building the future of AI

Quickchat — Let your AI assistant communicate like a human.