Popular GitHub Repositories for Flash Attention

中文LLaMA-2 & Alpaca-2大模型二期项目 + 64K超长上下文模型 (Chinese LLaMA-2 & Alpaca-2 LLMs with 64K long context models)

Official release of InternLM series (InternLM, InternLM2, InternLM2.5, InternLM3).

?A curated list of Awesome LLM/VLM Inference Papers with codes: WINT8/4, Flash-Attention, Paged-Attention, Parallelism, etc. ??

FlashInfer: Kernel Library for LLM Serving

MiniMax-01 is a simple implementation of the MiniMax algorithm, a widely used strategy for decision-making in two-player turn-based games like Tic-Tac-Toe. The algorithm aims to minimize the maximum possible loss for the player, making it a popular choice for developing AI opponents in various game scenarios.

MoBA: Mixture of Block Attention for Long-Context LLMs

?FFPA(Split-D): Yet another Faster Flash Prefill Attention with O(1) GPU SRAM complexity for headdim > 256, ~2x↑?vs SDPA EA.

Train llm (bloom, llama, baichuan2-7b, chatglm3-6b) with deepspeed pipeline mode. Faster than zero/zero++/fsdp.

Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.

Decoding Attention is specially optimized for MHA, MQA, GQA and MLA using CUDA core for the decoding stage of LLM inference.

Fast and memory efficient PyTorch implementation of the Perceiver with FlashAttention.

A flexible and efficient implementation of Flash Attention 2.0 for JAX, supporting multiple backends (GPU/TPU/CPU) and platforms (Triton/Pallas/JAX).

Utilities for efficient fine-tuning, inference and evaluation of code generation models

An simple pytorch implementation of Flash MultiHead Attention

? Automated deployment stack for AMD MI300 GPUs with optimized ML/DL frameworks and HPC-ready configurations

Long term project about a custom AI architecture. Consist of cutting-edge technique in machine learning such as Flash-Attention, Group-Query-Attention, ZeRO-Infinity, BitNet, etc.

? Automated installation script for vLLM on HPC systems with ROCm support, optimized for AMD MI300X GPUs.