Popular GitHub Repositories for Flash Attention

中文LLaMA-2 & Alpaca-2大模型二期项目 + 64K超长上下文模型 (Chinese LLaMA-2 & Alpaca-2 LLMs with 64K long context models)

Official release of InternLM series (InternLM, InternLM2, InternLM2.5, InternLM3).

?A curated list of Awesome LLM/VLM Inference Papers with codes: WINT8/4, Flash-Attention, Paged-Attention, Parallelism, etc. ??

FlashInfer: Kernel Library for LLM Serving

MoBA: Mixture of Block Attention for Long-Context LLMs

?FFPA(Split-D): Yet another Faster Flash Prefill Attention with O(1) GPU SRAM complexity for headdim > 256, ~2x↑?vs SDPA EA.

Train llm (bloom, llama, baichuan2-7b, chatglm3-6b) with deepspeed pipeline mode. Faster than zero/zero++/fsdp.

Decoding Attention is specially optimized for MHA, MQA, GQA and MLA using CUDA core for the decoding stage of LLM inference.

Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.

Fast and memory efficient PyTorch implementation of the Perceiver with FlashAttention.

A flexible and efficient implementation of Flash Attention 2.0 for JAX, supporting multiple backends (GPU/TPU/CPU) and platforms (Triton/Pallas/JAX).

An simple pytorch implementation of Flash MultiHead Attention

Utilities for efficient fine-tuning, inference and evaluation of code generation models

? Automated deployment stack for AMD MI300 GPUs with optimized ML/DL frameworks and HPC-ready configurations

Long term project about a custom AI architecture. Consist of cutting-edge technique in machine learning such as Flash-Attention, Group-Query-Attention, ZeRO-Infinity, BitNet, etc.

MiniMax-01 is a simple implementation of the MiniMax algorithm, a widely used strategy for decision-making in two-player turn-based games like Tic-Tac-Toe. The algorithm aims to minimize the maximum possible loss for the player, making it a popular choice for developing AI opponents in various game scenarios.

? Automated installation script for vLLM on HPC systems with ROCm support, optimized for AMD MI300X GPUs.