Popular GitHub Repositories for Triton

Efficient Triton Kernels for LLM Training

Kernl lets you run PyTorch transformer models several times faster on GPU with a single line of code, and is designed to be easily hackable.

Quantized Attention that achieves speedups of 2.1-3.1x and 2.7-5.1x compared to FlashAttention2 and xformers, respectively, without lossing end-to-end metrics across various models.

???A collection of some wesome public projects about Large Language Model(LLM), Vision Language Model(VLM), Vision Language Action(VLA), AI Generated Content(AIGC), the related Datasets and Applications.

A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.

??? This repository lists some awesome public CUDA, cuda-python, cuBLAS, cuDNN, CUTLASS, TensorRT, TensorRT-LLM, Triton, TVM, MLIR, PTX and High Performance Computing (HPC) projects.

OpenDILab RL HPC OP Lib, including CUDA and Triton kernel

ClearML - Model-Serving Orchestration and Repository Solution

NVIDIA-accelerated, deep learned model support for image space object detection

Training-free Post-training Efficient Sub-quadratic Complexity Attention. Implemented with OpenAI Triton.

NVIDIA-accelerated DNN model inference ROS 2 packages using NVIDIA Triton/TensorRT for both Jetson and x86_64 with CUDA-capable GPU

Three examples of recommendation system pipelines with NVIDIA Merlin and Redis

Increase the inference speed of the model

Deploy KoGPT with Triton Inference Server

TensorFlow Lite backend with ArmNN delegate support for Nvidia Triton

Optimize, convert and deploy machine learning models as fast inference API using Triton and ORT. Currently support Hugging Face transformers, PyToch, Tensorflow, SKLearn and XGBoost models.

A FlashAttention backwards-over-backwards ⚡??

Example of deployment Pytorch model into the Triton inference server via MLFlow model registry

MLModelService wrapping Nvidia's Triton Server

Framework, Model & Kernel Optimizations for Distributed Deep Learning - Data Hack Summit