Popular GitHub Repositories for Cuda Kernels

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

Spiking Neural Networks in C++ with strong GPU acceleration through CUDA

Speed up image preprocess with cuda when handle image or tensorrt inference

Accelerate Multimodal AI | 3.7x faster E2E video gen and 1.6x faster E2E image gen | Custom sparse CUDA kernels 9.3x faster than FlashAttention-3 and 2.5x faster than cuBLAS

Faster Pytorch bitsandbytes 4bit fp4 nn.Linear ops

Quantum-inspired evolutionary algorithms for Optimization problems

Attention Kernels for Symmetric Power Transformers

Pytorch implementation of a message passing neural network with RNN sub-units

Code repository for ICLR 2025 paper "LeanQuant: Accurate and Scalable Large Language Model Quantization with Loss-error-aware Grid"

Spiral's Machine Learning Library

A set of Docker containers extensively used for both AI/HPC software development and deployment at Unum

Simple LLM Rest API using Rust, Warp and Candle. Dedicated for quantized version of either phi-2 ( default) , Mistral, or Llama. Work using CPU or CUDA