Popular GitHub Repositories for Tensorrt Llm

?A curated list of Awesome LLM/VLM Inference Papers with codes: WINT8/4, Flash-Attention, Paged-Attention, Parallelism, etc. ??

A nearly-live implementation of OpenAI's Whisper.

MiniMax-01 is a simple implementation of the MiniMax algorithm, a widely used strategy for decision-making in two-player turn-based games like Tic-Tac-Toe. The algorithm aims to minimize the maximum possible loss for the player, making it a popular choice for developing AI opponents in various game scenarios.

An Optimized Speech-to-Text Pipeline for the Whisper Model Supporting Multiple Inference Engine

??? This repository lists some awesome public CUDA, cuda-python, cuBLAS, cuDNN, CUTLASS, TensorRT, TensorRT-LLM, Triton, TVM, MLIR, PTX and High Performance Computing (HPC) projects.

OpenAI compatible API for TensorRT LLM triton backend

Deep Learning Deployment Framework: Supports tf/torch/trt/trtllm/vllm and other NN frameworks. Support dynamic batching, and streaming modes. It is dual-language compatible with Python and C++, offering scalability, extensibility, and high performance. It helps users quickly deploy models and provide services through HTTP/RPC interfaces.

Higher performance OpenAI LLM service than vLLM serve: A pure C++ high-performance OpenAI LLM service implemented with GPRS+TensorRT-LLM+Tokenizers.cpp, supporting chat and function call, AI agents, distributed multi-GPU inference, multimodal capabilities, and a Gradio chat interface.

This repository is an AI Bootcamp material that consist of a workflow for LLM

Chat With RTX Python API

TensorRT-LLM server with Structured Outputs (JSON) built with Rust

AI Infra LLM infer/ tensorrt-llm/ vllm

大模型推理框架加速，让 LLM 飞起来

Getting started with TensorRT-LLM using BLOOM as a case study

Whisper in TensorRT-LLM

A simple project demonstrating LLM assisted review of documentation on Atlasssian Confluence.