Popular GitHub Repositories for Int8

An innovative library for efficient LLM inference via low-bit quantization

tensorrt int8 量化yolov5 onnx模型

a simple pipline of int8 quantization based on tensorrt.

? Apply YOLOv8 exported with ONNX or TensorRT(FP16, INT8) to the Real-time camera

TensorRT Int8 Python version sample. TensorRT Int8 Python 实现例子。TensorRT Int8 Pythonの例です

A LLaMA2-7b chatbot with memory running on CPU, and optimized using smooth quantization, 4-bit quantization or Intel? Extension For PyTorch with bfloat16.

INT8 calibrator for ONNX model with dynamic batch_size at the input and NMS module at the output. C++ Implementation.

quantization example for pqt & qat

LLM-Lora-PEFT_accumulate explores optimizations for Large Language Models (LLMs) using PEFT, LORA, and QLORA. Contribute experiments and implementations to enhance LLM efficiency. Join discussions and push the boundaries of LLM optimization. Let's make LLMs more efficient together!

VB.NET api wrapper for llm-inference chatllm.cpp

C# api wrapper for llm-inference chatllm.cpp

mxnet GluonCV quantization binary ternary models