Popular GitHub Repositories for Npu

Tengine is a lite, high performance, modular inference engine for embedded device

Efficient Inference of Transformer models

Free TPU for FPGA with compiler supporting Pytorch/Caffe/Darknet/NCNN. An AI processor for using Xilinx FPGA to solve image classification, detection, and segmentation problem.

Samples code for world class Artificial Intelligence SoCs for computer vision applications.

LLM SDK for OnnxRuntime GenAI (OGA)

Easy usage of Rockchip's NPUs found in RK3588 and similar chips

FREE TPU V3plus for FPGA is the free version of a commercial AI processor (EEP-TPU) for Deep Learning EDGE Inference

Ollama alternative for Rockchip NPU: An efficient solution for running AI and Deep learning models on Rockchip devices with optimized NPU support ( rkllm )

hardware design of universal NPU(CNN accelerator) for various convolution neural network

ONNXim is a fast cycle-level simulator that can model multi-core NPUs for DNN inference

High-speed and easy-use LLM serving framework for local deployment

Simplified AI runtime integration for mobile app development

EmbeddedLLM: API server for Embedded Device Deployment. Currently support CUDA/OpenVINO/IpexLLM/DirectML/CPU

Small Heterogeneous & AI Powered Computing SBC Based on V853

? A seamless integration of HuggingFace Transformers & Diffusers with RBLN SDK for efficient inference on RBLN NPUs.

Whisper server based on useful-transformers for the RK3588

A simple Python script for running LLMs on Intel's Neural Processing Units (NPUs)

A python vlc player that transcribes subtitles on the Intel NPU

Running Large Language Model easily.