Popular GitHub Repositories for Edge Ai

FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.

Olares: An Open-Source Sovereign Cloud OS for Local AI

[ICLR 2020] Once for All: Train One Network and Specialize it for Efficient Deployment

Deep learning gateway on Raspberry Pi and other edge devices

speech to text benchmark framework

AI-in-a-Box leverages the expertise of Microsoft across the globe to develop and provide AI and ML solutions to the technical community. Our intent is to present a curated collection of solution accelerators that can help engineers establish their AI/ML environments and solutions rapidly and with minimal friction.

The Hailo Model Zoo includes pre-trained models and a full building and evaluation environment

Model Compression Toolkit (MCT) is an open source project for neural network model optimization under efficient, constrained hardware. This project provides researchers, developers, and engineers advanced quantization and compression tools for deploying state-of-the-art neural networks.

A DNN inference latency prediction toolkit for accurately modeling and predicting the latency on diverse edge devices.

? The best free Telegram bot for ChatGPT, Microsoft Copilot (aka Bing AI / Sidney / EdgeGPT), Microsoft Copilot Designer (aka BingImageCreator), Gemini and Groq with stream writing, requests with images, multiple languages, admin control, data logging and more!

Optimized local inference for LLMs with HuggingFace-like APIs for quantization, vision/language models, multimodal agents, speech, vector DB, and RAG.

TPI-LLM: Serving 70b-scale LLMs Efficiently on Low-resource Edge Devices

FEDn: An enterprise-ready open source federated learning framework. This repository contains the Python framework, CLI and API.

High-performance, optimized pre-trained template AI application pipelines for systems using Hailo devices

An open source light-weight and high performance inference framework for Hailo devices

Demo for training a convolutional neural network to classify words and deploy the model to a Raspberry Pi using TensorFlow Lite.

Create a smart video application using the Intel Distribution of OpenVINO toolkit. The toolkit uses models and inference to run single-class object detection.

Light Face Detection using PyTorch Lightning

EdgeFace: Efficient Face Recognition Model for Edge Devices [TBIOM 2024] the winner of compact track of IJCB 2023 Efficient Face Recognition Competition

Resources of our survey paper "Optimizing Edge AI: A Comprehensive Survey on Data, Model, and System Strategies"