MSRA-USTC-AISystemProject-2021

Making large AI models cheaper, faster and more accessible

Port of OpenAI's Whisper model in C/C++

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

ncnn is a high-performance neural network inference framework optimized for the mobile platform

SGLang is a fast serving framework for large language models and vision language models.

? The exhaustive Pattern Matching library for TypeScript, with smart type inference.

Example ? Jupyter notebooks that demonstrate how to build, train, and deploy machine learning models using ? Amazon SageMaker.

concurrent, cache-efficient, and Dockerfile-agnostic builder toolkit

Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.