BentoOCR

A high-throughput and memory-efficient inference and serving engine for LLMs

Free MLOps course from DataTalks.Club

Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

Simple and Distributed Machine Learning

RAG (Retrieval Augmented Generation) Framework for building modular, open source applications for production by TrueFoundry

FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

? Awesome System for Machine Learning ⚡️ AI System Papers and Industry Practice. ⚡️ System for Machine Learning, LLM (Large Language Model), GenAI (Generative AI). ? OSDI, NSDI, SIGCOMM, SoCC, MLSys, etc. ?️ Llama3, Mistral, etc. ?‍? Video Tutorials.