transformer-perf-estimates

Making large AI models cheaper, faster and more accessible

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Official PyTorch implementation of PTS/PSRN: Fast and efficient symbolic expression discovery through parallelized tree search. Evaluates millions of expressions simultaneously on GPU with automated subtree reuse.

A GPipe implementation in PyTorch

LiBai(李白): A Toolbox for Large-Scale Distributed Parallel Training

Slicing a PyTorch Tensor Into Parallel Shards

Easy Parallel Library (EPL) is a general and efficient deep learning framework for distributed model training.

? A Collection of Awesome Large Weather Models (LWMs) | AI for Earth (AI4Earth) | AI for Science (AI4Science)

A curated list of awesome projects and papers for distributed training or inference

Automated Hypothesis Testing with Agentic Sequential Falsifications