Popular GitHub Repositories for Fsdp

??? distributed training & serverless inference at scale on RunPod

Fast and easy distributed model training examples.

A script for training the ConvNextV2 on CIFAR10 dataset using the FSDP technique for a distributed training scheme.

Minimal yet high performant code for pretraining llms. Attempts to implement some SOTA features. Implements training through: Deepspeed, Megatron-LM, and FSDP. WIP

Framework, Model & Kernel Optimizations for Distributed Deep Learning - Data Hack Summit