Popular GitHub Repositories for Distributed Optimization

Fair Resource Allocation in Federated Learning (ICLR '20)

FedTorch is a generic repository for benchmarking different federated and distributed learning algorithms using PyTorch Distributed API.

FedDANE: A Federated Newton-Type Method (Asilomar Conference on Signals, Systems, and Computers ‘19)

Scalable, structured, dynamically-scheduled hyperparameter optimization.

We present a set of all-reduce compatible gradient compression algorithms which significantly reduce the communication overhead while maintaining the performance of vanilla SGD. We empirically evaluate the performance of the compression methods by training deep neural networks on the CIFAR10 dataset.

We present an algorithm to dynamically adjust the data assigned for each worker at every epoch during the training in a heterogeneous cluster. We empirically evaluate the performance of the dynamic partitioning by training deep neural networks on the CIFAR10 dataset.

Decentralized Sporadic Federated Learning: A Unified Algorithmic Framework with Convergence Guarantees

Implemented FedAvg & FedProx: Decentralized Optimization Algorithms for Neural Networks for an Image Classification Task- Distributed Optimization and Learning(DOL) Course Project

EF-BV: A Unified Theory of Error Feedback and Variance Reduction Mechanisms for Biased and Unbiased Compression in Distributed Optimization. NeurIPS, 2022

Error feedback based quantization aided and convergence guaranteed Communication Efficient Federated Linear and Deep GCCA