Popular GitHub Repositories for Second Order Optimization

Distributed K-FAC Preconditioner for PyTorch

FEDL-Federated Learning algorithm using TensorFlow (Transaction on Networking 2021)

This repository implements FEDL using pytorch

Implementation of PSGD optimizer in JAX

Federated Learning using PyTorch. Second-Order for Federated Learning. (IEEE Transactions on Parallel and Distributed Systems 2022)

LIBS2ML: A Library for Scalable Second Order Machine Learning Algorithms

FOSI library for improving first order optimizers with second order information

Stochastic Second-Order Methods in JAX

A collection of second-order optimizers and experiments in JAX

sophia optimizer further projected towards flat areas of loss landscape