PyHessian is a Pytorch library for second-order based analysis and training of Neural Networks

ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning

Pytorch implementation of preconditioned stochastic gradient descent (Kron and affine preconditioner, low-rank approximation preconditioner and more)

A C++ interface to formulate and solve linear, quadratic and second order cone problems.

An implementation of PSGD Kron second-order optimizer for PyTorch

Distributed K-FAC preconditioner for PyTorch

FEDL-Federated Learning algorithm using TensorFlow (Transaction on Networking 2021)

This repository implements FEDL using pytorch

Implementation of PSGD optimizer in JAX

PyTorch implementation of the Hessian-free optimizer

Tensorflow implementation of preconditioned stochastic gradient descent

This package is dedicated to high-order optimization methods. All the methods can be used similarly to standard PyTorch optimizers.

Hessian-based stochastic optimization in TensorFlow and keras

Compatible Intrinsic Triangulations (SIGGRAPH 2022)

A canon of deep learning optimizers.

Federated Learning using PyTorch. Second-Order for Federated Learning. (IEEE Transactions on Parallel and Distributed Systems 2022)

Modular optimization library for PyTorch (work-in-progress).

LIBS2ML: A Library for Scalable Second Order Machine Learning Algorithms

Minimal deep learning library with first and second-order optimization algorithms made for educational purpose