Dynamical-Isometry-from-Orthogonality-Neural-Nets
PublicMathematical consequences of orthogonal weights initialization and regularization in deep learning. Experiments with gain-adjusted orthogonal regularizer on RNNs with SeqMNIST dataset.
Mathematical consequences of orthogonal weights initialization and regularization in deep learning. Experiments with gain-adjusted orthogonal regularizer on RNNs with SeqMNIST dataset.