tldr- A slight change in SGD formulation, in terms of maximization of local approximation, leads to an interesting general connection to NGD via mirror descent.

TL;DR - Implicit differentiation can lead to an efficient computation of the gradient of reparametrized samples.

TL;DR - The Exponential family provides an elegant and easy method to compute Natural Gradients and thus can be used for Variational Inference.

TL;DR- Code snippets for various Lipschitz Regularization methods for WGAN - Gradient Clipping, Gradient Penalty, Spectral Normalization etc. in PyTorch.

TL;DR- Another trick for interchanging the gradient and Expectation of a function under the Gaussian distribution.