Abstract
We develop a new efficient method for high-dimensional sampling called
Subspace Langevin Monte Carlo. The primary application of these methods is to
efficiently implement Preconditioned Langevin Monte Carlo. To demonstrate the
usefulness of this new method, we extend ideas from subspace descent methods in
Euclidean space to solving a specific optimization problem over Wasserstein
space. Our theoretical analysis demonstrates the advantageous convergence
regimes of the proposed method, which depend on relative conditioning
assumptions common to mirror descent methods. We back up our theory with
experimental evidence on sampling from an ill-conditioned Gaussian
distribution.