Polynomial convergence of iterations of certain random operators in Hilbert space

Abstract

We study the convergence of a random iterative sequence of a family of operators on infinite-dimensional Hilbert spaces, inspired by the stochastic gradient descent (SGD) algorithm in the case of the noiseless regression. We identify conditions that are strictly broader than previously known for polynomial convergence rate in various norms, and characterize the roles the randomness plays in determining the best multiplicative constants. Additionally, we prove almost sure convergence of the sequence.