Asynchronous Randomized Trace Estimation

Randomized trace estimation is a popular technique to approximate the trace of an implicitly-defined matrix $\mA$ by averaging the quadratic form $\vx^{\top}\mA\vx$ across several samples of a random vector $\vx$. This paper focuses on the application of randomized trace estimators on asynchronous computing environments where the quadratic form $\vx^{\top} \mA\vx$ is computed partially by observing only a random row subset of $\mA$ for each sample of the random vector $\vx$. Our asynchronous framework treats the number of rows, as well as the row subset observed for each sample, as random variables, and our theoretical analysis establishes the variance of the randomized estimator for Rademacher and Gaussian samples. We also consider an extension where the entries of $\mA$ are stochastically rounded. We also present error analysis and sampling complexity bounds for the proposed asynchronous randomized trace estimator. Our numerical experiments illustrate that the asynchronous variant can be competitive even when a small number of rows is updated per each sample.