The increasing deployment of distribution-level phasor measurement units (PMUs) calls for dynamic distribution state estimation (DDSE) approaches that tap into high-rate measurements to maintain a comprehensive view of the distribution-system state in real time. Accordingly, this paper explores the development of a fast algorithmic framework by casting the DDSE task within the time-varying optimization realm. The time-varying formulation involves a time-varying robustified least-squares approach, and it naturally models optimal trajectories for the estimated states under streaming of measurements. The formulation is based on a linear surrogate of the AC power-flow equations, and it includes an element of robustness with respect to measurement outliers. The paper then leverages a first-order prediction-correction method to achieve simple online updates that can provably track the state variables from heterogeneous measurements. This online algorithm is computationally efficient as it relies on the Hessian of the cost function without computing matrix-inverse. Convergence and bounds on the estimation errors of proposed algorithm can be analytically established.