With the proliferation of experimental techniques able to probe properties of complex materials in non-equilibrium with increasing accuracy and resolution, and interesting properties such as hidden phases continuing to garner interest, there is a clear need for accurate theoretical techniques to describe complex non-equilibrium processes. Few exact, many-body theoretical solutions exist in non-equilibrium, and approximations are very important. Sum rules, which relate integrals of the spectral function to expectation values of observables, may represent a self-consistent way to check the accuracy of non-equilibrium calculations. And simple problems which can be solved exactly in non-equilibrium may provide insight on processes which may occur in more complicated non-equilibrium systems. We calculate the first three spectral moment sum rules for a general non-equilibrium Holstein-Hubbard model. These moments can be verified exactly in the atomic limit, where an exact Green's function can be obtained. The behavior of this simpler Green's function's photoemission spectrum in non-equilibrium leads us to propose a measure, the first moment of the photoemission spectrum, which may be useful in tracking non-equilibrium changes in electron-electron or electron-phonon couplings in systems with sufficiently separated energy bands. *This work was supported by the Department of Energy, Office of Basic Energy Sciences, Division of Materials Sciences and Engineering under Contract No. DE-SC0019126. J.K.F. was also supported by the McDevitt bequest at Georgetown.