The Variational Quantum Eigensolver (VQE) is a quantum algorithm that has emerged as a leading candidate to address quantum ground-state problems on noisy-intermediate scale quantum devices. An important subroutine in VQE involves performing a large number of quantum measurements to construct estimates of the expectation value of a Hamiltonian. Recent efforts have been directed towards measurement procedures based on randomized estimators, with the goal of minimizing the number of measurements required to achieve a given error in the expectation value estimate, compared to non-randomized standard estimators. However, a comparison of the performance of these procedures on real noisy quantum hardware is lacking. To this end, we present benchmark results of recent randomized estimators proposed theoretically. We first carry out a finite-sample analysis of these procedures backed by numerical experiments on an ideal simulator. We then analyze their noise-resiliency through numerical experiments on a noisy simulator with different readout error mitigation techniques. Finally, we test their performance on an IBM Quantum device on a variety of molecular Hamiltonians of increasing size. Further, we propose a novel procedure based on derandomization with prior quantum state information.