Scanning tunneling microscopy (STM) was employed to measure the three-dimensional surface topography of radial servo grooves etched in a glass substrate. The effect of groove depth (nominal values of 20, 40, and 70 nm) on tracking servo signals at a track pitch of 1.6 μm is examined. Using an optical disk tester, three groove related signals were measured for each groove depth: the on-track signal ratio, cross-track ratio, and the push-pull ratio. In a key feature of this study, scalar diffraction theory was used to calculate the groove related signals from the STM profiles and the beam profile of the optical disk tester. The calculated groove related signals were found to be in nearly quantitative agreement with those measured with the disk tester.