Quantum gate error rates have dropped steadily on IBM processors over the last years, making more sophisticated computations possible, but also increasing the challenge of characterizing the remaining errors. In particular, non-Markovian errors have been thought likely to limit gates on IBM systems, but directly detecting these errors has been elusive, limiting our ability to mitigate them efficiently. We probe our system for off-resonant excitations, a specific form of non-Markovian coherent error, using continuous phase amplification. Since errors due to excitations of an off-resonant transition are non-stationary they are hard to amplify and detect without using this technique. These errors exist in all microwave driven one- and two-qubit gates on IBM devices and can bottleneck performance near collisions (e.g. driving the cross-resonance interaction during single qubit gates or vice versa). Furthermore we can use this same methodology to access to the errors caused by TLS, showing evidence of coherent, off-resonant interactions with subsystems that are not qubits. We explore these results and their impact on gate error for IBM deployed devices. *Research was sponsored by the Army Research Office and was accomplished under Grant Number W911NF-21-1-0002. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Office or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.