Nanoscale resistive memory devices are being explored for neuromorphic and in-memory computing. However, non-ideal device characteristics of read noise and resistance drift pose significant challenges to the achievable computational precision. Here, it is shown that there is an additional non-ideality that can impact computational precision, namely the bias-polarity-dependent current flow. Using phase-change memory (PCM) as a model system, it is shown that this “current–voltage” non-ideality arises both from the material and geometrical properties of the devices. Further, we discuss the detrimental effects of such bipolar asymmetry on in-memory matrix-vector multiply (MVM) operations and provide a scheme to compensate for it.