Cognitive computing describes systems that learn at scale, reason with purpose, and interact with humans naturally . In this paper, we review our work towards enabling next generation cognitive computing using neuromorphic computational schemes that could potentially outperform present-day CPUs and GPUs. Here we use large arrays of Resistive Non-Volatile Memories (NVM) with device conductance serving as synaptic weight. We focus on training and classification using fully-connected networks based on the backpropagation algorithm, and show that our approach could offer power and speed advantages over conventional Von-Neumann processors. We also propose some circuit approximations that improve network parallelism without significantly degrading classification accuracy. Finally, we explore the requirements for a system implementation of on-chip learning.