Computation in its many forms is the engine that fuels our modern world. With the increasing amount of complex data availability and AI applications invading our daily life, conventional computing technologies based on Von-Neumann architectures are gradually reaching their limits in terms of performance, speed, energy efficiency, and miniaturization. A dedicated neuromorphic hardware can help overcome these issues as it is explicitly designed to support dynamic learning in the context of complex and unstructured data. The development of novel functional materials and devices incorporated into unique architectures will allow a technological leap toward the implementation of a neuromorphic computer that can efficiently implement the heavy vector - matrix manipulation inherent to AI workloads with O(1) time complexity. In this colloquium I will focus on the role of materials science in the development of various devices based on Phase Change Materials and Oxide Resistive Random Access Memories , key building blocks for the realization of the artificial neural and synaptic function in neuromorphic computing. These devices rely on diverse physical mechanisms and materials and their understanding via experimental and theoretical means is pivotal to the device optimization and coupling to the higher layers of the computer architecture.