The “zero trust” cybersecurity concept was initially discussed in network-based frameworks, which requires to “never trust, always verify” each network tenant or user. The needs of adopting this concept have been rapidly increasing in recent years since the COVID-19 pandemic significantly accelerated digital transformation and increased size of the global hybrid-cloud market. However, in today’s environments, only focusing on network is not sufficient to achieve zero trust. In fact, zero trust computing in public clouds requires to verify all the software and hardware components in systems, namely from the user application/interface all the way down to the hardware chips, which brings many research questions. For instance, what will be root of trust (RoT) in the zero trust architecture, and how can it help build a trust chain in the whole system? Many compute-intensive applications, such as data-driven artificial intelligence (AI) computations, need to be executed in CPU-accelerator heterogeneous systems. Will such computations raise complexity of implementing a zero trust architecture? We believe that there are many more questions to be answered, which will bring many research opportunities for both academia and industry. In this talk, we will primarily focus on current challenges and opportunities to achieve zero trust on hardware architectures. Then, we will elaborate on a few topics such as building hardware-based platform RoT and trusted execution environments (TEEs) for accelerators.