Cryptography plays a major role assuring security in computation and communication. In particular, public-key cryptography enables the asymmetrical ciphering of data along with the authentication of the parties that are attempting to share data. The computation of asymmetrical encryption is costly, thus it has motivated extensive research to efficiently accelerate the execution of the most relevant algorithms and improve resistance against Side-Channel Attacks (SCAs), which leverage exposed features by the cryptographic systems, such as power consumption and execution timings, to gain access to private information. Herein, we present a state-of-the-art overview of the use of the Residue Number System (RNS) to exploit parallelism in the computation of the most important public-key algorithms. We also address how it can be exploited to prevent side-channel attacks. The experimental results presented in the literature show that not only the currently used RSA and Elliptic Curve Cryptographic (ECC) algorithms but also emerging postquantum algorithms, namely the ones supporting Lattice-based Cryptosystems (LBCs), can take advantage of the RNS. It enables the design of more efficient cryptographic systems and also reinforces the prevention of side-channel attacks, improving their security. Finally, we also present the characteristics of the Computing with the Residue Number System Framework (CRNS), which aims to automatize the design of fully functional cryptographic accelerators based on RNS.