Superior stability and safety are key promises attributed to all-solid-state batteries (ASSBs) containing solid-state electrolyte (SSE) compared to their conventional counterparts utilizing liquid electrolyte. To unleash the full potential of ASSBs, SSE materials that are stable when in contact with the low and high potential electrodes are required. The electrochemical stability window is conveniently used to assess the SSE-electrode interface stability. In the present work, we review the most important methods to compute the SSE stability window. Our analysis reveals that the stoichiometry stability method represents a bridge between the HOMO-LUMO method and the phase stability method (grand canonical phase diagram). Moreover, we provide computational implementations of these methods for SSE material screening. We compare their results for the relevant Li- A nd Na-SSE materials LGPS, LIPON, LLZO, LLTO, LATP, LISICON, and NASICON, and we discuss their relation to published experimental stability windows.