Solar power forecasting improvements are mainly evaluated by statistical and economic metrics, and the practical reliability benefits of these forecasting enhancements have not yet been well quantified. This paper aims to quantify reliability benefits from solar power forecasting improvements. To systematically analyze the relationship between solar power forecasting improvements and reliability performance in power system operations, an expected synthetic reliability (ESR) metric is proposed to integrate multiple state-of-the-art independent reliability metrics. The absolute value and standard deviation of area control errors (ACEs), and the North American Electric Reliability Corporation Control Performance Standard 2 (CPS2) score are calculated through a multi-timescale scheduling simulation, including the day-ahead unit commitment, real-time unit commitment, real-time economic dispatch, and automatic generation control sub-models. The absolute ACE in energy, CPS2 violations, CPS2 score, and standard deviation of the raw ACE are all calculated and combined as the ESR metric. Numerical simulations show that the reliability benefits of multi-timescale power system operations are significantly increased due to the improved solar power forecasts.