Hierarchical weather generators (WGs) are commonly used to generate high-resolution precipitation fields for climate impact assessments. The stochastic nature of WGs also provide realistic representation of uncertainty given some independent climate information (e.g., the large-scale atmospheric state, annual precipitation amount, etc.). While several WGs exist in the literature which differ both in design configuration and complexity, their performance is typically assessed using single case studies. More specifically, their performance remains largely unknown in geographic regions and climates different from the ones used to develop them. Similarly, performance of WGs relative to other WG configurations is rarely evaluated. The lack of a systematic comparison limits proper assessment of these tools and wider uptake—especially by the commercial industry. To this end, we undertake an assessment of the design implications of two state-of-the-art WGs— one conditioned on annual forecasts of rainfall, the other conditioned on AI-based weather regimes. For the standard formulations of these WGs, as reported in Guevara et al (2021), we systematically vary the components in the hierarchical structure and evaluate the resulting simulations with respect to their ability to simulate climate characteristics present in the historical time series (daily averages, skewness, dry spells, and extreme quantiles). Our results provide insight into the implications of design choices of WGs and quantify/justify the benefits of additional complexity in WGs.