The combined challenges of a high fire risk and poor vehicle access in mountainous regions have led planners and emergency management authorities to consider non-traditional alternatives to complete evacuation of a region under threat. Community fire shelters have been put forward as one such alternative; however, their benefits remain contested. In this paper a series of experiments are designed in the Dandenong Ranges in Australia and presented to elucidate the relationship between shelters and community safety in wildfire scenarios. Our approach utilises a multi-stage simulation workflow to evaluate the outcome of 64 shelter configurations subject to three different fires. When compared to a scenario without shelters, some shelter configurations result in up to 10% reduction in the median exposure count, while some other configurations increase it. It is found that the efficacy of shelters strongly depends on the relative location to the ignition point and the trend of fire progression. The most effective shelters are identified for the specific fires that we simulate. The results demonstrate that sophisticated modelling and simulation is necessary for decision makers to determine a beneficial shelter placement strategy that remains effective across a number of likely wildfire spread scenarios.