Abstract
The Weather Research and Forecast (WRF) model is a model of the atmosphere for mesoscale research and operational numerical weather prediction (NWP). A petascale problem for WRF is a nature run that provides very high-resolution truth against which more coarse simulations or perturbation runs may be com-pared for purposes of studying predictability, stochastic parameterization, and fundamental dynamics. We carried out a nature run involving an idealized high resolution rotating fluid on the hemisphere, at a size and resolution never before attempted, and used it to investigate scales that span the k-3 to k-5/3 kinetic energy spectral transition, via simulations. We used up to 15,360 processors of the New York Blue IBM BG/L machine at Stony Brook Uni-versity and Brookhaven National Laboratory. The grid we employed has 4486 by 4486 horizontal grid points and 101 vertical levels (2 billion cells) at 5km resolution; this is 32 times larger than the previously largest 63 million cell 2.5km resolution WRF CONUS benchmark [10]). To solve a problem of this size, we worked through issues of parallel I/O and scalability and employed more processors than have ever been used in a WRF run. We achieved a sustained 3.4 Tflop/s on the New York Blue sys-tem, inputting and then generating an enormous amount of data to produce a scientifically meaningful result. More than 200 GB of data was input to initialize the run, which then generated output datasets of 40 GB each simulated hour. The cost of output was considered a key component of our investigation. Then we ran the same problem on more than 12K processors of the XT4 system at NERSC and achieved 8.8 Tflop/s. Our primary result however is not just scalability and a high Tflop/s number, but capture of atmosphere features never before represented by simulation, and taking an important step towards understanding weather predict-ability at high resolution. © 2008 IOP Publishing Ltd.