We present a nanoparticle size-separation device based on a nanofluidic rocking Brownian motor. It features a ratchet-shaped electrostatic particle potential with increasing barrier heights along the particle transport direction. The sharp drop of the particle current with barrier height is exploited to separate a particle suspension into multiple subpopulations. By solving the Fokker-Planck equation, we show that the physics of the separation mechanism is governed by the energy landscape under forward tilt of the ratchet. For a given device geometry and sorting duration, the applied force is thus the only tunable parameter to increase the separation resolution. For the experimental conditions of 3.5 V applied voltage and 20 s sorting, we predict a separation resolution of approximately 2 nm, supported by experimental data for separating spherical gold particles of nominal diameters of 80 and 100 nm.