Probabilistic bits (p-bits) based on magnetic tunnel junctions are of recent interest in probabilistic and neuromorphic computing architectures based on their small size, high operating speeds, and truly stochastic nature. In practical systems, the output probability of the bit can be tuned by an applied current, which is generally characterized by a quasi-static tuning curve. In this letter, we instead focus on the finite time it takes the p-bit's probabilistic distribution to respond to an applied bias current. We find that this settling time is in the range of hundreds of picoseconds for a typical junction, and is highly dependent on various parameters, including the device size, material properties, and magnitude of the applied current. These results provide a baseline understanding of the dynamic properties of a nanomagnetic p-bit's probability distribution, which is helpful for p-bit-related system architecture discussions.