Since the conventional multilevel thresholding approaches exhaustively search the optimal thresholds to optimize objective functions, they are computational expensive. In this paper, the modified particle swarm optimization (MPSO) algorithm is proposed to overcome this drawback. The MPSO employs two new strategies to improve the performance of original particle swarm optimization (PSO), which are named adaptive inertia (AI) and adaptive population (AP), respectively. With the help of AI strategy, inertia weight is variable with the searching state, which helps MPSO to increase search efficiency and convergence speed. Moreover, with the help of AP strategy, the population size of MPSO is also variable with the searching state, which mainly helps the algorithm to jump out of local optima. Here, the searching state is estimated as exploration or exploitation simply according to whether the gBest has been updated in (Formula presented.) consecutive generations or not, where the gBest stands for the position with the best fitness found so far among all the particles in the swarm. The MPSO has been evaluated on 12 unimodal and multimodal Benchmark functions, and the effects of AI and AP strategies are studied. The results show that MPSO improves the performance of the PSO paradigm. The MPSO is also used to find the optimal thresholds by maximizing the Otsu’s objective function, and its performance has been validated on 16 standard test images. The experimental results of 30 independent runs illustrate the better solution quality of MPSO when compared with the global particle swarm optimization and standard genetic algorithm.