Health monitoring technology in everyday situations is expected to improve quality of life and support aging populations. Mental fatigue among health indicators of individuals has become important due to its association with cognitive performance and health outcomes, especially in older adults. Previous models using eye-tracking measures allow inference of fatigue during cognitive tasks, such as driving, but they require us to engage in specific cognitive tasks. In addition, previous models were mainly tested by user groups that did not include older adults, although age-related changes in eye-tracking measures have been reported especially in older adults. Here, we propose a model to detect mental fatigue of younger and older adults in natural viewing situations. Our model includes two unique aspects: (i) novel feature sets to better capture fatigue in natural-viewing situations and (ii) an automated feature selection method to select a feature subset enabling the model to be robust to the target's age. To test our model, we collected eye-tracking data from younger and older adults as they watched video clips before and after performing cognitive tasks. Our model improved detection accuracy by up to 13.9% compared with a model based on the previous studies, achieving 91.0% accuracy (chance 50%).