In Huntington's disease (HD), accurate estimates of expected future motor impairments are key for clinical trials. Individual prognosis is only partially explained by genetics. However, studies so far have focused on predicting the time to clinical diagnosis based on fixed impairment levels, as opposed to predicting impairment in time windows comparable to the duration of a clinical trial. Here we evaluate an approach to both detect atrophy patterns associated with early degeneration and provide a prognosis of motor impairment within 3 years, using data from the TRACK-HD study on 80 premanifest HD (pre-HD) individuals and 85 age- and sex-matched healthy controls. We integrate anatomical MRI information from gray matter concentrations (estimated via voxel-based morphometry) together with baseline data from demographic, genetic and motor domains to distinguish individuals at high risk of developing pronounced future motor impairment from those at low risk. We evaluate the ability of models to distinguish between these two groups solely using baseline imaging data, as well as in combination with longitudinal imaging or non-imaging data. Our models show improved performance for motor prognosis through the incorporation of imaging features to non-imaging data, reaching 88% cross-validated accuracy when using baseline non-longitudinal information, and detect informative correlates in the caudate nucleus and the thalamus both for motor prognosis and early atrophy detection. These results show the plausibility of using baseline imaging and basic demographic/genetic measures for early detection of individuals at high risk of severe future motor impairment in relatively short timeframes.