The possibility that the stellar initial mass function (IMF) arises mostly from cloud structure is investigated with fractal Brownian motion (fBm) clouds that have power-law power spectra. An fBm cloud with a realistic projected power spectrum slope of β = 2.8 is found to have a mass function for clumps exceeding a threshold density that is a power law with a slope of α= 2.35, the same as in the Salpeter IMF. Any hierarchically structured cloud has a clump mass function with about the same slope. This result implies that turbulent interstellar clouds produce dense substructure with the observed pre-stellar core mass function built in from the start. Details of the clump formation processes are not critical. The conversion of clumps into stars involves a second step. A one-to-one correspondence between clump mass and star mass is not necessary to convert the clump mass spectrum into an IMF with the same power-law slope. As long as clumps have an internal stellar IMF from sub-fragmentation, protostellar accretion, coalescence and other processes, and the characteristic mass for this internal IMF scales with the clump mass, then the IMF slope above the minimum characteristic mass will equal the clump mass slope. A detailed review of IMF models illustrates the prominence of cloud structure as a major component in a wide class of theories. Tests are proposed to determine the relative importance of cloud structure and competitive accretion in the IMF. © 2010 The Authors. Journal compilation © 2010 RAS.