The emergence of noisy medium-scale quantum devices has led to proof-of-concept applications for quantum computing in various domains. Examples include Natural Language Processing (NLP) where sentence classification experiments have been carried out, as well as procedural generation, where tasks such as geopolitical map creation, and image manipulation have been performed. We explore applications at the intersection of these two areas by designing a hybrid quantum-classical algorithm for sentence generation. Our algorithm is based on the well-known simulated annealing technique for combinatorial optimisation. An implementation is provided, and used to demonstrate successful sentence generation on real quantum hardware. Finally, we discuss how a variant of our algorithm can be used for music generation, and compare its performance to the recently proposed Quanthoven toolkit for music composition. This paper aims to be self-contained, introducing all the necessary background on NLP and quantum computing along the way.