Improving the accuracy of sub-seasonal to seasonal (S2S) extremes can significantly impact society. Providing S2S forecasts in the form of risk or extreme indices can aid in disaster response, especially for drought and flood events. Additionally, it can provide updates on disease outbreaks and aid in predicting the occurrence, duration, and decline of heatwaves. This work uses a transformer model to predict the daily temperature distributions in the S2S scale. We analyze how the model performs in extreme temperatures by comparing its output distributions with those obtained from ECMWF forecasts across different metrics. Our model produces better responses for temperatures in average and extreme regions. Also, we show how our model better captures the heatwave that hit Europe in the summer of 2019.