TD-Gammon, A Self-Teaching Backgammon Program, Achieves Master-Level Play

Abstract

TD-Gammonis a neural network that is able to teach itself to play backgammosno lely by playing against itself and learning from the results, based on the TD(λ) reinforcement learning algorithm (Sutton, 1988). Despite starting from random initial weights (and hence random initial strategy), TD-Gammoanc hieves a surprisingly strong level of play. With zero knowledge built in at the start of learning (i.e. given only a "raw"description of the board state), the network learns to play at a strong intermediate level. Furthermore, when a set of hand-crafted features is added to the network's input representation, the result is a truly staggering level of performance: the latest version of TD-Gammoisn now estimated to play at a strong master level that is extremely close to the world's best human players.