The Dubuque Water Portal is a system aimed at supporting voluntary reductions of water consumption that is intended to be deployed city-wide. It provides each household with fine-grained, near real time feedback on their water consumption, as well as using techniques like social comparison, weekly games, and news and chat to encourage water conservation. This study used logs, a survey and interviews to evaluate a 15-week pilot with 303 households. It describes the Portal's design, and discusses its adoption, use and impacts. The system resulted in a 6.6% decrease in water consumption, and the paper employs qualitative methods to look at the ways in which the Portal was (or wasn't) effective in supporting its users and enabling them to reduce their consumption. The paper concludes with a discussion of design implications for residential feedback systems, and possible engagement models. Copyright 2012 ACM.