Matrix approximation (MA) methods are popular in recommendation tasks on explicit feedback data. However, in many real-world applications, only positive feedbacks are explicitly given whereas negative feedbacks are missing or unknown, i.e., implicit feedback data, and standard MA methods will be unstable due to incomplete positive feedbacks and inaccurate negative feedbacks. This paper proposes a stable matrix approximation method, namely StaMA, which can improve the recommendation accuracy of matrix approximation methods on implicit feedback data through dynamic weighting during model learning. We theoretically prove that StaMA can achieve sharper uniform stability bound, i.e., better generalization performance, on implicit feedback data than MA methods without weighting. Meanwhile, experimental study on real-world datasets demonstrate that StaMA can achieve better recommendation accuracy compared with five baseline MA methods in top-N recommendation task.