Collaborative Filtering (CF) is widely applied to personalized recommendation systems. Traditional collaborative filtering techniques make predictions through a user-item matrix of ratings which explicitly presents user preference. With the increasingly growing number of users and items, insufficient rating data still leads to the decreasing predictive accuracy with traditional collaborative filtering approaches. In the real world, however, many different types of user feedback, e.g. review, like or not, votes etc., co-exist in many online content providers. In this paper we integrate rating data with some other new types of user feedback and propose a multi-task matrix factorization model in order for flexibly using multiple data. We use a common user feature space shared across sub-models in this model and thus the model can simultaneously train the corresponding sub-models with every training sample. Our experiments indicate that new types of user feedback really work and show improvements on predictive accuracy compared to state-of-the-art algorithms.