Unsupervised learning approach to estimating user engagement with mobile applications: A case study of The Weather Company (IBM)

Abstract

User engagement (UE) is the quality of user experience that emphasizes any positive aspects of interaction with an application, and particularly the phenomena associated with being captivated by certain features included in it, thus being motivated to use it. In the context of mobile applications, measuring UE could provide insights to further explain usage behaviors, allowing developers, and product managers, to gain a better understanding of how users utilize their applications, and what drives their engagement with them. Numerous methods have been proposed in literature to measure UE in domains such as online services; however, not much had been done to model UE in the context of mobile applications. In response to this problem, our study proposes to measure UE with mobile applications by analyzing temporal changes in a defined set of usage metrics, yielding a general metric, a mobile application user engagement (MAUE) score, which is a linear combination of the UE time series metrics, accounting for the largest amount of the variance in usage data, and extracted by principal component analysis (PCA). Our proposed approach has been applied to the behavioral data of 40,004 unique users of The Weather Company mobile application. Our results indicate that time-dependent fluctuations of the MAUE score are characterized with a power-law decrease, in accordance with the power law of practice, suggesting different levels of UE stability for the different mobile platforms (i.e., IOS, Android). Additionally, the Multidimensional scaling distance between clusters of variables loadings, and among the variables loadings within each cluster with regards to the amount of usage days, could be used to map the UE motivations and thus provide product managers an improved understanding and prediction ability of the influence of different app updates and product interventions on UE.