IoT and distributed machine learning powered optimal state recommender solution

Abstract

Recommender systems add significant benefits to E-commerce in terms of sale conversion, revenues, customer experience, loyalty and lifetime value. But the recommendations from these systems do not change on inputs beyond user and item profile and transaction data. There have been some attempts in the past to optimize on more varied data in recommenders, example of which is the location based recommenders. But location is just one dimension of the state that a user could have shared with GPS/GLONASS/BaiDeu sensor available in most Smartphones. With an upcoming era of Smart-wears and pervasive IoTs, there are a lot many other dimensions of a user state which can be utilized to optimize upon the concept of Optimal State Recommender Solutions. This paper suggests upgrading from conventional recommendations that are based on user/item preferences alone with systems that provide the best recommendation at the most optimal state when the user is most receptive to accept the recommendation, the "optimal state recommendation solution" and proposes solutions and architectures to overcome the challenges of dealing with real time, distributed machine learning on IoT scale data in implementing this solution. The paper leverages some of the advance distributed machine learning algorithms like variants of Distributed Kalman Filters, Distributed Alternating Least Square Recommenders, Distributed Mini-Batch Stochastic Gradient Descent(SGD) based Classifiers, and highly scalable distributed computation and machine learning platforms like Apache Spark, (Apache) Spark MLlib, Spark Streaming, Python/PySpark, R/SparkR, Apache Kafka in an high performance, distributed, fault tolerant architecture. The solution also aspires to be compliant with upcoming IoT standards and architectures like IEEE P2413 to provide a standard solution for such problems beyond the current scope of this paper.