Ranking web-based partial orders by significance using a Markov reference model

Abstract

Mining web traffic data has been addressed in literature mostly using sequential pattern mining techniques. Recently, a more powerful pattern called partial order was introduced, with the hope of providing a more compact result set. A further approach towards this goal, valid for both sequential patterns and partial orders, consists in building a statistical significance test for frequent patterns. Our method is based on probabilistic generative models and provides a direct way to rank the extracted patterns. It leaves open the number of patterns of interest, which depends on the application, but provides an alternative criterion to frequency of occurrence: statistical significance. In this paper, we focus on the construction of an algorithm which calculates the probability of partial orders under a first-order Markov reference model, and we show how to use those probabilities to assess the statistical significance of a set of mined partial orders. © 2011 IEEE.