Unsupervised Search Algorithm Configuration using Query Performance Prediction

Abstract

Search engine configuration can be quite difficult for inexpert developers. Instead, an auto-configuration approach can be used to speed up development time. Yet, such an automatic process usually requires relevance labels to train a supervised model. In this work, we suggest a simple, yet highly effective, extension to the probabilistic query performance prediction (QPP) framework that allows to auto-configure search algorithms without relevance labels. Our solution only assumes the availability of a sample of queries in a given domain. We demonstrate the merits of our solution using two common auto-configuration tasks. The first task is similarity model selection, including selection over “compound” models such as re-ranking and fusion. The second one is similarity parameter auto-tuning, where we choose amongst dozens of possible parameter configurations so to optimize the potential search quality.