Multi-objective Relevance Ranking

In this paper, we introduce an Augmented Lagrangian basedmethod in a search relevance ranking algorithm to incorporate the multidimensional nature of relevance and business constraints, both of which are the requirements for building relevance ranking models in production. The off-the-shelf solutions cannot handle such complex objectives and therefore, modelers are left hand-tuning of parameters that have only indirect impact to the objectives, attempting to incorporate multiple objectives (MO) in a model. This process is time-consuming and tends to face sub-optimality. The proposed method is designed to systematically solve the MO problem in a constrained optimization framework, which is integrated with a popular Boosting algorithm and is, by all means, a novel contribution. Furthermore, we propose a procedure to specify the constraints to achieve business goals and the exploration scales linearly in the number of constraints, while existing methodology can scale exponentially. The experimental results show that the method successfully builds models that achieve MO criteria much more efficiently than existing methods. The potential impact includes significant reduction in model development time and allows for automation of model refresh even with presence of several MO criteria, in real world production system scale with hundreds of millions of records.