BERT for Target Apps Selection: Analyzing the Diversity and Performance of BERT in Unified Mobile Search

A unified mobile search framework aims to identify the mobile apps that can satisfy a user’s information need and route the user’s query to them. Previous work has shown that resource descriptions for mobile apps are sparse as they rely on the app’s previous queries. This problem puts certain apps in dominance and leaves out the resource-scarce apps from the top ranks. In this case, we need a ranker that goes beyond simple lexical matching. Therefore, our goal is to study the extent of a BERT-based ranker’s ability to improve the quality and diversity of app selection. To this end, we compare the results of the BERT-based ranker with other information retrieval models, focusing on the analysis of selected apps diversification. Our analysis shows that the BERT-based ranker selects more diverse apps while improving the quality of baseline results by selecting the relevant apps such as Facebook and Contacts for more personal queries and decreasing the bias towards the dominant resources such as the Google Search app.

[1]  James P. Callan,et al.  Query-based sampling of text databases , 2001, TOIS.

[2]  Fabio Crestani,et al.  Mobile Information Retrieval , 2017, SpringerBriefs in Computer Science.

[3]  W. Bruce Croft,et al.  Target Apps Selection: Towards a Unified Search Framework for Mobile Devices , 2018, SIGIR.

[4]  Jimmy J. Lin,et al.  Multi-Stage Document Ranking with BERT , 2019, ArXiv.

[5]  W. Bruce Croft,et al.  Context-aware Target Apps Selection and Recommendation for Enhancing Personal Mobile Assistants , 2021, ACM Trans. Inf. Syst..

[6]  Alexander Löser,et al.  How Does BERT Answer Questions?: A Layer-Wise Analysis of Transformer Representations , 2019, CIKM.

[7]  Jimmy J. Lin,et al.  End-to-End Open-Domain Question Answering with BERTserini , 2019, NAACL.

[8]  Nazli Goharian,et al.  CEDR: Contextualized Embeddings for Document Ranking , 2019, SIGIR.

[9]  Kyunghyun Cho,et al.  Passage Re-ranking with BERT , 2019, ArXiv.

[10]  Barry Smyth,et al.  Mobile information access: A study of emerging search behavior on the mobile Internet , 2007, TWEB.

[11]  Fernando Diaz,et al.  Learning to aggregate vertical results into web search results , 2011, CIKM '11.

[12]  W. Bruce Croft,et al.  In Situ and Context-Aware Target Apps Selection for Unified Mobile Search , 2018, CIKM.

[13]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[14]  James P. Callan,et al.  Context-Aware Document Term Weighting for Ad-Hoc Search , 2020, WWW.

[15]  Haohong Wang,et al.  Leveraging User Reviews to Improve Accuracy for Mobile App Retrieval , 2015, SIGIR.

[16]  Fernando Diaz,et al.  Vertical selection in the presence of unlabeled verticals , 2010, SIGIR '10.