Rico: A Mobile App Dataset for Building Data-Driven Design Applications

Data-driven models help mobile app designers understand best practices and trends, and can be used to make predictions about design performance and support the creation of adaptive UIs. This paper presents Rico, the largest repository of mobile app designs to date, created to support five classes of data-driven applications: design search, UI layout generation, UI code generation, user interaction modeling, and user perception prediction. To create Rico, we built a system that combines crowdsourcing and automation to scalably mine design and interaction data from Android apps at runtime. The Rico dataset contains design data from more than 9.7k Android apps spanning 27 categories. It exposes visual, textual, structural, and interactive design properties of more than 72k unique UI screens. To demonstrate the kinds of applications that Rico enables, we present results from training an autoencoder for UI layout similarity, which supports query- by-example search over UIs.

[1]  Antonella De Angeli,et al.  Computation of Interface Aesthetics , 2015, CHI.

[2]  Christos Faloutsos,et al.  Why people hate your app: making sense of user feedback in a mobile app store , 2013, KDD.

[3]  David P. Dobkin,et al.  A search engine for 3D models , 2003, TOGS.

[4]  Shahriyar Amini.  Analyzing Mobile App Privacy Using Computation and Crowdsourcing , 2014 .

[5]  Antti Oulasvirta,et al.  Computational Layout Perception using Gestalt Laws , 2016, CHI Extended Abstracts.

[6]  Katharina Reinecke,et al.  Predicting users' first impressions of website aesthetics with a quantification of perceived visual complexity and colorfulness , 2013, CHI.

[7]  Alireza Sahami Shirazi,et al.  Insights into layout patterns of mobile user interfaces by an automatic analysis of android apps , 2013, EICS '13.

[8]  Jia Deng,et al.  A large-scale hierarchical image database , 2009, CVPR 2009.

[9]  Ranjitha Kumar,et al.  ERICA: Interaction Mining Mobile Apps , 2016, UIST.

[10]  Antonella De Angeli,et al.  Visual impressions of mobile app interfaces , 2014, NordiCHI.

[11]  Biplab Deka,et al.  Interaction mining mobile apps , 2017 .

[12]  Scott R. Klemmer,et al.  Example-centric programming: integrating web search into the development environment , 2010, CHI.

[13]  Jason Flinn,et al.  AMC: verifying user interface properties for vehicular applications , 2013, MobiSys '13.

[14]  Dawn Xiaodong Song,et al.  Mining Permission Request Patterns from Android and Facebook Applications , 2012, 2012 IEEE 12th International Conference on Data Mining.

[15]  Radomír Mech,et al.  Learning design patterns with bayesian grammar induction , 2012, UIST.

[16]  Tom Yeh,et al.  Collect, Decompile, Extract, Stats, and Diff: Mining Design Pattern Changes in Android Apps , 2015, MobileHCI.

[17]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[18]  Antonella De Angeli,et al.  Pick me!: Getting Noticed on Google Play , 2016, CHI.

[19]  Anton van den Hengel,et al.  Image-Based Recommendations on Styles and Substitutes , 2015, SIGIR.

[20]  Kavita Bala,et al.  Learning visual similarity for product design with convolutional neural networks , 2015, ACM Trans. Graph..

[21]  Aaron Hertzmann,et al.  Learning Layouts for Single-PageGraphic Designs , 2014, IEEE Transactions on Visualization and Computer Graphics.

[22]  Omer Tsimhoni,et al.  Slow down, you move too fast: examining animation aesthetics to promote eco-driving , 2011, AutomotiveUI.

[23]  Brian P. Bailey,et al.  SEARCHING FOR INSPIRATION: AN IN-DEPTH LOOK AT DESIGNERS EXAMPLE FINDING PRACTICES , 2014 .

[24]  Alexandre N. Tuch,et al.  Vertical versus dynamic menus on the world wide web: Eye tracking study measuring the influence of menu design and task complexity on user performance and subjective preference , 2011, Comput. Hum. Behav..

[25]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[26]  Scott R. Klemmer,et al.  d.tour: style-based exploration of design example galleries , 2011, UIST.

[27]  Antonella De Angeli,et al.  Visual diversity and user interface quality , 2015, BCS HCI.

[28]  Jason Nieh,et al.  A measurement study of google play , 2014, SIGMETRICS '14.

[29]  Ranjitha Kumar,et al.  Webzeitgeist: design mining the web , 2013, CHI.

[30]  Suman Nath,et al.  Brahmastra: Driving Apps to Test the Security of Third-Party Components , 2014, USENIX Security Symposium.

[31]  Leonidas J. Guibas,et al.  Learning hierarchical shape segmentation and labeling from online repositories , 2017, ACM Trans. Graph..

[32]  Tuan Anh Nguyen,et al.  Reverse Engineering Mobile Application User Interfaces with REMAUI (T) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[33]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[34]  Iulian Neamtiu,et al.  Targeted and depth-first exploration for systematic testing of android apps , 2013, OOPSLA.

[35]  Claudia Eckert,et al.  Sources of inspiration: a language of design , 2000 .

[36]  Christopher Krügel,et al.  Challenges for Dynamic Analysis of iOS Applications , 2011, iNetSeC.

[37]  Claudia Eckert,et al.  REFERENCES TO PAST DESIGNS , 2005 .

[38]  Thea van der Geest,et al.  Testing the visual consistency of web sites , 2005 .