Automated Cross-Platform GUI Code Generation for Mobile Apps

Android and iOS are the two dominant platforms for building mobile apps. To provide uniform and smooth user experience, app companies typically employ two teams of programmers to develop UIs (and underlying functionalities) for these two platforms, respectively. However, this development practice is costly for both development and maintenance. To reduce the cost, we take the first step in this direction by proposing an automated cross-platform GUI code generation framework. It can transfer the GUI code implementation between the two mobile platforms. Specifically, our framework takes as input the UI pages and outputs the GUI code for the target platform (e.g., Android or iOS). It contains three phases, i.e., component identification, component type mapping, and GUI code generation. It leverages image processing and deep learning classification techniques. Apart from the UI pages of an app, this framework does not require any other inputs, which makes it possible for large-scale, platform -independent code generation.

[1]  Ting Su,et al.  FSMdroid: Guided GUI Testing of Android Apps , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering Companion (ICSE-C).

[2]  Yang Liu,et al.  Large-Scale Analysis of Framework-Specific Exceptions in Android Apps , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[3]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[4]  Tony Beltramelli,et al.  pix2code: Generating Code from a Graphical User Interface Screenshot , 2017, EICS.

[5]  Ying Zou,et al.  An Exploratory Study on the Relation between User Interface Complexity and the Perceived Quality , 2014, ICWE.

[6]  Yang Liu,et al.  From UI Design Image to GUI Skeleton: A Neural Machine Translator to Bootstrap Mobile GUI Implementation , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[7]  Yang Liu,et al.  Guided, stochastic model-based GUI testing of Android apps , 2017, ESEC/SIGSOFT FSE.

[8]  Yang Liu,et al.  Efficiently Manifesting Asynchronous Programming Errors in Android Apps , 2018, 2018 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[9]  Lingling Fan,et al.  StoryDroid: Automated Generation of Storyboard for Android Apps , 2019, 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE).

[10]  Tuan Anh Nguyen,et al.  Reverse Engineering Mobile Application User Interfaces with REMAUI (T) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).