AutoIAS: Automatic Integrated Architecture Searcher for Click-Trough Rate Prediction

Automating architecture design for recommendation tasks becomes a trending topic because expert efforts are saved, and better performance is expected. Neural Architecture Search (NAS) is introduced to discover powerful CTR prediction model architectures in recent works. CTR prediction model usually consists of three components: embedding layer, interaction layer, and deep neural network. However, existing automation works focus on searching single component and leaving other components hand-crafted. The isolated searching will cause incompatibility among components and lead to weak generalization ability. Moreover, there is not a unified framework for integrated CTR prediction model architecture searching. This paper presents Automatic Integrated Architecture Searcher (AutoIAS), a framework that provides a practical and general method to find optimal CTR prediction model architecture in an automatic manner. In AutoIAS, we unify existing interaction-based CTR prediction model architectures and propose an integrated search space for a complete CTR prediction model. We utilize a supernet to predict the performance of sub-architectures, and the supernet is trained with Knowledge Distillation(KD) to enhance consistency among sub-architectures. To efficiently explore the search space, we design an architecture generator network that explicitly models the architecture dependencies among components and generates conditioned architectures distribution for each component. Experiments on public datasets show the outstanding performance and generalization ability of AutoIAS. Ablation study shows the effectiveness of the KD-based supernet training method and the Architecture Generator Network.