Gating PatternPyramid for diversified image style transfer

Abstract. In diversified image style transfer tasks, the goal is to translate an everyday photograph to diverse stylized images conditioned on the style of an artwork. Although several style transfer methods have achieved certain diversity through noise injection, they still have two unsolved problems: (1) relatively limited diversity and (2) significant degradation of quality. In this work, we introduce an effective Gating PatternPyramid block (GPP) to resolve these issues by employing a well-designed multibranch architecture. The main idea is underpinned by a finding that different network architectures capture different style patterns from the same artwork. In addition, the GPP block is compatible with many feed-forward style transfer models and empowers them with the ability of generating diverse stylization results. To the best of our knowledge, this is the first style transfer method that achieves significant diversity without injecting random noise, which provides an inspiring perspective for diverse translation research. We perform qualitative and quantitative experiments, showing the effectiveness and superiority of our method against the state-of-the-art diversified style transfer methods.