Attention to Mean-Fields for Particle Cloud Generation

The generation of collider data using machine learning has emerged as a prominent research topic in particle physics due to the increasing computational challenges associated with traditional Monte Carlo simulation methods, particularly for future colliders with higher luminosity. Although generating particle clouds is analogous to generating point clouds, accurately modelling the complex correlations between the particles presents a considerable challenge. Additionally, variable particle cloud sizes further exacerbate these difficulties, necessitating more sophisticated models. In this work, we propose a novel model that utilizes an attention-based aggregation mechanism to address these challenges. The model is trained in an adversarial training paradigm, ensuring that both the generator and critic exhibit permutation equivariance/invariance with respect to their input. A novel feature matching loss in the critic is introduced to stabilize the training. The proposed model performs competitively to the state-of-art whilst having significantly fewer parameters.

[1]  B. Nachman,et al.  Fast point cloud generation with diffusion models in high energy physics , 2023, Physical Review D.

[2]  T. Golling,et al.  PC-JeDi: Diffusion for Particle Cloud Generation in High Energy Physics , 2023, SciPost Physics.

[3]  G. Kasieczka,et al.  EPiC-GAN: Equivariant Point Cloud Generation for Particle Jets , 2023, SciPost Physics.

[4]  I. Melzer-Pellmann,et al.  Point Cloud Generation using Transformer Encoders and Normalising Flows , 2022, 2211.13623.

[5]  I. Melzer-Pellmann,et al.  JetFlow: Generating Jets with Conditioned and Mass Constrained Normalising Flows , 2022, 2211.13630.

[6]  Javier Mauricio Duarte,et al.  Particle Cloud Generation with Message Passing Generative Adversarial Networks , 2021, NeurIPS.

[7]  Benjamin Nachman,et al.  A Living Review of Machine Learning for Particle Physics , 2021, ArXiv.

[8]  Yee Whye Teh,et al.  Set Transformer , 2018, ICML.

[9]  Patrick T. Komiske,et al.  Energy flow polynomials: a complete linear basis for jet substructure , 2017, 1712.07124.

[10]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[11]  Sitao Xiang,et al.  On the Effects of Batch and Weight Normalization in Generative Adversarial Networks , 2017, 1704.03971.

[12]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[13]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[14]  Yann Dauphin,et al.  Language Modeling with Gated Convolutional Networks , 2016, ICML.

[15]  Zhen Wang,et al.  Multi-class Generative Adversarial Networks with the L2 Loss Function , 2016, ArXiv.

[16]  Geoffrey E. Hinton,et al.  Layer Normalization , 2016, ArXiv.

[17]  Kevin Gimpel,et al.  Gaussian Error Linear Units (GELUs) , 2016, 1606.08415.

[18]  Wojciech Zaremba,et al.  Improved Techniques for Training GANs , 2016, NIPS.

[19]  Lucio Rossi,et al.  High-Luminosity Large Hadron Collider (HL-LHC) : Preliminary Design Report , 2015 .

[20]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[21]  Javier Mauricio Duarte,et al.  On the Evaluation of Generative Models in High Energy Physics , 2022, ArXiv.

[22]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.