Accurate Modeling of Frequency Selective Surfaces Using Fully-Connected Regression Model With Automated Architecture Determination and Parameter Selection Based on Bayesian Optimization

Surrogate modeling has become an important tool in the design of high-frequency structures. Although full-wave electromagnetic (EM) simulation tools provide an accurate account for the circuit characteristics and performance, they entail considerable computational expenditures. Replacing EM analysis by fast surrogates provides a way to accelerate the design procedures. Unfortunately, modeling of microwave passives is a challenging task due to their highly-nonlinear outputs. Frequency selective surfaces (FSSs) constitute a representative example with their multi-resonant reflection and transmission responses that need to be represented over broad frequency ranges. Deep neural networks (DNNs) seem to be the promising techniques for handling such cases. However, a serious practical issue associated with their employment is an appropriate selection of the model parameters, including its architecture. A common practice is experience-driven setup, heavily based on trial and error, which does not guarantee the optimum model determination and may lead to multiple problems such as poor generalization or high variance of the model predictive power with respect to the training data set selection. This paper proposes a novel modeling framework, referred to as a fully-connected regression model (FCRM), where the crucial role is played by Bayesian Optimization (BO), incorporated to determine the DNN-based model setup, including both its architecture and the hyperparameter values, in a fully automated manner. For validation, FCRM is applied to construct the model of a Minkowski Fractal-Based FSS. The efficacy of the methodology is demonstrated through comparisons with several benchmark techniques, including the DNN surrogates established using the traditional methods as well as conventional regression models. The numerical results indicate that FCRM exhibits considerably improved prediction power and reduced sensitivity to the training sample assignment.

[1]  Ming-Shing Lin,et al.  Bandpass Shielding Enclosure Design Using Multipole-Slot Arrays for Modern Portable Digital Devices , 2008, IEEE Transactions on Electromagnetic Compatibility.

[2]  Ramana V. Grandhi,et al.  Improved Distributed Hypercube Sampling , 2002 .

[3]  A. Žilinskas,et al.  On the use of statistical models of multimodal functions for the construction of the optimization algorithms , 1980 .

[4]  Jingon Joung,et al.  Machine Learning-Based Antenna Selection in Wireless Communications , 2016, IEEE Communications Letters.

[5]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[6]  Ataollah Ebrahimzadeh,et al.  Improved Fruit-Fly Optimization Algorithm and Its Applications in Antenna Arrays Synthesis , 2018, IEEE Transactions on Antennas and Propagation.

[7]  Philippe Ferrari,et al.  Accurate Parametric Electrical Model for Slow-Wave CPW and Application to Circuits Design , 2015, IEEE Transactions on Microwave Theory and Techniques.

[8]  P. Sheng,et al.  Dark acoustic metamaterials as super absorbers for low-frequency sound , 2012, Nature Communications.

[9]  Rowdra Ghatak,et al.  Multi-band bisected Hilbert monopole antenna loaded with multiple subwavelength split-ring resonators , 2018 .

[10]  Wei Zhang,et al.  Fast and simple technique for computing circuit noise figure from component noise model using artificial neural network , 2015, 2015 IEEE MTT-S International Conference on Numerical Electromagnetic and Multiphysics Modeling and Optimization (NEMO).

[11]  Abdallah Shami,et al.  Bayesian Optimization with Machine Learning Algorithms Towards Anomaly Detection , 2018, 2018 IEEE Global Communications Conference (GLOBECOM).

[12]  Jian Wang,et al.  Efficient gradient-based optimisation of pixel antenna with large-scale connections , 2018 .

[13]  Yu Peng,et al.  The Classification of Synthetic Aperture Radar Image Target Based on Deep Learning , 2016, J. Inf. Hiding Multim. Signal Process..

[14]  Hui Liu,et al.  Smart wind speed forecasting approach using various boosting algorithms, big multi-step forecasting strategy , 2019, Renewable Energy.

[15]  Sławomir Kozieł,et al.  Reduced-cost design closure of antennas by means of gradient search with restricted sensitivity update , 2023, Metrology and Measurement Systems.

[16]  Qijun Zhang,et al.  A Novel Deep Neural Network Topology for Parametric Modeling of Passive Microwave Components , 2020, IEEE Access.

[17]  Tianqi Chen,et al.  Empirical Evaluation of Rectified Activations in Convolutional Network , 2015, ArXiv.

[18]  Qi-Jun Zhang,et al.  Recent advances in neural network‐based inverse modeling techniques for microwave applications , 2020, International Journal of Numerical Modelling: Electronic Networks, Devices and Fields.

[19]  Filiz GÜNE,et al.  A Novel Design Approach to X-Band Minkowski Reflectarray Antennas using the Full-Wave EM Simulation-based Complete Neural Model with a Hybrid GA-NM Algorithm , 2014 .

[20]  Odysseas Tsilipakos,et al.  Antimatched Electromagnetic Metasurfaces for Broadband Arbitrary Phase Manipulation in Reflection , 2018, ACS photonics.

[21]  Jensen Li,et al.  Double-negative acoustic metamaterial. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[22]  Ayman Elboushi,et al.  Miniaturized triple band-notched quasi-self complementary fractal antenna with improved characteristics for UWB applications , 2019, AEU - International Journal of Electronics and Communications.

[23]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[24]  Filiz Güneş,et al.  GSM filtering of horn antennas using modified double square frequency selective surface , 2017 .

[25]  D. Goldberg,et al.  BOA: the Bayesian optimization algorithm , 1999 .

[26]  Xudong Chen,et al.  Deep-Learning Schemes for Full-Wave Nonlinear Inverse Scattering Problems , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[27]  N. Fang,et al.  Sub–Diffraction-Limited Optical Imaging with a Silver Superlens , 2005, Science.

[28]  B. Minasny,et al.  The Matérn function as a general model for soil variograms , 2005 .

[29]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[30]  Jonas Mockus,et al.  Application of Bayesian approach to numerical methods of global and stochastic optimization , 1994, J. Glob. Optim..

[31]  Seref Sagiroglu,et al.  Neural computation of resonant frequency of electrically thin and thick rectangular microstrip antennas , 1999 .

[32]  Philip D. Wasserman,et al.  Advanced methods in neural computing , 1993, VNR computer library.

[33]  Filiz Güneş,et al.  Symbolic Regression for Derivation of an Accurate Analytical Formulation Using “Big Data”: An Application Example , 2017 .

[34]  Franco Giannini,et al.  Small‐signal and large‐signal modeling of active devices using CAD‐optimized neural networks , 2002 .

[35]  Vincent Fusco,et al.  Pre-matched Eigenmode Antenna With Polarization and Pattern Diversity , 2019, IEEE Transactions on Antennas and Propagation.

[36]  Rabindra K. Mishra,et al.  Neural network-based CAD model for the design of square-patch antennas , 1998 .

[37]  Guoliang Huang,et al.  Analytical coupled vibroacoustic modeling of membrane-type acoustic metamaterials: membrane model. , 2013, The Journal of the Acoustical Society of America.

[38]  N. Yu,et al.  Light Propagation with Phase Discontinuities: Generalized Laws of Reflection and Refraction , 2011, Science.

[39]  Fabien Ferrero,et al.  Dual-Matching for Single Resonance Miniaturized Antenna for IoT applications , 2018, 2018 IEEE International Symposium on Antennas and Propagation & USNC/URSI National Radio Science Meeting.

[40]  Gianni Di Pillo,et al.  A SVM Surrogate Model-Based Method for Parametric Yield Optimization , 2016, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[41]  Matthew W. Hoffman,et al.  Predictive Entropy Search for Efficient Global Optimization of Black-box Functions , 2014, NIPS.

[42]  Matthias Hein,et al.  Low-Profile Penta-Band Automotive Patch Antenna Using Horizontal Stacking and Corner Feeding , 2019, IEEE Access.

[43]  Vicente E. Boria,et al.  Exploring the Tuning Range of Channel Filters for Satellite Applications Using Electromagnetic-Based Computer Aided Design Tools , 2018, IEEE Transactions on Microwave Theory and Techniques.

[44]  Slawomir Koziel,et al.  Computationally‐efficient design optimisation of antennas by accelerated gradient search with sensitivity and design change monitoring , 2019, IET Microwaves, Antennas & Propagation.

[45]  P. Sheng,et al.  Membrane-type acoustic metamaterial with negative dynamic mass. , 2008, Physical review letters.

[46]  Guoliang Huang,et al.  Analytical coupled vibroacoustic modeling of membrane-type acoustic metamaterials: plate model. , 2013, The Journal of the Acoustical Society of America.

[47]  Filippo Costa,et al.  Electromagnetic Absorbers Based on Frequency Selective Surfaces , 2016 .

[48]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[49]  Carlos Gershenson,et al.  Artificial Neural Networks for Beginners , 2003, ArXiv.

[50]  Richard J. Beckman,et al.  A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output From a Computer Code , 2000, Technometrics.

[51]  Alois Knoll,et al.  Gradient boosting machines, a tutorial , 2013, Front. Neurorobot..

[52]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[53]  Malathi Kanagasabai,et al.  A Wideband Frequency Tunable FSS for Electromagnetic Shielding Applications , 2018, IEEE Transactions on Electromagnetic Compatibility.

[54]  Zlatica Marinkovic,et al.  Temperature-dependent models of low-noise microwave transistors based on neural networks , 2005 .

[55]  F. Hutter,et al.  Fast Bayesian hyperparameter optimization on large datasets , 2017, Electronic Journal of Statistics.

[56]  Xiaoling Zhang,et al.  Target recognition of 3-D synthetic aperture radar images via deep belief network , 2016, 2016 CIE International Conference on Radar (RADAR).

[57]  Elson J. Silva,et al.  Infinitesimal Dipole Model Using Space Mapping Optimization for Antenna Placement , 2018, IEEE Antennas and Wireless Propagation Letters.

[58]  Guy A. E. Vandenbosch,et al.  An Efficient Method for Antenna Design Optimization Based on Evolutionary Computation and Machine Learning Techniques , 2014, IEEE Transactions on Antennas and Propagation.

[59]  Jasper Snoek,et al.  Practical Bayesian Optimization of Machine Learning Algorithms , 2012, NIPS.

[60]  Weicong Na,et al.  Deep Neural Network Technique for High-Dimensional Microwave Modeling and Applications to Parameter Extraction of Microwave Filters , 2019, IEEE Transactions on Microwave Theory and Techniques.

[61]  Xiaoli Li,et al.  Deep Convolutional Neural Network Based Regression Approach for Estimation of Remaining Useful Life , 2016, DASFAA.

[62]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.

[63]  Noemí DeCastro-García,et al.  Effect of the Sampling of a Dataset in the Hyperparameter Optimization Phase over the Efficiency of a Machine Learning Algorithm , 2019, Complex..

[64]  M. A. Belen,et al.  Performance enhancement of a microstrip patch antenna using substrate integrated waveguide frequency selective surface for ISM band applications , 2018 .

[65]  Colin P. Schwegmann,et al.  Very deep learning for ship discrimination in Synthetic Aperture Radar imagery , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[66]  Salih Demirel,et al.  Design and analysis of minkowski reflectarray antenna using 3‐D CST microwave studio‐based neural network model with particle swarm optimization , 2013 .

[67]  Li Yang,et al.  On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice , 2020, Neurocomputing.

[68]  Tom Dhaene,et al.  Low-Cost Surrogate Models for Microwave Filters , 2016, IEEE Microwave and Wireless Components Letters.

[69]  Slawomir Koziel,et al.  Reduced‐cost microwave filter modeling using a two‐stage Gaussian process regression approach , 2015 .

[70]  L. Grippo,et al.  On the convergence of the block nonlinear Gauss-Seidel method under convex constraints , 2000, Oper. Res. Lett..

[71]  Ram Narayan Yadav,et al.  Directivity Estimations for Short Dipole Antenna Arrays Using Radial Basis Function Neural Networks , 2015, IEEE Antennas and Wireless Propagation Letters.

[72]  Marvin Minsky,et al.  An introduction to computational geometry , 1969 .

[73]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[74]  Ahmed Salim,et al.  Metamaterial Inspired Radio Frequency-Based Touchpad Sensor System , 2020, IEEE Transactions on Instrumentation and Measurement.

[75]  J. Pendry,et al.  Negative refraction makes a perfect lens , 2000, Physical review letters.

[76]  Asifullah Khan,et al.  Wind power prediction using deep neural network based meta regression and transfer learning , 2017, Appl. Soft Comput..

[77]  Peng Chen,et al.  Bayesian Optimization for Broadband High-Efficiency Power Amplifier Designs , 2015, IEEE Transactions on Microwave Theory and Techniques.

[78]  Tzong-Lin Wu,et al.  Ultracompact Via-Based Absorptive Frequency-Selective Surface for 5-GHz Wi-Fi With Passbands and High-Performance Stability , 2018, IEEE Transactions on Components, Packaging and Manufacturing Technology.

[79]  R. Shelby,et al.  Experimental Verification of a Negative Index of Refraction , 2001, Science.

[80]  M. A. Belen,et al.  UWB Gain Enhancement of Horn Antennas Using Miniaturized Frequency Selective Surface , 2018 .

[81]  Peyman Mahouti,et al.  Deep learning base modified MLP model for precise scattering parameter prediction of capacitive feed antenna , 2019, International Journal of Numerical Modelling: Electronic Networks, Devices and Fields.

[82]  Qi-Jun Zhang,et al.  Neural Network Inverse Modeling and Applications to Microwave Filter Design , 2008, IEEE Transactions on Microwave Theory and Techniques.

[83]  Peyman Mahouti,et al.  Design optimization of a pattern reconfigurable microstrip antenna using differential evolution and 3D EM simulation‐based neural network model , 2019, International Journal of RF and Microwave Computer-Aided Engineering.

[84]  Seyed Mohsen Hosseini Varkiani,et al.  Compact and ultra-wideband CPW-fed square slot antenna for wearable applications , 2019, AEU - International Journal of Electronics and Communications.

[85]  Thomas J. Santner,et al.  Space-Filling Designs for Computer Experiments , 2003 .

[86]  John W. Bandler,et al.  Rapid Yield Estimation and Optimization of Microwave Structures Exploiting Feature-Based Statistical Analysis , 2015, IEEE Transactions on Microwave Theory and Techniques.

[87]  Peyman Mahouti,et al.  Application of artificial intelligence algorithms on modeling of reflection phase characteristics of a nonuniform reflectarray element , 2019, International Journal of Numerical Modelling: Electronic Networks, Devices and Fields.

[88]  Patrick Neary Automatic Hyperparameter Tuning in Deep Convolutional Neural Networks Using Asynchronous Reinforcement Learning , 2018, 2018 IEEE International Conference on Cognitive Computing (ICCC).

[89]  Slawomir Koziel,et al.  Gaussian process antenna modeling using neighborhood-data-expanded training sets , 2013, 2013 IEEE Antennas and Propagation Society International Symposium (APSURSI).

[90]  M. Cacciola,et al.  Microwave Devices and Antennas Modelling by Support Vector Regression Machines , 2006, IEEE Transactions on Magnetics.

[91]  Chao Zhang,et al.  Parametric Modeling of EM Behavior of Microwave Components Using Combined Neural Networks and Pole-Residue-Based Transfer Functions , 2016, IEEE Transactions on Microwave Theory and Techniques.

[92]  Jose Luis Chavez-Hurtado,et al.  Polynomial-Based Surrogate Modeling of RF and Microwave Circuits in Frequency Domain Exploiting the Multinomial Theorem , 2016, IEEE Transactions on Microwave Theory and Techniques.

[93]  Hao Li,et al.  Visualizing the Loss Landscape of Neural Nets , 2017, NeurIPS.

[94]  Yahya Rahmat-Samii,et al.  Fractal antennas: a novel antenna miniaturization technique, and applications , 2002 .

[95]  Donald F. Specht,et al.  A general regression neural network , 1991, IEEE Trans. Neural Networks.

[96]  Filiz Güneş,et al.  Cost‐effective GRNN‐based modeling of microwave transistors with a reduced number of measurements , 2017 .

[97]  Er-Ping Li,et al.  A 2.5-D Angularly Stable Frequency Selective Surface Using Via-Based Structure for 5G EMI Shielding , 2018, IEEE Transactions on Electromagnetic Compatibility.

[98]  Nando de Freitas,et al.  A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning , 2010, ArXiv.

[99]  Yong Bae Park,et al.  Analysis of FSS Radomes Based on Physical Optics Method and Ray Tracing Technique , 2014, IEEE Antennas and Wireless Propagation Letters.

[100]  Andreas Krause,et al.  Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting , 2009, IEEE Transactions on Information Theory.

[101]  Jooyoung Park,et al.  Approximation and Radial-Basis-Function Networks , 1993, Neural Computation.

[102]  Dong Liu,et al.  Dominant-Current Deep Learning Scheme for Electrical Impedance Tomography , 2019, IEEE Transactions on Biomedical Engineering.

[103]  Mahdi Nezhad Asad,et al.  Investigation of bone age assessment with convolutional neural network by using DoG filtering and à trous wavelet as preprocessing techniques , 2018, 2018 6th International Conference on Control Engineering & Information Technology (CEIT).

[104]  Filiz Güneş,et al.  Horn antennas with enhanced functionalities through the use of frequency selective surfaces , 2016 .

[105]  Jian Dong,et al.  Low-Cost Multi-Objective Optimization of Multiparameter Antenna Structures Based on the l1 Optimization BPNN Surrogate Model , 2019, Electronics.

[106]  Yoshua Bengio,et al.  Greedy Layer-Wise Training of Deep Networks , 2006, NIPS.

[107]  Peter I. Frazier,et al.  A Tutorial on Bayesian Optimization , 2018, ArXiv.

[108]  J. Friedman Greedy function approximation: A gradient boosting machine. , 2001 .

[109]  B. Matérn Spatial variation : Stochastic models and their application to some problems in forest surveys and other sampling investigations , 1960 .

[110]  Peyman Mahouti,et al.  A Novel Design of Non-Uniform Reflectarrays with Symbolic Regression and its Realization using 3-D Printer , 2019 .

[111]  S. Kawata,et al.  Unattenuated light transmission through the interface between two materials with different indices of refraction using magnetic metamaterials , 2006 .

[112]  Philipp Hennig,et al.  Entropy Search for Information-Efficient Global Optimization , 2011, J. Mach. Learn. Res..

[113]  Hao Xin,et al.  A 3-D-Printed W-Band Slotted Waveguide Array Antenna Optimized Using Machine Learning , 2018, IEEE Antennas and Wireless Propagation Letters.

[114]  Silvio Savarese,et al.  Learning to Track at 100 FPS with Deep Regression Networks , 2016, ECCV.

[115]  Yulei Rao,et al.  A deep learning framework for financial time series using stacked autoencoders and long-short term memory , 2017, PloS one.

[116]  Gurpreet Kumar,et al.  A survey on planar ultra-wideband antennas with band notch characteristics: Principle, design, and applications , 2019, AEU - International Journal of Electronics and Communications.

[117]  C. Pfeiffer,et al.  Metamaterial Huygens' surfaces: tailoring wave fronts with reflectionless sheets. , 2013, Physical review letters.

[118]  Seungtae Ko,et al.  Optically Invisible Antenna Integrated Within an OLED Touch Display Panel for IoT Applications , 2017, IEEE Transactions on Antennas and Propagation.

[119]  Slawomir Koziel,et al.  Reduced‐cost electromagnetic‐driven optimisation of antenna structures by means oftrust‐region gradient‐search with sparse Jacobian updates , 2019, IET Microwaves, Antennas & Propagation.

[120]  V. Vapnik Pattern recognition using generalized portrait method , 1963 .

[121]  Hang Wong,et al.  Multi-Polarization Reconfigurable Antenna for Wireless Biomedical System , 2017, IEEE Transactions on Biomedical Circuits and Systems.