Performance of neural network for indoor airflow prediction: sensitivity towards weight initialization

Abstract Neural networks (NNs) have been proposed as a promising alternative for fast and accurate prediction of indoor airflow. NN training is of great importance for acquiring accurate prediction results, which is essentially a nonconvex optimization process through gradient descent-based algorithms. NN performance at a certain solution is dependent on the initial parameter values from random initialization, crucial to the reliability of evaluation for model comparisons and hyperparameter tuning. In this study, the sensitivity of NN performance for indoor airflow prediction towards weight initialization is revealed by clarifying two issues on solution equivalence and the impact of weight initialization strategy. By reproducing non-isothermal indoor airflows, numerical experiments were conducted on various scenarios considering different initialization strategies. For each scenario, following the same convergence criteria, the training process was repeated to obtain multiple solutions concerning training / validation errors and temperature / velocity predictions. The results indicate that sensitivity of NN modeling capability to weight initialization for all scenarios are similar; while significant discrepancies among scenarios in sensitivity of generalization capability and convergence to weight initialization are demonstrated. NN with weight sampling intervals larger than [-1, 1] is more sensitive to initial weights than that with smaller sampling intervals.

[1]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[2]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[3]  Wangda Zuo,et al.  A systematic evaluation of accelerating indoor airflow simulations using cross-platform parallel computing , 2017 .

[4]  Claus Nebauer,et al.  Evaluation of convolutional neural networks for visual recognition , 1998, IEEE Trans. Neural Networks.

[5]  Aleksander Madry,et al.  How Does Batch Normalization Help Optimization? (No, It Is Not About Internal Covariate Shift) , 2018, NeurIPS.

[6]  Siddharth Krishna Kumar,et al.  On weight initialization in deep neural networks , 2017, ArXiv.

[7]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[8]  J. Xamán,et al.  Coupling CFD-BES Simulation of a glazed office with different types of windows in Mexico City , 2013 .

[9]  Nikos Komodakis,et al.  Exploring Weight Symmetry in Deep Neural Network , 2018, Comput. Vis. Image Underst..

[10]  Wangda Zuo,et al.  Fast and informative flow simulations in a building by using fast fluid dynamics model on graphics processing unit , 2010 .

[11]  Liping Wang,et al.  Coupled simulations for naturally ventilated rooms between building simulation (BS) and computational fluid dynamics (CFD) for better prediction of indoor thermal environment , 2009 .

[12]  Qingyan Chen,et al.  STRATEGIES FOR COUPLING ENERGY SIMULATION AND COMPUTATIONAL FLUID DYNAMICS PROGRAMS , 2001 .

[13]  J. Allegrini,et al.  Coupled CFD and building energy simulations for studying the impacts of building height topology and buoyancy on local urban microclimates , 2017 .

[14]  Wei Liu,et al.  Development of adaptive coarse grid generation methods for fast fluid dynamics in simulating indoor airflow , 2018 .

[15]  Rui Zhang,et al.  Coupled EnergyPlus and computational fluid dynamics simulation for natural ventilation , 2013 .

[16]  Sue Joseph,et al.  Australian Literary Journalism and “Missing Voices” , 2016 .

[17]  Qingyan Chen,et al.  On approaches to couple energy simulation and computational fluid dynamics programs , 2002 .

[18]  Zhiqiang John Zhai,et al.  Sensitivity analysis and application guides for integrated building energy and CFD simulation , 2006 .

[19]  Shi-Jie Cao,et al.  Ventilation control strategy using low-dimensional linear ventilation models and artificial neural network , 2018, Building and Environment.

[20]  Kenji Kawaguchi,et al.  Every Local Minimum Value Is the Global Minimum Value of Induced Model in Nonconvex Machine Learning , 2019, Neural Computation.

[21]  Wei Tian,et al.  COUPLED SIMULATION OF INDOOR ENVIRONMENT, HVAC AND CONTROL SYSTEM BY USING FAST FLUID DYNAMICS AND THE MODELICA BUILDINGS LIBRARY , 2014 .

[22]  Surya Ganguli,et al.  Identifying and attacking the saddle point problem in high-dimensional non-convex optimization , 2014, NIPS.

[23]  M. Sohn,et al.  Building energy simulation coupled with CFD for indoor environment: A critical review and recent applications , 2018 .

[24]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[25]  Zhiqiang Zhai,et al.  Numerical investigation on thermal performance and correlations of double skin façade with buoyancy-driven airflow , 2008 .

[26]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[27]  Michael Carbin,et al.  The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks , 2018, ICLR.

[28]  S-J Cao,et al.  On the construction and use of linear low-dimensional ventilation models. , 2012, Indoor air.

[29]  Yann LeCun,et al.  The Loss Surfaces of Multilayer Networks , 2014, AISTATS.

[30]  Surya Ganguli,et al.  Exact solutions to the nonlinear dynamics of learning in deep linear neural networks , 2013, ICLR.

[31]  Ryozo Ooka,et al.  Influence of data preprocessing on neural network performance for reproducing CFD simulations of non-isothermal indoor airflow distribution , 2021 .

[32]  Liping Wang,et al.  Coupled simulations for naturally ventilated residential buildings , 2008 .

[33]  Yun Kyu Yi,et al.  Dynamic integration between building energy simulation (BES) and computational fluid dynamics (CFD) simulation for building exterior surface , 2013 .

[34]  Daniel Soudry,et al.  No bad local minima: Data independent training error guarantees for multilayer neural networks , 2016, ArXiv.

[35]  Wangda Zuo,et al.  Coupling fast fluid dynamics and multizone airflow models in Modelica Buildings library to simulate the dynamics of HVAC systems , 2017 .

[36]  Trevor Darrell,et al.  Data-dependent Initializations of Convolutional Neural Networks , 2015, ICLR.

[37]  Shinsuke Kato,et al.  Building energy simulation considering spatial temperature distribution for nonuniform indoor environment , 2013 .

[38]  Dimitris P. Labridis,et al.  An ANN-based model for the prediction of internal lighting conditions and user actions in non-residential buildings , 2019, Journal of Building Performance Simulation.

[39]  Yoshua Bengio,et al.  Greedy Layer-Wise Training of Deep Networks , 2006, NIPS.

[40]  L. F. Abbott,et al.  Random Walk Initialization for Training Very Deep Feedforward Networks , 2014, 1412.6558.

[41]  Jie Ren,et al.  Incorporating online monitoring data into fast prediction models towards the development of artificial intelligent ventilation systems , 2019, Sustainable Cities and Society.

[42]  S. Kato,et al.  Control of indoor thermal environment based on concept of contribution ratio of indoor climate , 2010 .

[43]  Zaid Chalabi,et al.  Development of an England-wide indoor overheating and air pollution model using artificial neural networks , 2016 .

[44]  Laura Bellia,et al.  A coupled numerical approach on museum air conditioning: Energy and fluid-dynamic analysis , 2013 .

[45]  Saiprasad Koturwar,et al.  Weight Initialization of Deep Neural Networks(DNNs) using Data Statistics , 2017, ArXiv.

[46]  Wong Nyuk Hien,et al.  Effects of double glazed facade on energy consumption, thermal comfort and condensation for a typical office building in Singapore , 2005 .

[47]  Yoshua Bengio,et al.  Why Does Unsupervised Pre-training Help Deep Learning? , 2010, AISTATS.

[48]  Kazuhide Ito,et al.  Energy consumption analysis intended for real office space with energy recovery ventilator by integrating BES and CFD approaches , 2012 .

[49]  Olivier Ramalho,et al.  Machine learning and statistical models for predicting indoor air quality. , 2019, Indoor air.

[50]  Mengxuan Song,et al.  A neural-network enhanced modeling method for real-time evaluation of the temperature distribution in a data center , 2019, Neural Computing and Applications.

[51]  Yue Yuan,et al.  Improving prediction performance for indoor temperature in public buildings based on a novel deep learning method , 2019, Building and Environment.

[52]  Wei Tian,et al.  Coupling indoor airflow, HVAC, control and building envelope heat transfer in the Modelica Buildings library , 2016 .

[53]  Ryozo Ooka,et al.  Comparison of different deep neural network architectures for isothermal indoor airflow prediction , 2020, Building Simulation.

[54]  Zhiqiang John Zhai,et al.  Performance of coupled building energy and CFD simulations , 2005 .

[55]  Tian-hu Zhang,et al.  The use of genetic algorithm and self-updating artificial neural network for the inverse design of cabin environment , 2017 .

[56]  Q Chen,et al.  Real-time or faster-than-real-time simulation of airflow in buildings. , 2009, Indoor air.

[57]  Th. Frank,et al.  Thermal simulation of buildings with double-skin façades , 2005 .

[58]  Zhimin Du,et al.  Temperature sensor placement optimization for VAV control using CFD–BES co-simulation strategy , 2015 .