An optimizer ensemble algorithm and its application to image registration

The design of effective optimization algorithms is always a hot research topic. An optimizer ensemble where any population-based optimization algorithm can be integrated is proposed in this study. First, the optimizer ensemble framework based on ensemble learning is presented. The learning table consisting of the population members of all optimizers is constructed to share information. The maximum number of iterations is divided into several exchange iterations. Each optimizer exchanges individuals with the learning table in exchange iterations and runs independently in the other iterations. Exchange individuals are generated by a bootstrap sample from the learning table. To maintain a balance between exchange individuals and preserved individuals, the exchange number of each optimizer is adaptively assigned according to its fitness. The output is obtained by the voting approach that selects the highest ranked solution. Second, an optimizer ensemble algorithm (OEA) which combines multiple population-based optimization algorithms is proposed. The computational complexity, convergence, and diversity of OEA are analyzed. Finally, extensive experiments on benchmark functions demonstrate that OEA outperforms several state-ofthe-art algorithms. OEA is used to search the maximum mutual information in image registration. The high performance of OEA is further verified by a large number of registration results on real remote sensing images.

[1]  P. N. Suganthan,et al.  Ensemble of niching algorithms , 2010, Inf. Sci..

[2]  David J. Montana,et al.  Strongly Typed Genetic Programming , 1995, Evolutionary Computation.

[3]  Hujun Yin,et al.  Multi-Step Time Series Forecasting with an Ensemble of Varied Length Mixture Models , 2017, Int. J. Neural Syst..

[4]  Jacek M. Zurada,et al.  An approach to multimodal biomedical image registration utilizing particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[5]  Hossein Nezamabadi-pour,et al.  BGSA: binary gravitational search algorithm , 2010, Natural Computing.

[6]  Jian Ma,et al.  Sentiment classification: The contribution of ensemble learning , 2014, Decis. Support Syst..

[7]  G. Di Caro,et al.  Ant colony optimization: a new meta-heuristic , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[8]  Fazhi He,et al.  Service-Oriented Feature-Based Data Exchange for Cloud-Based Design and Manufacturing , 2018, IEEE Transactions on Services Computing.

[9]  Amir Hossein Gandomi,et al.  Chaotic cuckoo search , 2015, Soft Computing.

[10]  Ponnuthurai N. Suganthan,et al.  Ensemble strategies with adaptive evolutionary programming , 2010, Inf. Sci..

[11]  Yiteng Pan,et al.  A novel Enhanced Collaborative Autoencoder with knowledge distillation for top-N recommender systems , 2019, Neurocomputing.

[12]  Yuan Cheng,et al.  An optimized RGA supporting selective undo for collaborative text editing systems , 2019, J. Parallel Distributed Comput..

[13]  Hojjat Adeli,et al.  Water Drop Algorithms , 2014, Int. J. Artif. Intell. Tools.

[14]  María R. Sierra,et al.  Evolutionary one-machine scheduling in the context of electric vehicles charging , 2018, Integr. Comput. Aided Eng..

[15]  Andrew Lewis,et al.  The Whale Optimization Algorithm , 2016, Adv. Eng. Softw..

[16]  C. Lucas,et al.  A novel numerical optimization algorithm inspired from weed colonization , 2006, Ecol. Informatics.

[17]  Bilal Alatas,et al.  ACROA: Artificial Chemical Reaction Optimization Algorithm for global optimization , 2011, Expert Syst. Appl..

[18]  Xin-She Yang,et al.  A New Metaheuristic Bat-Inspired Algorithm , 2010, NICSO.

[19]  Thomas G. Dietterich Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[20]  Qi Li,et al.  Multimodality image registration using local linear embedding and hybrid entropy , 2013, Neurocomputing.

[21]  Thomas G. Dietterich Machine-Learning Research , 1997, AI Mag..

[22]  P. N. Suganthan,et al.  Differential Evolution Algorithm With Strategy Adaptation for Global Numerical Optimization , 2009, IEEE Transactions on Evolutionary Computation.

[23]  María José del Jesús,et al.  A Pareto-based Ensemble with Feature and Instance Selection for Learning from Multi-Class Imbalanced Datasets , 2017, Int. J. Neural Syst..

[24]  Yi Zhou,et al.  Parallel ant colony optimization on multi-core SIMD CPUs , 2018, Future Gener. Comput. Syst..

[25]  Mehmet Fatih Tasgetiren,et al.  An ensemble of discrete differential evolution algorithms for solving the generalized traveling salesman problem , 2010, Appl. Math. Comput..

[26]  Yuan Cheng,et al.  A novel CRDT-based synchronization method for real-time collaborative CAD systems , 2018, Adv. Eng. Informatics.

[27]  Hojjat Adeli,et al.  Spiral Dynamics Algorithm , 2014, Int. J. Artif. Intell. Tools.

[28]  Andrés Iglesias,et al.  Multilayer embedded bat algorithm for B-spline curve reconstruction , 2017, Integr. Comput. Aided Eng..

[29]  Shiwei Wei,et al.  A convergence-diversity balanced fitness evaluation mechanism for decomposition- based many-objective optimization algorithm , 2019, Integr. Comput. Aided Eng..

[30]  Thomas Bäck,et al.  Multi-objective inventory routing with uncertain demand using population-based metaheuristics , 2016, Integr. Comput. Aided Eng..

[31]  Hossein Nezamabadi-pour,et al.  GSA: A Gravitational Search Algorithm , 2009, Inf. Sci..

[32]  Giovanni Iacca,et al.  Multi-Strategy coevolving aging Particle Optimization , 2014, Int. J. Neural Syst..

[33]  Hojjat Adeli,et al.  A novel unsupervised deep learning model for global and local health condition assessment of structures , 2018 .

[34]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[35]  Michael G. Epitropakis,et al.  Progressive preference articulation for decision making in multi-objective optimisation problems , 2017, Integr. Comput. Aided Eng..

[36]  A. Kaveh,et al.  A new meta-heuristic method: Ray Optimization , 2012 .

[37]  Qiang Wang,et al.  Optimizing the energy-spectrum efficiency of cellular systems by evolutionary multi-objective algorithm , 2019, Integr. Comput. Aided Eng..

[38]  Zexuan Zhu,et al.  Adaptive multiple-elites-guided composite differential evolution algorithm with a shift mechanism , 2018, Inf. Sci..

[39]  Fazhi He,et al.  An Efficient Particle Swarm Optimization for Large-Scale Hardware/Software Co-Design System , 2017, Int. J. Cooperative Inf. Syst..

[40]  Fazhi He,et al.  IBEA-SVM: An Indicator-based Evolutionary Algorithm Based on Pre-selection with Classification Guided by SVM , 2019, Applied Mathematics-A Journal of Chinese Universities.

[41]  Xin Yao,et al.  Evolutionary programming made faster , 1999, IEEE Trans. Evol. Comput..

[42]  Ferrante Neri,et al.  Covariance matrix adaptation pareto archived evolution strategy with hypervolume-sorted adaptive grid algorithm , 2016, Integr. Comput. Aided Eng..

[43]  Liang Gao,et al.  An improved fruit fly optimization algorithm for continuous function optimization problems , 2014, Knowl. Based Syst..

[44]  Ignacio Rojas,et al.  Multi-Objective Genetic Algorithms to Find Most Relevant Volumes of the Brain Related to Alzheimer's Disease and Mild Cognitive Impairment , 2018, Int. J. Neural Syst..

[45]  María R. Sierra,et al.  Genetic algorithms for the scheduling problem with arbitrary precedence relations and skilled operators , 2016, Integr. Comput. Aided Eng..

[46]  Saeid Sanei,et al.  Deep Neural Architectures for Mapping Scalp to Intracranial EEG , 2018, Int. J. Neural Syst..

[47]  Gexiang Zhang,et al.  Multicriteria adaptive differential evolution for global numerical optimization , 2015, Integr. Comput. Aided Eng..

[48]  Fazhi He,et al.  Quantitative optimization of interoperability during feature-based data exchange , 2015, Integr. Comput. Aided Eng..

[49]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[50]  Huosheng Hu,et al.  A novel camera calibration technique based on differential evolution particle swarm optimization algorithm , 2016, Neurocomputing.

[51]  Jorge Puente,et al.  Satisfying flexible due dates in fuzzy job shop by means of hybrid evolutionary algorithms , 2018, Integr. Comput. Aided Eng..

[52]  Soonhung Han,et al.  An efficient approach to directly compute the exact Hausdorff distance for 3D point sets , 2017, Integr. Comput. Aided Eng..

[53]  Alex Alexandridis,et al.  A particle swarm optimization approach in printed circuit board thermal design , 2017, Integr. Comput. Aided Eng..

[54]  P. Suganthan,et al.  Differential evolution algorithm with ensemble of populations for global numerical optimization , 2009 .

[55]  Xin Yao,et al.  Ensemble learning via negative correlation , 1999, Neural Networks.

[56]  Francisco Javier Vera-Olmos,et al.  DeepEye: Deep convolutional network for pupil detection in real environments , 2018, Integr. Comput. Aided Eng..

[57]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[58]  Tshilidzi Marwala,et al.  Finite element model updating using fish school search and volitive particle swarm optimization , 2015, Integr. Comput. Aided Eng..

[59]  Khalid M. Mosalam,et al.  Deep Transfer Learning for Image‐Based Structural Damage Recognition , 2018, Comput. Aided Civ. Infrastructure Eng..

[60]  Fazhi He,et al.  A Novel Hardware/Software Partitioning Method Based on Position Disturbed Particle Swarm Optimization with Invasive Weed Optimization , 2017, Journal of Computer Science and Technology.

[61]  Dervis Karaboga,et al.  A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm , 2007, J. Glob. Optim..

[62]  P. Suganthan,et al.  Constrained multi-objective optimization algorithm with an ensemble of constraint handling methods , 2011 .

[63]  Qingwu Hu,et al.  Robust feature matching via support-line voting and affine-invariant ratios , 2017 .

[64]  Hojjat Adeli,et al.  Harmony Search Algorithm and its Variants , 2015, Int. J. Pattern Recognit. Artif. Intell..

[65]  Eric Bauer,et al.  An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants , 1999, Machine Learning.

[66]  Ferrante Neri,et al.  An Optimization Spiking Neural P System for Approximately Solving Combinatorial Optimization Problems , 2014, Int. J. Neural Syst..

[67]  Fabio Caraffini,et al.  An analysis on separability for Memetic Computing automatic design , 2014, Inf. Sci..

[68]  Bin Li,et al.  Two-stage based ensemble optimization for large-scale global optimization , 2010, IEEE Congress on Evolutionary Computation.

[69]  Yuping Wang,et al.  Multiobjective bilevel optimization for production-distribution planning problems using hybrid genetic algorithm , 2014, Integr. Comput. Aided Eng..

[70]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[71]  Yuan Cheng,et al.  Supporting selective undo of string-wise operations for collaborative editing systems , 2018, Future Gener. Comput. Syst..

[72]  Lorenzo Bruzzone,et al.  Robust Registration of Multimodal Remote Sensing Images Based on Structural Similarity , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[73]  Lior Rokach,et al.  Ensemble learning: A survey , 2018, WIREs Data Mining Knowl. Discov..

[74]  Alicia Troncoso Lora,et al.  Enhancing the scalability of a genetic algorithm to discover quantitative association rules in large-scale datasets , 2015, Integr. Comput. Aided Eng..

[75]  Natalio Krasnogor,et al.  Nature‐inspired cooperative strategies for optimization , 2009, Int. J. Intell. Syst..

[76]  Yan Su,et al.  Shape Generation of Grid Structures by Inverse Hanging Method Coupled with Multiobjective Optimization , 2018, Comput. Aided Civ. Infrastructure Eng..

[77]  Michael G. Epitropakis,et al.  HyperSPAM: A study on hyper-heuristic coordination strategies in the continuous domain , 2019, Inf. Sci..

[78]  Ponnuthurai N. Suganthan,et al.  Ensemble differential evolution algorithm for CEC2011 problems , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[79]  Yicheng Li,et al.  A Fast Detection Method via Region‐Based Fully Convolutional Neural Networks for Shield Tunnel Lining Defects , 2018, Comput. Aided Civ. Infrastructure Eng..

[80]  T. Mackie,et al.  Fast free-form deformable registration via calculus of variations , 2004, Physics in medicine and biology.

[81]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[82]  Yiteng Pan,et al.  A novel region-based active contour model via local patch similarity measure for image segmentation , 2018, Multimedia Tools and Applications.

[83]  Qingfu Zhang,et al.  Decomposition-Based Multiobjective Evolutionary Algorithm With an Ensemble of Neighborhood Sizes , 2012, IEEE Transactions on Evolutionary Computation.

[84]  A. Kaveh,et al.  A novel heuristic optimization method: charged system search , 2010 .

[85]  Georgios Dounias,et al.  Evolutionary computation for resource leveling optimization in project management , 2016, Integr. Comput. Aided Eng..

[86]  Maria do Carmo Nicoletti,et al.  A hardware oriented ad-hoc computer-based method for binary structuring element decomposition based on genetic algorithms , 2016, Integr. Comput. Aided Eng..

[87]  Abdolreza Hatamlou,et al.  Black hole: A new heuristic optimization approach for data clustering , 2013, Inf. Sci..

[88]  Yew-Soon Ong,et al.  Multifactorial Evolution: Toward Evolutionary Multitasking , 2016, IEEE Transactions on Evolutionary Computation.

[89]  Carlos Cotta,et al.  Memetic algorithms and memetic computing optimization: A literature review , 2012, Swarm Evol. Comput..

[90]  Marco Dorigo,et al.  The ant colony optimization meta-heuristic , 1999 .

[91]  Ye Tian,et al.  A region division based diversity maintaining approach for many-objective optimization , 2017, Integr. Comput. Aided Eng..

[92]  Wei Tang,et al.  Ensembling neural networks: Many could be better than all , 2002, Artif. Intell..