A parameter-optimizing model-based approach to the analysis of low-SNR image sequences for biological virus detection

This thesis presents the multi-objective parameter optimization of a novel image analysis process. The focus of application is automatic detection of nano-objects, for example biological viruses, in real-time. Nano-objects are detected by analyzing time series of images recorded with the PAMONO biosensor, after parameters have been optimized on synthetic data created by a signal model for PAMONO. PAMONO, which is short for Plasmon-Assisted Microscopy of Nano-Sized Objects, is a biosensor yielding indirect proofs for objects on the nanometer-scale by measuring the Surface Plasmon Resonance (SPR) effects they cause on the micrometer scale. It is an optical microscopy technique enabling the detection of biological viruses and other nano-objects within a portable device. The PAMONO biosensor produces time series of 2-D images on the order of 4000 half-megapixel images per experiment. A particular challenge for automatic analysis of this data emerges from its low Signal-to-Noise Ratio (SNR). Manual analysis takes approximately two days per experiment and analyzing person. With the automatic analysis process developed in this thesis, occurrences of nano-objects in PAMONO data can be counted and displayed in real-time while measurements are being taken. Analysis is divided into a GPU-based detector aiming at high sensitivity, complemented with a machine learning-based classifier aiming at high precision. The analysis process is embedded into a multi-objective optimization approach that automatically adapts algorithm choice and parameters to changes in physical sensor parameters. Such changes occur, for example, during sensor prototype development. In order to automatically evaluate the objectives undergoing optimization, a signal model for the PAMONO sensor is proposed, which serves to synthesize ground truth-annotated data. The parameters of the analysis process are optimized on this synthetic data, and the classifier is learned from it. Hence, the signal model must accurately mimic the data recorded by the sensor, which is achieved by incorporating real sensor data into synthesis. Both, optimized parameters and the learned classifier, achieve high quality results on the real sensor data to be analyzed: Nano-objects with diameters down to 100 nm are detected reliably in PAMONO data. Note that the median SNR over all nano-objects to be detected was below two in the examined experiments with 100 nm objects. While the presented analysis process can be used for real-time virus detection in PAMONO data, the optimization approach can serve in accelerating the advancement of the sensor prototype towards a final setup of its physical parameters: In this scenario, frequent changes in physical sensor parameters make the automatic adaptation of algorithmic process parameters a desirable goal. No expertise concerning the underlying algorithms is required in these use cases, enabling ready applicability in a lab scenario.

[1]  S. Baskar,et al.  NSGA-II algorithm for multi-objective generation expansion planning problem , 2009 .

[2]  W. James MacLean,et al.  CCD noise removal in digital images , 2006, IEEE Transactions on Image Processing.

[3]  Fionn Murtagh,et al.  Image Processing and Data Analysis - The Multiscale Approach , 1998 .

[4]  Gunnar Rätsch,et al.  An introduction to kernel-based learning algorithms , 2001, IEEE Trans. Neural Networks.

[5]  M Reyes Sierra,et al.  Multi-Objective Particle Swarm Optimizers: A Survey of the State-of-the-Art , 2006 .

[6]  Jason Weston,et al.  Multi-Class Support Vector Machines , 1998 .

[7]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[8]  High confidence AGN candidates among unidentified Fermi-LAT sources via statistical classification , 2013, 1306.6529.

[9]  Zelin Shi,et al.  Redundant DWT based translation invariant wavelet feature extraction for face recognition , 2008, 2008 19th International Conference on Pattern Recognition.

[10]  K. Überla,et al.  Real-time Detection of Single Immobilized Nanoparticles by Surface Plasmon Resonance Imaging , 2010 .

[11]  David A. Anderson,et al.  Virus-like particles: Passport to immune recognition , 2006, Methods.

[12]  Tobias Voigt,et al.  Gamma-Hadron-Separation in the MAGIC Experiment , 2012, GfKl.

[13]  Hirotaka Nakayama,et al.  Meta-Modeling in Multiobjective Optimization , 2008, Multiobjective Optimization.

[14]  G. Krishna,et al.  The condensed nearest neighbor rule using the concept of mutual nearest neighborhood (Corresp.) , 1979, IEEE Trans. Inf. Theory.

[15]  Bernd Bischl,et al.  Resampling Methods for Meta-Model Validation with Recommendations for Evolutionary Computation , 2012, Evolutionary Computation.

[16]  Gabriel Landini,et al.  Radicular cysts and odontogenic keratocysts epithelia classification using cascaded Haar classifiers , 2008 .

[17]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[18]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[19]  김찬홍,et al.  다중물리 해석 프로그램 : COMSOL Multiphysics , 2006 .

[20]  Tobias Scheffer,et al.  Unbiased assessment of learning algorithms , 1997, IJCAI 1997.

[21]  S. Baskar,et al.  Application of NSGA-II Algorithm to Generation Expansion Planning , 2009, IEEE Transactions on Power Systems.

[22]  Carl Tim Kelley,et al.  Iterative methods for optimization , 1999, Frontiers in applied mathematics.

[23]  Heike Trautmann,et al.  On the distribution of the desirability index using Harrington’s desirability function , 2006 .

[24]  H. Kundel,et al.  Measurement of observer agreement. , 2003, Radiology.

[25]  Daniel Kondermann,et al.  Is Crowdsourcing for Optical Flow Ground Truth Generation Feasible? , 2013, ICVS.

[26]  Desire L. Massart,et al.  Simultaneous optimization of several chromatographic performance goals using Derringer's desirability function , 1991 .

[27]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[28]  Heike Trautmann,et al.  Integration of Expert's Preferences in Pareto Optimization by Desirability Function Techniques , 2006 .

[29]  S. Mallat A wavelet tour of signal processing , 1998 .

[30]  Tristan Fletcher,et al.  Support Vector Machines Explained , 2008 .

[31]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[32]  Kalyanmoy Deb,et al.  Mechanical Component Design for Multiple Objectives Using Elitist Non-dominated Sorting GA , 2000, PPSN.

[33]  Friedrich Wahl Digitale Bildsignalverarbeitung : Grundlagen, Verfahren, Beispiele , 1989 .

[34]  Wouter Duivesteijn,et al.  Understanding Where Your Classifier Does ( Not ) Work — the SCaPE Model Class for Exceptional Model Mining Te ch ni ca lR ep or t , 2014 .

[35]  Wiro J. Niessen,et al.  Quantitative comparison of spot detection methods in live-cell fluorescence microscopy imaging , 2009, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[36]  Rafael C. González,et al.  Digital image processing, 3rd Edition , 2008 .

[37]  Yung-Hsiang Lu,et al.  Cloud Computing for Mobile Users: Can Offloading Computation Save Energy? , 2010, Computer.

[38]  Xinping Huang,et al.  Label-free imaging, detection, and mass measurement of single viruses by surface plasmon resonance , 2010, Proceedings of the National Academy of Sciences.

[39]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[40]  Ujjwal Maulik,et al.  Survey of Multiobjective Evolutionary Algorithms for Data Mining: Part II , 2014, IEEE Transactions on Evolutionary Computation.

[41]  Santosh K. Gupta,et al.  Multi-objective optimization of reverse osmosis desalination units using different adaptations of the non-dominated sorting genetic algorithm (NSGA) , 2005, Comput. Chem. Eng..

[42]  Bernard Branson,et al.  A rapid review of rapid HIV antibody tests , 2006, Current infectious disease reports.

[43]  John C. Russ,et al.  The Image Processing Handbook , 2016, Microscopy and Microanalysis.

[44]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Comparing Review , 2006, Towards a New Evolutionary Computation.

[45]  M. Engel,et al.  Signal Analysis and Classification for Surface Plasmon Assisted Microscopy of Nanoobjects , 2010 .

[46]  I. Johnstone,et al.  Wavelet Threshold Estimators for Data with Correlated Noise , 1997 .

[47]  J. .. Woehl,et al.  Realistic modeling of the illumination point spread function in confocal scanning optical microscopy. , 2010, Journal of the Optical Society of America. A, Optics, image science, and vision.

[48]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[49]  Sariel Har-Peled,et al.  On coresets for k-means and k-median clustering , 2004, STOC '04.

[50]  Qiang Wu,et al.  Microscope Image Processing , 2010 .

[51]  A. Kak,et al.  Constrained least squares filtering , 1977 .

[52]  Jeon-Hor Chen,et al.  Computer-Aided Tumor Detection Based on Multi-Scale Blob Detection Algorithm in Automated Breast Ultrasound Images , 2013, IEEE Transactions on Medical Imaging.

[53]  Y. Blanter,et al.  Shot noise in mesoscopic conductors , 1999, cond-mat/9910158.

[54]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[55]  Kalyanmoy Deb,et al.  An Improved Adaptive Approach for Elitist Nondominated Sorting Genetic Algorithm for Many-Objective Optimization , 2013, EMO.

[56]  Robert LIN,et al.  NOTE ON FUZZY SETS , 2014 .

[57]  Andy Liaw,et al.  Classification and Regression by randomForest , 2007 .

[58]  T. Lindeberg Scale-Space Theory : A Basic Tool for Analysing Structures at Different Scales , 1994 .

[59]  David Erickson,et al.  Nanobiosensors: optofluidic, electrical and mechanical approaches to biomolecular detection at the nanoscale , 2008, Microfluidics and nanofluidics.

[60]  Richard M. Karp,et al.  A n^5/2 Algorithm for Maximum Matchings in Bipartite Graphs , 1971, SWAT.

[61]  Stan Szpakowicz,et al.  Beyond Accuracy, F-Score and ROC: A Family of Discriminant Measures for Performance Evaluation , 2006, Australian Conference on Artificial Intelligence.

[62]  Frank Weichert,et al.  Mobile Detektion viraler Pathogene durch echtzeitfähige GPGPU-Fuzzy-Segmentierung , 2013, Bildverarbeitung für die Medizin.

[63]  Haibo He,et al.  ADASYN: Adaptive synthetic sampling approach for imbalanced learning , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[64]  S. Hell,et al.  Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy. , 1994, Optics letters.

[65]  Camille Couprie,et al.  Learning Hierarchical Features for Scene Labeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[66]  Pablo Moscato,et al.  On Evolution, Search, Optimization, Genetic Algorithms and Martial Arts : Towards Memetic Algorithms , 1989 .

[67]  Thomas Stützle,et al.  F-Race and Iterated F-Race: An Overview , 2010, Experimental Methods for the Analysis of Optimization Algorithms.

[68]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[69]  Lothar Thiele,et al.  Comparison of Multiobjective Evolutionary Algorithms: Empirical Results , 2000, Evolutionary Computation.

[70]  Mohamed-Jalal Fadili,et al.  Multiscale Variance-Stabilizing Transform for Mixed-Poisson-Gaussian Processes and its Applications in Bioimaging , 2007, 2007 IEEE International Conference on Image Processing.

[71]  Haibo He,et al.  Learning from Imbalanced Data , 2009, IEEE Transactions on Knowledge and Data Engineering.

[72]  Pedro M. Domingos MetaCost: a general method for making classifiers cost-sensitive , 1999, KDD '99.

[73]  Yoshua Bengio,et al.  An empirical evaluation of deep architectures on problems with many factors of variation , 2007, ICML '07.

[74]  Marco Laumanns,et al.  SPEA2: Improving the strength pareto evolutionary algorithm , 2001 .

[75]  Peter Marwedel,et al.  Multi-objective, Energy-Aware GPGPU Design Space Exploration for Medical or Industrial Applications , 2014, 2014 Tenth International Conference on Signal-Image Technology and Internet-Based Systems.

[76]  George B. Dantzig,et al.  Linear programming and extensions , 1965 .

[77]  Peter Norvig,et al.  Artificial intelligence - a modern approach, 2nd Edition , 2003, Prentice Hall series in artificial intelligence.

[78]  Christian Sohler,et al.  Signal/Background Classification of Time Series for Biological Virus Detection , 2014, GCPR.

[79]  Kalyanmoy Deb,et al.  U-NSGA-III : A Unified Evolutionary Algorithm for Single , Multiple , and Many-Objective Optimization , 2014 .

[80]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[81]  Fionn Murtagh,et al.  Multiresolution in astronomical image processing: A general framework , 1995, Int. J. Imaging Syst. Technol..

[82]  Priyabrata Pattnaik,et al.  Surface plasmon resonance , 2005, Applied biochemistry and biotechnology.

[83]  Obias,et al.  DSEA : A Data Mining Approach to Unfolding , 2013 .

[84]  Frank Weichert,et al.  Fuzzy-enhanced, Real-time Capable Detection of Biological Viruses using a Portable Biosensor , 2013, BIOSIGNALS.

[85]  Andrew W. Moore,et al.  The Racing Algorithm: Model Selection for Lazy Learners , 1997, Artificial Intelligence Review.

[86]  Seán McLoone,et al.  Organic Acid Prediction in Biogas Plants Using UV/vis Spectroscopic Online-Measurements , 2010, ICSEE 2010.

[87]  Peter Ertl,et al.  Microfluidic Systems for Pathogen Sensing: A Review , 2009, Sensors.

[88]  Christian Sohler,et al.  BICO: BIRCH Meets Coresets for k-Means Clustering , 2013, ESA.

[89]  O. Blanch,et al.  Monte Carlo simulation for the MAGIC telescope , 2005 .

[90]  G. Mie Beiträge zur Optik trüber Medien, speziell kolloidaler Metallösungen , 1908 .

[91]  Katarzyna Radecka,et al.  Automated diagnosis of knee pathology using sensory data , 2014, 2014 4th International Conference on Wireless Mobile Communication and Healthcare - Transforming Healthcare Through Innovations in Mobile and Wireless Technologies (MOBIHEALTH).

[92]  Jian-Jia Chen,et al.  Computation Offloading for Frame-Based Real-Time Tasks with Resource Reservation Servers , 2013, 2013 25th Euromicro Conference on Real-Time Systems.

[93]  Seyed Taghi Akhavan Niaki,et al.  Multi-response simulation optimization using genetic algorithm within desirability function framework , 2006, Appl. Math. Comput..

[94]  John W. Nicklow,et al.  Multi-objective automatic calibration of SWAT using NSGA-II , 2007 .

[95]  Evgeny L. Gurevich,et al.  Analytical features of particle counting sensor based on plasmon assisted microscopy of nano objects , 2011 .

[96]  J. F. Jarvis,et al.  Focas-Faint Object Classification And Analysis System , 1979, Other Conferences.

[97]  D. Hunter Valuation of mortgage-backed securities using Brownian bridges to reduce effective dimension , 2000 .

[98]  Zhengyou Zhang,et al.  Microsoft Kinect Sensor and Its Effect , 2012, IEEE Multim..

[99]  J. Tukey,et al.  Variations of Box Plots , 1978 .

[100]  Chih-Jen Lin,et al.  A Practical Guide to Support Vector Classication , 2008 .

[101]  Kalyanmoy Deb,et al.  Dynamic Multi-objective Optimization and Decision-Making Using Modified NSGA-II: A Case Study on Hydro-thermal Power Scheduling , 2007, EMO.

[102]  Kalyanmoy Deb,et al.  U-NSGA-III: A Unified Evolutionary Optimization Procedure for Single, Multiple, and Many Objectives: Proof-of-Principle Results , 2015, EMO.

[103]  Frank Weichert,et al.  Parameteroptimierte und GPGPU-basierte Detektion viraler Strukturen innerhalb Plasmonen-unterstützter Mikroskopiedaten , 2012, Bildverarbeitung für die Medizin.

[104]  Etienne E. Kerre,et al.  Fuzzy Random Impulse Noise Removal From Color Image Sequences , 2011, IEEE Transactions on Image Processing.

[105]  Thomas Bartz-Beielstein,et al.  Tuned data mining: a benchmark study on different tuners , 2011, GECCO '11.

[106]  G. Rudolph Evolutionary Search under Partially Ordered Fitness Sets , 2001 .

[107]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[108]  H. Trautmann,et al.  Preference-based Pareto optimization in certain and noisy environments , 2009 .

[109]  Ajay K. Ray,et al.  Design stage optimization of an industrial low-density polyethylene tubular reactor for multiple objectives using NSGA-II and its jumping gene adaptations , 2007 .

[110]  Joachim Hornegger,et al.  Wavelet denoising of multiframe optical coherence tomography data , 2012, Biomedical optics express.

[111]  J. Pyun,et al.  Application of SPR biosensor for medical diagnostics of human hepatitis B virus (hHBV) , 2005 .

[112]  T. Kanade,et al.  Learning to Detect Different Types of Cells under Phase Contrast Microscopy , 2009 .

[113]  Stephen T. C. Wong,et al.  Detection of molecular particles in live cells via machine learning , 2007, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[114]  Irina Rish,et al.  An empirical study of the naive Bayes classifier , 2001 .

[115]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[116]  In-Jun Jeong,et al.  An interactive desirability function method to multiresponse optimization , 2009, Eur. J. Oper. Res..

[117]  Victoria Shpacovitch,et al.  Optimal conditions for SPR-imaging of nano-objects , 2017 .

[118]  Ashutosh Tiwari,et al.  Introducing user preference using Desirability Functions in Multi-Objective Evolutionary Optimisation of noisy processes , 2007, 2007 IEEE Congress on Evolutionary Computation.

[119]  Constantin Timm,et al.  Resource efficient processing and communication in sensor/actuator environments , 2012 .

[120]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[121]  Michael J. Pazzani,et al.  Reducing Misclassification Costs , 1994, ICML.

[122]  Christian Kirches,et al.  Mixed-integer nonlinear optimization*† , 2013, Acta Numerica.

[123]  D. Altman,et al.  Statistics Notes: Diagnostic tests 2: predictive values , 1994, BMJ.

[124]  Takeo Kanade,et al.  Cell segmentation in microscopy imagery using a bag of local Bayesian classifiers , 2010, 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[125]  Jean-Christophe Olivo-Marin,et al.  Extraction of spots in biological images using multiscale products , 2002, Pattern Recognit..

[126]  Tilmann Bruckhaus The Business Impact of Predictive Analytics , 2007 .

[127]  Erik G. Learned-Miller,et al.  Data driven image models through continuous joint alignment , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[128]  G. Derringer,et al.  Simultaneous Optimization of Several Response Variables , 1980 .

[129]  M. Tahar Kechadi,et al.  Multi-objective feature selection by using NSGA-II for customer churn prediction in telecommunications , 2010, Expert Syst. Appl..

[130]  Daniel Heim,et al.  Detection and Segmentation of Cell Nuclei in Virtual Microscopy Images: A Minimum-Model Approach , 2012, Scientific Reports.

[131]  Thomas Stützle,et al.  A Racing Algorithm for Configuring Metaheuristics , 2002, GECCO.

[132]  Ful-Chiang Wu,et al.  Optimization of Correlated Multiple Quality Characteristics Using Desirability Function , 2004 .

[133]  Thomas Bartz-Beielstein,et al.  The Sequential Parameter Optimization Toolbox , 2010, Experimental Methods for the Analysis of Optimization Algorithms.

[134]  Chih-Jen Lin,et al.  Asymptotic Behaviors of Support Vector Machines with Gaussian Kernel , 2003, Neural Computation.

[135]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[136]  Sergei Vassilvitskii,et al.  k-means++: the advantages of careful seeding , 2007, SODA '07.

[137]  Hui Han,et al.  Borderline-SMOTE: A New Over-Sampling Method in Imbalanced Data Sets Learning , 2005, ICIC.

[138]  Peter Marwedel,et al.  Modellierung und Optimierung eines Biosensors zur Detektion viraler Strukturen , 2014, Bildverarbeitung für die Medizin.

[139]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[140]  H. Rebel,et al.  The Karlsruhe extensive air shower simulation code CORSIKA. , 1992 .

[141]  Ming-Kuei Hu,et al.  Visual pattern recognition by moment invariants , 1962, IRE Trans. Inf. Theory.

[142]  Hsuan-Tien Lin A Study on Sigmoid Kernels for SVM and the Training of non-PSD Kernels by SMO-type Methods , 2005 .

[143]  J. Moros,et al.  Adaptive approach for variable noise suppression on laser-induced breakdown spectroscopy responses using stationary wavelet transform. , 2012, Analytica chimica acta.

[144]  G. Landini Quantitative analysis of the epithelial lining architecture in radicular cysts and odontogenic keratocysts , 2006, Head & face medicine.

[145]  Adhemar Bultheel,et al.  Multiple wavelet threshold estimation by generalized cross validation for images with correlated noise , 1999, IEEE Trans. Image Process..

[146]  Marco Dorigo,et al.  Ant colony optimization theory: A survey , 2005, Theor. Comput. Sci..

[147]  Christian Wietfeld,et al.  Multi-objective computation offloading for mobile biosensors via LTE , 2014, 2014 4th International Conference on Wireless Mobile Communication and Healthcare - Transforming Healthcare Through Innovations in Mobile and Wireless Technologies (MOBIHEALTH).

[148]  Y. Shirshov,et al.  Detection of plant viruses using a surface plasmon resonance via complexing with specific antibodies. , 2004, Journal of virological methods.

[149]  R. K. Ursem Multi-objective Optimization using Evolutionary Algorithms , 2009 .

[150]  David C. Banks,et al.  Counting cases in marching cubes: toward a generic algorithm for producing substitopes , 2003, IEEE Visualization, 2003. VIS 2003..

[151]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[152]  Nicola Beume,et al.  SMS-EMOA: Multiobjective selection based on dominated hypervolume , 2007, Eur. J. Oper. Res..

[153]  Yoshua Bengio,et al.  Algorithms for Hyper-Parameter Optimization , 2011, NIPS.

[154]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[155]  P. Sorger,et al.  Automatic fluorescent tag detection in 3D with super‐resolution: application to the analysis of chromosome movement , 2002, Journal of microscopy.

[156]  Shixiong Xia,et al.  An Improved KNN Text Classification Algorithm Based on Clustering , 2009, J. Comput..

[157]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[158]  K. Niemax,et al.  Enhancement of the detection power of surface plasmon resonance measurements by optimization of the reflection angle. , 2007, Analytical chemistry.

[159]  Michael D. Buhrmester,et al.  Amazon's Mechanical Turk , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[160]  Fabrizio Angiulli,et al.  Fast condensed nearest neighbor rule , 2005, ICML.

[161]  David Corne,et al.  The Pareto archived evolution strategy: a new baseline algorithm for Pareto multiobjective optimisation , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[162]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[163]  Ethem Alpaydin,et al.  Voting over Multiple Condensed Nearest Neighbors , 1997, Artificial Intelligence Review.

[164]  Kevin Leyton-Brown,et al.  Sequential Model-Based Optimization for General Algorithm Configuration , 2011, LION.

[165]  Andrew Zisserman,et al.  Learning to Detect Cells Using Non-overlapping Extremal Regions , 2012, MICCAI.

[166]  Richard J. Beckman,et al.  A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output From a Computer Code , 2000, Technometrics.

[167]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[168]  Nancy Chinchor,et al.  MUC-4 evaluation metrics , 1992, MUC.

[169]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[170]  Robert M. Haralick,et al.  Feature normalization and likelihood-based similarity measures for image retrieval , 2001, Pattern Recognit. Lett..

[171]  Zhi-Hua Zhou,et al.  Exploratory Undersampling for Class-Imbalance Learning , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[172]  Michael Engel,et al.  Plasmon-based Virus Detection on Heterogeneous Embedded Systems , 2015, SCOPES.

[173]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[174]  Xiaoou Tang,et al.  Translation-Invariant Face Feature Estimation Using Discrete Wavelet Transform , 2001, WAA.

[175]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[176]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[177]  Carlos Ansótegui,et al.  A Gender-Based Genetic Algorithm for the Automatic Configuration of Algorithms , 2009, CP.

[178]  Guy Lapalme,et al.  A systematic analysis of performance measures for classification tasks , 2009, Inf. Process. Manag..

[179]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[180]  Lothar Thiele,et al.  The Hypervolume Indicator Revisited: On the Design of Pareto-compliant Indicators Via Weighted Integration , 2007, EMO.

[181]  M K Cheezum,et al.  Quantitative comparison of algorithms for tracking single fluorescent particles. , 2001, Biophysical journal.

[182]  Ludmila I. Kuncheva,et al.  A stability index for feature selection , 2007, Artificial Intelligence and Applications.

[183]  Toby P. Breckon,et al.  The application of support vector machine classification to detect cell nuclei for automated microscopy , 2010, Machine Vision and Applications.

[184]  Luc Vincent,et al.  Watersheds in Digital Spaces: An Efficient Algorithm Based on Immersion Simulations , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[185]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[186]  Thomas Hofmann,et al.  Support vector machine learning for interdependent and structured output spaces , 2004, ICML.

[187]  A. Hillas Cerenkov light images of EAS produced by primary gamma , 1985 .

[188]  Ujjwal Maulik,et al.  A Survey of Multiobjective Evolutionary Algorithms for Data Mining: Part I , 2014, IEEE Transactions on Evolutionary Computation.