Preventing Model Overfitting and Underfitting in Convolutional Neural Networks

Thecurrentdiscourse in themachine learningdomainconverges to theagreement thatmachine learningmethodsemergedassomeofthemostprominentlearningandclassificationapproachesover thepastdecade.TheCNNbecameoneofmostactivelyresearchedandbroadly-applieddeepmachine learningmethods.However,thetrainingsethasalargeinfluenceontheaccuracyofanetworkanditis paramounttocreateanarchitecturethatsupportsitsmaximumtrainingandrecognitionperformance. Theproblemconsideredinthisarticleishowtopreventoverfittingandunderfitting.Thedeficiencies areaddressedbycomparingthestatisticsofCNNimagerecognitionalgorithmstotheIsingmodel. Usingatwo-dimensionalsquare-latticearray,theimpactthatthelearningrateandregularizationrate parametershaveontheadaptabilityofCNNsforimageclassificationareevaluated.Theobtained resultscontributetoabettertheoreticalunderstandingofaCNNandprovideconcreteguidanceon preventingmodeloverfittingandunderfittingwhenaCNNisappliedforimagerecognitiontasks. KeywORdS Cognitive Systems, Convolutional Neural Networks, Image Processing, Ising Model, Learning Rate, Machine Learning, Overfitting, Regularization Rate, Underfitting

[1]  Yingxu Wang,et al.  Kinect Sensor Gesture and Activity Recognition: New Applications for Consumer Cognitive Systems , 2018, IEEE Consumer Electronics Magazine.

[2]  Marina L. Gavrilova,et al.  Facial Metamorphosis Using Geometrical Methods for Biometric Applications , 2008, Int. J. Pattern Recognit. Artif. Intell..

[3]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[4]  Saeed Shiry Ghidary,et al.  Convolutional Neural Networks for Image Processing: An Application in Robot Vision , 2003, Australian Conference on Artificial Intelligence.

[5]  Jeffrey M. Hausdorff,et al.  Gait and Cognition: A Complementary Approach to Understanding Brain Function and the Risk of Falling , 2012, Journal of the American Geriatrics Society.

[6]  Yingxu Wang,et al.  A Formal Knowledge Representation System (FKRS) for the Intelligent Knowledge Base of a Cognitive Learning Engine , 2011, Int. J. Softw. Sci. Comput. Intell..

[7]  Yingxu Wang,et al.  A Formal Knowledge Retrieval System for Cognitive Computers and Cognitive Robotics , 2013, Int. J. Softw. Sci. Comput. Intell..

[8]  Shusaku Tsumoto,et al.  Perspectives on Cognitive Computers and Knowledge Processors , 2013, Int. J. Cogn. Informatics Nat. Intell..

[9]  E. Ising.  Beitrag zur Theorie des Ferromagnetismus , 1925 .

[10]  Michael Unser,et al.  Convolutional Neural Networks for Inverse Problems in Imaging: A Review , 2017, IEEE Signal Processing Magazine.

[11]  Bir Bhanu,et al.  Individual recognition using gait energy image , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  R. Glauber.  Time‐Dependent Statistics of the Ising Model , 1963 .

[13]  Yingxu Wang,et al.  Cognitive Informatics: Towards Cognitive Machine Learning and Autonomous Knowledge Manipulation , 2018, Int. J. Cogn. Informatics Nat. Intell..

[14]  Yingxu Wang,et al.  A novel fuzzy multimodal information fusion technology for human biometric traits identification , 2011, IEEE 10th International Conference on Cognitive Informatics and Cognitive Computing (ICCI-CC'11).

[15]  Sotiris B. Kotsiantis,et al.  Machine learning: a review of classification and combining techniques , 2006, Artificial Intelligence Review.

[16]  Andrei Dmitri Gavrilov,et al.  Convolutional Neural Networks: Estimating Relations in the Ising Model on Overfitting , 2018, 2018 IEEE 17th International Conference on Cognitive Informatics & Cognitive Computing (ICCI*CC).

[17]  Chin-Teng Lin,et al.  Neural-Network-Based Fuzzy Logic Control and Decision System , 1991, IEEE Trans. Computers.

[18]  Yingxu Wang,et al.  Cognitive Intelligence: Deep Learning, Thinking, and Reasoning by Brain-Inspired Systems , 2016, Int. J. Cogn. Informatics Nat. Intell..