Optimization of the ISP Parameters of a Camera Through Differential Evolution

Within the design and development of a smartphone, an important phase arises regarding time, which is related to the tuning of the ISP (image signal processor) of the camera. The ISP is an element that allows the adjustment of the images captured by a sensor in order to achieve the best image quality. The ISP implements different image improvement algorithms such as white balancing, denoising, and demosaicing as well as other image enhancement algorithms. The purpose of the ISP tuning process is to configure the parameters of these algorithms so that the processed images are of the highest quality. This task is carried out by the camera tuning engineer, who iteratively adjusts the ISP parameters through trial and error procedures until the desired quality is achieved. The complete adjustment process can be extended to several weeks and even months. The authors present a novel solution based on differential evolution, which allows a first-adjusted approximation of the ISP in a few hours. This work presents an architecture based on an optimization through a differential evolution algorithm with which different ISP tuning tests are carried out, and the good results in quality and time are verified.

[1]  Scott H. Hawley,et al.  Visualizing Sound Directivity via Smartphone Sensors , 2018 .

[2]  Ali Wagdy Mohamed,et al.  Differential Evolution (DE): A Short Review , 2017, ICRA 2017.

[3]  Emmanuel S. Boss,et al.  The HydroColor App: Above Water Measurements of Remote Sensing Reflectance and Turbidity Using a Smartphone Camera , 2018, Sensors.

[4]  Wilfrido Gómez-Flores,et al.  On the selection of surrogate models in evolutionary optimization algorithms , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[5]  Clément Viard,et al.  Image quality benchmark of computational bokeh , 2018 .

[6]  Yufei Chen,et al.  Performance Analysis of Smartphone-Sensor Behavior for Human Activity Recognition , 2017, IEEE Access.

[7]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[8]  Gang Hua,et al.  High Quality Image Processing System for ADAS , 2019, 2019 IEEE International Conference on Electronics, Computing and Communication Technologies (CONECCT).

[9]  Natalia Chandra,et al.  Implementation of Gyroscope Sensor to Presentation Application on Android Smartphone , 2018, 2018 Indonesian Association for Pattern Recognition International Conference (INAPR).

[10]  Jian Lu,et al.  Assessing User Mental Workload for Smartphone Applications With Built-In Sensors , 2019, IEEE Pervasive Computing.

[11]  M. Shamim Hossain,et al.  An Emotion Recognition System for Mobile Applications , 2017, IEEE Access.

[12]  Chao Cai,et al.  Location-Based Augmented Reality With Pervasive Smartphone Sensors: Inside and Beyond Pokemon Go! , 2017, IEEE Access.

[13]  Ville Tirronen,et al.  Recent advances in differential evolution: a survey and experimental analysis , 2010, Artificial Intelligence Review.

[14]  Gary Tse,et al.  Resting and Postexercise Heart Rate Detection From Fingertip and Facial Photoplethysmography Using a Smartphone Camera: A Validation Study , 2017, JMIR mHealth and uHealth.

[15]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[16]  Chyuan-Tyng Wu,et al.  Automatic ISP Image Quality Tuning Using Nonlinear Optimization , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[17]  Marjan Mernik,et al.  Exploration and exploitation in evolutionary algorithms: A survey , 2013, CSUR.

[18]  Amit Konar,et al.  Differential Evolution Using a Neighborhood-Based Mutation Operator , 2009, IEEE Transactions on Evolutionary Computation.

[19]  Senthil Yogamani,et al.  Overview and Empirical Analysis of ISP Parameter Tuning for Visual Perception in Autonomous Driving , 2019, J. Imaging.

[20]  Glenn D. Boreman,et al.  Modulation Transfer Function in Optical and Electro-Optical Systems , 2001 .

[21]  Ponnuthurai N. Suganthan,et al.  Recent advances in differential evolution - An updated survey , 2016, Swarm Evol. Comput..

[22]  Ke Wang,et al.  AI Benchmark: Running Deep Neural Networks on Android Smartphones , 2018, ECCV Workshops.

[23]  Jason Brownlee,et al.  Clever Algorithms: Nature-Inspired Programming Recipes , 2012 .

[24]  R. Storn,et al.  Differential Evolution - A simple and efficient adaptive scheme for global optimization over continuous spaces , 2004 .

[25]  Ahmad Almogren,et al.  A robust human activity recognition system using smartphone sensors and deep learning , 2018, Future Gener. Comput. Syst..