Towards Hardware Acceleration of Neuroevolution for Multimedia Processing Applications on Mobile Devices

This paper addresses the problem of accelerating large artificial neural networks (ANN), whose topology and weights can evolve via the use of a genetic algorithm. The proposed digital hardware architecture is capable of processing any evolved network topology, whilst at the same time providing a good trade off between throughput, area and power consumption. The latter is vital for a longer battery life on mobile devices. The architecture uses multiple parallel arithmetic units in each processing element (PE). Memory partitioning and data caching are used to minimise the effects of PE pipeline stalling. A first order minimax polynomial approximation scheme, tuned via a genetic algorithm, is used for the activation function generator. Efficient arithmetic circuitry, which leverages modified Booth recoding, column compressors and carry save adders, is adopted throughout the design.

[1]  David A. Patterson,et al.  Computer Architecture: A Quantitative Approach , 1969 .

[2]  Brian R. Gaines,et al.  Stochastic Computing Systems , 1969 .

[3]  Jenq-Neng Hwang,et al.  A Unified Systolic Architecture for Artificial Neural Networks , 1989, J. Parallel Distributed Comput..

[4]  Jenq-Neng Hwang,et al.  Finite Precision Error Analysis of Neural Network Hardware Implementations , 1993, IEEE Trans. Computers.

[5]  I. Koren Computer arithmetic algorithms , 2018 .

[6]  Bernard Widrow,et al.  Neural networks: applications in industry, business and science , 1994, CACM.

[7]  Noel E. O'Connor,et al.  Object detection and tracking using an EM-based motion estimation and segmentation framework , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[8]  Noel E. O'Connor,et al.  Supervised Semantic Object Segmentation and Tracking via EM-based Estimation of Mixture Density Parameters , 1998, NMBIA.

[9]  Noel E. O'Connor,et al.  The DICEMAN description schemes for still images and video sequences , 1999 .

[10]  Noel E. O'Connor,et al.  Hierarchical visual description schemes for still images and video sequences , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[11]  Amos Omondi Neurocomputers: A Dead End? , 2000, Int. J. Neural Syst..

[12]  Noel E. O'Connor,et al.  Description schemes for video programs, users and devices , 2000, Signal Process. Image Commun..

[13]  Noel E. O'Connor,et al.  Evaluating and combining digital video shot boundary detection algorithms , 2000 .

[14]  Alan F. Smeaton,et al.  The Fischlar Digital Video Recording, Analysis and Browsing System , 2000, RIAO.

[15]  Alan F. Smeaton,et al.  Implementation and Analysis of Several Keyframe-Based Browsing Interfaces to Digital Video , 2000, ECDL.

[16]  Noel E. O'Connor,et al.  Supervised object segmentation and tracking for MPEG-4 VOP generation , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[17]  Noel E. O'Connor,et al.  Automatically detecting camera motion from MPEG-1 encoded video , 2000 .

[18]  P. Howarth,et al.  RECENT ADVANCES IN IMAGE AND VIDEO RETRIEVAL Robust texture features for still-image retrieval , 2000 .

[19]  Alan F. Smeaton,et al.  Fischlar: an on-line system for indexing and browsing broadcast television content , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[20]  Alan F. Smeaton,et al.  The físchlár digital video system: a digital library of broadcast TV programmes , 2001, JCDL '01.

[21]  Noel E. O'Connor,et al.  Speech-music discrimination from MPEG-1 bitstream , 2001 .

[22]  Noel E. O'Connor,et al.  User interface design for keyframe-based browsing of digital video , 2001 .

[23]  N. O'Connor,et al.  SEMI-AUTOMATIC VIDEO OBJECT SEGMENTATION USING RECURSIVE SHORTEST SPANNING TREE AND BINARY PARTITION TREE , 2001 .

[24]  Alan F. Smeaton,et al.  News story segmentation in the Fischlar video indexing system , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[25]  Noel E. O'Connor,et al.  Audio and video processing for automatic TV advertisement detection , 2001 .

[26]  Noel E. O'Connor,et al.  An automatic technique for visual quality classification for MPEG-1 video , 2001 .

[27]  Howard C. Card,et al.  Stochastic Neural Computation I: Computational Elements , 2001, IEEE Trans. Computers.

[28]  N. O'Connor,et al.  Rhythm detection for speech-music discrimination in MPEG compressed domain , 2002, 2002 14th International Conference on Digital Signal Processing Proceedings. DSP 2002 (Cat. No.02TH8628).

[29]  Alan F. Smeaton,et al.  Dublin City University Video Track Experiments for TREC 2002 , 2001, TREC.

[30]  Noel E. O'Connor,et al.  Audio processing for automatic TV sports program highlights detection , 2002 .

[31]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[32]  Noel E. O'Connor,et al.  MPEG-1 Bitstreams Processing for Audio Content Analysis , 2002 .

[33]  Noel E. O'Connor,et al.  Low Power Techniques for Video Compression , 2002 .

[34]  Noel Murphy,et al.  Automatic TV advertisement detection from MPEG bitstream , 2002, Pattern Recognit..

[35]  Noel E. O'Connor,et al.  MPEG audio bitstream processing towards the automatic generation of sports programme summaries , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[36]  Noel E. O'Connor,et al.  Temporal video segmentation for real-time key frame extraction , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[37]  Leonardo Maria Reyneri Implementation issues of neuro-fuzzy hardware: going toward HW/SW codesign , 2003, IEEE Trans. Neural Networks.

[38]  Alan F. Smeaton,et al.  Mobile Access to the Físchlár-News Archive , 2003, Mobile HCI Workshop on Mobile and Ubiquitous Information Access.

[39]  Noel E. O'Connor,et al.  Efficient contour-based shape representation and matching , 2003, MIR '03.

[40]  Noel E. O'Connor,et al.  Region and object segmentation algorithms in the Qimera segmentation platform , 2003 .

[41]  Noel E. O'Connor,et al.  Face detection and clustering for video indexing applications , 2003 .

[42]  Noel E. O'Connor,et al.  Hardware acceleration architectures for MPEG-Based mobile video platforms: a brief overview , 2003 .

[43]  Noel E. O'Connor,et al.  TV news story segmentation, personalisation and recommendation , 2003 .

[44]  Noel E. O'Connor,et al.  QIMERA: A SOFTWARE PLATFORM FOR VIDEO OBJECT SEGMENTATION AND TRACKING , 2003 .

[45]  Michael G. Strintzis,et al.  Advanced content-based semantic scene analysis and information retrieval: the SCHEMA project , 2003 .

[46]  N. O'Connor,et al.  Voice Processing for Automatic Tv Sports Program Highlights Detection , 2003 .

[47]  Noel E. O'Connor,et al.  A multiscale representation method for nonrigid shapes with a single closed contour , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[48]  Noel E. O'Connor,et al.  A framework for event detection in field-sports video broadcasts based on SVM generated audio-visual feature model. Case-study: soccer video , 2004 .

[49]  Noel E. O'Connor,et al.  Facial features and appearance-based classification for face detection in color images , 2004 .

[50]  Noel E. O'Connor,et al.  Action Sequence Detection in Motion Pictures , 2004, EWIMT.

[51]  Alan F. Smeaton,et al.  Experiments in Terabyte Searching, Genomic Retrieval and Novelty Detection for TREC 2004 , 2004, TREC.

[52]  Noel E. O'Connor,et al.  Relating Visual and Semantic Image Descriptors , 2004, EWIMT.

[53]  Alan F. Smeaton,et al.  A generic news story segmentation system and its evaluation , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[54]  D. B. Fogel,et al.  Evolving neural networks , 1990, Biological Cybernetics.

[55]  Noel E. O'Connor,et al.  Energy-Efficient Hardware Architecture for Variable N-point 1D DCT , 2004, PATMOS.

[56]  Noel E. O'Connor,et al.  Dialogue scene detection in movies using low and mid-level visual features , 2004 .

[57]  Noel E. O'Connor,et al.  Automatic detection and extraction of artificial text in video , 2004 .

[58]  Alan F. Smeaton,et al.  The Físchlár-News-Stories System: Personalised Access to an Archive of TV News , 2004, RIAO.

[59]  Noel E. O'Connor,et al.  An experiment in audio classification from compressed data , 2004 .

[60]  Noel E. O'Connor,et al.  Combining textual and visual information processing for interactive video retrieval: SCHEMA's participation in TRECVID 2004 , 2004 .

[61]  Noel E. O'Connor,et al.  Facial Feature Extraction and Principal Component Analysis for Face Detection in Color Images , 2004, ICIAR.

[62]  N. O'Connor,et al.  Current Developments in MPEG-4 Video , 2005 .

[63]  Noel E. O'Connor,et al.  Multiple image view synthesis for free viewpoint video applications , 2005, IEEE International Conference on Image Processing 2005.

[64]  Noel E. O'Connor,et al.  Event detection based on generic characteristics of field-sports , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[65]  Noel E. O'Connor,et al.  A hybrid technique for face detection in color images , 2005, IEEE Conference on Advanced Video and Signal Based Surveillance, 2005..

[66]  Noel E. O'Connor,et al.  Natural Scene Classification and Retrieval Using Ridgelet-Based Image Signatures , 2005, ACIVS.

[67]  Noel E. O'Connor,et al.  Ridgelet-based signatures for natural image classification , 2005, CORIA.

[68]  Seán Marlow,et al.  Road Traffic Monitoring using a Two-Microphone Array , 2005 .

[69]  Risto Miikkulainen,et al.  Real-time neuroevolution in the NERO video game , 2005, IEEE Transactions on Evolutionary Computation.

[70]  Alan F. Smeaton,et al.  Using Segmented Objects in Ostensive Video Shot Retrieval , 2005, Adaptive Multimedia Retrieval.

[71]  Noel E. O'Connor,et al.  An efficient motion estimation hardware architecture for MPEG-4 binary shape coding , 2005 .

[72]  Noel E. O'Connor,et al.  Region-based segmentation of images using syntactic visual features , 2005 .

[73]  Alan F. Smeaton,et al.  Background Modelling in Infrared and Visible Spectrum Video for People Tracking , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[74]  Noel E. O'Connor,et al.  FPGA-based conformance testing and system prototyping of an MPEG-4 SA-DCT hardware accelerator , 2005, Proceedings. 2005 IEEE International Conference on Field-Programmable Technology, 2005..

[75]  Noel E. O'Connor,et al.  Video analysis of events within chemical sensor networks , 2005 .

[76]  Alan F. Smeaton,et al.  User-interface to a CCTV video search system , 2005 .

[77]  Noel E. O'Connor,et al.  Scalable virtual viewpoint image synthesis for multiple camera environments , 2005, Ninth International Conference on Information Visualisation (IV'05).

[78]  Noel E. O'Connor,et al.  Coherent segmentation of video into syntactic regions , 2005 .

[79]  Noel E. O'Connor,et al.  An integrated approach for object shape registration and modeling , 2005 .

[80]  Alan F. Smeaton,et al.  Fusion of infrared and visible spectrum video for indoor surveillance , 2005 .

[81]  Noel E. O'Connor,et al.  Event detection in field sports video using audio-visual features and a support vector Machine , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[82]  P. Beardsley,et al.  Detecting shadows and low-lying objects in indoor and outdoor scenes using homographies , 2005 .

[83]  Noel E. O'Connor,et al.  Detecting the presence of large buildings in natural images , 2005 .

[84]  Alan F. Smeaton,et al.  Interactive Object-Based Retrieval Using Relevance Feedback , 2005, ACIVS.

[85]  Noel E. O'Connor,et al.  Efficient hardware architectures for MPEG-4 core profile , 2005 .

[86]  Peter Carr,et al.  Invited talks , 2005, NUSOD '05. Proceedings of the 5th International Conference on Numerical Simulation of Optoelectronic Devices, 2005..

[87]  Valentin Muresan,et al.  An optimal adder-based hardware architecture for the DCT/SA-DCT , 2005, Visual Communications and Image Processing.

[88]  Alan F. Smeaton,et al.  Using video objects and relevance feedback in video retrieval , 2005, SPIE Optics East.

[89]  Noel E. O'Connor,et al.  The acetoolbox: low-level audiovisual feature extraction for retrieval and classification , 2005 .

[90]  Noel E. O'Connor,et al.  Dialogue Sequence Detection in Movies , 2005, CIVR.

[91]  Alan F. Smeaton,et al.  Associating low-level features with semantic concepts using video objects and relevance feedback , 2005 .

[92]  Alan F. Smeaton,et al.  User evaluation of Físchlár-News , 2006, ACM Trans. Inf. Syst..

[93]  Alan F. Smeaton,et al.  Fischlar-TRECVid-2004: combined text- and image-based searching of video archives , 2006, 2006 IEEE International Symposium on Circuits and Systems.

[94]  Noel E. O'Connor,et al.  Comparison of Fusion Methods for Thermo-Visual Surveillance Tracking , 2006, 2006 9th International Conference on Information Fusion.

[95]  A. Smeaton,et al.  IDENTIFYING PERSON RE-OCCURRENCES FOR PERSONAL PHOTO MANAGEMENT APPLICATIONS , 2006 .

[96]  Alan F. Smeaton,et al.  A System for Event-Based Film Browsing , 2006, TIDSE.

[97]  Noel E. O'Connor,et al.  Image metadata estimation using independent component analysis and regression , 2006 .

[98]  Noel E. O'Connor,et al.  3D image analysis for pedestrian detection , 2006 .

[99]  Alan F. Smeaton,et al.  Word matching using single closed contours for indexing handwritten historical documents , 2006, International Journal of Document Analysis and Recognition (IJDAR).

[100]  Alan F. Smeaton,et al.  Interactive Experiments in Object-Based Retrieval , 2006, CIVR.

[101]  Noel E. O'Connor,et al.  Adaptive visual summary of lifeLog photos forpersonal information management , 2006 .

[102]  Ching Hau Chan,et al.  The CDVPlex biometric cinema: sensing physiological responses to emotional stimuli in film , 2006 .

[103]  Noel E. O'Connor,et al.  Pre-Classification for Automatic Image Orientation , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[104]  Noel E. O'Connor,et al.  Optimisation of Constant Matrix Multiplication Operation Hardware Using a Genetic Algorithm , 2006, EvoWorkshops.

[105]  Alan F. Smeaton,et al.  Security Considerations and Key Negotiation Techniques for Power Constrained Sensor Networks , 2006, Comput. J..

[106]  Alan F. Smeaton,et al.  Object-Based Access to TV Rushes Video , 2006, ECIR.

[107]  Alan F. Smeaton,et al.  Automatically selecting shots for action movie trailers , 2006, MIR '06.

[108]  Noel E. O'Connor,et al.  Movie indexing via event detection , 2006 .

[109]  Noel E. O'Connor,et al.  Detection thresholding using mutual information , 2006 .

[110]  Noel E. O'Connor,et al.  Exploiting context information to aid landmark detection in SenseCam images , 2006 .

[111]  Noel E. O'Connor,et al.  A low complexity hardware architecture for motion estimation , 2006, 2006 IEEE International Symposium on Circuits and Systems.

[112]  D. Diamond,et al.  Low-Energy Finite Field Arithmetic Primitives for Implementing Security in Wireless Sensor Networks , 2006, 2006 International Conference on Communications, Circuits and Systems.

[113]  Alan F. Smeaton,et al.  Multispectral Object Segmentation and Retrieval in Surveillance Video , 2006, 2006 International Conference on Image Processing.

[114]  Noel E. O'Connor,et al.  Searching Movies Based on User Defined Semantic Events , 2006, SIGMAP.

[115]  Alan F. Smeaton,et al.  MediAssist: Using Content-Based Analysis and Context to Manage Personal Photo Collections , 2006, CIVR.

[116]  Noel E. O'Connor,et al.  An Efficient Hardware Architecture for a Neural Network Activation Function Generator , 2006, ISNN.

[117]  Alan F. Smeaton,et al.  Automatic Text Searching For Personal Photos , 2006, SAMT.

[118]  Alan F. Smeaton,et al.  Pedestrian Detection Using Stereo and Biometric Information , 2006, ICIAR.

[119]  Noel E. O'Connor,et al.  Towards an optimised VLSI design algorithm for the constant matrix multiplication problem , 2006, 2006 IEEE International Symposium on Circuits and Systems.

[120]  Noel E. O'Connor,et al.  Interactive object contour extraction for shape modeling , 2006 .

[121]  Noel E. O'Connor,et al.  An Adder Based Hardware Architecture for the MPEG-4 Shape Adaptive IDCT , 2006 .

[122]  Alan F. Smeaton,et al.  Organising a daily visual diary using multifeature clustering , 2007, Electronic Imaging.

[123]  Alan F. Smeaton,et al.  An Improved Spatiogram Similarity Measure for Robust Object Localisation , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[124]  Alan F. Smeaton,et al.  Using text search for personal photo collections with the MediAssist system , 2007, SAC '07.