Learning Resource-Aware Classifiers for Mobile Devices: From Regularization to Energy Efficiency

Abstract Mobile devices are resource-limited systems that provide a large number of services and features. Smartphones, for example, implement advanced functionalities and services for the final user, in addition to conventional communication capabilities. Machine Learning algorithms can help in providing such advanced functionalities, but mobile systems suffer from issues related to their resource-limited nature such as, for example, limited battery capacity and processing power and, therefore, even simple pattern recognition activities can become too demanding, in this respect. We propose here a method to design a Human Activity Recognition algorithm, which takes into account the fact that only limited resources are available for its execution. In particular, we restrict the hypothesis space of possible recognition models by applying some advanced concepts from Statistical Learning Theory, so as to force the selection of models with good generalization ability but low computational complexity. Then, the learned model can be effectively implemented on a mobile and resource-limited device: the experiments, carried out on a current-generation smartphone, show the benefits of the proposed approach in terms of both model accuracy and battery duration.

[1]  Davide Anguita,et al.  Human Activity Recognition on Smartphones Using a Multiclass Hardware-Friendly Support Vector Machine , 2012, IWAAL.

[2]  Davide Anguita,et al.  A Learning Machine with a Bit-Based Hypothesis Space , 2013, ESANN.

[3]  Ambuj Tewari,et al.  Smoothness, Low Noise and Fast Rates , 2010, NIPS.

[4]  Gábor Lugosi,et al.  Introduction to Statistical Learning Theory , 2004, Advanced Lectures on Machine Learning.

[5]  Ingo Steinwart,et al.  Fast rates for support vector machines using Gaussian kernels , 2007, 0708.1838.

[6]  Davide Anguita,et al.  The Impact of Unlabeled Patterns in Rademacher Complexity Theory for Kernel Classifiers , 2011, NIPS.

[7]  Peter L. Bartlett,et al.  Localized Rademacher Complexities , 2002, COLT.

[8]  Manfred Mücke,et al.  Effects of Reduced Precision on Floating-Point SVM Classification Accuracy , 2011, International Conference on Conceptual Structures.

[9]  Michael G. Epitropakis,et al.  Hardware-friendly Higher-Order Neural Network Training using Distributed Evolutionary Algorithms , 2010, Appl. Soft Comput..

[10]  Carlo Vercellis,et al.  Discrete support vector decision trees via tabu search , 2004, Comput. Stat. Data Anal..

[11]  V. Koltchinskii.  Local Rademacher complexities and oracle inequalities in risk minimization , 2006, 0708.0083.

[12]  Davide Anguita,et al.  Energy Efficient Smartphone-Based Activity Recognition using Fixed-Point Arithmetic , 2013, J. Univers. Comput. Sci..

[13]  Andrew T. Campbell,et al.  Bewell: A smartphone application to monitor, model and promote wellbeing , 2011, PervasiveHealth 2011.

[14]  Davide Anguita,et al.  In-sample Model Selection for Trimmed Hinge Loss Support Vector Machine , 2012, Neural Processing Letters.

[15]  André Elisseeff,et al.  Stability and Generalization , 2002, J. Mach. Learn. Res..

[16]  M. Talagrand,et al.  Probability in Banach Spaces: Isoperimetry and Processes , 1991 .

[17]  A. Tsybakov,et al.  Fast learning rates for plug-in classifiers , 2007, 0708.2321.

[18]  Colin McDiarmid,et al.  Surveys in Combinatorics, 1989: On the method of bounded differences , 1989 .

[19]  Davide Anguita,et al.  A support vector machine classifier from a bit-constrained, sparse and localized hypothesis space , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[20]  L. Valiant.  Probably Approximately Correct: Nature's Algorithms for Learning and Prospering in a Complex World , 2013 .

[21]  Lawrence O. Hall,et al.  Bit reduction support vector machine , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[22]  Peter L. Bartlett,et al.  Rademacher and Gaussian Complexities: Risk Bounds and Structural Results , 2003, J. Mach. Learn. Res..

[23]  Davide Anguita,et al.  A Public Domain Dataset for Human Activity Recognition using Smartphones , 2013, ESANN.

[24]  P. Massart,et al.  Discussion: Local Rademacher complexities and oracle inequalities in risk minimization , 2006 .

[25]  Lorenzo Rosasco,et al.  Learning from Examples as an Inverse Problem , 2005, J. Mach. Learn. Res..

[26]  Raquel Valdés-Cristerna,et al.  An FPGA Implementation of Linear Kernel Support Vector Machines , 2006, 2006 IEEE International Conference on Reconfigurable Computing and FPGA's (ReConFig 2006).

[27]  B. Venkataramani,et al.  FPGA Implementation of Support Vector Machine Based Isolated Digit Recognition System , 2009, 2009 22nd International Conference on VLSI Design.

[28]  Sang-Woong Lee,et al.  Real-Time Implementation of Face Recognition Algorithms on DSP Chip , 2003, AVBPA.

[29]  Gert Cauwenberghs,et al.  Kerneltron: Support Vector 'Machine' in Silicon , 2002, SVM.

[30]  Enrique Alba,et al.  Using Variable Neighborhood Search to improve the Support Vector Machine performance in embedded automotive applications , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[31]  John Shawe-Taylor,et al.  Structural Risk Minimization Over Data-Dependent Hierarchies , 1998, IEEE Trans. Inf. Theory.

[32]  Davide Anguita,et al.  Selecting the hypothesis space for improving the generalization ability of Support Vector Machines , 2011, The 2011 International Joint Conference on Neural Networks.

[33]  Diane J. Cook,et al.  Pervasive computing at scale: Transforming the state of the art , 2012, Pervasive Mob. Comput..

[34]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[35]  M. Talagrand.  The Glivenko-Cantelli Problem , 1987 .

[36]  Davide Anguita,et al.  A support vector machine with integer parameters , 2008, Neurocomputing.

[37]  P. Bartlett,et al.  Local Rademacher complexities , 2005, math/0508275.

[38]  Eliathamby Ambikairajah,et al.  Classification of a known sequence of motions and postures from accelerometry data using adapted Gaussian mixture models. , 2006, Physiological measurement.

[39]  V. Koltchinskii,et al.  Oracle inequalities in empirical risk minimization and sparse recovery problems , 2011 .

[40]  Narayanan Vijaykrishnan,et al.  A Hardware Efficient Support Vector Machine Architecture for FPGA , 2008, 2008 16th International Symposium on Field-Programmable Custom Computing Machines.