Penalized eigendecompositions: motivations from domain adaptation for calibration transfer

Maintaining multivariate calibrations involves keeping models developed on an instrument applicable to predicting new samples over time. Sometimes, a primary instrument model is needed to predict samples measured on secondary instruments. This situation is referred to as calibration transfer. Sometimes, a primary instrument model is needed to predict samples that have acquired new spectral features (chemical, physical, and environmental influences) over time. This situation is referred to as calibration maintenance. Calibration transfer and maintenance problems have a long history and are well studied in chemometrics and spectroscopy. In disciplines outside of chemometrics, particularly computer vision, calibration transfer and maintenance problems are more recent phenomena, and these problems often go under the umbrella term domain adaptation. Over the past decade, domain adaptation has demonstrated significant successes in various applications such as visual object recognition. Since domain adaptation already constitutes a large area of research in computer vision and machine learning, we narrow our scope and report on penalty‐based eigendecompositions, a class of domain adaptation methods that has its motivational roots in linear discriminant analysis. We compare these approaches against chemometrics‐based approaches using several benchmark chemometrics data sets.

[1]  Barbara Caputo,et al.  A Deeper Look at Dataset Bias , 2015, Domain Adaptation in Computer Vision Applications.

[2]  Daniel Marcu,et al.  Domain Adaptation for Statistical Classifiers , 2006, J. Artif. Intell. Res..

[3]  Charles R. Hurburgh,et al.  Improving the transfer of near infrared prediction models by orthogonal methods , 2009 .

[4]  Christopher D. Brown,et al.  Discordance between net analyte signal theory and practical multivariate calibration. , 2004, Analytical chemistry.

[5]  Peter Willett,et al.  Combination of Similarity Rankings Using Data Fusion , 2013, J. Chem. Inf. Model..

[6]  M. Dyrby,et al.  Chemometric Quantitation of the Active Substance (Containing C≡N) in a Pharmaceutical Tablet Using Near-Infrared (NIR) Transmittance and NIR FT-Raman Spectra , 2002 .

[7]  Onno E. de Noord,et al.  The influence of data preprocessing on the robustness and parsimony of multivariate calibration models , 1994 .

[8]  Tinne Tuytelaars,et al.  Unsupervised Visual Domain Adaptation Using Subspace Alignment , 2013, 2013 IEEE International Conference on Computer Vision.

[9]  John Blitzer,et al.  Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification , 2007, ACL.

[10]  Desire L. Massart,et al.  Standardization of near-infrared spectra in the wavelet domain , 1997 .

[11]  P. Hansen Rank-Deficient and Discrete Ill-Posed Problems: Numerical Aspects of Linear Inversion , 1987 .

[12]  Steven D. Brown,et al.  Wavelet analysis applied to removing non‐constant, varying spectroscopic background in multivariate calibration , 2002 .

[13]  Eric O. Postma,et al.  Dimensionality Reduction: A Comparative Review , 2008 .

[14]  G. W. Small,et al.  Spectral Simulation Methodology for Calibration Transfer of Near-Infrared Spectra , 2007, Applied spectroscopy.

[15]  S. Wold,et al.  Orthogonal signal correction of near-infrared spectra , 1998 .

[16]  T. Fearn Standardisation and Calibration Transfer for near Infrared Instruments: A Review , 2001 .

[17]  Charles L. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[18]  Rama Chellappa,et al.  Visual Domain Adaptation: A survey of recent advances , 2015, IEEE Signal Processing Magazine.

[19]  Ivor W. Tsang,et al.  Domain Adaptation via Transfer Component Analysis , 2009, IEEE Transactions on Neural Networks.

[20]  Tom Fearn,et al.  Transfer by orthogonal projection: making near-infrared calibrations robust to between-instrument variation , 2004 .

[21]  Tom Fearn,et al.  Collaborative evaluation of universal calibrations for the measurement of protein and moisture in flour by near infrared reflectance , 2007 .

[22]  Olof Svensson,et al.  An evaluation of orthogonal signal correction applied to calibration transfer of near infrared spectra , 1998 .

[23]  Onno E. de Noord,et al.  Multivariate calibration standardization , 1994 .

[24]  Mengjie Zhang,et al.  Scatter Component Analysis: A Unified Framework for Domain Adaptation and Domain Generalization , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  D. Longsine,et al.  Simultaneous rayleigh-quotient minimization methods for Ax=λBx , 1980 .

[26]  Rong Yan,et al.  Cross-domain video concept detection using adaptive svms , 2007, ACM Multimedia.

[27]  Charles R. Hurburgh,et al.  Standardisation of near Infrared Spectrometers: Evaluation of Some Common Techniques for Intra- and Inter-Brand Calibration Transfer , 2008 .

[28]  Steven D. Brown,et al.  Transfer of multivariate calibration models: a review , 2002 .

[29]  James J. Jiang A Literature Survey on Domain Adaptation of Statistical Classifiers , 2007 .

[30]  Neil D. Lawrence,et al.  Dataset Shift in Machine Learning , 2009 .

[31]  David M. Haaland,et al.  New Prediction-Augmented Classical Least-Squares (PACLS) Methods: Application to Unmodeled Interferents , 2000 .

[32]  John H. Kalivas,et al.  Fundamentals of Calibration Transfer through Procrustes Analysis , 1999 .

[33]  Jieping Ye,et al.  Generalized Linear Discriminant Analysis: A Unified Framework and Efficient Model Selection , 2008, IEEE Transactions on Neural Networks.

[34]  J. Shenk,et al.  Calibration Transfer Between near Infrared Reflectance Spectrophotometers , 1985 .

[35]  Erik Andries,et al.  Model updating for spectral calibration maintenance and transfer using 1-norm variants of Tikhonov regularization. , 2010, Analytical chemistry.

[36]  Bruce R. Kowalski,et al.  Weighting schemes for updating regression models—a theoretical approach , 1999 .

[37]  Hal Daumé,et al.  Frustratingly Easy Domain Adaptation , 2007, ACL.

[38]  Desire L. Massart,et al.  Improvement of the piecewise direct standardisation procedure for the transfer of NIR spectra for multivariate calibration , 1996 .

[39]  Ahmed H. Sameh,et al.  Trace Minimization Algorithm for the Generalized Eigenvalue Problem , 1982, PPSC.

[40]  Barbara Caputo,et al.  A Deeper Look at Dataset Bias , 2015, Domain Adaptation in Computer Vision Applications.

[41]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[42]  Bruce R. Kowalski,et al.  Temperature-compensating calibration transfer for near-infrared filter instruments , 1993 .

[43]  B. Kowalski,et al.  Multivariate instrument standardization , 1991 .

[44]  Yehuda Koren,et al.  Ieee Transactions on Visualization and Computer Graphics Robust Linear Dimensionality Reduction , 2022 .

[45]  Tom Fearn,et al.  COLLABORATIVE EVALUATION OF NEAR-INFRARED REFLECTANCE ANALYSIS FOR THE DETERMINATION OF PROTEIN, MOISTURE AND HARDNESS IN WHEAT , 1983 .

[46]  Erik Andries,et al.  Calibration Maintenance and Transfer Using Tikhonov Regularization Approaches , 2009, Applied spectroscopy.