Effective Dimensionality: A Tutorial

The topic of this tutorial is the effective dimensionality (ED) of a dataset, that is, the equivalent number of orthogonal dimensions that would produce the same overall pattern of covariation. The...

[1]  L. Jost.  Entropy and diversity , 2006 .

[2]  David V Budescu,et al.  How to measure diversity when you must. , 2012, Psychological methods.

[3]  Trevor Hastie,et al.  An Introduction to Statistical Learning , 2013, Springer Texts in Statistics.

[4]  Gil Alterovitz,et al.  Is the reduction of dimensionality to a small number of features always necessary in constructing predictive models for analysis of complex diseases or behaviours? , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[5]  Ilker Yalcin,et al.  Nonlinear Factor Analysis as a Statistical Method , 2001 .

[6]  J. Li,et al.  Adjusting multiple testing in multilocus analyses using the eigenvalues of a correlation matrix , 2005, Heredity.

[7]  Daniel M. McNeish,et al.  Psychological Methods Thanks Coefficient Alpha , We ’ ll Take It From Here , 2022 .

[8]  Donald A. Jackson,et al.  How many principal components? stopping rules for determining the number of non-trivial axes revisited , 2005, Comput. Stat. Data Anal..

[9]  N. Higham.  Computing the nearest correlation matrix—a problem from finance , 2002 .

[10]  Ata Kabán,et al.  When is 'nearest neighbour' meaningful: A converse theorem and implications , 2009, J. Complex..

[11]  P. Costa,et al.  Normal Personality Assessment in Clinical Practice: The NEO Personality Inventory. , 1992 .

[12]  M. Hill.  Diversity and Evenness: A Unifying Notation and Its Consequences , 1973 .

[13]  M. Gurven,et al.  What Explains Personality Covariation? A Test of the Socioecological Complexity Hypothesis , 2017 .

[14]  Minimum Entropy as a Measure of Effective Dimensionality , 2016 .

[15]  Seungmin Jahng,et al.  Determining the number of factors using parallel analysis and its recent variants. , 2019, Psychological methods.

[16]  K. A. Remley,et al.  Reverberation Chamber Measurement Correlation , 2012, IEEE Transactions on Electromagnetic Compatibility.

[17]  William Revelle,et al.  Cronbach’s α, Revelle’s β, and Mcdonald’s ωH: their relations with each other and two alternative conceptualizations of reliability , 2005 .

[18]  Hans-Peter Kriegel,et al.  A survey on unsupervised outlier detection in high‐dimensional numerical data , 2012, Stat. Anal. Data Min..

[19]  Jordan B Peterson,et al.  Between facets and domains: 10 aspects of the Big Five. , 2007, Journal of personality and social psychology.

[20]  Alessandro Laio,et al.  Estimating the intrinsic dimension of datasets by a minimal neighborhood information , 2017, Scientific Reports.

[21]  J. Heckman,et al.  Bayesian Exploratory Factor Analysis , 2014, Journal of econometrics.

[22]  Michael Wolf,et al.  Spectrum Estimation: A Unified Framework for Covariance Matrix Estimation and PCA in Large Dimensions , 2013, J. Multivar. Anal..

[23]  Andrzej Cichocki,et al.  Bayesian estimation of the number of principal components , 2006, 2006 14th European Signal Processing Conference.

[24]  M. Kirkpatrick.  Patterns of quantitative genetic variation in multiple dimensions , 2009, Genetica.

[25]  J. Berge,et al.  Least-squares approximation of an improper correlation matrix by a proper one , 1989 .

[26]  P. Campadelli,et al.  Intrinsic Dimension Estimation: Relevant Techniques and a Benchmark Framework , 2015 .

[27]  Martin Krzywinski,et al.  Points of Significance: Regularization , 2016, Nature Methods.

[28]  A. Goriely,et al.  Component retention in principal component analysis with application to cDNA microarray data , 2007, Biology Direct.

[29]  J. Cheverud,et al.  A simple correction for multiple comparisons in interval mapping genome scans , 2001, Heredity.

[30]  Wijnand A.P. van Tilburg.  It's not unusual to be unusual (or: A different take on multivariate distributions of personality) , 2019, Personality and Individual Differences.

[31]  Martin Krzywinski,et al.  The curse(s) of dimensionality , 2018, Nature Methods.

[32]  Hans-Peter Kriegel,et al.  Can Shared-Neighbor Distances Defeat the Curse of Dimensionality? , 2010, SSDBM.

[33]  Sacha Epskamp,et al.  Exploratory graph analysis: A new approach for estimating the number of dimensions in psychological research , 2017, PloS one.

[34]  Eric O. Postma,et al.  Dimensionality Reduction: A Comparative Review , 2008 .

[35]  Fritz Drasgow,et al.  Polychoric and Polyserial Correlations , 2006 .

[36]  Xavier Mestre,et al.  Improved Estimation of Eigenvalues and Eigenvectors of Covariance Matrices Using Their Sample Estimates , 2008, IEEE Transactions on Information Theory.

[37]  Miguel Á. Carreira-Perpiñán,et al.  A Review of Dimension Reduction Techniques , 2009 .

[38]  Tim Kaiser.  Nature and evoked culture: Sex differences in personality are uniquely correlated with ecological stress , 2019, Personality and Individual Differences.

[39]  H. Tuomisto.  A diversity of beta diversities: straightening up a concept gone awry. Part 1. Defining beta diversity as a function of alpha and gamma diversity , 2010 .

[40]  Anthony J Bishara,et al.  Reducing Bias and Error in the Correlation Coefficient Due to Nonnormality , 2015, Educational and psychological measurement.

[41]  John A. Johnson.  Measuring thirty facets of the Five Factor Model with a 120-item public domain inventory: Development of the IPIP-NEO-120 , 2014 .

[42]  Duane T. Wegener,et al.  Evaluating the use of exploratory factor analysis in psychological research. , 1999 .

[43]  Olivier Ledoit,et al.  Nonlinear Shrinkage Estimation of Large-Dimensional Covariance Matrices , 2011, 1207.5322.

[44]  Charu C. Aggarwal,et al.  On the Surprising Behavior of Distance Metrics in High Dimensional Spaces , 2001, ICDT.

[45]  G. Wagner,et al.  Pleiotropic scaling of gene effects and the ‘cost of complexity’ , 2008, Nature.

[46]  Thomas J. Dunn,et al.  From alpha to omega: a practical solution to the pervasive problem of internal consistency estimation. , 2014, British journal of psychology.

[47]  Jochen Einbeck,et al.  Intrinsic Dimensionality Estimation for High-dimensional Data Sets: New Approaches for the Computation of Correlation Dimension , 2013 .

[48]  J. Ruscio,et al.  Determining the number of factors to retain in an exploratory factor analysis using comparison data of known factorial structure. , 2012, Psychological assessment.

[49]  L. Guttman.  Some necessary conditions for common-factor analysis , 1954 .

[50]  Martin Vetterli,et al.  The effective rank: A measure of effective dimensionality , 2007, 2007 15th European Signal Processing Conference.

[51]  Jonathan Goldstein,et al.  When Is ''Nearest Neighbor'' Meaningful? , 1999, ICDT.