THE GENERALIZED GAUSSIAN MIXTURE MODEL USING ICA

THEGENERALIZEDGAUSSIANMIXTUREMODELUSINGICATe-WonLeInstituteforNeuralComputation,UniversityofCalifornia,SanDiego,9500GilmanDr.LaJolla,California92093-0523,USAtewon@inc.ucsd.eduMichaelS.LewickiComputerScienceDepartment&CenterfortheNeuralBasisofCognitionCarnegieMellonUniversity4400FifthAve.Pittsburgh,PA15213lewicki@cs.cmu.eduABSTRACTAnextensionoftheGaussianmixturemo delispre-sentedusingIndep endentComp onenAnalysis(ICA)andthegeneralizedGaussiandensitymo del.Themix-turemo delassumesthattheobserveddatacanb ecat-egorizedintomutuallyexclusiveclasseswhosecomp o-nentsaregeneratedbyalinearcombinationofinde-p endentsources.Thesourcedensitiesaremo deledbygeneralizedGaussians(BoxandTiao,1973)thatpro-videageneralmetho dformo delingnon-Gaussiansta-tisticalstructureofunivariatedistributionsthathavetheformp(x)/exp(jq).Byinferringq,awideclassofstatisticaldistributionscanb echaracterizedin-cludinguniform,Gaussian,Laplacian,andothersub-andsup er-Gaussiandensities.ThegeneralizedGaus-sianmixturemo delusingICAinfersforeachclassthesourceparameters,thebasisfunctionsandbiasvec-tors.Thenewmetho dcanimproveclassi cationaccu-racycomparedwithstandardGaussianmixturemo delsandshowspromiseforaccuratelymo delingstructureinhigh-dimensionaldata.1.INTRODUCTIONInpatternclassi cation,thep erformanceofametho disoftendeterminedbyhowellitcanmo deltheun-derlyingstatisticaldistributionofthedata.Onere-centexampleofthisisindep endencomp onenanal-ysis(ICA).ThesuccessofICAonproblemssuchasblindsourceseparationandsignalanalysisresultsdi-rectlyfromitsabilitytomo delnon-Gaussianstatisticalstructure.Ifthesourcedistributionsareassumedtob eGaussian,thistechniqueisequivalenttoprincipalcomp onentanalysis(PCA).PCAassumesthedatatob edistributedaccordingtoamultivariateGaussian.Incontrast,ICAassumesthatthesourcedistributionsarenon-Gaussianallowingmo delingstruc-ture,e.g.,platykurticorleptokurticprobabilityden-sityfunctions.InmanapplicationsofICA,theformofthesourcedistribution(orequivalently\non-linearity")is xed.Morerecentworkhasextendedtheseresultssothattheformofdistributioncanalsob einferredfromthedataforexample,usingGaussianmixtures(Attias,1999)orofsub-Gaussianandsup er-Gaussiandensities(Leeetal.,1999a).Inmanypatternrecognitionproblemstherearedataclustersinwhicheacclustercanb e ttedbynon-Gaussiandistributions.Tomo deltheensembleofdataclassesamixturemo del(DudaandHart,1973)iscon-sideredwheretheobserveddatacanb ecategorizedintoseveralmutuallyexclusiveclasses.Whentheclassvari-ablesaremo deledasmultivariateGaussiandensities,itiscalledaGaussianmixturemo del.Thismo delcanb egeneralizedbyassumingthatthesourcesineachclassareindep endentandnon-Gaussian.InLeeetal.(1999b)wemo deledtheunderlyingsourcedensitybtwoprede nednon-Gaussiandensities(sup erandsub-Gaussian)asusedintheextendedinfomaxalgorithm.Abinaryparameterswitchedfromasub-orsup er-Gaussiandensity.Inthispap er,weareinterestedmo delingacon-tinuouslyde nedparametricformoftheunderlyingdensityforeachclass.Theexp onentialp owerdistri-bution(BoxandTiao,1973)1isusedtomo deldis-tributionsthatdeviatefromnormality.Theyprovideageneralmetho dformo delingnon-Gaussianstatisti-calstructureofunivariatedistributionsthathavetheformp(x)/expjq).Byinferringq,awideclassofstatisticaldistributionscanb echaracterizedinclud-inguniform,Gaussian,Laplacian,andothersub-sup er-Gaussiandensities.Thisformulationofamix-turemo delusingthegeneralizedGaussiancontainsassp ecialtheGaussianmixturemo delwhenallsource1alsocalledageneralizedLaplacianorGaussian.