Non-correlation methods for stereo matching

I S I o ~ ~ o ol' the stereo conlA pil* A t ion process II~I \JC lccl to the development of a n11n1l)er of' ~n;~chines which itre a l ~ l c to procluce good resdts in mimy cilses, particularly if' aitlerl 1)y a h u ~ i l m operator. 011 difficult terrain, they !id. These ~niichines ;ire based on inoclifici~tions of the correli~tion i d r i ~ Ibr finding the disparity. The a s s~~mpt ion has Ijeen that f b thcr rdincrrients of' this m ~ ) r o ; ~ c h will solve the . . remilining prol)lc~ns or thilt t h y cannot I)e solved. 'I'herr are methods other t h m ~ correliition availihlc. for idc~ltifying corrcsponcling points. Some delwncl on optimization of the ~i i i~ tch fount1 with wrying distortions of the hvo images ( h t without i~s ing correl;~tion)-ott~crs syrn1)olicdly match feiitr~res extracted from the image. liesc:~rch on several alterni~tc proposed methods is showing pronlisc. Some q )p roxhes even lend themsclvcs to in~plerne~~ta~t ion in special p ~ r p o s c I~ardware to ilttaiu ri~pid throughput. W l ~ y 1x1s tlirrc been sucli single nlinclctl conceutri~tion on co r rc l i~ t io~~ and its modifications for ~ ~ ~ i ~ t c l i i n g image p~tc l ics? One rei~son is speed. l t wiis ~ I S S I I I ~ ~ ~ ~ t l ~ t a nlilchine h i~d to Iw f i ~ t to be llsefid iind tl1i11 only s in~ple con~puti~tions could I)c perlbnned at the rcquirc1tl speed. Correlation l e ~ ~ d s itself' to implcmcntation in special-purposc harclware or more generid-purpose piudlel cornpriters. This cniplrasis oil speed was perhaps prematrlrtB. A m ~ c h i n c which rapidly prodr~ccs output which requires estensivo post-editing is not contri l)~~ting to increased pn)ductivity. If' the hvo images were simply shifted versions of one ;mother, then thc intrgral of the squitre of their tlifkrt!nce (with one ilni~ge shifted) ciln I)e rni~tlc cv111aI to zero. This integral thrls h i~s il mini~ilunl when the images are shifted the correct iinlorlnt rclativc to one ;~notlicr. Correspondingly, the corrcli~tion (the integral of their product) has 21 m ; ~ s i n ~ ~ ~ n ~ . Does this m;~s in~~ i rn correspond to ;I unique tlispl;~cemcnt? If the two inii~ge patclies iire rmifi)rni in I)rigtitness, then unfortmately a whole range of displi~cements will do. Also, if' the image patches conti~in ;I periodic ~ i ~ t t c r n , there are multiple niaxinia. So tliere arc pro1)lems, even when the imi~ges arc just shifted versions of' one another. Even more serious is the h c t that the hvo ini;~gcs are in lict ,lot shifted versions of one another. For large Ix~se-height ri~tios in hilly tcrrain. or in 1)uilt-nr, areas. some surfkcs niilv h e sccn only in onc image. Those thilt are seen in I~otli will appear with d i f k r e ~ ~ t Ibreshortening in the two iniagcs; that is, the s ~ ~ r f i ~ c e s ilppcilr compressed along one dimension to ditkring dcgrees. Simple correlation will not work hcw. At the lewt, 21 kind of ri11)lwr-sheet distortion h;~s to I)r introduced so that the patclies heing compared ;we in f k t simili~r. Even if this is titken into : ~ c c o ~ ~ n t , one is still f:,lccd with the fact thilt the same piitch on the su14i1cc will appc;~r with difltrent 1)rightnesses when viewed lion1 two tlifkrent tlircctions. Only if the Idirectional reflecti~ncc distriI)utio~i h e ti011 (BRDY) hi~ppens to I)c inc1el)enclent of the tlirection of reflection will the image irradiance 1)e the same in the two images (ignoring atmospheric ef'f'ects). Specular retlections from water sr~rfhces and hot-spots ( I h c k reflections) fi-om vegctation canopies (and the i~t~nosphere) ;we estreme es ;~n~p les of violations of this criterion. If a su~filcc is viewed with iI device which does not resolve srlrfilce patches 01' difrerent orient. I t' 1011, then the lwightness is a weighted ;Iver;igc of the 1)rightnesscs of the indivitl~~id pi~tclws, with weights proportional to the foreshortend areas. Thus, even if the material of a s ~ n l k e satisfies the ihove criterion, ;I region of it may appear with tlifrerent brightnesses in different images if its s ~ ~ r f i c e ~mduli~tions are not resolved. Shi~dows too small to I)e ~rsolvctl contriI)r~te to this efi'ect. These efTects and others ( s ~ ~ c h as noise) gui~rantec that the lxisic nlodcl on which the idea of correlation is 1)ased is not s o ~ ~ n d . T l ~ c rcsult is that operator intervention is required thrring