Advanced Visual Sensor Systems �

This report is on the Lehigh/ColumbiaMURI contract. While the original focus was on sensors for manufacturing, the natural evolution of our basic researchhas led us to more general problemsin more generic settings. As a multifaculty multi-disciplinaryprojectmuchof the work is naturally done in smaller subgroup. The major resultsover the past year were on 3D modeling/sensorplanning,omni-directionalimaging,and reflectance/image/noise modeling.Thereweremore focusedresultsin deformablemodels,featuredetection, appearancematching,andvideosegmentation. Notethatsomeprojects(e.g.,theoutdoor3D model building) bring togethermany of the above topics. This reportprovidesshortsummariesof our significant contributions with citationsto relatedpapers. Lengthof presentationhereindoesnot reflect level of effort nor our view of its significance– many of the most importantareashave paperselsewherein theseproceedings. 1 AutomatedSiteModeling 3-D modelsof outdoorenvironments,known assite models,areusedin many differentapplicationsthat includecity planning,urbandesign,fire andpolice planning,surveillanceandvirtual reality modeling. Creatingsite modelsfor urbansceneswhich contain large structures(i.e., buildings) thatencompass a wide rangeof geometricshapesandcontainhigh occlusionareasis quitechallenging. Thesemodels are typically createdby hand in a painstakinganderror proneprocess.We arebuilding a systemto automatethis procedurethat extendsour previous work in 3-D model acquisition This work supported by ONR/ARPA MURI program ONR N00014-95-1-0601.Several other agenciesand companieshave also supportedpartsof this research. Figure1: Mobile robotfor automatedsitemodeling. usingrangedata[Reed-1998, ReedandAllen-1998, Reedet al.-1997, Allen and Yang-1998]. This is an incrementalvolumetricmethodthat canacquire andmodelmultiple objectsin a sceneandcorrectly merge models from different views of the scene. Modelsarebuilt from successi ve sensingoperations that aremergedwith the currentmodelbeingbuilt, called the compositemodel. The merging is performedusingaregularizedsetintersectionoperation. Thesepartial,compositemodelscanserveasinputto oursensorplanningsystemthatcanreducethenumber of views neededto fully acquirea scene.Planningeachsensingoperationreducesthedatasetsizes andacquisitiontime for a complex model.Most existing systemsdo not useplanning,but rely on humaninteractionor theneedfor largeoverlapsin imagesto assureadequatecoverageof thescene.Our plannercanincorporatedifferentconstraintsincluding visibility, field-of-view andsensorplacemento find the next viewing position that will reducethe model’s uncertainty . Thesystemhasbeentestedon Figure2: Recovered3-D modelsof a blocksworld city. All 3 objectswererecoveredat once. Sensor planningalgorithmswere usedto reducethe numberof rangescansandfind unoccludedviewing positions. indoor modelsandwe arenow extendingit to outdoorsceneswith multipleobjects. At eachstepof theprocess,a partialsolid modelof the sceneis created. The facesof this model consistof correctlyimagedfacesandfacesthatareocclusionartifacts. We can label thesefacesas “imaged”or “unimaged”andpropagate/update theselabelsasnew imagesareintegratedinto thecomposite model. The faceslabeled“unimaged”are thenthe focusof thesensorplanningsystemwhichwill try to positionthesensorto allow these“unimaged”faces to be scanned.The set intersectionoperationmust be ableto correctlypropagatethe surface-typetags fromsurfacesin themodelsthroughto thecomposite model.Retainingthesetagsaftermergingoperations allows viewpoint planningfor unimagedsurfacesto proceed. Figure2 is a recovered3-D solid modelof a simulatedcity scenemadeupof 3 toy buildingsplacedon our laserscannerturntable.Four initial views were takenat intervals,andthismergedpartialmodel wasusedby theview plannerto choosethenext view to reducethemodel’s uncertainty . This processof a partial modeldriving the plannerfor the next view wasuseduntil 8 moreimagesweretaken, reducing the model’s uncertaintyto a small threshold. The modelis accurateandthemethodcanrecover structurethatis occluded. For automatingthis task outdoors,we are equipping a mobile vehiclewith sensorsand algorithms to accomplishthis task. A pictureof the vehicle is shown in figure1. Theequipmentconsistsof anRWI ATRV mobile robot base,a spot rangescanner(80 meterrangespotscannerwith 2-DOFscanningmirrors for acquiringa whole rangeimage,not shown in image),centimeteraccuracy on-boardGPS,color camerasfor obtainingphotometryof thescene,and mobilewirelesscommunicationsfor transmissionof dataandhigh level controlfunctions. Theplanningalgorithmscanbeusedto navigatethe mobilerobotscannersystemto a positionfor a new scanthat will reducethe uncertaintyin the scene. Given a partial modelof the scene,we cantag the surfacesacquiredby the scannerin sucha way as to know whatregionsareimagedandwhich areocclusion surfaces. We can thenusetheseocclusion surfacesto find unobstructedviewpoints. Oncewe computetheseviewpoints, we can then command themobilesystemto navigateinsidea visibility volumeandtake a new scan,thuscompletinga partial modeland“filling in the blanks” to build the completemodel.Detailscanbefoundin thisproceedings [Allen et al.-1998]. 2 Interacti veSensorPlanning Theautomatedsitemodelingsystemwill bemerging therangedatawith 2-D imageryto enhancethemodels. This alsorequiresa view planningcomponent. In clutteredandcomplex environmentssuchasurban scenes, it canbevery difficult to determinewherea camerashouldbeplacedto view multipleobjectsand regionsof interest.We have built aninteracti ve sensor planningsystem[Stamosand Allen-1998] that canbe usedto selectviewpointssubjectto camera visibility, field of view and task constraints. This work buildsuponourearlierwork in sensorplanning [Abrams-1997 ]. Application areasfor this method includesurveillanceplanning,safetymonitoring,architecturalsite designplanning,andautomatedsite modeling. Givena descriptionof thesensor’ s characteristics,theobjectsin the3-D scene,andthetargetsto be viewed, our algorithmscomputethe set of admissibleview pointsthatsatisfytheconstraints. The systemfirst builds topologically correct solid modelsof the scenefrom a variety of datasources. Viewing targetsarethenselected,andvisibility volumesandfield of view conesarecomputedandintersectedto createviewing volumeswherecameras can be placed. The usercan interacti vely manipulate the sceneandselectmultiple target featuresto beviewedby acamera.Theusercanalsoselectcan-