Grand Challenges in Virtual Environments

In his 1999 article that reviewed the stateof virtual reality (VR) Prof. Fred BrooksJr. noted that at that time the field hadmadegreattechnicaladvancesoverthepre-vious 5years.Brooks (1999) wrote“I thinkour technology has crossed over the pass –VR that used to almost work now barelyworks. VR is now really real.” What is thestate today? Needless to say 15years later,it is the case that not only does VR “reallywork” but that it has become a common-placetoolinmanyareasof science,technol-ogy, psychological therapy, medical reha-bilitation, marketing, and industry, and issurely about to become commonplace inthe home.Let us very briefly reprise the majortechnologies reviewed by Brooks (1999).On the display side, projection systemssuchasCaves(Cruz-Neiraetal.,1992)haveadvanced to very high resolution, basedon multiple panel displays with automaticand seamless image alignment (Brownet al., 2005; Defanti et al., 2011). Head-mounted displays (HMDs) have improveddramatically – presenting wide field-of-view high-resolution images, with veryrecent advances toward lightweight, lowcost, and consumer oriented devices withwide field-of-view and acceptable resolu-tion.Beyondspecializeddevicesthereisthepromise of yet a further development withalready usable HMDs made from 3D print-able plastic frames and cheap lenses thathouse a Smartphone (Olson et al., 2011;Hoberman et al., 2012; Steed and Julier,2013).Devices for virtual and augmented real-itydisplaystypicallyrequiretheparticipantto wear specialized glasses. Autostereo dis-plays obviate the need for this but offera limited continuous field-of-view (Hol-liman et al., 2011). An important grandchallengeinthedisplayareaistheprovisionof high quality full field-of-view stereo-displays that do not require special glasses,where there is a seamless blend betweenreality and VR. Steps are being made inthis direction (Hilliges et al.,2012) and alsothere is the development of such displaysbased on the low cost consumer devices(Maimone et al., 2012).The other side of the equation to dis-play is tracking. In recent years whole bodytracking has become a relatively low costproduct enabling real-time motion cap-ture. But just as with HMDs, the arrivalof consumer oriented tracking systemsand depth cameras, marketed for com-puter games, is likely to revolutionize howreal-time full body tracking and probablyhead-tracking are likely to develop. Headand body tracking over wide areas withlow latency and high accuracy remains asignificant challenge.One thing is clear from the above – VRis moving to the home. However, the vastmajority of the studies that assess vari-ous aspects of the responses of people toVR experiences typically involve very shortone-off exposure times – a notable excep-tion being (Steed et al., 2003). Due to themovetotheconsumermarketthissituationobviously will and must change – althoughsimulator sickness may still be a significantproblem. If people in their millions will beusing these systems for hours a week, wereally need to understand the impact ofthis on their lives and the resulting socialimpact, as well as seeing this as an oppor-tunity to carry out massive experimentalstudiesof scientificinterestthatalsoinformthe technology.The above gives the impression that VRis defined by devices and associated soft-ware.However,itismoreusefultoconsiderVR conceptually as a technological sys-tem that can precisely substitute a person’ssensory input and transform the meaningof their motor outputs with reference to anexactly