iPhone in vivo: video analysis of mobile device use

Despite the widespread use of mobile devices, details of mobile technology use 'in the wild' have proven difficult to collect. This paper uses video data to gain new insight into the use of mobile computing devices. Our new method combines screen-capture of iPhone use with video recordings from wearable cameras. We use this data to analyse how mobile device use is threaded into other co-present activities, focusing on the use of maps and internet searches. Close analysis reveals novel aspects of gestures on touch screens, how they serve 'double duty' - both as interface gestures but as as resources for ongoing joint action. We go on to describe how users 'walk the blue dot' to orientate themselves, and how searches are occasioned by the local environment. In conclusion, we argue that mobile devices - rather than pushing us away from the world around us - are instead just another thread in the complex tapestry of everyday interaction.