A survey of evaluation methods for personal route and destination prediction from mobility traces

Personal mobility data can nowadays be easily collected by personal mobile phones and used for analytical modeling. To assist in such an analysis, a variety of computational approaches have been developed. The goal is to extract mobility patterns in order to provide traveling assistance, information, recommendations or on‐demand services. While various computational techniques are being developed, research literature on destination and route prediction lacks consistency in evaluation methods for such approaches. This study presents a review and categorization of evaluation criteria and terminology used in assessing the performance of such methods. The review is complemented by experimental analysis of selected evaluation criteria, to highlight the nuances existing between the evaluation measures. The experimental study uses previously unpublished mobility data of 15 users collected over a period of 6 months in Helsinki metropolitan area in Finland. The article is primarily intended for researchers developing approaches for personalized mobility analysis, as well as a guideline for practitioners to select criteria when assessing and selecting between computational approaches. Our main recommendation is to consider user‐specific accuracy measures in addition to averaged aggregates, as well as to take into consideration that for many users accuracy does not saturate fast and the performance keeps evolving over time. Therefore, we recommend using time‐weighted measures. WIREs Data Mining Knowl Discov 2018, 8:e1237. doi: 10.1002/widm.1237

[1]  Jian Dai,et al.  Personalized route recommendation using big trajectory data , 2015, 2015 IEEE 31st International Conference on Data Engineering.

[2]  Yoshitaka Deguchi,et al.  HEV Charge/Discharge Control System Based on Navigation Information , 2004 .

[3]  Susanne Robra-Bissantz,et al.  Context-Awareness in the Car: Prediction, Evaluation and Usage of Route Trajectories , 2013, DESRIST.

[4]  Piotr Jankowski,et al.  Privacy and spatial pattern preservation in masked GPS trajectory data , 2016, Int. J. Geogr. Inf. Sci..

[5]  Stathes Hadjiefthymiades,et al.  Mobility Prediction Based on Machine Learning , 2011, 2011 IEEE 12th International Conference on Mobile Data Management.

[6]  Jukka Riekki,et al.  Personalised assistance for fuel-efficient driving , 2015 .

[7]  Wen-Jing Hsu,et al.  Predictability of individuals' mobility with high-resolution positioning data , 2012, UbiComp.

[8]  D. Levinson,et al.  Activity, Travel, and the Allocation of Time , 1995 .

[9]  Walter Zucchini,et al.  Series of Seminars: Hidden Markov Models for Time Series , 2013 .

[10]  Ling Chen,et al.  A system for destination and future route prediction based on trajectory mining , 2010, Pervasive Mob. Comput..

[11]  D. T. Lee,et al.  Travel-time prediction with support vector regression , 2004, IEEE Transactions on Intelligent Transportation Systems.

[12]  Henry A. Kautz,et al.  Learning and inferring transportation routines , 2004, Artif. Intell..

[13]  Cecilia Mascolo,et al.  NextPlace: A Spatio-temporal Prediction Framework for Pervasive Systems , 2011, Pervasive.

[14]  Indre Zliobaite,et al.  Mobile Sensing Data for Urban Mobility Analysis: A Case Study in Preprocessing , 2014, EDBT/ICDT Workshops.

[15]  Chris Schmandt,et al.  A User-Centered Location Model , 2002, Personal and Ubiquitous Computing.

[16]  Harry Zhang,et al.  Traffic Advisories Based on Route Prediction , 2007 .

[17]  Anna Monreale,et al.  WhereNext: a location predictor on trajectory pattern mining , 2009, KDD.

[18]  Mykola Pechenizkiy,et al.  Context-Aware Personal Route Recognition , 2011, Discovery Science.

[19]  Gediminas Adomavicius,et al.  Context-aware recommender systems , 2008, RecSys '08.

[20]  Luca Martino,et al.  Multi-label methods for prediction with sequential data , 2015, Pattern Recognit..

[21]  Xing Xie,et al.  Destination prediction by sub-trajectory synthesis and privacy protection against such prediction , 2013, 2013 IEEE 29th International Conference on Data Engineering (ICDE).

[22]  Le Minh Kieu,et al.  Transit passenger segmentation using travel regularity mined from Smart Card transactions data , 2014 .

[23]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[24]  Ling Chen,et al.  A personal route prediction system based on trajectory data mining , 2011, Inf. Sci..

[25]  Jesse Read,et al.  Labeling sensing data for mobility modeling , 2016, Inf. Syst..

[26]  Luis González Abril,et al.  Trip destination prediction based on past GPS log using a Hidden Markov Model , 2010, Expert Syst. Appl..

[27]  Eric Horvitz,et al.  Trip Router with Individualized Preferences (TRIP): Incorporating Personalization into Route Planning , 2006, AAAI.

[28]  Xing Xie,et al.  Solving the data sparsity problem in destination prediction , 2015, The VLDB Journal.

[29]  João Gama,et al.  Validating the coverage of bus schedules: A Machine Learning approach , 2015, Inf. Sci..

[30]  Lorenzo Blanco,et al.  Future Locations Prediction with Uncertain Data , 2013, ECML/PKDD.

[31]  Vibhav Gogate,et al.  Modeling Transportation Routines using Hybrid Dynamic Mixed Networks , 2005, UAI.

[32]  Eamonn J. Keogh,et al.  A symbolic representation of time series, with implications for streaming algorithms , 2003, DMKD '03.

[33]  César A. Hidalgo,et al.  Unique in the Crowd: The privacy bounds of human mobility , 2013, Scientific Reports.

[34]  Kari Laasonen.  Route prediction from cellular data , 2005 .

[35]  João Gama,et al.  Predicting Taxi–Passenger Demand Using Streaming Data , 2013, IEEE Transactions on Intelligent Transportation Systems.

[36]  Tsutomu Terada,et al.  A destination prediction method using driving contexts and trajectory for car navigation systems , 2009, SAC '09.

[37]  Apollinaire Nadembega,et al.  A Destination Prediction Model based on historical data, contextual knowledge and spatial conceptual maps , 2012, 2012 IEEE International Conference on Communications (ICC).

[38]  Jure Leskovec,et al.  Friendship and mobility: user movement in location-based social networks , 2011, KDD.

[39]  Jukka Riekki,et al.  Continuous Travel Time Prediction for Transit Signal Priority Based on a Deep Network , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[40]  Qiming Chen,et al.  PrefixSpan,: mining sequential patterns efficiently by prefix-projected pattern growth , 2001, Proceedings 17th International Conference on Data Engineering.

[41]  Hui Xiong,et al.  Achieving Guaranteed Anonymity in GPS Traces via Uncertainty-Aware Path Cloaking , 2010, IEEE Transactions on Mobile Computing.

[42]  Stephen P. Boyd,et al.  Finding Ultimate Limits of Performance for Hybrid Electric Vehicles , 2000 .

[43]  Yin Wang,et al.  Mining large-scale, sparse GPS traces for map inference: comparison of approaches , 2012, KDD.

[44]  Alexander Mendiburu,et al.  A Review of Travel Time Estimation and Forecasting for Advanced Traveler Information Systems , 2012 .

[45]  Brett Browning,et al.  Learning to Predict Driver Route and Destination Intent , 2006, 2006 IEEE Intelligent Transportation Systems Conference.

[46]  John Krumm,et al.  Route Prediction from Trip Observations , 2008 .

[47]  Michael Hunter,et al.  Data-Driven Vehicle Trajectory Prediction , 2016, SIGSIM-PADS.

[48]  John Krumm,et al.  A survey of computational location privacy , 2009, Personal and Ubiquitous Computing.

[49]  David Barber,et al.  Bayesian reasoning and machine learning , 2012 .

[50]  Eric Horvitz,et al.  Predestination: Inferring Destinations from Partial Trajectories , 2006, UbiComp.

[51]  John Krumm,et al.  From GPS traces to a routable road map , 2009, GIS.

[52]  Hassan A. Karimi,et al.  A predictive location model for location-based services , 2003, GIS '03.

[53]  Zhi-Hua Zhou,et al.  B-Planner: Night bus route planning using large-scale taxi GPS traces , 2013, 2013 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[54]  Gennady L. Andrienko,et al.  A visual analytics framework for spatio-temporal analysis and modelling , 2013, Data Mining and Knowledge Discovery.

[55]  Mingqi Lv,et al.  Route Pattern Mining From Personal Trajectory Data , 2015 .

[56]  John Krumm,et al.  From destination prediction to route prediction , 2013, J. Locat. Based Serv..

[57]  Fredrik Gustafsson,et al.  On-road trajectory generation from GPS data: A particle filtering/smoothing application , 2012, 2012 15th International Conference on Information Fusion.

[58]  Andrew P. Bradley,et al.  The use of the area under the ROC curve in the evaluation of machine learning algorithms , 1997, Pattern Recognit..

[59]  Albert-László Barabási,et al.  Understanding individual human mobility patterns , 2008, Nature.

[60]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[61]  Thad Starner,et al.  Using GPS to learn significant locations and predict movement across multiple users , 2003, Personal and Ubiquitous Computing.

[62]  Albert-László Barabási,et al.  Limits of Predictability in Human Mobility , 2010, Science.

[63]  John Krumm,et al.  Real Time Destination Prediction Based On Efficient Routes , 2006 .

[64]  Anna Monreale,et al.  MyWay: Location prediction via mobility profiling , 2017, Inf. Syst..

[65]  Geoff Holmes,et al.  Evaluation methods and decision theory for classification of streaming data with temporal dependence , 2015, Machine Learning.

[66]  Karin Brundell-Freij,et al.  Optimizing route choice for lowest fuel consumption - Potential effects of a new driver support tool , 2006 .

[67]  Antonio F. Gómez-Skarmeta,et al.  Online route prediction based on clustering of meaningful velocity-change areas , 2016, Data Mining and Knowledge Discovery.

[68]  Gregory D. Abowd,et al.  A Conceptual Framework and a Toolkit for Supporting the Rapid Prototyping of Context-Aware Applications , 2001, Hum. Comput. Interact..

[69]  R. Pendyala,et al.  A structural equations analysis of commuters' activity and travel patterns , 2001 .

[70]  Yoshua Bengio,et al.  Markovian Models for Sequential Data , 2004 .

[71]  Guanling Chen,et al.  A Survey of Context-Aware Mobile Computing Research , 2000 .

[72]  Ling Chen,et al.  Predict Personal Continuous Route , 2008, 2008 11th International IEEE Conference on Intelligent Transportation Systems.

[73]  Anind K. Dey,et al.  Navigate like a cabbie: probabilistic reasoning from observed context-aware behavior , 2008, UbiComp.

[74]  M.N.S. Swamy,et al.  Neural Networks and Statistical Learning , 2013 .

[75]  John D. Hunter,et al.  Matplotlib: A 2D Graphics Environment , 2007, Computing in Science & Engineering.

[76]  Kazunari Nawa,et al.  Combining prediction methods with cyber information for proactive route recommendation , 2013, 2013 IEEE International Conference on Cyber Technology in Automation, Control and Intelligent Systems.