Visuelle Verfahren für ortsbezogene Dienste

Durch die rasante Entwicklung und Verbreitung mobiler Endgerate, wie z.B. Smartphones, Tablets und Wearables, ist die Anzahl ortsbezogener Dienste in den letzten Jahren enorm angestiegen. Die Positionsbestimmung des Nutzers ist fur solche Dienste zwingend notwendig. Funkbasierte Verfahren wie zum Beispiel GPS oder WLAN ermoglichen die Lokalisierung des Nutzers. Trotz der hohen Genauigkeit und der stetigen Weiterentwicklung der Verfahren haben diese Technologien ihre Grenzen und stehen nicht immer zur Verfugung, weshalb andere Losungen unterstutzend oder als Alternative zum Einsatz kommen mussen. Optische Sensoren, speziell Kamerasensoren oder sogenannte Actioncams, bieten hierfur eine Alternative an. Die Kombination aus hochauflosenden Kameras und der Leistungsfahigkeit mobiler Endgerate bietet die Moglichkeit Methoden zur Positionsbestimmung und Aktivitatserkennung zu realisieren. Obwohl bereits heute einige visuelle Verfahren fur ortsbezogene Dienste genutzt werden, steckt die Entwicklung auf diesem Gebiet immer noch in den Kinderschuhen. Daher werden in der vorliegenden Arbeit neuartige und erweiterte visuelle Verfahren zur Positionsbestimmung und Aktivitatserkennung vorgestellt, welche rein auf der Kamera mobiler Endgerate basierend umgesetzt werden. Dafur werden markante visuelle Merkmale aus Bildsequenzen extrahiert und fur die Bestimmung der relativen und absoluten Position verwendet. Zusatzlich werden die selben markanten Merkmale zur Bestimmung der aktuellen Aktivitat eines Nutzers genutzt. In der vorliegenden Arbeit wird ein Verfahren zum effizienten Vergleich von Bildinformationen vorgestellt. Dazu wird ein Prozess entwickelt, der aus mehreren Vergleichsstufen besteht. Ahnlich wie bei einem Siebverfahren werden dabei stufenweise falsche Bilder aussortiert. Ziel ist es, ein Verfahren zu entwickeln, das fotografierte Bildsequenzen von Objekten oder ganzen Standorten in kurzester Zeit eindeutig wieder erkennen kann. Anschliesend werden drei neue Erweiterungen fur ein visuelles Positionierungssystem, namens MoViPS, vorgestellt. Diese Erweiterungen sollen das System fur einen Echtzeitbetrieb nutzbar machen. Zwei Erweiterungen verkurzen allgemein die Antwortzeit und machen gleichzeitig den gesamten Prozess effizienter und robuster. Die dritte Erweiterung verbessert das bestehende Positionskorrekturverfahren aus MoViPS. Im letzten Teil dieser Arbeit wird ein Verfahren der visuellen Odometrie vorgestellt, das auf Basis eines Kamerasensors funktioniert, der aus der Ego-Perspektive einer Person aufzeichnet. Anders als bekannte State-of-the-Art-Verfahren basiert das Verfahren auf einem visuellen Schrittzahler und einem visuellen Kompass. Zusatzlich wird eine visuelle Aktivitatserkennung vorgestellt. Mit Hilfe der Eigenschaften markanter Merkmale, die aus einer Bildsequenz extrahiert werden, konnen Schritte sowie Aktivitaten erkannt werden. Die Beitrage der vorliegenden Arbeit liefern somit wichtige Grundbausteine fur die Entwicklung und Erweiterung visueller Positionierungssysteme. Sie bieten zusatzlich visuelle Verfahren fur ortsbezogene Dienste an, die die einfache Position und Aktivitaten eines Nutzers bestimmen konnen.

[1]  Dariu Gavrila,et al.  The Visual Analysis of Human Movement: A Survey , 1999, Comput. Vis. Image Underst..

[2]  Andrew Zisserman,et al.  Near Duplicate Image Detection: min-Hash and tf-idf Weighting , 2008, BMVC.

[3]  Chadly Marouane,et al.  Indoor positioning using smartphone camera , 2011, 2011 International Conference on Indoor Positioning and Indoor Navigation.

[4]  James R. Bergen,et al.  Visual odometry for ground vehicle applications , 2006, J. Field Robotics.

[5]  A. El-Rabbany Introduction to GPS: The Global Positioning System , 2002 .

[6]  Taku Komura,et al.  A Virtual Reality Dance Training System Using Motion Capture Technology , 2011, IEEE Transactions on Learning Technologies.

[7]  Tony Lindeberg,et al.  Feature Detection with Automatic Scale Selection , 1998, International Journal of Computer Vision.

[8]  Luc Van Gool,et al.  Content-Based Image Retrieval Based on Local Affinely Invariant Regions , 1999, VISUAL.

[9]  David L. Milgram,et al.  Computer Methods for Creating Photomosaics , 1975, IEEE Transactions on Computers.

[10]  C. Schmid,et al.  Indexing based on scale invariant interest points , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[11]  Claudia Linnhoff-Popien,et al.  Step and activity detection based on the orientation and scale attributes of the SURF algorithm , 2016, 2016 International Conference on Indoor Positioning and Indoor Navigation (IPIN).

[12]  Francesco Camastra,et al.  Automatic Face Recognition , 2015 .

[13]  Diogo R. Ferreira,et al.  Preprocessing techniques for context recognition from accelerometer data , 2010, Personal and Ubiquitous Computing.

[14]  Daniel P. Huttenlocher,et al.  Comparing Images Using the Hausdorff Distance , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Yan Ke,et al.  PCA-SIFT: a more distinctive representation for local image descriptors , 2004, CVPR 2004.

[16]  Xiao Zhang,et al.  An iBeacon-based Indoor Positioning Systems for Hospitals , 2015 .

[17]  Hans P. Morevec Towards automatic visual obstacle avoidance , 1977, IJCAI 1977.

[18]  Ardeshir Goshtasby,et al.  On the Canny edge detector , 2001, Pattern Recognit..

[19]  Jitendra Malik,et al.  Shape Context: A New Descriptor for Shape Matching and Object Recognition , 2000, NIPS.

[20]  C. Koch,et al.  Computational modelling of visual attention , 2001, Nature Reviews Neuroscience.

[21]  Kurt Konolige,et al.  CenSurE: Center Surround Extremas for Realtime Feature Detection and Matching , 2008, ECCV.

[22]  Hermann Winner,et al.  Handbuch Fahrerassistenzsysteme, Grundlagen, Komponenten und Systeme für aktive Sicherheit und Komfort , 2009 .

[23]  Jiri Matas,et al.  Linear Regression and Adaptive Appearance Models for Fast Simultaneous Modelling and Tracking , 2010, International Journal of Computer Vision.

[24]  Widyawan,et al.  Smartphone-based Pedestrian Dead Reckoning as an indoor positioning system , 2012, 2012 International Conference on System Engineering and Technology (ICSET).

[25]  Matthew S. Gast Building Applications with iBeacon: Proximity and Location Services with Bluetooth Low Energy , 2014 .

[26]  Chia-Yen Chen,et al.  Visual Odometry with Improved Adaptive Feature Tracking , 2014, IVCNZ '14.

[27]  Da-Wen Sun,et al.  Improving quality inspection of food products by computer vision: a review , 2004 .

[28]  Ramakant Nevatia,et al.  Tracking multiple humans in crowded environment , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[29]  Kostas Daniilidis,et al.  Monocular visual odometry in urban environments using an omnidirectional camera , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[30]  D. J. Langridge,et al.  Curve encoding and the detection of discontinuities , 1982, Comput. Graph. Image Process..

[31]  Garry Wei-Han Tan,et al.  NFC mobile credit card: The next frontier of mobile payment? , 2014, Telematics Informatics.

[32]  Adam Baumberg,et al.  Reliable feature matching across widely separated views , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[33]  Larry H. Matthies,et al.  Visual odometry on the Mars Exploration Rovers , 2005, 2005 IEEE International Conference on Systems, Man and Cybernetics.

[34]  Peter I. Corke,et al.  A Hybrid AUV Design for Shallow Water Reef Navigation , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[35]  Edificio Ada Byron Scaled Monocular SLAM for Walking People , 2013 .

[36]  Ales Leonardis,et al.  High-Dimensional Feature Matching: Employing the Concept of Meaningful Nearest Neighbors , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[37]  Kiyoharu Aizawa,et al.  Image-based indoor positioning system: fast image matching using omnidirectional panoramic images , 2010, MPVA '10.

[38]  Andrew Zisserman,et al.  Automated Scene Matching in Movies , 2002, CIVR.

[39]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[40]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[41]  Marco Maier,et al.  Fine-Grained Activity Recognition of Pedestrians Travelling by Subway , 2013, MobiCASE.

[42]  Farzin Mokhtarian,et al.  Robust Image Corner Detection Through Curvature Scale Space , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[43]  Pittsburgh,et al.  The MOPED framework: Object recognition and pose estimation for manipulation , 2011 .

[44]  Cordelia Schmid,et al.  A Performance Evaluation of Local Descriptors , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[45]  Ronald L. Rivest,et al.  Introduction to Algorithms, Second Edition , 2001 .

[46]  Haim J. Wolfson,et al.  Geometric hashing: an overview , 1997 .

[47]  A. Bab-Hadiashar,et al.  An Overview to Visual Odometry and Visual SLAM: Applications to Mobile Robotics , 2015 .

[48]  Paolo Pirjanian,et al.  The vSLAM Algorithm for Robust Localization and Mapping , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[49]  F. Fraundorfer,et al.  Visual Odometry : Part II: Matching, Robustness, Optimization, and Applications , 2012, IEEE Robotics & Automation Magazine.

[50]  Claudia Linnhoff-Popien,et al.  Visual Odometry for Pedestrians Based on Orientation Attributes of SURF , 2016, IntelliSys.

[51]  Klaus Wehrle,et al.  FootPath: Accurate map-based indoor navigation using smartphones , 2011, 2011 International Conference on Indoor Positioning and Indoor Navigation.

[52]  Serge J. Belongie,et al.  Context based object categorization: A critical survey , 2010, Comput. Vis. Image Underst..

[53]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[54]  Zachary Fitz-Walter,et al.  Simple classification of walking activities using commodity smart phones , 2009, OZCHI '09.

[55]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[56]  Sergio A. Velastin,et al.  Detection and classification of vehicles for urban traffic scenes , 2008 .

[57]  Larry H. Matthies,et al.  Two years of Visual Odometry on the Mars Exploration Rovers , 2007, J. Field Robotics.

[58]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[59]  Elena A. Fedorovskaya,et al.  Digital Image Processing and Analysis , 2010 .

[60]  Hugh F. Durrant-Whyte,et al.  Simultaneous localization and mapping: part I , 2006, IEEE Robotics & Automation Magazine.

[61]  Sergei Vassilvitskii,et al.  k-means++: the advantages of careful seeding , 2007, SODA '07.

[62]  Trevor Darrell,et al.  Integrated Person Tracking Using Stereo, Color, and Pattern Detection , 2000, International Journal of Computer Vision.

[63]  Anas Al-Nuaimi,et al.  Mobile Visual Location Recognition , 2013 .

[64]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[65]  Friedrich Fraundorfer,et al.  Visual Odometry Part I: The First 30 Years and Fundamentals , 2022 .

[66]  Hend Suliman Al-Khalifa Utilizing QR Code and Mobile Phones for Blinds and Visually Impaired People , 2008, ICCHP.

[67]  C. Cobelli,et al.  A Markerless Motion Capture System to Study Musculoskeletal Biomechanics: Visual Hull and Simulated Annealing Approach , 2006, Annals of Biomedical Engineering.

[68]  Maren Bennewitz,et al.  Humanoid robot localization in complex indoor environments , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[69]  Bernd Girod,et al.  Mobile Visual Search , 2011, IEEE Signal Processing Magazine.

[70]  Antonio Torralba,et al.  Context-based vision system for place and object recognition , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[71]  James W. Cooper,et al.  Early jump-out corner detectors , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[72]  Andreas Schierwagen,et al.  Bildverstehen in der KI: Konzepte und Probleme * , 1999 .

[73]  Ravindra C. Thool,et al.  Moving Vehicle Detection for Measuring Traffic Count Using OpenCV , 2013 .

[74]  M. Darianian,et al.  Smart Home Mobile RFID-Based Internet-of-Things Systems and Services , 2008, 2008 International Conference on Advanced Computer Theory and Engineering.

[75]  Gabe Sibley,et al.  Sliding window filter with application to planetary landing , 2010 .

[76]  I. Haritaoglu,et al.  Background and foreground modeling using nonparametric kernel density estimation for visual surveillance , 2002 .

[77]  Oliver Schreer Stereoanalyse und Bildsynthese , 2007 .

[78]  Seth J. Teller,et al.  Online pose classification and walking speed estimation using handheld devices , 2012, UbiComp '12.

[79]  G. Klein,et al.  Parallel Tracking and Mapping for Small AR Workspaces , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[80]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[81]  Marco Maier,et al.  Visual positioning systems — An extension to MoVIPS , 2014, 2014 International Conference on Indoor Positioning and Indoor Navigation (IPIN).

[82]  Takeo Kanade,et al.  Image matching in large scale indoor environment , 2009, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[83]  P. Anandan,et al.  About Direct Methods , 1999, Workshop on Vision Algorithms.

[84]  Tony Lindeberg,et al.  Detecting salient blob-like image structures and their scales with a scale-space primal sketch: A method for focus-of-attention , 1993, International Journal of Computer Vision.

[85]  Gaurav S. Sukhatme,et al.  Combined Visual and Inertial Navigation for an Unmanned Aerial Vehicle , 2008, FSR.

[86]  James L. Crowley,et al.  A Representation for Shape Based on Peaks and Ridges in the Difference of Low-Pass Transform , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[87]  Rolf P. Würtz,et al.  Corner detection in color images through a multiscale combination of end-stopped cortical cells , 2000, Image Vis. Comput..

[88]  Chadly Marouane,et al.  Classification of Vehicle Types in Car Parks using Computer Vision Techniques , 2015, EAI Endorsed Trans. Energy Web.

[89]  Peter I. Corke,et al.  Omnidirectional visual odometry for a planetary rover , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[90]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[91]  Claudia Linnhoff-Popien,et al.  Visual odometry using motion vectors from visual feature points , 2016, 2016 International Conference on Indoor Positioning and Indoor Navigation (IPIN).

[92]  Axel Küpper Location-based Services: Fundamentals and Operation , 2005 .

[93]  Masahiro Fujita,et al.  A Floor and Obstacle Height Map for 3D Navigation of a Humanoid Robot , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[94]  Andrew Walsh,et al.  Blurring the boundaries between our physical and electronic libraries: Location-aware technologies, QR codes and RFID tags , 2011, Electron. Libr..

[95]  Davide Scaramuzza,et al.  SVO: Fast semi-direct monocular visual odometry , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[96]  Matthew A. Brown,et al.  Invariant Features from Interest Point Groups , 2002, BMVC.

[97]  Jing Liu,et al.  Survey of Wireless Indoor Positioning Techniques and Systems , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[98]  Paul Beaudet,et al.  Rotationally invariant image operators , 1978 .

[99]  J. Scharinger,et al.  Neue Dimension von mobilen Tourismusanwendungen durch Near Field Communication-Technologie , 2010 .

[100]  P.V.C. Hough,et al.  Machine Analysis of Bubble Chamber Pictures , 1959 .

[101]  Edgar Stüssi,et al.  Ganganalyse beim Gehen und Laufen , 2008 .

[102]  Chadly Marouane,et al.  SURFLogo - Mobile Tagging with App Icons , 2015, MobiCASE.

[103]  Wolfram Höpken,et al.  Application of QR Codes in Online Travel Distribution , 2010, ENTER.

[104]  Herbert Freeman,et al.  Computer Processing of Line-Drawing Images , 1974, CSUR.

[105]  Hans-Hellmut Nagel,et al.  Volumetric model and 3D trajectory of a moving car derived from monocular TV frame sequences of a street scene , 1981, Comput. Graph. Image Process..

[106]  Jiri Matas,et al.  Robust wide-baseline stereo from maximally stable extremal regions , 2004, Image Vis. Comput..

[107]  Ana Belén Barragáns-Martínez,et al.  QR-Maps: An efficient tool for indoor user location based on QR-Codes and Google maps , 2011, 2011 IEEE Consumer Communications and Networking Conference (CCNC).

[108]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[109]  J. Tukey,et al.  An algorithm for the machine calculation of complex Fourier series , 1965 .

[110]  Ruizhi Chen,et al.  Ubiquitous Positioning and Mobile Location-Based Services in Smart Phones , 2012 .

[111]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[112]  Zhongfei Zhang,et al.  A survey of appearance models in visual object tracking , 2013, ACM Trans. Intell. Syst. Technol..

[113]  Maxime Lhuillier,et al.  Automatic scene structure and camera motion using a catadioptric system , 2008, Comput. Vis. Image Underst..

[114]  Carsten Isert,et al.  Self-contained indoor positioning on off-the-shelf mobile devices , 2010, 2010 International Conference on Indoor Positioning and Indoor Navigation.

[115]  Robin Ashford,et al.  QR codes and academic libraries Reaching mobile users , 2010 .

[116]  Hans P. Moravec Obstacle avoidance and navigation in the real world by a seeing robot rover , 1980 .

[117]  L. Kitchen,et al.  The dissimilarity corner detector , 1991, Fifth International Conference on Advanced Robotics 'Robots in Unstructured Environments.

[118]  Anupam Agrawal,et al.  Vision based hand gesture recognition for human computer interaction: a survey , 2012, Artificial Intelligence Review.

[119]  Agata Brajdic,et al.  Walk detection and step counting on unconstrained smartphones , 2013, UbiComp.

[120]  Mehmet N. Aydin,et al.  Development of an Indoor Navigation System Using NFC Technology , 2011, 2011 Fourth International Conference on Information and Computing.

[121]  Luc Van Gool,et al.  Wide Baseline Stereo Matching based on Local, Affinely Invariant Regions , 2000, BMVC.

[122]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[123]  Paul Geladi,et al.  Principal Component Analysis , 1987, Comprehensive Chemometrics.

[124]  Hugh Durrant-Whyte,et al.  Simultaneous localization and mapping (SLAM): part II , 2006 .

[125]  Matthew Turk,et al.  Car-Rec: A real time car recognition system , 2011, 2011 IEEE Workshop on Applications of Computer Vision (WACV).

[126]  J. MacQueen Some methods for classification and analysis of multivariate observations , 1967 .

[127]  Markus Maurer,et al.  Autonomes Fahren , 2015, Handbuch Fahrerassistenzsysteme.

[128]  David W. Murray,et al.  Parallel Tracking and Mapping on a camera phone , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[129]  Osama Masoud,et al.  Detection and classification of vehicles , 2002, IEEE Trans. Intell. Transp. Syst..

[130]  Javier Ibanez Guzman,et al.  Accurate visual odometry from a rear parking camera , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[131]  Nidhi Sethi,et al.  Automatic License Plate Recognition System using SURF Features and RBF Neural Network , 2013 .

[132]  Stefano Messelodi,et al.  A computer vision system for the detection and classification of vehicles at urban road intersections , 2005, Pattern Analysis and Applications.

[133]  Richard T. Watson,et al.  Location-based services , 2008, CACM.

[134]  Hideki Koike,et al.  Robust vSLAM for Dynamic Scenes , 2011, MVA.

[135]  Hans P. Moravec Visual Mapping by a Robot Rover , 1979, IJCAI.

[136]  James R. Bergen,et al.  Visual odometry , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[137]  Gaetano Borriello,et al.  Information Overlay for Camera Phones in Indoor Environments , 2007, LoCA.

[138]  Larry H. Matthies,et al.  Visual odometry on the Mars exploration rovers - a tool to ensure accurate driving and science imaging , 2006, IEEE Robotics & Automation Magazine.

[139]  Andrew Zisserman,et al.  Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[140]  Andrew J. Davison,et al.  DTAM: Dense tracking and mapping in real-time , 2011, 2011 International Conference on Computer Vision.

[141]  Heinrich Mensen Handbuch der Luftfahrt , 2013 .

[142]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[143]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[144]  Dong-Hwan Hwang,et al.  A Step, Stride and Heading Determination for the Pedestrian Navigation System , 2004 .

[145]  石井 聡,et al.  改良型 Wiener filter の研究 , 2009 .