Unveiling the technological trends of augmented reality: A patent analysis

Abstract This paper investigates the Augmented Reality (AR) technology with a novel approach based on patent research. We searched the USPTO for AR-related granted patents in the period 1993–2018, we selected and manually browse a total of 2,373, we classified them in five key technological classes i.e., display device, tracking, user interaction, application, and system, and we finally analyzed the results. The main contribution of this paper is the investigation of the technological trends, with outcomes that can be useful for researchers and developers for technical steering, but also for policymakers, managers and entrepreneurs for technology scouting and forecasting. Our study found that AR technological development has especially increased in the last decade. In particular, we evidenced a remarkable steady of 82 % annual growth rate of the number of granted patents after 2012. From geographical distribution, we found that North America is the leader (68 %); Asia (18 %) and Europe (13 %) are lagging behind despite dedicated Industry 4.0 policies actuated by the governments. Another nontrivial result is the incoherency between the owners of a high quantity of patents and those highly impacting. In fact, only Microsoft Corporation and Amazon Technologies are at the same time in the top 10 of the most patent-intensive organizations and the top 10 of highly impacting organizations. Moreover, the majority of the patents are owned by companies, albeit some of the highly impacting ones come from universities or research centers. These findings provide theoretical, managerial, and policy implications for future research activities in the AR domain.

[1]  D. Teece Profiting from technological innovation: Implications for integration, collaboration, licensing and public policy , 1993 .

[2]  Steven K. Feiner,et al.  Exploring MARS: developing indoor and outdoor user interfaces to a mobile augmented reality system , 1999, Comput. Graph..

[3]  Mu-Hsuan Huang,et al.  Industry–academia collaboration in fuel cells: a perspective from paper and patent analysis , 2015, Scientometrics.

[4]  Mark Billinghurst,et al.  A Survey of Augmented Reality , 2015, Found. Trends Hum. Comput. Interact..

[5]  Elena Novelli,et al.  An examination of the antecedents and implications of patent scope , 2015 .

[6]  Katerina Mania,et al.  Near‐Eye Display and Tracking Technologies for Virtual and Augmented Reality , 2019, Comput. Graph. Forum.

[7]  So Young Sohn,et al.  Technological convergence in standards for information and communication technologies , 2016 .

[8]  Bruce H. Thomas,et al.  First Person Indoor/Outdoor Augmented Reality Application: ARQuake , 2002, Personal and Ubiquitous Computing.

[9]  Sangwon Lee,et al.  Determinants of ICT innovations: A cross-country empirical study , 2016 .

[10]  M. Schijven,et al.  Systematic review on the effectiveness of augmented reality applications in medical training , 2016, Surgical Endoscopy.

[11]  C. Moro,et al.  The effectiveness of virtual and augmented reality in health sciences and medical anatomy , 2017, Anatomical sciences education.

[12]  Yong Gu Ji,et al.  Usability Principles for Augmented Reality Applications in a Smartphone Environment , 2013, Int. J. Hum. Comput. Interact..

[13]  Scott G. Dacko Enabling smart retail settings via mobile augmented reality shopping apps , 2017 .

[14]  John Ahmet Erkoyuncu,et al.  A systematic review of Augmented Reality content-related techniques for knowledge transfer in maintenance applications , 2018, Comput. Ind..

[15]  George Papagiannakis,et al.  A survey of mobile and wireless technologies for augmented reality systems , 2008 .

[16]  Dieter Schmalstieg,et al.  ARToolKitPlus for Pose Trackin on Mobile Devices , 2007 .

[17]  Suya You,et al.  Fusion of vision and gyro tracking for robust augmented reality registration , 2001, Proceedings IEEE Virtual Reality 2001.

[18]  Robert W. Lindeman,et al.  Hear-Through and Mic-Through Augmented Reality: Using Bone Conduction to Display Spatialized Audio , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[19]  Desney S. Tan,et al.  Developing a Generic Augmented-Reality Interface , 2002, Computer.

[20]  Volker Paelke,et al.  Augmented reality in the smart factory: Supporting workers in an industry 4.0. environment , 2014, Proceedings of the 2014 IEEE Emerging Technology and Factory Automation (ETFA).

[21]  Megan J. MacGarvie,et al.  Patents, Thickets and the Financing of Early-Stage Firms: Evidence from the Software Industry , 2007 .

[22]  A. Petruzzelli,et al.  Mapping innovation dynamics in the Internet of Things domain: Evidence from patent analysis , 2017, Technological Forecasting and Social Change.

[23]  Steven K. Feiner,et al.  Evaluating the benefits of augmented reality for task localization in maintenance of an armored personnel carrier turret , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[24]  Mihail C. Roco,et al.  Patent citation network in nanotechnology (1976–2004) , 2007 .

[25]  Takuji Narumi,et al.  Augmented reality flavors: gustatory display based on edible marker and cross-modal interaction , 2011, CHI.

[26]  Paul L. Sauer,et al.  Is Augmented Reality Technology an Effective Tool for E-commerce? An Interactivity and Vividness Perspective , 2017 .

[27]  N. Navab,et al.  Advanced Medical Displays: A Literature Review of Augmented Reality , 2008, Journal of Display Technology.

[28]  Deborah Hix,et al.  An Augmented Reality System for Military Operations in Urban Terrain , 2002 .

[29]  L. Anselin,et al.  Patents and innovation counts as measures of regional production of new knowledge , 2002 .

[30]  J. Edward Swan,et al.  Military Applications of Augmented Reality , 2011, Handbook of Augmented Reality.

[31]  Rinke Hoekstra,et al.  Patents as instruments for exploring innovation dynamics: geographic and technological perspectives on “photovoltaic cells” , 2014, Scientometrics.

[32]  Steven K. Feiner,et al.  A touring machine: Prototyping 3D mobile augmented reality systems for exploring the urban environment , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[33]  Lena L. Kronemeyer,et al.  Monitoring Competitors’ Innovation Activities: Analyzing the Competitive Patent Landscape Based on Semantic Anchor Points , 2021, IEEE Transactions on Engineering Management.

[34]  T. P. Caudell,et al.  Augmented reality: an application of heads-up display technology to manual manufacturing processes , 1992, Proceedings of the Twenty-Fifth Hawaii International Conference on System Sciences.

[35]  Dusan Tatic,et al.  The application of augmented reality technologies for the improvement of occupational safety in an industrial environment , 2017, Comput. Ind..

[36]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[37]  Andy Cockburn,et al.  User-defined gestures for augmented reality , 2013, INTERACT.

[38]  Ronald J. Mann,et al.  Patents, Venture Capital, and Software Start-Ups , 2007 .

[39]  Kangsoo Kim,et al.  Revisiting Trends in Augmented Reality Research: A Review of the 2nd Decade of ISMAR (2008–2017) , 2018, IEEE Transactions on Visualization and Computer Graphics.

[40]  Kumiko Miyazaki,et al.  Evaluating the effectiveness of keyword search strategy for patent identification , 2013 .

[41]  A. Petruzzelli,et al.  Understanding the development trends of low-carbon energy technologies: A patent analysis , 2014 .

[42]  Dieter Schmalstieg,et al.  The World as a User Interface: Augmented Reality for Ubiquitous Computing , 2007, Location Based Services and TeleCartography.

[43]  Steven K. Feiner,et al.  Exploring the Benefits of Augmented Reality Documentation for Maintenance and Repair , 2011, IEEE Transactions on Visualization and Computer Graphics.

[44]  Martin G. Moehrle,et al.  Technological speciation as a source for emerging technologies. Using semantic patent analysis for the case of camera technology , 2019, Technological Forecasting and Social Change.

[45]  Henry Fuchs,et al.  Comparison of optical and video see-through, head-mounted displays , 1995, Other Conferences.

[46]  Hayoung Choi,et al.  Innovation Topic Analysis of Technology: The Case of Augmented Reality Patents , 2018, IEEE Access.

[47]  Kiyoshi Kiyokawa,et al.  Trends and Vision of Head Mounted Display in Augmented Reality , 2012, 2012 International Symposium on Ubiquitous Virtual Reality.

[48]  Daniele Archibugi,et al.  Innovation surveys and patents as technology indicators: the state of arts , 1996 .

[49]  John Ahmet Erkoyuncu,et al.  A systematic review of augmented reality applications in maintenance , 2018 .

[50]  Andy Cockburn,et al.  FingARtips: gesture based direct manipulation in Augmented Reality , 2004, GRAPHITE '04.

[51]  Ronald Azuma,et al.  Augmented Reality: Approaches and Technical Challenges , 2001 .

[52]  John C. Tang,et al.  3D Collaboration Method over HoloLens™ and Skype™ End Points , 2015, ImmersiveME@ACM Multimedia.

[53]  Eliot Winer,et al.  Overcoming Limitations of the HoloLens for Use in Product Assembly , 2019 .

[54]  Henry Been-Lirn Duh,et al.  Trends in augmented reality tracking, interaction and display: A review of ten years of ISMAR , 2008, 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality.

[55]  K. Pavitt,et al.  Patent statistics as indicators of innovative activities: Possibilities and problems , 2005, Scientometrics.

[56]  Xiangyu Wang,et al.  Mutual awareness in collaborative design: An Augmented Reality integrated telepresence system , 2014, Comput. Ind..

[57]  Dieter Schmalstieg,et al.  Robust and unobtrusive marker tracking on mobile phones , 2008, 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality.

[58]  G. L. Cascella,et al.  Evaluating the effectiveness of spatial augmented reality in smart manufacturing: a solution for manual working stations , 2017, The International Journal of Advanced Manufacturing Technology.

[59]  Shu-Hao Chang,et al.  Identification of the technology life cycle of telematics: A patent-based analytical perspective , 2016 .

[60]  Gianpaolo Francesco Trotta,et al.  Design of a Projective AR Workbench for Manual Working Stations , 2016, AVR.

[61]  Fotis Liarokapis,et al.  An augmented reality interface for visualizing and interacting with virtual content , 2007, Virtual Reality.

[62]  Steven K. Feiner,et al.  Perceptual issues in augmented reality revisited , 2010, 2010 IEEE International Symposium on Mixed and Augmented Reality.

[63]  Malik Mallem Augmented Reality: Issues, trends and challenges , 2010, 2010 2nd International Conference on Image Processing Theory, Tools and Applications.

[64]  Soumyo D. Moitra,et al.  Innovation assessment through patent analysis , 2001 .

[65]  Inchae Park,et al.  SAO2Vec: Development of an algorithm for embedding the subject–action–object (SAO) structure using Doc2Vec , 2020, PloS one.

[66]  Janne Paavilainen,et al.  The Pokémon GO Experience: A Location-Based Augmented Reality Mobile Game Goes Mainstream , 2017, CHI.

[67]  Charles B. Owen,et al.  Review on cybersickness in applications and visual displays , 2016, Virtual Reality.

[68]  Can Huang,et al.  Nanoscience and technology publications and patents: a review of social science studies and search strategies , 2011 .

[69]  Joachim Henkel,et al.  IP Modularity: Profiting from Innovation by Aligning Product Architecture with Intellectual Property , 2012 .

[70]  Ivan E. Sutherland,et al.  A head-mounted three dimensional display , 1968, AFIPS Fall Joint Computing Conference.

[71]  Dieter Schmalstieg,et al.  Challenges of Large-Scale Augmented Reality on Smartphones , 2011 .

[72]  S. Botden,et al.  What is going on in augmented reality simulation in laparoscopic surgery? , 2008, Surgical Endoscopy.

[73]  Byeongki Jeong,et al.  Competitive Intelligence Analysis of Augmented Reality Technology Using Patent Information , 2017 .

[74]  Liangcai Cao,et al.  Progress in virtual reality and augmented reality based on holographic display. , 2018, Applied optics.

[75]  Hirokazu Kato,et al.  Marker tracking and HMD calibration for a video-based augmented reality conferencing system , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[76]  William W. Fisher,et al.  Strategic Management of Intellectual Property: An Integrated Approach , 2013 .

[77]  Nola Hewitt-Dundas,et al.  Knowledge stocks, knowledge flows and innovation: Evidence from matched patents and innovation panel data , 2015 .

[78]  J. Davidson Frame,et al.  Measuring national technological performance with patent claims data , 1994 .

[79]  Paul Coulton,et al.  Exploring the Evolution of Mobile Augmented Reality for Future Entertainment Systems , 2015, CIE.

[80]  Olivier Bau,et al.  REVEL: tactile feedback technology for augmented reality , 2012, ACM Trans. Graph..

[81]  D. Harhoff,et al.  Citation Frequency and the Value of Patented Inventions , 1999, Review of Economics and Statistics.

[82]  Sehat Ullah,et al.  A Survey on Augmented Reality Challenges and Tracking , 2016 .

[83]  M. Trajtenberg A Penny for Your Quotes : Patent Citations and the Value of Innovations , 1990 .

[84]  Giuseppe Monno,et al.  Augmented reality on large screen for interactive maintenance instructions , 2014, Comput. Ind..

[85]  So Young Sohn,et al.  Predicting the pattern of technology convergence using big-data technology on large-scale triadic patents , 2015 .

[86]  Egui Zhu,et al.  Augmented reality in healthcare education: an integrative review , 2014, PeerJ.

[87]  D. W. F. van Krevelen,et al.  A Survey of Augmented Reality Technologies, Applications and Limitations , 2010, Int. J. Virtual Real..