Interactive Sports Analytics

Analytics in professional sports has experienced a dramatic growth in the last decade due to the wide deployment of player and ball tracking systems in team sports, such as basketball and soccer. With the massive amount of fine-grained data being generated, new data-points are being generated, which can shed light on player and team performance. However, due to the complexity of plays in continuous sports, these data-points often lack the specificity and context to enable meaningful retrieval and analytics. In this article, we present an intelligent human--computer interface that utilizes trajectories instead of words, which enables specific play retrieval in sports. Various techniques of alignment, templating, and hashing were utilized by our system and they are tailored to multi-agent scenario so that interactive speeds can be achieved. We conduct a user study to compare our method to the conventional keywords-based system and the results show that our method significantly improves the retrieval quality. We also show how our interface can be utilized for broadcast purposes, where a user can draw and interact with trajectories on a broadcast view using computer vision techniques. Additionally, we show that our method can also be used for interactive analytics of player performance, which enables the users to move players around and see how performance changes as a function of position and proximity to other players.

[1]  Piotr Indyk,et al.  Similarity Search in High Dimensions via Hashing , 1999, VLDB.

[2]  Rong Yan,et al.  Semantic concept-based query expansion and re-ranking for multimedia retrieval , 2007, ACM Multimedia.

[3]  Cheng Jin,et al.  Sketch-Based Image Retrieval via Adaptive Weighting , 2014, ICMR.

[4]  Filip Radlinski,et al.  Large-scale validation and analysis of interleaved search evaluation , 2012, TOIS.

[5]  Ye Zhao,et al.  TenniVis: Visualization for Tennis Match Analysis , 2014, IEEE Transactions on Visualization and Computer Graphics.

[6]  Yaser Sheikh,et al.  Point-less calibration: Camera parameters from gradient-based alignment to edge images , 2012, 2012 IEEE Workshop on the Applications of Computer Vision (WACV).

[7]  Ehsanollah Kabir,et al.  A query-by-example music retrieval system using feature and decision fusion , 2018, Multimedia Tools and Applications.

[8]  Gita Reese Sukthankar,et al.  Activity Recognition for Dynamic Multi-Agent Teams , 2011, TIST.

[9]  Nicu Sebe,et al.  Academic Coupled Dictionary Learning for Sketch-based Image Retrieval , 2016, ACM Multimedia.

[10]  Hung-Kuo Chu,et al.  Court Reconstruction for Camera Calibration in Broadcast Basketball Videos. , 2016, IEEE transactions on visualization and computer graphics.

[11]  Jianhui Chen,et al.  Mimicking Human Camera Operators , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[12]  Dafna Shahaf,et al.  Metro maps of science , 2012, KDD.

[13]  Anthony K. H. Tung,et al.  SpADe: On Shape-based Pattern Detection in Streaming Time Series , 2007, 2007 IEEE 23rd International Conference on Data Engineering.

[14]  Matthew Louis Mauriello,et al.  BodyVis: A New Approach to Body Learning Through Wearable Sensing and Visualization , 2015, CHI.

[15]  Kristina Höök,et al.  Designing and evaluating kalas: A social navigation system for food recipes , 2005, TCHI.

[16]  Zhouyu Fu,et al.  Semantic-Based Surveillance Video Retrieval , 2007, IEEE Transactions on Image Processing.

[17]  Filip Radlinski,et al.  Inferring and using location metadata to personalize web search , 2011, SIGIR.

[18]  Tieniu Tan,et al.  Comparison of Similarity Measures for Trajectory Clustering in Outdoor Surveillance Scenes , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[19]  Florian Daiber,et al.  ClimbSense: Automatic Climbing Route Recognition using Wrist-worn Inertia Measurement Units , 2015, CHI.

[20]  Yaser Sheikh,et al.  Representing and Discovering Adversarial Team Behaviors Using Player Roles , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Bir Bhanu,et al.  Discrete Cosine Transform Locality-Sensitive Hashes for Face Retrieval , 2014, IEEE Transactions on Multimedia.

[22]  Yisong Yue,et al.  Chalkboarding: A New Spatiotemporal Query Paradigm for Sports Play Retrieval , 2016, IUI.

[23]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[24]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[25]  Tieniu Tan,et al.  Semantic interpretation of object activities in a surveillance system , 2002, Object recognition supported by user interaction for service robots.

[26]  Loong Fah Cheong,et al.  Inserting 3D projected virtual content into broadcast tennis video , 2006, MM '06.

[27]  Olivier Chapelle,et al.  Expected reciprocal rank for graded relevance , 2009, CIKM.

[28]  Eamonn Keogh Exact Indexing of Dynamic Time Warping , 2002, VLDB.

[29]  Xuelong Li,et al.  EMIF: Towards a Scalable and Effective Indexing Framework for Large Scale Music Retrieval , 2015, ICMR.

[30]  Matt Duckham,et al.  Trajectory similarity measures , 2015, SIGSPACIAL.

[31]  Andreas Krause,et al.  Scalable Training of Mixture Models via Coresets , 2011, NIPS.

[32]  L. Bornn,et al.  Counterpoints : Advanced Defensive Metrics for NBA , 2022 .

[33]  Sridha Sridharan,et al.  Large-Scale Analysis of Soccer Matches Using Spatiotemporal Tracking Data , 2014, 2014 IEEE International Conference on Data Mining.

[34]  Sridha Sridharan,et al.  Large-Scale Analysis of Formations in Soccer , 2013, 2013 International Conference on Digital Image Computing: Techniques and Applications (DICTA).

[35]  Christos Faloutsos,et al.  FTW: fast similarity search under the time warping distance , 2005, PODS.

[36]  Carlos Guestrin,et al.  Beyond keyword search: discovering relevant scientific literature , 2011, KDD.

[37]  Gabriela Csurka,et al.  Semantic combination of textual and visual information in multimedia retrieval , 2011, ICMR.

[38]  Ling Shao,et al.  Fast action retrieval from videos via feature disaggregation , 2017, Comput. Vis. Image Underst..

[39]  Mubarak Shah,et al.  Multi feature path modeling for video surveillance , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[40]  Bo Li,et al.  3D Sketch-Based 3D Model Retrieval , 2015, ICMR.

[41]  Christoph Bartneck,et al.  Digitally Augmenting Sports: An Opportunity for Exploring and Understanding Novel Balancing Techniques , 2016, CHI.

[42]  Jaime Teevan,et al.  Trajectory-aware mobile search , 2012, CHI.

[43]  Christopher J. C. Burges,et al.  From RankNet to LambdaRank to LambdaMART: An Overview , 2010 .

[44]  Mark J. Perry,et al.  Lean collaboration through video gestures: co-ordinating the production of live televised sport , 2009, CHI.

[45]  Woohun Lee,et al.  Recording and Sharing Non-Visible Information on Body Movement while Skateboarding , 2017, CHI.

[46]  Alexandre Yahi,et al.  Aurigo: an Interactive Tour Planner for Personalized Itineraries , 2015, IUI.

[47]  Piotr Indyk,et al.  Approximate nearest neighbors: towards removing the curse of dimensionality , 1998, STOC '98.

[48]  Deb Roy,et al.  Unsupervised content-based indexing of sports video , 2007, MIR '07.

[49]  Yisong Yue,et al.  “Win at Home and Draw Away”: Automatic Formation Analysis Highlighting the Differences in Home and Away Team Behaviors , 2014 .

[50]  Sariel Har-Peled,et al.  On coresets for k-means and k-median clustering , 2004, STOC '04.

[51]  Ryan P. Adams,et al.  Factorized Point Process Intensities: A Spatial Analysis of Professional Basketball , 2014, ICML.

[52]  Zhouyu Fu Improving Feature Aggregation for Semantic Music Retrieval , 2015, ACM Multimedia.

[53]  Giorgio Gandolfi NBA coaches playbook : techniques, tactics, and teaching points , 2009 .

[54]  Liqing Zhang,et al.  Sketch-based Image Retrieval via Shape Words , 2015, ICMR.

[55]  Frank M. Shipman,et al.  DOTS: support for effective video surveillance , 2007, ACM Multimedia.

[56]  Jenna Wiens,et al.  Automatically Recognizing On-ball Screens , 2014 .

[57]  Emanuel Zgraggen,et al.  Evaluating Subjective Accuracy in Time Series Pattern-Matching Using Human-Annotated Rankings , 2015, IUI.

[58]  Thorsten Joachims,et al.  Dynamic ranked retrieval , 2011, WSDM '11.

[59]  Joe Marshall,et al.  HCI and sports , 2014, Interactions.

[60]  Alan Fern,et al.  Play type recognition in real-world football video , 2014, IEEE Winter Conference on Applications of Computer Vision.

[61]  Filip Radlinski,et al.  A support vector method for optimizing average precision , 2007, SIGIR.

[62]  Kirk Goldsberry,et al.  POINTWISE: Predicting Points and Valuing Decisions in Real Time with NBA Optical Tracking Data , 2014 .

[63]  Tao Mei,et al.  Just-for-Me: An Adaptive Personalization System for Location-Aware Social Music Recommendation , 2014, ICMR.

[64]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[65]  J. Shane Culpepper,et al.  Indexing Word Sequences for Ranked Retrieval , 2014, TOIS.

[66]  Roger Levy,et al.  A new approach to cross-modal multimedia retrieval , 2010, ACM Multimedia.

[67]  Eamonn J. Keogh,et al.  A symbolic representation of time series, with implications for streaming algorithms , 2003, DMKD '03.

[68]  Harold W. Kuhn,et al.  The Hungarian method for the assignment problem , 1955, 50 Years of Integer Programming.

[69]  Yaser Sheikh,et al.  One-man-band: a touch screen interface for producing live multi-camera sports broadcasts , 2013, MM '13.

[70]  Radford M. Neal,et al.  Multiple Alignment of Continuous Time Series , 2004, NIPS.

[71]  Ming Yang,et al.  Query Specific Rank Fusion for Image Retrieval , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[72]  Aniket Kittur,et al.  Apolo: making sense of large network data by combining rich user interaction and machine learning , 2011, CHI.

[73]  Joe Marshall,et al.  HCI with sports , 2013, CHI Extended Abstracts.

[74]  Yisong Yue,et al.  Learning Fine-Grained Spatial Models for Dynamic Sports Play Prediction , 2014, 2014 IEEE International Conference on Data Mining.

[75]  François Brémond,et al.  A framework for surveillance video indexing and retrieval , 2008, 2008 International Workshop on Content-Based Multimedia Indexing.

[76]  Eamonn J. Keogh,et al.  Scaling up dynamic time warping for datamining applications , 2000, KDD '00.

[77]  Changsheng Xu,et al.  A generic virtual content insertion system based on visual attention analysis , 2008, ACM Multimedia.