Automated Discovery of Network Cameras in Heterogeneous Web Pages

Reduction in the cost of Network Cameras along with a rise in connectivity enables entities all around the world to deploy vast arrays of camera networks. Network cameras offer real-time visual data that can be used for studying traffic patterns, emergency response, security, and other applications. Although many sources of Network Camera data are available, collecting the data remains difficult due to variations in programming interface and website structures. Previous solutions rely on manually parsing the target website, taking many hours to complete. We create a general and automated solution for aggregating Network Camera data spread across thousands of uniquely structured web pages. We analyze heterogeneous web page structures and identify common characteristics among 73 sample Network Camera websites (each website has multiple web pages). These characteristics are then used to build an automated camera discovery module that crawls and aggregates Network Camera data. Our system successfully extracts 57,364 Network Cameras from 237,257 unique web pages.

[1]  Mingjing Li,et al.  Web mining for Web image retrieval , 2001, J. Assoc. Inf. Sci. Technol..

[2]  Wenyi Chen,et al.  Worldview and route planning using live public cameras , 2015, Electronic Imaging.

[3]  Dan Brickley,et al.  Google Dataset Search: Building a search engine for datasets in an open Web ecosystem , 2019, WWW.

[4]  Paul A Pisano,et al.  Automated Extraction of Weather Variables from Camera Imagery , 2005 .

[5]  Peter Norvig,et al.  The Unreasonable Effectiveness of Data , 2009, IEEE Intelligent Systems.

[6]  George K. Thiruvathukal,et al.  See the World Through Network Cameras , 2019, Computer.

[7]  Nurulfajar Abd Manap,et al.  Face detection and stereo matching algorithms for smart surveillance system with IP cameras , 2010, 2010 2nd European Workshop on Visual Information Processing (EUVIP).

[8]  Andrea Cavallaro,et al.  Privacy Protection in Online Multimedia , 2017, ACM Multimedia.

[9]  William H. Widen.  Smart Cameras and the Right to Privacy , 2008, Proceedings of the IEEE.

[10]  Yuxiao Hu,et al.  MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition , 2016, ECCV.

[11]  Indranil Gupta,et al.  Measurement and modeling of a large-scale overlay for multimedia streaming , 2007, QSHINE.

[12]  Jie Liu,et al.  Challenges in Building a Portal for Sensors World-Wide , 2006 .

[13]  Bernhard Rinner,et al.  A Smart Camera for Traffic Surveillance , 2003 .

[14]  Alan Davis,et al.  Georgia Department of Transportation , 2016 .

[15]  Bo Zhang,et al.  Semantic Concept Learning through Massive Internet Video Mining , 2008, 2008 IEEE International Conference on Data Mining Workshops.

[16]  George K. Thiruvathukal,et al.  Comparison of Visual Datasets for Machine Learning , 2017, 2017 IEEE International Conference on Information Reuse and Integration (IRI).

[17]  Héctor M. Pérez Meana,et al.  An Early Fire Detection Algorithm Using IP Cameras , 2012, Sensors.

[18]  Ryan Dailey,et al.  Creating the World's Largest Real-Time Camera Network , 2017 .

[19]  Yue Mao,et al.  CitySense: A Data Collection Approach for City Computing Applications , 2018, SenSys.

[20]  Robert Pless,et al.  The global network of outdoor webcams: properties and applications , 2009, GIS.

[21]  Wen-Tsuen Chen,et al.  Design and Implementation of a Real Time Video Surveillance System with Wireless Sensor Networks , 2008, VTC Spring 2008 - IEEE Vehicular Technology Conference.

[22]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.