相关论文
TRECVID 2004 Experiments in Dublin City University
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Abstract:In a move away from previous years' participation in TRECVid ((1) (2) (3)), this year our team focused on the instance search task (INS). We improved our system from last year by applying large vocabulary quantization, soft assignment of visual words, spatial verications and query expansion. Overall, four automatic runs have been submitted for evaluation. In this paper, we present rst our system, then we discuss the results and ndings of our experiments.
相关论文分享
暂无分享，去 创建一个
参考文献
[1] Carl Gutwin,et al. Design for individuals, design for groups: tradeoffs between power and workspace awareness , 1998, CSCW '98.
[2] Darren Leigh,et al. DiamondTouch: a multi-user touch technology , 2001, UIST '01.
[3] Alan F. Smeaton,et al. Using video objects and relevance feedback in video retrieval , 2005, SPIE Optics East.
[4] Alan F. Smeaton,et al. TRECVid 2006 Experiments at Dublin City University , 2012, TRECVID.
[5] Edward A. Fox,et al. Combination of Multiple Searches , 1993, TREC.
[6] Gerald Salton,et al. Automatic text processing , 1988 .
[7] Jong-Hak Lee,et al. Analyses of multiple evidence combination , 1997, SIGIR '97.
[8] Alan F. Smeaton,et al. Físchlár-DiamondTouch: collaborative video searching on a table , 2006, Electronic Imaging.
[9] Alan F. Smeaton,et al. The Físchlár-News-Stories System: Personalised Access to an Archive of TV News , 2004, RIAO.
[10] Alan F. Smeaton,et al. Design, implementation and testing of an interactive video retrieval system , 2003, MIR '03.
[11] Alexander G. Hauptmann,et al. Successful approaches in the TREC video retrieval evaluations , 2004, MULTIMEDIA '04.
[12] Noel E. O'Connor,et al. The acetoolbox: low-level audiovisual feature extraction for retrieval and classification , 2005 .
[13] Jorma Laaksonen,et al. PicSOM Experiments in TRECVID 2018 , 2015, TRECVID.
[14] Alan F. Smeaton,et al. Video retrieval using dialogue, keyframe similarity and video objects , 2005, IEEE International Conference on Image Processing 2005.
[15] Alan F. Smeaton,et al. Fischlár @ TRECVID2003: system description , 2004, MULTIMEDIA '04.
[16] Noel E. O'Connor,et al. A multiscale representation method for nonrigid shapes with a single closed contour , 2004, IEEE Transactions on Circuits and Systems for Video Technology.
[17] Alan F. Smeaton,et al. Designing the User Interface for the Físchlár Digital Video Library , 2006, J. Digit. Inf..
[18] Cordelia Schmid,et al. A Comparison of Affine Region Detectors , 2005, International Journal of Computer Vision.
[19] Jean-Luc Gauvain,et al. The LIMSI Broadcast News transcription system , 2002, Speech Commun..
[20] Stephen E. Robertson,et al. Some simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval , 1994, SIGIR '94.
[21] Alan F. Smeaton,et al. A Comparison of Score, Rank and Probability-Based Fusion Methods for Video Shot Retrieval , 2005, CIVR.
[22] Djoerd Hiemstra,et al. Combining Information Sources for Video Retrieval , 2003, TRECVID.
[23] Takeo Kanade,et al. A statistical approach to 3d object detection applied to faces and cars , 2000 .
[24] Alan F. Smeaton,et al. Collaborative searching for video using the Fischlar system and a DiamondTouch table , 2006, First IEEE International Workshop on Horizontal Interactive Human-Computer Systems (TABLETOP '06).
[25] Alan F. Smeaton,et al. Físréal: A Low Cost Terabyte Search Engine , 2005, ECIR.
[26] Ivan Laptev,et al. On Space-Time Interest Points , 2005, International Journal of Computer Vision.
[27] Andrew Zisserman,et al. Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.
[28] B. S. Manjunath,et al. Color and texture descriptors , 2001, IEEE Trans. Circuits Syst. Video Technol..
[29] Alan F. Smeaton,et al. Experiments in Terabyte Searching, Genomic Retrieval and Novelty Detection for TREC 2004 , 2004, TREC.
[30] Alan F. Smeaton,et al. Dublin City University Video Track Experiments for TREC 2002 , 2001, TREC.
[31] Jorma Laaksonen,et al. Semantic Concept Detection from News Videos with Self-Organizing Maps , 2006, AIAI.
[32] John Adcock,et al. FXPAL Experiments for TRECVID 2004 , 2004, TRECVID.
[33] Paul Over,et al. Evaluation campaigns and TRECVid , 2006, MIR '06.
CONTENT BASED VIDEO RETRIEVAL BASED ON HDWT AND SPARSE REPRESENTATION
2016
摘要
Video retrieval has recently attracted a lot of research attention due to the exponential growth of video datasets and the internet. Content based video retrieval (CBVR) systems are very useful for a wide range of applications with several type of data such as visual, audio and metadata. In this paper, we are only using the visual information from the video. Shot boundary detection, key frame extraction, and video retrieval are three important parts of CBVR systems. In this paper, we have modified and proposed new methods for the three important parts of our CBVR system. Meanwhile, the local and global color, texture, and motion features of the video are extracted as features of key frames. To evaluate the applicability of the proposed technique against various methods, the P(1) metric and the CC_WEB_VIDEO dataset are used. The experimental results show that the proposed method provides better performance and less processing time compared to the other methods.
A Survey on Visual Content-Based Video Indexing and Retrieval
IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews)
2011
摘要
Video indexing and retrieval have a wide spectrum of promising applications, motivating the interest of researchers worldwide. This paper offers a tutorial and an overview of the landscape of general strategies in visual content-based video indexing and retrieval, focusing on methods for video structure analysis, including shot boundary detection, key frame extraction and scene segmentation, extraction of features including static key frame features, object features and motion features, video data mining, video annotation, video retrieval including query interfaces, similarity measure and relevance feedback, and video browsing. Finally, we analyze future research directions.
Combining Relevance Information in a Synchronous Collaborative Information Retrieval Environment
2008
摘要
Traditionally information retrieval (IR) research has focussed on a single user interaction modality, where a user searches to satisfy an information need. Recent
advances in both web technologies, such as the sociable web of Web 2.0, and computer hardware, such as tabletop interface devices, have enabled multiple users to collaborate on many computer-related tasks. Due to these advances there is an increasing need to support
two or more users searching together at the same time, in order to satisfy a shared information need, which we refer to as Synchronous Collaborative Information Retrieval.
Synchronous Collaborative Information Retrieval (SCIR) represents a significant paradigmatic shift from traditional IR systems. In order to support an effective SCIR search, new techniques are required to coordinate users' activities. In this chapter we explore the effectiveness of a sharing of knowledge policy on a collaborating group. Sharing of knowledge refers to the process of passing relevance information across users,
if one user finds items of relevance to the search task then the group should benefit in the form of improved ranked lists returned to each searcher.
In order to evaluate the proposed techniques we simulate two users searching together through an incremental feedback system. The simulation assumes that users decide on an initial query with which to begin the collaborative search and proceed through the search by providing relevance judgments to the system and receiving a new ranked list. In order to populate these simulations we extract data from the interaction logs of various
experimental IR systems from previous Text REtrieval Conference (TREC) workshops.
Design, deployment and assessment of a movie archive system in a film studies context
2009
摘要
This thesis describe our work in developing a movie archive system for students of Film Studies in Dublin City University. In particular, our system uses several recent multimedia technologies to automatically process digital video content but at the same time we use the usability engineering process to relate to the real tasks of real users in their real environments.
We investigate how real users take advantage of technologies in a movie browsing system. By designing, building, deploying and assessing the usage of a technology in a user-focused way, the overall impact of a movie browsing system can be determined holistically.
The application domain we work in is film studies where students need to study movie contents and analyse movie sequences. Our work began by identification of user
needs through observations, focus groups and usability testing, followed by sketching and prototyping a web-based system. We then deployed the system to film study classes over a semester, monitored usage and gathered quantitative as well as qualitative data. Focused experiments were carried out to assess students’ performance and satisfaction levels. Our findings show expected patterns of usage of a real-user setting outside the lab, but at the
same time highlighted issues that need to be further investigated. In general, students found most of the provided features were beneficial for their studies. Findings from the experiment shows better performance in the essay assessments and higher satisfaction levels.
An interesting finding shows students are more engaged with the newly-introduced software application and take longer time to complete the same task than without the advanced
features of the application. This phenomenon was rationalized from established learning theory from the psychology domain. In a technologically-oriented multimedia field today, we attempted to bring in a user-centred approach of end-user interactions throughout a 3-year development process, and we identified benefits and challenges in trying to align the technical perspectives of novel multimedia features to real-world setting.
Detection of Multiple Instances of Video Objects
2011 Seventh International Conference on Signal Image Technology & Internet-Based Systems
2011
摘要
This paper tackles the issue of retrieving different instances of an object of interest within a given video document or in a video database. The principle consists of considering a semi-global image representation based on an over-segmentation of image frames. An aggregation mechanism is then applied in order to group a set of sub-regions into an object similar to the query, under a global similarity criterion. Two different types of approaches are proposed. The first one involves a greedy, dynamic region construction method. The second is based on simulated annealing, and aims at determining a global optimum of the similarity function. Experimental results show promising performances, with FT and BE detection rates of up to 66% and 86%, respectively.
Topological Spatial Verification for Instance Search
IEEE Transactions on Multimedia
2015
摘要
This paper proposes an elastic spatial verification method for Instance Search, particularly for dealing with non-planar and non-rigid queries exhibiting complex spatial transformations. Different from existing models that map keypoints between images based on a linear transformation (e.g., affine, homography), our model exploits the topological arrangement of keypoints to address the non-linear spatial transformations that are extremely common in real life situations. In particular, we propose a novel technique to elastically verify the topological spatial consistency with the triangulated graph through a “sketch-and-match” scheme. The spatial topology configuration, emphasizing relative positioning rather than absolute coordinates, is first sketched by a triangulated graph, whose edges essentially capture the topological layout of the corresponding keypoints. Next, the spatial consistency is efficiently estimated as the number of common edges between the triangulated graphs. Compared to the existing methods, our technique is much more effective in modeling the complex spatial transformations of non-planar and non-rigid instances, while being compatible to instances with simple linear transformations. Moreover, our method is by nature more robust in spatial verification by considering the locations, rather than the local geometry of keypoints, which are sensitive to motions and viewpoint changes. We evaluate our method extensively on three years of TRECVID datasets, as well as our own dataset MQA, showing large improvement over other methods for the task of Instance Search.
Using Segmented Objects in Ostensive Video Shot Retrieval
Adaptive Multimedia Retrieval
2005
摘要
This paper presents a system for video shot retrieval in which shots are retrieved based on matching video objects using a combination of colour, shape and texture. Rather than matching on individual objects, our system supports sets of query objects which in total reflect the user's object-based information need. Our work also adapts to a shifting user information need by initiating the partitioning of a user's search into two or more distinct search threads, which can be followed by the user in sequence. This is an automatic process which maps neatly to the ostensive model for information retrieval in that it allows a user to place a virtual checkpoint on their search, explore one thread or aspect of their information need and then return to that checkpoint to then explore an alternative thread. Our system is fully functional and operational and in this paper we illustrate several design decisions we have made in building it.
Locating Information in Video by Browsing and Searching
2006
摘要
This chapter describes tools for browsing and searching through video to enable users to quickly locate video passages of interest. Digital video databases containing large numbers of video programs ranging from several minutes to several hours in length are becoming increasingly common. In many cases, it is not sufficient to search for relevant videos, but rather to identify relevant clips, typically less than one minute in length, within the videos. We offer two approaches for finding information in videos. The first approach provides an automatically generated interactive multi-level summary in the form of a hypervideo. When viewing a sequence of short video clips, the user can obtain more detail on the clip being watched. For situations where browsing is impractical, we present a video search system with a flexible user interface that incorporates dynamic visualizations of the underlying multimedia objects. The system employs automatic story segmentation, and displays the results of text and image-based queries in ranked sets of story summaries. Both approaches help users to quickly drill down to potentially relevant video clips and to determine the relevance by visually inspecting the material.
User-interface to a CCTV video search system
2005
摘要
The proliferation of CCTV surveillance systems creates a problem of how to effectively navigate and search the resulting video archive, in a variety of security scenarios. We are concerned here with a situation where a searcher must locate all occurrences of a given person or object within a specified timeframe and with constraints on which camera(s) footage is valid to search. Conventional approaches based on browsing time/camera based combinations are inadequate. We advocate using automatically detected video objects as a basis for search, linking and browsing. In this paper we present a system under development based on users interacting with detected video objects. We outline the suite of technologies needed to achieve such a system and for each we describe where we are in terms of realizing those technologies. We also present a system interface to this system, designed with user needs and user tasks in mind.
Situated topic complexity in interactive video retrieval
J. Assoc. Inf. Sci. Technol.
2011
摘要
The overarching goal of this study is to place additional emphasis on the ability to observe, extend, and apply findings across different experimental studies and/or digital projects by providing further insight into some of the broader processes within interactive (user-centered) video retrieval. The variable at the center of this particular investigation included a user-centered definition of topic complexity, or “situated topic complexity,” where its influence on other factors within the interactive video process, namely users' actions, satisfaction, performance, and judgments on other topical qualities, was analyzed. Findings revealed that as users' impressions of complexity increased during experimental search topics, search times and the number of user actions also increased, but even more compelling was the fact that users started to convincingly shift away from keyword (transcript) searching, as predominantly used for easy to moderate topics, to categorical and linear browse strategies at a clear point within the interactive process. Different users' assessments, regarding their search experiences, and topic performances were also varied and shown to be significantly correlated with the situated complexity of a given topic. Such findings are relevant to the design of digital libraries in that researchers should continue improving ways to deliver video through keyword search functions, but also further recognize the multidimensionality of video by offering other paths or interface options that provide support for highly complex needs and also for exploring the collection's boundaries on easier topics. © 2011 Wiley Periodicals, Inc.
A review of text and image retrieval approaches for broadcast news video
Information Retrieval
2007
摘要
The effectiveness of a video retrieval system largely depends on the choice of underlying text and image retrieval components. The unique properties of video collections (e.g., multiple sources, noisy features and temporal relations) suggest we examine the performance of these retrieval methods in such a multimodal environment, and identify the relative importance of the underlying retrieval components. In this paper, we review a variety of text/image retrieval approaches as well as their individual components in the context of broadcast news video. Numerous components of text/image retrieval have been discussed in detail, including retrieval models, text sources, temporal expansion methods, query expansion methods, image features, and similarity measures. For each component, we conduct a series of retrieval experiments on TRECVID video collections to identify their advantages and disadvantages. To provide a more complete coverage of video retrieval, we briefly discuss an emerging approach called concept-based video retrieval, and review strategies for combining multiple retrieval outputs.
Modeling Human Judgment of Digital Imagery for Multimedia Retrieval
IEEE Transactions on Multimedia
2007
摘要
The application of machine learning techniques to image and video search has been shown to boost the performance of multimedia retrieval systems, and promises to lead to more generalized semantic search approaches. In particular, the availability of large training collections allows model-driven search using a substantial number of semantic concepts. The training collections are obtained in a manual annotation process where human raters review images and assign predefined semantic concept labels. Besides being prone to human error, manual image annotation is biased by the view of the individual annotator because visual information almost always leaves room for ambiguity. Ideally, several independent judgments are obtained per image, and the inter-rater agreement is assessed. While disagreement between ratings bears valuable information on the annotation quality, it complicates the task of clearly classifying rated images based on multiple judgments. In the absence of a gold standard, evaluating multiple judgments and resolving disagreement between raters is not trivial. In this paper, we present an approach using latent structure analysis to solve this problem. We apply latent class modeling to the annotation data collected during the TRECVID 2005 Annotation Forum, and demonstrate how to use this statistic to clearly classify each image on the basis of varying numbers of ratings. We use latent class modeling to quantify the annotation quality and discuss the results in comparison with the well-known Kappa inter-rater agreement measure.
Simulated Testing of an Adaptive Multimedia Information Retrieval System
2007 International Workshop on Content-Based Multimedia Indexing
2007
摘要
The Semantic Gap is considered to be a bottleneck in image and video retrieval. One way to increase the communication between user and system is to take advantage of the user's action with a system, e.g. to infer the relevance or otherwise of a video shot viewed by the user. In this paper we introduce a novel video retrieval system and propose a model of implicit information for interpreting the user's actions with the interface. The assumptions on which this model was created are then analysed in an experiment using simulated users based on relevance judgements to compare results of explicit and implicit retrieval cycles. Our model seems to enhance retrieval results. Results are presented and discussed in the final section.
User evaluation of Físchlár-News
ACM Trans. Inf. Syst.
2006
摘要
Technological developments in content-based analysis of digital video information are undergoing much progress, with ideas for fully automatic systems now being proposed and demonstrated. Yet because we do not yet have robust operational video retrieval systems that can be deployed and used, the usual HCI practise of conducting a usage study and an informed iterative system design is thus not possible. Físchlár-News is one of the first automatic, content-based broadcast news analysis and archival systems that process broadcast news video so that users can search, browse, and play it in an easy-to-use manner with a conventional web browser. The system incorporates a number of state-of-the-art research components, some of which are not yet considered mature technology, yet it has been built to be robust enough to be deployed to users who are interested in access to daily news throughout a university campus. In this article we report and discuss a user-evaluation study conducted with 16 users, each of whom utilized the system freely for a one month period. Results from a detailed qualitative analysis are presented, looking at collected questionnaires, incident diaries, and interaction-log data. The findings suggest that our users employed the system in conjunction with their other news update methods, such as watching TV news at home and browsing online news websites at their workplace, their major concerns being up-to-dateness and coverage of the news content. They tried to accommodate the system to fit their established web browsing habits, and they found local news content and the ability to play self-contained news stories on their desktop as major values of the system. Our study also resulted in a detailed wishlist of new features which will help in the further development of both our and others' systems.
Division of labour and sharing of knowledge for synchronous collaborative information retrieval
Inf. Process. Manag.
2010
摘要
Synchronous collaborative information retrieval (SCIR) is concerned with supporting two or more users who search together at the same time in order to satisfy a shared information need. SCIR systems represent a paradigmatic shift in the way we view information retrieval, moving from an individual to a group process and as such the development of novel IR techniques is needed to support this. In this article we present what we believe are two key concepts for the development of effective SCIR namely division of labour (DoL) and sharing of knowledge (SoK). Together these concepts enable coordinated SCIR such that redundancy across group members is reduced whilst enabling each group member to benefit from the discoveries of their collaborators. In this article we outline techniques from state-of-the-art SCIR systems which support these two concepts, primarily through the provision of awareness widgets. We then outline some of our own work into system-mediated techniques for division of labour and sharing of knowledge in SCIR. Finally we conclude with a discussion on some possible future trends for these two coordination techniques.
Circular Reranking for Visual Search
IEEE Transactions on Image Processing
2013
摘要
Search reranking is regarded as a common way to boost retrieval precision. The problem nevertheless is not trivial especially when there are multiple features or modalities to be considered for search, which often happens in image and video retrieval. This paper proposes a new reranking algorithm, named circular reranking, that reinforces the mutual exchange of information across multiple modalities for improving search performance, following the philosophy that strong performing modality could learn from weaker ones, while weak modality does benefit from interacting with stronger ones. Technically, circular reranking conducts multiple runs of random walks through exchanging the ranking scores among different features in a cyclic manner. Unlike the existing techniques, the reranking procedure encourages interaction among modalities to seek a consensus that are useful for reranking. In this paper, we study several properties of circular reranking, including how and which order of information propagation should be configured to fully exploit the potential of modalities for reranking. Encouraging results are reported for both image and video retrieval on Microsoft Research Asia Multimedia image dataset and TREC Video Retrieval Evaluation 2007-2008 datasets, respectively.
Hierarchical Hidden Markov Model for Rushes Structuring and Indexing
CIVR
2006
摘要
Rushes footage are considered as cheap gold mine with the potential for reuse in broadcasting and filmmaking industries. However, it is difficult to mine the “gold” from the rushes since usually only minimum metadata is available. This paper focuses on the structuring and indexing of the rushes to facilitate mining and retrieval of “gold”. We present a new approach for rushes structuring and indexing based on motion feature. We model the problem by a two-level Hierarchical Hidden Markov Model (HHMM). The HHMM, on one hand, represents the semantic concepts in its higher level to provide simultaneous structuring and indexing, on the other hand, models the motion feature distributions in its lower level to support the encoding of the semantic concepts. The encouraging experimental results on TRECVID′05 BBC rushes demonstrate the effectiveness of our approach.
Video Retrieval Using High Level Features: Exploiting Query Matching and Confidence-Based Weighting
CIVR
2006
摘要
Recent research in video retrieval has focused on automated, high-level feature indexing on shots or frames. One important application of such indexing is to support precise video retrieval. We report on extensions of this semantic indexing on news video retrieval. First, we utilize extensive query analysis to relate various high-level features and query terms by matching the textual description and context in a time-dependent manner. Second, we introduce a framework to effectively fuse the relation weights with the detectors' confidence scores. This results in individual high level features that are weighted on a per-query basis. Tests on the TRECVID 2005 dataset show that the above two enhancements yield significant improvement in performance over a corresponding state-of-the-art video retrieval baseline.
Content Coverage and Redundancy Removal in Video Summarization
2017
摘要
Over the past decade, research in the field of Content-Based Video Retrieval Systems (CBVRS) has attracted much attention as it encompasses processing of all the other media types i.e. text, image and audio. Video summarization is one of the most important applications as it potentially enables efficient and faster browsing of large video collections. A concise version of the video is often required due to constraints in viewing time, storage, communication bandwidth as well as power. Thus, the task of video summarization is to effectively extract the most important portions of the video, without sacrificing the semantic information in it. The results of video summarization can be used in many CBVRS applications like semantic indexing, video surveillance copied video detection etc. However, the quality of the summarization task depends on two basic aspects: content coverage and redundancy removal. These two aspects are both important and contradictory to each other. This chapter aims to provide an insight into the state-of-the-art approaches used for this booming field of research.
Interactive Searching and Browsing of Video Archives: Using Text and Using Image Matching
2006
摘要
Over the last number of decades much research work has been done in the general area of video and audio analysis. Initially the applications driving this included capturing video in digital form and then being able to store, transmit
and render it, which involved a large effort to develop compression and encoding standards. The technology needed to do all this is now easily available and cheap, with applications of digital video processing now commonplace,
ranging from CCTV (Closed Circuit TV) for security, to home capture of broadcast TV on home DVRs for personal viewing.
One consequence of the development in technology for creating, storing and distributing digital video is that there has been a huge increase in the volume of digital video, and this in turn has created a need for techniques to allow effective management of this video, and by that we mean content management. In the BBC, for example, the archives department receives approximately 500,000 queries per year and has over 350,000 hours of content in its library. Having huge archives of video information is hardly any benefit if we have no effective means of being able to locate video clips which are of relevance to whatever our information needs may be. In this chapter we report our work on developing two specific retrieval and browsing tools for digital video information. Both of these are based on an analysis of the captured video for the purpose of automatically structuring into shots or higher level semantic units like TV news stories. Some also include analysis of the video for the automatic detection of features such as the presence or absence of faces. Both include some elements of searching, where a user specifies a query or information need, and browsing, where a user is allowed to browse through sets of retrieved video shots. We support the presentation of these tools with illustrations of actual video retrieval systems developed and working on hundreds of hours of video content.