Logic-based interpretation of geometrically observable changes occurring in dynamic scenes

The work presented here is about employing a theory of updates to study geometrically observable changes that occur in spatial information about image sequences of a dynamic scene. The logical framework consists of a formalism for specifying the geometrical content of a scene, as well as the changes that occur in this geometry, and an algorithm for constructing a description for such changes from logical deductions. In this approach, a database state represents the available sensor data at a particular time instant. Transitions in sensor data are modeled by changes in the database and interpreted based on axioms encoding commonsense spatial reasoning. The main contribution of this work is that it provides the theoretical foundations for symbolically interpreting long sequences of sensor data transitions. For testing the framework and its implementation, the problem of interpreting rotational movements of objects in a sequence of images was used. Our experiments show that the system correctly interprets rotational movements for objects of different colors and provides satisfactory results for interpreting such movements from perceptually indistinguishable objects.

[1]  John K. Tsotsos,et al.  A framework for visual motion understanding , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Stuart Sutherland Lines of sight , 1987, Nature.

[3]  Enforcing Global Spatio-Temporal Consistency to Enhance Reliability of Moving Object Tracking and Classification , 2005, Künstliche Intell..

[4]  John K. Tsotsos,et al.  Knowledge organization and its role in representation and interpretation for time-varying data: the ALVEN system , 1987 .

[5]  John F. Santore,et al.  Identifying Perceptually Indistinguishable Objects: Is That the Same One You Saw Before? , 2002 .

[6]  M. Negnevitsky,et al.  Email communications analysis: how to use computational intelligence methods and tools? , 2005, CIHSPS 2005. Proceedings of the 2005 IEEE International Conference on Computational Intelligence for Homeland Security and Personal Safety, 2005..

[7]  Gordon I. McCalla,et al.  The knowledge frontier: essays in the representation of knowledge , 1987 .

[8]  Gérard Ligozat,et al.  Reasoning about Cardinal Directions , 1998, J. Vis. Lang. Comput..

[9]  Jeffrey Mark Siskind,et al.  Visual Event Classification via Force Dynamics , 2000, AAAI/IAAI.

[10]  R. Weale Vision. A Computational Investigation Into the Human Representation and Processing of Visual Information. David Marr , 1983 .

[11]  Patrick Bouthemy,et al.  Motion segmentation and qualitative dynamic scene analysis from an image sequence , 1993, International Journal of Computer Vision.

[12]  Jeffrey Mark Siskind,et al.  Grounding language in perception , 1993, Other Conferences.

[13]  Anthony G. Cohn,et al.  A Spatial Logic based on Regions and Connection , 1992, KR.

[14]  Lawrence Birnbaum,et al.  Sensible Scenes: Visual Understanding of Complex Structures through Causal Analysis , 1993, AAAI.

[15]  Nicholas Mark Gotts,et al.  How Far Can We 'C'? Defining a 'Doughnut' Using Connection Alone , 1994, KR.

[16]  Bernd Neumann,et al.  On the logics of image interpretation: model-construction in a formal knowledge-representation framework , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[17]  Raymond Reiter,et al.  Logical Foundations for Cognitive Agents: Contributions in Honor of Ray Reiter , 2001 .

[18]  John G. Gibbons Knowledge in Action , 2001 .

[19]  Stephan Winter,et al.  Spatial Information Theory, 8th International Conference, COSIT 2007, Melbourne, Australia, September 19-23, 2007, Proceedings , 2007, COSIT.

[20]  Gerd Herzog,et al.  VIsual TRAnslator: Linking perceptions and natural language descriptions , 1994, Artificial Intelligence Review.

[21]  Anthony G. Cohn,et al.  Qualitative Simulation Based on a Logical Formalism of Space and Time , 1992, AAAI.

[22]  Eliseo Clementini,et al.  Qualitative Distances , 1995, COSIT.

[23]  Christian Freksa,et al.  Qualitative spatial reasoning , 1990, Forschungsberichte, TU Munich.

[24]  Pedro Cabalar,et al.  Holes, Knots and Shapes: A Spatial Ontology of a Puzzle , 2007, AAAI Spring Symposium: Logical Formalizations of Commonsense Reasoning.

[25]  A. Galton Qualitative Spatial Change , 2001 .

[26]  Anthony G. Cohn,et al.  Computing Transivity Tables: A Challenge For Automated Theorem Provers , 1992, CADE.

[27]  Travé-Massuyès Conceptual Neighborhood and its role in temporal and spatial reasoning , 1991 .

[28]  Anthony G. Cohn,et al.  Combining Multiple Answers for Learning Mathematical Structures from Visual Observation , 2004, ECAI.

[29]  Antony Galton,et al.  Towards a Qualitative Theory of Movement , 1995, COSIT.

[30]  Frank Wolter,et al.  Spatio-temporal representation and reasoning based on RCC-8 , 2000, International Conference on Principles of Knowledge Representation and Reasoning.

[31]  Anthony G. Cohn,et al.  Protocols from perceptual observations , 2005, Artif. Intell..

[32]  Michael Kifer,et al.  Transaction Logic Programming , 1993, ICLP.

[33]  Anthony G. Cohn,et al.  Abducing Qualitative Spatio-Temporal Histories from Partial Observations , 2002, KR.

[34]  Mark Witkowski,et al.  From Images to Bodies: Modelling and Exploiting Spatial Occlusion and Motion Parallax , 2001, IJCAI.

[35]  Hans-Hellmut Nagel,et al.  From image sequences towards conceptual descriptions , 1988, Image Vis. Comput..

[36]  Patrick J. Hayes,et al.  The second naive physics manifesto , 1995 .

[37]  Anthony G. Cohn,et al.  Representing and Reasoning with Qualitative Spatial Relations About Regions , 1997 .

[38]  Martin Erwig,et al.  Toward Spatio-Temporal Patterns , 2004 .

[39]  Hans-Hellmut Nagel,et al.  Deriving Textual Descriptions of Road Traffic Queues from Video Sequences , 2002, ECAI.

[40]  Dana H. Ballard,et al.  Computer Vision , 1982 .

[41]  Bernd Neumann,et al.  On scene interpretation with description logics , 2006, Image Vis. Comput..

[42]  Murray Shanahan,et al.  What Sort of Computation Mediates Best between Perception and Action , 1999 .

[43]  Peter Gärdenfors,et al.  Conceptual spaces - the geometry of thought , 2000 .

[44]  Chung-Lin Huang Contour generation and shape restoration of the straight homogeneous generalized cylinder , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[45]  Hans-Hellmut Nagel,et al.  Image sequence evaluation: 30 years and still going strong , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[46]  Tonya Lewis,et al.  Knowledge in Action , 1977 .

[47]  Andrew U. Frank,et al.  Spatio-Temporal Databases , 2003, Lecture Notes in Computer Science.

[48]  Patrick Bouthemy,et al.  Computation and analysis of image motion: A synopsis of current problems and methods , 1996, International Journal of Computer Vision.

[49]  Raymond Reiter,et al.  A Logical Framework for Depiction and Image Interpretation , 1989, Artif. Intell..

[50]  Hans W. Guesgen,et al.  Reasoning About Distance Based on Fuzzy Sets , 2002, Applied Intelligence.

[51]  Takashi Matsuyama,et al.  SIGMA: A Knowledge-Based Aerial Image Understanding System , 1990 .

[52]  Michael Kifer,et al.  Transaction Logic Programming (or, A Logic of Procedural and Declarative Knowledge) , 1995 .

[53]  Anthony G. Cohn,et al.  Qualitative Spatial Representation and Reasoning , 2008, Handbook of Knowledge Representation.

[54]  Salvatore Gaglio,et al.  Understanding dynamic scenes , 2000, Artif. Intell..

[55]  D. Randell,et al.  Qualitative Simulation Based on a Logic of Space and Time 3 , 1992 .

[56]  David Toman,et al.  Logics for Databases and Information Systems , 1998 .

[57]  Hans W. Guesgen,et al.  Spatial and Temporal Reasoning , 2003, AI Commun..

[58]  Matthew Brand,et al.  Understanding manipulation in video , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[59]  Enrico Pontelli,et al.  Proceedings of the 24th International Conference on Logic Programming , 2008 .

[60]  Matthew Brand,et al.  Physics-Based Visual Understanding , 1997, Comput. Vis. Image Underst..

[61]  John Ross,et al.  Lines of sight , 1995 .

[62]  David Harel,et al.  Process Logic: Expressiveness, Decidability, Completeness , 1980, FOCS.

[63]  Randy Goebel,et al.  Theorist: A Logical Reasoning System for Defaults and Diagnosis , 1987 .

[64]  Hans-Hellmut Nagel,et al.  Characterization of Occlusion Situations Occurring in Real-World Traffic Scenes , 1996 .

[65]  Hans-Hellmut Nagel,et al.  Analysing Sequences of TV-Frames , 1977, IJCAI.

[66]  Wolfgang Wahlster,et al.  From Visual Input to Verbal Output in the Visual Translator , 2003 .

[67]  Paulo E. Santos,et al.  Reasoning about Depth and Motion from an Observer's Viewpoint , 2007, Spatial Cogn. Comput..

[68]  Mark Witkowski,et al.  Building Large Composition Tables via Axiomatic Theories , 2002, KR.

[69]  Bernhard Nebel,et al.  On the Complexity of Qualitative Spatial Reasoning: A Maximal Tractable Fragment of the Region Connection Calculus , 1999, Artif. Intell..

[70]  Murray Shanahan,et al.  Hypothesising Object Relations from Image Transitions , 2002, ECAI.

[71]  Benjamin Kuipers,et al.  Qualitative reasoning: Modeling and simulation with incomplete knowledge , 1994, Autom..

[72]  Markus Schneider,et al.  Spatio-Temporal Predicates , 2002, IEEE Trans. Knowl. Data Eng..

[73]  Christian Freksa,et al.  Using Orientation Information for Qualitative Spatial Reasoning , 1992, Spatio-Temporal Reasoning.

[74]  Hilary Buxton,et al.  Learning and understanding dynamic scene activity: a review , 2003, Image Vis. Comput..

[75]  Allan D. Jepson,et al.  The Computational Perception of Scene Dynamics , 1997, Comput. Vis. Image Underst..

[76]  P. Burrough,et al.  Geographic Objects with Indeterminate Boundaries , 1996 .

[77]  Oscal T.-C. Chen,et al.  Robust Image Segmentation Using Modified Edge-Following Scheme with Automatically-Determined Thresholds , 2006, First International Conference on Innovative Computing, Information and Control - Volume I (ICICIC'06).

[78]  Anthony G. Cohn,et al.  Constructing qualitative event models automatically from video input , 2000, Image Vis. Comput..

[79]  Pedro Cabalar,et al.  Strings and Holes: An Exercise on Spatial Reasoning , 2006, IBERAMIA-SBIA.

[80]  Gunter Saake,et al.  Logics for databases and information systems , 1998 .

[81]  Murray Shanahan,et al.  Robotics and the Common Sense Informatic Situation , 1996, ECAI.

[82]  Reinhard Moratz,et al.  Qualitative Spatial Reasoning about Line Segments , 2000, ECAI.

[83]  Allen Newell,et al.  The Knowledge Level , 1989, Artif. Intell..

[84]  Yogesh Rathi,et al.  Shape-Based Approach to Robust Image Segmentation using Kernel PCA , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[85]  Michael Kifer,et al.  A Logic for Programming Database Transactions , 1998, Logics for Databases and Information Systems.

[86]  Anthony G. Cohn,et al.  Qualitative Spatial Representation and Reasoning: An Overview , 2001, Fundam. Informaticae.

[87]  Murray Shanahan,et al.  A Logic-based Algorithm for Image Sequence Interpretation and Anchoring , 2003, IJCAI.

[88]  Johan van Benthem,et al.  Space, time, and computation: Trends and problems , 2004, Applied Intelligence.

[89]  Philippe Muller,et al.  Topological Spatio–Temporal Reasoning and Representation , 2002, Comput. Intell..