With the complex building composition and imaging condition, urban areas show versatile characteristics in remote sensing images. In the literature of land-cover analysis, many algorithms utilize the features with structural information to characterize urban areas. Typically, these are more successful on some types of imagery than others, since they usually use only one kind or a few kinds of structural information. On the other hand, since levels of development in neighboring areas are not statistically independent, the multiple features (encoding the multilevel structural information) of each site in urban area depend on that of neighboring sites. In this paper, a new-come discriminative model, i.e., conditional random field (CRF), is introduced to learn the dependencies and fuse the multilevel structural information to obtain the essential detection. To meet the higher needs of some users, we introduce a two-component-based Markov random field model and show how to integrate it tightly with CRF model to refine the results from essential detection. Experiments on a wide range of images show that our algorithms are competitive with recent results in urban area detection

While learning the maximum likelihood value of parameters of an undirected graphical model is hard, modelling the posterior distribution over parameters given data is harder. Yet, undirected models are ubiquitous in computer vision and text modelling (e.g. conditional random fields). But where Bayesian approaches for directed models have been very successful, a proper Bayesian treatment of undirected models in still in its infant stages. We propose a new method for approximating the posterior of the parameters given data based on the Laplace approximation. This approximation requires the computation of the covariance matrix over features which we compute using the linear response approximation based in turn on loopy belief propagation. We develop the theory for conditional and 'unconditional' random fields with or without hidden variables. In the conditional setting we introduce a new variant of bagging suitable for structured domains. Here we run the loopy max-product algorithm on a 'super-graph' composed of graphs for individual models sampled from the posterior and connected by constraints. Experiments on real world data validate the proposed methods.

We show that solving the LP relaxation of the min-sum labeling problem (also known as MAP inference problem in graphical models, discrete energy minimization, or valued constraint satisfaction) is not easier than solving any linear program. Precisely, every polytope is linear-time representable by a local marginal polytope and every LP can be reduced in linear time to a linear optimization (allowing infinite costs) over a local marginal polytope. The reduction can be done (though with a higher time complexity) even if the local marginal polytope is restricted to have a planar structure.

We proposed using Conditional Random Fields with adaptive data reduction for the classification of 3D point clouds acquired from a Riegl Terrestrial laser scanner. The training and inference of the acquired large outdoor urban data can be time consuming. We approach the problem by computing an adaptive support region for each data point using 3D scale theory. For training and inference of the discriminative Conditional Random Fields, smaller set of data samples that contains relevant information within the support region is selected instead of using all point cloud data. We tested the algorithm on synthetically generated data and urban point clouds data acquired from the laser scanner. The computed support region is also used in feature extraction for urban point clouds data. The results showed improvement in the training and inference rate while maintaining comparable classification accuracy.

We propose in this paper a new online handwritten flowchart database and perform some first experiments to have a baseline benchmark on this dataset. The collected database consists of 419 flowcharts labeled at the stroke and symbol levels. In addition, an isolated database of graphical and text symbols was extracted from these collected flowcharts. Then, we tackle the problem of online handwritten flowchart recognition from two different points of view. Firstly, we consider that flowcharts are correctly segmented, and we propose different classifiers to perform two tasks, text/non-text separation and graphical symbol recognition. Tested with the extracted isolated test database, we achieve up to 90% and 98% in text/non-text separation and up to 93.5% in graphical symbols recognition. Secondly, we propose a global approach to perform flowchart segmentation and recognition. For this latter, we adopt a global learning schema and a recognition architecture that considers a simultaneous segmentation and recognition. Global architecture is trained and tested directly with flowcharts. Results show the interest of such global approach, but regarding the complexity of flowchart segmentation problem, there is still lot of space to improve the global learning and recognition methods.

We propose and evaluate the use of a mirror for improving the scanning process when using laser-stripe acquisition devices. We propose to exploit mirrors for two different purposes: automatizing the 3D acquisition process and allowing the scanning of hard-to-reach parts. The combined use of a flat mirror and a rotary table allows to scan, in a completely automatic and unattended manner, small complex objects with an high level of completeness. Moreover we show how the use of a small hand-held mirror can to be effective for scanning parts that are difficult to be reached or scanned because the physical dimension of the scanner device.

We present our recent model of a diagram recognition engine. It extends our previous work which approaches the structural recognition as an optimization problem of choosing the best subset of symbol candidates. The main improvement is the integration of our own text separator into the pipeline to deal with text blocks occurring in diagrams. Second improvement is splitting the symbol candidates detection into two stages: uniform symbols detection and arrows detection. Text recognition is left for post processing when the diagram structure is already known. Training and testing of the engine was done on a freely available benchmark database of flowcharts. We correctly segmented and recognized 93.0% of the symbols having 55.1% of the diagrams recognized without any error. Considering correct stroke labeling, we achieved the precision of 95.7%. This result is superior to the state-of-the-art method with the precision of 92.4%. Additionally, we demonstrate the generality of the proposed method by adapting the system to finite automata domain and evaluating it on own database of such diagrams.

We introduce a new, online, stroke-based recognition system for hand-drawn diagrams which belong to a group of documents with an explicit structure obvious to humans but only loosely defined from the machine point of view. We propose a model for recognition by selection of symbol candidates, based on evaluation of relations between candidates using a set of predicates. It is suitable for simpler structures where the relations are explicitly given by symbols, arrows in the case of diagrams. Knowledge of a specific diagram domain is used—the two domains are flowcharts and finite automata. Although the individual pipeline steps are tailored for these, the system can readily be adapted for other domains. Our entire diagram recognition pipeline is outlined. Its core parts are text/non-text separation, symbol segmentation, their classification and structural analysis. Individual parts have been published by the authors previously and so are described briefly and referenced. Thorough evaluation on benchmark databases shows the accuracy of the system reaches the state of the art and is ready for practical use. The paper brings several contributions: (a) the entire system and its state-of-the-art performance; (b) the methodology exploring document structure when it is loosely defined; (c) the thorough experimental evaluation; (d) the new annotated database for online sketched flowcharts and finite automata diagrams.

This work deals with the on-line recognition of hand-drawn graphical sketches with structure. We present a novel approach, in which the search for a suitable interpretation of the input is formulated as a combinatorial optimization task - the max-sum problem. The recognition pipeline consists of two main stages. First, groups of strokes possibly representing symbols of a sketch (symbol candidates) are segmented and relations between them are detected. Second, a combination of symbol candidates best fitting the input is chosen by solving the optimization problem. We focused on flowchart recognition. Training and testing of our method was done on a freely available benchmark database. We correctly segmented and recognized 82.7% of the symbols having 31.5% of the diagrams recognized without any error. It indicates that our approach has promising potential and can compete with the state-of-the-art methods.

This study aims to digitalize the Poppelreuter's overlapping figures test. The Poppelreuter's test used in psychology and neurology to assess visual perceptual function. Its recent modification performed with pencil and paper. Replacing the pencil and paper by the tablet computer equipped with the stylus, allows recording and analyzing fine motor motions observed during the test. On the one hand, this provides an opportunity to compute the measures describing condition of the participant. On the other hand, this possess two major problems to be tackled. The first one is to recognize the contours of the overlapping objects drawn by the participant. In the case of severe neurologic disorder, dissimilarity between the etalon shape and drawn contour may be very high. The second problem is to identify errors made during the drawing. The both problems are addressed within this study. Traditional machine learning techniques K-means, k-nearest neighbors and random forest used in this study to identify drawn contours and drawing mistakes. Finally, to demonstrate applicability of the proposed approach, kinematic parameters analyzed for the pilot groups of Parkinson Disease patients and healthy individuals.

This paper proposes new potential functions for Conditional Random Fields (CRF) in image labeling framework based on generalized Gaussian mixture Modeling (GGMM) of the potential functions. Laplacian mixture potential functions have previously been applied to CRF. However, Laplacian potentials fail to capture data characteristics where data fluctuations happen very smoothly; so that they even give rise to induction of atypical results due to erroneous modeling of data. Having an additional shape manipulation parameter, generalized Gaussian mixtures (GGM) can model data characteristics and fluctuations precisely. In this paper, we propose to deploy GGM in the CRF framework to formulate the potential functions. Expectation maximization (EM) technique is used to estimate GGM parameters. Belief propagation and stochastic gradient descent algorithms are utilized for CRF inference and training, respectively. We show that proposed GGM feature functions effectively improve labeling accuracy of nature images in comparison with Laplacian mixtures. Qualitative labeling results show that the proposed framework performs well particularly for labeling simple even backgrounds where the Laplacian counterparts impose irregular outcomes. That is, despite Laplacian mixtures, GGM-based feature functions can correctly model smooth image color and texture variations.

This paper proposes a novel framework for offline signature verification. Different from previous methods, our approach makes use of online handwriting instead of handwritten images for registration. The online registrations enable robust recovery of the writing trajectory from an input offline signature and thus allow effective shape matching between registration and verification signatures. In addition, we propose several new techniques to improve the performance of the new signature verification system: 1. we formulate and solve the recovery of writing trajectory within the framework of conditional random fields; 2. we propose a new shape descriptor, online context, for aligning signatures; 3. we develop a verification criterion which combines the duration and amplitude variances of handwriting. Experiments on a benchmark database show that the proposed method significantly outperforms the well-known offline signature verification methods and achieve comparable performance with online signature verification methods.

This document presents a method of a logical links structure recognition between elements on diagrams. The applied approach intuitively mimics a human way of recognition that relies on merging already found connectors into more complex ones. This procedure is modeled by our method where simple and obvious connectors and gradually extended to more complex structures. Each iteration may lead to modification of connectors set obtained so far. The modifications are managed by a rules set describing logical and graphical constraints that should be satisfied by the connectors structure. If the extension leads to violation of constraints defined by the rules then the modification is not carried out. In this way, the recognized diagram structure is consistent with the assumed principles. The method was experimentally validated using the set of diagrams from three domains. In conclusions, method's advantages and drawbacks are discussed.

This article proposes a sparse conditional random field (SCRF) model to exploit contextual information for classification problems, and select relevant features to prevent overfitting derived from excessively large numbers of features. The sparsity arises from the use of Laplacian priors on the parameters of CRFs, which encourages the parameter estimates to be either significantly large or exactly zero. Since the Laplacian distribution is nondifferentiable at the origin, we developed a simple but efficient sub-gradient-based training algorithm to compute a maximum a posteriori (MAP) point estimate of the CRF parameters. We used SCRF for the classification between urban and non-urban areas in an optical remote sensing image database. The results attest to the accuracy, sparsity and effectiveness of the proposed model.

There has been considerable interest in constructing pen-based intelligent tutoring systems due to the natural interaction metaphor and low cognitive load afforded by pen-based interaction. We believe that pen-based intelligent tutoring systems can be further enhanced by integrating animation techniques. In this work, we explore methods for recognizing and animating sketched physics diagrams. Our methodologies enable an Intelligent Tutoring System (ITS) to understand the scenario and requirements posed by a given problem statement and to couple this knowledge with a computational model of the student’s handwritten solution. These pieces of information are used to construct meaningful animations and feedback mechanisms that can highlight errors in student solutions. We have constructed a prototype ITS that can recognize mathematics and diagrams in a handwritten solution and infer implicit relationships among diagram elements, mathematics and annotations such as arrows and dotted lines. We use natural language processing to identify the domain of a given problem, and use this information to select one or more of four domain-specific physics simulators to animate the user’s sketched diagram. We enable students to use their answers to guide animation behavior and also describe a novel algorithm for checking recognized student solutions. We provide examples of scenarios that can be modeled using our prototype system and discuss the strengths and weaknesses of our current prototype.

The partition function plays a key role in probabilistic modeling including conditional random fields, graphical models, and maximum likelihood estimation. To optimize partition functions, this article introduces a quadratic variational upper bound. This inequality facilitates majorization methods: optimization of complicated functions through the iterative solution of simpler sub-problems. Such bounds remain efficient to compute even when the partition function involves a graphical model (with small tree-width) or in latent likelihood settings. For large-scale problems, low-rank versions of the bound are provided and outperform LBFGS as well as first-order methods. Several learning applications are shown and reduce to fast and convergent update rules. Experimental results show advantages over state-of-the-art optimization methods.

The low accuracy rates of text-shape dividers for digital ink diagrams are hindering their use in real world applications. While recognition of handwriting is well advanced and there have been many recognition approaches proposed for hand drawn sketches, there has been less attention on the division of text and drawing ink. Feature based recognition is a common approach for text-shape division. However, the choice of features and algorithms are critical to the success of the recognition. We propose the use of data mining techniques to build more accurate text-shape dividers. A comparative study is used to systematically identify the algorithms best suited for the specific problem. We have generated dividers using data mining with diagrams from three domains and a comprehensive ink feature library. The extensive evaluation on diagrams from six different domains has shown that our resulting dividers, using LADTree and LogitBoost, are significantly more accurate than three existing dividers.

The increasing availability of pen-based hardware has recently resulted in a parallel growth in sketch-based user interfaces. Sketch-based user interfaces aim to combine the expressive power of free-hand sketching with the processing power of computers. Most sketch-based systems require intelligent ink processing capabilities, which makes the development of robust sketch recognition algorithms a primary concern in the field. So far, the research in sketch recognition has produced various independent approaches to recognition, each of which uses a particular kind of information (e.g., geometric and spatial constraints, image-based features, temporal stroke-ordering patterns). These methods were designed in isolation as stand-alone algorithms, and there has been little work treating various recognition methods as alternative sources of information that can be combined to increase sketch recognition accuracy. In this paper, we focus on two such methods and fuse an image-based method with a time-based method in an attempt to combine the knowledge of how objects look (image data) with the knowledge of how they are drawn (temporal data). In the course of combining spatial and temporal information, we also introduce a mathematically well founded fusion method for combining recognizers. Our combination method can be used for isolated sketch recognition as well as full diagram recognition. Our evaluation with two databases shows that fusing image-based and temporal features yields higher recognition rates. These results are the first to confirm the complementary nature of image-based and temporal recognition methods for full sketch recognition, which has long been suggested, but never supported by data.

Structuring and rearranging free-hand sketches on large interactive surfaces typically requires making multiple stroke selections. This can be both time-consuming and fatiguing in the absence of well-designed selection tools. Investigating the concept of automated clustering, we conducted a background study which highlighted the fact that people have varying perspectives on how elements in sketches can and should be grouped. In response to these diverse user expectations, we present cLuster, a flexible, domain-independent clustering approach for free-hand sketches. Our approach is designed to accept an initial user selection, which is then used to calculate a linear combination of pre-trained perspectives in real-time. The remaining elements are then clustered. An initial evaluation revealed that in many cases, only a few corrections were necessary to achieve the desired clustering results. Finally, we demonstrate the utility of our approach in a variety of application scenarios.

Sketch-based modeling shares many of the difficulties of the branch of computer vision that deals with single image interpretation. Most obviously, they must both identify the parts observed in a given 2D drawing or image.We draw on constellation models first proposed in the computer vision literature to develop probabilistic models for object sketches, based on multiple example drawings. These models are then applied to estimate the most-likely labels for a new sketch. A multi-pass branch-and-bound algorithm allows well-formed sketches to be quickly labelled, while still supporting the recognition of more ambiguous sketches. Results are presented for five classes of objects.