Joint Alignment and Stitching of Non Overlapping Meshes

We contribute a novel algorithm for aligning and stitching non-overlapping 3D meshes. Mesh aligning and stitching are often combined to construct a complete 3D model of an object. There are standard algorithms that address this task for the case of overlapping meshes, i.e., meshes that share common parts of a given object. The overlap allows the registration and alignment of different meshes. The algorithm here proposed, Joint Alignment and Stitching of Non-Overlapping Meshes (JASNOM), addresses the case when the overlap is not possible due to acquisition or application constraints. JASNOM takes advantage that both meshes can only be connected by their boundary to reframe the alignment problem as a search of the best assignment between boundary vertices. Given an assignment, the boundaries can be aligned with a rigid transformation that aligns the two boundaries by minimizing the distance between corresponding vertices while preventing the meshes from intersecting. However, not all assignments are valid. The boundaries are oriented curves and assignments between the two boundaries must preserve the order of the vertices. JASNOM takes advantage of the reduced number of order preserving assignments to perform exhaustive search for the best assignment. The outcome of the search is not only the assignment between points in the boundaries, but also the rigid transformation that aligns the two meshes. After alignment, meshes can then be stitched by using the assignment between vertices. One of the applications we envision for JASNOM is the modeling of 3D objects retrieved from range cameras that cannot overlap such as the Kinect camera. Another possible use of JASNOM is to fill holes in a mesh using parts of other meshes. In this work, we propose a new algorithm to solve the problem of aligning and stitching two non-overlapping meshes of the same object. The problem of stitching meshes has long been introduced in 3D object modeling, e.g. in [1]. However, to the best of our knowledge, no work has yet been devoted to deal with meshes that share no common parts. Our objective, as depicted in Figure 1, is to align two meshes, M1 and M2, that belong to the same object but are complementary. After alignment, meshes should be glued together in order to create a single mesh, M that models the surface of the whole object. A possible application for our algorithm, Joint Alignment and Stitching of Non Overlapping Meshes (JASNOM), is the construction of object models from meshes captured from range sensors in diametrically opposite positions with respect to the object. Placed in such a geometry, the sensors provide an almost complete view of the object in a single time frame. JASNOM is able to align and stitch meshes returned by each sensor. Because meshes from the different cameras would be acquired with little time lapse, our algorithm can model 1 ECE departments at CMU and IST-UTL sbrandao@ece.cmu.edu ECE department at IST-UTL, jpc@isr.ist.utl.pt 2 CS department at CMU, veloso@cmu.edu Fig. 1. Our objective is to construct a mesh M from two other meshes, M1 and M2. Both meshes have a boundary B1 and B2 that do not overlap. To construct M , we align both boundaries through a rotation R and a translation t and stitch the two meshes together by introducing a connection mesh Mc. dynamic objects, such as humans or animals, as rigid objects. This application is of particular value in cases where the range sensors cannot be used in overlap with each other, as is the case of the Kinect cameras. Due to interference between the light patterns of different cameras, the Kinect cameras do not allow multiple acquisition of images from the same surface at the same time instant. Another possible application of JASNOM is to fill holes in a mesh. In the case of interactive object modeling, our algorithm allows a user to select parts from a mesh or library of meshes and use them to fill holes in an incomplete 3D model. JASNOM automatically aligns meshes with boundaries and stitches the two meshes. Because it is designed for complementary meshes, the user would not need to select the exact part of the the mesh to fill the hole. Instead, the user would only need to select a part that would fit inside the hole. The capacity of filling holes from other meshes is of valuable use for the modeling of objects that have self similar surfaces such as planes or cylinders. There are several algorithms stitch two meshes. To our knowledge, one of the first was presented in [1] and is available in mesh manipulation software programs such as Meshlab. However, the algorithm proposed in [1] assumes that meshes are initially aligned. The initial alignment requires the two meshes to have surfaces in common and thus the algorithm does not handle complementarity between shapes. Furthermore, holes in 3d object models are so common that several algorithms have been proposed to remove them. Some algorithms, e.g. [2], [3], also use information from other meshes to fill in the holes. However, they again require overlap between meshes. There are also algorithms for closing holes that do not require overlap such as [4] and [5]. However, those try to guess how the missing mesh should look based on its neighborhood. Our proposed algorithm, JASNOM, differs from the previous works by not requiring previous alignment nor mesh overlap. JASNOM aligns meshes by iteratively assigning correspondences between vertices in the two meshes. Given an assignment between vertices, the algorithm computes the rigid transformation that minimizes a cost function that depends on the distance between corresponding vertices. All the possible assignments are tested and the best assignment is the one that has the minimum cost function after alignment. Because the search is exhaustive, JASNOM returns the global best assignment and respective rigid transformation. One of the contributions of this paper is on how to decrease the search space from the Nb1 × Nb2, the number of possible pairings between the boundary vertices of the two boundaries, to min(Nb1, Nb2). The reduction of the assignment search space is possible because the boundaries define a directed path. Correspondences between the two boundaries must respect the path topology and direction. In practice, given a single correspondence between one vertex in B1 and another vertex in B2, all correspondences between the remaining vertices are automatically defined. The search space is thus reduced to finding the first correspondence. JASNOM introduces the order preserving constraint by parameterizing the boundaries. An example of a possible parameterization is the distance over the boundary to a given vertex. The parameterization also allows the use of sampling in the boundary vertices and further decreases the search space. Given a set of correspondences, JASNOM aligns the boundaries by minimizing the distance between corresponding vertices, while preventing the two meshes from intersecting each other. The intersection is avoided by modeling the contact region between boundaries as a plane. There is no intersection if each mesh keeps to their own side of the plane. One of the contributions of JASNOM is the introduction, in the alignment cost function, of a term that penalizes the number of vertices that cross the plane to the wrong side. Finally, we contribute an algorithm to stitch meshes after alignment. The stitching makes use of the assignments from the alignment stage and ensures that properties like mesh manifoldness are locally preserved. The remaining of the paper is organized as follows: Section I overviews different approaches to 3D object modeling through stitching of different meshes; Section II motivates JASNOM by addressing the problem of mesh stitching from the point of view of topology; Section III introduces the JASNOM algorithm itself; Section IV presents experimental results and, finally, Section V concludes and presents future research.