Deformable Shape Recovery by Factorization Based on a Spatiotemporal Measurement Matrix

An efficient technique based on factorization is presented for recovering the 3-D shape of deformable objects. The original factorization solely recovers the 3-D shape of rigid objects. The present technique applies it to shape recovery of deformable objects. In the proposed technique, multiple video cameras at fixed locations take video images of an object. The obtained image sequences are analyzed to yield measurement matrices at respective sample times and they are merged into a single matrix called a spatiotemporal measurement matrix. Factorization is applied to the matrix once and this results in the entire 3-D shape recovery during observation in a single stage. In the performed experiment, some human motions and the inflation/deflation process of a toy balloon successfully recovered their 3-D shape. Advantages of the present technique are that calibration is not necessary with the employed multiple video cameras, that the entire deformation is recovered by single application of the factorization, which eliminates the alignment of recovered shape in the time axis, that the recovered shape contains less recovery errors because of averaging effect by the employment of the proposed matrix, and that the technique is applicable to any object on which feature points can be specified. Various applications related to human motions recovery are expected.