A Tracking Method for 2D Canvas in MR-Based Interactive Painting System

We have proposed a mixed reality based painting system. In this paper, we tackle a problem in our previous painting system that fully relies on magnetic sensor that is attached on the canvas, the users needed to detach and attach a sensor on the canvas during painting when they want to switch the canvas. Instead of that, in this paper, we aim to automatically detect the shape of the canvas for registration purpose. Using the shape or region detection method such as MSER (maximally stable extremal regions), we detect and track the shape on the canvas on the captured camera image. We then compute the camera pose for virtually overlay the painting result. Using the brush device, we can draw and paint freely on the tracked canvases. We show that using visual based tracking method, we can generate the equivalent result compared to the result of using the sensor.

[1]  David H. Douglas,et al.  ALGORITHMS FOR THE REDUCTION OF THE NUMBER OF POINTS REQUIRED TO REPRESENT A DIGITIZED LINE OR ITS CARICATURE , 1973 .

[2]  David Nistér,et al.  Linear Time Maximally Stable Extremal Regions , 2008, ECCV.

[3]  Ivan Poupyrev,et al.  Virtual object manipulation on a table-top AR environment , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[4]  Jihad El-Sana,et al.  Shape recognition and pose estimation for mobile augmented reality , 2009, ISMAR.

[5]  Dieter Schmalstieg,et al.  ARToolKitPlus for Pose Trackin on Mobile Devices , 2007 .

[6]  Hideyuki Tamura,et al.  AR/sup 2/Hockey: a case study of collaborative augmented reality , 1998, Proceedings. IEEE 1998 Virtual Reality Annual International Symposium (Cat. No.98CB36180).

[7]  Peter Kontschieder,et al.  Robust planar target tracking and pose estimation from a single concavity , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[8]  Ramesh Raskar,et al.  Dynamic shader lamps : painting on movable objects , 2001, Proceedings IEEE and ACM International Symposium on Augmented Reality.

[9]  Yoon-Ho Cho,et al.  Development of an immersive VR display system for 3D digital art , 2004, Proceedings. International Conference on Computer Graphics, Imaging and Visualization, 2004. CGIV 2004..

[10]  Peter Schröder,et al.  Surface drawing: creating organic 3D shapes with the hand and tangible tools , 2001, CHI.

[11]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[12]  Jihad El-Sana,et al.  In-place 3D sketching for authoring and augmenting mechanical systems , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[13]  Fumihisa Shibata,et al.  MAI painting brush: an interactive device that realizes the feeling of real painting , 2010, UIST.

[14]  Gerard Jounghyun Kim,et al.  Immersive authoring: What You eXperience Is What You Get (WYXIWYG) , 2005, CACM.

[15]  Thomas K. Peucker,et al.  2. Algorithms for the Reduction of the Number of Points Required to Represent a Digitized Line or its Caricature , 2011 .

[16]  Hirokazu Kato,et al.  Marker tracking and HMD calibration for a video-based augmented reality conferencing system , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).