Patient-specific bronchoscopy visualization through BRDF estimation and disocclusion correction

This paper presents an image-based method for virtual bronchoscope with photo-realistic rendering. The technique is based on recovering bidirectional reflectance distribution function (BRDF) parameters in an environment where the choice of viewing positions, directions, and illumination conditions are restricted. Video images of bronchoscopy examinations are combined with patient-specific three-dimensional (3-D) computed tomography data through two-dimensional (2-D)/3-D registration and shading model parameters are then recovered by exploiting the restricted lighting configurations imposed by the bronchoscope. With the proposed technique, the recovered BRDF is used to predict the expected shading intensity, allowing a texture map independent of lighting conditions to be extracted from each video frame. To correct for disocclusion artefacts, statistical texture synthesis was used to recreate the missing areas. New views not present in the original bronchoscopy video are rendered by evaluating the BRDF with different viewing and illumination parameters. This allows free navigation of the acquired 3-D model with enhanced photo-realism. To assess the practical value of the proposed technique, a detailed visual scoring that involves both real and rendered bronchoscope images is conducted.

[1]  Guang-Zhong Yang,et al.  pq-Space Based 2D/3D Registration for Endoscope Tracking , 2003, MICCAI.

[2]  S K Nayar,et al.  Visual appearance of matte surfaces , 1995, Science.

[3]  Paul E. Debevec.  Pursuing Reality with Image-Based Modeling, Rendering, and Lighting , 2000, SMILE.

[4]  Pierre Poulin,et al.  Interactive Virtual Relighting and Remodeling of Real Scenes , 1999, Rendering Techniques.

[5]  Cristian Lorenz,et al.  Simultaneous segmentation and tree reconstruction of the airways for virtual bronchoscopy , 2002, SPIE Medical Imaging.

[6]  Ivan Bricault,et al.  Registration of real and CT-derived virtual bronchoscopic images to assist transbronchial biopsy , 1998, IEEE Transactions on Medical Imaging.

[7]  Kun Zhou,et al.  Synthesis of progressively-variant textures on arbitrary surfaces , 2003, ACM Trans. Graph..

[8]  Jeremy S. De Bonet,et al.  Multiresolution sampling procedure for analysis and synthesis of texture images , 1997, SIGGRAPH.

[9]  Robert L. Cook,et al.  A Reflectance Model for Computer Graphics , 1987, TOGS.

[10]  Greg Turk,et al.  Generating textures on arbitrary surfaces using reaction-diffusion , 1991, SIGGRAPH.

[11]  Gregory J. Ward,et al.  Measuring and modeling anisotropic reflection , 1992, SIGGRAPH.

[12]  David J. Kriegman,et al.  Fast texture synthesis on arbitrary meshes , 2003, SIGGRAPH 2003.

[13]  Jr. Leonard McMillan,et al.  An Image-Based Approach to Three-Dimensional Computer Graphics , 1997 .

[14]  Greg Turk,et al.  Texture synthesis on surfaces , 2001, SIGGRAPH.

[15]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Donald P. Greenberg,et al.  A comprehensive physical model for light reflection , 1991, SIGGRAPH.

[17]  Jun Sugiyama,et al.  Tracking of a bronchoscope using epipolar geometry analysis and intensity-based image registration of real and virtual endoscopic images , 2002, Medical Image Anal..

[18]  William E. Higgins,et al.  Technique for registering 3D virtual CT images to endoscopic video , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[19]  André Gagalowicz,et al.  Image-based rendering of diffuse, specular and glossy surfaces from a single image , 2001, SIGGRAPH.

[20]  Guang-Zhong Yang,et al.  Enhancement of Visual Realism with BRDF for Patient Specific Bronchoscopy Simulation , 2004, MICCAI.

[21]  Guang-Zhong Yang,et al.  Real-Time Photo-Realistic Rendering for Surgical Simulations with Graphics Hardware , 2004, MIAR.

[22]  Paul E. Debevec,et al.  Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography , 1998, SIGGRAPH '08.

[23]  Hans-Peter Seidel,et al.  Image-based reconstruction of spatial appearance and geometric detail , 2003, TOGS.

[24]  James T. Kajiya,et al.  The rendering equation , 1998 .

[25]  Shree K. Nayar,et al.  Reflectance and texture of real-world surfaces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  Marc Levoy,et al.  Texture synthesis over arbitrary manifold surfaces , 2001, SIGGRAPH.

[27]  Donald P. Greenberg,et al.  The Irradiance Volume , 1998, IEEE Computer Graphics and Applications.

[28]  David S. Ebert,et al.  Texturing and Modeling: A Procedural Approach , 1994 .

[29]  Alexei A. Efros,et al.  Texture synthesis by non-parametric sampling , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[30]  Marc Levoy,et al.  Fast texture synthesis using tree-structured vector quantization , 2000, SIGGRAPH.

[31]  Adam Finkelstein,et al.  Lapped textures , 2000, SIGGRAPH.

[32]  Reyer Zwiggelaar,et al.  Synthesis and analysis of solid texture: application in medical imaging , 2002 .

[33]  Bruno Lévy,et al.  Least squares conformal maps for automatic texture atlas generation , 2002, ACM Trans. Graph..

[34]  Paul Debevec,et al.  Inverse global illumination: Recovering re?ectance models of real scenes from photographs , 1998 .

[35]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH.

[36]  Ivan Bricault.  A fast morphology-based registration - application to computer-assisted bronchoscopy , 1997, CVRMed.

[37]  Paul E. Debevec,et al.  Acquiring the reflectance field of a human face , 2000, SIGGRAPH.

[38]  Paul Debevec.  Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography , 2008, SIGGRAPH Classes.