Incremental acquisition and visualization of 3d ultrasound images

This dissertation describes work on 3D visualization of ultrasound echography data. The future goal of this research is the in-place volume visualization of medical 3D ultrasound images acquired and visualized real-time. For example, using such a system, a doctor wearing a special glasses would see a volume-visualized image of the fetus in the mother‘s abdomen. This dissertation discusses two feasibility study systems that have been developed in order to push the state of the art toward this goal. The work on the first system, the static viewpoint 3D echography system, shows that it is possible with current graphics hardware to visualize, at an interactive rate, a stationary object from a series of 2D echography image slices hand-guided with 3 degrees-of-freedom. This work includes development of an incremental volume reconstruction algorithm for irregularly spaced samples and development of an efficient volume visualization algorithm based on a spatial bounding technique. The work on the second system, the dynamic viewpoint 3D echography system, shows the feasibility of a system that uses a video see-through head-mounted display to realize in-place visualization of ultrasound echography datasets.

[1]  James T. Kajiya,et al.  Ray tracing complex scenes , 1986, SIGGRAPH.

[2]  Christian Barillot,et al.  Multidimensional ultrasonic imaging for cardiology , 1988, Proc. IEEE.

[3]  D. Jackél,et al.  The Graphics PARCUM * System: A 3D Memory Based Computer Architecture for Processing and Display of Solid Models , 1985, Comput. Graph. Forum.

[4]  Ivan E. Sutherland,et al.  A head-mounted three dimensional display , 1968, AFIPS Fall Joint Computing Conference.

[5]  P. Hanrahan,et al.  Area and volume coherence for efficient visualization of 3D scalar functions , 1990, VVS.

[6]  John F. Hughes,et al.  Sculpting: an interactive volumetric modeling technique , 1991, SIGGRAPH.

[7]  Dan Gordon,et al.  Image space shading of 3-dimensional objects , 1985, Comput. Vis. Graph. Image Process..

[8]  M Halliwell,et al.  Ultimate limits in ultrasonic imaging resolution. , 1991, Ultrasound in medicine & biology.

[9]  John B. Shoven,et al.  I , Edinburgh Medical and Surgical Journal.

[10]  Stephen W. Smith,et al.  Beam Steering with Linear Arrays , 1983, IEEE Transactions on Biomedical Engineering.

[11]  Marc Levoy,et al.  Volume rendering on scalable shared-memory MIMD architectures , 1992, VVS.

[12]  Jayaram K. Udupa,et al.  Surface and volume rendering in 3D imaging: a comparison , 1991 .

[13]  Kevin L. Novins,et al.  An efficient method for volume rendering using perspective projection , 1990, VVS.

[14]  Raffaele Perego,et al.  Parallel volume visualization on a hypercube architecture , 1992, VVS.

[15]  F. Harris.  On the use of windows for harmonic analysis with the discrete Fourier transform , 1978, Proceedings of the IEEE.

[16]  James T. Kajiya,et al.  Ray tracing volume densities , 1984, SIGGRAPH.

[17]  Nelson Max,et al.  Light diffusion through clouds and haze , 1986, Comput. Vis. Graph. Image Process..

[18]  James J. Clark,et al.  A transformation method for the reconstruction of functions from nonuniformly spaced samples , 1985, IEEE Trans. Acoust. Speech Signal Process..

[19]  Peter Shirley,et al.  A polygonal approximation to direct scalar volume rendering , 1990, SIGGRAPH 1990.

[20]  Jean-Daniel Boissonnat,et al.  Shape reconstruction from planar cross sections , 1988, Comput. Vis. Graph. Image Process..

[21]  Henry Fuchs,et al.  High-Speed Interaction On A Vibrating-Mirror 3D Display , 1984, Optics & Photonics.

[22]  Thomas W. Parsons,et al.  Digital signal processing: theory, applications, and hardware , 1991 .

[23]  Michael Ian Shamos,et al.  Computational geometry: an introduction , 1985 .

[24]  Ross T. Whitaker,et al.  Direct visualization of volume data , 1992, IEEE Computer Graphics and Applications.

[25]  Bernard Gibaud,et al.  3-D Representation Of Anatomic Structures From CT Examinations , 1986, Other Conferences.

[26]  Arie E. Kaufman,et al.  Memory and processing architecture for 3D voxel-based imagery , 1988, IEEE Computer Graphics and Applications.

[27]  John G. Eyles,et al.  PixelFlow: high-speed rendering using image composition , 1992, SIGGRAPH.

[28]  Rae A. Earnshaw,et al.  Virtual Reality Systems , 1993 .

[29]  Hiroshi Abe,et al.  Three‐dimensional echocardiography for spatial visualization and volume calculation of cardiac structures , 1981, Journal of clinical ultrasound : JCU.

[30]  Marc Levoy,et al.  Gaze-directed volume rendering , 1990, I3D '90.

[31]  Donald P. Greenberg,et al.  Perceiving spatial relationships in computer-generated images , 1992, IEEE Computer Graphics and Applications.

[32]  J. Owxzarczyk,et al.  Evaluation of true 3D display systems for visualizing medical volume data , 1990 .

[33]  M Fatemi,et al.  Ultrasonic B-scan imaging: theory of image formation and a technique for restoration. , 1980, Ultrasonic imaging.

[34]  Carolyn A. Bucholtz,et al.  Shape-based interpolation , 1992, IEEE Computer Graphics and Applications.

[35]  Pat Hanrahan,et al.  Fast algorithms for volume ray tracing , 1992, VVS.

[36]  G. C. Fox,et al.  Solving Problems on Concurrent Processors , 1988 .

[37]  David S. Ebert,et al.  Rendering and animation of gaseous phenomena by combining fast volume and scanline A-buffer techniques , 1990, SIGGRAPH.

[38]  K B Chandran,et al.  Computerized three-dimensional finite element reconstruction of the left ventricle from cross-sectional echocardiograms. , 1984, Ultrasonic imaging.

[39]  Warren Robinett,et al.  Virtual environment display system , 1987, I3D '86.

[40]  Jay Beck,et al.  Generalized two- and three-dimensional clipping , 1978, Comput. Graph..

[41]  Philip K. Robertson,et al.  Volume rendering on the MasPar MP-1 , 1992, VVS.

[42]  Arie E. Kaufman,et al.  Voxel-Based Architecture for Three-Dimensional Graphics , 1986, IFIP Congress.

[43]  J C Bamber,et al.  Ultrasonic B-scanning: a computer simulation , 1980, Physics in medicine and biology.

[44]  Ryutarou Ohbuchi,et al.  Merging virtual objects with the real world: seeing ultrasound imagery within the patient , 1992, SIGGRAPH.

[45]  E. Grosse.  A catalogue of algorithms for approximation , 1990 .

[46]  Paul A. Wetzel,et al.  Evaluation of eye tracking measurement systems for use with the fiber optic helmet-mounted display , 1990, Other Conferences.

[47]  Warren Robinett,et al.  A Computational Model for the Stereoscopic Optics of a Head-Mounted Display , 1993, Virtual Reality Systems.

[48]  Nelson L. Max,et al.  Atmospheric illumination and shadows , 1986, SIGGRAPH.

[49]  Daniel L. Toth,et al.  On ray tracing parametric surfaces , 1985, SIGGRAPH.

[50]  Wolfgang Krueger.  Volume rendering and data feature enhancement , 1990, SIGGRAPH 1990.

[51]  J. M. Levaillant,et al.  Three-dimensional ultrasound imaging of the female breast and human fetus in utero: Preliminary results , 1989 .

[52]  Paul Hudak,et al.  Memory coherence in shared virtual memory systems , 1986, PODC '86.

[53]  Takayuki Tanaka,et al.  ARTS: Accelerated Ray-Tracing System , 1986, IEEE Computer Graphics and Applications.

[54]  Peter Schröder,et al.  Fast rotation of volume data on parallel architectures , 1991, Proceeding Visualization '91.

[55]  Daniel P. Westra,et al.  Simulator Design Features for Helicopter Landing on Small Ships , 1984 .

[56]  Marc Levoy,et al.  Efficient ray tracing of volume data , 1990, TOGS.

[57]  T. A. Foley,et al.  Three-stage interpolation to scattered data , 1984 .

[58]  Gregory J. Ward,et al.  Measuring and modeling anisotropic reflection , 1992, SIGGRAPH.

[59]  Robert E. Barnhill,et al.  Representation and Approximation of Surfaces , 1977 .

[60]  W. A. Verhoef,et al.  Texture of B-Mode Echograms: 3-D Simulations and Experiments of the Effects of Diffraction and Scatterer Density , 1985 .

[61]  J. Ophir,et al.  Digital scan converters in diagnostic ultrasound imaging , 1979, Proceedings of the IEEE.

[62]  Donald P. Greenberg,et al.  A comprehensive physical model for light reflection , 1991, SIGGRAPH.

[63]  Steven K. Feiner,et al.  Knowledge-based augmented reality , 1993, CACM.

[64]  Jane Wilhelms,et al.  A coherent projection approach for direct volume rendering , 1991, SIGGRAPH.

[65]  Randy F. Pausch,et al.  A Literature Survey for Virtual Environments: Military Flight Simulator Visual Systems and Simulator Sickness , 1992, Presence: Teleoperators & Virtual Environments.

[66]  J.F. Havlice,et al.  Medical ultrasonic imaging: An overview of principles and instrumentation , 1979, Proceedings of the IEEE.

[67]  G T Herman,et al.  Dynamic three-dimensional reconstruction of the left ventricle from two-dimensional echocardiograms. , 1986, Journal of the American College of Cardiology.

[68]  Ronald Azuma,et al.  A demonstrated optical tracker with scalable work area for head-mounted display systems , 1992, I3D '92.

[69]  Pat Hanrahan,et al.  Hierarchical splatting: a progressive refinement algorithm for volume rendering , 1991, SIGGRAPH.

[70]  Bjørn Olstad,et al.  Visualizing 4-D medical ultrasound data , 1991, Proceeding Visualization '91.

[71]  J Perrin,et al.  Specular reflector noise: effect and correction for in vivo attenuation estimation. , 1985, Ultrasonic imaging.

[72]  Craig Upson,et al.  V-buffer: visible volume rendering , 1988, SIGGRAPH.

[73]  F. P. Brooks,et al.  Grasping reality through illusion—interactive graphics serving science , 1988, CHI '88.

[74]  R. Bernstein,et al.  Shading 3D-Images from CT Using Gray-Level Gradients , 1986, IEEE Transactions on Medical Imaging.

[75]  M. R. Harris,et al.  Exploring Virtual Worlds With Head-Mounted Displays , 1989, Photonics West - Lasers and Applications in Science and Engineering.

[76]  Stephen H. Westin,et al.  A global illumination solution for general reflectance distributions , 1991, SIGGRAPH.

[77]  David Salesin,et al.  A fast and accurate light reflection model , 1992, SIGGRAPH.

[78]  Anselmo Lastra,et al.  Virtual Environments: A Survey of the Technology , 1993 .

[79]  Wolfgang Straßer,et al.  Reconstructing Solids from Tomographic Scans - The PARCUM II System , 1987, Advances in Computer Graphics Hardware.

[80]  S. Leeman,et al.  Pulse Scattering in Dispersive Media , 1982 .

[81]  R. L. Hardy.  Multiquadric equations of topography and other irregular surfaces , 1971 .

[82]  L. Schumaker.  Fitting surfaces to scattered data , 1976 .

[83]  Holly Rushmeier,et al.  Realistic image synthesis for scenes with radiatively participating media , 1988 .

[84]  K. C. Knowlton.  Computer displays optically superimposed on input devices , 1977, The Bell System Technical Journal.

[85]  Donald P. Greenberg,et al.  Echocardiographic Three-Dimensional Visualization of the Heart , 1990 .

[86]  S.W. Smith,et al.  High-speed ultrasound volumetric imaging system. II. Parallel processing and image display , 1991, IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control.

[87]  Henry Fuchs,et al.  Pixel-planes 5: a heterogeneous multiprocessor graphics system using processor-enhanced memories , 1989, SIGGRAPH.

[88]  Isao Shirakawa,et al.  Links-1 - a parallel pipelined multimicrocomputer system for image creation , 1983, ISCA '83.

[89]  Stephen H. Westin,et al.  Predicting reflectance functions from complex surfaces , 1992, SIGGRAPH.

[90]  D. Shepard.  A two-dimensional interpolation function for irregularly-spaced data , 1968, ACM National Conference.

[91]  Arie Kaufman.  Memory Organization for a Cubic Frame Buffer , 1986, Eurographics.

[92]  Lee Westover,et al.  Footprint evaluation for volume rendering , 1990, SIGGRAPH.

[93]  Henry Stark,et al.  Direct Fourier Reconstruction in Fan-Beam Tomography , 1987, IEEE Transactions on Medical Imaging.

[94]  Vincent G. McDermott.  Diagnostic Sonography Principles and Clinical Applications , 1996 .

[95]  K. A. Dines,et al.  Ultrasonic Attenuation Tomography of Soft Tissues , 1979 .

[96]  Robert E. Barnhill,et al.  Multistage trivariate surfaces , 1984 .

[97]  J F Brinkley,et al.  Ultrasonic three-dimensional imaging and volume from a series of arbitrary sector scans. , 1978, Ultrasound in medicine & biology.

[98]  Marc Levoy,et al.  A hybrid ray tracer for rendering polygon and volume data , 1990, IEEE Computer Graphics and Applications.

[99]  Steven A. Johnson,et al.  WAVE EQUATIONS AND INVERSE SOLUTIONS FOR SOFT TISSUE , 1982 .

[100]  A N Venetsanopoulos,et al.  Modelling and restoration of ultrasonic phased-array B-scan images. , 1985, Ultrasonic imaging.

[101]  S.W. Smith,et al.  High-speed ultrasound volumetric imaging system. I. Transducer design and beam steering , 1991, IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control.

[102]  R Bernstein,et al.  Combined surface display and reformatting for the three-dimensional analysis of tomographic data. , 1987, Investigative radiology.

[103]  Jayaram K. Udupa,et al.  Surface Shading in the Cuberille Environment , 1985, IEEE Computer Graphics and Applications.

[104]  B. Fornage,et al.  Sonographic appearance and ultrasound‐guided fine‐needle aspiration biopsy of breast carcinomas smaller than 1 cm3. , 1990, Journal of ultrasound in medicine : official journal of the American Institute of Ultrasound in Medicine.

[105]  M Itoh,et al.  A computer-aided three-dimensional display system for ultrasonic diagnosis of a breast tumour. , 1979, Ultrasonics.

[106]  Judy Challinger,et al.  Direct volume rendering of curvilinear volumes , 1990, SIGGRAPH 1990.

[107]  Marc D Weinshenker,et al.  Explososcan: a parallel processing technique for high speed ultrasound imaging with linear phased arrays. , 1984 .

[108]  Henry Fuchs,et al.  Optimal surface reconstruction from planar contours , 1977, CACM.

[109]  Mark R. Mine.  Characterization of End-to-End Delays in Head-Mounted Display Systems , 1993 .

[110]  R. Franke.  A Critical Comparison of Some Methods for Interpolation of Scattered Data , 1979 .

[111]  Mark Riley,et al.  CARVUPP: Computer Assisted Radiological Visualisation Using Parallel Processing , 1990 .

[112]  Alan Liu,et al.  A fast ray casting algorithm using adaptive isotriangular subdivision , 1991, VIS '91.

[113]  W. Lorensen,et al.  Two algorithms for the three-dimensional reconstruction of tomograms. , 1988, Medical physics.

[114]  Tatsuo Miyazawa.  High-speed integrated rendering algorithm for interpreting multiple-variable 3-D data , 1991, Electronic Imaging.

[115]  R. L. Hardy,et al.  LEAST SQUARES PREDICTION , 1977 .

[116]  W. Moritz,et al.  A new three-dimensional echocardiographic method of right ventricular volume measurement: in vitro validation. , 1986, Journal of the American College of Cardiology.

[117]  Henry Fuchs,et al.  Incremental 3D ultrasound imaging from a 2D scanner , 1990, [1990] Proceedings of the First Conference on Visualization in Biomedical Computing.

[118]  Michael Stumm,et al.  Algorithms implementing distributed shared memory , 1990, Computer.

[119]  Henry Fuchs,et al.  3D ultrasound display using optical tracking , 1990, [1990] Proceedings of the First Conference on Visualization in Biomedical Computing.

[120]  Mario Tokoro,et al.  A Three-dimensional Shaded Display Method for Voxel-based Representation , 1985, Eurographics.

[121]  W H Round,et al.  Modification of spectra of pulses from ultrasonic transducers by scatters in non-attenuating and in attenuating media. , 1987, Ultrasonic imaging.

[122]  Georgios Sakas,et al.  Interactive visualization of large scalar voxel fields , 1992, Proceedings Visualization '92.

[123]  Kellogg S. Booth,et al.  Evaluating 3D task performance for fish tank virtual worlds , 1993, TOIS.

[124]  Eric Keppel,et al.  Approximating Complex Surfaces by Triangulation of Contour Lines , 1975, IBM J. Res. Dev..

[125]  S. Rippa,et al.  Numerical Procedures for Surface Fitting of Scattered Data by Radial Functions , 1986 .

[126]  F.L. Thurstone,et al.  Biomedical Ultrasonics , 1970, IEEE Transactions on Industrial Electronics and Control Instrumentation.

[127]  J. Thijssen,et al.  Texture in tissue echograms. Speckle or information? , 1990, Journal of ultrasound in medicine : official journal of the American Institute of Ultrasound in Medicine.

[128]  D. H. McLain,et al.  Drawing Contours from Arbitrary Data Points , 1974, Comput. J..

[129]  Samuel M. Goldwasser,et al.  Real-time display and manipulation of 3-d medical objects: the voxel processor architecture , 1987 .

[130]  Jayaram K. Udupa,et al.  Fast surface tracking in three-dimensional binary images , 1989, Comput. Vis. Graph. Image Process..

[131]  R. E. Barnhill,et al.  Three- and four-dimensional surfaces , 1984 .

[132]  Kenneth E. Torrance,et al.  The zonal method for calculating light intensities in the presence of a participating medium , 1987, SIGGRAPH.

[133]  J. R. Higgins.  A sampling theorem for irregularly spaced sample points (Corresp.) , 1976, IEEE Trans. Inf. Theory.

[134]  W. J. Gordon,et al.  Shepard’s method of “metric interpolation” to bivariate and multivariate interpolation , 1978 .

[135]  Takeo Kanade,et al.  A Multiple-Baseline Stereo , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[136]  Steven E. Molnar,et al.  Combining Z-buffer Engines for Higher-Speed Rendering , 1988, Advances in Computer Graphics Hardware.

[137]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[138]  D. King,et al.  Three‐dimensional spatial registration and interactive display of position and orientation of real‐time ultrasound images. , 1990, Journal of ultrasound in medicine : official journal of the American Institute of Ultrasound in Medicine.