Three-dimensional ultrasound imaging

INSTITUTE OF PHYSICS PUBLISHING

PHYSICS IN MEDICINE AND BIOLOGY

Phys. Med. Biol. 46 (2001) R67–R99

www.iop.org/Journals/pb

PII: S0031-9155(01)12089-0

TOPICAL REVIEW

Three-dimensional ultrasound imaging Aaron Fenster1 , D´onal B Downey and H Neale Cardinal The John P Robarts Research Institute, London, Canada and the Department of Diagnostic Radiology and Nuclear Medicine, The University of Western Ontario, London, Canada E-mail: [email protected]

Received 6 November 2000 Abstract Ultrasound is an inexpensive and widely used imaging modality for the diagnosis and staging of a number of diseases. In the past two decades, it has benefited from major advances in technology and has become an indispensable imaging modality, due to its flexibility and non-invasive character. In the last decade, research investigators and commercial companies have further advanced ultrasound imaging with the development of 3D ultrasound. This new imaging approach is rapidly achieving widespread use with numerous applications. The major reason for the increase in the use of 3D ultrasound is related to the limitations of 2D viewing of 3D anatomy, using conventional ultrasound. This occurs because: (a) Conventional ultrasound images are 2D, yet the anatomy is 3D, hence the diagnostician must integrate multiple images in his mind. This practice is inefficient, and may lead to variability and incorrect diagnoses. (b) The 2D ultrasound image represents a thin plane at some arbitrary angle in the body. It is difficult to localize the image plane and reproduce it at a later time for follow-up studies. In this review article we describe how 3D ultrasound imaging overcomes these limitations. Specifically, we describe the developments of a number of 3D ultrasound imaging systems using mechanical, free-hand and 2D array scanning techniques. Reconstruction and viewing methods of the 3D images are described with specific examples. Since 3D ultrasound is used to quantify the volume of organs and pathology, the sources of errors in the reconstruction techniques as well as formulae relating design specification to geometric errors are provided. Finally, methods to measure organ volume from the 3D ultrasound images and sources of errors are described.

1 Address for correspondence: The J P Robarts Research Institute, Imaging Research Laboratories, 100 Perth Drive, London, Ontario, Canada N6A 5K8.

0031-9155/01/050067+33$30.00

© 2001 IOP Publishing Ltd

Printed in the UK

R67

R68

A Fenster et al

1. Introduction For most of the last century, medical imaging has involved visualizing the interior structures of the human body with two-dimensional (2D) x-ray images. Although 2D x-ray imaging remains an indispensable imaging modality, it has the disadvantage that almost all three-dimensional (3D) information about the interior of the human body is lost to the physician. In the early 1970s, the introduction of computed tomography (CT) revolutionized diagnostic radiology by providing, for the first time, true 3D anatomical information to the physician. At first this information was simply presented as a series of contiguous 2D image slices of the body. However, the advent of 3D images from CT, and later magnetic resonance imaging (MRI), provided a major impetus to the field of 3D visualization, which has lead to the development of a wide variety of applications in diagnostic medicine (Fishman et al 1991, Robb and Barillot 1989, Vannier et al 1984). Although the origins of medical ultrasound imaging can be traced back half a century to the pioneering work of Wild and Reid (1952), progress was slow for the first three decades, and, outside of research laboratories, medical ultrasound was largely limited to applications in cardiology, obstetrics, and gynaecology. However, in the past two decades, medical ultrasound has benefited from major advances in technology and has become an indispensable imaging modality, due to its flexibility and non-invasive character. Moreover, due to continuous improvements in image quality, and the availability of blood flow information via the Doppler effect, ultrasonography is progressively achieving a greater role in radiology and cardiology, and is finding new roles in image-guided surgery and therapy. In the last decade, research investigators and commercial companies have further advanced ultrasound imaging by combining these advances in ultrasound image quality with recent advances in 3D image visualization, to develop 3D ultrasound imaging. This review article discusses this new imaging modality, which is rapidly achieving widespread application in medical imaging. 2. Limitations of ultrasonography addressed by 3D imaging In a conventional ultrasound examination, the ultrasound transducer is manipulated to obtain a series of 2D ultrasound images, which are mentally combined by the operator to form a subjective impression of the 3D anatomy and pathology. The success of the diagnostic or interventional procedure is thus largely dependent on the skill and experience of the operator in performing these tasks. This is a suboptimal approach, due to the following limitations: • Mentally transforming multiple 2D images to form a 3D impression of the anatomy and pathology is not only time-consuming and inefficient, but is also more importantly, variable and subjective, which can lead to incorrect decisions in diagnosis, and in the planning and delivery of therapy. • Diagnostic (e.g. obstetric) and therapeutic (e.g. staging and planning) decisions often require accurate estimation of organ or tumour volume. Conventional 2D ultrasound techniques calculate volume from simple measurements of height, width and length in two orthogonal views, by assuming an idealized (e.g. ellipsoidal) shape. This practice can potentially lead to low accuracy, high variability and operator dependency. • Conventional 2D ultrasound is suboptimal for monitoring therapeutic procedures, or for performing quantitative prospective or follow-up studies, due to the difficulty in adjusting the transducer position so that the 2D image plane is at the same anatomical site and in the same orientation as in a previous examination.

Three-dimensional ultrasound imaging

R69

• Because the location and orientation of conventional 2D ultrasound images are determined by those of the transducer, some views are impossible to achieve because of restrictions imposed by the patient’s anatomy or position. In a 3D ultrasound examination, the 2D ultrasound images are combined by a computer to form an objective 3D image of the anatomy and pathology. This image can then be viewed, manipulated and measured in 3D by the physician on the same or another computer. Also, a 2D cross-sectional image can be generated in any orientation, without restriction, at any anatomical site, which may easily be registered with a previous or subsequent 3D image. Thus, 3D ultrasound imaging promises to overcome the limitations of 2D ultrasound imaging described above. Moreover, unlike CT and MR imaging, in which 2D images are usually acquired at a slow rate as a stack of parallel slices, in a fixed orientation, ultrasound provides tomographic images at a high rate (15–60 s−1 ), and in arbitrary orientations. The high acquisition rate and arbitrary orientation of the images provide unique problems to overcome and opportunities to exploit in extending 2D ultrasound imaging to 3D and dynamic 3D (i.e. 4D) visualization. Over the past two decades, many investigators have attempted to develop 3D ultrasound imaging techniques (Brinkley et al 1982, Fenster and Downey 1996, Ghosh et al 1982, Greenleaf et al 1993, King et al 1993, Nelson and Pretorius 1992, Rankin et al 1993). However, progress has been slow due to the enormous computational requirements which must be met in order to acquire, reconstruct and view 3D information in near real time on low-cost systems. Advances in low-cost computer technology and visualization techniques in the past few years have made 3D ultrasound imaging viable. A number of laboratories have led the way in demonstrating the feasibility of 3D ultrasound imaging and, more recently, commercial companies have begun to provide these options on their ultrasound equipment. A few review articles and two books describing the development of 3D ultrasound imaging have been published in the past few years (Fenster and Downey 1996, Greenleaf et al 1993, Rankin et al 1993, Belohlavek et al 1993, Fenster 1998, Ofili and Nanda 1994, Nelson et al 1999, Nelson and Pretorius 1998, Baba and Jurkovic 1997, De Castro et al 1998). In this review we summarize the various approaches that have been used in the development of 3D ultrasound imaging systems, with particular emphasis on the quantitative aspects of geometric accuracy in viewing the anatomy in 3D and in measuring anatomical volumes with 3D ultrasound imaging. 3. 3D ultrasound scanning techniques Most 3D ultrasound imaging systems make use of conventional 1D ultrasound transducers to acquire a series of 2D ultrasound images, and differ only in the method used to determine the position and orientation of these 2D images within the 3D image volume being examined. The production of 3D images without distortions requires that three factors be optimized: • The scanning technique must be either rapid or gated, to avoid image artefacts due to involuntary, respiratory or cardiac motion. • The locations and orientations of the acquired 2D images must be accurately known, to avoid geometric distortions in the 3D image that would lead to measurement errors. • The scanning apparatus must be simple and convenient to use, so that the scan is not complicated or awkward to perform and hence easily included in the examination procedure. Over the past two decades, four different 3D ultrasound imaging approaches have been pursued: mechanical scanners, free-hand techniques with position sensing, free-hand

R70

A Fenster et al Table 1. Summary of 3D scanning methods, acquisition methods of images used to produce the 3D image, and the major disadvantages of the scanning methods.

Scanning method Mechanical Linear Tilt Rotational Free-Hand Acoustic

Articulated arms Magnetic sensor

Image correlation

No position sensing 2D arrays

Image acquisition method

Disadvantages

Acquired images are parallel to each other with equal spacing Acquired images are fan-like with equal angular spacing Acquired images are propeller-like with equal angular spacing

Bulky device

Measure time-of-flight of sound from spark gaps on transducer to microphones above patient Measure angulation between movable arms Measure magnetic field generated by transmitter beside the patient with receiver on transducer Measure speckle decorrelation between adjacent images Distance or angle between images is assumed 2D phased array transmits a diverging pyramidal beam and returned echoes are displayed in real time as multiple planes

Resolution degrades with depth Motion of axis of rotation results in artefacts Line of sight required and sound velocity varies with humidity Scanning volume limited, flexing of arms Ferrous metals distort magnetic field Special computer processor required, compound motion is difficult to track Cannot measure distance System cost and signal/noise

techniques without position sensing and 2D arrays. These approaches are summarized in table 1 and are discussed below. 3.1. Mechanical scanners In this approach, the anatomy is scanned by using a motorized mechanical apparatus to translate, tilt or rotate a conventional transducer as it rapidly acquires a series of 2D ultrasound images spanning the volume of interest, which are recorded by a computer. Because the scanning protocol is predefined and precisely controlled, the relative position and orientation of each 2D image can be known accurately. Using current computer technology, the acquired 2D images may either be stored in their original digital format in the ultrasound system’s computer memory, or the analogue video output from the ultrasound machine may be digitized and stored in an external computer memory. The reconstruction and viewing of the 3D image can then be carried out either in the ultrasound machine’s computer or in an external computer, using the predefined geometric parameters that describe the orientation and position of each 2D image within the 3D image volume. Usually, the angular or spatial interval between successive 2D images is made adjustable, so that it can be optimized to minimize the scanning time while still adequately sampling the volume (Smith and Fenster 2000). Various kinds of scanning apparatus have been developed to translate or rotate the conventional transducer in the required manner. These vary in size from small integrated 3D


R71

Figure 1. Schematic diagrams showing the four mechanical scanning approaches. (a) Linear scanning approach, in which a series of parallel 2D images are collected and used to reconstruct the 3D image. (b) Tilt scanning approach, in which a series of 2D images are collected as the transducer is tilted and then reconstructed into a 3D image. (c) Tilt scanning approach used with a side-firing transrectal (TRUS) transducer to produce 3D images of the prostate. (d) Rotational scanning approach used with a end-firing endocavity transducer in gynaecological and urological imaging.

probes that house the scanning mechanism within the transducer housing, to external fixtures that mechanically hold the housing of a conventional ultrasound probe. Although the integrated 3D probes are larger and heavier than conventional probes, they are usually easy for the operator to use. However, they require the purchase of a special ultrasound machine that can interface with them. On the other hand, while external fixtures are generally bulkier than integrated probes they can be adapted to hold the transducer of any conventional ultrasound machine, obviating the need to purchase a special-purpose 3D ultrasound machine. The image quality and imaging techniques offered by any conventional ultrasound machine can thus also be achieved in 3D. Three basic types of mechanical scanners have been developed, as shown schematically in figure 1: linear, tilt and rotational. 3.1.1. Linear 3D scanning. In this approach a motorized drive mechanism is used to linearly translate the transducer across the patient’s skin, while the 2D images are acquired at regular spatial intervals so that they are uniformly spaced and parallel (figure 1(a)). The ability to vary the translation speed and sampling interval is necessary in order to match the sampling rate to the frame rate of the ultrasound machine and to match the sampling interval to (half) the elevational resolution of the transducer (Smith and Fenster 2000). The simple predefined geometry of the acquired 2D image planes allows for a fast 3D image reconstruction. Thus, using this approach a 3D image can be obtained immediately after performing the scan, and its resolution can be optimized. Because the 3D image is produced from the series of conventional 2D images, its resolution will not be isotropic. In the direction parallel to the acquired 2D image planes, it will equal that of the original 2D images, but in the perpendicular direction it will be equal to the elevational resolution of the transducer. Thus, the 3D image resolution will generally be the worst in the 3D scanning direction, and for optimal results a transducer with good elevational resolution should be used. The linear scanning approach has been successfully implemented in many vascular imaging applications, using B-mode and colour Doppler images of the carotid arteries (Downey

R72

A Fenster et al (a)

(b)

Figure 2. 3D ultrasound images of the carotid arteries of a patient with carotid atherosclerotic disease. The 3D images were obtained using the mechanical linear scanning approach and have been oriented and ‘sliced’ to reveal the details of the plaque. (a) Transverse ‘slice’ revealing the complex plaque. (b) Longitudinal ‘slice’ revealing the extent of the plaque.

and Fenster 1995a, Fenster et al 1995, Guo and Fenster 1996, Dicot et al 1991, 1993, Pretorius and Nelson 1992), tumour vascularization (Bamber et al 1992, Carson et al 1993, Downey and Fenster 1995b, King et al 1991), test phantoms (Guo and Fenster 1996, Guo et al 1995, Dabrowski et al 2001) and power Doppler images (Downey and Fenster 1995a, b, Guo and Fenster 1996). An example of a linearly scanned 3D ultrasound image made with an external fixture is given in figure 2, which shows a 3D image of carotid arteries with plaques. In addition to radiological applications, the utility of linear scanning in echocardiology has also been demonstrated, using pullback of a transesophageal (TE) probe whose imaging plane is horizontal (i.e. perpendicular to the probe axis) (Pandian et al 1992, Ross et al 1993, Wollschlager et al 1993).


R73

3.1.2. Tilt 3D scanning. In this approach, a motorized drive mechanism is used to tilt a conventional transducer about an axis parallel to the transducer face while the 2D images are acquired at regular angular intervals, so that they form a fan of images radial to the axis as shown in figure 1(b). This type of motion is possible with an integrated 3D probe as well as with an external fixture. In either case, the probe housing remains at a single location on the patient’s skin. Because of the fan-like geometry of the acquired 2D images, a large region of interest can be swept out with an angular separation (typically 0.5◦ to 1.0◦ ) that can be adjustably predefined to yield high quality images in a variety of applications (Fenster et al 1995, Delabays et al 1995, Downey et al 1995a, b, Gilja et al 1994, Sohn et al 1992). This approach lends itself to compact designs, both for integrated 3D probes and for external fixtures. Special 3D ultrasound systems with integrated 3D probes, well-suited for abdominal and obstetric imaging, have been successfully demonstrated by Kretztechnik (Zipf, Austria) and Aloka (Korea). This compactness allows for easy hand-held positioning and manipulation. Also, with suitable a total scanning angle, angular interval and ultrasound frame rate, the scanning time can be made short. For example, with a total scanning angle of 90◦ , an angular interval of 1◦ and a 30 Hz acquisition rate, the scanning time is only 3 s. As with linear scanning, the simple predefined geometry of the 2D image planes allows for a fast 3D image reconstruction, which can therefore be performed immediately after the scan is complete. Also, the 3D image resolution will again not be isotropic. Typically, the resolution will degrade away from the axis of rotation, for two reasons. First, because of the fan-like geometry of the acquired 2D images, the linear distance between acquired image planes increases with distance from the axis, resulting in decreased spatial sampling and lower spatial resolution. Second, because of beam spreading in the elevational (or scan) direction, as well as within the acquired image plane, the resolution will degrade with distance from the transducer, which is located near the axis (Blake et al 2000). In a variation of tilt scanning, an external fixture is used to rotate a cylindrical endocavity probe, such as a transoesophageal (TE) or transrectal ultrasound (TRUS) probe, about its axis to perform the scan. The 2D images are acquired by a side-firing linear transducer array parallel to the probe axis, and they again form a fan of images radial to the axis, as shown in figure 1(c), which span a total scanning angle of about 80◦ to 110◦ (Tong et al 1996, 1998). This approach has been successfully applied to prostate imaging (Downey and Fenster 1995a, b, Fenster et al 1995, Tong et al 1996, Elliot et al 1996), as illustrated in figure 3, and to 3D ultrasound guided cryosurgery (Downey et al 1995c, Chin et al 1998, 1999, Onik et al 1996), and it has been commercialized by Life Imaging Systems Inc. (London, Canada). Also, an application of this approach to echocardiography has been demonstrated by TomTec Inc. (Munich, Germany) using a TE probe whose imaging plane is vertical (i.e. parallel to the probe axis) (Belohlavek et al 1993, Martin and Bashein 1989). 3.1.3. Rotational 3D scanning. In this approach, a motorized mechanism rotates the transducer array about a fixed axis that perpendicularly bisects the transducer array through an angle of at least 180◦ , while the 2D images are acquired. Thus, in this scanning geometry, the acquired images sweep out a conical volume about the axis, in a propeller-like pattern, as shown in figure 1(d). This approach has also been implemented with both an integrated 3D probe and an external fixture (e.g. attached to an end-firing TRUS probe). As with the other scanning approaches, the 3D image resolution will not be isotropic. Because the acquired 2D images all intersect along the axis, spatial sampling will be highest near the axis and poorest away from it. Also, both axial and elevational resolution in the acquired 2D images will decrease with distance from the transducer. The combination of these

R74

A Fenster et al (a)

(b)

Figure 3. 3D images of two prostates with cancer. Both images were obtained with the tilt scanning approach using a side-firing transrectal transducer. The image has been ‘sliced’ to reveal a tumour (arrow pointing to dark lesion) and its relationship to the seminal vesicle and rectal wall. Image (b) is courtesy of Life Imaging Systems Inc.

effects will cause the 3D image resolution to vary in a complicated manner, being highest near the transducer and near the axis, and degrading with distance from either. This technique is particularly sensitive to operator or patient motion. Because the acquired 2D images all intersect along the rotational axis at the centre of the 3D image, any motion of the axis or the patient during the scan will cause the axial pixels to have different values in different 2D images. For example, images acquired 180◦ apart will then not be mirror images of each other. These inconsistencies will result in image artefacts or resolution loss on the axis, depending on the method of reconstruction. Similarly, artefacts will occur if the axis of rotation is not accurately known, or if it is not within the 2D image plane of the transducer. This approach has successfully been used for transrectal imaging of the prostate and for endovaginal imaging, as illustrated in figure 4. For imaging the prostate, Downey and others


R75

Figure 4. 3D image of a pregnant uterus with twins. The image has been ‘sliced’ to reveal the two gestational sacs. This 3D image was obtained using the rotational scanning approach using an end-firing endovaginal transducer.

acquired 200 images spanning 200◦ in 13 s (Downey and Fenster 1995b, Tong et al 1996, Elliot et al 1996). Also, rapid imaging of the heart with a multiplane transoesophageal transducer has been demonstrated by other investigators (Ghosh et al 1982, McCann et al 1987, Roelandt et al 1994). 3.2. Free-hand scanning with position sensing The mechanical scanning approaches described above offer short imaging times, high-quality 3D images and fast reconstruction times. However, the bulkiness and weight of the scanning apparatus sometimes make it inconvenient to use, and large structures are difficult to scan. To overcome this problem, free-hand scanning techniques that do not require a motorized fixture have been developed by many investigators. In these approaches, a sensor is attached to the transducer to measure its position and orientation. Thus, the transducer can be held by the operator and be manipulated in the usual manner over the anatomy to be imaged. While the transducer is being manipulated, the acquired 2D images are stored by a computer together with their positions and orientations. This information is then used to reconstruct the 3D image. Since the relative locations of the acquired 2D images are not predefined, the operator must ensure that the spatial sampling is appropriate and that the set of 2D images has no significant gaps. Several free-hand scanning approaches have been developed, making use of four different sensing approaches: articulated arms, acoustic sensing, magnetic field sensing and image-based sensing. 3.2.1. Articulated arm 3D scanning. Position and orientation sensing can be achieved by mounting the ultrasound transducer on a multiple-jointed mechanical arm system.

R76

A Fenster et al

Figure 5. Schematic diagram showing two free-hand scanning approaches: (a) acoustic tracking, (b) magnetic field sensing. The arrangement of the acquired 2D images has no regular geometry. For accurate 3D reconstruction, the position and orientation of each plane must be known accurately.

Potentiometers located at the joints of the movable arms provide the information necessary to calculate the relative position and orientation of the acquired 2D images. This arrangement allows the operator to manipulate the transducer while the computer records the 2D images and the relative angles of all the arms. Using this information, the position and orientation of the transducer is calculated for each acquired 2D image, allowing the 3D image to be reconstructed. To avoid 3D image distortions and inaccuracies, the arms must not flex. Improved performance can be achieved by keeping each arm as short as possible and reducing the number of movable joints. Thus, increased 3D image quality can be achieved by reducing the scanning flexibility and the maximum size of the scanned volume (Geiser et al 1982). 3.2.2. Acoustic free-hand 3D scanning. In this approach, an array of three sound-emitting devices, such as spark gaps, are mounted on the transducer, and an array of fixed microphones are mounted above the patient, as illustrated in figure 5(a). The microphones continuously receive sound pulses from the emitters while the transducer is being manipulated and the 2D images are being acquired. The position and orientation of the transducer for each acquired 2D image can then be determined from knowledge of the speed of sound in air, the measured timesof-flight from each emitter to each microphone and the fixed locations of the microphones. The microphones must be placed over the patient, to provide unobstructed lines of sight to the emitters, and must be close enough that the sound pulses can be detected with a good signal-to-noise ratio (Brinkley et al 1982, 1984, King et al 1990, 1992, Levine et al 1989, Moritz et al 1983, Rivera et al 1994, Weiss et al 1983). 3.2.3. Free-hand 3D scanning with magnetic field sensors. The most popular free-hand scanning approach makes use of a magnetic field sensor with six degrees of freedom. In this approach, a transmitter is used to produce a spatially varying magnetic field and a small receiver containing three orthogonal coils senses the magnetic field strength, as illustrated in figure 5(b). By measuring the strength of three components of the local magnetic field, the position and orientation of the transducer can be calculated each time a 2D image is acquired. Typically, the transmitter is placed beside the patient and the receiver is mounted on the hand-held transducer. Magnetic field sensors are small and unobtrusive, allowing the transducer to be tracked with less constraint than the previously described approaches. However, electromagnetic


R77

(a)

(b)

Figure 6. 3D B-mode image of the carotid arteries obtained with the free-hand scanning approach using a magnetic position and orientation measurement (POM) device. (a) The 3D image has been ‘sliced’ to reveal an atherosclerotic plaque at the entrance of the internal carotid artery. The plaque is heterogeneous, with calcified regions casting a shadow on the common carotid artery. (b) The same image has been volume rendered to obtain a view of the vessel wall and the surface of the plaque.

interference from sources such as CRT monitors, ac power cables and electrical signals from the transducer itself can compromise the tracking accuracy. Geometric distortions in the final 3D image can occur if ferrous or highly conductive metals are located nearby. In addition, errors in the determination of the location of the moving transducer will occur if the magnetic field sampling rate is low and the transducer is moved too quickly. This results in an image ‘lag’ artefact, which can be avoided by using a sampling rate of about 100 Hz or higher. By ensuring that the environment of the scanned volume is free of metals and electrical interference, and using an adequate sampling rate, high-quality 3D images can be obtained, as illustrated in figure 6.

R78

A Fenster et al

Magnetic position and orientation measurement (POM) devices of sufficient quality for 3D ultrasound imaging are currently being produced by two companies: the Fastrack by Polhemus, and Flock-of-Birds by Ascension Technologies. These devices have been used successfully in many diagnostic applications, including echocardiography, obstetrics, and vascular imaging (Fenster et al 1995, Nelson and Elvins 1993, Bonilla-Musoles et al 1995, Detmer et al 1994, Ganapathy and Kaufman 1992, Hodges et al 1994, Hughes et al 1996, Leotta et al 1997, Gilja et al 1997, Nelson and Pretorius 1995, Ohbucki et al 1992, Pretorius and Nelson 1994, Raab et al 1979, Riccabona et al 1995). 3.2.4. 3D tracking by speckle decorrelation. The free-hand scanning techniques described above require a POM device to record the position and orientation of each acquired image. An alternative technique, not requiring any device, uses the acquired images themselves to extract their relative positions. This can be accomplished using the well-known phenomenon of speckle decorrelation. When a source of coherent energy interacts with scatterers, the reflected spatial energy pattern will vary, due to interference, and appear as a speckle pattern. Ultrasound images are characterized by image speckle, which can be used to make velocity images of moving blood (Friemel et al 1998). In this situation if the red blood cells remain stationary then two sequential signals scattered by these cells will be correlated and the speckle pattern will be identical. However, if the blood cells move, the two sequential signals will become decorrelated, with the degree of decorrelation being proportional to the distance moved. The same principle can be used to determine the spacing between two adjacent 2D images. If two images are acquired from the same location, then the speckle pattern will be the same, so that there will be no decorrelation. However, if one of the images is moved with respect to the first, then the degree of decorrelation will be proportional to the distance moved, the exact relationship depending on the beam width in the direction of the motion (Tuthill et al 1998). Speckle decorrelation techniques for 3D ultrasound imaging are complicated by the fact that a pair of adjacent 2D images may not necessarily be parallel to each other. Thus, in order to determine whether the transducer has been tilted or rotated, the acquired images are subdivided into smaller regions, and similar regions in adjacent images are cross-correlated. In this manner, a pattern of decorrelation values is generated, which in turn generates a pattern of distance vectors. These are then analysed to determine the relative position and orientation of the two 2D images. 3.3. Free-hand scanning without position sensing An alternative 3D scanning approach, without any position sensing, involves manipulating the transducer over the patient while acquiring 2D images, and then reconstructing the 3D image by assuming a predefined scanning geometry. Because no geometrical information is recorded during the transducer’s motion, the operator must be careful to move the transducer at a constant linear or angular velocity so that the acquired 2D images are obtained with a regular spacing. With uniform motion and knowledge of the approximate total distance or total angle scanned, very good 3D images may be reconstructed, as illustrated by figure 7 (Downey and Fenster 1995a). However, because this approach offers no guarantee that the 3D images are geometrically accurate, they should not be used for any measurements. 3.4. 2D Arrays for dynamic 3D ultrasound The mechanical and free-hand scanning approaches for 3D ultrasound images described above all require the 2D ultrasound image produced by a conventional transducer, with a 1D array,


R79

Figure 7. Maximum intensity projection (MIP) of a 3D power Doppler image of a kidney. The image was obtained with the free-hand scanning approach without any positioning sensing. The image has been ‘sliced’ to demonstrate that excellent 3D images of vascular structures can be obtained.

to be mechanically or manually swept across the anatomy. However, by using a transducer with a 2D array, the transducer could remain stationary, and electronic scanning could be used to sweep the broad ultrasound beam over the entire volume under examination. Although a number of 2D array designs have been developed, the most advanced is the one at Duke University, which was developed for real-time 3D echocardiography and has been used for clinical imaging (Shattuck et al 1984, Smith et al 1991, 1992, Snyder et al 1986, Turnbull and Foster 1991, von Ramm and Smith 1990, von Ramm et al 1991). In this approach, a 2D phased array of transducer elements is used to transmit a broad beam of ultrasound diverging away from the array, sweeping out a volume shaped like a truncated pyramid. The returned echoes detected by the 2D array are processed to display multiple planes from this volume in real time. These planes can be manipulated interactively to allow the user to explore the whole volume under investigation. Although this approach has been successfully used in echocardiology, some problems must be overcome before its use can become widespread in radiology. These problems are related to high cost of the 2D transducer arrays, which results from the low yield of the manufacturing process, which requires many electronic leads to be properly connected to the numerous small elements in the array. 4. Reconstruction of 3D ultrasound images Image reconstruction refers to the process of generating a 3D representation of the anatomy by first placing the acquired 2D images in their correct relative positions and orientations in the 3D image volume, and then using their pixel values to determine the voxel values in the 3D image. Two distinct reconstruction methods have been used: feature-based and voxel-based reconstruction.

R80

A Fenster et al

4.1. Feature-based reconstruction In this method, desired features or surfaces of anatomical structures are first determined and then reconstructed into a 3D image. For example, in echocardiographic or obstetric imaging, the ventricles or foetal structures may be outlined (either manually or automatically by computer) in the 2D images, and only the resulting boundary surfaces presented to the viewer in 3D. The surfaces of different structures may be assigned different colours and shaded, and some structures may be eliminated to enhance the visibility of others. This approach has been used extensively in 3D echocardiography to identify the surfaces of ventricles (Ofili and Nanda 1994, Nadkarni et al 2000a, Coppini et al 1995, Wang et al 1994). Then, using ventricle surfaces from 3D images obtained at different phases of the cardiac cycle, the complex motion of the heart can be viewed with a computer workstation, as a cine loop. A similar approach has been also used to reconstruct 3D images of the vascular lumen from intravascular ultrasound (IVUS) images (Reid et al 1995). Because this approach reduces the 3D image content to the surfaces of a few anatomical structures, the contrast of these structures can be optimized. Also, by using multiple (e.g. cardiac gated) images, their dynamic behaviour can be easily appreciated. In addition, it also enables the 3D image to be manipulated efficiently using inexpensive computer display hardware (e.g. to produce ‘fly-through’ views). However, this approach also has major disadvantages. Because it represents anatomical structures by simple boundary surfaces, important image information, such as subtle anatomical features and tissue texture, is lost at the initial stage of the reconstruction process. Moreover, the artificial contrast between structures may misrepresent subtle features of the anatomy or pathology. Furthermore, the boundary identification step is tedious and time-consuming if done manually, and subject to errors if done automatically by a computer segmentation algorithm. 4.2. Voxel-based reconstruction The more popular approach to 3D ultrasound image reconstruction is based on using a set of acquired 2D images to build a voxel-based image, i.e. a regular Cartesian grid of volume elements in three dimensions. Reconstruction is accomplished in two steps: first, the acquired images are embedded in the image volume, by placing each image pixel at its correct 3D coordinates (x, y, z), based on the 2D coordinates (x ∗ , y ∗ ) of that pixel in its 2D image, and the position and orientation of that image with respect to the 3D coordinate axes. Then, for each 3D image point, the voxel value (colour or grey-scale image intensity) is calculated by interpolation, as the weighted average of the pixel values of its nearest neighbours among the embedded 2D image pixels. For mechanically scanned images, the interpolation weights may be precomputed and placed in a look-up table, allowing the 3D image to be rapidly reconstructed (Tong et al 1996). This voxel-based reconstruction approach preserves all the information originally present in the acquired 2D images. Thus, by taking suitable cross sections of the 3D image volume, the original 2D images can be recovered. Moreover, new views not found in the original set of images can be generated. However, if the scanning process does not sample the object volume adequately, so that there are gaps between acquired 2D images which are greater than about half the elevational resolution, then the interpolated voxel values will not depict the true anatomy in the gap. In this situation, spurious information is introduced, and the resolution of the reconstructed 3D image is degraded. To avoid this situation, the volume must be well sampled. This generates large data files which require efficient 3D viewing software. Typical image files can range


R81

from 16 MB to 96 MB in size, depending on the application. For example, 96 MB images have been reported in 3D TRUS imaging of the prostate for cryosurgical guidance (Downey et al 1995c, Chin et al 1996). Moreover, because all the original image information is preserved, the 3D image can be processed repeatedly using a variety of rendering techniques. For example, the operator can scan through the data (review the complete 3D image) and then choose the technique which displays the features of interest to best advantage. The operator may then apply segmentation and classification algorithms to segment boundaries, measure volumes or perform various volume-based rendering operations. Then, if the selected process does not achieve the desired results, the operator may return to the original 3D image and try a different procedure. 4.2.1. Freehand 3D scanning. Here it is assumed that the transducer housing is rigidly attached to the ‘sensor’ of a position and orientation measurement (POM) device (such as a magnetic field sensor). This device determines the position and orientation of the ‘sensor’ relative to a ‘source’ in a fixed remote location (such as a magnetic field generator). Suppose now that a 2D image pixel has coordinates (x ∗ , y ∗ ) relative to the transducer, 3D coordinates (x , y , z ) relative to the ‘sensor’, 3D coordinates (x , y , z ) relative to the ‘source’ and 3D voxel coordinates (x, y, z) in the reconstructed image volume. Suppose further that the x ∗ –y ∗ plane would have to be sequentially rotated through the three angles (α, β, γ ) about three (given) axes and translated by the vector (X, Y, Z) in order to coincide with the x –y plane; that the x –y plane would similarly have to be rotated through (α , β , γ ) and translated by (X , Y , Z ) in order to coincide with the x –y plane; and, finally, that the x –y plane would have to be rotated through (α , β , γ ) and translated by (X , Y , Z ) in order to coincide with the x–y plane. Then the embedding transformation from 2D pixel coordinates (x ∗ , y ∗ ) to 3D voxel coordinates (x, y, z) can be written as the product of three homogeneous linear transformations, so that:    ∗ x x ∗ y   y  (1)   = T T T  z 0 1 1 where, in terms of the 3D rotation matrix Rij = rij (α, β, γ )   R11 R12 R13 X R22 R23 Y  R (2) T =  21  R31 R32 R33 Z 0 0 0 1 with similar definitions for T in terms of Rij = rij (α , β , γ ) and T in terms of Rij = rij (α , β , γ ). Because T is defined by the user and T is given by the output of the POM device, only T needs to be determined in order to fully specify the embedding transformation. This is done by calibrating the rotation angles (α, β, γ ) and the translation vector components (X, Y, Z). Methods for doing this are described by Detmer et al (1994) and Prager et al (1998). Using commercially available magnetometer-based POM devices, Prager et al (1998) found a root-mean-square (rms) precision of 1.1–2.2 mm in each 3D coordinate, depending on the calibration method, and a mean accuracy of better than 0.25 mm for each of the methods tested. These are similar to results obtained earlier by Detmer et al (1994) and Leotta et al (1997) for specific calibration methods. Also, the precision and accuracy of various in vitro and in vivo volume measurements has been reported by Hodges et al (1994) and Hughes et al (1996).

R82

A Fenster et al

4.2.2. Mechanical 3D scanning. Here, the transducer housing is rigidly mounted on a mechanical device, often hand-held, which then uses a stepper motor to scan the object by translating, rotating or tilting the transducer according to a predetermined protocol, often under microcomputer control. The embedding transformations for mechanical linear and tilt scanners are described in connection with the next section. 5. Effects of errors in the reconstruction Except for systems using 2D arrays, 3D ultrasound images are reconstructed from multiple 2D images, using knowledge of their relative positions and orientations. Thus, any errors in the position or orientation of the 2D images will cause geometric distortion in the reconstructed 3D image, resulting in errors in the measured lengths, areas and volumes of anatomical features in these images. Although other sources of error such as tissue motion during the scan (possibly caused by the scanning procedure itself) or image segmentation errors can also affect these measurements, they overlie this fundamental geometric image distortion. In the literature, analyses of these geometric distortions have been performed for linear and tilt mechanical scanners. 5.1. Linear 3D scanning The analysis was performed by Cardinal et al (2000) for the general case of an ultrasound transducer oriented as shown in figure 8 and scanned parallel to the z-axis. Here, the transducer array is initially oriented along the x-axis, and aimed in the y direction. After being tilted by an angle θ about the x-axis, and then swiveled by an angle φ about the y-axis, it is translated in the z direction, in steps of size d, to acquire a series of parallel 2D images. Thus, for the 2D image acquired after n steps, the pixel at coordinates (x ∗ , y ∗ ) has 3D Cartesian coordinates (x, y, z) given by the linear transformation ∗ x cos φ sin φ sin θ 0 x (3) y∗ . y = 0 cos θ 0 n z − sin φ cos φ sin θ d Moreover, the embedded 2D image planes are all normal to the unit vector U = (sin φ cos θ, − sin θ, cos φ cos θ)

(4)

and have a normal spacing D = d cos φ cos θ > 0

(5) ◦

assuming (without loss of generality) that θ and φ are each less than π/2 (90 ) in magnitude. When φ = 0, the embedded 2D image planes are all parallel to the x-axis, and hence the 3D image interpolation reduces to a 2D interpolation in the y–z plane, which is identical for each value of x. Thus, when φ = 0, the 3D image may be efficiently reconstructed by computing a single set of 2D interpolation coefficients and applying it to each x plane in succession. The error analysis assumes that the 3D image is reconstructed with the nominal parameter values (d, θ, φ), but that the 3D object was scanned with the actual parameter values (d0 , θ0 , φ0 ). Because the embedding transformation is linear, straight lines are always imaged as straight lines. However, the errors d = d − d0 , θ = θ − θ0 and φ = φ − φ0 still cause relative errors L/L, A/A and V /V in image length, area and volume. If the error vector E is now defined as:

θ d −φ , , (6) E= cos φ cos θ cos θ d cos φ cos θ


R83

Figure 8. Schematic diagram showing the scanning geometry for linear scanning with a conventional transducer. The transducer array is initially located on the object’s surface, along the x-axis, and aimed perpendicularly into the object, in the y direction. The transducer is then tilted by an angle θ about the x-axis and subsequently swiveled by an angle φ about the y-axis, before being translated across the object’s surface in the z direction (in steps of size d, not shown) to acquire a series of parallel 2D images.

then, assuming that |d/d| 1, |θ | 1 and |φ| 1, it is found that all three relative errors are well described by the linear approximations K K = L, A, V (7) = E · CK K where L × (U × L) (U · L)L CL = U − = (8a) L2 L2 which is the component of U parallel to the image line vector L A × (U × A ) ( U · A )A CA = U − = (8b) 2 A A2 which is the component of U perpendicular to the image area vector A, and CV = U .

Hence, because |CK | |U | = 1, within these linear approximations K K = L, A, V . K |CK ||E | |E |

(8c)

(9)

For volume, the linear approximation is explicitly V d = − tan θ θ − tan φφ (10) V d which is insensitive to φ when φ = 0. Also, the exact ratio of image volume V to object volume V0 is given by the formula V d cos φ cos θ D = = (11) d0 cos φ0 cos θ0 D0 V0 because the determinant of the embedding linear transformation is D.

R84

A Fenster et al

Figure 9. Schematic diagram illustrating the scanning geometry for tilt scanning with a side-firing transrectal ultrasound (TRUS) probe. The acquired 2D images form a radial fan about the probe axis, with an inner radius R0 , and spanning a total scan angle .

For the case φ0 ≈ φ = 0, θ0 = 32.7◦ , the exact volume error formula was verified experimentally, using phantom images, within an experimental uncertainty of less than 0.5%. Also, it was verified numerically that all three linear approximations are perfectly adequate for parameter errors of any likely magnitude (Cardinal et al 2000). 5.2. Tilt 3D scanning The analysis was performed by Tong et al (1998) for the case of a side-firing TRUS probe rotated about its axis, as shown in figure 9. Here, the scanned images comprise a radial fan of 2D images, each of size (X, Y ), whose inner edges all lie at a distance R0 from the axis, which forms the z-axis. By convention, the X and Y directions are respectively chosen to be parallel (lateral) and perpendicular (axial) to the probe axis. Thus, for the 2D image at angle θ, the pixel at (x ∗ , y ∗ ) has 3D cylindrical coordinates (r, θ, z) = (R0 + y ∗ , θ, x ∗ ) and hence 3D Cartesian coordinates (x, y, z) = (r cos θ, r sin θ, z). Hence, in this case, the 3D interpolation reduces to a 2D interpolation in the x–y plane, which is identical for each value of z. The 3D image may therefore be efficiently reconstructed by computing a single set of 2D interpolation coefficients and applying it to each z plane in succession. The error analysis assumes that the angular interval between acquired 2D images is uniform, and proportional to the total scan angle (Tong et al 1998). However, it applies equally to the case where the angular interval is a variable number of steps, provided the step size remains proportional to . Errors R0 in R0 and in will cause relative errors L/L, A/A and V /V in image length, area and volume, as well as a transverse deviation h from linearity in the image of a line of length L (defined so that h vanishes at the image endpoints), due to the nonlinearity of the embedding transformation. It is found that all three relative errors, as well as h/L, vary closely as

K R0 =P +Q K = L, A, V (12) K R where |P | 1, |Q| 1 and R is the average radius of the object from the axis, which is usually well approximated by the radius of the object’s centroid. Hence, within these linear approximations K R0 + K = L, A, V . (13) K R


R85

For line images, the coefficients P and Q for L/L and h/L are simple functions of the line’s orientation and size, and the relative location of the point at which h is to be calculated. For area and volume, P = Q = 1, and the exact ratio of image size K to object size K0 is given by the formula

K R0 1+ K = A, V . (14) = 1+ K0 R Moreover, this formula applies to cross-sectional areas in any orientation. Tong et al (1998) also describe methods for using simple phantoms to calibrate R0 and accurately. Moreover, the linear length, area and volume formulae were verified numerically, using simulated images, and experimentally, using phantom images (and the calibrated values of R0 and ), within an experimental uncertainty of less than 0.5%. 6. 3D ultrasound image display Over the past decade, many algorithms and software utilities have been developed to manipulate 3D images interactively using a variety of visualization tools (Udupa 1999, Udupa et al 1991, Robb 1995). Although the quality of the 3D image depends critically on the method of image acquisition and the fidelity of the 3D image reconstruction, the viewing technique used to display the 3D image often plays a dominant role in determining the information transmitted to the operator. There are many approaches for displaying 3D images, which differ in the details of the display implementation. Many of these techniques have been investigated for use with 3D ultrasound images, and these can be divided into three broad classes: surface rendering (SR), multiplanar reformatting (MPR) and volume rendering (VR). 6.1. Surface rendering (SR) A common 3D display technique used in medical imaging is based on visualizing the surfaces of organs. Because the 3D image must first be reduced to a description of surfaces, image classification and segmentation steps precede the rendering. In the first step, each voxel (or voxel group) in the 3D image must be either manually or automatically classified as to which structure it (or they) belongs (Bezdek et al 1993). Organ boundaries are then identified using manual contouring (Coppini et al 1995, Neveu et al 1994, Lobregt and Viergever 1995), or computer-based segmentation techniques (Gill et al 2000). Once the organs have been classified and segmented, the boundary surfaces are each represented by a mesh and then texture mapped with a colour and texture which appropriately represents the corresponding anatomical structure. Because early 3D ultrasound imaging techniques did not have the benefit of the high computational speed now available in inexpensive computers, only simple wire-frame rendering techniques were used at first. These early approaches were used for displaying the foetus (Brinkley et al 1984, Sohn et al 1989, Sohn and Rudofsky 1989, Brinkley et al 1982), various abdominal structures (Sohn and Grotepass 1989, Sohn et al 1988, Sohn 1989), and the endocardial and epicardial surfaces of the heart (Moritz et al 1983, Coppini et al 1995, Fine et al 1988, Nixon et al 1983, Martin et al 1990, Linker et al 1986, Sawada et al 1983). More recent approaches use more complex surface representations, which are texturemapped, shaded and illuminated, so that both the surface topography and its 3D geometry are more easily comprehended. These approaches typically allow user-controlled motion for viewing the anatomy from various perspectives. This approach has been used successfully by many investigators in the rendering of 3D echocardiographic (Greenleaf et al 1993, Ofili

R86

A Fenster et al

and Nanda 1994, Ross et al 1993, Delabays et al 1995, Wang et al 1994, McCann et al 1998, Belohlavek et al 1994, Delabays et al 1995) and obstetric images (Nelson and Pretorius 1992, Lees 1992). 6.2. Multiplanar reformatting (MPR) Rather than displaying only the surfaces of structures, as in the SR technique, the MPR technique provides the viewer with planar cross-sectional images extracted from the 3D image. Thus, this approach requires that either a voxel-based 3D image be first reconstructed, or that an algorithm be used that can extract an arbitrarily oriented plane from the original set of acquired 2D images (Robb 1995). Because the extracted plane will generally not coincide with one of these original image planes, interpolation is required to provide an image similar in appearance to an acquired 2D image. Thus, the extracted image resembles a conventional 2D ultrasound image, which is already familiar to the operator; the operator can then easily position and orient it to the optimal image plane for the examination. This technique is easy to learn and is therefore preferred by radiologists. Two MPR techniques have been used to view 3D ultrasound images: orthogonal planes and cube-view. 6.2.1. Orthogonal planes. In this approach, computer–user interface tools provide three perpendicular planes which are displayed on the screen simultaneously, with graphical cues indicating their relative orientations. The operator can then select any single or multiple planes and move them within the 3D image volume to provide a cross-sectional view at any desired location or orientation, including oblique views (Nelson and Pretorius 1992, Zosmer et al 1996, Gerscovich et al 1994, Kuo et al 1992, Kirbach and Whttingham 1994). 6.2.2. Cube-view. In this approach, illustrated by figures 2, 3, 4, 6(a) and 10(a), the 3D image is presented as a polyhedron which represents the boundaries of the reconstructed volume. Each face of the multifaceted polyhedron is rendered with the appropriate ultrasound image for that plane, using a texture-mapping technique (Fenster et al 1995, Tong et al 1996, Robb 1995). With the computer–user interface tools provided, the operator can select any face, and move it in or out, parallel to the original face or reorient it obliquely, at any angle to the original face, while the appropriate ultrasound data are continuously texture-mapped in real time onto the new face. In addition, the entire polyhedron may be arbitrarily rotated in any direction to obtain the desired orientation of the 3D image. In this way, the operator always has 3D image-based cues relating the plane being manipulated to the anatomy being viewed (Rankin et al 1993, Fenster et al 1995, Picot et al 1993, Blake et al 2000). 6.3. Volume rendering (VR) The volume rendering technique presents a display of the entire 3D image after it has been projected onto a 2D plane, with the image projection typically being accomplished via raycasting techniques (Tuy and Tuy 1984, Levoy 1990a, b). The 3D image voxels that intersect each ray are weighted and summed to achieve the desired result in the rendered image. Although many VR algorithms have been developed, 3D ultrasound imaging currently makes use of two main approaches: maximum (minimum) intensity projection and translucency rendering. 6.3.1. Maximum (minimum) intensity projection (MIP). A common approach is to display only the maximum (minimum) voxel intensity along each ray. This approach is easy to


R87

Figure 10. 3D image of a foetal face, displayed using (a) the multiplanar reformatting cube-view approach, and (b) the translucency rendering approach, done with ray-casting. In (a) the 3D image is presented as a polyhedron with the appropriate 2D image texture-mapped onto each face. This polyhedron can then be arbitrarily ‘sliced’ and rotated to reveal the profile of the foetal face. In (b), the amniotic fluid has been rendered as transparent, and tissues have been rendered as translucent to opaque, as a function of their voxel intensity.

implement and fast to compute, allowing real-time manipulation of the MIP image on inexpensive computers. Excellent results are typically achieved when the image information is sparse, such as it is in 3D power Doppler images (see figure 7) (Guo and Fenster 1996, Downey and Fenster 1995b). 6.3.2. Translucency rendering. This is the VR approach most commonly used in 3D ultrasound imaging. In this approach, the accumulated luminance C for a given ray is calculated by tracing the ray through the 3D image and adding the contribution from each voxel along the ray. Tracing the ray forwards, towards the projected image plane, the ray-tracing equation is C = C[1 − α(i)] + c(i)α(i)

(15)

where C and C are respectively the accumulated luminance not including and including the contribution from the ith voxel, α(i) is the opacity (1 minus the transmittance) of the ith voxel, and c(i) is its luminance. The parameters α(i) and c(i) are generally functions of the ith voxel intensity, which are chosen to control the specific type of rendered image desired. Thus, if α(i) = 0, the ith voxel is transparent, while if α(i) = 1, then it is opaque or luminescent, depending on the value of c(i). However, the contribution of voxels far from the projected image plane is typically negligible, due to the high total opacity (low total transmittance) of the intervening voxels. Thus, it is often preferred to trace the ray backwards, away from the projected image plane. In this case, the ray-tracing equation becomes C = C + c(i)α(i)[1 − A(i)]

(16)

R88

A Fenster et al

where C and C are respectively the accumulated luminance not including and including the contribution from the ith voxel (now numbered in reverse order), and A(i) is the accumulated opacity at the ith voxel, given by [1 − A(i)] = [1 − α(j )] (17) j