Image Processing and Pre-Processing for Medical U1trasound Frederic L. Lizzi, Eng.Sc.D. and Ernest J. Feleppa, Ph.D. Riverside Research Institute, New York, NY
[email protected] [email protected] Abstract
investigations aimed at augmenting the clinical information conveyed by ultrasound. The report addresses these investigations in terms of three general topics: first, techniques designed to enhance image features such as resolution and tissue contrast; second, techniques designed to indicate tissue type and status (e.g., following radiotherapy); and third, techniques designed for interactive 3-D imaging and biometry. Our techniques €or image enhancement start with frequency-domain pre-processing of digitally acquired radio-frequency (RF) echo signals. These RF signals are captured at the transducer before the application of conventional signal processing, including envelope detection, employed in standard “B-mode” ultrasound imaging. Our tissue-typing investigations employ calibrated spectrum analysis of acquired RF signals. Clinical databases of previously measured spectra are used to classify tissues, and tissue type is indicated by color-coded image overlays. Our 3-D tissue assays integrate images obtained from the above techniques; they are designed to clarify anatomic relationships and to compute 3-D biometric data (e.g., surface areas and volume) for selected tissue structures.
Much attention is now being focused on techniques to improve the quality and information content of ultrasonic images of the body. Many of these techniques employ digital pre-processing of coherent echo signals prior to image generation. Examples of these procedures include: resolution enhancement; contrast enhancement (using ji-equency-domain techniques) to suppress speckle; and imaging of spectral parameters (which sense the sizes and concentrations of sub-resolution tissue constituents). Combinations of spectral parameters and ancillary clinical data (e.g., PSA blood levels) are also being used with statistical classifiers (e.g., discriminant analysis neural network) to generate color-coded, images that indicate tissue type (e.g., cancer) or tissue regions responding to therapy. Sets of these images, obtained from serial-plane scans, promise to be particularly useful when presented in interactive three-dimensional (3-0) formats.
1. Introduction Visualization with pulse-echo ultrasound has become a well-established imaging technique in medical practice. Most modern systems employ versatile array transducers and digital processing to derive high-quality crosssectional images for use in obstetrics, cardiology, and other medical specialties. A variety of specialized systems have been developed using, e.g., higher frequencies (for improved resolution), and miniaturized probes (for intracavitary and intravascular procedures). While ultrasonic imaging has improved dramatically over the past decade, emerging techniques promise further advances. Among these are ultrasonic contrast agents (for high-precision blood-flow assays) [ 11, harmonic imaging (exploiting non-linear propagation phenomena for improved image quality) [2], and elastography [3] (for remotely assessing tissue stiffness by applying external displacement and characterizing induced internal strains). This report presents a brief summary of some of our
2. Methods and illustrative results This report presents illustrative clinical results obtained from ocular and prostate examinations. The ocular examinations employed a digital ultrasonic scanner developed in conjunction with the Weill Medical College of Cornel1 University [4]. The computer-controlled system scans a PVDF 40-MHz transducer across the eye of a supine patient, using a small saline bath for acoustic coupling. RF data are typically acquired (250 MHz sampling) along 128 scan lines in a set of 20 - 80 parallel planes (0.2 - 0.05 mm separation, respectively). Prostate examinations [5] employed a B&K (North Billerica, MA) transrectal scanner that incorporates a 7MHz sector-scanned transducer. We have interfaced this
187 0-7695-0978-9/00$10.00 0 2000 IEEE
improve axial resolution by a factor near two.
system with our synchronous acquisition module that is used to acquire RF data (50-MHz sampling) from entire scans of the prostate immediately before a biopsy needle is fired at a known, imaged site. Data are also acquired from sequential scan planes as the probe is manually translated along a linear path orthogonal to the scan planes. This research is being conducted in collaboration with several medical institutions; results presented below were obtained with the Memorial Sloan-Kettering Cancer Center. In both ocular and prostate examinations, RF data from single entire planes are acquired at real-time rates (0.3 s / plane) and transferred to Pentium computers for subsequent processing and display, using a variety of software packages developed in our laboratories.
2.1. Image enhancement Two of the most important features of an ultrasonic image are its spatial resolution and gray-scale contrast. In some applications, such as corneal biometry, resolution is of most importance. In many applications, the more pressing need is to obtain a high degree of gray-scale image-contrast among different tissue structures. Our approaches for both resolution and contrast enhancement involve RF-signal pre-processing, prior to image formation. Figure 1 shows segments of RF echo data obtained from adjacent scan lines in the prostate; the scan lines were positioned within the region-of-interest (ROI) demarcated in the upper B-mode image. Figure 2 shows the average power spectrum of these signals; it also displays the spectrum of echoes from a planar calibration
plate placed in the transducer’s focal plane. The calibration spectrum defines the transfer function of the transducer and associated pulser / receiver electronics. [6] As seen in the figure, these system artifacts exert a significant influence on the observed tissue spectrum. To enhance axial (thickness) resolution, we apply an inverse-filtering technique. We first divide the complex spectrum of tissue data from each scan line by the complex calibration spectrum. As seen in the normalized spectrum of Fig. 2, this “pre-whitening” operation broadens the signal spectrum (e.g., at the -3 dB points) by removing the constraining influence of the system. The whitened spectrum is then multiplied by a digital passband filter function that passes in-band components (2 - 8 MHz in Fig. 3) and suppresses out-of-band noise. An inverse Fourier transform is then performed, and the analytic-signal magnitude (ASM) is computed to derive unipolar video signals along each scan line. Because of their augmented bandwidth, the processed signals from each scatterer are shorter and, therefore, more-closely spaced tissue elements can be resolved. These signals are then displayed in standard B-mode image formats. In practice, we have found that this procedure can typically
I
1
2
0
ti rne “(p)
6
8
Figure 1
-
RF Spectrum
90 -,
- - -Glass Plate Spectrum
0 - \
80 -
1
\
\
/
3 Q
50
E
Q 40
’”ICJ 20
0
, 5
Frequency (MHz)
Figure 2 188
I
10
15
-Normalized Spectrum Linear Fit of spectrum
--
2
3
4
5
6
7
8
Figure 4b
Frequency (MHz)
Contrast enhancement is achieved by suppressing speckle -- the modulation of image brightness by random dark and bright regions. Speckle [8] arises because of the superposition of RF echo signals from many unresolved tissue scatterers; these can interfere in constructive or destructive fashions, affecting video signal levels. Our technique uses the fact that images formed in different frequency bands exhibit statistically independent speckle, because the RF interference patterns vary with frequency. Our contrast enhancement procedure starts with the computation of calibrated RF spectra, as described for resolution enhancement. It then divides these spectra into N (typically three) partly overlapping frequency bands using Hamming filter functions. [9] It then computes corresponding video images (via inverse transforms and ASM), equalizes their mean brightness levels, and averages them to obtain the final image. The resultant ratio of signal (mean brightness) to speckle (brightness standard deviation) is expected to improve by dN. This process involves a reduction in axial resolution (since each filter’s bandwidth is reduced) but, as for resolution enhancement, pre-whitening expands the usable bandwidth. Overall, when three filters are employed, axial resolution is usually reduced by 1.5 with respect to its original value, while contrast is enhanced by a factor of 1.7. Figure 5 shows an example of speckle suppression applied to a 40-MHz scan of a traumatically injured eye. The images display anterior ocular structures labelled in the final image, shown in Fig. 6, where hyphema refers to blood in the anterior chamber near the tom iris. The original image (Fig. 5, left) has significant speckle that is especially noticeable in the upper, bright sclera. The grayscale in the processed image (Figs. 5, right; enlarged in Fig. 6) has improved stability and smoother gray-scale variations. This image was processed using three filters whose -6 dB pass-bands were 15 - 25 MHz, 25 - 35 MHz, and 35 - 50 MHz, respectively. Histogram measurements in the hyphema region showed that speckle was reduced by a factor of 1.5, close to the theoretical value of 1.7.
Figure 3 This resolution-enhancement technique is especially valuable in ophthalmology for visualizing thin layers within the cornea. As an example, Fig. 4a shows the RF echo signals obtained from the central region of the cornea. The two large echoes clearly delineate the anterior and posterior boundaries of the cornea, which is typically 550 pm thick. Close examination of the first echo shows that is consists of two superimposed components, arising from the anterior and posterior boundaries of the corneal epithelium, a cellular layer of 50-pm thickness. These two boundaries are too closely spaced to be resolved by the
Figure 4a Figure 4b shows an analogous case in which resolution has been enhanced using the inverse-filtering technique. The processed signal is labelled with measured depths starting at the outer epithelial boundary. Note the clear resolution of the epithelium, whose thickness was 49 pn. The overall cornea thickness was 586 pm. This technique is now being employed to construct comeal-layer thickness maps for planning and evaluating excimer-laser corneal resculpting. [7] Using repeated measurements, we have found a precision near 1 pm can be achieved.
189
Third, calibrated spectra are expressed in dB and analyzed with linear regression techniques to derive: the spectral intercept (linear extrapolation to zero frequency), spectral slope, and midband fit (regression value at the center frequency). Fourth, basic spectral parameter images are formed by imaging local values of the above spectral parameters along each scan line. Compensation for rangedependent attenuation is employed, when needed, in slope and midband fit images using assumed or measured attenuation coefficients. (Intercept values are not affected by attenuation.) [ 111 Tissue-type images are generated using diseasespecific database information to analyze the above spectral-parameter images. The databases are compiled in clinical studies in which spectral parameters are measured and collated with subsequent biopsy findings that definitively identify tissue type (e.g., normal prostate, prostate cancer, etc.). Extensive databases have been constructed for the eye [lo] and prostate [13], and are being compiled for other applications (e.g., breast cancer [ 141, vascular plaque [ 151, and lymph nodes [16]). The database for each organ defines specific conjoint ranges of spectral-parameter values that are associated with different tissue types. Our clinical investigations of the prostate have shown that prostate cancer (identified with subsequent biopsies) can be differentiated from other prostate tissue based on spectral intercept, midband fit, and prostate specific antigen (PSA) levels. [5,12,13] Receiver operator characteristics (ROC) results showed that neural networks or nearest-neighbor analysis provide the most effective means of detecting cancer, yielding ROC areas near 0.8. Figure 7 shows a tissue-type image of the prostate; here, a neural net classifier was used to analyze local values of intercept and midband fit together with the patient’s PSA level. The neural net was trained using database results; its output provides a level-of-suspicion (LOS) for the presence of cancer. Regions whose LOS exceeds a preselected value are displayed in corresponding “flat” shades of gray (red in the original). RRI and Spectrasonics Imaging, Inc. (Wayne, PA) are now collaborating to investigate whether real-time displays of such images can improve biopsy guidance. This technique could significantly decrease the number of cancers that are now missed because conventional ultrasound does not provide the distinctive images of cancer needed for precise biopsy placement. Similar approaches, using linear discriminant classifiers, have been employed to image ocular tumors [10,111, using color-coded “stains” to indicate tumor type (e.g., malignant melanomas and metastatic carcinomas). The status of ocular tumors treated with radiation or ultrasonic hyperthermia have been followed using a similar technique. [I 11 This technique is based on initial experiments which showed that spectral parameters are
Figure 5
Figure 6
2.2. Tissue-type and tissue-status images Images of tissue-type and status are generated using RF spectrum analysis which sense the sizes, concentrations and mechanical properties of tissue scatterers. [ 10,111 A set of derived spectral parameters are compared to diseaseindexed data files (to determine tissue type) or to previous results for the same patient (to evaluate tissue status). Tissue-type images are obtained using a sequence of procedures. [lo-121 First, acquired RF data along each scan line are multiplied by a sliding Hamming gatefunction and undergo Fourier transformation to obtain local tissue spectra as a function of position. Second, the resultant spectra are squared and divided by the calibration spectrum (Fig. 2) to obtain calibrated power spectra.
190
altered by tissue changes induced with radiotherapy or ultrasonic hyperthermia.[ 171 Images of tissue status are constructed by first determining the range of pre-treatment spectral parameters within a tumor. After treatment, the observations are repeated, and color-coding is used to indicate tissue regions whose properties lie outside the pre-treatment range.
Figure 8 In addition to diagnostic applications, we are also employing these 3-D mappings to optimize techniques for non-invasive tumor treatments using intense, focused ultrasound. [20]
3. Summary Figure 7
Modem digital facilities are expediting advanced processing of ultrasonic signals and broadening the options available for tissue imaging and analysis. The results presented in this report show how pre-processing of RF echo signals can enhance the spatial resolution and tissue contrast of ultrasonic images, providing clinicians with a selection of techniques to be applied in specific types of examinations. Spectrum-analysis procedures are increasing the information content of ultrasonic images. They provide a means of using stored database information to convey tissue type (for diagnosis) and using previous patient data to convey tissue status information (for treatment monitoring). The potential utility of these imaging procedures is augmented by the synthesis of interactive 3-D displays. These can be employed to clarify anatomical relationships and delineate the 3-D extent of tumors and disease processes. In addition, they provide a means of obtaining 3-D biometric descriptions, such as volume, of selected tissue regions for use in planning and monitoring therapy regimens. The techniques described in this report can be used to complement other new procedures, including harmonic imaging, elastography, and contrast agents, that are rapidly evolving. Ultimately, a combination of such techniques and those treated in this report may provide truly
2.3. Three-dimensional tissue assays Many, if not most, normal and diseased tissues comprise 3-D structures that are not totally characterized in single cross-sectional images. To better evaluate tissue anatomy, we are investigating means of integrating crosssectional images in interactive 3-D displays and including software tools for comprehensive biometric studies. [ 18,191 The cross-sectional images that are combined include conventional B-mode images as well as the feature-enhanced and tissue-type images described above. Figure 8 shows an example of a 3-D isometric presentation of the eye, clearly demonstrating the cornea, sclera, and iris. This “biopsied” view shows the eye after the removal of a tissue segment passing through a ciliarybody malignant melanoma. The figure shows how the excised specimen can be “extracted”; it can also be rotated for a clearer view of the removed tissue. [19] In these images, spectral parameters (derived from slope and midband fit) were used to display the tumor, while Bmode values were used to represent other tissue segments. (Subsequent post-radiation images from this tumor showed 3 - 6 dB spectral changes within the tumor.) Volumetric assessments [19] of the tumor were also performed to document its subsequent growth or
191
[lo] E.J. Feleppa et al., “Diagnostic spectrum analysis in ophthalmology: A physical perspective,“ Ultrasound Med. Biol.,vol. 12(8), 1986, pp. 623-631.
comprehensive 3-D mappings of sets of complementary tissue properties. Such mappings have a substantial potential for advancing medical care and improving fundamental knowledge of disease processes.
[ l l ] F.L. Lizzi et al., “Ultrasonic spectrum analysis for tissue assays and therapy evaluation,” lilt. J. Imag. Sys. Tech., vol. 8, 1997, pp. 3-10.
4. Acknowledgments
[12] E.J. Feleppa, T. Liu, A. Kalisz, M.C. Shao, W.R. Fair, N. Fleshner, and V. Reuter, “Ultrasonic spectral-parameter imaging of the prostate,” Int. J. h a g . Svs. Tech., vol. 8, 1997, pp. 11-25.
This work has been supported in part by NIH Grants EY01212 and CA53561. We also wish to acknowledge the invaluable contributions of Drs. Coleman and Silverman of the Weill Medical College of Cornel1 University and Dr. Fair of the Memorial Sloan-Kettering Cancer Center.
[13] E.J. Feleppa, W.R. Fair, T. Liu, W. Larchian, H. Tsai, ic and D.J. Coleman, “High-resolution, three-dimensional visualization and morphology assays of the in-vivo ciliary body,” in Proceedings of the 2000 IEEE Ultrasonics Svmposiurn, (in press).
5. References [ l ] B.B. Goldberg, J. Liu, and F. Fosberg, “Ultrasound contrast agents: a review,” Ultrasound Med. Biol., vol. 20, 1994,pp. 319-333.
[14] S.K. Alam, F.L. Lizzi, E.J. Feleppa, T. Liu, and A. Kalisz, “Ultrasonic multifeature analysis procedures for breast lesion classification,”in Medical Imaging 2000: Ultrasonic Imaging and Signal Processing, Society of Photo-Optical Instrumentation Engineers, K.K. Shung and M. Insana (Eds.), vol. 3982, pp. 196-201,2000, Bellingham, WA.
[2] Y. Li and J.A. Zagzebski, “Computer model for harmonic ultrasound imaging,” IEEE Trans. Ultrason., Ferroelec., Freq. Contr., vol. 47(5), 2000, pp. 1259-1272. [3] J. Ophir, S.K. Alam, B. Garra, F. Kallel, E. Konofagou, T. Krouskop, and T. Varghese, “Elastography: ultrasonic estimation and imaging of the elastic properties of tissues,” Proceedings of the Institution of Mechanical Engineers, Part H: Journal of Engineering in Mediciiie, vol. 213(3), 1999, pp. 203-233.
[15] T. Noritomi, B. Sigel, V. Gahtan, V. Swami, J. Justin, E.J. Feleppa, and K. Shirouzu, “In vivo detection of carotid plaque thrombus by ultrasonic tissue characterization,” J. Ultrasound Med., vol. 16(2), 1997, pp. 107-111. [16] T. Tateishi, J. Machi, E.J. Feleppa, R.H. Oishi, J. Jucha, E. Yanagihara, L.J. McCarthy, T. Noritomi, and K. Shirouzu, “In vitro diagnosis of axillary lymph node metastases in breast cancer by spectrum analysis of radio frequency echo signals,” Ultrasound Med. Biol., vol. 24(8), 1998, pp. 11511159.
[4] F.L. Lizzi, M.C. Rorke, J.B. Sokil-Melgar, A. Kalisz, and J. Driller, “Interfacing very-high-frequency transducers to digital-acquisition scanning systems,“ Proc. of the Society for Photo-Optical Instrumentation Engineering (SPIE), vol. 1844, 1992, pp. 313-321.
[17] R.H. Silverman, D.J. Coleman, F.L. Lizzi, J.H. Torpey, J. Driller, T. Iwamoto, S.E.P. Burgess, and A. Rosado, “Ultrasonic tissue characterization and histopathology in tumor xenografts following ultrasonically induced hyperthermia,“ Ultrasound Med. Biol.,vol 12(8), 1986, pp. 639-645.
[5] E.J. Feleppa et al., “Typing of prostate tissue by ultrasonic spectrum analysis,” IEEE Trans. Ultrason., Ferroelect., Freq. Contr., vol. 43(4), 1996, pp. 609-619. [6] F.L. Lizzi, M. Greenebaum, E.J. Feleppa, M. Elbaum, and D.J. Coleman, “Theoretical framework for spectrum analysis in ultrasonic tissue characterization,“J. Acous. Soc. Am., vol. 73(4), 1983, pp. 1366-1373.
[ 181 F.L. Lizzi et al., “Three-dimensional biometric images of
ocular structures using very-high-frequency ultrasound,” in Proceedings of the 1999 Ultrasonics Symposium, Institute of Electrical and Electronics Engineers, Piscataway, 2000, pp. 1581-1583.
[7] R.H. Silverman, D.Z. Reinstein, T. Raevsky, and D.J.
Coleman, “Improved system for sonographic imaging and biometry of the cornea,” J. Ultrasound Med., vol. 16, 1997, pp. 117-124.
[ 191 F.L. Lizzi, E.J. Feleppa, A. Kalisz, R.H. Silverman, and D.J.
Coleman, “High-resolution, three-dimensional visualization and morphology assays of the in-vivo ciliary body,” in Proceedings of the 2000 IEEE Ultrasonics Symposium, (in press).
[8] J.G. Abbott and F.L. Thurstone, “Acoustic speckle: theory and experimental analysis,” Ultrason. Imag., vol. 1, 1979, pp. 303-324. [9] Lizzi, F.L., Feleppa, E.J., Coleman, D.J., “Ultrasonicocular tissue characterization,“ in Characterization of Tissue with Ultrasound, J.F. Greenleaf (Ed.), CRC Press, Boca Raton, pp. 41-60, 1986.
[20] F.L. Lizzi et al., “Ocular tumor treatments with focused ultrasound: effects of beam geometry, tissue morphology, and adjacent tissues,” Proceedings of the 2000 IEEE Ultrasonics Symposium, (in press).
192
