established. These parameters were then combined to give detective quantum efficiency (DQE(u)) and used in conjunction with signal detection theory to.
INSTITUTE OF PHYSICS PUBLISHING
PHYSICS IN MEDICINE AND BIOLOGY
Phys. Med. Biol. 46 (2001) 1631–1649
www.iop.org/Journals/pb
PII: S0031-9155(01)21836-3
The practical application of signal detection theory to image quality assessment in x-ray image intensifier–TV fluoroscopy N W Marshall Regional Medical Physics Department, Newcastle General Hospital, Westgate Road, Newcastle upon Tyne NE4 6BE, UK
Received 12 February 2001 Abstract This paper applies a published version of signal detection theory to x-ray image intensifier fluoroscopy data and compares the results with more conventional subjective image quality measures. An eight-bit digital framestore was used to acquire temporally contiguous frames of fluoroscopy data from which the modulation transfer function (MTF(u)) and noise power spectrum were established. These parameters were then combined to give detective quantum efficiency (DQE(u)) and used in conjunction with signal detection theory to calculate contrast-detail performance. DQE(u) was found to lie between 0.1 and 0.5 for a range of fluoroscopy systems. Two separate image quality experiments were then performed in order to assess the correspondence between the objective and subjective methods. First, image quality for a given fluoroscopy system was studied as a function of doserate using objective parameters and a standard subjective contrast-detail method. Following this, the two approaches were used to assess three different fluoroscopy units. Agreement between objective and subjective methods was good; doserate changes were modelled correctly while both methods ranked the three systems consistently.
1. Introduction Subjective methods of image quality assessment such as limiting spatial resolution and contrastdetail tests are often used in the quality assurance of x-ray image intensifier fluoroscopy systems (IPEM 1997). These techniques are quick to perform and minimally invasive, two key factors in fluoroscopy quality assurance (QA) where the time available on a particular system is often limited. Contrast-detail methods are well established in the field of fluoroscopy (Hay et al 1985) and provide reproducible results if used with care (Marshall et al 1992). The test generates an image of known contrast content set against a uniform background which is evaluated visually and provides an overall impression of image quality, including any display aspects. Contrast-detail tests therefore utilize a static, signal known exactly/background known exactly (SKE/BKE) task. The observer decides upon a threshold criterion for detection of the 0031-9155/01/061631+19$30.00
© 2001 IOP Publishing Ltd
Printed in the UK
1631
1632
N W Marshall
last visible disc and tries to maintain this threshold constant over the range of disc diameters in the test object and for the period of time over which the QA tests are performed. The method is not free from problems; there can be difficulty when comparing results obtained by different observers because contrast-detail methodology fails to provide an estimate of observer detection efficiency. Methods of image quality evaluation which allow for observer detection efficiency, such as a receiver operating characteristic (ROC) (Chesters 1992) and alternative forced choice (AFC) (Ohara et al 1986), can be time consuming and are of limited use in routine QA testing. Objective measures such as modulation transfer function (MTF) and noise power spectrum (NPS) are a well established means of describing the image quality of x-ray imaging systems (Metz et al 1995). While these methods do not require an experienced observer to obtain reproducible measurements, the techniques themselves can be quite difficult to perform in the field. Access to image data is often a problem for older, conventional x-ray imaging systems such as film/screen radiography and fluoroscopy units. Data access via some nominal format, however, should eventually enable the routine use of these methods with a variety of digital radiography imaging systems. Ultimately, some form of system self-testing should be possible using these methods. Once established, these parameters are useful both in the evaluation of objective image quality (Giger and Doi 1984, Giger et al 1984, Hillen et al 1987, Cowen and Workman 1992, ICRU 1996) and in the optimization of various components in the relevant imaging chain (Siewerdsen and Antonuk 1998). Application of objective image quality methods to fluoroscopy systems poses particular problems because of the range of video systems employed by the different manufacturers. Furthermore, full evaluation of the spatio-temporal aspects of fluoroscopy image quality requires access to a contiguous set of fluoroscopy images. Papers by Goldman (1992) and Tapiovaara (1993) describe NPS measurements for fluoroscopy units; however the relationship between NPS and an established image quality metric currently used to express a QA standard is far from clear. The principal aim of this work is therefore to link the NPS, measured for a fluoroscopy system, to contrast-detail results obtained using the same imaging system. A digital frame store was used to acquire the fluoroscopy image data from which the NPS, MTF and detective quantum efficiency (DQE) were calculated. The version of signal detection theory given by Aufrichtig (1999) was then used to derive observer performance from the objective parameters and the results compared with the contrast-detail measurements. 2. Theory 2.1. Observer model of contrast-detail performance The application of statistical decision theory to medical imaging given in the paper by Wagner and Brown (1985) is well established and has been widely used in diagnostic radiology imaging. Tapiovaara and Wagner (1993) further discuss implications of this theory for medical imaging systems, with reference to both ideal and sub-optimal algorithmic observers. Uses of the theory include the study of imaging system parameters such as pixel size (Giger and Doi 1985), image quality assessment of various imaging systems (Wagner et al 1979, Sandrik and Wagner 1982, Tapiovaara 1993, Aufrichtig 1999) and the investigation of image processing and display (Loo, Doi and Metz 1985, Aufrichtig 1999). In these models, the observer performing the detection task (usually in white noise) uses a matched filter strategy. Briefly, the received signal in frequency space is matched by the observer against a template of the spectrum of the expected signal; detection occurs when the matched signal exceeds a threshold level. A detection task often modelled is the SKE/BKE task, which is relevant to the contrast-detail method described above.
The practical application of signal detection theory in fluoroscopy
1633
Aufrichtig (1999) presents a version of matched filter theory for the contrast-detail task and its application to both a film/screen system and a prototype amorphous selenium (a-Se) imaging system; this is the basis for the observer model used in this paper. The signal-to-noise ratio (SNR) for the detection task, that of detecting a disc of spectrum S(u, v) against a noisy background, can be written as C |S(u, v)MTF(u, v)O(u, v)|2 du dv SNR = (1) ( |S(u, v)MTF2 (u, v)O 2 (u, v)|2 (q0 DQE(u, v))−1 du dv)1/2 with the signal defined as the square root of the peak power. S(u, v) is the spectrum of the object in the frequency domain, MTF(u, v) is the imaging system modulation transfer function and O(u, v) represents the spatial frequency response function of the observer’s visual system. The contrast of the object (C) is introduced by writing the object spectrum S(u, v) = CS(u, v). For a circular object, the signal spectrum is simply the Fourier transform of a disc: √ d J1 (π d u2 + v 2 ) S(u, v) = . (2) √ 2 u2 + v 2 The system detective quantum efficiency DQE(u, v) has been used in equation (1) instead of the noise power spectrum W E/E (u, v) according to the equation DQE(u, v) =
MTF2 (u, v) q0 W E/E (u, v)
(3)
where q0 is the x-ray fluence (defined in terms of number of photons or energy per unit area) onto the image receptor. Equation (3) presents a practical implementation of DQE, using the noise power spectrum and MTF to describe the signal and noise properties of the x-ray system in question. A more general definition of DQE is given by Dainty and Shaw (1974): DQE(u, v) =
SNR2out (u, v) SNR2in (u, v)
(4)
where SNRin and SNRout are the input (i.e. before detection) and output signal-to-noise ratios, respectively. As discussed by Tapiovaara and Wagner (1985), two options are available when evaluating the SNR of the incident x-ray beam. We can either treat the x-ray detector as a photon counter or as an energy integrating device. The implications of this are important when it comes to calculating DQE; we either compare the x-ray detector against an ideal photon counter or against an ideal energy integrating device. In this study, when assigning a value to q0 in equation (3) (i.e. to SNRin ) we assume that the x-ray detector is acting as an energy integrating device (in line with Hillen et al 1987 and Stierstorfer and Spahn 1999), rather than a photon counter, as the CsI scintillator in image intensifiers integrates the incident x-ray energy rather than counts photons. Stierstorstorfer and Spahn (1999) state that switching from an energy fluence to a photon number picture will only decrease the DQE by a few per cent. Isolating such a fine effect would probably require a laboratory setup rather than the field measurements presented here. The SNR of the input energy fluence, characterized by spectral distribution (E), is given by ( (E)E dE)2 q0 = . (5) (E)E 2 dE Stierstorfer and Spahn (1999) refer to q0 as the noise equivalent (input) quanta and this quantity uses the variance of the energy fluence to quantify the input quantum noise.
1634
N W Marshall
The NPS in terms of relative exposure W E/E (u, v) (i.e. input fluctuation) is related to the NPS in terms of pixel grey scale fluctuations W P /P (u, v) via the average gradient of the macro transfer function (G) (Dainty and Shaw 1974): W E/E (u, v) = W P /P (u, v)/G2 .
(6)
At the threshold for detection, the SNR will have exceeded the value of the threshold criterion for detection, SNRT , maintained by the observer. The contrast will then be the threshold contrast, CT : −1 |S(u, v)MTF(u, v)O(u, v)|2 du dv . (7) CT = SNRT ( |S(u, v)MTF2 (u, v)O 2 (u, v)|2 (q0 DQE(u, v))−1 du dv)1/2 The function O(u, v) describes the contrast sensitivity response of the human visual system as a function of spatial frequency and is taken from a paper by Kelly (1979). All the curves presented in this study were calculated using a viewing distance of 50 cm. The value given to SNRT depends on the detection probability assumed by the observer and is discussed later. 2.2. Noise power spectrum Tapiovaara (1993) presents an excellent discussion of the measurement of the full 3D NPS, i.e. the 2D spatial component together with the temporal component. To achieve this, Tapiovaara (1993) acquired a 2D spatial patch (approximately 20×14 mm2 for the 23 cm image intensifier mode) over a period of 16 frames (0.64 s). The NPS was then calculated from this 3D block of spatio-temporal data to give the 2D spatial NPS at various temporal frequencies. The 2D spatial NPS is required because the spatial component demonstrates some anisotropy from the raster nature of the video system readout. Aufrichtig (1999), studying a film/screen system and an a-Se flat panel detector, could make the valid assumption of rotational symmetry for the noise power spectrum and MTF. Here, we work with the 1D NPS, measured in either the horizontal or vertical direction, and accept that this is only an approximation. As will be shown for a range of fluoroscopy systems, the approximation of using just the 1D NPS and MTF is not unreasonable. A full derivation of the method adopted in this study to measure NPS is given by Goldman (1992). We are interested in evaluating a one dimensional slice through the full three dimensional NPS. This section can be either parallel or perpendicular to the TV raster lines (here defined as the horizontal and vertical directions, respectively). Goldman (1992) gives the one dimensional section of a three dimensional noise power spectrum W (u, 0, 0) at spatial frequency u for a noise process characterized by an output function S(x) as 2 −1 Mdy T dt N −2π iux/N W (u, 0, 0) = lim S(x) e dx Tw−1 (8) N →∞ N dx x=0 where the symbol denotes an ensemble average. Implementation of equation (8) is described as follows for a horizontal NPS. Assuming a TV frame has been captured, the first step is to form a long narrow slit of length M video lines (down the screen) and scan this horizontally across the image data P (x, y) producing a scan S(x). The value of S(x) at a given point x in the horizontal scan is found from the mean of the slit values in the y direction: S(x) =
1 M−1 P (x, y). M y=0
(9)
The physical length of each scan is given by N dx where N is the scan length in pixels and dx is the horizontal sampling distance. Likewise, dy is the vertical spatial sampling distance.
The practical application of signal detection theory in fluoroscopy
1635
As stated earlier, the full fluoroscopy NPS is 3D and contains a temporal component that must be accounted for. This can be done by evaluating the NPS over a range of temporal frequencies (Tapiovaara 1993)—a method that requires 3D Fourier transformation of the (3D) noise data. Alternatively, the NPS can be calculated from temporally integrated TV frames (Goldman 1992), an approach which yields the spatial NPS at a temporal frequency of approximately 0 Hz (Tapiovaara 1993). This is the method adopted in this study and hence we are making the further approximation that the NPS measured at low temporal frequencies will be sufficient to model a static object, imaged in dynamic noise. Parameter T in equation (8) represents the number of contiguous video frames used to form the temporal slit. When multiplied by the temporal sampling distance (dt), T dt gives the temporal slit length in seconds. The values of M and T are determined experimentally as they depend on the spatio-temporal blurring characteristics of the fluoroscopy system in question. The term (Tw )−1 in equation (8) corrects for the effect of the finite aperture width of the framestore used to acquire the image data. Here, Tw = sinc2 (u/N dx). The vertical NPS can be studied by making the appropriate changes to equations (8) and (9). 3. Materials and methods Before describing the experiments comparing objective and subjective methods, the following sections detail how the individual components such as MTF, NPS and the macro transfer function were measured. It should be noted that, prior to all the measurements described here, nominal tube potentials for each system were checked using a calibrated non-invasive tube potential divider (Keithley Instruments, USA) connected to an oscilloscope. 3.1. Measurement of noise power spectra Application of equation (7) requires the measurement of the MTF(u, v) and W E/E (u, v) along with the macro transfer function. The fluoroscopy image data were acquired using a Matrox IP8 (Matrox, Canada) digital framestore. This framestore captures data at eight bits/pixel, with a resolution of 512 × 512 pixels and requires a 625 lines/frame, 25 frame s−1 interlaced video signal. A personal computer was used to control the acquisitions and process the resulting data. While the use of a framestore is not free from problems, for example a small additional (additive) noise component from the framestore may be present in the results and these devices often have limited bandwidth when digitizing data along the horizontal video scan lines, this type of system has been used successfully in previous studies (Goldman 1992, Tapiovaara 1993). As a first step, horizontal and vertical sampling distances were established by imaging a 60 mm square piece of 0.1 mm thick copper sheet. Following this, the gradient of the macro transfer function was measured. This parameter is required when relating the NPS measured in terms of pixel values of the acquired image data (the output variable) back to relative x-ray intensity (the input variable), although the presence of veiling glare in the image intensifier makes the accurate measurement of G difficult. Veiling glare arises from light scattering in the output phosphor of the image intensifier and the associated optics which couple the output phosphor to the TV camera. The veiling glare component is of a low spatial frequency and can travel a considerable distance in the image (perhaps 4 or 5 cm), depending on image intensifier design (Gagne et al 1993). For the case of a noise power measurement, veiling glare will flood the area of a uniformly exposed image intensifier field containing the noise samples of interest. This additional light component will tend to reduce the detected noise fluctuations, when compared with the relative x-ray intensity which generated the fluctuations,
1636
N W Marshall Table 1. Contrast of 0.125 mm thick copper square and number of noise equivalent photons per unit doserate as a function of tube potential (tungsten anode, 3.0 mm Al inherent +1.5 mm Cu added filtration, 15◦ target angle, 5% waveform ripple). Tube potential (kV)
log10 (relative contrast)
q0 /doserate (µGy−1 mm−2 s−1 )
55 60 65 70 80 90
0.1555 0.1255 0.1130 0.0988 0.0785 0.0660
31 600 33 200 34 200 34 800 35 000 34 400
by a factor of 1 minus the veiling glare fraction (Fujita et al 1986). Newer designs of output phosphor/coupling optics have significantly reduced veiling glare in the current generation of image intensifier systems however consideration of this effect is still important. Several methods of measuring the macro transfer function are described in the literature (Fujita and Doi 1986, Fujita et al 1986, Goldman 1992, Cowen and Workman 1992, Tapiovaara 1993). Initially, a stepwedge method was adopted, in which a copper stepwedge made of 0.11 mm steps was placed over a thin aperture cut into 2 mm thick lead plate and imaged using an x-ray beam filtered with 1.5 mm of additional copper filtration. The lead collimation is used to suppress the contribution from veiling glare to pixel intensity (Fujita et al 1986). Giger et al (1986) used the method outlined by Fujita et al (1986) when expressing the noise power spectrum in terms of relative x-ray fluctuation. However, this technique involves a measurement of the veiling glare fraction for the image intensifier in question and hence is more suited to a laboratory measurement of the noise power spectrum. An alternative method was also used, which does not try to eliminate the effects of veiling from the measurement. First, the x-ray beam was filtered using 1.5 mm Cu. A sheet of Cu 0.125 mm thick and 42 mm square was then imaged at the tube potential of interest. The contrast in terms of pixel value ( P ) of the 0.125 mm copper sheet was measured using regions of interest placed within and beside the sheet. This technique produces a pixel value contrast close to the mean pixel value of the background, usually at a point midway along the macro transfer function. There are limits to the thickness of attenuation which may be used to produce the contrast (Goldman 1992). In practice, P should at least cover the pixel contrast of the noise fluctuations to be measured. Log10 (relative contrast) of the 0.125 mm thick copper square ( C) was calculated as follows. First, the radiation contrast (in terms of air kerma) between the copper square and the background (effectively 100% transmission) was calculated for a given x-ray energy spectrum and tube filtration (3 mm inherent aluminium and 1.5 mm additional copper filtration) using the x-ray spectrum generation software and mass attenuation data provided by Cranley et al (1997). The quantity log10 (relative contrast) was then found as a function of tube potential (see table 1). Finally, G was taken from the relationship G = P / C and the NPS transformed using equation (6). Figure 1 illustrates the two different approaches for measuring G described above. System 1 in this figure uses an older generation of image intensifier where there is significant veiling glare. A linear fit to the macro transfer function gives G ≈ 146. The dotted line shows G ≈ 216, as measured via the copper square technique. Hence the use of the stepwedge value would tend to overestimate the noise power spectrum and ultimately underestimate DQE(u). Both methods suffer from light scatter, although in this instance the scatter is worse for the stepwedge due to high light levels generated at the low attenuation end of the stepwedge. System 2, on the other hand, uses a newer generation of image intensifier with an output
The practical application of signal detection theory in fluoroscopy
1637
250 System 1
225
G = 216 (Cu square) G = 146 (stepwedge)
200
pixel value
175 150 125 100 75 G = 213 (Cu square) G = 216 (stepwedge)
50 25
System 2
0
1.0
1.1
1.2
1.3
1.4
1.5
1.6
1.7
1.8
1.9
2.0
log (relative transmission)
Figure 1. Measurement of macro transfer function using two methods: copper stepwedge (points) and copper square technique.
phosphor that eliminates much of the veiling glare from the image. Both the stepwedge method and the copper square method give similar values of G for this unit (G ≈ 215). The degree to which veiling glare ultimately affects the measurements will depend on the design of the image intensifier system, with older systems being more seriously affected. The image data P (x, y) required for the NPS were taken from a homogeneous field, imaged with the added 1.5 mm Cu in the x-ray beam at the tube potential of interest. During each acquire stage, data were extracted from a 256 × 64 pixel area at the image intensifier field centre using a partial area acquire and stored in an adjacent frame buffer. This was repeated until 16 (or a maximum of 32) contiguous frames had been captured and the spatial scan S(x) was synthesized for each frame. The full spatio-temporal slit was then formed by summing these spatial scans over the 16 contiguous frames. For a given fluoroscopy system, both spatial and temporal slit length were increased until a plateau in the NPS was reached. Following measurements for a range of systems, spatial slit length was taken to be 5 to 8 mm, depending on sample distance. Temporal slit length was 16 frames (0.64 s). The entire acquisition process was repeated until 128 separate spatio-temporally integrated scans S(x) had been acquired. Using 128 scans for the NPS leads to a relative standard deviation of approximately 8.8% on the NPS (Dainty and Shaw 1974). A polynomial was then fitted to each scan and the mean value removed in order to suppress low frequency effects such as vignetting and the uneven radiation field (Giger et al 1986). The actual order of the polynomial appeared to make little difference provided it was second order or higher; here we used a fourth order polynomial (figure 2). Following removal of the mean by a polynomial, the square modulus of the Fourier transformed noise scan (Press et al 1992) was added to the ensemble average to give the NPS as defined in equation (8). 3.2. Measurement of modulation transfer function Cunningham and Reid (1992) give an extensive account of signal and noise in the estimation of MTF using slit, wire and edge techniques. They conclude that an MTF derived from the edge
1638
N W Marshall 0.01 th
0 order (simple mean) 1st order 2nd order 3rd order th 4 order
2
noise power (mm s)
1E-3
1E-4
1E-5
1E-6 0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
1.1
1.2
-1
spatial frequency (mm )
Figure 2. Use of polynomial subtraction to remove low frequency effects such as uneven radiation field from noise power spectrum.
spread function has a superior SNR at low spatial frequencies. As the limiting resolution of fluoroscopy systems is less than 2.5 cycles mm−1 , we use the differential of the edge spread function to generate the line spread function (McRobbie and Nieto-Camero 1994). In this work we do not use the angled edge method described by McRobbie and Nieto-Camero (1994) to achieve spatial oversampling along the TV lines. Instead, a simple normal edge method is used (with no oversampling) to provide the MTF up to the same Nyquist frequency as the NPS routine. To generate the contrast edge, a lead edge (the Leeds E1 test object (Hay et al 1985)) was placed at the input plane of the image intensifier and imaged under fluoroscopy with 1.5 mm Cu added filtration in the x-ray beam. A pixel template of lines parallel and perpendicular to the TV lines was superimposed on the TV signal to ensure accurate alignment of the edge. Profiles 256 pixels long were sampled across the edge from a region at the centre of the image. These were then differentiated and Fourier transformed (Press et al 1992). A mean of ten MTF measurements from a single image was taken to give the final MTF(u) or MTF(v), depending on scan direction. Hence, MTF(u) came from a roughly isoplanatic patch approximately 3 mm wide (depending on sample distance) at the image centre. Figure 3 shows MTF(u) measured for a Thomson CSF TH 9435 E GKV1 CsI image intensifier (Thomson Electronic Components, France) coupled to a Pulnix TM765 CCD camera (Pulnix, USA) via a tandem lens of focal length 24 mm. This is a 1.7 cm interline transfer CCD of resolution 756 (H) × 581 (V) pixels, giving horizontal and vertical sampling distances of 0.51 mm and 0.35 mm, respectively. A tube potential of 60 kV was used for these measurements. Results for the edge method are represented by the solid line in figure 3. By way of a check on the edge method, MTF(u) was also calculated using the Coltman technique from the square wave response function of a periodic bar pattern (Hillen et al 1987). Modulation within each line pair group for a H¨uttner type 18 line pair test object was measured from an acquired frame and normalized to the modulation of a contrast at 0.05 cycles mm−1 . The Coltman formula was then applied to the square wave modulation, giving the MTF. Results from this method are indicated by the dashed curve in figure 3.
The practical application of signal detection theory in fluoroscopy
1639
1.0 0.9
MTF using edge technique MTF from Coltman method
0.8
MTF from Coltman method scaled to include 15% Low Frequency Drop
0.7
MTF(u)
0.6 0.5 0.4 0.3 0.2 0.1 0.0 0.00
0.25
0.50
0.75
1.00
1.25
-1
spatial frequency (mm )
Figure 3. Modulation transfer function derived using two techniques: edge spread function method and the Coltman method.
Agreement between the two methods appears poor. There is a steep initial drop in modulation transfer of approximately 15% at frequencies below 0.1 mm−1 for the MTF derived from the edge; this is related to the low frequency drop of the image intensifier (Beekmans et al 1981, Brock and Slump 1989). The LFD is a result of long range light scattering in the image intensifier, principally in the output phosphor, and depends on output phosphor window assembly design. Such a drop is not present in the Coltman method as veiling glare is present in the measurement of both line pair modulation and the normalization contrast. Hence the Coltman method does not give an indication of large area contrast loss as any veiling glare tends to cancel out when the normalization is performed. However, once scaled by 15% to include a nominal LFD there is far closer agreement between the edge and Coltman methods (figure 3). It is not clear whether the LFD is fully characterized by this simple edge method; however an LFD of 13% is typical for a Thomson E Series II image intensifier (Thomson Electronic Components 1996) and this agrees closely with the value of 15% taken from figure 3. As this study will show, there can be considerable variation in the shape of the MTF at low spatial frequency. It is important to include some measure of contrast loss in the estimation of MTF as this directly affects signal transfer through the image intensifier–TV chain. We therefore include any LFD which appears in the measurement of MTF using the edge method and hence in the calculation of the theoretical contrast-detail results. 3.3. Contrast-detail and doserate measurements Initially, the Leeds GS1 test object, which contains a stepwedge, was imaged under AEC control to ensure correct setting of brightness and contrast controls on the TV monitor. The TO10 contrast-detail test object (Hay et al 1985) was then placed at the input plane of the image intensifier and imaged, again under AEC control, with 1.5 mm added Cu filtration in the x-ray beam. Two experienced observers scored the images from a distance of approximately 50 cm. Each observer counted to the last disc visible at each diameter and the contrast of this disc was found from published contrast data (MDD 1994). This is defined as the threshold contrast and when plotted against disc diameter gives the contrast-detail graph.
1640
N W Marshall
An MDH 2025 electrometer (Radcal, USA) connected to a calibrated 60 cm3 flat ionization chamber was used to monitor the doserate at the image intensifier during both the objective and subjective measurements. Doserate was measured using an air gap of 10 cm or greater between the chamber and the input plane in order to reduce the effect of any backscattered radiation. The anti-scatter grid was left in place and a measured factor used to correct for grid attenuation. The noise equivalent number of x-ray photons incident on the input phosphor (q0 ) was then calculated via equation (5) using the doserate and the spectral data of Cranley et al (1997)—see table 1. 3.4. Effect of change in image intensifier input doserate One of the most important parameters monitored as part of a QA program is image intensifier input doserate (IPEM 1997) as this parameter directly influences both patient and staff dose and image quality. The contrast-detail measurements were performed on a C-arm system with nominal image intensifier field sizes of 23, 17 and 15 cm and a fibre optic output phosphor assembly linked to a CCD TV camera. Contrast-detail results were acquired at a tube potential of 71 kV for doserates of 0.05, 0.13, 0.24, 0.45 and 0.87 µGy s−1 in the 23 cm field mode using the methodology described above. NPS measurements were then made for each of these input doserates. The macro transfer function was also measured with changing doserate to ensure correct normalization of the NPS. Measurements of the MTF suffered from considerable noise at very low doserates and therefore MTF(u) was calculated from images at the centre of the doserate range, as images acquired at 0.13, 0.24 and 0.45 µGy s−1 gave similar results. Detective quantum efficiency (equation (3)) was calculated from the measured NPS and MTF together with the noise equivalent number of input x-ray photons (q0 ) (equation (5)). These parameters were then used to calculate contrast-detail performance via equation (7). SNRT , required for this calculation, simply scales the calculated curve with respect to the y-axis. A value of 3.5 was assigned to SNRT (found by fitting to the measured contrast-detail results at 0.05 µGy s−1 with SNRT as the free parameter), which agrees closely with the figure of 3.8 given by Ishida et al (1984) for 50% detection probability. This value was used for SNRT when calculating all the contrast-detail curves presented in this paper. 3.5. Image quality at a fixed doserate Given that contrast-detail methodology is an accepted method of assessing image quality of fluoroscopy systems (Hay et al 1985, IPEM 1997), one of the aims was discover whether, for a fixed input doserate, objective and subjective measures ranked three systems consistently. Alternatively stated, the aim of this section is to evaluate how successfully these two techniques can assess noise sources arising from within the imaging chain, given a fixed level of quantum noise as input. Three systems were assessed using the objective and subjective methods described above. System A was a static system installed in 1986 with an undercouch x-ray tube and a three field (30 cm, 23 cm and 17 cm) image intensifier with a conventional vacuum TV camera. System B was a three field (23 cm, 17 cm and 15 cm) mobile C-arm image intensifier unit in use since 1999 with a CCD TV camera and a fibre optic output phosphor assembly. System C had a two field image intensifier (22 cm and 16 cm) coupled to a CCD TV camera and was less than one year old at the time of evaluation. All of these units were tested using the nominal image intensifier mode closest to 23 cm in diameter. A doserate of 0.34 µGy s−1 was set at the input phosphor and the objective and subjective measurements described above were then obtained. Tube potential settings were 70 kV, 71 kV and 63 kV for systems A, B and C, respectively.
The practical application of signal detection theory in fluoroscopy
1641
0.01
scans parallel to TV lines scans vertical to TV lines
2
noise power (mm s)
1E-3
1E-4
1E-5
1E-6 0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
1.1
1.2
-1
spatial frequency (mm )
Figure 4. Noise power spectrum measured parallel and perpendicular to TV scan lines.
4. Results and discussion Figure 4 shows two example noise power spectra, acquired parallel and perpendicular to the TV raster lines, for a typical C-arm unit operated in 23 cm field mode. Noise power parallel to the TV lines plateaus at lower frequencies and then drops dramatically above 0.8 mm−1 while the perpendicular NPS falls steadily as a function of spatial frequency. The parallel NPS has a similar shape to the NPS given by Cowen and Workman (1992) for a digital fluorography system and this has been ascribed to underlying TV camera noise. This noise generally increases along the raster lines and is due to high frequency compensation in the camera applied to correct for signal roll-off at high frequencies (Cowen and Workman 1992, Goldman 1992). This shape of NPS parallel to the TV lines was seen for all systems studied. The effect of varying doserate on a given fluoroscopy system is seen in figure 5, for measurements parallel to the TV scan. The first point to note is that the MTF (figure 5(a)) shows excellent contrast, with little or no LFD. The MTF is unusual in having a triangular shape instead of the more typical ‘exponential’ form often seen in image intensifier systems (Brock and Slump 1989). This may be due to the fibre optic output phosphor assembly found on this unit, which is designed to reduce veiling glare. There is considerable change in noise power with doserate (figure 5(b)), with the NPS falling from 2 × 10−3 at 0.05 µGy s−1 to approximately 1 × 10−4 at 0.87 µGy s−1 (for low spatial frequencies). NPS is a measure of absolute noise power, including x-ray quantum noise, and this parameter will usually scale with doserate. DQE, on the other hand, is largely independent of doserate (provided the measurements are made within the normal operating range of the imaging system). This is expected as DQE reflects changes in SNR transmission by the system as a function of spatial frequency channel, for example the presence of any additional noise sources or loss of signal transmission that my occur as a function of doserate. DQE(u) in figure 5(c) shows some reduction at low spatial frequencies (0.2 mm−1 ) for the 0.87 µGy s−1 measurement. These results were taken from a very bright image as this doserate is significantly greater than the doserate measured under AEC for this system (0.13 µGy s−1 ). This quantity of light could therefore cause some long distance light scatter, despite the fibre optic output phosphor assembly, and thus reduce signal transfer at lower frequencies. At higher
1642
N W Marshall 1.0 0.9 0.8
a) MTF(u)
MTF(u)
0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
1.1
1.2
0.9
1.0
1.1
1.2
spatial frequency (mm-1 )
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8 -1
1.0 0.9 0.8 c) DQE (u)
0.7
DQE(u) at 0.05 µGy/s
DQE(u)
0.6
DQE(u) at 0.87 µGy/s
0.5 0.4 0.3 0.2 0.1 0.0 0.0
0.1
0.2
0.3
0.4 0.5 0.6 0.7 spatial frequency (mm-1 )
0.8
0.9
1.0
1.1
Figure 5. (a) Modulation transfer function measured parallel to TV scan lines using edge spread function method; (b) noise power spectrum measured parallel to TV scan lines for image intensifier input doserates of 0.05 µGy s−1 and 0.87 µGy s−1 ; (c) detective quantum efficiency parallel to TV lines for image intensifier input doserates of 0.05 µGy s−1 and 0.87 µGy s−1 .
The practical application of signal detection theory in fluoroscopy
1643
measured at 0.05 µGy/s calculated for 0.05 µGy/s
threshold contrast (%)
100
measured at 0.87 µGy/s calculated for 0.87 µGy/s
10
1 0.1
1 detail diameter (mm)
10
Figure 6. Measured and calculated contrast-detail results for image intensifier input doserates of 0.05 µGy s−1 and 0.87 µGy s−1 . Points show measured results; lines show results calculated using equation (7). The error bar shows an uncertainty of 11%.
spatial frequencies (>0.5 mm−1 ) the two DQE results match quite closely. If the quantum noise can be thought of as predominantly low spatial frequency noise (Tapiovaara 1993), the similarity in DQE at higher spatial frequencies is probably a result of a (fixed) amount of system noise limiting the SNR. The measured contrast-detail results are plotted as points in figure 6; the error bars in this figure represent an uncertainty of 11% for two observers reading the test object once (Marshall et al 1992). Also shown in this diagram are the curves calculated from the objective image quality results using equation (7) (both derived using a viewing distance of 50 cm and SNRT = 3.5). There is good agreement between measured and calculated curves, although an exact match is not expected. The contrast-detail evaluation includes the display stage, in this case a CRT, while objective measures only evaluate the imaging process up to the point of display with an added parameter to account for the spatial frequency response of the human visual system. CRT contrast and brightness were optimized before viewing the images and therefore the display appears to have had little influence on the results for this unit. The shapes of the calculated curves match the measured results well, but more importantly the calculated curves follow the measured results with changing doserate. This is particularly important for measurements made as part of a routine QA programme, where it is important to quantify changes in an image quality metric due to fluctuations in input air kerma rate. The fact that the contrast-detail results scale with doserate indicates that quantum noise is the principal noise component limiting contrast-detail performance. This is corroborated by the result that DQE(u) for this unit remains approximately constant over the range of doserate studied. Figure 7 shows the objective image quality results for systems A, B and C, respectively, measured at a fixed doserate of 0.34 µGy s−1 . The results presented here were measured parallel to the TV scan lines. The MTFs for systems B and C show no drop in modulation at low frequencies and this is probably a consequence of the fibre optic windows used in these systems. System A, however, contains an older generation of image intensifier tube (with a conventional glass output phosphor window), which shows an LFD of almost 30%. There is also significant noise on this MTF measurement (figure 7(a)), despite being an average of ten scans.
N W Marshall
0.01
0.01
1E-3
1E-3
1E-3
1E-4
1E-5
noise power (mm 2 s)
0.01
noise power (mm 2 s)
noise power (mm 2 s)
1644
1E-4
1E-5
1E-6
1E-5
1E-6
0.0 0.1 0.2 0.3
0.4 0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4
1E-4
1E-6
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
1.1
1.2
1.0
1.0
1.0
0.9
0.9
0.9
0.8
0.8
0.8
0.7
0.7
0.7
0.6
0.6
0.6
0.5
0.5
0.5
0.4
0.4
0.4
0.3
0.3
0.3
0.2
0.2
0.2
0.1
0.1
0.1
0.0
0.0 0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
system A
0.8
0.9
1.0
1.1
1.2
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
1.1
1.2
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
1.1
1.2
0.0 0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
system B
0.8
0.9
1.0
1.1
1.2
system C
Figure 7. (a)–(c) Modulation transfer function for systems A, B and C; (d)–(f) noise power spectrum measured parallel to TV scan lines at an image intensifier input doserate of 0.34 µGy s−1 for systems A, B and C; (g)–(i) detective quantum efficiency for systems A, B and C.
The NPS results for the three units (figures 7(d)–(f)) all have a similar shape due to the camera noise along the TV lines as discussed earlier (Cowen and Workman 1992). The NPS for systems B and C meets the ordinate at approximately 4 × 10−4 mm2 s while system A has a far greater NPS of approximately 1 × 10−3 mm2 s at 0.1 mm−1 . Combining MTF(u) and NPS(u) via equation (3) to give DQE(u) is not free from problems. There can be significant noise in MTF(u) and this leads to an artificially high DQE(u) at higher spatial frequencies, e.g. above 1.0 mm−1 . There is certainly evidence of this in the results for the three units presented here. It may be possible to fit the MTF data and thus reduce the influence of high frequency noise, although care must be taken not to bias measurements. Significant differences in DQE are seen between the three units. At low spatial frequencies (i.e. 0.5 mm−1 ), system B has a DQE of approximately 0.35, system C has a DQE of roughly 0.5 while DQE barely reaches 0.1 for system A. These figures are of the correct order of magnitude for an x-ray image intensifier based imaging system, i.e. between approximately 0.1 and 0.5 (Rowlands and Taylor 1983). Tapiovaara (1993) found a mean
The practical application of signal detection theory in fluoroscopy
1645
100
threshold contrast (%)
system A system B system C
10
a) 1 0.1
1
10
detail diameter (mm)
100
threshold contrast (%)
measured contrast-detail results:
10
calculated results:
system A system B system C
b) 1
0.1
1 detail diameter (mm)
10
Figure 8. (a) Measured contrast-detail results for systems A, B and C. Curves are second order polynomial fits to the contrast-detail results. The error bar shows an uncertainty of 11%. (b) Contrast-detail results calculated using equation (7) together with measured contrast-detail results.
DQE of 0.54 for a Philips Imagica HC 23 cm image intensifier connected to a vidicon pick-up tube, by taking an average of the NPS data for spatial frequencies less than 0.087 mm−1 (but excluding the (0, 0) frequency). The measured contrast-detail results (figure 8(a)) show some difference in imaging performance between the three systems, although the differences are not particularly pronounced. Systems B and C have similar contrast-detail curves, while the result for system A is poorer. The theoretical contrast-detail curves (for a viewing distance of 50 cm and SNRT = 3.5) are plotted in figure 8(b) together with the measured contrast-detail data. Overall agreement is considerably poorer in this figure than the doserate results presented in figure 6.
1646
N W Marshall
A reasonable match is seen for system A; however, threshold contrast predicted by equation (7) is too low for systems B and C, especially for details greater than 3mm in diameter. While agreement between the measured and theoretical curves is not particularly close, inspection of figure 8 shows that the objective and subjective measurements have ranked the three units consistently. Differences in contrast-detail performance between the systems also appear more marked for the calculated curves compared to the measured curves. This suggests that the objective measurements are showing greater sensitivity to image quality changes than the observer contrast-detail results. This is quite plausible as it is impossible to guarantee that the observer maintains a constant decision threshold over a wide range of noise (x-ray, electronic etc), veiling glare, TV monitor contrast and brightness settings and general image appearance. It should be emphasized that the objective parameters presented here have been measured with veiling glare affecting both the signal and noise channels. This is inevitable for a field measurement of this type; however it is anticipated that veiling glare will be less of a problem for newer systems which have improved output window assemblies. If objective image quality measurements are to be truly portable, allowing comparison not just between similar systems but across modalities, then measurement conditions have to be stated explicitly. Further discussion of differences between the calculated and measured data leads into a general discussion of the difficulties involved in modelling subjective data from objective measurements. First, we are using 1D spectra (in effect assuming rotational symmetry) and results calculated from 1D spectra are an approximation only. It is surprising, however, that such good agreement with the subjective measurements can be found with 1D objective image quality parameters. Use of the full 2D spatial spectra should produce closer accord between the subjective measurements and the calculated results; this is a point for further study. The observer model, as presented, may not fully account for all the variables present, for example noise internal to the observers visual system may start to limit performance for lower noise (higher DQE) units (Ishida et al 1984) although this is unlikely because fluoroscopy images generated at 0.34 µGy s−1 contain a considerable amount of x-ray quantum noise. The subjective measurements themselves are not free from problems. It is possible that the observers are failing to hold their decision threshold constant for the period of time over which the measurements were acquired (4 months), a known problem with the contrast-detail method. There may be some influence on the results due to the CRT display, despite the optimization of monitor brightness and contrast before acquiring the contrast-detail results. This is not necessarily a negative point, as the subjective results can highlight a problem with the display stage. Objective and subjective results used together can provide a comprehensive evaluation of the entire imaging chain. While assumptions of shift invariance and noise stationarity are generally valid for fluoroscopy system images (Tapiovaara 1993), there will be some limit to their applicability. The NPS and MTF data were taken from the centre of the image and it is assumed that statistics and image quality parameters derived within this region will apply to the entire image. This may not be the case; for example focusing can vary slightly across x-ray image intensifier images (IPEM 1997). As the details within the contrast-detail test object cover much of the image area, strict stationarity would be required for exact modelling. The temporal slit method presented here evaluates NPS at a low temporal frequency (approximately 0 Hz (Tapiovaara 1993)) and this appears to be sufficient for modelling a static SKE task such as contrast-detail. Furthermore, Tapiovaara (1993) found that quantum noise dominated at low spatial and temporal frequencies (at typical image intensifier doserates), while video system noise began to dominate at high temporal frequencies, although this will be system dependent to some extent. Video noise is far from isotropic and a full 2D evaluation of the spatial NPS would be needed for units exhibiting considerable video noise.
The practical application of signal detection theory in fluoroscopy
1647
Both TV camera lag/camera target persistence and any applied digital frame averaging will affect the NPS as lag/frame averaging effectively increases DQE (reducing NPS in equation (3) leads to an increase in DQE(u)). Tapiovaara (1993) measured lag factors for a fluoroscopy system and found that SNR for a single video frame increased at low air kerma rates as TV camera lag increased due to the high video gain (the result of a dark image intensifier output screen). Systems exhibiting excessive persistence may therefore defeat both objective and subjective image quality measurements and hence future work needs to address the accurate measurement of this quantity in fluoroscopy. 5. Conclusions This study has described a practical method for making field measurements of objective image quality parameters (MTF, NPS and DQE) in x-ray image intensifier quality assurance. It has also demonstrated that a version of signal detection theory can be used to model the contrastdetail performance of a range of fluoroscopy systems with reasonable accuracy. This latter step provides a potential link between the objective and subjective approaches. It is doubtful whether objective image quality parameters will supersede well established subjective methods used to frame current image quality standards (e.g. contrast-detail) in the immediate future. This will certainly be the case while access to image datasets remains limited. Methods calculated directly from acquired image data, however, offer the significant advantage of providing objective information about the system, even in the reduced 1D form presented here. The relative ease with which these parameters can be established may lead to their eventual inclusion as components of x-ray system self-testing routines. Accurate subjective evaluation of image quality will always be difficult, regardless of the methodology used to acquire the subjective data (AFC, ROC, contrast-detail etc). Using objective techniques to reinforce subjective image quality assessment can only help to improve the accuracy of imaging system evaluation. Acknowledgments I would like thank Dr Tony Whittingham of the Regional Medical Physics Department Ultrasound section for the loan of the framestore and accompanying notebook computer. The computing expertise of George Mitchell is gladly acknowledged. Finally, thanks to Dr John Kotre for helpful comments regarding the manuscript. This work was partially supported by the European Union’s radiation protection programme (DIMOND III). References Aufrichtig R 1999 Comparison of low contrast detectability between a digital amorphous silicon and a screen–film based imaging system for thoracic radiography Med. Phys. 26 1349–58 Beekmans A A G, den Boer J, Haarman J W and van der Eijk B 1981 Quality control of image intensifiers in the manufacturing process Proc. AAPM Summer School on Acceptance Testing pp 1–28 Brock M and Slump C H 1989 Automatic determination of image quality parameters in digital radiographic imaging systems Proc. Soc. Photoelectron. Instrum. Eng. 1090 246–56 Chesters M S 1992 Human visual perception and ROC methodology in medical imaging Phys. Med. Biol. 37 1433–76 Cowen A R and Workman A 1992 A physical image quality evaluation of a digital spot fluorography system Phys. Med. Biol. 37 325–42 Cranley K, Gilmore B J, Fogarty G W A and Desponds L 1997 Catalogue of diagnostic x-ray spectra and other data IPEM Report 78 (York: Institute of Physics and Engineering in Medicine) Cunningham I A and Reid B K 1992 Signal and noise in modulation transfer function determinations using slit, wire and edge techniques Med. Phys. 19 1037–44
1648
N W Marshall
Dainty J C and Shaw R 1974 Image Science—Principles, Analysis and Evaluation of Photographic-Type Imaging Processes (London: Academic) Fujita H and Doi K 1986 Accurate measurement of characteristic curves of II-TV digital systems by use of aluminium stepwedge Med. Phys. 13 922–4 Fujita H, Doi K, Giger M L and Chan H-P 1986 Investigation of basic imaging properties in digital radiography. 5. Characteristic curves of II-TV digital systems Med. Phys. 13 13–18 Gagne R M, West C N, Wagner R F and Quinn P W 1993 Laboratory measurements of sensitometry, MTF, veiling glare, Wiener spectrum and DQE for image intensifier tubes Physics of Medical Imaging, Proc. Soc. Photoelectron. Instrum. Eng. 1896 248–58 Giger M L and Doi K 1984 Investigation of basic imaging properties in digital radiography. 1 Modulation transfer function Med. Phys. 11 287–94 ——1985 Investigation of basic imaging properties in digital radiography. 3 Effect of pixel size on SNR and threshold contrast Med. Phys. 12 201–8 Giger M L, Doi K and Fujita H 1986 Investigation of basic imaging properties in digital radiography. 7 Noise Wiener spectra of II–TV digital imaging systems Med. Phys. 13 131–8 Giger M L, Doi K and Metz C E 1984 Investigation of basic imaging properties in digital radiography. 2 Noise Wiener spectrum Med. Phys. 11 797–805 Goldman L 1992 Fluoroscopic performance tests using a portable computer/frame grabber: Wiener spectrum measurements Med. Phys. 19 45–52 Hay G A, Clarke O F, Coleman N J and Cowen A R 1985 A set of x-ray test objects for quality control in television fluoroscopy Br. J. Radiol. 58 335–44 Hillen W, Schiebel U and Zaengel T 1987 Imaging performance of a digital storage phosphor system Med. Phys. 14 744–51 Ishida M, Doi K, Loo L-N, Metz C E and Lehr J L 1984 Digital image processing: effect on detectability of simulated low-contrast radiographic patterns Radiology 150 569–75 ICRU 1996 Medical Imaging—the assessment of image quality, Report 54 International Commission on Radiation Units and Measurements (Bethesda, MD: ICRU) IPEM 1997 Recommended standards for the routine performance testing of diagnostic x-ray systems The Institute of Physics and Engineering in Medicine Report vol 77 (York: IPEM) Kelly D H 1979 Motion and vision II Stabilized spatio-temporal threshold surface J. Opt. Soc. Am. 69 1340–9 Loo L-N, Doi K and Metz C E 1985 Investigation of basic imaging properties in digital radiography. 4 Effect of unsharp masking on the detectability of simple patterns Med. Phys. 12 209–14 Marshall N W, Faulkner K, Kotre C J and Robson K 1992 Analysis of variations in contrast detail measurements performed on image intensifier-television systems Phys. Med. Biol. 37 2297–302 McRobbie D W and Nieto-Camero J J 1994 Measurement of image intensifier system modulation transfer function by video signal analysis Med. Phys. 21 257–64 MDD 1994 The testing of x-ray image intensifier-television systems Medical Devices Directorate Report MDD 94/07 (London: HMSO) Metz C E, Wagner R F, Doi K, Brown D G, Nishikawa R M and Myers K J 1995 Toward consensus on quantitative assessment of medical imaging systems Med. Phys. 22 1057–61 Ohara K, Chan H-P, Doi K, Giger M L and Fujita H 1986 Investigation of basic imaging properties in digital radiography. 8 Detection of simulated low-contrast objects in digital subtraction angiographic images Med. Phys. 13 304–11 Press W H, Teukolsky S A, Vetterling W T and Flannery B P 1992Numerical Recipes in C: the Art of Scientific Computing (Cambridge: Cambridge University Press) Rowlands J A and Taylor K W 1983 Absorption and noise in cesium iodide x-ray image intensifiers Med. Phys. 10 786–95 Sandrik J M and Wagner R F 1982 Absolute measures of physical image quality: measurement and application to radiographic magnification Med. Phys. 9 540–9 Siewerdsen J H and Antonuk L E 1998 DQE and system optimization for indirect-detection flat-panel imagers in diagnostic radiology Physics of Medical Imaging, Proc. Soc. Photoelectron. Instrum. Eng. vol 3336, pp 546–55 Stierstorfer K and Spahn M 1999 Self-normalizing method to measure the detective quantum efficiency of a wide range of x-ray detectors Med. Phys. 26 1313–19 Tapiovaara M J 1993 SNR and noise measurements for medical imaging: II. Application to fluoroscopic x-ray equipment Phys. Med. Biol. 38 1761–88 Tapiovaara M J and Wagner R F 1985 SNR and DQE analysis of broad spectrum imaging Phys. Med. Biol. 30 519–26 ——1993 SNR and noise measurements for medical imaging: I. A practical approach based on statistical decision theory Phys. Med. Biol. 38 71–92 Thomson Electronic Components 1996 Private communication
The practical application of signal detection theory in fluoroscopy
1649
Wagner R F and Brown D G 1985 Unified SNR analysis of medical imaging systems Phys. Med. Biol. 30 489–518 Wagner R F, Brown D G and Pastel M S 1979 Application of information theory to the assessment of computed tomography Med. Phys. 6 83–94
