Quality Control evaluation of 37 liquid crystal ... - SPIE Digital Library

Quality Control evaluation of 37 liquid crystal displays used in diagnostic services J.J. Moranta, M. Chevalierb, E. Guibelaldeb,1, M.E. Brandanb,c a

b

Servei de Proteccio Radiologica, Universitat Rovira i Virgili, 43201 Reus, Spain Dept. de Radiología, Fac. de Medicina, Universidad Complutense Madrid, 28040 Madrid, Spain c Instituto de Física, Universidad Nacional Autónoma de México, 04510 DF, Mexico ABSTRACT

The performance of 37 primary class liquid crystal display devices (2, 3 and 5 Mpixel matrix size) used in 9 different diagnostic services in Spain has been determined in terms of 13 quantitative and visual evaluations. The equipment had never been subjected to calibration or to QC tests since commissioning by vendors, between 2 and 18 months before measurements. Tests, using calibrated luminance meters and TG18 patterns, have evaluated ambient light conditions and other basic performance indicators, namely, display geometric distortion, artefacts, resolution and low-contrast visibility, contrast luminance response compliance to DICOM standard, luminance extreme values and uniformity between pairs of monitors associated to a same workstation. The principal sources of non-compliance are failures to visualize low-contrast test objects (73% of displays), excessive differences with the DICOM contrast response standard (57%), and nonuniform response of monitor pairs (54%). Also, 43% of LCD were found located in places with excessive illumination and presenting specular reflections from faceplates. The analysis of ten 5 Mpixel displays, of possible use in mammography services, indicates similar performance as the rest of monitors, except for the ambient luminance (100% complying with recommendations) and larger non-compliance with the DICOM response standard (80%). No correlation between image quality indicators and monitor hours of operation was found. Keywords: Softcopy image display, liquid crystal display, LCD, TG18, QC, monitor, luminance, DICOM GSDF, European mammography protocol, Spanish mammography protocol

1. INTRODUCTION Digital techniques currently used in radiology are transforming the clinical environment –among other reasons- replacing the traditional light-boxes by softcopy image displays (monitors), either cathode-ray tubes (CRT) or liquid crystal display (LCD) devices. While the advent of modern equipment might be erroneously interpreted as implying a simplification in the Quality Control (QC) procedures that guarantee their optimum performance, the new technology in CRT and LCD monitors requires a number of novel QC procedures. In 2005, exhaustive QC recommendations associated to radiological monitors were published in the USA, after work by the American Association of Physicists in Medicine Task Group 18 (TG18) [1,2]. The report recommends assessing display performance in terms of evaluations and measurements of room illumination, geometric distortion, display reflection, luminance response, luminance spatial and angular dependencies, resolution, noise and glare, among others. In Europe, the 2005 guidelines for mammography screening [3] included 8 QC recommendations for monitors, to be applied either daily or up to every 6 months. Of particular interest for this work, the recent Spanish QC protocol for digital mammography [4] recommends 8 tests –all considered as “essential”-- to be applied with various frequencies to displays included into digital mammography systems. All these protocols make use of the dedicated QC patterns developed by TG18 (freely available to users [5]) to homogenize the evaluation of the monitors. Compliance with the DICOM Greyscale Standard Display Function (GSDF) [6] has been proposed by all guidelines in order to guarantee consistent high-quality image appearance with respect to luminance contrast in all display devices. 1

[email protected]

Phone (+34) 91394 1555; fax (+34) 91394 1675; http://www.ucm.es/centros/webs/gi5037/

Medical Imaging 2009: Image Perception, Observer Performance, and Technology Assessment, edited by Berkman Sahiner, David J. Manning, Proc. of SPIE Vol. 7263, 72631K © 2009 SPIE · CCC code: 1605-7422/09/$18 · doi: 10.1117/12.810955 Proc. of SPIE Vol. 7263 72631K-1

A few applications of these protocols have been reported so far. In Ref. 7, seven tests considered likely to affect image quality were applied to 14 LCD monitors in a radiology department under optimized illumination conditions, finding that 4 of them did not comply with tolerances stated by TG18. In Ref. 8, an 18 month follow-up of calibration and performance for 4 CRT and 10 LCD monitors (seven of them not manufactured for medical uses) was reported. The main results indicated gradual deterioration over time of the maximum luminance in the non-medical displays. An independent study of 3 LCD [9] indicated absence of an effect from the monitor age (new displays up to 2.5 years old) in an observer performance evaluation. For mammography, the DMIST trial [10], performed before the publication of TG18, reports that two out of the three evaluated systems performed in acceptable terms with respect to the softcopy display luminance response; for the third, the monitor had not been calibrated in terms of the DICOM standard and thus, did not comply with it. All these reports have emphasized the need to incorporate softcopy displays into the Quality Control programs in medical diagnostic services. In this work, we have performed an evaluation of 37 primary class (diagnostic) monitors currently used in hospitals and clinics in Spain. Searching for a compromise between the completeness of TG-18 recommendations and the contents of the European and Spanish guidelines, we have selected 13 among the tests, measuring and evaluating room lighting, display reflections, geometric distortions, monitor luminance, low-contrast visibility, artefacts, resolution, and conformance of luminance contrast with the DICOM standard. The selected tests include all the measurements recommended by the European and Spanish mammography guidelines.

2. MATERIALS AND METHODS 37 LCD monitors belonging to 9 Spanish medical diagnostic services (between 1 and 16 devices each) in 7 Spanish cities were evaluated. Devices were either 2, 3 or 5 Mpixel flat panel displays, had either 8-, 10- or 12- bits pixel depth, many had autocalibration and some had ambient light compensation systems. They were used in general radiology, CT, MRI or mammography services. The equipment had never been subjected to a QC test or calibration except from initial commissioning done by the vendor, from a couple of months up to 1.5 year before these measurements. Table 1 summarizes the main technical characteristics of the evaluated equipment. 13 quantitative and visual evaluations were applied to the monitors under normal conditions of use of the equipment, including ambient lighting. Quantitative luminance measurements were performed using calibrated near-focus Unfors Light-O-Meter P10 Standard, Unfors XI (Unfors Instruments AB, Billdal, Sweden), and VeriLUM® Dual-Mode pod (IMAGE Smiths, Inc., Maryland, USA) luminance meters, and a telescopic Konica Minolta LS-100 photometer (Konica Minolta Sensing, Inc., Osaka, Japan). Most visual evaluations used the test patterns developed by AAPM Task Group 18, and all these evaluations were performed by one clinical medical physicist, familiar with digital LCD devices. Tests, listed in Table 2, were recorded and evaluated using ad hoc Excel datasheets to assure homogeneous data collection and analysis. These spread sheets, available (in Spanish) for download [11], were designed to display and analyze the collected information simultaneously for both monitors (in the case of a workstation with left and right devices) thus facilitating the collection task.

Table 1. Main features of the evaluated LCD monitors.

Monitor brand BARCO BARCO BARCO BARCO IMAGE SYSTEMS PHILIPS PHILIPS WIDE

Model

Matrix size E2621MA 1200x1600 E3621 1536x2048 MDCG3120CB 1536x2048 MFGD5421 2048x2560 FP2080M 1536x2048 MML2032 1536x2048 MML2152 2048x2560 IF2103MP 1536x2048

Bit depth 8 8 8 8 8 10 10 12

Proc. of SPIE Vol. 7263 72631K-2

Auto Ambient Quantity calibration light comp Y N 2 N N 5 Y Y 2 Y Y 8 N N 2 Y N 14 Y N 2 Y N 2

Test 01 measured Lamb, the ambient lighting level. In order to guarantee that the contrast in dark regions is, at least, 80% of the contrast observed in conditions of darkness, it is recommended [1] that Lamb for primary class devices is less than 0.25 times the minimum luminance Lmin produced by the device. This restriction limits Lamb to values, typically, between 7 and 25 lux. In this work, the requirement was for the ambient level to be < 10 lux, a value considered as normal for Xray diagnostic reading stations [1]. A limit of 10 lux under clinical conditions is also a requirement for mammography rooms, according to the European and Spanish protocols [3,4]. The measurement of Test 01 used a luminance meter at the LCD faceplate center, with monitor turned off, recording (for repeatability in future measurements) the location and orientation of the display device. Test 02 visually evaluated the speculated reflection of bright objects by examining the LCD faceplate at a distance of ≈ 30–60 cm within an angular view of +/- 150 for the presence of specularly reflected light sources or illuminated objects. No reflections should be observed, according to recommendations. Test 03 visually evaluated the presence of dust and smudges on the LCD faceplate and after evaluation, if necessary, the plate was cleaned following the manufacturer’s recommendations. Test 04 visually inspected the presence of visible light sources behind the monitor; since these might difficult the radiologist performance, no lights should be visible. Next series of tests refer to visualization performance using the test patterns provided by TG18. First, the TG18-QC pattern is displayed to visually assess the overall performance of the display, as suggested for daily tests by TG18 [1] and for weekly evaluations by the Spanish mammography protocol [4]. Test 05 looks for monitor geometric distortions. The pattern should appear centred in the display active area and all pattern borders and lines should be visible and straight. The TG18-QC pattern is used in Test 06 to visually evaluate general image quality and detect the presence of gross artefacts. The overall pattern appearance should be assessed, and non-uniformities or artefacts should be noted, especially at black-to-white and white-to-black transitions. The ramp bars should appear continuous, free from contour lines, thus indicating that the graphics card is working according to the monitor pixel depth and no serious misadjustment with respect to the DICOM standard exists. Luminance response and low-contrast visibility are evaluated in Test 07 using the TG18-QC pattern. Evaluation consists on verifying that the 5% and 95% and all 16 luminance patches are distinctly visible. The appearance of all low contrast QUALITY CONTROL letters and all luminance patches is evaluated under normal illumination conditions. Test 08 visually evaluates the display resolution using TG18-LPH10, TG18-LPH50, TG18-LPV10, and TG18-LPV50 patterns; all single-pixel-wide horizontal and vertical lines across the display at different luminance levels should be visible. Luminance uniformity is evaluated by Test 09. Luminance is measured with a photometer at the centre of each of the 5 uniform squares in TG18-UNL10 and TG18-UNL80 patterns; it is expected that the maximum luminance deviation for an individual device is < 30%. Test 10 evaluates compliance of the monitor display response with the DICOM Grayscale Standard Display Function (GSDF). Luminance levels L`= L + Lamb are measured using a telescopic photometer, at 18 luminance values using the TG18-LN12-01 … -18 patterns (Lamb is a fixed contribution from diffusely reflected ambient light). The 18 gray levels are transformed into JND (Just Noticeable Differences) indices, the quantity ΔL/L is evaluated and the measured contrast ΔL/L vs JND is compared with the expected contrast per JND from the DICOM display function [5]. The European and Spanish documents [3,4] set a maximum deviation of 10% between the observed contrast and the DICOM standard. Test 11 evaluates the maximum luminance, Lmax, using the maximum luminance value measured with TG18-LN12-18 in the previous test. It is expected that Lmax > 170 cd/m2. Test 12 evaluates the ratio of extreme luminances Lmax/Lmin using values from Test 10; it is expected that Lmax/Lmin is > 250. Finally, Test 13 measures the uniformity of luminance between pairs of monitors in a same workstation, comparing the left and right values of Lmin and Lmax; the differences of the values (which were measured as part of Test 10) should be < 5%.


Table 2 Applied QC tests

Test description

Method

Expected performance

Observations

01. Ambient lighting level

Measurement using calibrated luminance meter

Lamb < 10 lux [3,4]

Test done under conditions similar to those for normal equipment use. See section 3.4.4 in [1].

02. Specular reflections

Visual inspection

Absence of visible patterns [1,4].

See section 4.2.3 in [1].

03. Dust and smudges Visual inspection

Absence of visible dust and stains [1]

When necessary, the faceplate was cleaned following the manufacturer’s recommendations. See 3.4.3 in [1].

04. Intense light sources behind the monitor

Visual inspection

Absence of visible lights.

05. Geometric distortion

Visual inspection of TG18QC pattern

Pattern borders and lines are visible Not passed if one (or more) and straight and the pattern appears features fail. See section 4.1.3 in to be centred in the LCD active area Ref. [1]. [1,4]

06. General image quality and artefacts


Evaluate the overall pattern appearance, look for nonuniformities or artefacts [1,3,4].

07. Luminance response and lowcontrast visibility


Verify that the 5% and 95% and all Not passed if one (or more) 16 luminance patches are distinctly objects are not visible. See visible. Evaluate the visibility of all section 4.10.1 in Ref. [1]. low-contrast letters [1,3,4].

Not passed if one (or more) features fail. See section 4.10.1 in Ref. [1].

08. Display resolution Visual inspection of TG18LPH10, TG18-LPH50, TG18-LPV10, and TG18LPV50 patterns

Verify that horizontal and vertical bars at 1 pixel width, 1/16 modulation, and 3 luminance levels are distinctly visible [1,3,4]

The use of a magnifying glass was allowed, but not zooming in. Not passed if one (or more) lines are not visible. See 4.10.4 in Ref. [1].

09. Luminance uniformity

Measurement of TG18UNL10 and TG18-UNL80 patterns using calibrated luminance meter.

Maximum deviation among pattern squares in an individual monitor < 30% [1,3,4].

See section 4.4.4 in [Ref. 1].

10. Comparison with DICOM standard luminance response

Measurement of TG18-LN12 Contrast response computed from patterns using a telescopic 18 gray levels within 10% of the luminance meter. DICOM GSDF [3,4].

11. Maximum luminance

Measurement of Lmax using TG18-LN12-18 pattern.

Lmax > 170 cd/m2 [1,4].

Lmax is measured in Test 10. See section 4.3.4 in Ref. [1]

12. Maximum to minimum luminance ratio

Measurement of Lmax and Lmin using TG18-LN12-18 pattern.

Lmax/Lmin > 250 [1,3,4].

Lmax and Lmin are measured in Test 10. See section 4.3.4 in Ref. [1]

13. Luminance differences between monitors

Measurement of Lmax and Lmin using TG18-LN12-18 pattern

Difference between Lmax and Lmin for different monitors of the same workstation < 5% [3].

Lmax and Lmin are measured in Test 10. See section 4.3.4 in Ref. [1]


See section 4.3.4 in Ref. [1].

3. RESULTS AND DISCUSSION Table 3 summarizes the results of the tests applied to the 37 LCD, and dark bars in Fig. 1 show the percentage compliance for each of the evaluations. The first 4 tests are related to the ambient conditions of visualization and basic equipment maintenance, while the next 9 refer to the monitor calibration and performance evaluated from image quality indicators. With respect to the first group of tests (01-04), on the average, 31% of the evaluations were negative. 14 out the 37 monitors operated under ideal ambient conditions (i.e. passed all four tests) and this optimum evaluation is not correlated to belonging to a given hospital. In fact, the institution that concentrates the largest number of evaluated LCD in this study (16 out of 37) received an equal number of pass and fail for tests 01-04. This suggests that the lack of a QC program that guarantees the compliance with good visualization procedures for all devices in a given hospital, leaves the visualization conditions up to random conditions. Tests 05-13 refer to a fixed pattern image quality as visualized by the device. 22% of the evaluations were negative for these tests, particularly Test 07 (visibility of the 16 luminance patches and all low-contrast letters in the pattern) failed in 73% of the devices, by far the poorest performance indicator in this study. In 30 displays the 16 patches were visible (in the rest, only 15) and only in 10 monitors all the low-contrast letters were distinctly visible. Compliance with the DICOM standard, Test 10, was unacceptable (according to tolerances in Refs. [3,4]) in 21 monitors, probably due to the lack of a regular calibration of luminance contrast. In those units with an autocalibration system, after these tests were finished displays were calibrated and its luminance response curve agreed with the DICOM standard. This was shown to radiologists in the service as an indication of the practical convenience of implementing a QC program for the monitors. Numbers in Table 3 indicate that results for Tests 07 and 10 did not necessarily agree. Even if 16 monitors that failed the DICOM evaluation also failed the low-contrast visualization, there were 19 displays where the results for one test did not agree with the other. Our experience indicates that the visual evaluation in Test 07 might be influenced by subjective factors, particularly when visualizing the letters in the pattern, previously known to exist. In this respect, an objective measurement such as Test 10, presents advantages. Fig. 2 shows the dL/L contrast curves generated by the worksheets for Test 10, for a couple of monitors in a same workstation and only the left monitor passes the test. For this particular case, both monitors failed to show all the low contrast letters in Test 07. Another “difficult” test was number 13, which requires homogeneous luminance response in both monitors associated to a same workstation, which was passed by about half of the devices. On the other hand, Tests 08, 09, 11 and 12 (resolution, luminance uniformity over the faceplate, and extreme values of luminance) were passed by all, or almost all the monitors. Table 3 Results of tests (failure to comply with recommendations). Total number of evaluated LCD monitors is N=37.

Test ID

Number (%) of LCD failing to comply

01. Ambient lighting

16 (43)

02. Reflections

16 (43)

03. Dust and smudges

5 (14)

04. Intense light behind

9 (24)

05. Geometric distortion

1 (3)

06. Image quality and artefacts

1 (3)

07. Low-contrast visibility

27 (73)

08. Resolution

1 (3)

09. Luminance uniformity

1 (3)

10. DICOM GSDF response

21 (57)

11. Maximum luminance

0 (0)

12. Max/Min luminance

2 (5)

13. Luminance differences

20 (54)


57

1 2

50

100

57

3

80 76

Test identification

4

86

80

5

90

6

90 27

7

97

30 97

8 9 10

97

90

97

43

20

100 100

11 95

12 13

100

40

100

46

Percentage compliance Fig. 1. Percentage compliance for tests described in Table 2. Black bars refer to the total set of evaluated LCD (N=37). Grey bars refer to the subset of 5 Mpixel monitors (N=10).

Three monitors passed all the tests in the group 05-13, and one of these devices also passed all the tests in group 01-04. This device belongs to a hospital with an equal number of pass and fails for the rest of its monitors, suggesting random conditions. No correlation was found between compliance with tests 05-13 and bit pixel depth (two of the three monitors were 8, and one was a 10 bit depth display), nor with existence of an autocalibration system. None of the three monitors passing all tests 05-13 had ambient light compensation. It has been suggested that failure to keep the ideal technical conditions could be associated to the monitor lifetime [8]. An analysis of test results and age does not suggest a correlation. The three devices passing all the technical performance tests (05-13) belong to the group with a “medium” life (from 360 to 680 hours of operation), not the oldest (over 5200 h) nor the newest ones (about 150 h). Of special interest is the performance of monitors in mammography services. According to the European guidelines [3], only 5 Mpixel displays can be used as class 1 monitors in mammography. As shown in Table 1, 10 of the studied LCD have the required matrix size to be used for mammographic images. We have evaluated their performance, and their compliance is shown by the light bars in Fig. 1. Data indicates that all 5 Mpixel units do function in the low ambient lighting conditions required by regional and local protocols, however, they fail to comply with the rest of tests in a way similar to the complete set of devices. Particularly worrisome is the fact that only two of the 5 Mpixel monitors show contrast luminance adjusted to the DICOM standard. This can be interpreted as resulting from a general failure to provide monitors with QC actions, except for special attention paid to the required ambient lighting conditions.


1,0

1,0

Right monitor DICOM standard +/- 10%

d L /L

d L /L

Left monitor DICOM standard +/- 10%

p-values

p-values

0,1

0,1 0

1000

2000

3000

4000

5000

0

1000

2000

3000

4000

5000

Figure 2. Contrast response data and DICOM standard for left and right monitors associated to the same workstation. Only the left monitor passes the test.

4. CONCLUSIONS We have evaluated 37 liquid crystal displays working in Spanish radiological centers. The visual evaluation and quantitative measurements have include all that is recommended by the European and Spanish protocols for monitors used in digital mammography, and constitute a subset of the tests proposed by the AAPM TG18 for softcopy used as class 1 displays in radiology. Only one of the evaluated displays passed all the tests, and this device belonged to a hospital with an equal number of pass and fails for the rest of its monitors, suggesting random conditions. Some basic image quality indicators, such as the contrast response matched to the DICOM standard, failed in almost half of the equipment. The subsequent application of calibration procedures by the medical physicist performing the tests achieved compliance of the image quality indicators with recommendations, indicating that the lack of frequent calibration could be the cause for deficient performance. In absence of a QC program that includes softcopy displays, the performance of these systems could be far from the optimum, even a couple of months after purchase of the monitor.

5. ACKNOWLEDGMENTS We thank the collaboration of Esteban Velasco from ASIGMA during measurements. MEB acknowledges partial support from DGAPA-UNAM and CONACYT-Mexico for her academic stay at the Universidad Complutense de Madrid.

REFERENCES [1] [2] [3]

AAPM, [Assessment of display performance for medical imaging systems], American Association of Physicists in Medicine Task Group 18, on-line Report No. 3 (http://www.aapm.org) (2005). Samei,E. et al., "Assessment of display performance for medical imaging systems: Executive summary of AAPM 18 report," Med. Phys. 32, 1205-1225 (2005). European Commission, [European Guidelines for Quality Assurance in Breast Cancer Screening and Diagnosis] (http://www.euref.org) (2005).


[4] [5] [6] [7] [8] [9] [10] [11]

Sociedad Española de Física Médica, [Protocolo de control de calidad en mamografía digital] (in Spanish) (2008). http:deckard.mc.duke.edu/~samei/tg18 and http://www.euref.org National Electric Manufacturers Association, [Digital Imaging and Communication in Medicine (DICOM) Part 14] (http://www.nema.org/stds/ps3-14.cfm) (2007). Thompson,D.P., Koller,C.J., and Eatough,J.P., "Practical assessment of the display performance of radiology workstations", Br. J. Radiol. 80, 256-260 (2007). Crespi,A., Bonsignore,F., Paruccinin,N., and Macchi,I., "Grayscale calibration and quality assurance in diagnostic monitors in a PACS system", Radiol. Med. 111, 863-875 (2006). Krupinski, E.A., Roehrig, H., and Fan, J., "Does the age of liquid crystal displays influence observer performance?," Acad. Radiol. 14(4), 463-467 (2007). Bloomquist, A.K, et al., "Quality control for digital mammography in the ACRIN DMIST trial: Part I", Med. Phys. 33(3), 719-736 (2006). http://www.ucm.es/centros/webs/gi5037/
