778
Blaivas et al.
d
ULTRASOUND MACHINE COMPARISON
Ultrasound Image Quality Comparison between an Inexpensive Handheld Emergency Department (ED) Ultrasound Machine and a Large Mobile ED Ultrasound System Michael Blaivas, MD, RDMS, Larry Brannam, MD, RDMS, Daniel Theodoro, MD, RDMS Abstract Questions have been raised regarding image quality (IQ) provided by portable ultrasound (US) machines. Objectives: To determine if a difference exists between images obtained with a common portable US machine and those obtained with a more expensive, larger US machine when comparing typical views used by emergency physicians. Methods: The authors performed a cross-sectional, blinded comparison of images from similar sonographic windows obtained on healthy models using a SonoSite 180 Plus and a General Electric (GE) 400 US machine. Both machines were optimized by company representatives. Images obtained included typical abdominal and vascular applications using the abdominal and linear transducers on each machine. All images were printed on identical high-resolution printers and then digitized using a bitmap format at 300 dots-per-inch resolution (RES). Images were then cropped, masked, and placed into random order comparing each view per model by a commercial Web design company (loracs.com). Three credentialed emergency physician sonologists, blinded to machine type, rated each image pair
for RES, detail (DET), and total IQ as previously defined in the literature using a ten-point Likert scale; 10 was the best rating for each category. Paired t-test, 95% confidence intervals (95% CIs), and interobserver correlation were calculated. Results: A total of 49 image pairs were evaluated. Mean GE 400 RES, DET, and IQ scores were 6.8, 6.8, and 6.6, respectively. Corresponding SonoSite means were 6.3, 6.3, and 6.0, respectively. The difference of 0.5 (95% CI = 0.13 to 1.1) for DET was not statistically significant (p = 0.06). The differences of 0.5 (95% CI = 0.1 to 1.1) and 0.6 (95% CI = 0.2 to 1.2) for RES and IQ were statistically significant, with p = 0.01 and 0.01. There was good interobserver agreement (k = 0.71; 95% CI = 0.67 to 0.78). Conclusions: A statistically significant difference was seen between GE 400 and SonoSite in IQ and RES, but not DET. Key words: emergency ultrasonography; portable ultrasound; ultrasound; emergency medicine; ultrasound machine comparison. ACADEMIC EMERGENCY MEDICINE 2004; 11:778–781.
Emergency ultrasound (US) has been bolstered significantly by the emergence of smaller and less expensive US equipment. Many emergency departments (EDs) have significant budget limitations, and costs of $200,000 and higher per machine frequently made this equipment inaccessible. Size and weight of the equipment also matter to those pushing it through the small rooms and tight turns of many EDs. In a typical emergency US environment, the machine is moved from room to room as opposed to a radiology suite, where a machine may remain in one room for months or even years.
Since the late 1990s, several manufacturers have introduced portable or handheld US machines. SonoSite (Bothell, WA) in particular has marketed heavily to emergency physicians (EPs) and other specialists outside of radiology. The manufacturer’s most popular unit to date, the 180 Plus, is marketed for its portability, durability, and relatively low cost. However, its miniaturization may create compromises in function, and concerns have been raised regarding image quality (IQ) in these smaller and less expensive machines.1 We performed a comparison between the SonoSite 180 Plus and General Electric (GE) 400 Pro (Milwaukee, WI), both relatively common machines in EDs and marketed to EPs by their respective manufacturers. Our goal was to determine if a significant difference existed between images from similar, standard emergency US images obtained on the same subjects using the two machines.
From the Department of Emergency Medicine, Medical College of Georgia, Augusta, GA (MB, LB); and the Department of Emergency Medicine, North Shore University Hospital, Manhasset, NY (DT). Received September 15, 2003; revision received December 16, 2003; accepted December 18, 2003. Address for correspondence and reprints: Michael Blaivas, MD, RDMS, Department of Emergency Medicine, Medical College of Georgia, 1120 15th Street, AF-2056, Augusta, GA 30912-4007. Fax: 706-721-7718; e-mail:
[email protected]. doi:10.1197/j.aem.2003.12.030
METHODS Study Design. This was a cross-sectional, blinded, and randomized observational study that examined
ACAD EMERG MED
d
July 2004, Vol. 11, No. 7
d
779
www.aemj.org
whether there was a significant difference in images from similar sonographic windows obtained on two different US machines commonly found in EDs. The study was approved by the institutional review board, with human models giving consent before study onset. No external or internal funding was available for this project.
high-resolution computer monitors with similar settings to review images. Reviewer responses for each category for each image in a pair were stored on an online database and then transferred to a Microsoft Access database (Microsoft Corporation, Redmond, WA).
Study Setting and Population. The study was conducted in an urban teaching ED with an emergency medicine residency program. Hospital credentialing is available for EPs wishing to use bedside ultrasonography. Healthy models were used to evaluate image quality of two US machines under consideration by the ED. Both machines were optimized by representatives from each company, but representatives were not present during the comparison.
Data Analysis. Paired t-test, confidence intervals (CIs), and interobserver correlation were calculated. All statistical calculations were performed using StatsDirect (StatsDirect Software Inc., Ashwell, UK).
Study Protocol. SonoSite 180 Plus and GE 400 Pro US machines were used. Each was fitted with identical high-resolution thermal printers with identical settings and printer paper. Images were printed on thermal printers because this is a typical way to compare still images, and these printers allow for high-resolution images, capturing the entire US machine screen.2 An emergency sonologist with extensive experience obtained all of the images (MB). Images obtained included Morison’s pouch, splenorenal recess, gallbladder, pelvis, aorta (proximal, middle, and distal), femoral vein, and brachial vein. Two types of transducers were used on each machine: a linear array transducer (both broadband, ranging from 5 to 11 MHz) for superficial imaging and a curved linear transducer for abdominal imaging (both broadband up to 5 MHz). Gain, depth, frequency, and lighting conditions remained unchanged for the all images. Images were sent to a Web design company (loracs.com, Cedar, MI) where they were digitized using bitmap format at 300 dotsper-inch resolution (RES). They were then cropped and masked to leave only gray-scale images without additional information. Three sonologists reviewed the image groups. Each sonologist was board-certified in emergency medicine, had extensive US training and hospital credentialing in US, and was American Registry of Diagnostic Medical Sonographers–certified. Pretesting showed that approximately 49 image pairs were repeatedly evaluated during 45 minutes. The 45-minute cutoff was chosen for reviewer convenience. Image pairs were selected for review using a random-number generator (Figure 1). Each reviewer rated every image pair for RES, detail (DET), and total IQ as previously defined in the literature using a ten-point Likert scale.3 Images appeared side by side on the same screen. A score of 10 was set as the best possible rating for each category. Measures. All reviews took place on a Web site specifically designed for this project by loracs.com. Reviewers used high-speed Internet connections and
RESULTS Five healthy models were used for image acquisition. No equipment failures were encountered during study performance. Each image pair consisted of images from similar sonographic windows on the same patient model using the two different US machines. During the review process, one computer failure occurred, necessitating a reviewer to repeat the Likert score entry for one image pair. The mean GE 400 RES Likert scale score was 6.8 as compared with 6.3 for the SonoSite 180 Plus. The 0.5 (95% CI = 0.1 to 1.1) difference was statistically significant, with p = 0.01. The mean GE 400 DETscore was 6.8 as compared with a mean of 6.3 for the 180 Plus. The difference of 0.5 (95% CI = 0.13 to 1.1) was not statistically significant (p = 0.06). The mean IQ score for the GE 400 was 6.6 as compared with 6.0 for the 180 Plus. The difference of 0.6 (95% CI = 0.2 to 1.2) was statistically significant, with p = 0.01. There was good interobserver agreement, with a kappa of 0.71 (95% CI = 0.67 to 0.78). Upon specific query, reviewers were unable to identify images as belonging to a specific machine.
DISCUSSION The challenge of combining ease of use, portability/ mobility, and IQ is a significant one for US machine manufacturers. The smaller the machine, the easier it is for the EP to move between patients and introduce US into new environments such as ground and air ambulances, remote locations, or even triage stations such as first aid areas at large sporting or social events.4–6 We had previously expressed concern that potential falsenegative studies could result if small portable units are used.1 A counterargument is that the types of focused US examinations performed in emergent situations may not require the same detail as a complete diagnostic study in a radiology suite. However, even subtle amounts of fluid in the abdomen, for example, are important to detect in a trauma patient. A number of factors can affect US IQ, including patient habitus and the equipment used. Patients who are obese or have very anhydrous subcutaneous fat may prove exceedingly difficult to examine with US.7
780
Blaivas et al.
d
ULTRASOUND MACHINE COMPARISON
Figure 1. A comparison of a Morison’s pouch image from the same model obtained in the same orientation. The left shows the SonoSite 180 Plus image and the right shows the GE 400 Pro image.
This is highlighted by the rate of indeterminate scans that has been reported over a range of applications such as trauma and vascular US examinations.8,9 To date, several manufacturers have introduced portable US equipment. According to the company, SonoSite equipment is designed to be light, portable, and durable. Another manufacturer (GE) has recently started marketing to EPs and other nonradiology US users. The GE 400 Pro is approximately three times more expensive than the SonoSite 180 Plus and weighs about 40 times as much. If image quality of the SonoSite is significantly inferior to that of the GE 400 Pro, such a discrepancy should be taken into account before investing in a US for ED use. In order to compare these US options, we chose typical US examinations that would be performed by EPs, including the gallbladder, abdominal aorta, trauma US views, and peripheral and central veins. No attempt was made to differentiate between US studies and transducer type. Although true patients were not evaluated, the results can still be useful to the EP in comparing US RESs, DETs, and IQs. A previous experience, which prompted this study, looked at only three cases in an uncontrolled setting with no blinded reviewers and no side-by-side comparison.1 In this study, we compared multiple images from similar sonographic windows over a range of examinations commonly used in emergency US as noted previously. The differences in two out of three (IQ and RES) categories were statistically significant but minor on a ten-point Likert scale. For the third category (DET), no statistical difference existed at all between the two US machines. Based on our study results, small, portable US machines appear to compare favorably with larger machines in terms of image DET provided. Small, statistical differences in IQ and RES were noted but are not likely to be of clinical significance. In most situa-
tions, it is best to compare US machines head to head in the ED when contemplating the purchase of equipment. The actual scanning of real patients and moving the machines from room to room will allow the US director to make rapid conclusions about which machine will fit the particular needs of his or her ED.
LIMITATIONS This study has several limitations, including its use of healthy models rather than actual patients. Only one of the models was of normal habitus, whereas the rest were overweight, purely by chance. It should be noted that no comparisons were made using morbidly obese models. There is a potential for degradation of IQ with the multiple steps before evaluation by the reviewers. However, all of the steps were digital, and information loss was not expected because RES settings were kept high. Lighting conditions and monitor settings for each reviewer were not strictly controlled other than requesting low-light conditions. Conclusions regarding the clinical significance of our findings cannot be made.
CONCLUSIONS There was a small but statistically significant difference between General Electric 400 and SonoSite 180 Plus in IQ and RES, but not DET. The clinical importance of this finding is uncertain. References 1. Blaivas M, Theodoro D. Intraperitoneal blood missed on a FAST examination using portable ultrasound. Am J Emerg Med. 2002; 20:105–7. 2. American College of Emergency Physicians. ACEP emergency ultrasound guidelines—2001. Ann Emerg Med. 2001; 38:470–81.
ACAD EMERG MED
d
July 2004, Vol. 11, No. 7
d
www.aemj.org
3. Blaivas M, DeBehnke D, Sierzenski PR, Phelan MB. Tissue harmonic imaging improves organ visualization in trauma ultrasound when compared with standard ultrasound mode. Acad Emerg Med. 2002; 9:48–53. 4. Melanson SW, McCarthy J, Stromski CJ, Kostenbader J, Heller M. Aeromedical trauma sonography by flight crews with a miniature ultrasound unit. Prehosp Emerg Care. 2001; 5:399–402. 5. Kirkpatrick AW, Brown R, Diebel LN, Nicolaou S, Marshburn T, Dulchavsky SA. Rapid diagnosis of an ulnar fracture with portable hand-held ultrasound. Mil Med. 2003; 168:312–3.
781 6. Moore CL. Utility of portable ultrasound in patient care in a remote area of Nicaragua [abstract]. Ultrasound Med Biol. 2003; 29(5 suppl):S152. 7. Tanabe K, Belohlavek M, Greenleaf JF, Seward JB. Tissue harmonic imaging: experimental analysis of the mechanism of image improvement. Jpn Circ J. 2000; 64:202–6. 8. Frazee BW, Snoey ER, Levitt A. Emergency department compression ultrasound to diagnose proximal deep vein thrombosis. J Emerg Med. 2001; 20:107–12. 9. Boulanger BR, Brenneman FD, Kirkpatrick AW, McLellan BA, Nathens AB. The indeterminate abdominal sonogram in multisystem blunt trauma. J Trauma. 1998; 45:52–6.
