THE ANATOMICAL RECORD (PART B: NEW ANAT.) 272B:91–97, 2003
TUTORIAL
Using a Modified Standard Microscope to Generate Virtual Slides DAVID J. ROMER,* KURTIS H. YEARSLEY,
AND
LEONA W. AYERS
A standard microscope was reconfigured as a virtual slide generator by adding a Prior Scientific H101 robotic stage with H29 controller and 0.1-m linear scales and a Hitachi HV-C20 3CCD camera. Media Cybernetics Image Pro Plus version 4 (IP4) software controlled stage movement in the X-, Y-, and Z-axis, whereas a Media Cybernetics Pro-Series Capture Kit captured images at 640 ⴛ 480 pixels. Stage calibration, scanning algorithms, storage requirements, and viewing modes were standardized. IP4 was used to montage the captured images into a large virtual slide image that was subsequently saved in TIF or JPEG format. Virtual slides were viewed at the workstation using the IP4 viewer as well as Adobe Photoshop and Kodak Imaging. MGI Zoom Server delivered the virtual slides to the Internet, and MicroBrightField’s Neuroinformatica viewing software provided a browser-based virtual microscope interface together with labeling tools for annotating virtual slides. The images were served from a Windows 2000 platform with 2 GB RAM, 500 GB of disk storage, and a 1.0 GHz P4 processor. To conserve disk space on the image server, TIF files were converted to the FlashPix (FPX) file format using a compression ratio of 10:1. By using 4ⴛ, 10ⴛ, 20ⴛ, and 40ⴛ objectives, very large gigapixel images of tissue whole-mounts and tissue arrays with high quality and morphologic detail are now being generated for teaching, publication, research, and morphometric analysis. Technical details and a demonstration of our system can be found on the Web at http://virtualmicroscope.osu.edu. Anat Rec (Part B: New Anat) 272B:91–97, 2003. © 2003 Wiley-Liss, Inc. KEY WORDS: virtual slide; virtual microscope; microscopy; digital imaging; computer-assisted learning; CAL; medical curriculum; teaching
INTRODUCTION Static film images of tissue sections can currently be replaced in a variety Mr. Romer is a Systems Engineer in the Department of Pathology at The Ohio State University. His research interests include scanning microscopy technologies, tissue array data and imaging systems, and Web-based medical education technologies. Dr. Yearsley, member of Sigma Xi, is presently conducting research in Chronic Pathobiology in Alografts for quantification of fibrosis and other parameters for The Ohio State University Department of Pathology. His research includes quantitative analysis of very large digital images. Dr. Ayers is Professor of Pathology at The Ohio State University College of Medicine and Public Health with teaching and research at University Hospitals, the James Cancer Hospital, and Solove Research Institute. Her research interests are in infection-related diseases, including malignancies, and tissue array technology, including array digitization. *Correspondence to: David J. Romer, Department of Pathology, The Ohio State University, 129 Hamilton Hall, 1645 Neil Avenue, Columbus, OH 43210. Fax: 614-292-7072; E-mail:
[email protected] DOI 10.1002/ar.b.10017 Published online in Wiley InterScience (www.interscience.wiley.com).
© 2003 Wiley-Liss, Inc.
of settings by computer-based digital images. Such digital images can be used independently or assembled together to form a large digital mosaic of a salient histological feature or tissue whole-mount section. Efforts to produce high-quality digital gray scale mosaics from microscopic specimens were successful in the early 1980s (Silag and Gil, 1985) and continued into the 1990s from advances in image acquisition and mosaicing techniques (Westerkamp and Gahm, 1993; Swidbert, 1997). Today, high-quality digital color mosaics of histological features and tissue wholemount sections are commonplace and often referred to as “virtual slides” and deployed over a network for virtual microscopy (Harris et al., 2001; Heidger et al., 2002). The virtual slide is typically viewed with a virtual microscope application that enables a microscopist to navigate the virtual slide on a computer monitor screen in a manner that simulates a standard microscope as shown in Figure 1. Simulated magnification and navigation capabilities
(pan and zoom) are provided through a mouse interface, but illumination and focus capabilities typically are not available. Systems that create virtual slides—virtual slide generators—vary in complexity but, in general, are designed to scan a region of interest (ROI), programmatically acquire a series of small images from an analog or digital camera, and tile the small images into a very large montage image. The montage file size can range from a few megabytes to several gigabytes. Until recent years, the large image has been difficult to create, store, transmit, and view. Advances in image acquisition systems, reductions in disk storage costs, and improvements in broadband connectivity have greatly improved the management, distribution, and viewing of these images. Browser-based virtual microscopes are now commonplace, and virtual slides are now easily shared using a variety of storage and retrieval media. Commercial systems such as ScanScope (Aperio Technologies) and BLISS (Bacus Laboratories, Inc., Slide Scanner) acquire overlapping
92 THE ANATOMICAL RECORD (PART B: NEW ANAT.)
TUTORIAL
40⫻. Objectives were chosen to minimize spherical and chromatic aberrations and, thus, produce very flat images for tiling. The 10⫻, 20⫻, and 40⫻ objectives were selected for scanning, and the 2⫻ and 4⫻ objectives were selected primarily for ROI setup. To the microscope we added a Prior Scientific H101 stage with H29 controller and linear scales with encoders for positioning the stage with high accuracy and resolution as illustrated in Figure 2. The stage has an XY travel of 108 ⫻ 73 mm (3 ⫻ 4 inches) and accommodates a traditional glass slide (75 ⫻ 25 mm). The linear scales have a resolution of 0.1 m, which we believed necessary to accommodate the 0.37 m optical resolution of the 40⫻ objective. We chose Media Cybernetics Image Pro Plus Version 4 (IP4) to control stage movement in the X-, Y-, and Z-axis. An auto focus option was available for the Z-axis but not implemented.
Image Acquisition
Figure 1. Screen snapshots of a virtual slide and browser-based virtual microscope. A: Lowpower (1.25⫻) view of appendix, also illustrating mouse-driven navigation window in upper right corner and panning cursor on large image. B: Change in magnification to 20⫻ and navigation to goblet cells as indicated by the small yellow square in the navigation window.
images and use intelligent pattern recognition techniques to stitch the images together. The system we developed, however, relies solely on precise stage positioning to acquire nonoverlapping images. We discuss the components used for converting a standard microscope, as well as the tools we selected for viewing, annotating, and analyzing virtual slides.
MATERIALS, METHODS, AND DETAILED PROCEDURES Microscope and Robotic Stage Selection We selected an Olympus BX51 microscope with the following objectives: Plan 2⫻ and 4⫻, U Plan Apochromat 10⫻ and 20⫻, and U Apochromat
The Hitachi HV-C20 3CCD camera is a three-chip (red, green, blue) chargecoupled device (CCD) and was selected for its simplicity, image quality, and history of use on our video microscopes. The camera was coupled to the microscope with a 0.5⫻ C-mount adapter. We interfaced it to a Media Cybernetics Pro-Series Capture Kit to capture images at 640 ⫻ 480 pixels. The camera was set to 1/2,000-second manual shutter, auto gain was disabled, and the white balance was set to 3,200 K. Gross adjustments to the illumination were made by means of neutral density filters and minor adjustments were made by means of the lamp intensity control near 3,200 K. The aperture iris diaphragm (located on condenser, directly below slide) was set to approximately 80% of the numerical aperture of the objective used for scanning. The field iris (located on base) was set to produce the sharpest image without producing vignetting. This setup provided good contrast and helped moderate CCD blooming effects, which can occur when the illumination is too great, causing CCD saturation. The workstation consisted of a 1.4 GHz P4 processor, 2 GB RAM,
TUTORIAL
THE ANATOMICAL RECORD (PART B: NEW ANAT.) 93
Figure 2. Left: Modified standard microscope with camera (A), 0.5⫻ adapter (B), X-axis motor (C), Y-axis motor (D), Z-axis motor (E), linear scales (F), slide leveling assembly and stage (G), and joystick (H). Right: X-axis linear encoder and scale strip for position feedback (A), X-axis left/right over travel limit switches (B), glass slide holder (C), and nylon screw (D), for stage leveling (one of four). [Color figure can be viewed in the online issue, which is available at www.interscience.wiley.com.]
60 GB disk storage, and Windows 2000 was chosen as the operating system. Images captured during the scanning process described below required additional manipulations before insertion into the montage. Because the border pixels in the 640 ⫻ 480 image were not reliable, we developed macros using the IP4 Visual Basic language to crop the acquired 640 ⫻ 480 image down to 632 ⫻ 472 during the scan. We added a background correction macro to remove image artifacts such as vignetting and variations in color and luminance from each image as illustrated in Figure 3. Both macros were implemented immediately after image acquisition and before tiling. Additional information on the photomicrographic technique we chose can be found at the Olympus Microscope Resource Center at http://www.olympusmicro.com.
Camera Alignment (Squaring) Of paramount importance to image acquisition is the alignment between the camera and stage as shown in Figure 4. Squaring the camera with the
stage is an iterative process that involves rotating the camera slightly, scanning an ROI to create a montage, inspecting the montage image, and then iterating the process until satisfactory tile-to-tile alignment is observed. To facilitate alignment, we used a 40 line/mm (12.5 m line width) Ronchi ruling, which is a precision glass target with equally spaced parallel lines typically used for evaluating resolution and field distortion. By using a 40⫻ objective, we scanned a small portion of the Ronchi ruling and inspected the montage image for indications of vertical skewing. We then rotated the camera slightly and iterated the process until skewing was minimal.
Stage Leveling The stage must be nearly level from front-to-back and left-to-right to produce images that are tightly focused. If the stage slopes excessively from left-to-right, then an acquired image will be in focus at one side and out of focus at the opposite, which creates a problem during tiling. The stage does not need to be perfectly level, how-
ever, because programmatic focusing is performed during the scan to account for minor variations in level as well as tissue flatness. To level the stage, we used a Ronchi ruling and first observed the quality of focus at the far four corners of the ruling area, which was approximately 25 ⫻ 12 mm. By using only the Z-axis control, we focused on one of the four corners. We then moved the stage approximately 25 mm along the X-axis and focused by adjusting only the stage leveling screws. We moved the stage approximately 12 mm along the Y-axis and again focused by adjusting only the stage leveling screws. With all four corners reasonably in focus, we moved the stage again to each of the four corners and focused using the Z-axis control, recording the Z-axis position data for each corner. From these data, we calculated the stage slope from front-to-back and also leftto-right. We achieved a slope of approximately 1 m/mm in each direction. Thus, for our 40⫻ objective with a camera X-axis field of view (FOV) of 0.39 mm, the variation in focus across a static 640 ⫻ 480 image was deter-
94 THE ANATOMICAL RECORD (PART B: NEW ANAT.)
TUTORIAL
5). First, to determine approximate X and Y displacement values for each of the objects, we used a 2.00-mm stage micrometer with 0.01-mm divisions, captured a 640 ⫻ 480 image, and then measured the camera X-axis FOV (Table 1). For our 10⫻, 20⫻, and 40⫻ objectives, X-axis FOVs were approximately 1.24 mm, 0.63 mm, and 0.31 mm, respectively. For a cropped 632 ⫻ 472 pixel image at 10⫻, this corresponded to a displacement of approximately 0.002 mm/pixel (2 m/ pixel). By using this as a seed value in the scanning algorithm, a preliminary montage of the stage micrometer was produced. After inspection of the montage seam quality, the seed value was refined and the process was iterated until the seam was minimized. For 10⫻, the final value for X-axis displacement was determined to be 1.944 m/pixel, or 1.228 mm/tile. The process was repeated for the 20⫻ and 40⫻ objectives and displacements were found to be 0.972 and 0.485 m/ pixel, respectively. At 40⫻, seams that were less than 1.0 m were obtained, as shown in Figure 5.
Scanning Algorithms
Figure 3. Background correction removes artifacts from acquired image before tiling. Top image (A) is a mosaic of tiles that have not been corrected for variations in color and luminance (note vignetting). Bottom image (E) is a mosaic of tiles that have been corrected using the “normal” background image shown in the center series of images (B–D). This middle series of images demonstrates the correction process, which uses a background image to correct each newly acquired image before tiling. Images in this figure were purposely underexposed to demonstrate efficiency and range of correction algorithm. [Color figure can be viewed in the online issue, which is available at www.interscience. wiley.com.]
mined to be 0.39 m, which was unnoticeable in the montage image.
Stage Calibration Because images were not acquired using the overlap method, our system
was very dependent on precise stage positioning for capturing adjacent images. An iterative process of scanning and inspection of the montage seams determined precise XY displacement values for each objective (see Figure
Although IP4 includes menus for scanning images and creating montages, we created a user interface and scanning algorithm using IP4 macros to produce montages more efficiently and with better quality. Our scanning algorithm produced raster scans over rectangular ROIs with no overlapping among images. To perform a scan, the user clicks on a menu item and follows the directions provided through a short series of dialog boxes. First, the user defines an ROI by moving the stage to the four edges of a feature or wholemount. After this, a properly sized blank IP4 workspace is created for the large montage file. Next, the user focuses at three different points on the feature to locate a plane in the Z-axis that best fits the feature. The XYZ position values are recorded at the three points and applied to an equation that returns a new value of Z for any XY position during the scan. We refer to this as programmatic focusing rather than auto focusing, which is known to increase the scan time and sometimes produce focusing artifacts in the final
TUTORIAL
THE ANATOMICAL RECORD (PART B: NEW ANAT.) 95
Zoom Server and MicroBrightField’s Neuroinformatica viewing software, which also catalogs images in a MYSQL database. The images were served from a Windows 2000 platform supported with 2 GB RAM, 500 GB of disk storage, and a 1.0 GHz P4 processor. The Neuroinformatica software also allows users to annotate virtual slides with drawings and text by means of the virtual microscope interface. Additional information on the viewing software can be found at the MicroBrightField web site http://www.microbrightfield.com.
PRACTICAL ASPECTS
Figure 4. Improper alignment between camera and stage (A), resulting in poor tiling of nine adjacent images of a Ronchi ruling (B) and histological feature (C) scanned at high power. Proper alignment of same (D–F). [Color figure can be viewed in the online issue, which is available at www.interscience.wiley.com.]
montage. Finally, the user is asked to move the stage to an area void of specimen and uniformly lighted to acquire an image for background correction. After the last step, the scan commences in a raster mode from right to left and bottom to top. Stage translations are not continuous and are descretized according to the camera FOV for the scanning power. For example, scanning a 20 ⫻ 20 mm ROI with a 10⫻ objective causes the stage to move 1.228 mm in the X-axis and then pause briefly for image acquisition before moving again. In this example, the process would be repeated 17 times along the X-axis, because 20 is not evenly divisible by 1.228, and fractional frames (i.e., 16.3) are not possible. Each acquired image, thus, is stable and suitable for cropping, correcting, and copying into the mon-
tage workspace with no separation between images. At completion, the montage file is saved as a TIF or JPEG. We also developed a scanning algorithm that does not produce a montage in the IP4 workspace but instead acquires images and saves them to disk with a row– column name, like 001-001.JPG and 001-002.JPG. These images have been cropped and background-corrected and can be tiled outside of the IP4 environment.
Presentation and Viewing The IP4 workspace is used to view the virtual slide at the workstation, but Adobe Photoshop and Kodak Imaging, integral to Windows 2000, can also be used. To present virtual slides on the Internet, we used the MGI
Adding the Prior robotic stage to the standard microscope required no special tools and was accomplished in a few minutes. The IP4 stage control software installed quickly, and after studying the IP4 manuals, the first virtual slide at 10⫻ was generated a few hours later using standard IP4 menus. The stage hardware and IP4 software were designed for compatibility and ease-of-installation and use. The first virtual slide at 40⫻ was generated several hours later, due to the iterative procedure of leveling the stage and determining the proper stage calibration (pixel/step), which are both very critical at 40⫻. Our virtual slide generator continues to produce very high quality images from tissue sections, including tissue arrays. Camera alignment is difficult to setup initially but does not appear to drift unless the camera is jarred severely or the equipment is moved. Stage calibration, on the other hand, does drift and requires periodic adjustment to a software parameter if high-power objectives are used for diagnostic or image analysis purposes. Specifically, we have observed that large ambient temperature swings of 10 –20°F affect the thermal expansion of sensitive stage and camera components and, thus, affect the resolution values used for stage calibration. For example, for our 40⫻ objective, we have observed the stage resolution varies between 0.490 m/pixel and 0.475 m/pixel. Future plans include automating the stage calibration procedure by using a stage micrometer and IP4 image analysis routines to determine stage resolution values pro-
96 THE ANATOMICAL RECORD (PART B: NEW ANAT.)
TUTORIAL
Figure 5. Montage of images from a 2.00-mm stage micrometer at high power (40⫻). Top image was generated with a stage calibration value set too low, resulting in separation of tiles at approximately 0.27 mm as indicated by an additional division line between 0.25 and 0.30 mm. Middle image shows correct montaging. Bottom image shows overlapping of tiles at 0.27 mm, and loss of a division line between 0.25 and 3.0 mm. [Color figure can be viewed in the online issue, which is available at www. interscience.wiley.com.]
grammatically and at the discretion of the operator. The effect of variations in the stage calibration is shown in Figure 5. Because our installation of IP4 could not create montages greater than 220 megapixels— corresponding to 831 mm2 at 10⫻, 207 mm2 at 20⫻, and 51 mm2 at 40⫻—we developed an IP4 macro to save individual images to disk and then developed a program to tile these images outside of the IP4 workspace. By using this approach, the size limitation of the montage depends only on available storage capacity. The time it takes to scan a feature depends not only on the size of the ROI but also on the objective used as presented in Table 1. We observed scan coverage rates of 33.6 mm2/min at 10⫻, 9.0 mm2/min at 20⫻, and 2.4 mm2/min at 40⫻. Doubling the scanning power roughly quadrupled the
number of acquired images, the time to scan the images, and the amount of required storage. Scanning an ROI of 20 ⫻ 20mm would require approximately 12 minutes at 10⫻, 44 minutes at 20⫻, and 166 minutes at 40⫻. The same ROI would produce uncompressed RGB image files of
318 MB at 10⫻, 1,207 MB at 20⫻ (1.2 GB), and 5,101 MB (5.1 GB) at 40⫻. Scanned images are initially saved in TIF format, which has a size limitation of approximately 4 GB. Our experience has been limited to file sizes less than 1 GB. To conserve disk space on the Internet image server the TIF file is converted to a FlashPix (FPX) file using a compression ration of 10:1. Further reduction in file size may be realized in the future with JPEG 2000, which promises similar quality images but with improved compression ratios of 20:1. Additional information on JPEG 2000 can be found at http:// www.jpeg.org/JPEG2000.html. Creating montages of blood smears or targets that are not flat presents a problem, because the scanning algorithm uses programmatic focusing based on 2D planar geometry. We have been successful in scanning only sloping sections that “rise” or “fall” and are mostly planar; however, these are relatively small ROIs. Future plans include modifying the focusing method to accommodate nonplanar targets by constructing a 3D surface mesh of the target as a basis for focusing.
DISCUSSION A virtual slide generator can produce virtual slides for many different uses. First, in conjunction with the virtual microscope (Ferreira et al., 1997), it is considered an emerging learning aid in histology and pathology laboratory instruction (Heidger et al., 2002). Teaching with conventional glass histology slides is difficult, because they are breakable, difficult to distribute,
TABLE 1. Virtual Slide Generator Characteristics Nominal Coverage Rate (mm2/min)
20⫻20 mm ROI Scan Time (min)
20⫻20 mm File Size(2)
Objective
Resolution (um/pixel)
Camera(1) X-axis Field of View (FOV) (um)
10 20 40
1.944 0.972 0.485
1,240 620 310
33.6 9.0 2.4
12 44 166
318 1,270 5,101
(Mb)
Notes: (1) Camera is couple to microscope via 0.5X adapter. (2) Uncompressed file size calculated from resolution (um/pixel) for 3-color (RGB) image.
TUTORIAL
and fade over time. A virtual slide image, on the other hand, is an indestructible digital medium with extremely stable image quality and is easily accessible by means of the Internet (Harris et al., 2001). Second, the virtual slide can be used in telepathology, where published studies indicate that the degree of concordance between viewing digital microscopy images and viewing histology through conventional microscope eyepieces is high enough for this to be feasible (Leong and McGee, 2001). Third, if the ROI is the whole-mount, then the corresponding virtual slide will be a very high-resolution image of a low-power view of the overall tissue architecture, which is difficult to obtain using normal photomicrographic techniques, but which is highly desirable for presentation, publication, and teaching. Fourth, because the virtual slide is rich in morphologic detail, it is well suited to quantitative morphometric analysis using standard analysis tools like those available in IP4. For example, success has been achieved scanning specimens at 40⫻, to produce large montage images of 14,000 ⫻ 11,000 pixels in approximately 15 minutes. When viewed at 10% of original size, there is enough detail present to subsequently select an area for further analysis. From the selected area, 5 to 10 small ROIs are determined and subimages of 640 ⫻ 480 pixels are extracted using cut-andpaste, and saved for morphometric analysis. By using macros written in IP4 Basic, data collection and quantitative analysis usually takes less than a minute per ROI. Specific analysis has been performed on murine heart, lung, liver, and skin allograft tissues to study the amount of fibrosis, macrophage infiltration, and/or arterial hyperplasia. Overall image analysis using the VSG is less time consuming
THE ANATOMICAL RECORD (PART B: NEW ANAT.) 97
and much more accurate than visually rating each specimen manually. Finally, the authors are deploying virtual microscopy in the research setting to study and document virtual slides of tissue arrays. Investigators will be able to navigate to a core of interest, store, and retrieve visual as well as bioinformation from a database. Sharing images and reviewing interpretations on-line with collaborators and reviewers are of special interest. The system continues to evolve while it produces very useful virtual slides. We recently used virtual slides at the “2nd Annual Update Course in Surgical Pathology” sponsored by the
Our virtual slide generator continues to produce very highquality images from tissue sections including tissue arrays. Ohio State University Department of Pathology in Columbus, OH, August 25–27, 2002 (Romer and Suster, 2003). Medium-power images (10⫻) are the easiest and fastest to acquire and produce excellent teaching slides as well as high-quality megapixel images for presentation, publication, and research. The virtual slide generator has been in operation since November 2001 and has not required any mechanical adjustments, other than those required after equipment relocation. The Prior stage and IP4 software cost under $20,000 and, once properly configured, produce images comparable to commercial systems costing over $60,000. Addition of
these components can extend the usefulness of a standard video microscope by enabling educators and researchers to acquire very large digital images for many different purposes.
ACKNOWLEDGMENTS We thank all those who have helped in the design of our system, and in particular Dr. Saul Suster, Director of Anatomic Pathology, for enthusiastically supporting our technology by using virtual slides at national conferences, and also Dr. Charles Hitchcock, Director of Medical Education, for identifying interesting slides to evaluate and for introducing students to this technology. This project was supported in part by a grant (CA66531-07) from NCI/NIH.
LITERATURE CITED Ferreira R, Moon B, Humphries J, et al. 1997. The virtual microscope. Proceedings AMIA Annual Fall Symposium. p 449 – 453. Harris T, Leaven T, Heidger P, Kreiter C, Duncan J, Dick F. 2001. Comparison of a virtual microscope laboratory to a regular microscope laboratory for teaching histology. Anat Rec 265:10 –14. Heidger PM, Dee F, Consoer D, Leaven T, Duncan J, Kreiter. 2002. Integrated approach to teaching and testing in histology with real and virtual imaging. Anat Rec 269:107–112. Leong FJ, McGee JO. 2001. Automated complete slide digitization: A medium for simultaneous viewing by multiple pathologists. J Pathol 195:508 –514. Romer D, Suster S. 2003. Use of virtual microscopy for didactive live-audience presentation in anatomic pathology. Anal Diag Pathol 7(1):67–72. Silage DA, Gil J. 1985. Digital image tiles: A method for the processing large sections. J Microsc 138(Pt 2):221–227. Swidbert RO. 1997. Acquisition of highresolution digital images in video microscopy: Automated image mosaicking on a desktop microcomputer. Microsc Res Tech 38:335–339. Westerkamp D, Gahm T. 1993. Non-distorted assemblage of the digital images of adjacent fields in histological sections. Anal Cell Pathol 5:235–247.
