Micron 38 (2007) 492–499 www.elsevier.com/locate/micron
Use of Autostitch for automatic stitching of microscope images Bin Ma a,*, Timo Zimmermann b, Manfred Rohde c, Simon Winkelbach d, Feng He a, Werner Lindenmaier a, Kurt E.J. Dittmar a a
Division of Molecular Biotechnology, German Research Centre of Biotechnology, Mascheroder Weg 1, D-38124 Braunschweig, Germany b Advanced Light Microscopy Facility, EMBL Heidelberg, Heidelberg, Germany c Department of Microbial Pathogencity, German Research Centre of Biotechnology, Braunschweig, Germany d Institute for Robotics and Process Control, Technical University of Braunschweig, Braunschweig, Germany Received 9 June 2006; received in revised form 26 July 2006; accepted 28 July 2006
Abstract Image stitching is the process of combining multiple images to produce a panorama or larger image. In many biomedical studies, including those of cancer and infection, the use of this approach is highly desirable in order to acquire large areas of certain structures or whole sections, while retaining microscopic resolution. In this study, we describe the application of Autostitch, viz. software that is normally used for the generation of panoramas in photography, in the seamless stitching of microscope images. First, we tested this software on image sets manually acquired by normal light microscopy and compared the performance with a manual stitching approach performed with Paint Shop Pro. Secondly, this software was applied to an image stack acquired by an automatic microscope. The stitching results were then compared with that generated by a selfprogrammed rectangular tiling macro integrated in Image J. Thirdly, this program was applied in the image stitching of images from electron microscopy. Thus, the automatic stitching program described here may find applications in convenient image stitching and virtual microscopy in the biomedical research. # 2006 Elsevier Ltd. All rights reserved. Keywords: Autostitch; Image stitching; Automatic microscopy; Image masaicing; Virtual microscopy
1. Introduction Image stitching is the process of combining multiple images to produce a panorama or larger image (Shum and Szeliski, 1997; Chen and Klette, 1999; Zomet et al., 2006) and has found many applications in the acquisition of high resolution images. For example, for bright-field or fluorescence microscopy, analysis of the whole section of several centimetres at high resolution cannot be performed, even at a low power objective and even if cameras with high resolution are applied. Image stitching is a powerful approach for solving this problem by creating a single composite image by overlapping multiple images acquire from different parts of the section with high resolution. The image sets needed for the generation of the large single images can be acquired either manually with a normal microscope or with an automatic microscope with a motorised x, y stage. If small numbers of images are handled,
* Corresponding author. Tel.: +49 531 6181 293; fax: +49 531 6181 202. E-mail address:
[email protected] (B. Ma). 0968-4328/$ – see front matter # 2006 Elsevier Ltd. All rights reserved. doi:10.1016/j.micron.2006.07.027
the image sets can be acquired manually and stitched together either manually with normal image processing software or with automatic stitching software. In this case, manual stitching can produce good results, although it is difficult and time consuming. If large numbers of sections are handled, manual stitching becomes almost impossible and automatic mosaicing must be applied. In addition, the acquisition of data must be performed by a microscope system with motorised x, y stage. If stage moves along the x, y direction precisely, the mosaicing can be performed through Tile Scan, i.e. by arrangement of the images one by one in their natural position. This approach can be applied in confocal microscopy to generate a large single image. However, in practice, the motorised stage does not move as exactly as we had expected and this leads to misalignment of adjacent images resulting in the failure of Tile Mosaicing to produce a seamless image. Computer software is often used to interpolate the final image if the component images are not in precise alignment. Therefore, mosaicing with blending, i.e. assembling a set of overlapped images to generate seamless images, must be performed (Bhosle et al., 2002).
B. Ma et al. / Micron 38 (2007) 492–499
The other application is image stitching for virtual microscopy; this has found application in histology and pathology (Lundin et al., 2004). The aim of virtual microscopy is to replace traditional light microscopy by personal computers in some scenarios (Fontelo et al., 2005). The microscope slide is automatically scanned with a normal microscope and the large data set is stored, processed and browsed as if it were physically present. Furthermore, if virtual slices from serial virtual slides are available, they can be used for three-dimensional reconstructions of tissue or organs. Because of the large number of images and extremely large data sizes, image acquisition must be performed with a microscope equipped with a motorised x, y stage and corresponding software for data acquisition, processing and stitching. Normally the process of image stitching is divided into three steps: image registration (Brown, 1992) and image merging and blending. During the image registration, multi-images are compared to find the translations that can be used for the alignment of images. After registration, these images are merged together to form a single image. This step of image merging is performed to make the transition between adjacent images visually undetectable. In most cases neighbouring image edges show undesirable intensity discrepancies. These variations in intensity are present even when registration of two images is almost perfect to the eye. In order to eliminate such effects and improve visual quality of the composite image, a blending or ‘feathering’ algorithm is applied. The blending is to choose the final value of a pixel in a location where two images are overlapping. The easiest algorithm, which is called ‘‘noblend mode’’, is just to choose the value of one image and nothing from others pictures. Another algorithm, which is called linear blending, is to use a mean value of every pixels value for the final pixel value on an overlapping zone. More advanced algorithm, such as multi-band blend algorithm can also be applied for the blending. These kinds of algorithms decompose the picture using high and low filter, blend them apart and recompose every layers. Autostitch is a fully automatic two-dimensional image stitcher capable of stitching full view panoramas without any user input whatsoever. The algorithm used for extraction of feature for the registration of two images is scale-invariant feature transform (SIFT) (Lowe, 2004); for blending algorithms, two options are available: linear blending and multiband blending. The multi-band blending used in this approach is pyramid-based blending method (Adelson et al., 1984). In this study, application of this software is extended to the biomedical research field for the stitching of microscope images obtained by normal microscopy, automatic microscopy and electron microscopy. 2. Materials and methods 2.1. Mice Balb/c mice (8–12 weeks old) were obtained from Harlan Winkelmann and used in all the reported experiments. They were maintained under specific pathogen-free conditions in
493
the animal facility of the German Research Centre of Biotechnology. 2.2. Section preparation and haematoxylin–eosin staining Mice were sacrificed by CO2 narcosis and organs were quickly removed, embedded in Tissue-Tek1 OCT Compound (Sakura Finetek, Torrance, CA, USA) and snap-frozen in liquid nitrogen. Cryosections (7 or 10 mm) were prepared in a Reichert-Jung 2800 cryostat. Cryosections were mounted on glass slides (Menzel Glaeser, Braunschweig, Germany) coated with polylysine; they were air-dried for 30 min, fixed in 10% neutral buffered formaldehyde (Carl Roth, Karlsruhe, Germany) for 20 min, washed with running tap water for 3 min, stained with Mayer’s haemalaun (Merck, Darmstadt, Germany) for 10 min, differentiated in freshly prepared 3.75% HCl (in 70% ethanol), washed again with running tap water for 5 min, dehydrated in 70% and 90% ethanol for 2 min, respectively, stained with alcoholic eosin (containing 0.1% phloxine in 90% ethanol; Sigma–Aldrich, Steinheim, Germany) for 5 min, dehydrated in 100% ethanol and xylene and finally mounted with Entellan New (Merck, Darmstadt, Germany) mounting medium. 2.3. Image stitching and mosaicing (A) Manual acquisition for the haematoxylin–eosin (HE)stained sections was performed with a Zeiss Microscope S 100 (Carl Zeiss, Jena, Germany) equipped with a Zeiss Axicam HRC colour camera (Carl Zeiss, Jena, Germany). The images were taken by 5 or 10 objective lens. (B) A Zeiss Axiovert 135 microscope (Carl Zeiss, Jena, Germany), equipped with an Bioprecision automated slide positioning stage operated through a LEP MAC 5000 controller (LUDL Electronic Products Ltd., Hawthorne, NY, USA) and a Coolsnap HQ camera (Photometrics, Tucson, AZ, USA) was used for the automatic acquisition of images from HE-stained sections of lymph node. The repetition accuracy of the positioning stage is 0.75 mm. The images were taken by a 20 0.75 NA objective lens. The software used was MetaMorph Version 6 (Molecular Devices Corp., Sunnydale, CA, USA). The complete system was set up by Visitron GmbH (Puchheim, Germany). 2.4. Electron microscopy Lymph node were fixed with 2% glutaraldehyde and 5% formaldehyde for 12 h at 4 8C, washed with TE-buffer (20 mM Tris, 2 mM EDTA, pH 7.0) and dehydrated with a graded series of acetone before being critical-point dried with liquid CO2. The dried samples were mounted onto sticky carbon attached to aluminium sample holders. Samples were fractured by pressing against another sample holder with sticky carbon and separating the two holders. The freshly fractured sample surfaces were sputter coated with a thin gold-film. Samples were examined in a Zeiss DSM982 Gemini field emission scanning electron microscope, equipped with an Everhart–Thornley SE-detector
494
B. Ma et al. / Micron 38 (2007) 492–499
and the inlens-SE detector at a 50:50 ratio with an acceleration voltage of 5 kV. Images were saved as TIFF files and transferred to JPEG files for the test of stitching. 2.5. Image stitching and mosaicing For image processing and running of program, a Dell Optiplex GX620 PC (Pentium IV processor) was used. CPU: 3.20 GHz; RAM: 3.5G. The operation system is Microsoft Windows XP (Professional edition). 2.5.1. Manual stitching with Jasc Paint Shop Pro 9 The software was purchased from Jasc software, Inc. (Minneapolis, MN, USA). To perform stitching, image I was copied onto image II as an additional layer 1. The transparency of layer 1 was then changed for the visualization of the registration of the two images. Layer 1 was moved along x-, yaxes in order to find the overlap between two images. After completely overlapping the identical part of the two images, they were reduced to one layer to obtain a merged image. In the same way, a third or more images were added onto the image template to generate the composite images. 2.5.2. Image stitching with Autostitch Autostitch (Version 2.184, http://www.cs.ubc.ca/mbrown/ mb/mb.html) was used for the stitching of multiple images. The options for stitching are—auto straighten: selected; system memory: 2 GB; JPEG quality: 100%. Other settings are tested from default settings. The sizes of the images are optimized after stitching. 2.5.3. Image mosaicing with a self-programmed tiling macro The image stacks were stitched with a self-programmed rectangular tiling macro-integrated in Image J v1.36 (http:// rsb.info.nih.gov/ij/). The background in each acquired image was subtracted using an empty background image. Then the image stack containing all images used for the mosaic image was rotated by the value needed to perfectly align the x/y movement of the stage with the x/y orientation of the camera CCD chip. The image stack is cropped vertically as well as horizontally to take away the overlapping regions of neighbouring image fields. The ‘Make Montage’ function is used to generate a combined image of the image fields. 3. Results 3.1. Test of performance of Autostitch in image stitching
Fig. 1. Application test of Autostitch in stitching of microscopic images. Please note that the sizes of the images are optimized. Three spots indicate the positions of three coordinates used for check the validity of the stitching. In fact the original spots selected as coordinates are smaller. (A) Original image of a mouse lymph node. HE staining. Objective lens: 10. The original size of the image is 3840 3072 pixels. File format: JPEG. The ratio between the lengths of three coordinate lines of the triangle is 1.295:1.206:1. (B) Four small images (1, 2, 3 4) from four parts of the original images (with about 15% overlap with
The performance of Autostitch in the image stitching of multiple images was first tested before its application on multiple microscopic images. A 10 mm cryosection from mouse lymph node was stained with HE and the image was adjacent images) were extracted from (A). (C) The mosaic image is generated by stitching the four small extracted images in (B) with Autostitch. The ratio between the lengths of three coordinate line of the triangle is 1.290:1.198:1.
B. Ma et al. / Micron 38 (2007) 492–499
495
acquired with a normal light microscope equipped a colour camera. Four small images were extracted from this image and stitched together with the Autostitch. The original image and stitched results are shown in Fig. 1. To test the performance of the stitching, a simple quantitative test was performed. Three coordinates were selected in the original images and after stitching they were found on the stitching results. Three lines were drawn between three coordinates to generate a triangle and the ratio between the lengths of three sides was calculated (Fig. 1). More similar coordinates were selected and analysed in the same way (data are not shown). To test the different algorithm available in the Autostitch, the brightness of one small image (image 2) in Fig. 1 was adjusted and stitching was performed using the similar settings. The stitching results using no-blend, linear blending and multiblending are shown in Fig. 2A–C, respectively. The seams were observed in Fig. 2A, but not in Fig. 2B and C. The brightness of the overlap region, but not other regions, were optimized. The results generated with linear and multi-band blending is similar at low resolution images. However, if checked at high resolution, the quality generated with multi-band blending is better than that generated by linear blending (Fig. 2B and C, right panels) 3.2. Stitching of light microscope images A 7 mm cryosection of mouse intestine was stained with HE and eight images taken from different regions of the Peyer’s patch (50 magnification) were stitched together with Autostitch to generate a single overview image (Fig. 3A and B). To compare the performance of this approach with other methods, manual stitching with Paint Shop Pro was also performed and the results are presented in Fig. 3C. The efficiency of this software was also tested on image stacks acquired by automatic microscopy. From one section of lymph node, 49 images were taken to generate an image stack (Fig. 4A). These images were then stitched together with Autostitch to generate a large image (Fig. 4B). In addition, a montage of this image series was generated by a selfprogrammed tiling macro integrated in Image J (Fig. 4C).
4.1. Automatic image stitching with Autostitch
Fig. 2. Effect of blending on the stitching results. Four small images are from Fig. 1. Three small images (images 1, 3, 4) for stitching are same as that in Fig. 1B. The brightness of image 2 was adjusted in Photoshop (brightness from 0 to 38, the background from about (255,255,255) to about (216,216,216) according the RGB model). The stitching results of these four small images generated with no-blending, linear blending and multi-band blending (two bands) are shown in (A)–(C), respectively. Small parts of each stitching results are also shown at a higher resolution on upright panels. The options for stitching are—output size: 100%; SIFT image size: Min dim 400 pixels; auto straighten: selected; other settings: default. Please note that the sizes of the images are optimized. (A) In the overlap region, the overlap part from one image is put on that of adjacent image. Seams are visible in the image. The time for the stitching is 16 s. (B and C) Seams are not visible in both stitched results after blending. The visual quality of overlap region in (C) is better than that in (B). The brightness of the pixels out of the object was not adjusted (see the green circles in three stitched images, for examples). (B) Time for the stitching with linear blending is 20 s. (C) Time for the stitching with two bands blending is 69 s. Blending sigma: 5.
Autostitch is software designed for the stitching of multiple images to obtain a panoramic image in photography. In this study, we describe its application in the stitching of microscope images. However, image acquisition and manipulation are
different in the two applications. In making panoramic images, the cameras are moved or rotated and sometimes the objects in the scene also move. In the acquiring of microscope images, the camera position is fixed and only the slices are moved mainly
3.3. Stitching of electron-microscopic images The application of Autostitch was also tested on scanning electron microscopic images. Six images acquired from part of a lymph node were used for this study (Fig. 5A). Each of images has at least 12% overlap with the adjacent images. The output images after stitching are shown in Fig. 5. 4. Discussion
496
B. Ma et al. / Micron 38 (2007) 492–499
Fig. 3. Application of Autostitch in the stitching of images from mouse Peyer’s Patch. Please note that the sizes of the images are optimized. HE staining. Objective lens: 5. The size of the individual image is 640 480 pixels. File format: JPEG. Scale bar: 50 mm. (A) Eight images taken from different parts of a Peyer’s patch. (B) The image is generated by stitching the eight images in (A) with Autostitch. (C) The composite image is created by manually stitching the eight images with Paint Shop Pro 9. Seams are visible in this images (green arrows indicate). The edge lines are horizontal, which are different from that in (B).
along the x-, y-axes, making the stitching easier to perform. Thus, rotation and scaling are assumed to stay the same throughout the experiments and processing of the images. In the stitching of microscope images, high accuracy is needed in order to provide a genuine representation of the tissue structure, which is different from the panorama applications. To date,
Fig. 4. Application of Autostitch for stitching an image stack acquired by an automatic microscope. HE staining of lymph node. Objective lens: 20. Please note that the sizes of the images are optimized. (A) An image stack consisting of 49 images is taken from one section of lymph node, which is shown with part of the spleen. The montage of these images was performed by Image J. The large black circle around the lymph node is a marker used for finding the lymph node during image acquisition. (B) Stitching results with Autostitch. Big part of stack is cut from the image because of not-enough-match. High magnification of part of the image is shown in right panel. (C) Image of lymph node extracted from a mosaic image generated by a self-programmed rectangular tiling macro (in Image J) on the same stack. High magnification of part of the image, which shows the similar region as in (B), is shown in right panel. The arrow indicates one seam visible in the high resolution image.
many types of stitching software have been developed for applications in photography. However, after the stitching test, it was found that most of them could not be applied for the stitching of microscope images. For example, PanoramaStudio can only align images in the horizontal direction, which makes the generation of a composite image not possible. In addition, programs such as Panorama Tools, Pixtra Panostitcher were also found not suitable for our stitching purpose. To perform stitching, several kinds of microscope systems can be used. The first is a normal light microscope equipped
B. Ma et al. / Micron 38 (2007) 492–499
Fig. 5. Application of Autostitch in the stitching of scanning electron microscopic images. The original images were acquired at a magnification of 2000. The sizes of the images were optimized. Scale bar: 10 mm. (A) Six images were acquired from the follicle region of mouse lymph node. Each of them has overlap with the adjacent image. The original size of the images was 640 512 pixels. The montage of six images was created by Image J. (B) The composite image after stitching with Autostitch.
with either a colour or monochrome camera. With this microscope, the slide is moved along the x- or y-axes manually to acquire images for different parts of images. This method is usually time consuming and difficult to perform. The images must have sufficient overlap for stitching to work. The second is a microscope system with a motorised stage but without stitching software. The images can be collected along the x direction and then the stage is then moved ‘‘down’’ to the next row and moved back across the section in the opposite direction. After data acquisition, images can be stitched with other programs. The third type is a fully automatic microscope system with stitching software package. These kinds of work stations seem to be the best solution for image acquisition and stitching. However, when dealing with some serial sections, large areas of the section may be blank and without tissue information and, thus, misalignment of two adjacent images might occur with some programs because of ‘‘match of background’’ (data are not shown). The accuracy of stage for repeatable results is normally commensurate with the cost of the work stations. For less accurate stage systems, more overlap is needed for perfect image stitching. Furthermore, alignment of the work stations is also important since it will affect the accuracy of stage. Therefore, the microscope system should be calibrated before the image acquisition. For example, if the camera is not parallel to the stage, it should be taken into account in the tiling process. Pre-processing before image acquisition is important for image stitching. At first, for large data sets, the images must be
497
re-sized for easier manipulation. Sometime the data set is so big that the normal computer cannot handle. If several cameras are available, using the camera with least pixels across and down will make size-reducing factor smaller. Secondly, uneven illumination across the field is almost always present at different level and should be corrected. This problem becomes nearly impossible to avoid at low magnifications (using 2.5, 5, or 10 objective lens). In this case, more overlap must be provided for better stitching results. Thirdly, it is good practice to subtract the background from the image before stitching. In this study, background images were subtracted from the images taken by an automatic microscope. This kind of manipulation reduced the artifacts, such as the dirt on the objective lens. Both manual and automatic stitching methods can be used for image stitching. In our study, three approaches have been applied and comparison of them is summarized in Table 1. For manual manipulations, the images are stitched with normal commercially available software. Paint Shop Pro 9 was used for stitching eight images to create a single big image of Peyer’s patch and the results were compared with the image generated with Autostitch. The quality of image generated by the Autostitch is better than that the manually stitched image. Certainly, this kind of stitching can also be accomplished by Adobe Photoshop or other image-processing software. One advantage of Paint Shop Pro is that the transparency of the layers can be adjusted for easier image registration. After registration, two layers can be reduced to one layer to obtain an output image. Both manual stitching and automatic stitching have advantages and disadvantages. Manual stitching is easy to perform with normal image-processing software. For example, manual stitching can be applied in investigations of the pathological changes in spleen follicles at high resolution after infection or immunization. Since no blending is performed after registration, sometimes good results cannot always be obtained due to the visible seams in the overlap region of two images. Assuming good protocol from the start (good alignment of camera, stage and microscope slide, shading correction, manual settings on the camera to avoid uneven brightness) images should be seamless or nearly seamless. For image masaicing without making cross-correlation some studies have been reported for generation of super-images in light microscopy (Su¨ss et al., 2002; Chow et al., 2006). Although automatic tiling or similar masaicing approaches are rapid and easy ways to generate a compact image, they rely on a highly precise, motorised x, y stage and normally cannot produce results as good as the image stitching approaches. The first step of stitching with Autostitch is registration, which has been achieved by a correlation method based on the extracted features present in two adjacent images. Therefore, the provision of images with enough overlap is crucial for the stitching process. Although the correlation method is not the fastest algorithm, it will generate the smallest error (Flynn et al., 1999). If a motorised stage is used, the overlap area normally covers 10%, whereas when using a manual stage, the overlap of two adjacent images is different, although it must also be larger than 10% for efficient stitching. Another advantage of this stitching is that it is not necessary to arrange
498
B. Ma et al. / Micron 38 (2007) 492–499
Table 1 Comparison of three methods of image mosaicing
Image acquisition motorised stage Stage accuracy dependent User input Pre-processing Stitching algorithm X,Y translation Rotation Projection Change in exposure, uneven illumination Scale invariant Blending Colour correction Brightness/contrast correction Output Seam
Time needed Automatization
Manual stitching
Rectangular tiling
Autostitch
Without work stations
With work stations motorised stage Highly precise stages needed Needed Integrated in the macro, background correction Overlay two images, rectangular selection Yes Yes, before masaicing No Invariant
With or without work stations motorised stage Yes, bigger overlap needed for less accurate stages No need No, resize is needed
Yes, only dependent on rectangular selection No blending
Yes
Overlap region selected visually Needed No, can be done manually with other programs Overlay two images, manually registration Yes No No Variant due to manual operation Not No, can be done with other programs No correction, can be done with other programs No Any types of format Seam normally visible, especially in some cases, sometime nearly invisible Long, tedious Manual operation
the images in the certain order for the stitching. This is demonstrated in Fig. 3, in which input images are in an order different from their natural order. In the registration step, SIFT was applied for finding the corresponding features in the overlap region of two images. Therefore, this method is a feature-based registration method, which is different from the intensity correlation registration methods described by Rankov et al. (2005). These features are invariant to image scale, rotation and partially invariant (i.e. robust) to changing view points, illumination. In the present study, we have assumed the camera or detector is precisely calibrated and that the rotation and scaling stay the same during data acquisition. However, accuracy depends upon the performance of the motorised stage. In the stitched image in Figs. 3–5, the edge lines of the images are not straight, which indicates that the stitching has been obtained not only through translation, but also by a little rotation or perspective correction. In automatic image acquisition in Fig. 4, small ration of 0.678 were performed before the rectangular tiling, although this step is not necessary for the Autostitch. This kind of small rotation or projection, which is absent in some tiling or stitching programs, should be corrected for the perfect image match. In Autostitch, this kind of manipulation through the feature-based cross-correlation is performed automatically in the registration step. Stitching quality is measured visually by the similarity of the stitched image to each of the input images and by the visibility of the seam between the stitched images. In this study, we have used
SIFT, cross-correlation Yes Yes, with registration Yes, spherical Partially invariant
No
Pyramid-based blending, also linear blending Yes, especially in overlap region
No correction after stitching
Yes, especially in overlap region
Any types of format Seam may be visible
JPEG (often), TIFF or other types Normally no seam
Fast Automatic, for very large stacks easy to perform
Fast, but not fast as tiling Automatic for single image, for large stacks, additional macro-needed
several extracted images from one image as a test of stitching, since a comparison of the output image with the original images is easy to perform. To improve the quality of the image, blending is necessary after the registration of two adjacent images. However, one disadvantage of blending is that using the blending method, which applies changes in intensity, makes quantitative analysis of intensities invalid. The process of image blending is usually restricted to zones of overlap that are determined during the stitching procedure. Although a seam is sometimes not clearly visible when uneven illumination and background are not present, blending will still improve the image quality of the composite image. In the present study, three blending algorithms in the Autostitch were tested for their performance. Multi-band blending could produce images with better quality although it is much slower than linear blending method. In this kind of pyramid-based multi-band blending, different frequency bands are combined with different alpha masks. In addition, better stitching results can be obtained if pyramid blending is applied on the gradients (Levin et al., 2004). Furthermore, blending in gradient domain can also reduce the compression artifacts in highly compressed JPEG images. 4.2. Potential application in virtual microscopy Virtual microscopy is a computer-based technology that offers the full range of traditional microscope functionality (or more). A virtual slide is a digitally captured glass slide composed
B. Ma et al. / Micron 38 (2007) 492–499
of high quality images. The convenience of a computer is then used to browse the virtual slide with speed and ease. For example, in clinical studies of cancer, it is easier to store an image set for the whole section, while retaining microscopic resolution, than to keep the normal pathological slide. In this study, we have tested this application in a section of lymph node. Although the test has been performed on grey scale images, the principle also applies to colour images, since for the stitching of colour pictures, only the intensities component is used during the search for the best correlation of two adjacent images. The virtual microscope principle can also be applied in electron microscopy. For example, the whole section can be scanned with the electron microscope at very high magnification and stitched together. The data base generated is useful for studying the histology and pathology of organs, especially in research groups without direct access to an electron microscope. With respect to transmission electron microscopy, if such a virtual section is produced, other workers could use this data set to study cell–cell interactions inside organs without having to prepare their own material for electron microscopy. For example, interactions of the various cell populations inside a lymph node could be studied by several groups of workers using virtual sections; this would be extremely useful for determinations of the immunodynamics of lymph nodes in infection and other pathological conditions. In summary, Autostitch is an easy but very efficient approach for stitching microscope images. Although our manipulations have been performed on JPEG format images, it will also work on the images in TIFF or other format. In addition, it is applicable on both grayscale and RGB true colour images. In addition to the applications in the light microscopy and electron microscopy, it may be also applied in the fluorescence microscopy and confocal microscopy. We hope that this image stitching program will find increasing applications in biomedical research requiring morphological studies. Acknowledgements We acknowledge the financial supports from University Hospital of Ulm, Germany and continual support of the
499
Advanced Light Microscopy Facility of EMBL by Visitron GmbH. We also want to thank Wayne Rasband for the Image J and Matthew Brown for the Autostitch program. References Adelson, E.H., Anderson, C.H., Bergen, J.R., Burt, P.J., Ogden, J.M., 1984. Pyramid methods in image processing. RCA Eng. 29, 33–41. Bhosle, U., Chaudhuri, S., Roy, S.D., 2002. A fast method for image mosaicing using geometric hashing. IETE J. Res. Special Issue Visual Media Process. 48, 317–324. Brown, L.G., 1992. A survey of image registration techniques. ACM Comput. Surv. 24, 325–376. Chen, C., Klette, R., 1999. Image stitching—comparisons and new techniques. Comp. Anal. Imag. Patterns 1689, 615–622. Chow, S.K., Hakozaki, H., Price, D.L., MacLean, N.A.B., Deerinck, T.J., Bouwer, J.C., Martone, M.E., Peltier, S.T., Ellisman, M.H., 2006. Automated microscopy system for mosaic acquisition processing. J. Microsc. 222, 76–84. Flynn, A., Green, A., Boxer, G., Pedley, R., Begent, R., 1999. A comparison of image registration techniques for the correlation of radiolabeled antibody distribution with tumor morphology. Phys. Med. Biol. 44, 151–159. Fontelo, P., DiNino, E., Johansen, K., Khan, A., Ackerman, M., 2005. Virtual microscopy: potential applications in medical education and telemedicine in countries with developing economies. In: Proceedings of the 38th Hawaii International Conference on System Sciences, Hawaii. Levin, A., Zomet, A., Peleg, S., Weiss, Y., 2004. Seamless image stitching in the gradient domain. In: Proceedings of the Eighth European Conference on Computer Vision, vol. 4. pp. 377–389. Lowe, D.G., 2004. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60, 91–110. Lundin, M., Lundin, J., Isla, J., 2004. Virtual microscopy. J. Clin. Pathol. 57, 1250–1251. Rankov, V., Locke, R.J., Edens, R.J., Barber, P.R., Vojnovic, B., 2005. An algorithm for image stitching and blending. In: Conchello, J.A., Cogswell, C.J., Wilson, T. (Eds.), Proceedings of SPIE. Three-Dimensional and Multidimensional Microscopy: Image Acquisition and Processing XII, vol. 5701, San Jose, CA, USA. Shum, H.Y., Szeliski, R., 1997. Panoramic image mosaics. Technical report. http://research.microsoft.com/pubs/view.aspx?tr_id=11. Su¨ss, M., Washausen, S., Kuhn, H.J., Knabe, W., 2002. High resolution scanning and three-dimensional reconstruction of cellular events in large objects during brain development. J. Neurosci. Methods 113, 147–158. Zomet, A., Levin, A., Peleg, S., Weiss, Y., 2006. Seamless image stitching by minimizing false edges. IEEE Trans. Image Process. 15, 969–977.
