DUAL-VIEW MEDICAL IMAGE VISUALIZATION BASED ... - IEEE Xplore

DUAL-VIEW MEDICAL IMAGE VISUALIZATION BASED ON SPATIAL-TEMPORAL PSYCHOVISUAL MODULATION Zhongpai Gao, Guangtao Zhai, Chunjia Hu and Xiongkuo Min Institute of Image Communication and Information Processing, Shanghai Jiao Tong University, Shanghai, China Email: {gaozhongpai, zhaiguangtao, hcj-sjtu, minxiongkuo }@sjtu.edu.cn ABSTRACT Medical imaging technologies such as magnetic resonance imaging (MRI) and computerized tomographic (CT) are used to diagnose a wide range of medical diseases. Medical images are generated by detecting density differences between different tissues in the body. Multiple medical image visualization is of critical importance to diagnosis. This paper introduces a dual-view medical image visualization prototype based on spatial-temporal psychovisual modulation (STPVM). Temporal psychovisual modulation (TPVM) enables a single display to generate multiple visual content for different viewers. Spatial psychovisual modulation (SPVM) extends the idea of TPVM to spatial domain. STPVM combines TPVM and SPVM by exploiting both temporal and spatial redundancy of modern displays. Based on STPVM technology, one display can present even more images simultaneously. In this demo, two kinds of medical images e.g. T1 and T2 weighted MRI images, are presented simultaneously. Physicians can switch between either image by just moving the eye fixations. Since T1 and T2 are shown simultaneously and are aligned on the screen, it is more convenient for the physicians to get different information of the same spot from the T1 and T2 images. The developed demo is useful for physicians during surgery navigation and effectively reduces the burden of mental transfer. Index Terms— temporal psychovisual modulation, Medical Image Visualization, magnetic resonance imaging 1. INTRODUCTION Magnetic resonance imaging (MRI) [1] is a diagnostic medical imaging tool used in radiology to investigate the anatomy and function of the body in both health and disease. This technique has been proven high valuable for the diagnosis of a broad range of conditions in all parts of the body, including cancer, heart and vascular disease, stroke, and joint and musculoskeletal disorders. On an MRI image, some tissues This work was supported in part by NSFC (61025005, 61371146, 61221001), 973 Program (2010CB731401) and FANEDD (201339).

978-1-4799-5751-4/14/$31.00 ©2014 IEEE

appear to be brighter or darker than other tissues. Darkness depends on the density of protons in that area – an increased density being associated with a darker area. Relaxation times for protons can vary and two times are commonly measured, known as T1 and T2. White matter is darker than grey matter in T1-weighted images and brighter than grey matter in T2weighted images [2]. T1 and T2 of pathologic tissues usually become longer than those of normal tissues, making MRI very useful in diagnosis of various diseases. However, in existing practice, the physicians have to alternate their visual attention between T1 and T2 images during diagnosis, a process known as mental transfer which needs a time of practice and is quite laborious. Therefore, a dual-view medical image visualization device that enables simultaneous display of T1 and T2 weighted images on the same screen, is of high interest to physicians. Temporal psychovisual modulation (TPVM) [3] was proposed as a new information display technology using the interplay of signal processing, optoelectronics and psychophysics. TPVM is based on the fact that the human visual system (HVS) cannot resolve temporally rapidly changing optical signals beyond flicker fusion frequency (about 60 Hz for most of people). Nowadays, modern display technologies offer much higher refresh rates. Thus, a single display has extra capacity, i.e. psychovisual redundancy, to generate multiple visual percepts for different viewers. A TPVM based display device broadcasts out a set of images called atom frames at a speed higher than the flicker fusion frequency. The atom frames are then weighted by liquid crystal (LC) shutter based viewing devices that are synchronized with the display before entering the human visual system and integrating into some desired visual signals. Therefore, through different viewing devices, people can see different contents on the same display. Spatial psychovisual modulation (SPVM) extends the idea of TPVM to spatial domain. Nowadays modern displays also have very high pixel density that is far beyond the resolution of the human retina [4]. Study indicates that the unaided human eye can generally not differentiate detail beyond 300 PPI (pixels per inch) [5]. However, generally,

2168

ICIP 2014

Red Green Blue

Brightness value

40 30 20 10 0 0

50

100 150 200 Gray scale value

250

Fig. 2. Setup of the system

Fig. 1. The correspondence between gray scale and brightness of LG D2343PB-BN monitor measured by Konica Minolta Display Color Analyzer CA-210 in our lab. people views the display screen in a farther distance than idea condition, which makes the display has more psychovisual redundancy. So we can exploit the spatial redundancy of the screen to generate multiple visual percepts concurrently. For example, a SPVM based display device can broadcast out multiple images concurrently on interlaced polarized display. The scan lines of display are divided into different polarized directions. The desired image can be viewed through matched polarized glasses. STPVM combines TPVM and SPVM by exploiting both temporal redundancy and spatial redundancy of display devices. Based on STPVM technology, a single display can present even more images for different viewers simultaneously. STPVM can be realized by off the shelf technologies. For TPVM, active shutter 3D system can be used. An active shutter 3D system is a technique of displaying stereoscopic 3D images. The refresh rate of such display is 120Hz, the double of flicker fusion frequency (60Hz). So two atom frames can be presented alternately. A pair of LC shutter glasses with both eyes of the same time slot is used to view the atom frames (even-time frames or odd-time frames). For SPVM, polarized 3D system can be used. For such display device, to create the 3D illusion, even and odd scan lines of the devices are polarized in different directions. So two atom frames can be presented simultaneously. A pair of passive polarized glasses with both eyes of the same polarized direction is used to view the atom frames (even-line frames or odd-line frames). Although a display device which can work by both active shutter and passive polarized 3D technologies does not exist nowadays, we can combine these two systems by ourselves. For example, we can let two 3D projectors project to a single screen and put a polarizer in front of one of the projectors. This method can provide four atom frames. In this work, based on STPVM, we develop a dual-view medical im-

age visualization system for two kinds of medical images (T1 and T2 weighted MRI images) simultaneously. 2. SYSTEM DESIGN The dual-view medical image visualization system provides two atom frames include T1 and T2. So a straightforward way to implement the system is based on SPVM. SPVM provides a simple and stable solution to the problem. First, we let t1 and t2 be the images of T1 and T2 respectively. The odd-line frames are denoted as O and the even-line frames are denoted as E. We choose a polarizing coating which has the same polarization direction as the even-line frames as the passive polarized glasses lenses. In order to facilitate the physicians to see images. We cut the lenses from the middle and leave the upper parts. So physicians can switch between both images by just moving the eye fixations. Through the upper parts of glasses we can see the even-line frames E (glasses-aided view) representing T1 and through the lower parts we can see all the frames O + E (normal view) representing T2. Thus we let E = t1 and O + E = t2. Then O = t2 − t1. However, for an 8-bit image, the gray scale value is in [0, 255]. So E and O should be adjust to prevent the gray scale value overflowing. On the other hand, the correspondence between brightness and gray scale values are not linear for most display devices. If we directly compute O = t2 − t1, the normal view cannot get T2. Fig. 1 shows the relationship between brightness and gray scale values of an LG D2343PB-BN monitor measured by Konica Minolta Display Color Analyzer CA210. First, we make a mapping table between gray scale value and brightness value. Based on the table, we transform gray scale value to brightness value to compute the even-line frames. At last, we transform the brightness value back to gray scale value. 3. SYSTEM SETUP AND RESULTS The demo system is written in C++ language in Microsoft Visual Studio 2010. SDKs from DirectX, OpenCV and CEGUI

2169

ICIP 2014

(a)

(b)

(c)

(d)

(e)

(f)

Fig. 3. Results of the system. (a) and (d) show two pair of T1 and T2 weighted MRI images. (b) and (e) T1 weighted images viewed through the upper part of the glasses. (c) and (f) T2 weighted images viewed through the lower part of the glasses. are used to fully realized the system. The hardware equipments of this system includes a PC, a polarized 3D display and a pair of passive polarized glasses, as shown in Fig 2. The results of the system are shown in Fig 3. Physicians can view T1 and T2 through the passive polarized glasses. The glasses are designed to be half lens half empty as shown in Fig. 2. Through the upper part of the glasses physicians can view T1 and through the lower part of the glasses physicians can view T2. It is not necessary for physicians to control the glasses manually and they just need to move the eye fixations to switch between the images. So it is very convenient for physicians to view T1 and T2 simultaneously. On the other hand, T1 and T2 are presented at the same place, reducing the burden of mental transfer, which is a very convenient feature during surgery navigation.

disease. Physicians can diagnose the diseases by observing the T1 and T2 of tissues. In this demo system, the glasses are designed to be half lens half empty, so physicians can switch between ether image by just moving the eye fixations. Since T1 and T2 are shown simultaneously and are aligned on the screen, it is more convenient for the physicians to get different information of the same spot from the T1 and T2 images. The developed demo is useful for physicians during surgery navigation and effectively reduces the burden of mental transfer.

4. CONCLUSION

[2] Stewart C Bushong, Magnetic resonance imaging, Elsevier Health Sciences, 2003.

This paper introduces a dual-view medical image visualization system based on spatial-temporal psychovisual modulation (STPVM). STPVM combines temporal psychovisual modulation (TPVM) and spatial psychovisual modulation (SPVM) by exploiting both temporal and spatial redundancy of modern display devices. Based on STPVM technology, a single display can present multiple images simultaneously. As an example, two kinds of medical images: T1 and T2 weighted magnetic resonance imaging (MRI) images of are presented simultaneously in this demo system. MRI is a diagnostic medical imaging tool used in radiology to investigate the anatomy and function of the body in both health and

[3] Xiaolin Wu and Guangtao Zhai, “Temporal psychovisual modulation: A new paradigm of information display [exploratory dsp],” Signal Processing Magazine, IEEE, vol. 30, no. 1, pp. 136–141, 2013.

5. REFERENCES [1] Ann L Scherzinger and William R Hendee, “Basic principles of magnetic resonance imagingłan update,” Western Journal of Medicine, vol. 143, no. 6, pp. 782, 1985.

[4] Wikipedia, “List of displays by pixel density,” http://en.wikipedia.org/wiki/List_ of_displays_by_pixel_density, July 2014. [5] Jonesblog, “Apple retina display,” http: //prometheus.med.utah.edu/˜bwjones/ 2010/06/apple-retina-display/, Jun. 2010.

2170

ICIP 2014