Super-resolution on Data Acquired in Polar Format

INTERNATIONAL JOURNAL OF COMPUTATIONAL INTELLIGENCE AND HEALTHCARE INFORMATICS Vol. 4, No. 2, July-December 2011, pp. 63-72, Published by Serials Publications, ISSN: 0973-7413

Super-resolution on Data Acquired in Polar Format Debashis Nandi1 and Sudipta Mukhopadhyay2 1

2

Department of Information Technology, NIT, Durgapur, Mahatma Gandhi Avenue, Durgapur-713209, West Bengal, India. E-mail: [email protected]

Department of Electronics and Electrical Communication Engineering, IIT Kharagpur, Kharagpur-721302, West Bengal, India. E-mail: [email protected]

ABSTRACT: This paper presents a new super-resolution (SR) technique that is applied to the imaging systems (e.g. ultrasound sector scanner) where image data are acquired in polar format. Resolution of images obtained from such imaging systems is inherently poor. Though harmonic imaging (in case of ultrasound image) has been prescribed as one of the solutions to this problem, it fails to provide high-resolution image at higher depth due to its rapid attenuation along the way. Harmonic imaging also requires a major change in the system hardware and increases the cost of the system. Another problem in the conventional image formation techniques from polar format data is the non-uniform data sampling in the field of view of the image. In particular, the available data density is less at the higher depth in the case of ultrasound imaging. Due to low density of in the region of higher depth, the image quality becomes poor at that region. Since the gap between the successive scan lines increases in the image with the depth of penetration, very small objects may be missed from the higher depth of the reconstructed image. The Super-resolution reconstruction technique may be considered as a good solution to this problem. This article illustrates the new software-centric SR technique, which is low cost and superior to conventional SR technique. Simulation results confirm the usefulness of the proposed SR algorithm. The model also works well if the data is acquired in the rectangular co-ordinate system. Since it is performed at scan conversion level, it can be used with harmonic imaging also. Keywords: Ultrasound image enhancement; Super-resolution; Scan conversion; Low-resolution images; Registration parameters.

1. INTRODUCTION

Ultrasound (US) imaging is an important example where the scan data is acquired in polar domain and later it is converted to raster scan for the purpose of display on conventional video display unit. It has immense importance in the medical diagnostic due to its non-invasive and non-ionizing nature. The use of diagnostic ultrasound covers a broad area of medical science such as Cardiology, Gastroenterology, Gynecology, Neurology, Ophthalmology, Urology etc. The resolution of the ultrasound image influences the degree of accuracy in medical diagnosis. Hence, enhancement of ultrasound image resolution is an active area of research and has got a lot of attention. Though a considerable amount of works are done in the direction of scan conversion (Ophir and Maklad, 1979; Ophir and Brinch, 1982; Robinson and Knight, 1982; Lee et al., 1986; Dah-Chung Chang et al., 1992; Bakeoff et al., 1994; Richard and Arthur, 1994), interpolation (Robinson and Knight, 1982; Richard and Arthur, 1994; Lehmann et al., 1999; Unser, 1999), speckle reduction (Dutt and Greenleaf, 1996; Michailovich and Tannenbaum, 2006; Dantas and Costa, 2007; Lee et al., 2008) and Harmonic inaging for enhancement of ultrasound image (Jang et al., 2000; Li and Zagzebski, 2000; Kudo, 2001; Taxt and Jirik, 2004; Wang et al.,2009 ), only a few works (Carotenuto et

al., 2002; Clement et al., 2005) are found in the direction of software based super-resolution for enhancing the spatial resolution of ultrasound image. Most of the works for ultrasound image resolution enhancement are done at hardware level. These techniques include (a) Harmonic imaging and (b) Beam width reducing increasing the number of transducers in the array (c) Providing good focusing arrangement and (d) increasing number of scan-lines per frame. All these improvement techniques require major change in the hardware of the existing system, and consequently, the complexity and the cost of the system increases. But the application of super-resolution algorithm is not yet explored to enhance the spatial resolution of ultrasound image. In this work, we point out a few causes of degradation of ultrasound image resolution and suggest a good solution to this problem. Four types of resolution are defined for ultrasound image: (a) axial resolution, (b) lateral resolution, (c) temporal resolution and (d) contrast resolution. The axial and the lateral resolution are termed as spatial resolution and we are considering this spatial resolution in our literature. The axial resolution of the ultrasound image depends on the ultrasound frequency and the lateral resolution depends on the ultrasound frequency, the number of transducers used in the transducer array, beam width and

64

International Journal of Computational Intelligence and Healthcare Informatics

the focusing arrangement. Tissue harmonic imaging techniques were introduced to enhance the image resolution. The frequency bandwidth of the fundamental and the harmonic components often overlap in Tissue harmonic imaging (THI). The spectral overlap causes degradation of spatial resolution and the undesirable artifact. The SNR of tissue harmonic imaging at higher depth is also poor. The resolution at the higher depth of the ultrasound image is not only dependent on the ultrasound frequency but also on the local sample density in the image area. Since the ultrasound envelope data is available in the polar format, it is clear that the sample density is less at the higher depth. It reduces the resolution of the ultrasound image at the higher depth. In this paper, we develop a software based Super-resolution technique, which is more flexible and can be used (as and when required), in the existing system with minimum modification of hardware, to enhance the resolution of ultrasound image. It is pointed out that, the image data obtained in ultrasound sector scanner is in polar format. The image is formed from the available data by scan conversion and interpolation. Several types of interpolations exist in literature (Robinson and Knight, 1982; Richard and Arthur, 1994; Lehmann et al., 1999; Unser, 1999), which may be used for ultrasound image formation. Bilinear interpolation is considered to be a popular technique for its good speed quality combination though cubic spline interpolation gives better quality. Cubic interpolation is also lack good resolution due to low-pass filtering effect. Implementation of super-resolution algorithm is a thrust area of research and is successfully applied in many area of image processing including medical imaging (Kennedy et al., 2006). The basic philosophy of super-resolution reconstruction is the reconstruction of a high resolution image from a number of sub-pixel shifted, aliased low resolution images (Stark and Oskoui, 1989; Kim and Bose, 1990; Irani and Peleg, 1991; Tekalp et al., 1992; Elad and Feuer, 1997; Borman and Stevenson, 1998; Nguyen et al., 2001; Chowdhuri, 2002; Park et al., 2003; Ben-Ezra et al., 2005). The LR image data used in these SR reconstruction techniques are available in rectangular raster format. In our literature we term these techniques as conventional SR (CSR) techniques. Conventional super-resolution technique cannot enhance spatial resolution of ultrasound image to a great extent. The reason behind this is: the LR images are first formed by scan conversion and interpolation. Then SR reconstruction is applied to those LR images in the second phase. In the second phase, there is another interpolation stage. These repeated interpolations blur the image and hence a very little improvement is achieved after SR reconstruction. Considering these facts, it is assumed that if the SR reconstruction is applied to the raw data (available in polar format) available at the output of the preprocessing

stage of the Ultrasound system, the spatial resolution of the image can be improved. Hence, in this work, we have applied SR reconstruction technique to the raw scan data of the ultrasound imaging system and obtained high-resolution ultrasound image. The rest of the article is divided into following sections: Section 2 gives a brief description of conventional super-resolution technique and a brief introduction to medical ultrasound imaging system and Section 3 illustrates the architecture of the proposed algorithm; Section 4 gives the definition of a few objective quality metrics; the simulation results, a comparison among the reconstructed images in terms of quality metric and detectibility of the small objects is given in Section 5; and the article ends with conclusion in Section 6. 2. CONVENTIONAL SUPER-RESOLUTION

Conventionally, the super-resolution is the enhancement of spatial resolution of an image by using multiple lowresolution (LR) images (or frames) captured from the same scene. The LR images are nothing but the different “looks” of the same scene (Park et al., 2003). Hence, the LR images are sub-sampled (aliased) and shifted with sub-pixel precision. If the LR images are shifted by integer units, then each image contains the same information, and thus no new information can be used to reconstruct an HR image. On the other hand, if the LR images have different sub-pixel shifts from each other and if aliasing is present, then each image has some new information compared to others. Consequently, the new information contained in each LR image can be exploited to obtain an HR image. Multiple scenes can be obtained from one camera with several captures or from multiple cameras located in different positions. These scene motions can occur due to the controlled motions in imaging systems. The same is true for uncontrolled motions, e.g., movement of local objects or vibrating imaging systems. If these scene motions are known or can be estimated within sub-pixel accuracy by applying image registration technique, we combine these LR images to reconstruct a SR image. Accuracy of image registration is important in SR reconstruction. Various image registration techniques are reported in different articles (Dvorchenko, 1983; Tian and Huhns, 1986; Bernstein et al., 1987; Brown, 1992). For SR reconstruction we should pick up high speed and most accurate image registration. The recorded LR images usually suffer from blur, noise, and aliasing effects. Although the main concern of an SR algorithm is to reconstruct HR images from sparsely sampled LR images, it covers image restoration techniques that produce high quality images from noisy, blurred images. Therefore, the goal of SR techniques is to restore an HR image from several degraded and aliased LR images. The schematic diagram of a SR system is shown in Fig. 1.

65

Super-resolution on Data Acquired in Polar Format

Figure 1: Schematic Diagram of a SR System

The image is formed by computing the pixel values at the rectangular grid points with the use of the available scan data by employing a suitable interpolation technique. The motivation of this work is to employ software based super-resolution (SR) image processing technique to form high-resolution ultrasound image even at higher depth of penetration.

2.1 Brief Introduction to Medical Ultrasound Imaging System

3. CONVENTIONAL SUPER-RESOLUTION FROM SCAN CONVERTED ULTRASOUND LR IMAGES (CARTESIAN CO-ORDINATE)

Conventional ultrasound (US) sector scanner consists of two major blocks: The data acquisition system and the conventional ultrasound image formation system. The conventional image formation system contains a memory unit for storing the scan data obtained from the data acquisition system followed by the scan conversion, imageprocessing unit. A schematic diagram is shown in Fig. 2 and a block diagram is shown in Fig. 3.

It is proved that high-resolution image can be obtained from a number of sub-pixel shifted aliased low-resolution (LR) images (Borman and Stevenson, (1998); Chowdhuri, 2002; Park et al., 2003). By utilizing this philosophy we can also improve the image quality of the ultrasound image. The conventional super-resolution techniques are applied on the LR images in rectangular grid. If these conventional SR techniques are applied for ultrasound image resolution enhancement, they cannot improve much because scan converted LR images (frames) lose a considerable amount of information due to low pass nature of prior non-ideal repeated interpolation. They can only provide marginal improvement over the constituent LR images. The conventional SR scheme that can be used in reconstruction of ultrasound image is shown in Fig. 5.

Figure 2: Schematic Diagram of Ultrasound Imaging System

Figure 5: Conventional SR Reconstruction used in SR Ultrasound Image Reconstruction Figure 3: Block Diagram of Ultrasound Imaging System

3.1 Proposed Super-resolution from Sampled Raw Ultrasound Data (in Polar Format)

After the acquisition of data, the data (in polar format) is temporarily stored in the memory unit and then it is passed through the scan conversion, image-processing unit to form the ultimate image. Since the data is obtained in polar format, scan conversion and interpolation is required to form the image in the rectangular raster. The geometry of scan conversion and image formation is shown in Fig. 4.

In view of this result, we propose a novel SR reconstruction algorithm based on non-uniform interpolation for the enhancement of spatial resolution of ultrasound image, where scan conversion and SR reconstruction is done using raw scan-data (available in polar format) of captured multiple frames. Non-uniform interpolation technique is chosen since it is simple and adaptable to the proposed framework, fast and easy to implement. Since SR reconstruction is done on the raw polar format data during scan conversion from the radial lines to rectangular raster, the proposed SR is termed as Radial SR (RSR). Figure 6 illustrates the structure of super-resolution reconstruction of ultrasound image.

Figure 4: Scan Conversion Geometry

66

International Journal of Computational Intelligence and Healthcare Informatics

3.2 Interpolation

Figure 6: Proposed Radial SR Reconstruction for SR Ultrasound Image Reconstruction

In the proposed technique, we take raw scan data of a number of shifted images, which can be obtained by capturing multiple frames of same object with sub-pixel shift/rotation of the transducer probe with respect to the object in a plane. The angular shift between the images with respect to reference one should be fractional multiple of angular sampling interval for the case of only rotation. The LR images are formed from the data using conventional ultrasound image formation technique. These LR images are registered to evaluate the registration or motion estimation parameters. Rigid registration is selected in the proposed algorithm for its simplicity. The registration is done with affine transformation by minimizing the correlation measure between the reference image and transformed images. Scan conversion and interpolation is done then on the raw scan data using the registration parameters obtained from the previous stage. Figure 7 shows the proposed radial SR ultrasound scan conversion mechanism.

Figure 7: Proposed Radial SR Ultrasound Scan Conversion Mechanism

Interpolation is one of the important tasks for this SR technique. Figure 8(a) depicts the scanned lines from two LR images after alignment. Figure 8(b) shows a cropped area of Figure 8(a). It shows the aligned radial lines of two sets of LR frame on a rectangular grid. Broken lines represent the radial lines of one LR frame and solid lines indicate the radial lines of the other. The lines are aligned according to the measured registration parameters. The points marked by triangle represent the positions of the available raw scan data. The problem is to evaluate the pixel value at a point P (solid rhombus), which is a grid point of the rectangular raster using the available scan data. There are a number of ways to evaluate the pixel value at the point P by interpolation technique. These include: 1. nearest neighbor (NN) interpolation along radial direction followed by horizontal directions, 2. nearest neighbor (NN) among non-uniformly distributed data, 3. bilinear interpolation, 4. distance weighted average of four nearest neighbors, 5. linear interpolation along radial direction followed by horizontal directions, 6. Cubic spline interpolation, etc.. Considering the speed of operation, generally, nearest neighbor, linear interpolation and bilinear interpolation have become popular in practice for conventional ultrasound imaging. Nearest neighbor (NN) interpolation along radial and horizontal direction has less computational complexity, but output image is poor. Nearest neighbor interpolation can also be done by considering all the samples. But it will be more computationally intensive for exhaustive searching of the nearest point of a point (point P) to be interpolated. The computational complexity for this searching will be O(MNmnp), where each raw scan data of size is m × n, interpolated image size is M × N and p is the number of LR frames. The complexity of the bilinear interpolation is also more since we have to find out four surround points of the point P (as in Figure 8(b)) by exhaustive searching. Linear interpolation is successfully applied in ultrasound image reconstruction with considerable speed quality combination (Richard and Arthur, 1994). Considering all these facts linear interpolation along radial and horizontal directions is selected, which is relatively fast, and give comparatively good quality result. In the first stage, for a particular radial line, those points are linearly interpolated, which are the intersections of the radial lines and the horizontal lines. For this linear interpolation, we take two sample points on the either side of the point to be interpolated. In the second stage, the points where the vertical lines intersect the horizontal lines passing through the already interpolated points are found out. Then, each of these points is linearly interpolated by taking two already interpolated sample points on the either side of it along the horizontal lines. For example, first we calculate the pixel value at the point M (solid circle) on

67

Super-resolution on Data Acquired in Polar Format

the radial line B from the pixel values at the two nearest sample points O and Q (solid triangles) on the same line using linear interpolation technique (Figure 8(b)) as follows: IM =

( y1 − y ) I Q + ( y2 − y ) I o y1 − y2

where the co-ordinate of M, O and Q are (x, y), (x1, y1) and (x2, y2) respectively and the pixel value of M, O and IM, IO, and IQ respectively. In a similar way, we can calculate the pixel value at the point N (solid circle) on the radial line A using the pixel values at the two points R and S (solid triangles) on the same radial line. Next, the pixel value at rectangular grid point P using pixel values at two nearest interpolated sample points (on both sides) M and N lying on the same horizontal line by linear interpolation. As an alternative method we could first interpolate those points where vertical lines cut the radial lines along radial direction. Then the points, where horizontal lines cut the vertical lines, could be interpolated along vertical direction. But since the number of intersection points between the vertical lines and the radial lines with larger slopes become less, the image quality is degraded. Thereby the proposed scan conversion of the radial super-resolution image gets completed.

where S(i, j) and S 0(i, j) denote the original and reconstructed image of size M × N, respectively. Smaller value of MSE indicates better performance. Peak signal to noise ratio: The Peak signal to noise ratio (PSNR) (in dB) is defined as,

 Max _ level 2  PSNR = 10 log10   MSE  Larger value of PSNR indicates better performance. Correlation Co-efficient: The correlation coefficient between the input and output image is given by the following relation, σ xy ρ= σxσ y where σx, σy and σxy are given by, 1 MN

σx =

σy = σ xy =

(a)

(b)

4. QUALITY METRICS

To compare the reconstructed images obtained from different approaches five well-defined quality metrics are introduced. These are: 1. Means Square Error (MSE) 2. Peak Signal to Noise Ratio (PSNR) 3. Correlation co-efficient 4. Universal quality index 5. Blur metric Mean square error: The mean square error (MSE) is given by, M

N

∑ ∑ (S (i, j ) − S (i, j )) 0

MSE =

i =1 j =1

MN

2

avg

)2

i =1 j =1

N

∑ ∑ (S (i, j ) − S 0

0 avg

)2

i =1 j =1

M

N

∑ ∑ ( S (i , j ) − S

avg

) ( S 0 (i, j ) − S 0 avg )

i =1 j =1 M

1 and S avg = MN

Figure 8: Scan-conversion by Interpolation for SR Ultrasound Reconstruction. (a) Aligned Scan Lines, (b) A Cropped Region from Figure (a)

N

M

1 MN 1 MN

M

∑ ∑ ( S (i , j ) − S

N

∑ ∑ S (i, j ), i =1 j =1

1 M N S0 avg = ∑ ∑ S0 (i, j ) MN i =1 j =1 σx and σy represent the standard deviation of the original and the reconstructed image respectively, σ xy denotes co-variance between the original and the reconstructed image, Savg and S0avg denote the average value of the image pixels of the original and the reconstructed image respectively. The range of correlation coefficient is [–1, 1], where ρ = 1 implies perfect positive correlation and indicates the highest quality; ρ = –1 indicates perfect negative correlation. Universal Quality Index: The universal quality metric (UQM) (Wang and Bovik, 2002) is given as follows: Q=

(σ

4.σ xy S avg S 0 avg 2 x

+ σ 2y

) (S

2 avg

+ S 02avg

)

where Savg, S0avg, σx, σy and σxy are as already defined in the previous part of this section. The dynamic range of Q is [–1, 1]. The best value 1 is achieved if and only if S(i, j) = S0(i, j), ∀ i, j. This quality index models image distortion as a combination of three different factors: loss of correlation, mean distortion and variance distortion. In order to understand this we rewrite the definition of Q as a product of three components: σ xy 2Savg S0 avg 2σ x σ y Q = σ σ S2 + S2 σ2 + σ2 x

y

avg

0 avg

x

y

68

International Journal of Computational Intelligence and Healthcare Informatics

The first component is the linear correlation co-efficient between S(i, j) and S0(i, j), whose dynamic range is [–1, 1]. The second component measures how close the mean values are between S(i, j) and S0(i, j). The dynamic range of the second component is [0, 1]. The best value is 1 for Savg = S 0avg. The third component measures how similar the variances of the images are. Its range of values is also [0, 1] where the best value is achieved if and only if σx = σy. Blur metric: Blur metric is a no-reference perceptual blur estimation proposed by Crete et al. (2007). According to Crete et al., if a sharp image is blurred, gray levels of neighboring pixels will undergo with a major variation. On the other hand, if an already blurred image is blurred, gray levels of neighboring pixels will still change, but the change is very small. Based on this philosophy, blur effect on an image can be quantified as follows: 1. Compute the intensity variations between neighboring pixels of the input image 2. In the second step, apply a low-pass filter on the same image and also compute the intensity variation between the neighboring pixels. A comparison between these intensity variations allows us to evaluate the blur effect. The no-reference numerical value of the blur estimate ranges from 0 to 1. The lower the numerical value of the blur estimate the lesser the blur effect present on the image. The value 0 implies the best and 1 implies the worst quality in terms of blur perception. Figure 9 shows the diagram of blur metric computation.

the function and for generating four LR images. The LR images are obtained by conventional ultrasound scan conversion technique. One of the reconstructed images is shown in Figure 10(b). It is observed that the image quality becomes poorer with increasing the depth. The reconstructions of LR frames are done by the popular bilinear interpolation technique.

(b)

(a)

Figure 10 (a) : Original Image, (b) Reconstructed LR Image

The simulated LR, conventional SR and the proposed Figure 11. It is clear that the quality of the Proposed Radial SR image is best among all the images. To compare the objective and subjective quality of the images, quality metrics are described and employed. The values of the quality metrics for the simulated images are given in Table 1 and plotted in the form of bar chart in Figure 12. r

a

d

i

a

l

S

R

o

f

t

h

e

g

(a)

Figure 9: Schematic Diagram of Blur Metric Computation

5. SIMULATION RESULTS

(d)

i

v

e

n

f

u

n

c

t

i

o

n

a

r

e

s

h

o

w

n

i

n

(b)

(c)

(e)

(f)

Figure 11: (a) - (d) LR frames; (e) Conventional SR Image; (f) Proposed Radial SR Image.

In this section some results of the simulations are introduced. For the purpose of illustrating the performance of the algorithm, we have taken an analytic function, f(x, y) = abs(sin(ωx)) where abs(•) returns the absolute value of the argument. The function is sampled along angular and radial direction with 90° field of view and 128 scan lines (the number of samples taken along radial direction is 1500) analogous to the sampling in the ultrasound imaging system. Figure 10 (a) shows the original image obtained from the function f(x, y). Besides this, three more sets of data are obtained by sampling radially the same analytic function after applying slight rotation and translation parameters to

(a)

(b)

(c)

(d)

Figure 12: Bar Chart of Simulated ‘Sine’ Phantom Image. (a) MSE, (b) PSNR, (c) Universal Quality Metric, (d) Blur Metric

69

Super-resolution on Data Acquired in Polar Format Table 1 Quality Metrics for Simulated ‘Sine’ Phantom Metrics

LR (Average)

Conventional SR (CSR)

MSE

387.3076

355.5768

22.2770

22.6215

0.3445 dB

28.1536

5.8766 dB

5.5321dB

CCF

0.9927

0.9934

0.0705%

0.9981

0.5440 %

0.4731%

Q

0.9921

0.9927

0.0580%

0.9980

0.5922 %

0.5339%

Blur

0.5161

0.5165

0.0872%

0.5032

–2.4901 %

–2.5750%

PSNR(dB)

Improvement of CSR w.r.t LR –8.1926%

The proposed algorithm is applied to ultrasoundsimulated data of a liver image (Figure 13 (a)). Field II software is used to obtain simulated ultrasound data of the liver. Field II is a software for simulating ultrasound transducer fields and ultrasound imaging. The executable program is copyrighted freeware by Jørgen Arendt Jensen. The program is capable of calculating the emitted and pulse-echo fields for both the pulsed and continuous ultrasound wave for a large number of different transducers. The parameters and their values used for simulation of ultrasound image are given in the A1 in Appendix A.

Radial SR (RSR) 99.4771

(b)

(e)

–74.3157 %

–72.0237%

(b)

(c)

(c) (d)

Improvement of RSR w.r.t CSR

Four sets of ultrasound data are generated from four shifted frames of same image. Four LR frames are formed from those four data sets. Finally, SR image is formed from those four LR frames. The original, LR, Conventional SR and Proposed SR images are shown in Fig. 13. The quality metrics of the Liver images are given in Table 2. Figure 14 shows the bar chart of the Table 2 for pictorial representation.

(a) (a)

Improvement of RSR w.r.t LR

(f)

(g)

Figure 13: Ultrasound Simulated LR and SR Images of Liver. (a) Original Image of Liver, (b) - (e) LR Frames; (f) Conventional SR Image; (g) Radial SR Image.

(d)

Figure 14: Bar Chart of Ultrasound Simulated Liver Image. (a) MSE, (b) PSNR, (c) Universal Quality Metric, (d) Blur Metric

The quality metrics in the Table 1 and Table 2 express the superiority of the proposed Radial SR with respect to LR and the conventional SR images. It found from Table 2 that % of improvement in case of ultrasound-simulated image is lesser than the simulated synthetic phantom. This is because of the presence of noise and distortion in the reconstructed image. The values of MSE, PSNR, Correlation coefficient and Universal Quality Metric depend on the reference image. The liver image has sharp edges compared to the phantom images. Part of it is due to speckle noise, which increases the error in radial SR but reduce the blur metric. The distortion and noise in ultrasound simulated images changes the shape of the image with respect to the reference image. This reduces the quality metrics of the ultrasound-simulated image.

70

International Journal of Computational Intelligence and Healthcare Informatics Table 2 Quality Metrics for Ultrasound Simulated Liver Images

Metrics

LR (Average)

Conventional SR (CSR)

Improvement of CSR w.r.t LR

Radial SR (RSR)

Improvement of RSR w.r.t LR

Improvement of RSR w.r.t CSR

MSE

3798.38

3747.60

–1.34%

3346.50

–11.90%

–10.70%

PSNR(dB)

12.34

12.39

0.06 dB

12.88

0.55 dB

0.49 dB

CCF

0.8122

0.8141

0.23%

0.8278

1.92 %

1.68%

Q

0.7947

0.7981

0.43%

0.8192

3.08%

2.64%

Blur

0.3422

0.3365

–1.67%

0.1901

–44.45%

–43.51%

5.1 Performance in Terms of Object Detectibility from an Image of the Proposed Algorithm To illustrate the performance of the algorithm in terms of detectibility of objects of different sizes, we consider an image, generated from an analytical function g(x, y). The function g(x, y) is given as, 1 g(x, y) = [sin(ωx)] + 0.5 4 Fifteen objects (black) of different sizes are created within the image at different depths as shown in Fig. 15. 128 Scan lines are generated by sampling the function in 900 field of view with uniform angular step-size. The LR and SR images reconstructed from the scan lines obtained from the analytical function are shown in the Fig. 16. The LR and SR images are obtained in the same way as described in Section 5.

Figure 15: Phantom Image Generated by g(x, y) with a Fifteen Holes of Different Sizes on It.

(e)

(f)

Figure 16: LR and SR Reconstructed Images of g(x, y). (a) - (d) LR Images; (e) Conventional SR Image; (f) Radial SR Image.

Observation confirms that, not a single LR frame can reconstruct all the objects of the original image. For example, the first LR frame loses two small objects; the second LR frame loses eight objects and so on. The reconstructed SR image contains all the objects of the original image. Consequently, the detectibility of small objects is possible by using the proposed SR reconstruction techniques, which is not possible by conventional ultrasound imaging systems. The performance of object detectibility of the proposed SR reconstruction technique based on the above images is better and quantitatively is given in Table 3. From Table 3 we see that the average loss of object in the LR frames is about 33.33% whereas there is no the loss of object in case of SR image is. These data may slightly vary depending on the size and number of the objects and the accuracy of the registration. For the given image the conventional SR image contains all the objects, but the contrast resolution of the image is lower than the image reconstructed by the proposed technique (Table 4). From Table 4, it is evident that the detectibility of the proposed SR reconstruction is much better than the conventional ultrasound reconstruction techniques. The local contrast of the objects is measured using the following equation: C=

(a)

f −b f +b

(b)

where f is the average value of the foreground and b is the average value of the background. The Threshold value for separating the foreground and the background is obtained by Otsu method locally around the objects. The values of the contrast of the objects in the Convention SR image and (c)

(d)

71

Super-resolution on Data Acquired in Polar Format Table 3 Object Detectibility Images

Number of objects visible

Number of objects lost

% of object loss

Average loss of objects (%)

LR frame 1

13

2

13.33

LR frame 2

7

8

53.33

LR frame 3

9

6

40

LR frame 4

11

4

26.67

Conventional SR image

15

0

0

0

Proposed SR image

15

0

0

0

33

Table 4 Contrast of the Objects O1

O2

O3

O4

O5

O6

O7

O8

O9

O10

O11

O12

O13

O14

O15

CSR

0.65

0.67

0.54

0.16

0.15

0.52

0.50

0.13

0.13

0.09

0.47

0.48

0.14

0.07

0.06

RSR

0.57

0.64

0.36

0.51

0.77

0.62

0.66

0.40

0.49

0.42

0.54

0.59

0.56

0.39

0.46

the proposed SR image are given Table 4 with reference to the image of Fig. A. 1 is shown in Appendix A. The objects are marked and numbered in Fig. A.1 in the Appendix A. It is observed from the table that average loss of objects from the LR frames is 33.33% and for conventional SR three objects (O10, O14 and O15) have the very low contrast level i.e. 0.089, 0.075 and 0.058 respectively. These contrast levels are too low and there is a major chance to miss the objects at the time of detection in noisy images. 5. CONCLUSION

A new Super-resolution reconstruction algorithm in polar domain is developed which is applicable directly on image data available in polar format. The new technique can be employed to the existing ultrasound imaging systems with minimal change of existing hardware of the systems. It is demonstrated that one can use the conventional SR techniques to the LR frames obtained after scan conversion. But conventional SR after scan-conversion is unable to provide adequate improvement. This problem is addressed by developing a new SR reconstruction procedure, which is applied directly on the sampled radial lines. The new technique provides a way to model relative motion of multiple low-resolution data (in polar co-ordinate) through the registration of the images generated from low-resolution data (polar co-ordinate). SR in polar domain provides more information and better image quality than the LR images and conventional SR images (which performed in Cartesian co-ordinate) as there is no loss of information due to lowpass nature of interpolation in the successive stages of reconstruction. The new technique improves the image quality, detectivity with respect to conventional SR and LR output images. The improvement is more evident at higher depth and for small objects. This makes the new technique more promising. More research is needed to test the robustness of the proposed RSR technique in presence of noise before its application to the real field.

References [1] Ben-Ezra M., Zomet A., Nayar S.K., (2005). “Video Resolution from Controlled Sub-pixel Detector Shifts”, IEEE Trans. Pattern Analys. and Machine Intel., 27(6), pp. 977-987. [2] Berkhoff A.P., Huisman H.J., Thijssen J.M., Jacobs E.M.G.P. and Homan R.J.F., (1994). “Fast Scan Conversion Algorithms for Displaying Ultrasound Sector Images”, Ultrasonic Imaging, Vol. 16, pp. 87-108. [3] Bernstein C.A., Kanal L.N., Lavin D., and Olson E.C., (1987). “A Geometric Approach to Subpixel Registration Accuracy”, Computer Vsion, Graphics, and Image Proc., Vol. 40, pp. 334-360. [4] Borman S., and Stevenson R.L., (1998). “Spatial Resolution Enhancement of Low-resolution Image Sequences. A Comprehensive Review with Directions for Future Research”, Lab. Image and Signal Analysis, University of Notre Dame, Tech. Rep. [5] Brown L.G., (1992) “A Survey of Image Registration Techniques”, ACM Comput. Surveys, 24(4), pp. 325-376. [6] Carotenuto R., Sabbi G., and Pappalardo M., (2002). “Spatial Resolution Enhancement of Ultrasound Images Using Neural Networks”. IEEE Trans. on Ultra., Ferro., and Freq. Control, 49(8), pp. 1039-1059. [7] Chowdhuri S., (2002). “Super-resolution Imaging”, Book Chapter 1', Kluwer Academic Publisher. [8] Clement G.T., Hutunen J., and Hynynen K., (2005). “Superresolution Ultrasound Imaging Using Back-projected Reconstruction”. J. Acoust. Soc. America. 118(6), pp. 3953-3960. [9] Crete F., Dolmiere T., Ladret P., and Nicolas M., (2007). “The Blur Effect: Perception and Estimation with a New No-Reference Perceptual Blur Metric”, SPIE Electronic Imaging Symposium Conf. Human Vision and Electronic Imaging, San Jose: ÉtatsUnis d’Amérique (2007). [10] Dah-Chung Chang, Wen-Rong Wu, Jiann-Hwa Jeng, (1992). “An Efficient Architecture of Ultrasonic Scan Conversion for Implementing the Cubic Convolution Interpolation”, Nuclear Science Symposium IEEE, pp. 1546-1550. [11] Dantas R.G., and Costa E.T., (2007). “Ultrasound Speckle Reduction Using Modified Gabor Filters”, IEEE Trans. Ultra., Ferro., and Freq. Control, 54(3), pp. 530-538.

72

International Journal of Computational Intelligence and Healthcare Informatics

[12] Dutt V., and Greenleaf J.F., (1996). “Adaptive Speckle Reduction for Log-Compressed B-Scan images”, IEEE Trans. Med. Imag., 15(6), pp. 802-813.

[31]

[13] Dvorchenko V.N., (1983). “Bounds on (Deterministic) Correlation Functions with Applications to Registration”, IEEE Trans. Pattern Anal. Machine Intell., 5(2), pp. 206-213.

[32]

[14] Elad M., and Feuer A., (1997). “Restoration of a Single Super-resolution Image from Several Blurred”, Noisy, and Undersampled Measured Images”, IEEE Trans. Imag. Proc., 6(12), pp. 1646-1658. [15] Field_II Simulation Software: http://server.oersted.dtu.dk/ personal/jaj/field/

[33]

[34]

[16] Irani M., and Peleg S., (1991). “Improving Resolution by Image Registration”, CVGIP: Graphical Models and Image Proc., Vol. 53, pp. 231-239. [17] Jang H.J., Lim H.K., Lee W.J., Kim S.H., Kim K.A., and Kim E.Y., (2000). “Ultrasonographic Evaluation of Focal Hepatic Lesions: Comparison of Pulse Inversion Harmonic, Tissue Harmonic and Conventional Imaging Techniques”, American Institute of Ultrasound in Medicine, J Ultrasound Med. Vol. 19, pp. 293-299. [18] Kennedy J.A., Israel O., Frenkel A., Bar-Shalom R., Haim Azhari (2006). “Super-resolution in PET Imaging”, IEEE Trans. on Med. Imag., 25(2), pp. 137-147. [19] Kim S.P., and Bose N.K., (1990). “Reconstruction of 2-D Bandlimited Discrete Signals from Nonuniform Samples”, Proc. IEE., Vol. 137, pp. 197-204. [20] Kudo M., (2001). “Contrast Harmonic Ultrasound is a Breakthrough Technology in the Diagnosis and Treatment of Hepatocellular Carcinoma”. J. Med Ultrasonics, 28(3), pp. 79-81. [21] Lee M.H., Kim J.H., and Park S.B., (1986). “Analysis of a Scan Conversion Algorithm for a Real Time Sector Scanner”, IEEE Trans. Med. Imag., Vol. 5, pp. 96-105. [22] Lehmann T.M., Gonner C., and Spitzer K., (1999). “Survey: Interpolation Methods in Medical Image Processing”, IEEE Trans. Med. Imag. 18(11), pp. 1049-1075. [23] Li Rui, Sun Zhuoxin, and Zhang C., (2008). “Adaptive Filter for Speckle Reduction with Feature Preservation in Medical Ultrasound Images”, 10th Intl. Conf. on Control, Automation, Robotics and Vision, Hanoi, Vietnam, 17-20 December 2008, pp. 1787-1792. [24] Li Y., and Zagzebski J.A., (2000). “Computer Model for Harmonic Ultrasound Imaging”. IEEE Trans. on Ultra, Ferro., and Freq. Control, 47(4), pp. 1259-1272. [25] Michailovich O.V., and Tannenbaum A., (2006). “Despeckling of Medical Ultrasound Images”, IEEE Trans. on Ultra., Ferro. and Freq. Control, 53(1), pp. 64-78. [26] Nguyen N., Milanfar P., and Golub G., (2001). “Efficient Generalized Cross-validation with Applications to Parametric Image Restoration and Resolution Enhancement”, IEEE Trans. Imag. Proc., 10(9), pp. 1299-1308.

Conversion via Linear Interpolation of Oversampled Vectors”, Ultrasonic Imaging, Vol. 16, pp. 109-123. Robinson D.E., and Knight P.C., (1982). “Interpolation Scan Conversion in Pulse-echo Ultrasound”, Ultrasonic Imaging, Vol. 4, pp. 297-310. Stark H., and Oskoui P., (1989). “High Resolution Image Recovery from Image-plane Arrays, using Convex Projections”, J. Opt. Soc. America A, 6(11), pp. 1715-1726. Taxt T., and Jirik R., (2004). “Superresolution of Ultrasound Images Using the First and Second Harmonic Signal”, IEEE Trans. on Ultra., Ferro., and Freq. Control, 51(2), pp. 163-175. Tekalp A.M., Ozkan M.K., and Sezan M.I., (1992). “Highresolution Image Reconstruction from Lower-resolution Image Sequences and Space Varying Image Restoration”, Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing (ICASSP), San Francisco, CA., Vol. 3, pp. 169-172.

[35] Tian Q., and Huhns M.N., (1986). “Algorithm for Subpixel Registration”, Computer Vision, Graphics, Image Proc., Vol. 35, pp. 220-233. [36] Unser M., (1999). “Splines A Perfect Fit for Signal and Image Processing”, IEEE Signal Processing Magazine, November. [37] Wang W.P., Wu Y., Luo Y., Li R., Zhou X.D., Zhang J., Qian C. W., Tan X.Y., Xu Q. H., Wang Y., and Yuan J.J., (2009). “Clinical Value of Contrast-enhanced Ultrasonography in the Characterization of Focal Liver Lesions: A Prospective Multicenter Trial”, Hepatobiliary Pancreat Dis. Int., 8(4), pp. 370-376. [38] Wang Z., and Bovik A.C., (2002). “A Universal Image Quality Index”, IEEE Signal Processing Letter, 9(3), pp. 81-84.

APPENDIX A

A1. Parameters used for the simulation for the ultrasound image: • • • • • • • •

Number of scan-lines: Field of View: Angular step-size: Ultrasound frequency: Envelop sampling frequency: The Velocity of ultrasound (c): Number of elements: Wavelength (λ):

• Width of the element: • Element height: • Kerf: • Number of scatters:

[27] Ophir J., and Maklad N.F., (1979). “Digital Scan Converters in Diagnostic Ultrasound Imaging”, Proceedings, IEEE, 67(4), pp. 654-664. [28] Ophir J., and Brinch J.M., (1982). “Moire Undersampling Artifacts in Digital Ultrasound Image”, Ultrasonic Imaging, Vol. 4, pp. 311-320. [29] Park S.C., Park M.K., and Kang M.G., (2003). “Super-resolution Image Reconstruction: A Technical Overview”, IEEE Signal Processing Magazine, May(2003), pp. 21-36. [30] Richard W.D., and Arthur R.M., (1994). “Real-time Scan

Figure A1: Different Objects on the Image

128 90° 0.7301° 5 MHz 100 MHz 1540 meters/sec 128 3.08 × 10–4 meters λ 2 0.005 meters λ meters 10 100000