Real Time Image Segmentation using watershed ...

Bhupinder Verma et al. / International Journal of Engineering Science and Technology (IJEST)

Real Time Image Segmentation using watershed algorithm on FPGA BHUPINDER VERMA Department of ECE Lovely Professional University Phagwara Punjab India [email protected]

Dr. H.K.Sardana Dy. Director Central Scientific Instruments Organization Chandigarh India [email protected] Abstract : Watershed transformation is a powerful technique that can be efficiently used for image segmentation. Its use for classifying and grading rice kernels has been discussed followed by its implementation on reconfigurable devices. In this paper, we implement a watershed based segmentation algorithm on a Virtex-6 platform. The main contribution of this work is the low execution time and minimal internal hardware resource occupation. Watershed has been written in C-code and also in Verilog hardware descriptive language. The Verilog code is synthesized on Virtex-6 Xilinx platform. Execution timing in both these approaches have been analyzed and the hardware approach has been found to be much faster than its software counterpart. Keywords: Image segmentation, Watershed, Reconfigurable Systems, FPGA, Computer vision, Embedded imaging 1. Introduction: A recent survey on modern methods and tools for reconfigurable embedded systems reveals that the area of reconfigurable systems is very promising and offers quite new opportunities for development [1]. It is further observed that these reconfigurable systems are enhancing capabilities of image processing and computer vision domains as well. Applications such as high speed automated visual inspection; intelligent robotics, surveillance, and transportation all require real-time complex vision systems. The nature of these applications and the hard timing constraints associated with them makes the use of reconfigurable architectures such as FPGA (Field Programmable Gate Arrays) a suitable implementation platform [2]. One such embedded system has been reported for high speed inspection of grains recently [3]. Keeping in view these developments, it was felt to implement the watershed based image segmentation algorithm used in classification and grading of rice [4]. Rice classification and grading as per standards defined by USDA has been done using Matrox imaging library on a personal computer (PC) having 128MB RAM. Rice kernels are defined as broken when they are smaller than three-fourths of whole kernel by USDA, 1995 [5]. Broken rice kernels are separated during milling of rice (removal of the hull and bran layers from the rough rice) and re-added later depending on the desired quality and

ISSN : 0975-5462

Vol. 3 No. 6 June 2011

5282


price. Consumers prefer as few broken rice kernels as possible. This PC based system involved image analysis based on binarization of a scanned image followed by parameter extraction using watershed segmentation. The organization of this paper is as follows: section 2 presents the Watershed based segmentation process and its underlying principle, section 3 details our design approach and implementation results. Finally, we end with a brief conclusion. 2. Watershed algorithm Watershed transformation is a powerful tool that demonstrates good performances in several applications like object based motion estimation [7], medical [8], multi-spectral satellite imagery [9], image processing and computer vision [10], etc. Image segmentation is an important pre-processing step for most image analysis tasks. The general segmentation problem involves the partitioning of a given image into a number of homogeneous regions. Therefore, segmentation can be considered as a pixel labeling process where pixels belonging to the same homogenous regions are assigned the same label. However, this definition hides a tremendous complexity which makes us facing many challenges in the implementation. Several approaches have been proposed for the implementation of the watershed algorithm [10-12]. In [10], a watershed hardware implementation derived from Meyer's simulated flooding-based algorithm is described is based on ordered queues which add high complexity in the design. Hardware implementation offers the advantage of speeding up the processing. But it can be a very hard and time consuming task facing hardware resources limitations on FPGA or complexity in the algorithm. Whereas, software based implementation offers simplicity and enables generality and flexibility. However, it often slows down the application. Hardware/software co-design methodology to achieve a trade-off between speed and hardware occupation has also been reported [2]. In this paper, we have implemented an immersion based algorithm proposed by Vincent-Soille [6]. Accordingly, during immersion process, water progresses in the relief, starting with valleys until adjacent lakes meet. The intercepting points constitute Watershed. Fig 1 illustrates the immersion principle

Fig1. Immersion principle

Implementation of the Watershed algorithm includes the following steps: • • •

Sorting of the image grey levels. Extraction of the coordinates of all the pixels for each level. Processing of the image level by level in the following way:

For each pixel, if it has a labeled neighbor, it inherits this label, otherwise a new label is assigned and the pixel takes the new label. If two neighbors are labeled differently, then the pixel is considered as a watershed and so on until processing the entire image. Figure 2a presents an image of size 128*128 and figure 2b is its watershed transform. It is observed that the image obtained is over-segmented. To reduce this over segmentation we did quantification of the image. Figure 2c presents the image obtained after the application of the watershed algorithm on a quantified image.

ISSN : 0975-5462


5283


(a)

(b)

(c)

Fig.2.Application of the watershed algorithm on a cameraman image; a) Initial image. b) its watershed transform. c) The same processing done for a quantified image.

3. Methodology and implementation results An image taken using flatbed scanner [4] has been converted into bmp image using a C-code. It is pre-processed to achieve Binarized image. Another C-code has been written to implement Watershed algorithm on this Binarized image for segmentation. The execution time using this C-code watershed algorithm has been calculated using time.h library for a 900x900 image and is found to be 5µs on IBM opteron workstation with linux operating system. The input and output images obtained after application of the watershed segmentation is shown in Figure 3.

Fig.3: (a) Input image

(b) output image

We have divided the architecture into three main blocks - erosion, dilation and watershed (algorithm by S. Beucher and F. Meyer). The Verilog code reads a bit stream from memory and would write raw image array into external memory through 8/16 bit port. Camera

Image Acquisition (Camera Link)

External Memory Controller

FPGA

Frame Control

Image Processing Block

Frame buffering


Vision Blocks

One or more processing stages Image Acquisition (USB Interface)


Control Registers

Other Processing stages

To PC Host (USB Interface)

Synchronization & Packing

Reconfigurable Image Processing & Vision Stage

From PC Host (USB Interface) (a)

architecture

ISSN : 0975-5462


5284


Parameters PIXEL CLOCK Pixel Data Input Valid Pixel End of line signal End of frame signal

Image Processing Blocks

Pixel Data Input Valid Pixel End of line signal End of frame signal

RESET

(a)

Processing element

Fig.4: Reconfigurable architecture for Vision Systems

The proposed architecture has been modeled in Verilog for a 50*50 image and synthesized for Virtex-6 and the synthesis report is shown in figure 5. Execution time for blocks of nine pixels, in parallel, is calculated to be 0.924ns which is approximately five thousand times faster than C-code implementation.

Fig.5 Watershed implementation results on Xilinx xc6slx150t-3fgg676 device

Further, the hardware resource occupation is shown in table I below;

ISSN : 0975-5462


5285

Bhupinder Verma et al. / International Journal of Engineering Science and Technology (IJEST) Logic utilization No. of Slice registers No. of Slice LUTs No. of fully used LUTSFF pairs No. of. Bonded I/OBs No. of BUFGs / BUFG CTRLS

Available 184304 92152 80

Used 37 80 37

Utilization 0.02% 0.09% 46%

396 16

13 2

3% 12%

Table-I: Device utilization summary

Further a comparison between execution speed / frequency of operations reported till date for the watershed segmentation implemented by different authors and using different techniques is tabulated below; Table-II Comparison between the different methods of watershed implementation:

Algorithm Parallel & pipelined watershed Watershed flooding based algorithm Gradient + Watershed Techniques (Hw/Sw Co-design) C-Code watershed implementation Parallel implementation Virtex-6

on

Image

Slices

f[MHz)

Ref.

512x512

92%

66 MHz

Ref.11

30x30

64%

55 MHz

Ref.10

128x128

21%

100 MHz

Ref.2

900x900

Softw are only

0.2 MHz

This paper

50x50

0.02%

1000MHz

This paper

4. Conclusion: Parallel implementation of a popular image processing algorithm for segmentation namely watershed algorithm has been tried and presented in this paper. Results obtained reveal that with the availability of powerful reconfigurable devices, it is possible to achieve much faster execution of image processing algorithms, which may help realize real time computer vision systems. Towards future prospects, it is visualized to develop handy grader for rice and grain kernels.

Acknowledgments: The authors wish to thank Lovely Professional University for allowing EDA tools available in VLSI Design Lab for conducting this research. We also thank engineers at Cadence Design Systems India for technical support from time to time References [1] [2] [3] [4] [5]

[6] [7]

J. Lech, N. Nadia and Figueroa M “Modern development methods and tools for embedded reconfigurable systems: a survey” Integration, the VLSI journal Volume 43(2010) pp, 1-33, Elsevier. Elhossini. A, and Moussa. M “A Reconfigurable Architecture for Real-Time Vision Systems on FPGA” proceedings of IEEE, 22nd International Conference on Microelectronics (ICM 2010) Pearson T. “Hardware-based image processing for high speed inspection of grains” Computers and Electronics in agriculture, Volume 69(2009), pp. 12-18, Elsevier. Verma Bhupinder “Image Processing Techniques for Grading & Classification of Rice” proceedings IEEE International Conference on Computer & Communication Technology, 2010 USDA. United States standards for rice, 868—definition of terms. United States Department of Agriculture, Washington, DC. Available at http://www.usda.gov/gipsa/referencelibrary/standards/rice.htmUnited%20States%20Standards%20for%20Milled%20Rice1995 Vincent, L., and Soi1le, P.: 'Watersheds in digital spaces: an efficient algorithm based on immersion simulations', IEEE Trans. Pattern Anal. Mach. Intell., 1991, 13, (6), pp. 583-598 R.i, S.Yu "Efficient Spatio-temporal Segmentation for Extracting Moving Objects in Video Sequences" Institute of Image Communication and Information Processing of Shanghai Jiao Tong University, Shanghai, P.R. July 2007

ISSN : 0975-5462


5286

Bhupinder Verma et al. / International Journal of Engineering Science and Technology (IJEST) [8]

P. Dillinger, J. F. Vogelbruch, J. Leinen, S. Suslov, R. Patzak, H. Winkler, and K. Schwan "FPGA-Based Real-Time Image segmentation for Medical Systems and Data Processing" IEEE Transactions on nuclear Science VOL. 53, NO.4, August 2006 [9] S.Chenl, J.Luol, Z.Shenl, X.Hul, L. Gaol "Segmentation of Multi-spectral Satellite Images Based on Watershed Algorithm" Institute ofRemote Sensing Application, Chinese Academy of Scienses, Beijing, 100101, China 2008 [10] C. Rambabu, Chakrabarti and A. Mahanta "Flooding-based watershed algorithm and its prototype hardware architecture" lEE Proc.Vis. Image Signal Process., Vol. 151, No.3, June 2004 [11] Dang Ba Khac, Trieu Maruyama, Tsutomu Maruyama Implementation of parallel and pipelined watershed algorithm on FPGA Systems and Information Engineering, University of Tsukuba 2006 [12] Dang Ba Khac, Trieu Maruyama, Tsutomu Maruyama "A pipeline Implementation of a watershed algorithm on FPGA" Systems and Information Engineering, University of Tsukuba 2007.

ISSN : 0975-5462


5287