Processing of Image Data Using FPGA-Based MicroBlaze Core Swagata Samanta1, Soumi Paik1, Shreedeep Gangopadhyay1, and Amlan Chakrabarti2 1
Dept. of Electronics and Communication Techno India, Kolkata, India
[email protected],
[email protected],
[email protected] 2 A.K. Choudhury School of Information Technology, University of Calcutta, India
[email protected]
Abstract. This paper proposes the technique of storing of image data in the FPGA memory and subsequent processing of the stored image data using MicroBlaze processor core of the Xilinx FPGA device. Though related research work exists in processing image data using DSP blocks available in FPGA devices but very little work exists in processing the same using FPGA based processor core. This type of design is extremely important for real time embedded system design for image processing applications. Our work deals with the inversion of a binary image in the FPGA memory and the recovery of the inverted image into its original form for verification of the inversion process. We have used the Xilinx EDK 11.1 tool, Spartan 3E FPGA kit and MATLAB is used for pre and post processing of the image data. Keywords: EDK, FPGA, image inversion, image processing, System C.
1 Introduction The current available technologies like Microcontroller-based systems, DSP processor-based systems possess programmable processors, which are based on the Von-Neumann architectural concept, where a given application (program) is stored in the memory in sequential fashion. So, these Von-Neumann processors fail to exploit the concurrency in the algorithm even though they offer flexibility. Thus performance is not achieved for many applications especially where high speed is required. This leads to an alternative solution where, speed of the hardware can be achieved retaining the flexibility of the software (programmability). Field programmable gate array (FPGA) represents an evolutionary improvement in gate array technology, which offers potential reductions in prototype system costs and product time-to-market, and simplifies debugging and verification. Re-programmable FPGAs are capable of dynamically changing their logic and interconnect structure to adapt to changing system requirements. Hardware design techniques such as parallelism and pipelining techniques can be developed on a FPGA, which is not possible in dedicated DSP designs 3. A. Mantri et al. (Eds.): HPAGC 2011, CCIS 169, pp. 241–246, 2011. © Springer-Verlag Berlin Heidelberg 2011
242
S. Samanta et al.
During the last few years’ researchers made an unprecedented effort in the field of image processing in hardware. Prior research may be categorized based on the types of hardware and the image processing algorithms implemented. The types of hardware considered for image processing include Application Specific Integrated Chips (ASIC), Digital Signal Processors (DSP) and Reconfigurable Logic Devices (FPGA).The image processing algorithms considered for hardware implementation include: convolution, image filtering and edge detection (Sobel’s, Prewitt’s and Canny’s edge detection). Some researchers have also considered hardware implementations specific to FPGA vendors like Xilinx, Actel and Altera. Most of the system level hardware programming languages introduced and commonly used in the industry are highly hardware specific and requires intermediate to advance hardware knowledge to design and implement the system. In order to overcome this bottleneck various C-based hardware descriptive languages have been proposed over the past decade 3, 4.C-based hardware descriptive languages have been proposed and developed since the late 1980s. Some of the C-based hardware descriptive languages include: Cones, HardwareC, TransmogrifierC, SystemC, OCAPI, C2Verilog, Cyber, SpecC, NachC, CASH. This paper deals with FPGA technology and the development environment used is Xilinx embedded development kit (EDK) 1, 2. EDK is a Xilinx software suite for designing complete embedded programmable systems and this enables the integration of both hardware and software components of an embedded system. The soft-core processor that is used is MicroBlaze. The MicroBlaze has a versatile interconnect system to support a variety of embedded applications. It consists of about 900 LUTs having RISC architecture 2. The programming platform that is required here is SystemC 7. Another feature of our design is that in our FPGA system (Spartan 3E) there was no dedicated graphics port but still we were successful in transmitting the image data using the serial RS232 interface. Image processing involves processing or altering an existing image in a desired manner. It is a computational process used to convert an image data into a desired image representation. Image information demands visual perception. Therefore, displaying image information in various forms is a necessity 6.The first step is obtaining an image in a readable format. Once the image is in a readable format, image processing software needs to read it so that it can be processed and written back as an image file. The organization of the paper is as follows: Section 2 shows the image data basics. Section 3 describes the proposed methodology. Section 4 deals with the results. Applications, limitations and suggestion for further improvement are presented in sections 5 and 6 respectively. In Section 7 we discuss on some important issues of our work.
2 Image Data Basics An image consists of a two-dimensional array of numbers. The color or gray shade displayed for a given picture element (pixel) depends on the number stored in the array for that pixel. In binary image, image data is black and white. Here each pixel is either 0 or 1.Gray-scale images appear like common black-and-white photographs;
Processing of Image Data Using FPGA-Based MicroBlaze Core
243
they are black, white, and shades of gray. This paper discusses methods of viewing an image into a desired form. This paper presents a program to display the raw pixel data of an image in text format and the recovery of the image in its original form.
Fig. 1. Gray-scale image
3 Proposed Methodology From the 2-D image the raw pixel data is transferred into an environment where by taking the pixel data our desired operation is accomplished. Here the environment is text environment. Inversion logic 5 is applied on the raw readable image data through MicroBlaze processor and EDK tool. The inverted image is shown in Figure 5.
Fig. 2. Inverted pixel data obtained by applying inversion logic
Inverted text raw image data is obtained. After obtaining the inverted data reverse process is applied in order to get the original image. The pixel data of the inverted image is viewed at hyper terminal through RS-232 serial port which is shown in Figure 8.
Fig. 3. Flow diagram of the proposed methodology
244
S. Samanta et al.
Figure 1 is the example gray scale image, which is converted into binary image. Figure 3 shows the Flow diagram of the proposed methodology. First block represents the MATLAB-based pre-processing of the image. MATLAB-based pre-processing generates the required binary data of the image which is downloaded to the FPGA memory through RS-232 serial port. Next the MicroBlaze processing unit processes the image data and stores the modified data back to the memory. Through the RS-232 serial port the binary data of the image is transmitted to the computer for subsequent MATLAB-based post-processing and verification of the processed image data. Ultimately we get the inverted image of the initial input image.
4 Results The IO devices that are needed to perform the entire process are: DDR_SDRAM, RS232_DCE, LEDs, DIP_Switches_4bit, Buttons_4bit. Table1 shows the device utilization summary. Table 1. Device utilization summary
logic utilization no. of slice flip flops no. of 4 input LUTs no. of occupied slices no. of slices containing only related logic no. of slices containing unrelated logic Total no. of 4 input LUTs
used
available
utilization
2,561
9,312
27%
2,625
9,312
28%
2,429
4,656
52%
2,429
2,429
100%
0
2,429
0%
2,747
9,312
29%
Figure 4 shows the binary data which is being converted from gray scale image (Figure 1) using MATLAB. The binary image is then processed in EDK and the corresponding pixel data is viewed at hyper terminal (Figure 5). After processing in EDK the inverted image data is obtained. Next, the inverted image data is transferred to the computer from the SDRAM memory of the FPGA kit through the RS232 serial link. The image data is recovered into its binary form by the use of MATLAB based post processing. Figure 6 and 7 shows the pixel data of the binary image and the inverted image respectively. The final inverted image is obtained as shown in Figure 8. This experimental result verifies the inversion processing using the Microblaze processor core.
Processing of Image Data Using FPGA-Based MicroBlaze Core
Fig. 4. Binary image
Fig. 5. Pixel data viewed at hyper terminal
Fig. 6. Pixel data of binary image
Fig. 7. Pixel data of inverted image
Fig. 8. Inverted image
245
246
S. Samanta et al.
5 Applications This proposed development work is useful for the real time embedded applications where processing of image data is needed. In surveillance applications there is a need of processing data from visual sensors in real time, for effective decision making and hence embedded systems can give a better solution. Our work addresses this problem and we have tried to develop FPGA based embedded system for processing image data. We have adopted the processor core based design which will be effective for interfacing the system with different data communication protocols easily compared to the DSP based image processing existing in the related research works
6 Limitations and Suggestions for Improvement We have implemented our design on Spartan3E, which has small RAM size so our test image sizes are small. In higher versions of FPGA i.e. Vertex II or Vertex IV we expect to implement the design for larger image sizes and also aim for video data processing.
7 Discussion As EDK is a new development tool in embedded system design domain, inverting a binary image is really a challenging job. In Spartan 3E device, there is no graphics port to download an image directly so the image data is converted to raw binary data and is transferred as a file to the FPGA using the RS232 link. So it can be concluded that this is a useful design for devices which don’t have a dedicated graphics port but still we can transfer the image data if any other communication link is available.
References 1. Introduction to EDK 10.1 and the Digilent V2Pro Board” – online tutorial 2. EDK Concepts, Tools and techniques Version EDK 9.2i, http://www.xilinx.com 3. Rao, D.V., Patil, S., Babu, N.A., Muthukumar, V.: Implementation and Evaluation of Image Processing Algorithms on Reconfigurable Architecture using C-based Hardware Descriptive Languages. International Journal of Theoretical and Applied Computer Sciences 1(1), 9–34 (2006) 4. Edwards, S.A.: The challenges of hardware synthesis from C-like languages. International Journal of Theoretical and Applied Computer Science (June 2004); Proc. IWLS, Temecula, California 5. Image arithmetic-invert/Logical NOT –online tutorial 6. Displaying different image types, http://www.mathworks.com/help/toolbox/images/f10-30847.html 7. Grotker, T., Liao, S., Martin, G., Swan, S.: System Design with System. Kluwer, Dordrecht (2002)
