electronics Article
FPGA-Based Real-Time Motion Detection for Automated Video Surveillance Systems Sanjay Singh 1, *, Chandra Shekhar 1,† and Anil Vohra 2,† 1 2
* †
CSIR—Central Electronics Engineering Research Institute (CSIR-CEERI), Pilani 333031, Rajasthan, India;
[email protected] Electronic Science Department, Kurukshetra University, Kurukshetra 136119, Haryana, India;
[email protected] Correspondence:
[email protected]; Tel.: +91-1596-252214; Fax: +91-1596-242294 These authors contributed equally to this work.
Academic Editors: Ignacio Bravo-Muñoz, Alfredo Gardel-Vicente and José L. Lázaro-Galilea Received: 21 December 2015; Accepted: 4 March 2016; Published: 11 March 2016
Abstract: Design of automated video surveillance systems is one of the exigent missions in computer vision community because of their ability to automatically select frames of interest in incoming video streams based on motion detection. This research paper focuses on the real-time hardware implementation of a motion detection algorithm for such vision based automated surveillance systems. A dedicated VLSI architecture has been proposed and designed for clustering-based motion detection scheme. The working prototype of a complete standalone automated video surveillance system, including input camera interface, designed motion detection VLSI architecture, and output display interface, with real-time relevant motion detection capabilities, has been implemented on Xilinx ML510 (Virtex-5 FX130T) FPGA platform. The prototyped system robustly detects the relevant motion in real-time in live PAL (720 ˆ 576) resolution video streams directly coming from the camera. Keywords: motion detection; VLSI architecture; FPGA implementation; video surveillance System; smart camera system
1. Introduction The importance of motion detection for designing an automated video surveillance system can be gauged from the availability of a large number of robust and complex algorithms that have been developed to-date, and the even larger number of articles that have been published on this topic so far. The problem of motion detection can be stated as “given a set of images of the same scene taken at several different times, the goal of motion detection is to identify the set of pixels that are significantly different between the last image of the sequence and the previous images” [1]. The simplest approach to motion detection is the frame differencing method in which motion detection can be achieved by finding the difference of the pixels between two adjacent frames. If the difference is higher than a threshold, the pixel is identified as foreground, otherwise background. The threshold is chosen empirically. Different methods and criteria for choosing the threshold have been surveyed and their comparative results have been reported in the literature [2–4]. Researchers have reported several motion detection methods that are closely related to simple differencing e.g., change vector analysis [5–7], image rationing [8], and frame differencing using sub-sampled gradient images [9]. The simplicity of frame differencing based approaches comes at the cost of motion detection quality. For a chosen threshold, simple differencing based approaches are unlikely to outperform the more advanced algorithms proposed for real-world surveillance applications. There are several other motion detection techniques such as predictive models [10–14], adaptive neural networks [15], and shading
Electronics 2016, 5, 10; doi:10.3390/electronics5010010
www.mdpi.com/journal/electronics
Electronics 2016, 5, 10
2 of 18
models [16–18]. A comprehensive description and comparative analysis of these methods has been presented by Radke et al. [1]. The practical real-world video surveillance applications demand a continuous updating of the background frame to incorporate any permanent scene change, for example, light intensity changes in the day time must be a part of the background. For this purpose, several researchers [19–21] have described adaptive background subtraction techniques for motion detection using a Gaussian Density Function. They are capable of handling illumination changes in a scene with stationary background scenarios. Due to pseudo-stationary nature of the background in real-world scenes, assuming that background is perfectly stationary for surveillance applications is a serious flaw. For example, in a real-world video scene, there may be swaying branches of trees, moving tree leaves in windows of rooms, moving clouds, the ripples of water on a lake, or a moving fan in the room. These are small repetitive motions (typically not important) and so should be incorporated into background. The single background model based approaches mentioned above are incapable of correctly modeling such pseudo-stationary backgrounds. Stauffer and Grimson [22] recognized that these kinds of pseudo-stationary backgrounds are inherently multi-model and hence they developed the technique of an Adaptive Background Mixture Models, which models each pixel by a mixture of Gaussians. According to this method, every incoming pixel value is compared against the existing set of models at that location to find a match. If there is a match, the parameters of the matched model are updated and the incoming pixel is classified as a background pixel. If there is no match, the incoming pixel is motion pixel and the least-likely model (model having minimum weighted Gaussian) is discarded and replaced by a new one with incoming pixel as its mean and a high initial variance. However, maintaining these mixtures for every pixel is an enormous computational burden and results in low frame rates when compared to previous approaches. Butler et al. [23] proposed a new approach, similar to that of Stauffer and Grimson [22], but with a reduced computational complexity. The processing, in this approach, is performed on YCrCb video data format, but it still requires many mathematical computations and needs large amounts of memory for storing background models. In parallel with the algorithm development, many researchers from VLSI design and system design research community published several research articles describing different hardware design based approaches to address the issue of real-time implementation of motion detection algorithms. These methods of hardware based motion detection reported in literature differ from each other due to design methodologies/approaches and design tool chains used. Based on design methodologies, various hardware based methods can be categorized as general purpose processor based approach, digital signal processor (DSP) based approach [24–26], complex programmable logic device (CPLD) based approach [27], application specific integrated circuit (ASIC) based approach [28,29], FPGA based hardware design approach [30–49], and FPGA based hardware/software co-design approach [50–52]. From design tool-chain perspective, the differences can be based on the use of VHDL/Verilog, high level hardware description language like Handle-C or SystemC, MATLAB-Simulink software, an Embedded Development kit, and a System Generator tool. In current surveillance scenario, motion detection is one component of a potentially complex automated video surveillance system, intended to be used as a standalone system. Therefore, in addition to being accurate and robust, a successful motion detection technique must also be economic in the use of computational resources on FPGA development platform. This is because many other complex algorithms of an automated video surveillance system also run on the same FPGA platform. In order to address this problem of reducing the computational complexity, Chutani and Chaudhury [53] proposed a block-based clustering scheme with a very low complexity for motion detection. On one hand, this scheme is robust enough for handling pseudo-stationary nature of background, and on the other it significantly lowers the computational complexity and is well suited for designing standalone systems for real-time applications. For this reason we have selected the clustering based motion detection scheme for designing the real-time standalone motion detection
Electronics 2016, 5, 10
3 of 18
system which features real-time processing and is capable of discarding irrelevant motion using adaptive background Electronics 2016, 5, 10 model. The implemented motion detection system can be used as a standalone 3 of 18 system for automated video surveillance applications. background model. The implemented motion detection system can be used as a standalone system for automated video surveillance applications. 2. Motion Detection Algorithm
In this section, theAlgorithm clustering based motion detection scheme is briefly described. For more 2. Motion Detection detailed description refer to [53] and [23]. Clustering based motion detection uses a block-based In this section, the clustering based motion detection scheme is briefly described. For more similarity computation scheme. To start with, each incoming video frame is partitioned into 4 ˆ 4 pixel detailed description refer to [53] and [23]. Clustering based motion detection uses a block-based blocks. Each 4computation ˆ 4 pixel block is modeled by a group of four clusters where each cluster similarity scheme. To start with, each incoming video frame is partitioned intoconsists 4 × 4 of a block centroid (in RGB) and ablock frame numberby which updated cluster most Optionally, pixel blocks. Each 4 × 4 pixel is modeled a group of four the clusters where eachrecently. cluster consists for each blockcentroid there may be aand motion flag field. which The group of the four clusters necessary to correctly of a block (in RGB) a frame number updated cluster mostisrecently. Optionally, for each block there may be a motion flag field. The group of four clusters is necessary to correctly model the pseudo-stationary background, as a single cluster is incapable of modeling multiple modes model pseudo-stationary background, as a single cluster incapable ofis modeling that can be the present in pseudo-stationary backgrounds. Theisgroup size selectedmultiple as fourmodes because it that can be present in pseudo-stationary backgrounds. size per is selected fourabecause it has been reported by Chutani and Chaudhury [53] thatThe fourgroup clusters group as yield good balance has been reported by Chutani and Chaudhury [53] that four clusters per group yield a good balance between accuracy and computational complexity. The basic computational scheme is shown in between accuracy and computational complexity. The basic computational scheme is shown in Figure 1 and the pseudo-code is shown in Figure 2. The sequence of steps for motion detection using a Figure 1 and the pseudo-code is shown in Figure 2. The sequence of steps for motion detection using clustering-based scheme is given below. a clustering-based scheme is given below.
Figure 1. 1.Clustering-Based MotionDetection Detection Scheme. Figure Clustering-Based Motion Scheme.
1.
2.
3.
1. Block Centroid Computation:Each Eachincoming incoming frame into 4 ×44ˆblocks. For each Block Centroid Computation: frameisispartitioned partitioned into 4 blocks. For each block, the block centroid for RGB color space is computed by taking the average color block, the block centroid for RGB color space is computed by taking the averagevalue colorofvalue the 16 pixels of that block. The block centroid is of 24-bits (8-bits for each R, G, and B color of the 16 pixels of that block. The block centroid is of 24-bits (8-bits for each R, G, and B component). color component). 2. Cluster Group Initialization: During the initial four frames, initialization is performed. In the Cluster Initialization: theisinitial fourwith frames, initialization is block performed. first Group frame, the first cluster ofDuring each block initialized its centroid set to the centroidIn the firstof frame, the first cluster each initialized its is centroid to the block centroid corresponding block ofof the firstblock frame is and its frame with number set to 1. set In the second frame, of corresponding block of the firstisframe and its frame number 1. Incentroid the second frame, the second cluster of each block initialized with its centroid setistoset thetoblock of the corresponding of the second and itswith frame is set 2. the In the thirdcentroid frame, theof the the second clusterblock of each block is frame initialized itsnumber centroid settoto block third clusterblock of each is initialized its centroid to to the block of the corresponding of theblock second frame and with its frame numberset is set 2. In the centroid third frame, corresponding block of the third frame and its frame number is set to 3. In the fourth frame, the third cluster of each block is initialized with its centroid set to the block centroid of corresponding last/fourth cluster of each is initialized to theframe, block centroid of block of the third frame and block its frame number with is setitstocentroid 3. In thesetfourth the last/fourth corresponding block of the fourth frame and its frame number is set to 4. In this way, during cluster of each block is initialized with its centroid set to the block centroid of corresponding initialization all the four clusters of the cluster group are initialized. block of the fourth frame and its frame number is set to 4. In this way, during initialization all the 3. Cluster Matching: After initialization, the next step for motion detection in incoming frames is fourtoclusters ofeach the cluster group are initialized. compare of the incoming blocks against the corresponding cluster group. The goal is to Cluster Matching: After initialization, the next For step for motion detection incoming frames find a matching cluster within the cluster group. finding a matching cluster,infor each cluster the cluster group, centroid the incoming current centroid is toincompare each of the difference incomingbetween blocks its against theand corresponding clusterblock group. The goal computed. The cluster with minimum centroidgroup. difference theauser defined cluster, threshold is each is toisfind a matching cluster within the cluster Forbelow finding matching for considered as a matching cluster. In orderbetween to simplify this computation, Manhattancurrent distanceblock cluster in the cluster group, the difference its centroid and the incoming (sum is of absolute differences) is used which avoids the overheads of multiplication in difference centroid computed. The cluster with minimum centroid difference below the user defined
Electronics 2016, 5, 10
4.
5.
6.
4 of 18
threshold is considered as a matching cluster. In order to simplify this computation, Manhattan distance (sum of absolute differences) is used which avoids the overheads of multiplication in difference computation [23]. Eliminating multiplications is very beneficial in terms of reducing computational complexity of the algorithm as multiplications are costly in hardware. Electronics 2016, 5, 10 4 of 18 Cluster Update: If, for a given block, a matching cluster is found within the cluster group, then computation [23]. EliminatingThe multiplications is very beneficial in cluster terms of reducing by the the matching cluster is updated. frame number of the matching is replaced complexity the algorithm as multiplications are costly in hardware. currentcomputational frame number and theofcentroid of the matching cluster is replaced by the average value 4. Cluster Update: If, for a given block, a matching cluster is found within the cluster group, then of the matching cluster centroid and the incoming current block centroid. the matching cluster is updated. The frame number of the matching cluster is replaced by the Cluster Replace: for a and given block, no matching be by found withinvalue the group, current frameIf, number the centroid of the matchingcluster cluster iscould replaced the average then the oldest cluster which has not updated for the longest period of time (cluster with of the matching cluster centroid and been the incoming current block centroid. 5. Cluster Replace: If, forisa given block, cluster could be having found within the group, thencentroid minimum frame number) deleted andnoa matching new cluster is created the current block the oldest cluster which has not been updated for the longest period of time (cluster with as its centroid and the current frame number as its frame number. minimum frame number) is deleted and a new cluster is created having the current block Classification: For a given block, if no matching cluster is found and the oldest cluster is replaced, centroid as its centroid and the current frame number as its frame number. then implies that the current is notcluster matching withand thethe background models and 6. itClassification: For incoming a given block, if noblock matching is found oldest cluster is it is marked asthen motion detected block by setting motion flag field of the thebackground block to “1”. If a replaced, it implies that the incoming current the block is not matching with models and itisis found markedwithin as motion detected by setting the matching motion flag cluster field of the block to matching cluster the clusterblock group and the is updated, then “1”. If a matching cluster is found within the cluster group and the matching cluster is updated, the incoming current block belongs to the background and therefore, the motion flag field of the then the incoming current block belongs to the background and therefore, the motion flag field block is set to “0” (i.e., no motion detected). of the block is set to “0” (i.e., no motion detected).
Figure 2. Pseudo-code for Clustering-based Motion Detection Scheme.
Figure 2. Pseudo-code for Clustering-based Motion Detection Scheme.
Electronics 2016, 5, 10
5 of 18
The above-mentioned clustering-based motion detection scheme has been implemented Electronics 2016, 5, 10 5 of 18by us in C/C++ programming language. For running the code, a Dell Precision T3400 workstation (with ® Core™2 clustering-based motion detection scheme has been implemented by us WindowsThe XP above-mentioned operating system, quad-core Intel Duo Processor with 2.93 GHz Operating in C/C++ programming language. For running the code, a Dell Precision T3400 workstation (withused Frequency, and 4GB RAM) was used. The Open Computer Vision (OpenCV) libraries have been Windows XP operating system, quad-core Intel® Core™2 Duo Processor with 2.93 GHz Operating in the code for reading video streams (either stored or coming from camera) and displaying motion Frequency, and 4GB RAM) was used. The Open Computer Vision (OpenCV) libraries have been used detection results. The frame rate of this software-based implementation for standard PAL (720 ˆ 576) in the code for reading video streams (either stored or coming from camera) and displaying motion resolution wasresults. much The lower than theofreal-time requirements of automated video surveillance systems. detection frame rate this software-based implementation for standard PAL (720 × 576) resolution was much lower than the real-time requirements of automated video surveillance systems.
3. Motion Detection System
3. Motion System performance, as required in an automated video surveillance system, In order toDetection achieve real-time we have proposed a dedicated hardware architecture for clustering-based motion scheme In order to achieve real-time performance, as required in an automated videodetection surveillance system, we have proposed a dedicated hardware architecture clustering-based motion detection and its implementation as a prototype system using the Xilinxfor ML510 (Virtex-5 FX130T) FPGA board scheme and its implementation as a prototype system using the Xilinx ML510 (Virtex-5 FX130T) for real-time motion detection. FPGA board for real-time motion A simplified conceptual block detection. diagram of the proposed and developed FPGA-based motion A simplified conceptual block diagram of the proposed and developed FPGA-based motion detection system is shown in Figure 3 to illustrate the data flow within the system. The main detection system is shown in Figure 3 to illustrate the data flow within the system. The main components of a complete FPGA-based standalone motion detection system are: analog Camera, components of a complete FPGA-based standalone motion detection system are: analog Camera, VDEC1 Video Decoder Board for analog to digital video conversion, custom designed Interface PCB, VDEC1 Video Decoder Board for analog to digital video conversion, custom designed Interface PCB, XilinxXilinx ML510 (Virtex-5 FX130T) FPGA real-time motion detection, a ML510 (Virtex-5 FX130T) FPGAplatform platform for for performing performing real-time motion detection, and and a display device (Display Monitor). display device (Display Monitor).
Figure 3. Dataflow Diagram theProposed Proposed and and Developed Detection System. Figure 3. Dataflow Diagram ofofthe DevelopedMotion Motion Detection System.
Input video, captured by a Sony Analog Camera, is digitized by Digilent VDEC1 Video Decoder
Input captured bysignals a Sonyfrom Analog Camera, is digitized bytransferred Digilent VDEC1 Video Decoder Board.video, The digital output the video decoder board are to FPGA platform Board. Thehigh digital output signals from theavailable video decoder transferred to FPGA FPGABoard platform using speed I/O ports (HSIO PINS) on Xilinxboard ML510are (Virtex-5 FX130T) Interface Theavailable components inside dashed line areFX130T) available FPGA on Xilinx usingusing highcustom speed designed I/O ports (HSIOPCB. PINS) on Xilinx ML510blue (Virtex-5 Board ML510 (Virtex-5 FX130T) FPGA Board. These include FPGA Device (shown by dashed red line), High using custom designed Interface PCB. The components inside dashed blue line are available on Xilinx Speed InputFX130T) Output Interface (HSIO These PINS),include and DVIFPGA Interface. The (shown Camera by Interface module usesHigh ML510 (Virtex-5 FPGA Board. Device dashed red line), the digital video signals available at the FPGA interface and extracts RGB data and generates Video Speed Input Output Interface (HSIO PINS), and DVI Interface. The Camera Interface module uses Timing Signals. The DVI Display Controller is used to display the processed data on Display Monitor. the digital video signals available at the FPGA interface and extracts RGB data and generates Video Data arrives from the Camera Interface module row by row. As the motion detection scheme is Timing Signals. The DVI Display Controller is used to display the processed data on Display Monitor. based on the processing of 4 × 4 image blocks, streaming video processing cannot be used for Data arrives from the Camera module row the by row. As the motion detection scheme clustering based motion detectionInterface scheme. For this reason, four rows of image data are buffered is based on the processing 4 ˆ 4 image streaming video processing cannot be from used for in Input Memory before of processing begins.blocks, The Motion Detection architecture takes its input clustering based motion detection reason, the four rows module of image data are buffered in the Input Memory and processesscheme. the data.For Thethis output of Motion Detection is stored in Output forbefore synchronization sending Detection it for display. This is because InputMemory Memory processingpurpose begins.before The Motion architecture takesthe itsoutput input data from the Motion Detection modulethe is for 4 × 4The pixel block while the DVI Display Controller the in input InputofMemory and processes data. output of Motion Detection module istakes stored Output datafor row by row. The processed image pixel data from Memory withthe video timing Memory synchronization purpose before sending it forOutput display. This isalong because output data of Motion Detection module is for 4 ˆ 4 pixel block while the DVI Display Controller takes the input data
Electronics 2016, 5, 10
6 of 18
row by row. The processed image pixel data from Output Memory along with video timing signals Electronics 2016, 5, 10 6 of 18 is sent for display on Display Monitor through DVI Interface available on FPGA board. The three modules (Input Memory, Motion Detection, andthrough OutputDVI Memory) green dashed line form signals is sent for display on Display Monitor Interfacewithin available on FPGA board. The the clustering based motion detection architecture. three modules (Input Memory, Motion Detection, and Output Memory) within green dashed line form the clustering based motion detection architecture.
4. Proposed Architecture
4. Proposed Architecture
In order to achieve real-time performance, as required in an automated video surveillance system, In order toa achieve real-time performance, as required in an automated videodetection surveillance we have proposed dedicated hardware architecture for clustering-based motion scheme system, we have proposed a dedicated hardware architecture for clustering-based motion detection and its implementation. The detailed VLSI architecture proposed and designed for clustering-based scheme and its implementation. The detailed VLSI architecture proposed and designed for motion detection scheme is shown in Figure 4. The first module is INPUT MEM. This module receives clustering-based motion detection scheme is shown in Figure 4. The first module is INPUT MEM. the incoming pixel data as input and buffers it in the memory. Buffering is required because the This module receives the incoming pixel data as input and buffers it in the memory. Buffering is motion detection algorithm works on 4 ˆ 4 pixel windows. The output from INPUT MEM is four required because the motion detection algorithm works on 4 × 4 pixel windows. The output from pixelINPUT data coming parallel. BLCENT computes the average centroid MEM isout fourinpixel data coming outCOMPUTATION in parallel. BLCENTUNIT COMPUTATION UNIT computes for 4 the ˆ 4average image centroid block byfortaking pixel data from the input memory (four clock cycles are required 4 × 4 image block by taking pixel data from the input memory (four clock for reading 16required pixels from the input memory buffer). This computation done by adding cycles are for reading 16 pixels from the input memory buffer). Thisiscomputation is done16 by pixel adding 16 pixel values current block and then dividing theread sum address, by 16. The read address, address, write values of current block andofthen dividing the sum by 16. The write and write address, signals input memory generated byINPUT-MEM the corresponding INPUTenable signalsand forwrite inputenable memory are for generated by the are corresponding RADD-WRADD MEM RADD-WRADD WEN module. WEN module.
Figure 4. Proposed & Implemented VLSI Architecture for Clustering-based Motion Detection Scheme. Figure 4. Proposed & Implemented VLSI Architecture for Clustering-based Motion Detection Scheme.
The motion computation enable (MCEN) signal is generated by MOTION COMPUTATION
The motion computation generated by MOTION COMPUTATION ENABLE UNIT. This signal isenable used as(MCEN) the enablesignal signal is in different modules of the designed motion ENABLE UNIT. This signal is used as the enable signal in different modules of the designed motion detection architecture. detectionAs architecture. mentioned in the algorithm section, the clustering-based scheme stores centroid and frame number information foralgorithm each 4 × 4section, image block. It, therefore, requires the assignment of a unique As mentioned in the the clustering-based scheme stores centroid and frame identity (or address) each4 block. This isblock. done by row and column generated number information fortoeach ˆ 4 image It, using therefore, requires thecounters assignment of a by unique COUNTERS & CONTROLLER MODULE. This unit takes row the video timing signals fromgenerated camera by identity (or address) to each block. This is done by using and column counters COUNTERS & CONTROLLER MODULE. This unit takes the video timing signals from camera
Electronics 2016, 5, 10
7 of 18
interface module and generates the different counter values (row counter, column counter, frame number counter) and control signals required for the proper functioning of complete system. For storing background related information in the clustering-based algorithm, a parameter memory is used. It has two components viz. Centroid Memory (CENT MEM) and Frame Number Memory (FRNM MEM). Each Centroid Memory location contains four Centroid values (corresponding to four clusters) which contain the background color and intensity related information. Each Frame Number Memory location stores four Frame Number values (corresponding to four clusters) which are used to keep the record of Centroid value updating or replacement history i.e., for the particular 4 ˆ 4 pixel block when (at what time or for what Frame Number) the cluster Centroid value is updated or replaced. The read address and write address signals for CENT-MEM and FRNM MEM are generated by the corresponding units CENT-MEM RADD-WRADD UNIT and FRNM-MEM RADD-WRADD UNIT. During initial four frames, control signals are generated in such a way that the four clusters are initialized (one in each frame). For all subsequent frames, the generated control signals enable the motion detection process. After initialization, the matching cluster is searched within the cluster group of four clusters. For this purpose, first difference between cluster Centroid value (CENT DATA) and incoming current block Centroid value (BLCENT) is computed for all four clusters by reading the cluster Centroid values (4 CENT DATA) from CENT-MEM corresponding to current 4ˆ4 image block and taking absolute sum of differences with current block Centroid value (BLCENT). From the four difference values, minimum Centroid difference value is selected. This complete task is carried out by the MINIMUM CENT DIFF COMPUTATION UNIT. It outputs MCD (minimum centroid difference value) and CINDX (Centroid Index). CINDX gives the cluster number corresponding to MCD (minimum centroid difference value). In parallel with this, MINIMUM FRAME NUM COMPUTATION UNIT finds the frame index (FINDX). FINDX gives the cluster number having minimum frame number value corresponding to current 4 ˆ 4 block in the frame number memory (FRNM MEM). The MCD and BLCENT values are used by CENT-MEM WDATA COMPUTATION UNIT to compute the write data for Centroid Memory (CENT MEM). MCD is compared with a user defined threshold. For MCD less than or equal to the threshold (i.e., matching cluster is found), the write data for CENT MEM is the average value of current block Centroid value (BLCENT) and matching cluster Centroid value (matching cluster number is given be CINDX). For MCD greater than threshold (i.e., no matching cluster is found), the write data for CENT MEM is current block Centroid value (BLCENT) and, in this case, the cluster number for which value is replaced is determined by frame number index value (FINDX) which corresponds to the oldest cluster in the cluster group. The write data for frame number memory (FRNM-MEM) is generated by FRNM-MEM WDATA COMPUTATION UNIT and it is the present or the current frame number (PFRNM) value. The CINDX and FINDX values are used by CENT-MEM FRNM-MEM WEN UNIT for generating the write enable signals for CENT MEM and FRNM MEM. The write enable signals help for selecting the cluster for which the centroid value and the frame number value is to be updated or replaced. The MD FLAG takes MCD as input and compares it with a user defined threshold. A 1-bit Flag signal is generated which is low if difference is less than the threshold (i.e., current block matches with background model and therefore, no motion is detected) and high if the difference is greater than the threshold (i.e., current block is motion detected block). This motion information data of 4x4 pixel block is written to the output memory (OUTPUT MEM) and corresponding addresses for this memory are generated by OUTPUT-MEM RADD-WRADD WEN module. Finally, the motion detected data is read from this output buffer memory and sent to the display controller for display on the screen. The details of individual blocks of the proposed and implemented motion detection architecture are presented in the following sub-sections.
Electronics 2016, 5, 10
8 of 18
4.1. Input Buffer Memory Electronics 2016, 5, 10
8 of 18
The input buffer memory module (Figure 5) is used to buffer the four rows of input image data. buffer (MEM1 memoryand module (Figure 5) is used If tothe buffer theisfour rowsin offirst inputmemory image data. There are The twoinput memories MEM2) in parallel. data written then the There are two memories (MEM1 and MEM2) in parallel. If the data is written in first memory then reading is performed from the second memory and vice-versa. Each of the memory is constructed by the reading is performed from the second memory and vice-versa. Each of the memory is constructed four block RAMs (BRAMs) in FPGA. These four block RAMs are connected in parallel. Each block by four block RAMs (BRAMs) in FPGA. These four block RAMs are connected in parallel. Each block RAMRAM is used to buffer oneone row ofof image Thewidth widthofofeach each block RAM is 8-bit. The 8-bit is used to buffer row imagepixel pixeldata. data. The block RAM is 8-bit. The 8-bit inputinput data data is applied to the of all RAMs of MEM1 and MEM2. The is applied to inputs the inputs ofblock all block RAMs of MEM1 and MEM2. Theread readaddress addressand andwrite address foraddress the block of MEM1 MEM2and are generated RDADDbyWRADD write forRAMs the block RAMsand of MEM1 MEM2 are by generated RDADDGENERATOR. WRADD GENERATOR. The write enable signals are generated based on the(RW4C) row counter (RW4C)value. The write enable signals are generated separately basedseparately on the row counter and switch and switch value. The selection of four output pixels (read from MEM1 or MEM2) is done by usingbased The selection of four output pixels (read from MEM1 or MEM2) is done by using multiplexers multiplexers based on switch value. on switch value.
Figure 5. Proposed VLSI architecture for Input Buffer Memory. Figure 5. Proposed VLSI architecture for Input Buffer Memory.
4.2. Minimum Centroid Difference Computation
4.2. Minimum Centroid Difference Computation
The objective of this block (Figure 6) is to find the minimum centroid difference and
The objective of this value. block (Figure 6) is block to findcentroid the minimum difference and corresponding corresponding index The current value is centroid subtracted from centroid values of indexfour value. The current block centroid value subtracted centroid values of four clusters (read from CENT MEM). The isfour centroid from values from CENT MEM are clusters CMRD1,(read CMRD3, CMRD4. The absolute of current block centroid BLCENT is and from CMRD2, CENT MEM). Theand four centroid values fromdifference CENT MEM are CMRD1, CMRD2, CMRD3, computed in parallel with these four values. The first two differences are compared and the minimum CMRD4. The absolute difference of current block centroid BLCENT is computed in parallel with these four values. The first two differences are compared and the minimum difference among two (MDIFF1)
Electronics 2016, 5, 10
Electronics 2016, 5, 10
9 of 18
9 of 18
difference among two (MDIFF1) is computed using comparator and multiplexer. The corresponding
Electronics 2016, 5, 10comparator and multiplexer. The corresponding index value is also computed 9 of 18 is computed using index value is also computed (INDX1). Similarly, the last two differences are compared and the (INDX1). Similarly, the last two differences are compared and the minimum difference among the minimum among difference the is last two differences (MDIFF2) and index (INDX2) is difference twoamong (MDIFF1) computed using comparator and corresponding multiplexer. The corresponding last two differences (MDIFF2) and corresponding index (INDX2) the is computed. Finally,difference using a set of computed. a set of comparator and two minimum centroid index valueFinally, is also using computed (INDX1). Similarly, themultiplexers last two differences are compared and the comparator and two multiplexers the minimum centroid difference (MCDIFF) and the corresponding (MCDIFF) difference and the corresponding index (CINDX) is computed using MDIFF1 & MDIFF2 minimum among the lastcentroid two differences (MDIFF2) and corresponding index (INDX2) is centroid (CINDX) isrespectively. computed using MDIFF1 MDIFF2 and & INDX2, respectively. and index INDX1Finally, & INDX2, computed. using a set of comparator and two&multiplexers the INDX1 minimum centroid difference
(MCDIFF) and the corresponding centroid index (CINDX) is computed using MDIFF1 & MDIFF2 and INDX1 & INDX2, respectively.
Figure 6. Proposed VLSI architecture of Minimum Centroid Difference Computation Module.
Figure 6. Proposed VLSI architecture of Minimum Centroid Difference Computation Module.
4.3. Minimum Number Computation Figure Frame 6. Proposed VLSI architecture of Minimum Centroid Difference Computation Module.
4.3. Minimum Frame Number Computation
The objective of minimum frame number computation block (Figure 7) is to find the minimum 4.3. Minimum Number Computation The objective of minimum frame number computation block (Figure 7) isfrom to find the MEM minimum frame numberFrame and the corresponding index value. The four frame number values FRNM FMRD1, FMRD2, FMRD3, and FMRD4. Thevalue. first twoThe frame number values and FMRD2) framearenumber and the corresponding index four frame number values from FRNM The objective of minimum frame number computation block (Figure 7) is(FMRD1 to find the minimum are and thecorresponding minimum among is computed using comparator and and number and the index value.two The four number values from FRNM MEM MEMframe arecompared FMRD1, FMRD2, FMRD3,value and FMRD4. The(MFN1) firstframe two frame number values (FMRD1 multiplexer. The corresponding index value is also computed (INDX1). Similarly, the last frame are FMRD1, FMRD2, and FMRD3, and FMRD4.value The first two frame number values (FMRD1using and FMRD2) FMRD2) are compared the minimum among two (MFN1) is computed comparator number valuesand (FMRD3 and FMRD4) are compared and the minimum value among last two (MFN2) are compared the minimum value two (MFN1) is computed using comparator andframe and multiplexer. The corresponding indexamong value is also computed (INDX1). Similarly, the last and corresponding index (INDX2) computed. using a set of comparator andframe two multiplexer. The corresponding indexis value is alsoFinally, computed (INDX1). Similarly, the last number values (FMRD3 and FMRD4) are compared and the minimum value among last two (MFN2) multiplexers the minimum frame number (MFRNM) and corresponding frame number index number values (FMRD3 and FMRD4) are compared and the minimum value among last two (MFN2) and corresponding index using (INDX2) is computed. Finally, using a set of comparator and two multiplexers (FINDX) is computed MFN1 & is MFN2 and INDX1 & INDX2, and corresponding index (INDX2) computed. Finally, using respectively. a set of comparator and two
the minimum frame (MFRNM) and corresponding frame number index (FINDX) computed multiplexers the number minimum frame number (MFRNM) and corresponding frame numberis index using(FINDX) MFN1 is & computed MFN2 and INDX1 & INDX2, respectively. using MFN1 & MFN2 and INDX1 & INDX2, respectively.
Figure 7. Proposed VLSI architecture of Minimum Frame Number Computation Module. Figure 7. Proposed VLSI architecture of Minimum Frame Number Computation Module.
Figure 7. Proposed VLSI architecture of Minimum Frame Number Computation Module.
Electronics 2016, 5, 10
4.4. Parameter Memory Electronics 2016, 5, 10
10 of 18
10 of 18
There are two parameter memories as shown in Figure 8. The first one is CENT MEM which 4.4. Parameter Memory stores centroid value of each block for four clusters. This memory contains four memory modules There two parameterto memories as shown Figure 8. memory The first one is CENT MEMwide which in parallel eachare corresponding one cluster. Eachincentroid module is 8-bit and is stores centroid value of each block for four clusters. This memory contains four memory modules in 25,920 locations in size. The four centroid parameter memory modules are CM1, CM2, CM3, and CM4. parallel each corresponding to one cluster. Each centroid memory module is 8-bit wide and is 25,920 The data is written to and read from all the four memories from same locations at a time. Therefore, locations in size. The four centroid parameter memory modules are CM1, CM2, CM3, and CM4. The the read writes is common to allfrom centroid The write data address is writtenand to and readaddress from all the four memories samememories locations atmodules. a time. Therefore, the data (WRDATACM) and write enable signals (WRENCM) for these four memories are different. The second read address and writes address is common to all centroid memories modules. The write data parameter memory is FRNM MEM which stores frame number value of each block for four clusters. (WRDATACM) and write enable signals (WRENCM) for these four memories are different. The This memory containsmemory four memory modules in parallel corresponding Each second parameter is FRNM MEM which storeseach frame number value to of one eachcluster. block for fourframe clusters. This module memory is contains four memory modules in parallel each corresponding to one cluster. number memory 16-bit wide and 25,920 locations in size. The four frame number parameter Each modules frame number memory is 16-bit wide 25,920 locations size. Thefrom fourall frame memory are FM1, FM2,module FM3, and FM4. Theand data is written to in and read the four number parameter memory modules are FM1, FM2, FM3, and FM4. The data is written to and memories from same locations at a time. Therefore, the read address and writes address is read common from all the four memories from same locations at a time. Therefore, the read address and writes to all frame number memories modules. The write data (WRDATAFM) and write enable signals address is common to all frame number memories modules. The write data (WRDATAFM) and write (WRENFM) for these four memories are different. The four data can be written to and read from the enable signals (WRENFM) for these four memories are different. The four data can be written to and parameter memories at anymemories time in single clock The data bedata written to CENT MEM and read from the parameter at any time in cycle. single clock cycle. to The to be written to CENT FRNM MEM is computed by corresponding write data generation modules. MEM and FRNM MEM is computed by corresponding write data generation modules.
Figure 8. Parameter Memory(Centroid (Centroidand and Frame Frame Number) Motion Detection Architecture. Figure 8. Parameter Memory Number)ofof Motion Detection Architecture.
Electronics 2016, 5, 10 Electronics 2016, 5, 10
11 of 18 11 of 18
The write enable signals for CENT MEM and FRNM MEM are same. Therefore, a common 4-bit write enable is generated forCENT both memories. Each bit of the signal is corresponding Thesignals write enable signals for MEM and FRNM MEM are write same.enable Therefore, a common 4-bit write enablemodule signals of is the generated for both memories. memories. Each bit of the write enable signal is to each memory two parameter corresponding to enable each memory of9) theistwo parameter memories. The 4-bit write signalmodule (Figure generated based on Frame Number value, centroid The(CINDX), 4-bit writeframe enablenumber signal (Figure is generated Frame Number value, centroid index value index 9) (FINDX) value,based and on Minimum Difference Value (MDIFF). index value (CINDX), frame number index (FINDX) value, and Minimum Difference Value (MDIFF). If frame number is less than 5 then write enable signal is generated in such a way that it initializes If frame number is less than 5 then write enable signal is generated in such a way that it initializes the four memories. If MDIFF is less than that of user defined threshold then, the write enable signal the four memories. If MDIFF is less than that of user defined threshold then, the write enable signal is generated for updating the existing centroid values and frame number values. If the MDIFF is is generated for updating the existing centroid values and frame number values. If the MDIFF is greater thanthan thatthat of user defined threshold writesignal signalis is generated by replacing the existing greater of user defined thresholdthen, then, the the write generated by replacing the existing centroid and fame number values in parameter memories. This is done with the help of multiplexers centroid and fame number values in parameter memories. This is done with the help of multiplexers and comparators. and comparators.
Figure 9. Parameter Memory(Centroid (Centroid and and Frame Write Enable Signal Generation. Figure 9. Parameter Memory FrameNumber) Number) Write Enable Signal Generation.
4.5. Initialize-Update-Replace Module
4.5. Initialize-Update-Replace Module
The objective of this block (Figure 10) is to find the minimum centroid difference and The objective of this value. block (Figure 10) isblock to find the minimum centroid difference and corresponding corresponding index The current centroid value is subtracted from centroid values of indexfour value. The(read current block centroid value is subtracted from centroid clusters clusters from CENT MEM). Initialization-update-replace module values (Figure of 10)four computes the (read to be writtenInitialization-update-replace in parameter memories (CENTmodule MEM and FRNM10) MEM). The write fromdata CENT MEM). (Figure computes the data datafor to frame be written number parameter is always frame number (PFRNUM). It parameter is 16in parameter memoriesmemory (CENT(FMWD) MEM and FRNMcurrent/present MEM). The write data for frame number bit data and is written only when MDEN is high, will be zero. It For memory (FMWD) is always current/present frameotherwise numberit(PFRNUM). isinitial 16-bitfour dataframes, and isthe written initialization process take place, therefore the data written to centroid memory (CMWD) will be only when MDEN is high, otherwise it will be zero. For initial four frames, the initialization process take current block average centroid value (BLCENT) for initial four frames. For all subsequent frames, place, therefore the data written to centroid memory (CMWD) will be current block average centroid there are two possibilities. Either existing data will be updated or it will be replaced depending upon value (BLCENT) for initial four frames. For all subsequent frames, there are two possibilities. Either the value of MDIFF. If MDIFF value is greater than the user defined threshold, then replacing will existing be updated or itbewill be replaced depending upon the value ofIfMDIFF. If MDIFF takedata placewill and the old value will replaced by current block average centroid value. MDIFF value valueisisless greater than the user defined threshold, then replacing will take place and the old value will be than the user defined threshold then, updating process will take place. The existing centroid replaced by current block average centroid value. If MDIFF value is less than the user defined threshold value is updated with the average sum of old value of centroid and current block average centroid Which memory centroid is updated the CINDX value. Based onwith CINDX, the 8-bit sum then,value. updating process will take place. The depends existing on centroid value is updated the average
of old value of centroid and current block average centroid value. Which memory centroid is updated depends on the CINDX value. Based on CINDX, the 8-bit centroid value is selected and added with
Electronics 2016, 5, 10
12 of 18
Electronics 2016, 5, 10
12 of 18
centroid value is5,selected and added with BLCENT and the result is then divided by 2 using Electronics 2016, 10 12 of 18 right shift. This average value will be written to centroid parameter memory. BLCENT andvalue the result is then by 2BLCENT using right shift. This average value be written to centroid is selected anddivided added with and the result is then divided by will 2 using right shift.parameter This average value will be written to centroid parameter memory. centroid memory.
Figure 10. Proposed VLSI architecture of Initialization, Update, and Replace Module. Figure 10. Proposed Proposed VLSI VLSI architecture architecture of of Initialization, Initialization, Update, Update, and and Replace Figure 10. Replace Module. Module.
5. Synthesis Results
5. Synthesis SynthesisResults Results 5.
All design modules of proposed architecture for clustering based motion detection scheme have
All design design modules of proposed proposed architecture for clustering clustering based motion detection scheme have been coded in VHDL, simulated using ModelSim, and synthesizedbased using motion the Xilinx ISE (Version 12.1) have All modules of architecture for detection scheme been coded in VHDL, simulated using ModelSim, and synthesized using the Xilinx ISE (Version 12.1) tool chain. The resulting configuration (bit) file was stored in the Flash Memory to enable automatic been coded in VHDL, simulated using ModelSim, and synthesized using the Xilinx ISE (Version 12.1) configuration of the FPGA at power-on. Thus a complete standalone prototype system for real-time tool chain. The resulting configuration (bit) file was stored in the Flash Memory to enable automatic tool chain. The resulting configuration (bit) file was stored in the Flash Memory to enable automatic motion detection has been and is shown in Figure 11. The components ofsystem the system a configuration of the the FPGA FPGA atdeveloped power-on. Thus complete standalone prototype forare real-time configuration of at power-on. Thus aa complete standalone prototype system for real-time Xilinx ML510 (Virtex-5 FX130T) FPGA platform, a Sony EVI D-70P Camera, and a display monitor. motion detection detection has has been been developed developed and is shown shown in in Figure Figure 11. 11. The The components components of of the the system system are are aa motion and is Xilinx ML510 (Virtex-5 FX130T) FPGA platform, a Sony EVI D-70P Camera, and a display monitor. Xilinx ML510 (Virtex-5 FX130T) FPGA platform, a Sony EVI D-70P Camera, and a display monitor.
Figure 11. Complete Motion Detection System Setup.
Figure Figure 11. 11. Complete Complete Motion Motion Detection Detection System System Setup. Setup.
Electronics 2016, 5, 10
13 of 18
The FPGA resources utilized (post-place & route results) by camera interface (CI), display interface (DVI), proposed architecture (PA), and complete implemented motion detection system (camera interface, proposed architecture for motion detection, and display interface) are given in Table 1. For PAL (720 ˆ 576) resolution color video processing, implementation of the complete clustering-based motion detection system utilizes approximately 57% of the available Block RAMs on Xilinx ML510 (Virtex-5 FX130T) FPGA platform. The total power consumption (analyzed using Xilinx XPower Analyzer tool) is 1.97 W (Quiescent Power 1.82 W + Dynamic Power 0.15 W). The maximum operating frequency is 73.88 MHz and maximum possible frame rate for PAL (720 ˆ 576) size color video is 178 frames per second (fps). Frame rate achieved through the hardware implementation is far higher than that required for real-time processing and is far higher than that obtained for software implementation on a workstation. Table 1. FPGA Resource Utilization by Clustering based Motion Detection System.
Resources
Camera Interface (CI)
Display (DVI)
Proposed Architecture (PA)
Complete System (CI + PA +DVI)
Total Available Resources
Percentage of Utilization
Slice Registers Slice LUTs Route-thrus Occupied Slices BRAMs 36K Memory (Kb) DSP Slices IOs
391 434 42 199 3 108 3 16
79 101 39 33 0 0 0 22
239 1379 62 467 168 6048 0 36
701 1884 122 697 171 6156 3 36
81920 81920 163840 20840 298 10728 320 840
0.86% 2.30% 0.07% 3.34% 57.38% 57.38% 0.94% 4.28%
Performance of the proposed, designed, and implemented motion detection architecture is compared with some recently published motion detection implementations using different algorithms. The performance comparison is shown in Table 2. The motion detection/segmentation architectures presented by Genovese et al. [32] and Genovese and Napoli [33] were designed for OpenCV GMM algorithm and were implemented on Virtex5 (xc5vlx50-2ff1153) FPGA. For accurate performance comparison with their work, the proposed motion detection architecture has also been synthesized (including place & route) for Virtex5 (xc5vls50) FPGA devices using Xilinx ISE tool chain. Table 2. Performance Comparison with Existing Motion Detection Implementations. Target FPGA Device
Implementation
Maximum Clock Frequency (MHz
Frame Rate for PAL Resolution Video
Virtex5 (xc5fx130t-2ff1738)
Our Implementation
73.88 MHz
178
Virtex5 (xc5vlx50-2ff1153)
Our Implementation [33] [32]
73.88 MHz 50.50 MHz 47.00 MHz
178 121 113
The architecture proposed in this paper for motion detection outperforms existing implementations in terms of processing speed. An apple to apple comparison of FPGA resource utilization does not make proper sense as the motion detection algorithms used, number of background models considered, and the video formats and sizes considered in these implementations are different than in our implementation. For this reason, one to one FPGA resource utilization comparisons are not tabulated here. The system architecture for motion detection, proposed, designed, and implemented by us is adaptable and scalable for different video sizes. The proposed architecture is capable of processing HD (1920 ˆ 1080) resolution videos in real-time at a frame rate of 35 fps.
Electronics 2016, 5, 10
14 of 18
Electronics 2016, 5, 10
14 of 18
6. Motion Detection Results
6.The Motion Detection system Results was tested for different real-world scenarios (both indoor and outdoor), implemented
which are broadly classified into two static background situations(both and pseudo-stationary The implemented system wascategories tested fori.e., different real-world scenarios indoor and background Figureclassified 12 shows examples of real-world ofsituations static background outdoor), situations. which are broadly into two categories i.e., staticsituations background and pseudo-stationary background situations. Figure showsof examples real-world situations of scenarios captured by the camera. In both the12cases Figureof12, the background is static static and scenarios by the both thedetected cases of Figure 12,implementation the background isin static the background moving objects are captured present in the camera. scene. In Motion by our different and the moving objects are present in the scene. Motion detected by our implementation in different frames is shown just below the respective images. It can be clearly seen that only moving objects shown just below the respective images. It can be clearly seen that only moving objects have haveframes beenisdetected by the implemented motion detection system. Figure 13 shows the scenarios been detected by the implemented motion detection system. Figure 13 shows the scenarios of pseudoof pseudo-stationary background with moving foreground objects. In this case, there are moving stationary background with moving foreground objects. In this case, there are moving leaves of the leaves of the trees in the background. Despite these pseudo-stationary movements in background, trees in the background. Despite these pseudo-stationary movements in background, only moving onlyobjects moving objects in the have foreground detected and the movements of trees in the in the foreground detectedhave and the movements of leaves of trees of in leaves the background background motion) have been eliminated. Results show that the system is (irrelevant(irrelevant motion) have been eliminated. Results of the tests show of thatthe thetests system is robust enough robust enough to the detect only motion the relevant motion a live video scene and eliminatesunwanted the continuous to detect only relevant in a live videoinscene and eliminates the continuous movements in the background itself. All the above framescolor are offrames PAL (720 size and unwanted movements in the background itself. All color the above are ×of576) PAL (720 ˆare 576) size extracted from live video streams produced by the implemented system. and are extracted from live video streams produced by the implemented system.
(a)
(b) Figure 12. Moving Objects in Video Scene and Corresponding Motion Detected Outputs for Static Figure 12. Moving Objects in Video Scene and Corresponding Motion Detected Outputs for Static Background Scenarios. Background Scenarios.
The quantitative analysis of the motion detection quality of proposed hardware implementation quantitative analysis implementation of the motion detection quality of proposed hardware implementation isThe done against the software of the clustering-based motion detection algorithm by usingagainst video streams of different real-world scenarios. For this purpose, for every frame of each video is done the software implementation of the clustering-based motion detection algorithm by stream, mean square error (MSE) is calculated. MSE For is a common measure quality of video and video using videothe streams of different real-world scenarios. this purpose, forofevery frame of each is equivalent to square other commonly usedismeasures of quality. example, the peak signal-to-noise stream, the mean error (MSE) calculated. MSE isFor a common measure of quality of ratio video and (PSNR) is equivalent to MSE [34]. Some researchers measure the number of false positives (FP) and is equivalent to other commonly used measures of quality. For example, the peak signal-to-noise ratio false negatives (FN) whose sum is equivalent to the MSE [54]. MSE is defined as
Electronics 2016, 5, 10
15 of 18
(PSNR) is equivalent to MSE [34]. Some researchers measure the number of false positives (FP) and false negatives (FN) whose sum is equivalent to the MSE [54]. MSE is defined as Electronics 2016, 5, 10
MSE “
15 of 18
´1 M ´1 Nÿ ÿ
1 M ˚ N1 =
∗
m “0 n “0
pISOFTWARE pm, nq ´ IH ARDWARE pm, nqq2 ( , )−
(
( , ))
In the above equation, ISOFTWARE (m, n) is the motion detected binary output image produced In the equation, ISOFTWAREimplementation (m, n) is the motion detected binary output image produced by by running of above the software (C/C++) of clustering based algorithm, while IHARDWARE running of the software (C/C++) implementation of clustering based algorithm, while IHARDWARE (m, (m, n) is the motion detected binary output image produced by running of the proposed hardware n) is the motion detected binary output image produced by running of the proposed hardware (VLSI) (VLSI) implementation of a clustering based algorithm. M is the number of rows in a video frame and implementation of a clustering based algorithm. M is the number of rows in a video frame and N is N isthe thenumber numberofofcolumns columns in a video frame. The computed MSE for every frame of all the test videos in a video frame. The computed MSE for every frame of all the test videos is is zero the proposed proposedhardware hardware (VLSI) implementation produces the same zeroand andititconfirms confirms that that the (VLSI) implementation produces the same motionmotion detection results as the software implementation of the clustering based motion detection scheme detection results as the software implementation of the clustering based motion detection scheme but but at much higher frame rates. at much higher frame rates.
(a)
(b) Figure 13. Moving Objects in Video Scene and Corresponding Motion Detected Outputs for Pseudo-
Figure 13. Moving Objects in Video Scene and Corresponding Motion Detected Outputs for stationary Background Scenarios. Pseudo-stationary Background Scenarios.
7. Conclusions
7. Conclusions
In this research article, the hardware architecture for a clustering based motion detection In this realized researchusing article, the and hardware architecture for aML510 clustering motion detection scheme, VHDL implemented on a Xilinx FPGA based platform, has been presented. The complete working system, on including camera interface, detection scheme, realized using VHDL andprototype implemented a Xilinx ML510 FPGAmotion platform, has been VLSI architecture, and display interface has been implemented on camera Xilinx ML510 (Virtex-5 FX130T) presented. The complete working prototype system, including interface, motion detection Board. Theand implemented system can robustly and automatically in realVLSIFPGA architecture, display interface has been implemented ondetect Xilinxrelevant ML510motion (Virtex-5 FX130T) world scenarios (both for the static backgrounds and the pseudo-stationary backgrounds) for FPGA Board. The implemented system can robustly and automatically detect relevant motion in standard PAL (720x576) resolution live incoming video streams in real-time. It can be effectively used real-world scenarios (both for the static backgrounds and the pseudo-stationary backgrounds) for as a standalone component for motion detection in video surveillance systems.
standard PAL (720 ˆ 576) resolution live incoming video streams in real-time. It can be effectively usedAcknowledgments: as a standalone component motionofdetection in video surveillance systems. Technology The financialfor support Department of Electronics & Information (DeitY)/Ministry of Communications and Information Technology (MCIT), the Govt. of India is gratefully acknowledged.
Electronics 2016, 5, 10
16 of 18
Acknowledgments: The financial support of Department of Electronics & Information Technology (DeitY)/Ministry of Communications and Information Technology (MCIT), the Govt. of India is gratefully acknowledged. Author Contributions: All authors have contributed for this manuscript. Conflicts of Interest: The authors declare no conflict of interest. The founding sponsors had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, and in the decision to publish the results.
References 1. 2. 3. 4. 5. 6. 7.
8. 9.
10. 11. 12. 13. 14.
15. 16. 17. 18. 19.
20.
Radke, R.J.; Andra, S.; Kofahi, O.A.; Roysam, B. Image Change Detection Algorithms: A Systematic Survey. IEEE Trans. Image Process. 2005, 14, 294–307. [CrossRef] [PubMed] Rosin, P.L. Thresholding for Change Detection. In Proceedings of the Sixth International Conference on Computer Vision, Bombay, India, 4–7 January 1998; pp. 274–279. Rosin, P.L.; Ioannidis, E. Evaluation of Global Image Thresholding for Change Detection. Pattern Recognit. Lett. 2001, 24, 2345–2356. [CrossRef] Smits, P.C.; Annoni, A. Toward Specification-Driven Change Detection. IEEE Trans. Geosci. Remote Sens. 2000, 38, 1484–1488. [CrossRef] Bruzzone, L.; Prieto, D.F. Automatic Analysis of the Difference Image for Unsupervised Change Detection. IEEE Trans. Geosci. Remote Sens. 2000, 38, 1171–1182. [CrossRef] Colwell, J.E.; Weber, F.P. Forest Change Detection. In Proceedings of the 15th International Symposium on Remote Sensing of the Environment, Ann Arbor, MI, USA, 11–15 May 1981; pp. 839–852. Malila, W.A. Change Vector Analysis: An Approach for Detecting Forest Changes with Landsat. In Proceedings of the Symposium on Machine Processing of Remotely Sensed Data, West Lafayette, IN, USA, 3–6 June 1980; pp. 326–336. Singh, A. Review Article: Digital Change Detection Techniques using Remotely-sensed Data. Int. J. Remote Sens. 1989, 10, 989–1003. [CrossRef] Stefano, L.D.; Mattoccia, S.; Mola, M. A Change-detection Algorithm based on Structure and Color. In Proceedings of the IEEE Conference on Advanced Video and Signal-Based Surveillance, 21–22 July 2003; pp. 252–259. Hsu, Y.Z.; Nagel, H.H.; Rekers, G. New Likelihood Test Methods for Change Detection in Image Sequences. Comput. Vision Graph. Image Process. 1984, 26, 73–106. [CrossRef] Skifstad, K.; Jain, R. Illumination Independent Change Detection for Real World Image Sequences. Comput. Vision Graph. Image Process. 1989, 46, 387–399. [CrossRef] Elfishawy, A.S.; Kesler, S.B.; Abutaleb, A.S. Adaptive Algorithms for Change Detection in Image Sequence. Signal Process. 1991, 23, 179–191. [CrossRef] Jain, Z.S.; Chau, Y.A. Optimum Multisensor Data Fusion for Image Change Detection. IEEE Trans. Syst. Man Cybern. 1995, 25, 1340–1347. [CrossRef] Toyama, K.; Krumm, J.; Brumitt, B.; Meyers, B. Wallflower: Principles and Practice of Background Maintenance. In Proceedings of the Seventh International Conference on Computer Vision, Kerkyra, Greece, 20–27 September 1999; pp. 255–261. Clifton, C. Change Detection in Overhead Imagery using Neural Networks. Appl. Intell. 2003, 18, 215–234. [CrossRef] Durucan, E.; Ebrahimi, T. Change Detection and Background Extraction by Linear Algebra. In Proceedings of the IEEE, October 2001; Volume 89, pp. 1368–1381. [CrossRef] Li, L.; Leung, M.K.H. Integrating Intensity and Texture Differences for Robust Change Detection. IEEE Trans. Image Process. 2002, 11, 105–112. [PubMed] Liu, S.C.; Fu, C.W.; Chang, S. Statistical Change Detection with Moments under Time-Varying Illumination. IEEE Trans. Image Process. 1998, 7, 1258–1268. [PubMed] Cavallaro, A.; Ebrahimi, T. Video Object Extraction based on Adaptive Background and Statistical Change Detection. In Proceedings of the SPIE Visual Communications and Image Processing, San Jose, CA, USA, 20 January 2001; pp. 465–475. Huwer, S.; Niemann, H. Adaptive Change Detection for Real-Time Surveillance Applications. In Proceedings of the Third IEEE International Workshop on Visual Surveillance, Dublin, Ireland, 1 July 2000; pp. 37–46.
Electronics 2016, 5, 10
21.
22. 23. 24.
25. 26.
27.
28.
29. 30. 31. 32.
33. 34.
35.
36.
37.
38. 39.
40.
17 of 18
Kanade, T.; Collins, R.T.; Lipton, A.J.; Burt, P.; Wixson, L. Advances in Cooperative Multi-Sensor Video Surveillance. In Proceedings of the DARPA Image Understanding Workshop, San Francisco, USA, 20 November 1998; pp. 3–24. Stauffer, C.; Grimson, W.E.L. Learning Patterns of Activity using Real-Time Tracking. IEEE Trans. Pattern Anal. Mach. Intell. 2000, 22, 747–757. [CrossRef] Butler, D.E.; Bove, V.M.; Sridharan, S. Real-Time Adaptive Foreground/Background Segmentation. EURASIP J. Appl. Signal Process. 2005, 2005, 2292–2304. [CrossRef] Dumontier, C.; Luthon, F.; Charras, J.P. Real-Time Implementation of An MRF-based Motion Detection Algorithm on a DSP Board. In Proceedings of the IEEE Workshop on Digital Signal Processing, Leon, 1–4 September 1996; pp. 183–186. Dumontier, C.; Luthon, F.; Charras, J.P. Real-Time DSP Implementation for MRF-based Video Motion Detection. IEEE Trans. Image Process. 1999, 8, 1341–1347. [CrossRef] [PubMed] Bassignana, P.; Martina, M.; Masera, G.; Molino, A.; Vacca, F. DSP Implementation of a Low Complexity Motion Detection Algorithm. In Proceedings of the 39th Asilomar Conference on Signals, Systems and Computers, Pacific Groove, USA, 28 October–1 November 2005; pp. 1352–1355. Benkhalil, A.K.; Lpson, S.S.; Booth, W. A Novel CPLD based Implementation of a Motion Detection Algorithm for Surveillance Applications. In Proceedings of the International Conference on Custom Integrated Circuits, Santa Clara, CA, USA, 11–14 May 1998; pp. 105–108. Liu, Z.; Li, Z.; Li, G.; Zhang, X.; Zhang, H. A Novel Motion-Detection and Edge Detection Algorithm Based on Motion Estimation. In Recent Advances in Computer and Information Engineering; Springer: Berlin, Heidelberg, 2012; Volume 128, pp. 295–308. Luthon, F.; Dragomirescu, D. A Cellular Analog Network for MRF Based Video Motion Detection. IEEE Trans. Circuits Syst. I 1999, 46, 281–293. [CrossRef] Kristensen, F.; Hedberg, H.; Jiang, H.; Nilsson, P.; Öwall, V. An Embedded Real-Time Surveillance System: Implementation and Evaluation. J. Signal Process. Syst. 2008, 52, 75–94. [CrossRef] Jiang, H.; Ardö, H.; Öwall, V. A Hardware Architecture for Real-time Video Segmentation Utilizing Memory Reduction Techniques. IEEE Trans. Circuits Syst. Video Technol. 2009, 19, 226–236. [CrossRef] Genovese, M.; Napoli, E.; Petra, N. OpenCV Compatible Real Time Processor for Background Foreground Identification. In Proceedings of the International Conference on Microelectronics, Cairo, Egypt, 19–22 December 2010; pp. 467–470. Genovese, M.; Napoli, E. FPGA-Based Architecture for Real Time Segmentation and Denoising of HD Video. J. Real Time Image Process. 2013, 8, 389–401. [CrossRef] Genovese, M.; Napoli, E. ASIC and FPGA Implementation of the Gaussian Mixture Model Algorithm for Real-time Segmentation of High Definition Video. IEEE Trans. Very Large Scale Integr. 2014, 22, 537–547. [CrossRef] Yu, N.; Kim, K.; Salcic, Z. A New Motion Estimation Algorithm for Mobile Real-Time Video and Its FPGA Implementation. In Proceedings of the IEEE Region 10 Conference TENCON, Thailand, 21–24 November 2004; pp. 383–386. Saad, E.M.; Hamdy, A.; Abutaleb, M.M. Reconfigurable Hardware Implementation of a Fast and Efficient Motion Detection Algorithm. In Proceedings of the 10th International Conference on Mathematical Methods and Computational Techniques in Electrical Engineering, Corfu Island, Greece, 26–28 October 2008; pp. 40–45. Zhang, T.; Wu, H.; Borst, A.; Kuhnlenz, K.; Buss, M. An FPGA Implementation of Insect-Inspired Motion Detector for High-Speed Vision Systems. In Proceedings of the International Conference on Robotics and Automation, Pasadena, CA, USA, 19–23 May 2008; pp. 335–340. Hussian, S.N.; Rao, K.S.; Ashfaq, S.M. The Hardware Implementation of Motion Object Detection Based on Background Subtraction. Int. J. Adv. Res. Comput. Sci. Softw. Eng. 2013, 3, 1297–1302. Liang, H.; Morie, T. A Motion Detection Model Inspired by Hippocampal Function and Its FPGA Implementation. In Proceedings of the International Conference on Neural Information Processing, Shanghai, China, 13–17 November 2011; Volume 7064, pp. 522–529. McErlean, M. Hierarchical Motion Estimation for Embedded Object Tracking. In Proceedings of the IEEE International Symposium on Signal Processing and Information Technology, Vancouver, BC, Canada, 27–30 August 2006; pp. 797–802.
Electronics 2016, 5, 10
41.
42.
43.
44. 45.
46.
47.
48.
49.
50.
51.
52.
53.
54.
18 of 18
Sanchez-Ferreira, C.; Mori, J.Y.; Llanos, C.H. Background Subtraction Algorithm for Moving Object Detection in FPGA. In Proceedings of the Southern Conference on Programmable Logic, Bento Goncalves, Brazil, 20–23 March 2012; pp. 1–6. Bing, X.; Charoensak, C. Rapid FPGA Prototyping of Gabor-Wavelet Transform for Applications in Motion Detection. In Proceedings of the 7th International Conference on Control, Automation, Robotics and Vision, Singapore, 2–5 December 2002; Volume 3, pp. 1653–1657. Desmouliers, C.; Oruklu, E.; Sanile, J. FPGA-Based Design of a High-Performance and Modular Video Processing Platform. In Proceedings of the International Conference on Electro/Information Technology, Windsor, ON, Canada, 7–9 June 2009; pp. 393–398. Singh, S.; Mandal, A.S.; Shekhar, C.; Vohra, A. Real-time Implementation of Change Detection for Automated Video Surveillance System. ISRN Electron. 2013, 2013. Article ID 691930. [CrossRef] Singh, S.; Dunga, S.M.; Saini, R.; Mandal, A.S.; Shekhar, C.; Vohra, A.; Chaudhary, S. Hardware Accelerator Design for Change Detection in Smart Camera. In Proceedings of the SPIE International Conference on Graphics and Image Processing (ICGIP 2011), Cairo, Egypt, 30 September 2011. Bartosinski, R.; Danek, M.; Sykora, J.; Kohout, L.; Honzik, P. Video Surveillance Application Based on Application Specific Vector Processors. In Proceedings of the International Conference on Design and Architecture for signal and Image Processing, Karlsruhe, Germany, 23–25 October 2012; pp. 1–8. Saad, E.M.; Hamdy, A.; Abutateb, M.M. FPGA Implementation of a Low Cost and Area Real-Time Motion Detection. In Proceedings of the 15th International Conference on Mixed Design of Integrated Circuits and Systems, Poznan, Poland, 19–21 June 2008; pp. 249–254. Chowdary, M.K.; Babu, S.S.; Babu, S.S.; Khan, H. FPGA Implementation of Moving Object Detection in Frames by Using Background Subtraction Algorithm. In Proceedings of the International Conference on Communication and Signal Processing, Melmaruvathur, India, 3–5 April 2013; pp. 1032–1036. Hao, J.; Shibata, T. A VLSI-Implementation-Friendly EGO-Motion Detection Algorithm Based on Edge-Histogram Matching. In Proceedings of the International Conference on Acoustics, Speech and Signal Processing, Toulouse, 14–19 May 2006; Volume 2, pp. 245–248. Singh, S.; Dunga, S.M.; Mandal, A.S.; Shekhar, C.; Chaudhary, S. FPGA based Embedded Implementation of Video Summary Generation Scheme in Smart Camera Systems. Adv. Mater. Res. 2011, 403–408, 516–521. [CrossRef] Singh, S.; Dunga, S.M.; Mandal, A.S.; Chaudhary, S. Embedded Implementation of Change Detection Algorithm for Smart Camera Systems. In Proceedings of the IEEE International Conference on Recent Trends in Information, Telecommunication, and Computing, Kochi, Kerala, 12–13 March 2010; pp. 268–270. Singh, S.; Saini, R.; Saini, A.K.; Mandal, A.S.; Shekhar, C. Performance Evaluation of Different Memory Components for FPGA based Embedded System Design for Video Processing Application; International Journal of Intelligent Systems and Applications (IJISA), MECS Press: Hong Kong, China, 2013; Volume 5, pp. 113–119. Chutani, E.R.; Chaudhury, S. Video Trans-Coding in Smart Camera for Ubiquitous Multimedia Environment. In Proceedings of the International Symposium on Ubiquitous Multimedia Computing, Hobart, Australia, 13–15 October 2008; pp. 185–189. Rodriguez-Gomez, R.; Fernandez-Sanchez, E.J.; Diaz, J.; Ros, E. FPGA Implementation for Real-Time Background Subtraction Based on Horprasert Model. Sensors 2012, 12, 585–611. [CrossRef] [PubMed] © 2016 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons by Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/).
