FPGA-Based Robust Ellipse Estimation for Circular Road Sign Detection

FPGA-Based Robust Ellipse Estimation for Circular Road Sign Detection Samuele Martelli1 , Roberto Marzotto2 , Andrea Colombari2 , Vittorio Murino1,3 1 Dipartimento di Informatica, University of Verona, 2 eVS embedded Vision Systems S.r.l. 3 IIT Istituto Italiano di Tecnologia, Genova, Italy {samuele.martelli, vittorio.murino}@univr.it, {roberto.marzotto,andrea.colombari}@evsys.net Abstract

mination. Furthermore, RSs may be partly occluded, damaged, or other factors may corrupt the scene making the detection more difficult. In addition, the resulting implementation also needs to meet some important requirements like real-time constraints, high reliability and compact size if an embedded platform is employed.

Estimating parametric curves from images using robust fitting algorithms is a well-known and important computer vision task. We present a complete FPGA design and implementation of a fast and robust model fitting algorithm for real-time ellipse detection on video streams. The proposed solution relies on a the RANSAC algorithm, modified for FPGA deployment, in combination with an imagepreprocessing pipeline in order to perform the intensive pixel-level analysis, reducing each frame to a simple binary image of edges. The design has been developed in a parallel fashion and with specific architectural solutions so as to allow a fast response without degrading the functional performances. Experimental results on synthetic and real data show that our implementation, synthesized onto a Xilinx Spartan-3A DSP 3400A device, succesfully runs in real-time with a low resource occupation, while maintaining a functionality comparable with the floating-point software version.

While RS classification is often implemented using correlation techniques [15], or Artificial Neural Networks (ANNs), RS detection problem has been investigated by many research groups using either colour or shape information. Most of the leading approaches rely solely on shape analysis are curve fitting methods, like least-square fitting algorithms, Hough Transform (HT) [10], and Random Sample and Consensus (RANSAC) methods [7]. The disadvantages of least square fitting algorithms are well known, being sensitive to the presence of a small number of inaccurately detected edge points solely, due to a moderate noise level. Instead, HT and RANSAC methods became increasingly popular since they can deal with a large set of parametric curves even if data is affected by a large percentage of outliers. Actually, HT [11, 18] and RANSAC methods [2] are succesfully applied in a wide range of computer vision areas, and have a lot of variants aimed at increasing the efficiency of the basic methods.

1. Introduction Parametric curve detection and modeling is undoubtedly one of the fundamental issues in computer vision, which proved to be useful in many real-world applications. Among these, road sign detection constitutes a complex task, recently more and more investigated due to its inclusion in Driver Assistance Systems for improved automotive safety. In a typical circular road sign (RS) method, two main issues can be distinguished: the detection, aimed at identifying the region of interest (ROI) which may contain a RS; the classification, in which the RS is recognized, assuming that the ROI has been correctly detected [16]. The detection phase is the most important as errors in this stage may severely affect the rest of the processing. This task is also quite difficult since the video sequence is acquired from a moving vehicle, and the images are usually suffering from vibrations, and affected by blurring and non uniform illu-

Basically, HT-based methods transform input data from the image space to a parameter (or accumulator) space, then, by means of a voting mechanism, parametric curves are assigned a score, and the most voted points in the parameter space identify the winner (most probable) curves. In this formulation, it is clear that HT needs a huge amount of memory to store the accumulator space. Specifically in RS detection, several methods have been proposed that try to reduce the computational burden of HT, but embedded implementations still remain demanding in terms of resource allocation. RANSAC-based methods is not suffering from this problem and it is usually preferred because it does not needs to dynamically construct accumulators and dynamic memory allocation, which represent a crucial requirement in embedded system design.

1

RANSAC is an iterative algorithm composed by two steps: 1) generating a hypothesis (hypothesis formulation step) from random samples and 2) evaluating (voting step) it to the data. Further, it does not need complex optimization algorithms and huge amounts of memory. The hypothesis formulation and the voting steps are high parallelizable in hardware resulting in a large improvement of time performances as compared to standard implementation on general purpose processors. Moreover, RANSAC is robust to noisy data sets, hence, its use is particularly suitable in demanding computer vision applications like real time road sign recognition. In this paper, we present a field programmable gate array (FPGA) architecture for real-time processing of automotive road video sequences addressing the ellipse fitting problem with the aim of detecting circular road signs. The main goal of this work is to develop a highly parallel design optimizing the trade-off between area and speed of the circuit, while reaching optimal functional performances. We pursued an embedded solution based on FPGA since our interest is in the development of an automotive oriented system integrated in a compact and low-cost device. Although there have been previous approaches in curve fitting estimation adopting FPGA for HT-like algorithms implementations [13, 19], to the best of our knowledge this paper is one of the first studies addressing the problem with a parallel design of RANSAC in FPGA. The proposed architecture can achieve a minimum clock period of 25.837 ns, which is equivalent to 75.7 RANSAC operations per second for a data set of 512 points and 1000 iterations. Consequently, this design is capable of meeting the real-time data processing requirements of most automotive applications. Its performance in terms of robustness to noise has been measured and accuracy compared with an equivalent software implementation. The rest of the paper is organized as follows. In Section 2, we present an introduction to the RANSAC framework, focusing on the modifications carried out in our design leading to real-time performances. In Section 3, we describe the architecture and specific design issues, and the hardware optimizations faced during the implementation are also outlined. Section 4 reports the results and performances on simulated and real data, and Section 5 summarizes the main issues of the work and concludes the paper.

2. Ellipse estimation with RANSAC Extracting elliptical objects from image and video scenes has received much attention in computer vision due to the occurrence in traffic sign detection and for recognizing manufactured objects in industrial applications. RANSAC has been adopted as an effective parametric fitting method in a variety of different applications, especially for its intrinsic flexibility and large potential to recognize a huge amount

of important geometric features in a scene, as compared to competing techniques. Recent works using general purpose processors range from robust curb and ramp detection for safe parking [8], lane markings detection for lane departure warning systems [22], road sign detection and recognition for driver assistance systems [6] and ground plane detection for autonomous navigation [12]. RANSAC allows to extract interesting structures in images by fitting an a priori defined mathematical model on subsets of features such as edges, corners or regions. The model fitting algorithm adopted in this work is a modified version of the basic RANSAC algorithm [7], suitable to be implemented in a hardware platform meeting the real-time constraint and without decreasing functional performances.

2.1. Classic RANSAC formulation The RANSAC method (RANdom Sampling And Consensus) is an iterative algorithm able to fit a mathematical model (i.e., a parametric curve) on a data set of candidates points containing outliers [7]. The percentage of outliers handled by RANSAC can be larger than 50% of the entire data set [9]. Despite many modifications, RANSAC algorithm is essentially composed of two steps that are repeated in an iterative fashion (hypothesis-and-test framework). The classic formulation of RANSAC is the following: given a model that requires a Minimal Sample Set (MSS) of p data points to instantiate its free parameters, and a set of data points S containing outliers: 1. Hypothesis formulation: randomly select a subset of p points of S and instantiate the model from this subset. 2. Voting: determine the consensus set Si of data points that are within an error tolerance T of the model. 3. Repeat 1–2 for N trials and choose the model with the largest consensus set found. 4. Solution refinement: re-estimate the model (possibly using least-squares) using Si (the set of inliers). Given an image from which edge points set S containing outliers is extracted, first, a MSS is randomly selected from S and used to compute the model parameters. Given the ellipse equation a(x − p)2 + 2b(x − p)(y − q) + c(y − q)2 = 1

(1)

where p and q stand for the ellipse center and with restriction 4ac − b2 > 0, the ellipse model M = {p, q, a, b, c} is defined with five parameters (2 for the center, 2 for the lengths of the axes, and one for the orientation). Hence, in this case, the MSS cardinality is 5, which can be reduced to 3 using the Pascal’s theorem [1]. The center coordinates (p, q) can be computed given the tangents of only 3 points on

the ellipse, K1 , K2 , K3 . The center is located at the intersection of the bisections of the three tangents to the ellipse, as illustrated in Fig. 1. When the center coordinates are obtained, the coordinate system is shifted so that the point (p, q) becomes the origin. orem find the 2center q) of the using only 3 Then, thetoequation can be(p,applied to ellipse estimate the ellipse points estimation of tangents at each point. It allows a model M by given 3 points. linear estimation of ellipse using only 3 points. y2 x2 + = 1 Given 3 points P1 , P(2) 4.1.1 Ellipse from three points 2, A2 B2 P3 on an ellipse (see Figure 3) the center is computed as follows: where A and B are the semi-axes of the ellipse. Two points are needed to compute the hypothesis, the other is used for the consistency check. This implementation only detects el• Tangents at these 3 points (t1 , t2 , t3 ) are found. lipses with major axis 0◦ and 90◦ from the x axis. However, this limitation does not a 2great • Intersections of taffect t2 (Iperformances ) are 1 with the 1 ) and t2 with tto 3 (I degree incomputed. RS detection, with the advantage to save hardware resources. • Midpoints segments [P ] (M 2 ] and [Pof 2 P3 1 Afterwards, in the of testthe framework, the1 Pelements the enM2 )are areconsistent found. with the computed model are tire data and set that checked out. After a proper number of iterations N directly • The intersection of the segments [I1 M1 ] and [I2 M2 ] related to the outliers percentage, the model with the greatgives the ellipse center (C). est consensus set is chosen as the solution.

t1

I1

P11 K M1

C

P22 K M2

t3

P K33

I2 t2

Figure 1. Determining the ellipse center

Figure 3: Use of Pascal’s theorem for estimating ellipse center with 3 points. The basic RANSAC approach has the following advanWhen the center coordinates (p, q) are obtained the coorditages originated from random sampling: it is robust to outnate system is shifted such as (p, q) become origin. Then, liers and not memory consuming, flexible and adaptable to the Equation 3 can be applied to estimate the ellipse equadifferent curve3 points. models without radically changing the imtion using plementation, finally accuracy can be trimmed by changing the number of iterations. The importance of RANSAC as +one fitax2 + 2bxy cy 2of=the 1 most robust (3) ting technique is also evidenced by the variety of algorithms available. Some of these works aim to optimize the pro4.1.2 Ellipse estimation with RANSAC In the previcess of model verification, others seek to modify the samous section the ellipse estimation method was explained pling process in order to generate more useful hypotheses, when we have three points on the ellipse. The problem is while fewpoints efforts aim at developing implementato relatively obtain three belonging to the ellipse within the tionsnoise working in real-time. See [3] for a more comprehen(see Figure 4(a)). We used a RANSAC algorithm sive(Fischler review ofand RANSAC-based Bolles, 1981). Ittechniques. is composed of six steps:

2.2. Embedded RANSAC An FPGA implementation of the standard RANSAC can be a non trivial task, thus a modified formulation particularly suited for an embedded implementation has been chosen.If Our approach has been implemented in great, what itweis usu4. the number of support point is sufficiently allyaccept referred to as depth-first defined in We the stanthe model and exit themanner, loop with success. dard RANSAC and in theofscheme by Chum et assume that the number support proposed point is sufficient al [5]. thatthan all ascoring for aofparticular generated whenItitmeans is higher percentage the estimated theoretical circumference. hypothesis is ellipse performed before moving on to the next hypothesis. A real-time oriented implementation, the preemp5. Repeat the steps 1 to 4, n times. tive RANSAC framework [14], proposes another style, the breadth-first, number of hypotheses is fixed 6. If we arrivewhere to thisthe step, we declare a failure and there a priori.isThis scheme leads to generate all the hypotheses first, no ellipse found. and then to compare them in a parallel setting. Here, the appeal is that hypotheses can be measured against each other Suppose densitysome ratio of inlier isquality 50% andmetric, the probrather that thanthe against absolute and the ability that the algorithm withoutinfinding goodslot. fit is best solution is always exit evaluated a fixedatime chosen 5%, then, the number of needed iterations (n) is 25. Since this approach is very efficient but very resource demanding in hardware, we developed an hybrid procedure. the line of preemptive RANSAC, we the propose to tanuse a preInIn ellipse estimation, in order to compute needed determined number of hypotheses with the aim to meeting gent on each edge point, a line is fitted to its neighbours the real-time constraint, but preserving the iterative on the linked edges. A neighborhood of 2 pixels is cho-strucsen. to discretisation, it does not provide a good tureDue in order to reduce the amount of memory andtanprocessgent usingbypixel accuracy.scheme. This problem ingestimation resources when required the parallel One of the is main shownlimitation in Figuresof4(b) and 4(c). RANSAC, It causes more preemptive and frequent of our hybrid failure and less accurate result. In order to cope this [17], approach, is its inherent non-adaptiveness to with the data problem, the edge to provide a subi.e., it does not points achievearea delocalised fixed confidence in the solution. pixel accuracy using the method developed in (Devernay, However, since in embedded architectures the priority is the 1995). fixed-time constraint, we reached the goal to find the best solution, from a non-optimal restricted set of hypotheses, within a bounded runtimeofinterval. Figure 5 shows an example result obtained by this algoAs described above, the test step is the most intensive rithm. since, for all the points in the data set, consistency with the current model has to be checked. Since the number 5 HYPOTHESIS VERIFICATION AND TEXTURE of iterations N for each frame is fixed, the quality of the PATTERN RECOGNITION fitting results is enhanced avoiding to vote for bad hypothesis. Inspired by [20] and [4], a guided sampling module 5.1 Ellipse Rectification was added at the beginning of the pipeline, which attempts to quickly discard bad hypotheses by isfixing constraints Validation and recognition of road sign performed by on the hypotheses generation. The random search has been comparing the detected circular road sign with a set of ref- replaced by (See a guided strategy based on atexture prioriofknowledge erence ones Figure 7) . The inside sign is on the tangent values of edge points. Good hypotheses are genused to measure the its similarity with all reference signs. erated early on the sampling significantly reducing Correlation coefficient seems to process, be particularly interesting forthe this purpose. the detected signs are deformed search timeHowever (or the number of hypotheses required). toFinally, ellipse while theinspiration reference ones areMLESAC circular. It[21], makewe the use a taking from correlation process difficult. order likelihood, to resolve the prob-than a scoring function that is a Inrobust rather lem, we propose to rectify the texture of the detected sign simple inlier count. toIn match the geometry reference ones. The needed transthe voting step aofdatum is considered as inlier if its erformation must transform an ellipse to a circle of a given ror from a candidate model is within a predefined threshradius. This is performed using an 8 parameters projecoldtransformation. D. In case ofWe ellipse fitting, error can tive suppose that the the images arebe ap-a geometric distance Err from datum to the estimated ellipse. proximately horizontal or the orientation of the images are D is a tuning variable, highly related with magnitude of known so the transformation is unique. Figure 6 shows noise which is usually unknown. As pointed out by Torr some examples of resampled road signs. 5.2 Matching with texture DB

1. Pick randomly three points within the edges points. 2. Estimate the ellipse parameters (see Section 4.1.1). 3. Search how many edge points fit on the ellipse model

After rectification, in order to match only the pixel inside the road sign, we generate a circular mask and we apply the ZMNCC (Zero Mean Normalized Cross Correlation) function to compute the similarity of detected and refer-

Video Stream

Data Loader

Candidature

GNR

Ping-Pong Buffer Controller

Center Computation

Voting

Ellipse Model

Pre-Processing

Center-x Center-y Semi-Major axis

Support Evaluation

Semi-minor Axis

HST Dual Port RAM HV Gradient EETH

Hipothesis Formulation Find Best

Dual Port RAM

Hypothesis Verification

FPGA Spartan 3A-DSP

N iterations Data Out

Figure 2. General scheme of the architecture.

et al. [21], a binary logic-based RANSAC can be sensitive to the choice of the correct noise threshold D. The original RANSAC 0-1 voting logic has been replaced by a fuzzy one where the quality of a candidate model is evaluated calculating the likelihood of its consensus set and not the cardinality. Inspired by the fact that the likelihood measure depends on a function that weights edge points on the basis of their distance from the model, the proposed voting scheme is weighted (see details in the next section).

3. FPGA-Based Ellipse Detection Architecture The FPGA design of the RANSAC-based ellipse fitting algorithm consists in a pipeline composed by four main stages associated to correspondent processing modules. Starting from a set of candidate points, possibly corrupted by outliers, it returns the ellipse model M = {p, q, A, B}. Fig.2 shows the several modules of the architecture. The Pre-Processing module bufferizes the video stream and extracts edge points. The Data Point Loader aims at loading in memory the coordinates of the data points. The Model Candidature module randomly selects the hypotheses of the candidate ellipse models. Finally, the Model Voting module is responsible of the voting process for each candidate model, keeping track of the current winner solution. The RANSAC algorithm is characterized by a large degree of parallelism. For example, the Model Candidature stage is completely independent from the Model Voting task. Thus, the algorithm can be separated in two indepen-

dent pipelines, and the related computations can be carried out simultaneously (as illustrated in Fig. 3), iterating for a fixed number of times N . iter 1

Candidature

iter 2

iter 3

iter N

Voting

Voting

Voting

Candidature

Candidature

RES

Figure 3. Temporal diagram of Model Candidature and Model Voting stages working in parallel.

3.1. Pre-Processing The first stage of the pre-processing pipeline is a 2-D 5x5 Gaussian noise reduction (GNR) useful to improve the performance of the edge detection stage, typically affected by the noise. The second stage is histogram stretching (HST), which is used to enhance the contrast of the image exploiting as much as possible the whole gray-level scale. In the third step, the Horizzontal/Vertical Gradient module (HVG) apply a 5x5 Sobel filter with two 1D masks in cascade. The filters described above are designed to produce one pixel in output per clock cycle, as long as consecutive local areas (i.e., 5x5 neighbourhood areas every clock cycle) are continuously provided in input. A memory controller fetches every pixel and the related neighborhood to be processed from a 5-line memory buffer, and is responsible for maintaining this continuous flow of sequential neighborhood areas. Finally, the edge extraction and thinning (EETH) mod-

ule determines which points are edges by thresholding the gradient magnitude, and applying non-maximum suppression to generate thin contours (one-pixel thick). Using this procedure, the entire image is densely scanned, computing vertical and horizontal gradient for each pixel, as well as intensity and phase. The FPGA performs this processingintensive pixel-level analysis of each frame, and reduces the data from a 8-bit gray-scale image to a simple binary image for our downstream processing. The accuracy of this step is fundamental, especially regarding the phase, which is used in the ellipse center estimation module as described in Sec 2.1.

3.2. Data Point Loader The Data Point Loader is responsible to fill a table whose entries are given by the triple (xi , yi , ϕi ), that are the horizontal and vertical coordinates of an edge, and its phase, respectively, for each point coming from the EETH module. The table is implemented using the internal memory of the FPGA that is organized in elementary blocks (Block RAM, BRAM) of 18 Kbit. Therefore, the size of the table must be fixed a priori, and, in order to save FPGA memory and use a single BRAM, the size of the table is usually fixed at 512 points. The table is ready to be accessed by the Model Candidature and the Model Voting modules as soon as the whole data set S is completely stored. Since the Model Candidature and the Model Voting stages work in parallel, the table has been implemented using a dual port BRAM. To increase the throughput of the circuit two tables are allocated and involved in a “ping-pong” buffer strategy: while Data Point Loader is filling up the first buffer with the data set at instant t, S t , the second buffer was already filled with the data set S t−1 , and used by the RANSAC algorithm. Therefore as soon as RANSAC completes an iteration for S t−1 , the new edge points extracted from the current frame are already available in the other buffer, which are ready to be processed.

3.3. Model Candidature The Model Candidature module performs, at each iteration, a random sampling of a pair of points, and formulate an hypothesis by calculating the candidate model using the selected pair. This stage consists of two main steps: ellipse center computation, using the method described in Section 2.1, and ellipse parameter computation, which evaluates A and B, using equation 2. Once an hypothesis has been evaluated, the verification step checks if it satisfies two conditions, which are based on the size and shape of the ellipse. More precisely, an hypothesis must satisfy the condition Major-axis < 2 ∗ minor-axis, and they must be between a priori fixed minimum and maximum values. Many false hypotheses are identified and eliminated at this stage.

Using the method described in Section 2 to find the ellipse center, given 3 edge points and their tangents, 3 ellipse centers can be computed. Exploiting the intrinsic parallel nature of the FPGA, and using only few extra adders and multipliers, we have the advantage of calculating 3 ellipse hypotheses in the same iteration.

3.4. Model Voting The Model Voting module is responsible for the verification process, selecting the data points consistent with the model and finding the best hypothesis. For every iteration, it receives in input the three current models, which are then verified against all data points in the table. In RANSAC, we usually have to evaluate an Err function against a threshold D (see Section 2), in order to determine the so-called support, i.e., the number of data points consistent with the model. The Err function typically is a distance, or a geometrical measure reflecting the likelihood of the voting point with respect to the model. In the ellipse estimation framework, it is not trivial and resource demanding to compute the point-model distance, especially in hardware. Since the D value decides how many pixels an edge point is far from the ellipse to be considered an inlier, we defined a simple heuristics to find if a point belongs to the hypothesis, avoiding to compute any metric, and to compare it against D. Let Eh (x, y) be an ellipse hypothesis with semi-major axis A, and semi-minor axis B. A voting point should be located no more than D from Eh (x, y) to be an inlier, thus it has to belong to the range bounded by Ehext (x, y) and Ehint (x, y), where Ehext and Ehint are the external and internal bound of the region R, respectively. Thus the support is the locus of points between the two ellipses Ehext and Ehint . For a given voting point P = (v, u), it must be verified that  Ehext (v, u) < 0 (3) Ehint (v, u) > 0 in order to be part of the model. Using 10 system clocks, only few adders and multipliers and two comparators, this heuristics allowed to speed-up the intensive support calculation, saving also hardware resources. The module is optimized to compute a verification for each clock cycle. An accumulator is used to store the sum of the residue and a counter is used to register the number of inliers for each model. Both are reset at the end of each iteration. Since the Voting stage works in parallel with the Candidature stage, and implies a much larger number of operations, it is overclocked in order to work 9 times faster than the other blocks. It would be possible to further exploit the parallelization, but this would require more logic and a different configuration of the data point table. Once the voting is complete, the resulting vote is passed to a block responsible to updating current solution as the new one is better.

4. Experimental evaluation This section describes the preliminary results, figures of merit, obtained from our architecture, as well as the resources occupied. The adopted hardware platform is the XtremeDSP Video Starter Kit equipped with a Xilinx Spartan-3A DSP 3400A, an FPGA chip optimized for DSP applications. The FPGA resource occupation and the timing performances are estimated using the Xilinx development tools. The FPGA model was tested on both synthetic and real data sets, and the results are compared with those obtained using a behavioral software model of the algorithm and with the expected ground truth, respectively. Tests are first performed in simulation, and then in co-simulation with the hardware in the loop, providing the FPGA with input data coming from a software simulation environment.

4.1. Resource occupation and timing performance Our FPGA based ellipse fitting architecture was configured to compute 1000 iterations per frame. We selected an image size of 640 x 480, a data point table of 512 (32 bit) elements, and a pre-processing Sobel filter of 5 × 5 window size. It is worth to notice that the system was synthesized onto a Xilinx Spartan-3A DSP 3400A device utilizing less than 25% of system resources, leaving the possibility to implement in the same device other functionalities, like a classification module, which is the natural prosecution of a Driver Assistance system. Two BRAMs are used by the Data Point Loader module, one for each of two data point tables involved in the ping pong buffer. Other 6 BRAMs are used to implement the 1/x operator in the Model Candidature stage and for the trigonometric operations. A total of 36 DSP48 blocks are used to implement the multipliers in the Model Candidature and the Model Voting stages. The timing performances of the circuit are estimated through a Static Timing Analysis (STA), which reveals that the voting module is the bottleneck of our architecture, even if it was overclocked in order to work in parallel with the hypothesis formulation stage, as described in details in Sec.3. The estimated minimum clock period of the circuit is 25.837 ns, corresponding to a maximum clock frequency of 38.704 MHz. Since our algorithm takes 1 clock cycle for each vote and N = 1000, the time needed to perform a fitting process is given by 13.2 ms (which is also the block latency), corresponding to 75.7 ellipse fitting operations per second. Thus, we are allowed to set N to an upper bound of about 1200 iterations to meet the 60 fps constraint of highlevel frame grabbers.

4.2. Synthetic Data For these experiments, we estimate the performance using synthetic data over a wide range of parameters’ variations. A synthetic set of 512 points were used to evaluate the performances of our fixed-point precision RANSAC architecture as compared to a PC-based software version. The performances were measured computing accuracy and robustness to noise. Accuracy is quantified through the percentage error defined below (eq. 4), and a geometric (Euclidean) distance is used as error function. The robustness can be observed via variation of accuracy in varying configurations. Accuracy. The accuracy is evaluated as referred to the ellipse center, the semi-major, and the semi-minor axis. For each measurement, the results of both hardware and software implementations are compared with the ground truth. The two implementations are functionally equivalent, but while the hardware version works in fixed point precision, the software version works in floating point, therefore, the difference in the estimation is due only to numerical precision. For each measurement, we compute the maximum relative error, emax , and we also consider the mean of all the relative errors, eavg . For quantifying the accuracy of the fixed-point algorithm, and in the case of ellipse center coordinates estimation, we use the absolute percentage error, e, defined as: dist(Cf xp , Cref ) × 100 (4) e = Cref where dist(Cf xp , Cref ) is the geometric distance between the center point Cf xp , obtained from the fixed-point algorithm, and the reference point Cref , which is the ground truth. The same metric is adopted to evaluate the center point obtained with floating-point precision. We used an equivalent percentage error for evaluating the accuracy of semi-axes.

Ellipse Center X axis Y axis

fp : eavg 0.34 1.09 0.8

fxp : eavg 1.12 2.26 1.41

fp : emax 3.4 6.34 5.74

fxp : emax 7.2 18.4 11.2

Table 1. Accuracy - Synthetic Data

Table 1 presents the average and maximum percentage errors of the estimated parameters in floating-point (fp) and fixed-point (fxp) precisions. Larger errors of the axes with respect to the center are due to computational dependencies. As expected, the fxp parameters are less accurate, especially the axes, which results from the combined effect of several fixed-precision approximations. The computation of the ellipse parameters involves operations with the data in real ( 1000, allows to deal with datasets corrupted by more than 79% of outliers, which makes our method very robust in typical (noisy) road scenarios.

4.3. Circular RS Detection: Real Data Our RANSAC-based ellipse fitting architecture was also used to evaluate the performances on shape-based road sign detection. Real video sequences have been acquired in dense urban areas under changing light conditions. A set of 50 frames were extracted and selected for the experiments.The metric used to evaluate the proposed algorithm

is the percentage of correct detections. Experiments were performed 100 times on each image for statistically meaningful results. In Fig.4 (a), an experiment on a road scene with a traffic sign is presented. Fig.4 (b - c) respectively show the gradient magnitude and the edge map after the edge-thinning. Clearly, our framework is very effective at taking a roadway scene and reducing the data to only the primary ellipse-point candidates. In Fig.4 (d) the ellipse detected is overladed on the original frame. The red points represent the support. The detection algorithm performs properly with RSs under variations of scale, rotation and light conditions. In terms of scale, RSs as small as 25 pixels wide were detected. However it has been necessary to accurately design the pre-processing module in order to be robust with different lighting conditions, and to properly trim the number of RANSAC iterations to generate a sufficient number of hypotheses. Furthermore the data point table has been extended to 1024 elements and the search area was limited to about a quarter of the frame size in order to remove a consistent number of outliers and enhance the detection performance. It has been possible at the expense of hardware resources and time performance. The fitting accuracy has been easily trimmed by changing the number of clock cycles dedicated to the task during the system design and development phase; allowing to a complete customization for the specific application and without radically changing the general implementation. In order to achieve a detection rate of about 75% for the RS case study, N has been fixed to 2500 iterations, resulting in a architecture latency of 66.1ms and corresponding to a frame rate of 15 fps. However the fixed number of iterations and the large amount of outliers remain a challenge, especially in urban scenarios, where the outliers has been estimated to represent the 90-95% of the edges in the scene.

5. Conclusions In this paper we presented an implementation of a RANSAC-like algorithm for ellipse model fitting suitable for real time vision applications. Porting such a kind of algorithms to high performance computing devices is not a trivial task. This is particularly necessary when dealing with demanding applications which entail the extraction of structured features in real time. The proposed scheme was tested with synthetic data. A deepen analysis of the estimation error shows that the fixed point approximations used in the FPGA implementation does not affect the overall quality of the estimates. Moreover robust and real-time circular road sign detection using our architecture was demonstrated on real data. Future works will focus on the integration of the algorithm on a real road sign detection and classification architecture, and in the related validation phase, also looking fur-

(a)

(b)

(c)

Figure 4. Example of ellipse estimation on a real data set.

ther at a more efficient use of the hardware resources.

References [1] A. Arlicot, B. Soheilian, and N. Paparoditis. Circular Road Sign Extraction From Street Level Images Using Colour, Shape and Texture Database Maps. 2 [2] D. Capel and A. Zisserman. Automated mosaicing with super-resolution zoom. In Proc. CVPR, pages 885–891, 1998. 1 [3] S. Choi, T. Kim, and W. Yu. Performance Evaluation of RANSAC Family. pages 1–12, 2009. 3 [4] O. Chum and J. Matas. Matching with PROSAC-progressive sample consensus. In Proc. CVPR, volume 1, page 220, 2005. 3 [5] O. Chum, T. Werner, and J. Matas. Epipolar geometry estimation via RANSAC benefits from the oriented epipolar constraint. In Proc. ICPR, volume 1, 2004. 3 [6] M. Eichner and T. Breckon. Integrated Speed Limit Detection and Recognition from Real-Time Video. In publishing on the IEEE Intelligent Vehicles Symposium, 2008. 2 [7] M. A. Fischler and R. C. Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM, 24(6):381– 395, June 1981. 1, 2 [8] O. Gallo, R. Manduchi, and A. Rafii. Robust curb and ramp detection for safe parking using the Canesta TOF camera. In CVPR Workshops 2008, pages 1–8, 2008. 2 [9] R. Hartley and A. Zisserman. Multiple view geometry in computer vision. Cambridge Univ Pr, 2003. 2 [10] P. Hough. Method and means for recognizing complex patterns US Patent 3069654. Washington, DC: Patent and Trademark Office, 1962. 1 [11] N. Kehtarnavaz, N. Griswold, and D. Kang. Stop-sign recognition based on color/shape processing. Machine Vision and Applications, 6(4):206–208, 1993. 1 [12] K. Konolige, M. Agrawal, R. Bolles, C. Cowan, M. Fischler, and B. Gerkey. Outdoor mapping and navigation using stereo

[13] [14] [15] [16] [17]

[18]

[19] [20] [21] [22]

(d)

vision. In Proc. of the Intl. Symp. on Experimental Robotics (ISER). Springer, 2006. 2 M. Nakanishi and T. Ogura. Real-time line extraction using a highly parallel hough transform board. Image Processing, International Conference on, 2:582, 1997. 2 D. Nist´er. Preemptive RANSAC for live structure and motion estimation. Machine Vision and Applications, 16(5):321– 329, 2005. 3 P. Pacl´ık. The automatical Classification of Road Signs. Master’s thesis, Faculty of Transportation Sciences, Czech Technical Univeristy, 1998. 1 M. Prieto and A. Allen. Using self-organising maps in the detection and recognition of road signs. Image and Vision Computing, 27(6):673–683, 2009. 1 R. Raguram, J. Frahm, and M. Pollefeys. A Comparative Analysis of RANSAC Techniques Leading to Adaptive RealTime Random Sample Consensus. In European Conference on Computer Vision. Springer, 2008. 3 A. Ruiz, N. Guil, and M. Ujald´on. Recognition of circular patterns on GPUs: Performance analysis and contributions. Journal of Parallel and Distributed Computing, 68(10):1329–1338, 2008. 1 S. Tagzout, K. Achour, and O. Djekoune. Hough transform algorithm for fpga implementation. Signal Process., 81(6):1295–1301, 2001. 2 B. Tordoff and D. Murray. Guided sampling and consensus for motion estimation. Lecture Notes in Computer Science, pages 82–98, 2002. 3 P. Torr and A. Zisserman. MLESAC: A new robust estimator with application to estimating image geometry. Computer Vision and Image Understanding, 78(1):138–156, 2000. 3, 4 H. Wang and Q. Chen. Real-time lane detection in various conditions and night cases. In Proc. of the IEEE Intelligent Transportation Systems Conference, pages 1226–1231, 2006. 2