Background Introduction Problem to solve Methods Proposal Experimental results Conclusions Future work
Automated Functional Coverage for a Digital System Based on a Binary Differential Evolution Algorithm 1st. BRICS Countries Congress (BRICS-CCI) Recife, ’Porto de Galinhas beach, Brazil Alfonso Martínez Cruz Herón Molina Lozano Ricardo Barrón Fernández Centro de Investigación en Computación, CIC-IPN
Mexico D.F. September, 2013
1 / 22
Background Introduction Problem to solve Methods Proposal Experimental results Conclusions Future work
Content 1
Background
2
Introduction
3
Problem to solve
4
Methods
5
Proposal
6
Experimental results
7
Conclusions
8
Future work
2 / 22
Background Introduction Problem to solve Methods Proposal Experimental results Conclusions Future work
Definitions
1.- Fault is a deviation of a specified behavior system. 2.- Coverage is the percentage of faults covered. It means the amount of faults detected of all possible faults in a digital circuit. Number of points exercised Number of total points. 3.- Complex digital circuit is a circuit composed of a large number of semiconductor components. 4.- Coverage point is a scalar, a vector value or an expression.
3 / 22
Background Introduction Problem to solve Methods Proposal Experimental results Conclusions Future work
Definitions
5.- Coverage hole is a point which has not been exercised during the verification process. 6.- Coverage model is a functional space, which captures the behavior of the device. 7.- Coverage metric is a parameter or attribute that is used as a unit of measure for assessing progress in the verification of some dimension. 8.- Coverage Space is a multi-dimensional region defined by attributes, states and their values.
4 / 22
Background Introduction Problem to solve Methods Proposal Experimental results Conclusions Future work
Well known faults Ariane 5.- It exploded on its first flight; caused by reusing of some parts of a code from its predecessor without proper verification. Therac-25.- It was a radiation therapy machine. Due to a software error, six people died because of overdoses. Pentium II.- It had a design error in a floating point division unit; Intel was forced to offer replacements of all flawed processors.
5 / 22
Background Introduction Problem to solve Methods Proposal Experimental results Conclusions Future work
Selection of functional coverage points Bin
Digital system specification review Address: [0:255] = {0,1,2,3,4, ... , 255}
Full [0:1] = {0,1}
0 1
HDL Design
DUV
DUV Data_in[0:255] = {0,1,2, ... , 255} Data_out[0:255] = {0,1,2, ... , 255}
6 / 22
Background Introduction Problem to solve Methods Proposal Experimental results Conclusions Future work
Coverage model verification
Coverage analysis
Generator
1 0 1 0
1 0 1 0
1 0
0 0
...
...
1 0 1 1
1 0 0 0
1 0
1 0
...
DUV
...
Detection of holes
7 / 22
Background Introduction Problem to solve Methods Proposal Experimental results Conclusions Future work
General method Start Review the digital system specification
Analize the implementation of the design at the level RTL (VHDL, Verilog, Systemverilog)
Propose a functional coverage model Define a verification plan based on the design Perform the functional verification of the digital system based on a software system
not
Modifying tests
Analize the results and measure the advance in the coverage reached
yes
Process end 8 / 22
Background Introduction Problem to solve Methods Proposal Experimental results Conclusions Future work
Techniques
9 / 22
Background Introduction Problem to solve Methods Proposal Experimental results Conclusions Future work
Examples of techniques
10 / 22
Background Introduction Problem to solve Methods Proposal Experimental results Conclusions Future work
Proposed system architecture
Coverage metrics (coverage percentage)
Digital system Specification
Test vectors generation (Binary Evolution Differential Algorithm)
DUV
Simulator
(FIFO)
(Modelsim)
Output data analysis (Coverage holes)
Functional coverage model (Selection of coverage points)
11 / 22
Background Introduction Problem to solve Methods Proposal Experimental results Conclusions Future work
Binary Differential Evolution algorithm G=0 Create a random initial population ~xi,G ∀ i i = 1, ...NP Evaluate f (~xi,G )∀ i i = 1, ...NP Compute: for G = 1 → MAX GEN do for i = 1 → NP do Select randomly r1 6= r2 6= r3 : jrand = randint(1, D) for j = 1 → D do if randj [0, 1) < CR or j = jrand then Ui,j,G+1 = MOD2 (xr 1,j,G + (xr 2,j,G XOR xr 3,j,G )) end ui,j,G+1 = xi,j,G end if f (~ui,j,G+1 ≤ ~ui,j,G ) then xi,j,G+1 = ui,G+1 end xi,j,G+1 = xi,G end G = G+1 end
Algorithm 1: Differential Evolution algorithm (DE)
12 / 22
Background Introduction Problem to solve Methods Proposal Experimental results Conclusions Future work
Representation
The representation of the vectors is based on a fixed-legth binary string, wich represents a set of test vectors for the device under verification. In the case of the FIFO memory, each vector is composed of a sequence of 256 vectors of 8 bits. The representation of vectors is defined as follow: Pi = {~x1 , ~x2 , ~x3 , ..., ~xn }
(1)
where: ~xi ⊆ X , X ≡ {0, 1}8
13 / 22
Background Introduction Problem to solve Methods Proposal Experimental results Conclusions Future work
Fitness function Percentage of covereged bins = 100 ∗
Number of bins exercised Number of total bins
Percentage of not coveraged bins = 100 ∗
Number of not exercised bins Number of total bins
Coverage percentage =
P W × Ci |binscoverage | Pi Cg = , Ci = Wi
|bins|
Fitness function F(x) = Max
Percentage of
1 + Percentage of not exercised bins
covereged bins
14 / 22
Background Introduction Problem to solve Methods Proposal Experimental results Conclusions Future work
DE parameters
Scenarios Population CR value F value Number of experiments Stop criterion
1 2 3 10 8 20 0.25 0.25 0.25 0.50 0.85 0.8 30 30 30 Number of evaluations
15 / 22
Background Introduction Problem to solve Methods Proposal Experimental results Conclusions Future work
Comparison of different DE scenarios
Percentage of functional coverage
Results obtained for three different Scenarios using the binary DE Best, average and worst values for each Scenario 99 98,5 98 97,5
Best value Worst value Average value
97 96,5 96 95,5 95 1
2
3
Number of Scenario
16 / 22
Background Introduction Problem to solve Methods Proposal Experimental results Conclusions Future work
Results of the second scenario
This table shows the best and the worst solution, the average value and variance found with the binary Differential Evolution algorithm. Number of evaluations 500 1000 5000 10000 20000 50000
best 92.6269 92.8710 94.6289 95.7519 97.4121 98.7304
worst 91.4062 91.5527 93.4570 94.9218 96.3378 98.0468
average 91.9449 92.1793 94.0218 95.2897 96.8131 98.3837
variance 0.1133 0.1190 0.0642 0.0349 0.0644 0.0222
17 / 22
Background Introduction Problem to solve Methods Proposal Experimental results Conclusions Future work
Chart of convergence
18 / 22
Background Introduction Problem to solve Methods Proposal Experimental results Conclusions Future work
Conclusions
1.- The method based on the binary Differential Evolution algorithm permitted levels above 97 % functional coverage. 2.- In the case of twenty individuals, percentages of more than 93 % from 5000 evaluations of the algorithm stopping criterion were obtained. 3.- In the second scenario at the beginning of the algorithm execution, percentages of over 90 % were quickly reached. However, many generations were required to further increase this percentage.
19 / 22
Background Introduction Problem to solve Methods Proposal Experimental results Conclusions Future work
Future work
We propose to apply this methodology in more complex digital systems and comparate it with other techniques. To improve the methodology to implement constrained vector generation.
20 / 22
Background Introduction Problem to solve Methods Proposal Experimental results Conclusions Future work
Acknowledgment
21 / 22
Background Introduction Problem to solve Methods Proposal Experimental results Conclusions Future work
“Thank you, very much” E-mail: Alfonso Martínez Cruz
[email protected] Ricardo Barrón Fernández
[email protected] Herón Molina Lozano
[email protected]
22 / 22
