Proceedings of the World Congress on Engineering and Computer Science 2008 WCECS 2008, October 22 - 24, 2008, San Francisco, USA
Optimal Piecewise Linear Approximation of Convex Functions A. Imamoto, B. Tang, Member, IAENG
Abstract— The optimal minimax solution to the N segment piecewise linear approximation of arbitrary convex differentiable functions over a finite range is described. The optimal solution is uniquely described by the derivatives at N distinct points. The optimality of the solution is proven and a recursive algorithm is proposed. The efficacy of the algorithm and optimality of the solution are demonstrated in example solutions for common functions such as 1/x and sin(πx). Index Terms— approximation linear minimax optimal piecewise
ε
β i +1
f (x)
ε
βi
ε
I. INTRODUCTION Approximation of functions by piecewise linear (PWL) functions are widely used in many hardware and software design applications. PWL functions allow the representation of arbitrary functions to any accuracy by simply increasing the number of segments until the desired accuracy is met. The purpose of this paper is to present a methodology for obtaining an optimal PWL function approximation to a convex (or concave) function over a finite range, where the PWL function is constrained to a fixed number of segments, and the optimization criteria is to minimize the maximum absolute error over the range, i.e. the minimax solution.
αi
ti
α i +1
Figure 1 − 1 Segment Minimax Solution
2ε
β i +1 f (x)
ε
II. BACKGROUND
2ε
A function f ( x ) is approximated by a linear function
βi
g (x ) defined over the range α i ≤ x ≤ α i +1 by the segment
connection endpoints (αi , β i ) and (α i +1 , β i +1 ) . The minimax solution is defined as the function g(x) such that it minimizes the error ε given by
ε = max α
i
≤ x ≤α i + 1
g ( x) − f ( x) .
(1)
αi
ti
α i +1
Figure 2 − Minimax Solution, Alternate Construct
Figure 1 shows the construct of the minimax solution where f(x) is a convex, continuously differentiable function. The segment connecting the endpoints (α i , f (α i )) and
Figure 2 shows an alternate construct of the minimax solution, which we will then use to construct the N segment piecewise linear solution. We define ti as the segment pivot,
(α i +1 , f (α i +1 )) has a maximum error of 2ε . The segment (α i , f (α i ) + ε ) connecting the endpoints and
β i +1 − β i . The minimax solution can then be α i +1 − α i defined as g ( x ) = f ' (ti )( x − ti ) + f (ti ) − ε , where the maximum error is ε . The tangent at ti , given
(α i +1 , f (α i +1 ) + ε ) has a maximum error of ε and is the minimax solution. Manuscript received July 30, 2008. A. Imamoto is with West HS, Torrance, CA 90503 USA (e-mail:
[email protected]). B. Tang is with Primarion, an Infineon Technologies Company, Torrance, CA 90505 USA, ((310)602-5500, e-mail:
[email protected]).
ISBN: 978-988-98671-0-2
where f ' (ti ) =
by g~( x ) = f ' (ti )( x − ti ) + f (ti ) , has the property that the error function e( x ) = g~( x ) − f ( x ) is strictly positive, convex, with maximum error ε at the endpoints α i and α i +1 .
WCECS 2008
Proceedings of the World Congress on Engineering and Computer Science 2008 WCECS 2008, October 22 - 24, 2008, San Francisco, USA Furthermore, since the error function is convex, the minimax solution over an alternate range * * * α i ≤ x ≤ α i +1 where α i +1 < α i +1 results in ε < ε and
IV. PROOF OF OPTIMALITY Define g i (x) , α i , and t i such that
t *i < ti . Similarly, if α *i +1 > α i +1 , then ε * > ε and t *i > ti .
g i (α i ) − f (α i ) = ε , for i=0 to N.
III. PROPOSED SOLUTION A convex function f (x ) is approximated by an N segment continuous piecewise linear function g (x ) defined over the
range α 0 ≤ x ≤ α N by the set of points or knots (α i , β i ) iN= 0 connected by N segments given by
g ( x ) = ∑iN=−01 gi ( x )
(10)
Assume g * ( x) , α * , and t * exist to represent a better solution (i.e. one with a smaller ε ) such that g i * (α i * ) − f (α i * ) < ε , for i=0 to N
(11)
* * (2) and α 0 = α 0 and α N = α N .
, where
If
⎧ ⎪m x + bi g i ( x) = ⎨ i 0 ⎪ ⎩ β − βi mi = i +1 α i +1 − α i
for α i ≤ x ≤ α i +1 otherwise
* (3) g 0 (α 0 ) − f (α 0 ) < ε
and
(4)
bi = β i − miα i , for i=0 to N-1
Furthermore, we (t0 , t1 ,...t N −1 ) where
define
(5) the
(12)
set
of
pivots
f ' (ti ) = mi , α i ≤ ti ≤ ti +1
(6)
g1* (α 1* ) − f (α 1 ) < ε ,
(13)
then t 0* < t 0 , α1* < α1 . Similarly, α i* < α i , t i* < t i for i=1 to N-1. Since α N* = α N , if t N* −1 < t N −1 , α *N −1 < t N −1 , then
g *N −1 (α N ) − f (α N ) > ε .
(14)
We propose that g(x) is the minimax optimal N segment Equation (14) shows that g i * (α i * ) − f (α i * ) < ε cannot piecewise linear approximation, i.e. ε = maxα 0 ≤ x ≤α N g ( x ) − f ( x ) is minimized (7) be true for N. Therefore, a better solution with a smaller ε does not exist and g i (x) as defined in (10) is the optimal if for i= 0 to N, and j = 0 to N-1: g (α ) − f (α ) = g (t ) − f (t ) = ε . (8) minimax PWL solution. i
i
j
j
V. RECURSIVE ALGORITHM
ε
Recursive solutions to finding the minimax solution of piecewise linear approximations have been previously proposed in [1] and [2]. However, these solutions are based on splitting algorithms, where a search algorithm is used to find the segments that meet a certain error criteria. An upper bound and lower bound is used, and the algorithm searches for improved approximations by reducing the lower and upper bounds until it converges to an acceptable accuracy. We propose a recursive algorithm for finding the optimal N segment PWL approximation directly by finding the solution to (8), the locations of the pivots ti , j .
f (x)
ε ε
ε
ε α0
t0
α1
t1
α2
The algorithm, shown in Figure 4, starts with an estimate of the pivots ti , j . The initial estimate ti ,0 can simply be
Figure 3 − 2 Segment Minimax Solution
uniformly spaced on α 0 < x < α N .
Fig. 3 shows a representation of a 2 segment solution. Note that the set of pivots (t0 , t1 ,...t N −1 ) fully defines g(x) and that g(x) may be alternately written as g ( x) = max iN=−01 f ' (t i )( x − t i ) + f (t i ) − ε .
ISBN: 978-988-98671-0-2
The tangent segments
gi , j ( x ) and the knot locations α i , j ( x ) at the intersection of the tangent segments are then computed based on the pivots. The errors at the knots, ε i , j = gi , j (α i , j ) − f (α i , j ) , are
computed and then used to provide a better estimate of the (9) pivot locations t i , j +1 .
WCECS 2008
Proceedings of the World Congress on Engineering and Computer Science 2008 WCECS 2008, October 22 - 24, 2008, San Francisco, USA VI. EXAMPLES
Given: f ( x ), N , α 0 , α N
The efficacy of the recursive algorithm is shown in the following examples. Table 1 shows the 8 segment solution for the function 1 f ( x) = from 1 ≤ x ≤ 2 . The pivots ( t i ), knots (α i , β i ) , x and segments (mi x + bi ) are shown. As discussed
Find: N −1 g ( x) i =0 i
g ( x ) = max/ min
which minimizes ε = maxα0 < x max | ε i , j −1 |
Yes
j = j −1
Δ = Δ2
No
d i, j
N −1 i =0
= Δ
ti , j +1
ε i +1, j − ε i , j ε i +1, j ε i, j + α i +1, j − ti , j ti , j − α i , j N −1 i =0
i
ti
αi
βi
mi
bi
0 1 2 3 4 5 6 7 8
1.0380 1.1200 1.2121 1.3161 1.4341 1.5687 1.7231 1.9015
1.0000 1.0775 1.1643 1.2620 1.3726 1.4984 1.6423 1.8079 2.0000
0.9987 0.9268 0.8576 0.7911 0.7272 0.6661 0.6076 0.5518 0.4987
-0.9281 -0.7972 -0.6806 -0.5773 -0.4862 -0.4064 -0.3368 -0.2766
1.9274 1.7863 1.6506 1.5202 1.3953 1.2756 1.1614 1.0524
Figure 6 shows the piecewise linear approximation g(x) and the error e(x) = g(x)-1/x over the range 1 ≤ x ≤ 2 . The equal maximum errors in each segment clearly demonstrate that g(x) is the minimax solution and that the error of the approximation is 0.00067.
8 Segment PWL, 1/x
= ti , j + d i , j 1.0
j = j +1
0.0010
g(x) g(x)-(1/x)
Figure 4 − PWL Search Algorithm
0.9
The computation of ti , j +1 is depicted graphically in Figure g(x)
5. To ensure convergence, step control Δ is used to ensure that the error decreases with each iteration. The algorithm converges to ti when the errors are essentially equal within a desired accuracy δ .
0.8 0.0000 0.7
g(x)-(1/x)
0.0005
-0.0005 0.6
0.5
-0.0010 1
1.1
1.2
1.3
1.4
1.5
1.6
1.7
1.8
1.9
2
x
Figure 6 − 8 Segment PWL Error 1 , 1 ≤ x ≤ 2 , ε = 0.00067 x
ε i +1 gi ( x ) − f ( x )
εi
αi
ti , j
ti , j +1
Figure 7 shows the construction of the PWL using the 8 segments displayed over the entire range, showing the relationship between the pivots and tangent segments in creating the piecewise linear approximation.
α i +1
Figure 5 − Updating Pivot Estimates
ISBN: 978-988-98671-0-2
WCECS 2008
Proceedings of the World Congress on Engineering and Computer Science 2008 WCECS 2008, October 22 - 24, 2008, San Francisco, USA 64 segment PWL, sin(πx)
8 Segment PWL, 1/x 1.0
1.0
0.00004
0.9 0.00002
0.7
0.5
0.00000
g(x) - sin (πx)
g(x)
gi(x)
0.8
0.6 -0.00002
0.5 1
1.1
1.2
1.3
1.4
1.5
1.6
1.7
1.8
1.9
g(x) g(x)-sin(pi x)
2
x
0.0
-0.00004 0
Figure 7 − 8 Segment PWL 1 7 , 1 ≤ x ≤ 2 , g i ( x ) i =0 x
0.1
0.2
0.3
0.4
0.5
x
Figure 8 − 64 Segment PWL Error sin(πx ), 0 ≤ x ≤ 0.5 , ε = 0.000022
A second example shows the algorithms’ easy scalability to more complex solutions. Table 2 shows the pivots for the 64 segment approximation of sin(πx ) from 0 ≤ x ≤ 0.5 . The pivot locations are an efficient method of describing the solution and the knots and segments are easily computed from the pivots.
1.5
g(x)
TABLE 2 64 SEGMENT PWL APPROXIMATION, PIVOTS ti
64 segment PWL, sin(π x) 2.0
1.0
sin(πx ), 0 ≤ x ≤ 0.5 , ε = 0.000022 0.5
0.016198
0.191817
0.305997
0.406857
0.037175
0.199673
0.312562
0.412945
0.053101
0.207396
0.319083
0.419018
0.066903
0.214999
0.325561
0.425075
0.079413
0.222489
0.331999
0.431119
0.091019
0.229877
0.338399
0.437150
0.101943
0.237168
0.344764
0.443169
0.112327
0.244370
0.351095
0.449179
0.122270
0.251488
0.357395
0.455179
0.131841
0.258529
0.363665
0.461171
0.141095
0.265496
0.369906
0.467156
0.150074
0.272395
0.376122
0.473136
0.158811
0.279229
0.382312
0.479111
0.167333
0.286003
0.388480
0.485082
0.175663
0.292720
0.394625
0.491050
0.183819
0.299384
0.400750
0.497017
0.0 0
0.1
0.2
0.3
0.4
0.5
x
Figure 9 − 64 Segment PWL 63
sin(πx ), 0 ≤ x ≤ 0.5 , g i ( x ) i =0
REFERENCES [1]
[2]
J. Vandewalle, “On the Calculation of the Piecewise Linear Approximation to a Discrete Function,” IEEE Tr. on Computers, vol. C-24, Issue 8, pp. 843 – 846, Aug. 1975. M. G. Cox, “An algorithm for approximating convex functions by means of the first degree splines,” Computer Journal, vol. 14, pp. 272-275, Aug. 1971.
The PWL approximation and error of the approximation are shown in Figure 8. The maximum error for 64 segments is 0.000022. Figure 9 shows the construction of the PWL approximation from the 64 segments.
ISBN: 978-988-98671-0-2
WCECS 2008
