Theoretical Computer Science Theoretical
Computer
Science 2 19 (1999) 30 l-3 18 www.elsevier.com/locate/tcs
On the cost of uniform and nonuniform algorithms Erich
Novak a,*, Henryk
Woiniakowski
b,c
a Muthematisches
Institut, Universitiit Erlangen-Niirnberg. Bistnarckstr. I 112, 91054 Erlangen, Germane b Department of Computer Science, Columbia University, New York, NY 10027, USA ‘Institute of Applied Mathematics, University of Warsaw, ul. Banacha 2, 02-097 Wurszuwa, Polund
Abstract We compare the costs of uniform and nonuniform algorithms for approximate solutions of continuous problems assuming the real number model. We show that, in general, there is no relation between these costs. That is, the class of uniform algorithms may be empty; moreover, even if this class is nonempty then the cost of any uniform algorithm may be arbitrarily larger than the minimal cost of nonuniform algorithms. We also provide conditions under which there exist uniform algorithms whose cost is basically the same as the minimal cost of nonuniform algorithms. @ 1999 Elsevier Science B.V. All rights reserved. KeywonJs: Uniform algorithms; Information-based complexity
Continuous
problems
and algorithms;
1. Introduction We study the computation
of &-approximations
to exact solutions
of problems
and
assume the real number model without an oracle, as in [I], or with an oracle, as in [6] or, more formal, in [4]. Such approximations are usually computed by nonuniform algorithms (machines, programs). For nonuniform algorithms, the error parameter E is regarded as a fixed number. The choice of a nonuniform for different E’S we have, in general, different algorithms. part of the input of the nonuniform algorithm. The nonuniform algorithm P, has a finite number eters. The number of built-in parameters is small. The minimal cost of nonuniform is called the (nonuniform) the complexity.
* Corresponding
complexity.
algorithm depends on E, and The error parameter E is not
of built-in
(precomputed)
param-
usually depends on E and may be large if i: algorithms that compute an e-approximation For many problems
we know tight bounds
author. E-mail:
[email protected].
0304.3975/99/$-see front matter @ 1999 Elsevier Science B.V. All rights reserved PII: SO304-3975(98)00293-X
on
302
E. Novak,
Obviously, parameter
H. Wotniukow,skil
Theoretic&
it is more interesting
is no need to change the algorithm of built-in
than the minimal
the cost of a uniform
cost of nonuniform
for which the error algorithm
algorithms,
P has also
E.
We still want to
algorithm
P, which takes
but they do not depend on
but this time using the uniform
Obviously,
301-318
can be used for different E’S and there
when E varies. The uniform
parameters
an e-approximation,
E as part of its input.
Science 219 (1999)
uniform algorithms
to consider
E is part of the input. Such algorithms
a finite number compute
Cornputrr
algorithm
cannot
be smaller
i.e., the complexity.
It is tempting to define the unifbrm complexity as the minimal cost of uniform algorithms that compute an &-approximation. There is, however, a problem with such a definition. First of all, the class of uniform algorithms that compute an c-approximation may be empty. Furthermore, how to define the minimal and the minimal inf{q:
even if this class is nonempty, cost. For nonuniform
4: computes
is well-defined.
an r-:-approximation)
For uniform
algorithms
P, the cost cp is a function
of E which usually
we would like to find a uniform algoamong the cost functions of all uniform
i.e.,
cp* = inf{cp:
P with input E computes
But how to interpret
the last infimum?
cp- (E) = inf {c&a) : P with input Then cp* is essentially attained
we still have a problem P,, the cost cc is a number
cost
goes to infinity as e goes to zero. Intuitively, rithm P* whose cost function cp* is minimal algorithms,
algorithms
by a uniform
the (nonuniform)
an c-approximation}.
One can try to interpret E
computes
this pointwise,
i.e.,
an &-approximation}.
complexity,
but the last infimum
may not be
algorithm.
This explains why we do not define uniform complexity in this paper. Instead, we compare the costs of uniform algorithms with the (nonuniform) complexity. We study the real number model and show that, in general, anything can happen: l (i) For some problems, the class of uniform algorithms that compute an &-approximation is empty even though the class of nonuniform algorithms is nonempty and the complexity l
is relatively
algorithms exist. (ii) For some problems,
small.
We will provide
the cost of any uniform
conditions algorithm
under
which
is arbitrarily
uniform
larger than
the complexity. l (iii) For some problems, there exist uniform algorithms with essentially the same cost as the complexity. We now discuss these three cases in detail. (i) The class of uniform algorithms is empty for certain problems for which the error is measured by a metric. The choice of for a metric that forces us to compute this problem, the complexity is finite, depending on the specific form of the
the particular metric is crucial. This result holds some components of the solution exactly. For and can be a slowly increasing function of E-’ metric.
E. Novuk,
H. Woiniakowskil
Theoretic-d
Computer
Science 219 (1994)
303
301-318
The essence of this result also holds for certain problems whose solution does not require oracles. The sequence pk = 2’jk cannot be computed by a uniform (in k) algorithm, but trivially can be computed by nonuniform algorithms with small cost. Section
2 deals with the exact solution
of problems
without
oracles within the BSS-
model of computation. In Section complexity algorithms
3 we extend the results of Section
that require is nonempty
2 to problems
in information-based
the use of oracles. We also show that the class of uniform if the metric is replaced by a norm. The choice of the norm
relaxes the demand that some components of the solution must be computed exactly. It turns out that it is now enough to approximate the components of the solution. This implies the existence of uniform algorithms. Uniform algorithms may exist for problems
with metrics. It is enough to assume that
there exist nonuniform algorithms that are weakly continuous, see Section 4. Continuity means that a small change of the built-in parameters of the nonuniform algorithm causes a small change in its outputs. (ii) Even if the class of that for some problems
uniform
algorithms
the cost of uniform
(nonuniform) complexity. By “arbitrarily form algorithm that computes an
is
algorithms
nonempty,
it may
is arbitrarily
happen
larger than the
larger”, we mean that the cost of any unis-approximation can be of the form
exp(comp(s)), or exp(. . . exp(comp( c)) . .) where comp( s) denotes the complexity, and we can take arbitrarily many compositions of the exponent function. Such problems are presented in Sections 2 and 3. Once more, the choice of the metric is crucial. This bad property disappears if the metric is replaced by a norm. (iii) We now discuss the complexity. Section
when uniform
3, and for approximation
these two cases, we construct chosen
nonuniform
algorithms
have essentially
algorithms.
of certain linear functionals a uniform
algorithm.
from the magic number
algorithm
defined
parameters
parameters
algorithm
of appropriately
of the nonuniform parameter
for a given E, we decode
of the nonuniform
needed for this E. To control the cost of decoding, on the nonuniform algorithms.
in
defined in Section 5. In
number which is used as a single built-in
When we run the uniform the built-in
the same cost as problem
from a sequence
We code the built-in
into one single (magic)
of the uniform
algorithms
We prove that this holds for the approximation
algorithm
we need a continuity
which are assumption
As already mentioned, our construction of uniform algorithms requires one built-in parameter (a magic number) which captures the behavior of nonuniform algorithms for various F’S. It is usually very hard, if not impossible, to store such a number. This makes our results on uniform algorithms purely of theoretical interest. To address the practical side, we note that the difference between
uniform
and
nonuniform algorithms basically disappears if we do not let E go to zero, and instead consider E in the interval [slow, s,rr] with, say, slow = lo-’ and s,pp = 10e2. Then no magic numbers are needed to construct a uniform algorithm which works well for all I-:E [slow, EupP]. Details are provided in Section 6.
304
E. Novak,
H. Wo+ziakowskil
In this paper, we concentrate and uniform
algorithms
Throreticaal Computer
on the error parameter
integration
Y denotes the smoothness rth distributional additional
for functions of functions
derivatives
parameters:
r, p, d,M.
nonuniform
there are other parameters
suppose we want to find algorithms
from the Sobolev subclass
W~p([O, IId). Here,
defined over the d-dimensional
are bounded
301-318
E and we distinguish
by the way they treat E. Obviously,
which can play the same role as E. For instance, for multivariate
Science 219 (1999)
in the &-norm
There are natural
unit cube whose
by M. We now have four
restrictions
on them such as r and
d are positive integers, p E [ 1, co], M > 0, and to assure continuity of functions we need to assume that p >d/v for d > 1. Once more, nonuniform algorithms may take all these parameters as fixed. Then the nonuniform pute an s-approximation to the integral &,ld f(t)dt cle calls) and by performing uniform
algorithms
arithmetic
algorithm
ez,r,p,d,~ should com-
E, r, p, d,M
as inputs and compute
by sampling the function f (oraand branching operations. On the other hand,
may take all these parameters
an E-approximation to the integral by using oracle calls and by using arithmetic branching operations. We believe that a similar analysis of the cost of nonuniform uniform
algorithms
2. Computations We consider
could be performed
and and
for the case of many parameters.
without oracle the following
specific problem.
For a given sequence
of real numbers
we want to compute Sp(n) = pn for any n = 1,2,. . .. We use the BSS-model with the four standard arithmetic, copy and branching operations over the reals, see p = { pk},
[l] for exact definitions. We give a short and informal description of this model for the case when input and output consist of just one real number, as in our example. Any algorithm (machine) P is of the following
form. The algorithm P may use a finite number of built-in on P. Here the X-i parameters x_~,x_,+I , . . , x-1, where m is finite and may depend are real numbers. We input x0 and for j = 1,2,. . . , J the algorithm P performs either an arithmetic operation xj =
with ji,j2 Xj =
OP(Xi,>xjz > E [-m,j
- 11, and opt
{+, -;,/},
or a copy operation
X,j,
or a goto-operation if
X~,OP, Xj2
then goto X,
where op, E { Y-I >
with algebraic
numbers
y-i.
For this result,
see [2] (1959,
chap. V, paragraph
5,
exercise 7). However, as already mentioned, the sequence pk = 2’” is not contained 0 in an algebraic field extension Q(Y-~, . . . , y-1 ) = K. This is a contradiction. Theorem 1 states that for some sequences p = {pk} there are no uniform algorithms. Obviously for some p, it is easy to find uniform algorithms that compute S, and their cost is small or even equal to zero, i.e., it may be the same as the cost of the nonuniform algorithm. Indeed, take pk = k. Then P(n) =x0 computes S, with cost zero. We now show that for some sequences p, there exist uniform algorithms that compute SP although their cost can be arbitrarily large. This means that the costs of uniform and nonuniform algorithms can be arbitrarily different.
306
E. Novuk.
H. Wo5uukowskil
Let d be a positive /~(n,2)=2~”
Throwticul
integer.
Define
Computer
Scirnw
219 (1999)
h(n, 1) = 2” and
301-318
h(n,d)=2h(“,d-‘).
Hence,
and so on.
Theorem 2.2. For any integer d, there exist sequences p = { pk} such that inf
P: P computes4,
‘;:p:
cow>n) =
‘.
h(n, d)
Proof. Define pk = h(k, d + 2). First of all we find a uniform S, with cost h(n,d)( 1 + of 1)). Indeed, h(n, d + 2) = 22h’n’d’,wegetxJ=p,
algorithm
let x1 = 2 and xi+1 =xf,
that computes
Then xi = 22’-‘. Since,
forJ=h(n,d)+l.
We can compute the number J = h(n, d) + 1 in cost much smaller than the cost of h(k,d + 2) and then we perform the loop for i:= 1 step 1 until J do xi :=x2. If the cost of this loop is J then we are done. However, the cost of such a loop is basically 3J, because we also need branchings and an integer operation like i := i + 1. We can modify the loop in such a way that several operations of the form xi :=xf are performed before the next branching and the next integer operation. This way we can prove that the infimum Consider
in Theorem
now an arbitrary
may use the built-in a=max{n,IX_j/,
uniform
parameters l/lxil,
2.2 is at most one.
x-j,
j=
algorithm
P that computes
S,. The algorithm
P
j = 1,2,. . . , m and the input n. Let
1,2 ,..., m for X_,j#O}.
The algorithm P computes x,~= op(xi, ,xi2 ) for jr, j2
