CUTE: Constrained and Unconstrained Testing Environment i. BONGARTZ
and A. R, CONN
IBM T. J. Watson Research Center NICK GOULD Rutherford Appleton
Laboratory
and Ph. L. TOINT Facult6s Universitaires
ND de la Paix
The purpose of this article testing
small-
facilities
and
were
LANCELOT,
for
anonymous The by
originally
ftp
their from
with
input
the
format
number a specific
in mind,
bias
illustrate
to many
of the
everything
sufficiently Categories
with
and
Subject
Techniques—modules
and
This
research
ment
of Defense
was
Authors’
addresses:
218, Yorktown ing
The
Department,
email:
NY
ND
de
la
D.2.2
Ph.
61,
rue
Paix,
Force
A. R, Corm, email:
Toint,
de
and C
we do not
of potential confusion.
users
Tools
are
and
Optimization—constrained
Projects
Agency Research
to reproduce notation
T. J. Watson
of the
Depart-
under and
Contract distribute
hereon,
Research
Center,
P.O.
N. Gould,
Central
Comput-
Oxfordshire,
OX1l
of Mathematics,
B-5000,
and a
of problems,
of simplicity,
any copyright
Chilton,
not this
includes
is a Unix
to undue
of Scientific
Department
Bruxelles,
classes
is authorized IBM
the SIF
majority
[email protected];
Laboratory, L.
between
There
used
was
between
Engineering]:
Research Office
(SIF)
as each problem
Analysis]:
Government
Appleton ac.uk;
[Software
[Numerical
software
sake
to by
automatically.
most
particular
the
package
obtained
format
information.
not lead
be
input
a link
for the
that
will
notwithstanding
and
10598;
Rutherford
[email protected].
sitaires
States
L Bongartz
since,
by the Advanced
purposes
Heights,
article,
by the Air
provide
can
an interface
Additionally,
of this
We trust
G.1.6
in part
tools
these
be available
be installed
that
in identifying
examples
Descriptors:
United
for governmental
these
cases,
for
of
software
should
tools
in building
and OSL.
a database
in the
interfaces;
supported
These
generality.
that
and was monitored
F49620-91-C-O079. reprints
fullest
Unix
to assist
environment many
the
and
in the standard Recognizing
to be useful
and manage
with
right The
in many
written
MINOS
descriptions
in its
familiar
packages.
is designed
to build
may,
tools
own
versatile
a
Although
conjunction
software.
is described,
including
that
are provided
and
we provide
optimization
classification
facilities shell
package
in
in their
problems
of
algorithms.
optimization
of sources
software
packages,
authors
be useful
of
of test
and other
optimization the
will
collection
SIF
of existing
by
they
a number
LANCELOT
written
that
development
scope of a major the
nonlinear
produced
we believe
researchers
discuss the scope and functionality
is to
large-scale
Namur,
Box
OQX,
England;
Facult6s
Univer-
Belgium;
email:
pht@
math.fundp.ac.be. Permission to copy without fee all or part of this material not made or distributed for direct commercial advantage, of the
publication
Association specific
and
its
for Computing
date
appear,
Machinery.
and
notice
is granted provided the ACM copyright
is given
To copy otherwise,
that
copying
that the copies are notice and the title
is by permission
or to republish,
requires
of the
a fee and/or
permission.
@ 1995 ACM
0098-3500/95/0300-0123 ACM Transactions
$03.50
on Mathematical
Software,
Vol. 21, No. 1, March
1995, Pages 123-160.
124
.
1. Bongartzet
optimuatlon;
nonlinear
ware—certification General Additional
programming;
and
Terms:
al.
testing;
Algorithms,
Key
Words
and
G.4 [Mathematics
reliability
and
of Computing]:
robustness;
Mathematical
Soft-
uertficatlon
Performance Phrases:
Test
article
is
problems
1. INTRODUCTION The
purpose
devised
of this
for testing
small-
to
describe
and large-scale
an
environment
nonlinear
that
optimization
we
have
algorithms.
It is inevitable that, during the process of developing a software package, the designers concern themselves with problems of testing. Thus, in a major project, one has to consider how to obtain and specify a suitable collection of test problems. hensive, it will
If the collection require careful
of test problems is to be reasonably management. Additionally, one might
that researchers would like to compare different algorithmic problem, All of these situations occurred during our own indeed, duced
many
of the
and tested
facilities
described
in conjunction
Corm et al. [1992a]). In this article we discuss testing
environment
with
the
(CUTE)
in this
scope of our for
article
the software
constrained
nonlinear
approaches researches,
to a and
originally
pro-
LANCELOT
(see
were
package
comprepresume
and
programming
unconstrained algorithms.
We
describe in some depth a collection of test problems that we have gathered from a variety of academic and real-life sources. Furthermore, we discuss a number of tools that we have produced to manage the resulting test problem database. An additional set of Fortran tools, which are intended to facilitate the building of an interface between the test problem collection and potential optimization These
software
tools
have
packages
to be rather
optimization community problems and algorithms. We have
tried
to order
the
of this
—I
a (class
content
an ideal
article
user who might
of) problem(s).
short)
is
also
described.
is, as yet, no consensus
constitutes
of a potential
to solve
for
as there
as to what
the requirements questions: want
(interfacing,
general
What
interface
so that
in the between
it corresponds
pose the following
sequence
to of
is available?
—My problem is not available in the current test set. I would like to construct it and include it in my classification database. How do I proceed? —I
now know which problem(s) I want to solve. Which faces available in CUTE, and how do I use them?
—The
solver
Fortran
I would
interface
of choice
using
The facilities community—and
like
to use is not
between the tools
my
specific
available. (class
How
solvers can I build
of) problem(s)
and
tion
software,
much
TransactIons
on Mathematical
in the
inter-
a suitable my solver
supplied?
offered are intended to be useful to the general especially to researchers—for the development
ACM
have
same Software,
spirit
as those
Vol. 21, No
of Buckley
1, March
1995
optimization of optimiza-
[1992].
All
of the
Constrained
test problems used
by
classification the
test
Testing Environment
that
is designed
We provide problems
to be useful
an automatic
in any
facility
collection
in identifying
particular
classes
of
for selecting
interesting
subsets
of
conforming
to this
We also provide tools for maintaining a database allow for the introduction of new members. We provide fied
using
additional
the
SIF,
tools and
Since most software important to provide
to allow
other
was such
of classified
an interface
existing
classification
between
nonlinear
system.
problems
problems,
programming
and derivative a relatively easy
Details of both Of course, CUTE
include
any
authors
source, To
object,
obtain
(or, when
or executable
these
available,
files,
the
transfer
files user
for
should
the files
external
contact
from
TEST
PROBLEM
2.1 The Scope
SIF and does not
optimization
the
appropriate
the appropriate
We conclude the article by describing how to obtain the files that CUTE and how to install the accompanying tools on a user’s Further details are provided in a number of appendices. 2. THE
speci-
packages.
means of building interfaces with new algorithms. LANCELOT are provided by Corm et al. [1992a]. the
to
not written to interface with this format, it is tools. Furthermore, Fortran subroutines are sup-
plied to take input written in SIF and to return function values in various dense and sparse formats, thus providing
packages.
125
.
in our collection are written in the standard input format (SIF) LANCELOT software package. Each problem comes with a
the
problems.
and Unconstrained
ftp
site).
make up machine.
DATABASE
of the Collection
The provided collection of nonlinear set of test problems written in the
test problems is intended SIF used by LANCELOT.
to be a growing This collection
contains
a large number of nonlinear optimization problems of various sizes and “real-world” applications. and difficulty, representing both “academic” Both constrained and unconstrained examples are included. On December 15, 1994, the database consisted of 738 test problems. Because many of these problems allow various choices for the number of variables and/or the number of constraints, many more instances can be defined by the user. More than
a thousand
such
instances
the problem definitions. It is clearly impossible will
suffice
—the
to say that
“Argonne
1989], the the Dembo problems and Tone Tuyttens 1991]; —most
to describe the test
set”
[Mor6
by appropriate
all of these
set covers, et al.
among
1981],
examples
comments
in this
in
article.
It
others,
the
Testpack
report
[Buckley
Hock and Schittkowski collection [Hock and Schittkowski 1981], network problems [Dembo 1984], the Mor6-Toraldo quadratic [Mor6 and Toraldo 1991], the Boggs-Tolle problems [Boggs 1989], the Toint-Tuyttens network model problems [Toint and 1990], and Gould’s quadratic programming problems [Gould
problems
—problems
test
are suggested
from
inspired
the PSPMIN
by the orthogonal ACM
Transactions
collection
[Toint
regression on Mathematical
19831;
report Software,
by Gulliksson
[ 1990];
Vol. 21, No. 1, March
1995.
126
.
—some
problems
Mor6 tion
1. Bongartzet
1991;
the
not
because
Minpack-2
et al. 1991] 1987];
number
Moreover, but
from
Averick
[Schittkowski
—a large
al.
problems
LANCELOT
ming
because
format
problems
the
(see IBM
SIF
that
the
[Averick
Schittkowski
and collec-
of application
collection
will
agreement,
areas.
grow
which
steadily,
requires
the
by most users of the package, the new more user oriented than the present ones. is an extension
[1978],
(e.g., Gay [1985])
collection
second
a variety
licensing
submission of typical problems problems are likely to be even Furthermore,
the
from
can one assume
of the
problem
from
and
of original only
test
and
access
is possible.
of the
to suites Tables
MPS
of linear
linear
program-
programming
I and II are included
test
to indicate
the scope of the applications that are currently represented in the database. Also included are some other test problems of interest to optimization algorithm designers. As can be seen from in the current can also simple present
these
collection,
be seen that
bounds)
cover
collection
the
dimension
a wide
involve
large- and small-scale number of problems
tables,
many
and the coverage range.
constraints
and
application
are represented
to become
number
of constraints
About more
areas
is likely one-third general
of the than
even wider,
problems
simple
It
(excluding in the
bounds.
Both
problems are included. The collection also contains a posed as nonlinear systems of equations. The SIF file
associated with each problem contains a reference to its statement in the literature, when appropriate. A summary of details on the test results obtained with various of the options available with the software package LANCELOT is given by Corm et al. [ 1992 b]. A technical report containing the complete results (including a more detailed description of each problem’s characteristics and the complete set of results Corm The
obtained
on nearly
20,000
separate
runs)
is also available
(see
et al. [1992c]). current
size
of the
total
uncompressed
database
of test
problems
is
almost 100 MB. Since this database is substantial, it is available in two separate compressed files, one containing the smaller problems (mastsif.small. tar.Z) and the other the larger problems (mastsif.large. tar. Z). Individual problems are also available, In a Unixl environment, we assume that the directory in which these problems reside is pointed to by the environment variable MASTS IF. We also assume that the directory where CUTE is installed (see Section 7 for more details) is pointed to by the environment variable CUTEDI R. 2.2 The Classification
Scheme
Each
in our data
of the problems
set comes with
a simple
problem
classifica-
tion inspired by the scheme of Hock and Schittkowski [1981], which was itself a slight extension of a scheme adopted by Bus [1977]. The scope of such a
1UNIX
is a
registered trademark of AT & T Bell Laboratories.
ACM Transactions
on
Mathematical Software, Vol. 21, No 1, March 1995
Constrained
and Unconstrained
Table
I.
Typical
Problem ~craft
Testing Environment
variables
problem
Elastic-plastic
torsion problem
Membrane
problem
network
Optimization
gas flow problem
of electrical
Oscillation
problem
9 0
CLPLATEA
15625
0
TORSION1
mechanics
AIRCRFTA
8
1
HS106
15625
0
JNLBRNGA
13
15
2734
2727
6
4
network
in structural
SIF file name
9
separation
Nonlinear
constraints
4970
Heat exchanger design Journal-bearing
127
Examples
stability
Clamped-plate
.
HS116 BRIDGEND HS87
5041
0
SVANBERG
Static power scheduling
9
6
HS107
Time-optimal
2
1
HS88
heat conduction
Transformer
design
Three-stage
launch vehicle design
Minimal-weight Aircraft
rotating-disc
design
design
6
2
25
28
905
1081
49
32
HS93 LAUNCH ROTDISC AVION2
1-D variational
problem
from ODES
1001
0
BRATUID
2-D variational
problem
from PDEs
5184
0
BRATU2D
3-D variational
problem
from PDEs
4913
0
BRATU3D
Applied
6
15
2
0
ROSENBR
5002
0
TRIGGER
5002
0
BDVALUE
5000
0
VAREIGVL
10000
0
MSQRTA
3946
3690
15625
0
13284
6724
GRIDNETA
10000
5000
HAGER4
6
15
LEWISPOL
15625
0
OBSTCLBU
649
349
GOULDQP2
8197
4096
ORTHREGA
18
18
NYSTROM5
5
502
6
2654
geometry
Banana shaping Turning
point in curve tracing
Discretized
boundary
Eigenvalue
calculations
Matrix
value
square root
Pivot growth
in Gaussian Elimination
Surface problem
with nonlinear
Nonlinear
network
Nonlinear
optimal-control
problem
conditions
on square grid problem
Number theory Obstacle problem Optimal
knot placement
for ODE solution
Orthogonal
regression problem
Quadrature
rule determination
Rational
approximation
Smallest crescent containing
ACM
given points
Transactions
on Mathematical
Software,
PENTAGON
GAUSSELM NLMSURF
EXPFITC CRESC132
Vol. 21, No. 1, March
1995.
128
.
1. Bongartz et al. Table
II.
Typical
Problem
Examples
variables
SIF file name
constraints
Physics Quantum
physics
600
1
Radiative
transfer
100
100
CHANDHEQ
246
0
RAYBENDS
1002
1000
3
0
5000
10000
READING3
8
60
VIBRBEAM
Seismic ray bending Semiconductor Thermistor
analysis
modeling
Tide modeling Vibrating
beam modeling
Chemistry
LCH
SEMICON2 MEYER3
and Biology
Alkylation
10
11
HS114
43
14
HIMMELBJ
8
0
PALMERIC
5000
5000
99
99
modeling
4
0
KOWOSB
Dipole model of the heart
8
8
HEART8
19
8
FCCU
4
3
HS73
500
0
PRODPL1
4
2
HS68
Chemical
equilibrium
Chemical
kinetics
Chemical
reaction
Distillation
problem
column modeling
Enzyme reaction Fluid catalytic
cracker modeling
Economy and Operations production
Cost optimal
planning
inspection
HYDCAR20
Research
Cattle feeding Computer
CHEMRCTA
plan
Economic
equilibrium
1825
730
Economic
model from Thailand
2230
1112
BIGBANK
194
120
SSEBNLN
2017
1008
Electric
power generation
Hydro-electric Optimal Nonlinear
reservoir management
sample sizing blending
Traffic equilibrium
4
2 14
HIMMELBK
540
126
STEENBRE ODNAMUR
11276
0
906
666
Weapon assignment
100
12
classification
demand estimation
could
be very
broad
indeed,
HYDROELL
24
Water distribution
Transportation
MANNE
and
we
realize
HS72
DALLASL HIMMELBI
that
a general
scheme that encompasses much more than we have succeeded in doin~ could be very useful for large databases such as ours. However, we have consciously limited the scope of our classification as follows. A problem
is classified
by the string
XXXr-XX-n-m. ACM
TransactIons
on Mathematical
Software,
Vol. 21, No, 1, March
1995.
Constrained This
string
admissible character
must
not
contain
letters
and
integers,
in the
uppercase. The first function.
and Unconstrained
character
no objective
C
the objective
string.
in the
string
values
function
blanks.
together
classification
Its possible
N
any
Testing Environment In
with
Note
that
defines
the objective
function
is linear;
function
is quadratic;
the type
S
the objective
function
is a sum of squares;
O
the objective
function
is none
U
character
Its possible
the problem
in the
values
string
defines
problem
only
constraints
are fixed
only
constraints
are bounds
N
the problem’s network;
constraints
represent
L
the problem’s
constraints
are linear;
Q
the problem’s
constraints
are quadratic;
O
the problem’s
constraints
are more
The third character There are two choices
in the string
R
the problem is regular, that are continuous everywhere;
I
the problem
type
the
on the variables; adjacency
matrix
of a (linear)
and
general
indicates
than
any of the above alone.
the smoothness
is, its first or
and second
of the problem.
derivatives
(r) which
corresponds
to the
fourth
interest
of the problem.
Its possible
is academic, that is, has been to test one or more algorithms;
M
the problem
is part
The
character
of a modeling
exercise
is not used in a genuine
practical
values
of the
and
where
character contains
in
the
explicit ACM
specifically
the actual
application; used
value
in a real
string
indicates
whether
or not
variables
(see the
description
on Mathematical
Software,
is
by of the
and
internal Transactions
string
the problem O, 1, or 2. the primary
are
constructed
the problem’s solution is (or has been) actually tion for purposes other than testing algorithms. next
exist
is irregular.
the problem researchers
description
of the
variables;
A
R
objective
of constraints
the degree of the highest derivatives provided analytically within description. It is restricted to being one of the single characters The character immediately following the first hyphen indicates
solution
in
is unconstrained;
the problem’s
and/or
be given
and
the
the problem’s
origin
each
are
B
integer
must
of the
the
for
of the above.
X
The
all letters
state
is constant;
the objective
second
we
is defined;
function
L
The
follows, interpretation,
are
Q
problem.
what their
129
.
the
applica-
problem of group
Vol. 21, No. 1, March
1995.
130
1, Bongatlzet
.
partial
separability
There
al, Corm
in
are two possible
et al. [1990]
or Corm
Y
the problem
description
contains
N
the problem
description
does not contain
The symbol(s) variables
V
the
n
a positive
The
number
than
integer
not
the number
the
and third
in the
integer
Sect. 8.2.2.2]).
hyphen
hyphen
or
internal
indicate
variables.
the number
can be chosen
(fixed)
number
indicate
bounds)
in
actual
number
of
The
can be chosen (fixed)
or
variables.
Note
here.
user;
of constraints
problem.
constraints
the
by the
of problem
the
the
in the problem giving
variables;
are
as general
of constraints
internal
any explicit
problem
the actual
and
considered
explicit
values
third
variables
nonnegative
a
Possible
giving
after
fixed
variables are values are
m
the second
of variables
symbol(s)
(other
V
between
in the problem.
et al. [1992a,
namely,
values,
that two
fixed
possible
by the user;
or
number
of problem
the scope of the classification
to include,
constraints. In the future,
we intend
to expand
among other things, a more thorough subdivision of constraint information, structural considerations, and solution characteristics. We have refrained from
extending
to benefit
the classification
from
feedback
such extensions
be compatible
2.3 The Select The purpose problem
list
system CUTE
at the present
users.
with
time
It is, of course,
the present
because essential
we wish that
any
system.
Tool
of this
classifications.
contains all the The interrogation Thus, obtain
from
tool
is to interrogate
a file
In the distributed
current classification includes a facility
version
containing
details
of CUTE,
on a list
of
the file CLASS, DB
details for the test problem database. for choosing the file that is to be probed.
one is able to maintain and a list of problems that match
query several different .DB files and to interactively defined characteristics. The
can be saved in a designated file. One might imagine having several
different
problem
data
sets in several
different directories. Suppose, for example, one had two directories called academic and applications. One could then maintain a .DB file in each directory corresponding to the SIF files they contained. For each, there is then the possibility of setting up an interactive dialogue at the terminal to determine such data as, for example: “Which problems are constrained and have only linear constraints,” or “Which problems have more than 1000 constraints and have explicit second derivatives?” By default, the select tool assumes the .DB file resides in $MASTSIF / ClASSF. DB, but the user may give a full pathname for the classification file and thereby probe any file in any directory. The select tool prompts the user to specify the problem characteristics of interest, and then lists all problems in the classification file which satisfy ACM
these
Transactions
characteristics. on Mathematical
If the Software,
save
facility
is used,
Vol. 21, No. 1, March
1995
the
listing
will
be
Constrained saved
under
written
the file
name
in the current
pathname
and thus
and Unconstrained
specified
working write
Testing Environment
by the user.
directory.
the listing
By default,
The user
file
may,
131
.
the listing
however,
file
give
is
a full
in any directory.
The dialogue with the user is on the standard input\ output. Additional information about the select tool is given in the document file install. rdm contained
$CUTEDIR / doc (see Section
in the directory
7).
Note that the upper bound on the number of variables is 99,999,999, and one is not allowed more than 99,999,999 constraints. The output produced when running this program is given in Appendix C.
is meant
3. ADDING
NEW PROBLEMS
Large-scale
optimization
of information number
a format
a Fortran would
Our
including
that
compiler
(fortunately)
structure.
of objectives,
wanted that
problems
in their
would
that
to be self-explanatory.
be available with
the
typically format
exploiting
was capable
be compatible
input
that
contain
to anyone format
using
In
of IBM
deal
to satisfy
particular,
a machine
the problem
session
a great
was designed
structure.
of exploiting MPS
A typical
that
has
structure
[1978].
a we
and
When
users
wish to add new problems, they will first need to write the appropriate input files. It is not our intention to provide a manual for writing such files here. Details of the required syntax and a primer for beginners are included in the book describing
the LANCELOT
software
package
(Corm
et al. [1992a,
chaps.
7 and 2]). Although this article does not describe how to write problems in SIF, Section 3.1 outlines how the problem structure is exploited, and Appendix A gives
a small
classification
illustrative string
should occur in the must be of the form * classification
example
in
this
SIF
file
of a SIF
example. before
the
In
file.
general,
The
user
the
might
notice
classification
GROUPS or VARIABLES
the
string
section
and
XXXr-XX-n-m
or * CLASSIFICATION
XXXr-XX-n-m
where before entire
the first character must be a *; any number of blanks can appear or after the key word classification or CLASSIFICATION as long as the string is no more than 80 characters; and the characters in the string XXXr-XX-n-m must obey the classification scheme defined in Section 2.2.
3.1 Decoding
the Problem
Input File
A module (called the “SIF decoder”) reads the description of the problems in the standard input format. A structure called group partial separability, which we believe has many significant advantages, is accounted for in the input format. ACM Transactions on Mathematical Software, Vol. 21,
No, 1, March
1995.
132
1. Bongartz et al.
.
A function
f(x)
(1) the function
is said to be group can be expressed
partially
separable
if
in the form n&-
f-(x) = ~ where
n~ is the number
(2) each of the function
group
of groups;
functions
of the single
(1)
g,(a!,(x)),
g,(a)
variable
is a twice
continuously
differentiable
a;
(3) the function
is known
as the
i th group;
(4) each of the index sets ~ is a subset of {1, . . . , n,}, where of different nonlinear element functions; element (5) each of the nonlinear tiable function of a subset assumed by
to have comprising
xIJI
a large
(7) the This
Wi ~ are known
structure
sparse
matrix
of the
continuously
variables
subspace.
x.
Usually,
of the variables linear
element
Each this
differenfunction
is
is manifested
x;
functions
a~x – bi is, in
weights.
general.
may
~ is a twice
of the
fraction
as element
is extremely
Hessian
xIJI
invariant
a small
a, of each (6) the gradient general, sparse; and
functions
n, is the number
Indeed,
be written
any function
in this
form
with
a continuous
(see Griewank
and Toint
[1982]). A more thorough introduction by Corm et al. [1990]. The SIF decoder
for group partial separability is given assumes that the objective and general
constraint
with
functions
are of this
form,
the
proviso
that
each
constraint
uses only a single group, i.e., n~ = 1 for the constraints. Thus, for each problem in the SIF test set there is a file, named, for EXAM PLE.S IF. The SIF decoder interprets the statements found in example, the file below:
and
—ELFUNS.f
produces
several
contains
a Fortran
functions corresponding in the problem, as well
Fortran
subroutines
and
a data
subroutine that evaluates to the nonlinear element function as their derivatives;
file,
as given
the numerical types occurring
—GROUPS.f
contains a Fortran subroutine that evaluates the numerical functions corresponding to the group function types occurring in the problem, as well as their derivatives;
—RANGES.f contains a Fortran subroutine that computes the transformations from elemental to internal variables (see Corm el al. [1992a] or Corm et al. [19901) for all element types that use a nontrivial transformation;
—SETTYP.f nonlinear
contains elements
a Fortran
subroutine for the problem;
that
assigns
the correct
ACM Transactions on Mathematical Software, Vol. 21, No, 1, March 1995
type to the
Constrained
—EXTERN.f
(if present)
contains
at the end of the problem
—OUTSDIF,d names, These
contains
data
point,
are
subsequently
the LANCELOT
user-supplied
Fortran
133
.
functions
found
structure
(variable
and constraint
and the like).
used
codes (see Sections
Testing Environment
and
on the problem
starting
files
any
SIF file;
scalings,
optimization
and Unconstrained
by
the
tools
that
interface
5 and 6), and of course
they
with
other
are required
by
optimizer.
4. MANAGEMENT
OF CLASSIFICATION
Once the problems
is described This raises
in SIF, the next step is its inclusion in the the issue of database management. We now
classify
classall
problem database.
describe
the
purpose
tools
of these
and
tools is to create,
the classification
database
that
the user is on the standard classification Suppose (assuming
required
maintain,
the select
input/output.
for
this
management;
the
.DB files that contain
and update tool interrogates.
The dialogue
We now illustrate
the usage
with of the
commands. the
directory
pointed
one is in a Unix
BRYBND.SIF CLUSTER.SIF CORE1 .SIF CORE2.SIF COSHFUN.SIF which
DATABASES
all contain
to by
environment)
EQC.SIF FLETCHBV.SIF GENROSE.SIF HS25.SIF HS35.SIF suitable
the
environment
contains
lines.
MASTS IF
the SIF files
LEAKNET.SIF MINMAXBD.SIF MOREBV.SIF NETI .SIF NET2.SIF
classification
variable
NET3.SIF QC.SIF S268.SIF S368.SIF STANCMIN.SIF
Issuing
the command
classify LEAKNET (note that the the screen:
file
name
is case sensitive)
results
in the following
output
at
Your current classification file is : CLASSF.DB Your current SIF filename file is : LEAKNET.SIF If the file CI-ASSF, DB in the directory
pointed
MASTSIF does not already
it is created
exist,
then
to by the environment and contains
variable
the following
line; LEAKNET LOR2-RN- 156-153 Otherwise, classification If LEAKNET
LEAKNET, the if CLASSF.DB exists, but no entry for problem line is inserted in the file in its correct ASCII ordered position. already appears in ClASSF. DB, the users are asked if they wish
to overwrite the old entry, if it differs from the new one. To refer to a directory other than the directory indicated must issue the command
by $MASTSIF,
one
Vol. 21, No. 1, March
1995.
classify classdir probname, ACM
Transactions
on Mathematical
Software,
134
.
1. Bongartzet
al.
probname is the problem name with or without the S1F appended, and classdir is the name of the directory where this problem resides. In this case, CLASSF. DB is created in the directory classdir. Note that, when the classdir argument is not given, classify assumes that the SIF file probname.SIF is in It is recommended that the user not assign file the $MASTSIF directory. where
names with multiple .SIF endings, Finally, issuing the command
such
as probname,SIF.SIF.
classall
ClASSF. DB in the appropriate directory (pointed to by the MASTS IF in a Unix environment), together with screen messages similar to that produced by classify, for each SIF file processed. The process is completed by a listing of the . DB file on the screen. For our classall produces a ClASSF. DB file containing the following lines: example, produces
the file
environment
variable
BRYBND CLUSTER COREI CORE2 COSHFUN EQC FLETCHBV GENROSE HS25 HS35 LEAKNET MINMAXBD MOREBV NET1 NET2 NET3 QC S268 S368 STANCMIN To override mand
suR2-ANNOR2-ANLQ12-RNLQ12-RNLoR2-ANouR2-MYOUR2-ANSUR2-ANSUR2-ANQLR2-ANLoR2-RNLoR2-ANSUR2-MYoo12-RNoo12-RNoo12-RNOUR2-MYQLR2-ANOBR2-MYoL12-AYthe
environment
v265157v:: v33156:: 671815919:: 3-
V 2 59
134 v o v v o 1 153 20 v 57 160 521 0 5 v 2
variable
MASTSIF,
one must
issue
the
com-
classall classdir where classdk is the name of the directory classified. In this case, ClASSF. DB is created
5. THE INTERFACE
WITH EXISTING
So far, we have described
what
containing the in the directory
OPTIMIZATION
problems
exist
SIF
files
to be
classdir.
PACKAGES
in SIF, how they
are classified,
and how to add to a supplied or new database. This section lists the optimization packages for which interfaces have been developed. The purpose of these interfaces is to deal with the situation, presumably not uncommon among researchers and practitioners alike, where one wants to run a given problem on a range of algorithms to assess which algorithm is likely to be the most suitable for solving classes of related problems. ACM Transactions on Mathematical Software, Vol. 21, No. 1, March 1995
Constrained
and Unconstrained
Testing Environment
135
.
5.1 Scope At the present tion
time,
we have
available
interfaces
with
the following
optimiza-
packages:
COBYLA
of Powell
[1994].
This
package
is a direct
search
method
for
inequality-constrained problems that models the objective and constraint function by linear interpolation and does not use derivatives. It is available from M. J. D. Powell, DAMTP, Cambridge University, Cambridge, UK (email address:
[email protected].
MINOS
of Murtagh
and
ac.uk).
Saunders
[1993].
MINOS
solves
problems
of the
form minimize
~(x)
+ CTX + dTy
xER”,y=R~ subject
to
f(x)
+Aly
= bl
(3)
Azx+A~y=bz
The
nonlinear
contributions
to the
constraints
are linearized
so that
linear
programming technology can be exploited. MINOS allows matrices to be stored in either dense or sparse format and is therefore suitable for large sparse problems. Details are given in Murtagh and Saunders [1978; 1993]. Currently we have interfaces for MINOS versions !5.3, 5.4, and 5.5. MINOS is distributed by the Office of Technology Licensing (OTL) at Stanford University
and
is subject
Stanford
to certain
University.
Michael Saunders dress: Department
license
Readers
agreements.
interested
(email address: of Operations
in
MINOS
more
is copyrighted
details
should
by
contact
mike@ sol-michael.stanford. edu; postal adResearch, Stanford University, Stanford,
CA 94305-4022). NPSOL functions
of Gill et al. [1986]. subject to constraints,
This package is designed to minimize smooth which may include simple bounds, linear
constraints, and smooth nonlinear constraints. The software tial quadratic programming algorithm, where bounds, linear nonlinear all
constraints
matrices
problems. Stanford
in dense
NPSOL University.
OSL of IBM
are treated
separately.
format
is therefore
and
is available
[1990].
This
from
package
the
TENMIN problems
of Schnabel where
and
Chow
the cost of storing ACM
Transactions
not
Office
obtains
ming problems where the Hessian matrix is intended to be suitable for large-scale International Business Machines, subject is copyrighted by IBM Corporation.
Unlike
MINOS, intended
of Technology
solutions
uses a sequenconstraints, and NPSOL
stores
for large
sparse
Licensing
to quadratic
program-
is assumed positive semidefinite. problems. OSL is distributed to certain license agreements,
[1991].
one n-by-n
This matrix
on Mathematical
package (where
Software,
at
is intended
It by and
for
n is the number
Vol. 21, No. 1, March
1995.
136
.
1. Bongartzet
al.
of variables) and of factoring it at each iteration is acceptable. The software allows the user to choose between a tensor method for unconstrained optimization and an analogous standard tensor method bases each iteration model form,
of the store,
objective
via anonymous distribs/tensor.
ftp Any
[email protected]. UNCMIN
in
is not
on a quadratic constructed
significantly
quadratic
model.
edu, in software
more
model. The fourth-order expensive
TENMIN
to
is available
the directory pub/es/ should be addressed to
edu, et al.
Dennis and minimization
region
or difference sians (from
that
the standard
from ftp.cs.colorado. questions about this
of Koontz
pseudocode unconstrained and trust
function
or solve than
method based on a specially
[1985].
UNCMIN
Schnabel and has
approaches.
[1983]. options
The provided
corresponds This that
options
closely
to the
package is designed for include both line search include
analytic
gradients
approximations with analytic Hessians, or finite-difference Hesanalytic or finite-difference gradients) or secant methods (BFGS).
VA15 of Liu and Nocedal [1989]. This package solves general nonlinear unconstrained problems using a limited-memory BFGS method. It is intended for large-scale problems. VE09 of Gould [1991]. This nonconvex quadratic programming is intended
to be suitable
VE14
of
Corm
et
al.
package obtains local solutions to general, problems, using an active set method, and
for large-scale [1994].
This
problems. package
solves
quadratic programming problems using a barrier function again intended to be suitable for large-scale problems. [1982]. This package solves general using a sequential quadratic programming
bound-constrained method
VF13 strained
of Powell problems
VA15, [Harwell
VE09, VE14, and VF13 are part of the Harwell Subroutine 1993]. They are distributed by the United Kingdom Atomic
and
nonlinearly technique.
is
con-
Library Energy
Authority, Harwell, subject to certain license agreements. They are all copyrighted jointly by the UKAEA and SERC (Science and Engineering Research Council). We have also included an interface which allows the CUTE evaluation tools, described in Section 6, to be called from MATLAB [1993]. For pointers to more information, see Section 7. Of course, LANCELOT also solves problems in SIF, but it does not require an interface using the CUTE tools. Note that LANCELOT exploits much more structure than that provided by the interface tools. The natural next question is: how does one use the supplied interfaces? We begin
5.2
with
Using
After ment, ACM
one of the simplest.
the UNCMIN
installation there will TransactIons
Interface
of CUTE (see Section 7), for example, in a Unix environbe an empty subdirectory named uncmin, It is the user’s
on Mathematical
Software,
Vol. 21, No. 1, March
1995
Constrained responsibility single
to ensure
object
UNCMIN
file,
that
in single
version.
The
uncmind. o in the uncmin the
user
will
comments For that
have
to edit
describing
illustrative
the UNCMIN
uncmins.o,
named
is available
double-precision
and Unconstrained
alias
precision
only,
the required
subdirectory.
the user object
To run
sdunc
file
may
into
a
(Although
wish
should
137
.
codes are compiled
uncmin
double-precision
to create be stored
this double-precision and uric. These scripts
a as
version, include
changes.)
Assume one is in a Unix environment. CUTEDIR and MASTSIF are set appropriately the directory $CUTEDIR / uncmin. Now suppose suppose
variables
and that uncmins.o is in further that one wishes to MOREBV.SIF
source
in the
subdirectory. the scripts
purposes,
the environment
Testing Environment
is in the
to the command
solve
the
problem
$MASTSIF.
directory
MOREBV,
The
user
should
whose either
SIF
file
set an
sdunc using
alias sdunc ‘$CUTEDIR / interfaces / sdunc’ this alias would be set automatically $CUTEDIR / interfaces to his or her path. The $MASTSIF and issue the command directory
(typically,
sdunc (The –s following
by the user’s shell) or add user should then enter the
–s MOREBV. option output:
is explained
in
Section
5.3.)
This
command
results
in
the
Problem name: MOREBV Single-precision
version will be formed.
The objective function There are #FCN EVAL GRAD EVAL HSN EVAL ITRMCD FINAL F FINAL NORMG xl x2 x3 x4 x5 X6 x7 X8 x9 xl o
= = = = = =
10 nonlinear
groups
13 13 2 1 6.5557D-14 1.3728D-07
x – 3.5088D-02 –6.5312D-02 – 8.9785D-02 – 1.0742D-01 – 1.1686D-01 – 1.1641 D-01 – 1.0390D-01 – 7.6520D-02 – 3.0551 D-02 3.9023D-02
Set up time= Solve time = Total time = Further package
uses
10 free variables
G – 1.5189D-08 – 4.8354D-09 2.4506D-08 9.2885D-08 – 1.4974D-09 7.4553D-08 – 9.9286D-09 – 4.0877D-08 1.3728D-07 1.2241 D-07
0.05 0.02 0.07 seconds
details for both are given in the
this interface document file
and
the
install.rdm
others provided with the contained in the directory
ACM Transactions on Mathematical Software, Vol. 21, No. 1, March 1995.
138
1. Bongartz et al.
.
Table
Package
III.
Currently
Decode
The
syntax
Sdmns
mns
NPSOL
s dnps
nps
OSL
sdosl
0s1
TENMIN
sdt
ten
UNCMIN
sdunc
unc
VA15
sdlmq
lmq
VE09
sdqp
W
VE14
sdbqp
bqp
VF13
sdcns
cns
Ian, which
(described
perform
in Section
These
the
5.3) as the
are distributed
as indicated
$CUTEDIR / problem.
en
5.3. A summary of the available packages and is given in Table III. LANCELOT uses the
commands
part of CUTE. A few SIF files,
in
are
same
as part
Table
sufficient
IV,
functions
commands
and
listed
have
the
in Table
III.
of LANCELOT,
are
provided
to verify
that
in the
and not as the
directory
packages
of the shell
Interface scripts
for the interfaces
listed
in Table
III
execute
the
general steps and have the same syntax. Thus, we now give details usage of these scripts by describing the sdgen and gen commands generic
are
installed.
5,3 The Generic All
only
MINOS
sdlan and Ian scripts
correctly
Optimize cob
sdlan and
commands
and optimize Sdcob
corresponding
same
Interfaces
COBYLA
$CUTEDIR / doc and in Section the
Available
interface.
The
characters
inns, uric, or any of the other Table III. Besides
offering
a convenient
same on the for a
gen in sdgen and gen may be replaced interface names given in the third column means
to describe
details
on the usage
by in
of the
important current interfaces, the sdgen and gen shell scripts serve another purpose: they provide a template for the shell scripts required for new interfaces with optimization packages other than those listed in Table III. Details on the construction of a new interface are given in the document file in the directory $CUTEDIR / dot. interface. rdm contained
5.3.1 The sdgen and gen Commands. Suppose the scripts sdgen residing in the directory $CUTEDI R / interfaces, provide an interface optimization package GEN (for GENeric). The object module(s) for package reside in the directory $CUTEDIR / gen. The single-precision gend. o. named gens.o, while the double-precision object is named ACM
TransactIons
on Mathematical
Software,
Vol. 21, No, 1, March
1995.
and gen, with the the GEN object is (Because
Constrained Table
IV.
and Unconstrained
Interface
Package
Testing Environment
and Appropriate
Sample
Appropriate
SIF file
COBYLA
HS1l
MINOS
HS1l
NPSOL
HS1l
OSL
HS21
TENMIN
ROSENBR
UNCMIN
ROSENBR
VA15
ROSENBR
VE09
HS21
VE14
HS3
VF13
HS1l
.
139
SIF File
some optimization packages consist of only one precision source code, the user may w-ish to modify this source code to create an appropriate precision version.) Besides the shell scripts, the interface also includes a Fortran driver
genma.f that interleaves calls to the optimization subroutines in GEN with calls to the appropriate CUTE evaluation tools (described in Section 6 and Appendix B). The Fortran source file genma.f resides in $CUTEDIR / tools / the compiled object modules reside in $CUTEDI R / tools / sources, while genma.o, for objects /single / genma.o, and $CUTEDIR / tools /objects/double/ the single-
and double-precision
The main (1) check them.
steps
versions,
by the sdgen
executed
respectively. command
are as follows:
the argument of the command for inconsistencies, and interpret Also, check that the problem specified has an associated SIF file.
(2) Apply the SIF decoder to the problem SIF file, in order to produce the OUTSDIF.d file and the problem-dependent Fortran subroutines. Stop the process (3) call
if any error
is uncovered
the gen command
The main
steps
(1) Check them.
the
executed
next),
in order
by the gen command
arguments
(2) If a new executable Fortran
(described
in the SIF file.
of the module
command is to be built,
to continue
the process.
are as follows:
for inconsistencies, compile
and
interpret
the problem-dependent
subroutines.
the compiled problem-dependent subroutines, the appropri(3) Load together ate evaluation tool objects, the Fortran driver object genma.o, and the on whether appropriate GEN object module (gens.o or gend .o, depending the singleor double-precision version is selected) to produce the executable file genmin. (4) Execute
genmin in order ACM
to solve the problem.
Transactions
on Mathematical
Software,
Vol
21, No. 1, March
1995.
140
.
1. Bongartzet
5.3.2
Syntax
command
is
and
al.
Options
for
sdgen
the
Command,
The
format
of the
command
argu-
sdgen [-s] [-h] [-k] [-o j] [-l sees] probname where ments
optional arguments have the following
-s
runs
are within meaning:
the single-precision
square
brackets.
version
(default:
The
run
the double-preci-
sion version); -h
prints
a simple
description
of the possible
options
for the sdgen
command; -k
does not delete the executable the module is deleted);
-o j
if j = O, the tool runs in silent mode, while brief descriptions of the stages executed are printed if j = 1 (default: silent mode);
-1 sees
an upper GEN in
limit the
99,999,999);
probname
5.3.3 Syntax mand is
and
after
execution
(default:
of sees seconds is set on the CPU time used by numerical solution of the problem (default: and
is the name description
module
(without
extension)
of the problem
Options
for
of the file
containing
the
SIF
to solve.
the gen Command.
The
format
of the com-
gen [-n] [-h] [-s1 [-k] [-o i] [-l secsl where
optional
ments
have
-n
arguments
the following
reconstructs
are within
the executable
the SIF decoding without
-h
runs
module
stage (default:
reconstructing
prints a simple command;
-s
square
brackets.
The
command
argu-
meaning:
genmin from
the files
the current
executable
run
output
at
module
it);
description
the single-precision
of the
version
possible
(default:
options
run
for
the
gen
the double-precision
version);
-k
does not delete module
the executable
module
after
execution
(default:
the
is deleted);
-o j
if j = O, the tool runs in silent mode, while brief descriptions stages executed are printed if j = 1 (default: silent mode);
-1 sees
an upper limit of sees seconds is set on the CPU time used by GEN in the numerical solution of the problem (default: 99,999,999).
5.3.4 Running GEN on a Problem be able to run sdgen and gen from lines
in Standard Input Format. In order to any directory, the user should add the
alias sdgen ‘$ CUTEDIR I interfaces I sdgen’ alias gen ‘$ CUTEDIR / interfaces / gen’ ACM
Transactions
on Mathematical
Software,
of the and
Vol. 21, No. 1, March
1995
Constrained
and Unconstrained
Testing Environment
141
.
the C shell under AIX, HP-UX, UNICOS, to his or her .cshrc file if running SunOS, or Ultrix systems. Alternatively, the user could add $CUTEDIR / interfaces to his or her PATH. To run GEN on a problem in SIF, the user should move into the directory in which
the
following
command:
SIF
file
for the
probname.Sl
problem,
F, resides
and
issue
the
sdgen probname. This
command
will
decode the SIF file,
print
a short
summary
of the problem
characteristics, and attempt to solve the problem using GEN. If the SIF file has already been decoded, but the executable module has been deleted, the appropriate command is
genmin
gen -n whereas
it is
gen if (1) the
SIF
genmin still
file
has
exists.
the -k option
should
6. EVALUATION
already
been
decoded
(To keep the executable be used with
and
module
(2) the after
executable solving
module
the problem,
the sdgen or gen command.)
either
TOOLS
6.1 Overview We also provide the
data
have
reasons.
First,
general
than
some
a collection been the
data
sparsity
overhead
of tools
decoded
with
structure
respect
enable file.
one to manipulate
This
is important
is able to represent 3.1).
to the
Naturally,
complexity
structure this
of the
data for two that
capability
data.
The
once main
is more imposes
evaluation
the process; in other words, the tools take a group (that one might think of as a “scatter” operation)
and “gather” the information the second derivatives for
to produce the function the entire function. The
value, the gradient, second derivatives
or are
in three formats: as a dense matrix, as a sparse matrix whose are stored in “coordinate” form, and as a sparse matrix stored in
“finite-element” include
that
a SIF
(see Section
tools enable one to reverse partially separable format
available nonzeros
from
format
a routine
that
(e.g., see Duff forms
the
et al. [1986,
matrix
matrix, and we provide utilities to variables, and possibly its constraints. required
the
collection
of
vector
chap.
2]). Additionally,
product
with
the
we
Hessian
obtain the names of a problem, its Second, our testing of LANCELOT
a substantial
number
of
large
and
nontrivial
optimization problems (see Section 2.1). With the aid of the supplied tools, one can use the same input files to interface with other optimization software that is not designed to accept SIF (see Appendix B). As a corollary, we are hoping that practitioners will be encouraged to write test problems in the SIF. Of course, although it is not strictly speaking essential (see Corm et al. [1992a, chap. 8]), we would assume that most users of LANCELOT will write their
input
using
the SIF. ACM
Transactions
on Mathematical
Software,
Vol. 21, No. 1, March
1995.
142
.
1. Bongartz et al
Several
parameters
particular error
have
problem
message
values
or system
is produced
that
may be changed
configuration,
telling
which
by the user to suit
If any of these parameter(s)
a
are too small,
to increase.
an
Typically,
they concern the size of the workspace for what we consider the default values for big, medium, and small problems. Details on how to change the in the direcparameters are given in the document file instal 1.rdm contained tory $CUTEDIR / dot. The next two sections indicate the problem classes for which tools are available
for users
to manipulate
6.2 The Unconstrained We provide
tools
the decoded
and Bound-Constrained
that
specifically
relate
optimize subject
to the simple
x ● R“,
is group
and
partially
detailed
provide
tools
to the general
inequality
and the simple
function
from
and
is given
their
R’
purposes
in Appendix
into
that
is given
specifically
relate
to
the
general
constrained
~(x),
(6)
= o,
i~E,
(7)
constraints
bounds (i=l
XL5UL
with
the above
problem
L(X2 where c(x) is the vector the vector A are known TransactIons
in
B,
,.. ., n).
(9)
Here x ~ R’, ~, c, are assumed to be smooth functions from Rn into are group partially separable. Furthermore, 1 U E = 1, 2, . . . . m, with E disjoint.
ACM
then
equations
1,
