Benchmarking nonlinear optimization software in technical computing environments
János D. Pintér & Frank J. Kampas
TOP An Official Journal of the Spanish Society of Statistics and Operations Research ISSN 1134-5764 TOP DOI 10.1007/s11750-011-0209-5
1 23
Your article is protected by copyright and all rights are held exclusively by Sociedad de Estadística e Investigación Operativa. This e-offprint is for personal use only and shall not be self-archived in electronic repositories. If you wish to self-archive your work, please use the accepted author’s version for posting to your own website or your institution’s repository. You may further deposit the accepted author’s version on a funder’s repository at a funder’s request, provided it is not made publicly available until 12 months after publication.
1 23
Author's personal copy Top DOI 10.1007/s11750-011-0209-5 O R I G I N A L PA P E R
Benchmarking nonlinear optimization software in technical computing environments Global optimization in Mathematica with MathOptimizer Professional János D. Pintér · Frank J. Kampas
Received: 7 February 2011 / Accepted: 27 June 2011 © Sociedad de Estadística e Investigación Operativa 2011
Abstract Our strategic objective is to develop a broadly categorized, expandable collection of test problems, to support the benchmarking of nonlinear optimization software packages in integrated technical computing environments (ITCEs). ITCEs— such as Maple, Mathematica, and MATLAB—support concise, modular and scalable model development: their built-in documentation and visualization features can be put to good use also in test model selection and analysis. ITCEs support the flexible inclusion of both new models and general-purpose solver engines for future studies. Within this broad context, in this article we review a collection of global optimization problems coded in Mathematica, and present illustrative and summarized numerical results obtained using the MathOptimizer Professional software package. Keywords Nonlinear optimization in integrated technical computing environments · Optimization software benchmarking · Model library in Mathematica · Lipschitz Global Optimizer (LGO) solver suite for nonlinear optimization · MathOptimizer Professional (LGO linked to Mathematica) · Numerical performance results Mathematics Subject Classification (2000) 65K30 · 90C05 · 90C31
J.D. Pintér () Department of Industrial Engineering, Özye˘gin University, Istanbul, Turkey e-mail:
[email protected] J.D. Pintér PCS Inc., Halifax, NS, Canada e-mail:
[email protected] F.J. Kampas Physicist at Large Consulting, Ambler, PA, USA e-mail:
[email protected]
Author's personal copy J.D. Pintér, F.J. Kampas
1 Introduction Nonlinear optimization (NLO) problems have been of practical interest for centuries, and they have been systematically studied since the beginnings of modern operations research. We shall consider the general NLO model in the following concise form: min f (x)
x ∈ D := x : l ≤ x ≤ u, g(x) ≤ 0 .
(1) (2)
In (1)–(2) x ∈ R n is the decision vector to be optimized; l ∈ R n and u ∈ R n denote the (component-wise) finite n-vector bounds of x; f : R n → R is the objective function; g : R n → R m is the m-vector of constraint functions (hence, the zero in (2) is also an m-vector). The set of constraints g can be empty, thereby leading to a box-constrained model. Formally more general NLO models that include also = and ≥ constraint relations and/or explicit lower bounds on the constraint function values can be simply reduced to the model form (1)–(2). Without going into details, let us also note that e.g. all combinatorial optimization models defined with finitely bounded integer decision variables are also formally subsumed by the GO model. We assume that the feasible set D ⊂ R n is non-empty, and that the functions f and g (the latter component-wise) are continuous, without further postulated structural requirements such as model convexity or smoothness properties. Under the stated “minimal” assumptions we shall refer to (1)–(2) as the continuous global optimization (CGO) model, or simply as the GO model. By the classical theorem of Weierstrass, the GO model has a non-empty global solution set X ∗ : each point x ∗ ∈ X ∗ is a globally optimal solution, and z∗ = f (x ∗ ) is the global optimum value. The objective of global optimization is the development of theory and methods, in order to find the best solution(s) of GO models, in terms of finding the entire set X ∗ , or finding a single global solution x ∗ ∈ X ∗ , together with the value z∗ . Let us point out that most non-trivial instances of the GO model cannot be handled solely by analytical (symbolic computing) techniques. Some of the key reasons for this are the possibly massive multimodality of the objective function f ; the possibly complex structure of the set D; and the possible (theoretically exponential) increase of model complexity measured e.g. in terms of m and n. Partly due to the apparent limitations of analytical solution methods in the GO context, computational GO has become the subject of intensive studies, especially so in recent years. The remarkable progress in algorithm development and computer technology has led to increasingly more sophisticated nonlinear optimization software implementations. While the majority of GO software packages reviewed in the not-so-distant survey (Pintér 1996b) have been perhaps more “academic” than “professional,” more recently a growing array of professionally developed and maintained GO solvers has become available. This progress also motivates the need for optimization software performance analysis and comparative assessment. Benchmarking studies provide an objective approach and guidelines to evaluate the quality of optimization (or other) software products, by using these to handle “standard” or new test problems in a realistic, fair and completely reproducible manner. Our own strategic level objective is to develop and maintain a properly classified library of test models to benchmark GO software packages in integrated technical
Author's personal copy Benchmarking in technical computing environments
computing environments such as Maple (Maplesoft 2011a), Mathematica (Wolfram Research 2011), and MATLAB (The MathWorks 2011a). ITCEs offer model development transparency and scalability, and have built-in documentation and visualization features: they also support the direct addition of both new models and solvers to existing studies. Within this general context, in this article we review a selection of global optimization problems and present illustrative numerical results obtained using Mathematica as the model development and test environment, and MathOptimizer Professional (MOP) as a nonlinear optimization package. MOP is based on the external Lipschitz Global Optimizer (LGO) solver suite linked to Mathematica. Mathematica is extensively documented in hundreds of books. The comprehensive standard reference is Wolfram (2003); Mathematica’s many features are discussed in great detail also by Trott (2004, 2006). The entire standard documentation is also available through Mathematica’s hierarchically organized Help system. All information is presented in the form of concise definitions and ready-to-use basic examples; advanced examples, more extended tutorials and demonstration projects are also available. Our solver benchmarking environment is an extensive and easily expandable Mathematica document (called notebook) that in principle can be executed by a single command if required. Our model collection currently includes the most well-known simple global optimization tests, a selection of more difficult challenges, some potentially very challenging scalable problem-classes, and a range of examples from realworld applications. Due to the size of our current benchmarking test suite—several hundreds of models, even without the scalable “infinite” size test batteries—it would be tedious to include here all results in full detail. Therefore we present only illustrative results and add brief summaries regarding several model categories. We also highlight some key points that are thought of relevance also in a broader benchmarking context. The paper is organized as follows. Section 2 provides a concise review of the topical benchmarking literature. Section 3 introduces the key features of LGO and the recently revised MathOptimizer Professional package version used in this study. Sections 4 to 8 present illustrative results from our classified collection of test problems, starting from simple standard models, followed by more difficult ones including some tough challenges. Since some of the models presented are directly scalable to arbitrary dimension, we will consider only relatively small, but non-trivial model instances of such model prototypes. The conclusions (Sect. 9) are followed by a fairly extensive list of topical references. In the study presented here, mostly an average (several years old) Toshiba laptop computer has been used with the following key technical characteristics: 32-bit architecture, Intel™ Core™ 2 Duo CPU T5450 @ 1.66 GHz processor, 2.00 GB RAM, running under the Microsoft Windows Vista (Service Pack 2) operating system. Mathematica (Wolfram Research 2011) is the model development and test environment; the Lahey/Fujitsu FORTRAN 95 compiler (LF95) by Lahey Computer Systems (2004) is used for code generation (this aspect will be discussed later on). Most models presented here are solved in a fraction of a second on this PC (or on similar/better ones), unless explicitly stated otherwise. Absolute (total solution) timings will be reported when deemed necessary, including communication between Mathematica and the external LGO solver engine. In order to enhance the discussion, we
Author's personal copy J.D. Pintér, F.J. Kampas
also include the visual representation of some interesting models. The visualization features of ITCEs can assist not only to perceive the difficulty (or simplicity) of GO test problems, but they can also help to formulate and to readily test new GO models.
2 A brief literature review In recent decades, a considerable effort has been devoted to the development of nonlinear optimization test problems, and to the systematic presentation of numerical test results. It is next to impossible to aim at a complete review of the related research: therefore we will only highlight a selection of general and more specific information sources. We will primarily focus on work related to GO tests, but we shall also mention several benchmarking studies related to local (or more general, but not directly focused on global) nonlinear optimization. Comprehensive guidelines and methodology related to the subject of benchmarking optimization software and reporting results are discussed e.g. by Jackson et al. (1991), Barr et al. (1995), Dolan and Moré (2002), Moré and Wild (2009), Hare and Wang (2010). The websites of Mittelmann (2011), Neumaier (2011), GAMS Global World (2011) and GAMS Performance World (2011) present further in-depth information with many additional references and links. For collections of NLO test problems, consult, e.g., Moré et al. (1981), Hock and Schittkowski (1981, 1983), Schittkowski (1987, 2008), Averick and Moré (1991). Luksan and Vlçek (2000) present a specialized collection of non-smooth optimization test problems. The models in the studies listed above are all relatively small-scale, and they can be unconstrained, bound-constrained or linearly constrained, and convex or non-convex. The COPS model library (Dolan et al. 2004) includes a selection of medium- to large-scale (general) constrained optimization problems inspired by a range of real-world applications. The studies of Buckley (1992), Bongartz et al. (1995) and Gould et al. (2003) are focused on the development of environments to test NLO software. Specific aspects and features of such environments include a problem definition database, an optional problem decoder, the interface between the testing software and the optimization packages tested, and automatically generated result summaries and statistics. The collection of test problems associated with the Bongartz et al. (1995) and Gould et al. (2003) studies is the CUTE/CUTEr suite: many of these models are largescale and constrained. The Hock–Schittkowski problems and the CUTE problems are also part of the nonlinear optimization model library collected by Vanderbei (2011), with additional problems motivated by practical applications. Obviously, all the NLO test model libraries referred to above can be put to good use also in the context of benchmarking GO software. The benchmarking literature specifically focused on GO—including here also some of the derivative-free optimization benchmarking studies—is also substantial. For discussions of methodology, test problems and numerical results, consult e.g. Dixon and Szegö (1975, 1978), Hansen et al. (1992), Schoen (1993), Floudas et al. (1999), Liu and Zhang (2000), Pintér (2002, 2006a, 2007), Pintér et al. (2003), Casado et al. (2003), Gaviano et al. (2003), Shcherbina et al. (2003), Schichl (2003),
Author's personal copy Benchmarking in technical computing environments
Doye et al. (2004), Ali et al. (2005), Khompatraporn et al. (2005), Neumaier et al. (2005), Lavor et al. (2006), Addis and Locatelli (2007), Csendes et al. (2008), Pál and Csendes (2009), Rios and Sahinidis (2010), Tang et al. (2010). The last document cited in this long list of references is just one example of benchmarking studies conducted in the vast area of heuristic global optimization: for more details and great many further references on heuristic GO strategies, consult e.g. Michalewicz (1996) or Weise (2011). For a more complete picture, we mention also at least several GO books that include detailed numerical tests and discuss applications arising from various areas of engineering and the sciences. Examples of such volumes are Grossmann (1996), Pintér (1996a), Papalambros and Wilde (2000), Tawarmalani and Sahinidis (2002), Zabinsky (2003), and Pintér (2006b). Without adding other long lists of further references, here we just mention the significant research and computational studies related exclusively to rigorous global optimization, and to the area of mixed integercontinuous nonlinear programming. In addition to freely available research codes and other tools (that are often associated with some of the studies referred to above), a number of software companies offer professionally developed and maintained nonlinear optimization solvers and related test model libraries. Here we refer to the model libraries made available by the developers and contributors through the websites of AMPL LLC (2011), the GAMS Development Corporation (2011), LINDO Systems (2011), Maplesoft (2011a), The MathWorks (2011a), Maximal Software (2011), Paragon Decision Technology (2011), TOMLAB (2011), and Wolfram Research (2011). Although well-established model libraries are readily available (in coded form) in FORTRAN, C/C++, in optimization modeling environments such as AIMMS, AMPL, GAMS, LINGO, MPL, there is no published relatively comprehensive nonlinear optimization model collection for the Mathematica environment. In this study, we introduce such a model collection and present illustrative examples with summary notes regarding solver performance on several categories of test problems. Our current model library is partly based on some of the references and model libraries mentioned above, adding to these models from our own test model collection. It is obvious—and perhaps excusable—that we cannot include “every possible” test model covered by the topical benchmarking literature. However, we attempt to cover a rather broad range of GO tests and applications. Our categorized model library is directly expandable, and we plan to add to it further model types and classes as time and resources allow.
3 MathOptimizer Professional 3.1 The LGO solver suite The core of the MathOptimizer Professional (MOP) software package is the LGO solver suite that is linked to the Mathematica platform. LGO solves nonlinear optimization models numerically, using an integrated suite of global and local scope algorithms. LGO has been developed and maintained since the late 1980s, and it has
Author's personal copy J.D. Pintér, F.J. Kampas
been discussed in detail elsewhere: consult, for instance, Pintér (1996a, 1997, 2002). Currently, LGO is available as a stand-alone solver engine across all major hardware platforms and operating systems, for use with C/C++/C# and FORTRAN compilers. LGO is also implemented with links to prominent modeling environments such as Excel, optimization modeling languages (AIMMS, AMPL, GAMS, and MPL), and integrated computing systems (Maple, Mathematica, and MATLAB). Several of these implementations are discussed by Pintér (2002, 2005, 2006a, 2007, 2009a, 2009b), Pintér and Kampas (2005, 2006), Kampas and Pintér (2006), Pintér et al. (2006). We will briefly discuss LGO’s key features while introducing MOP. 3.2 MathOptimizer Professional The MOP software package offers a combination of Mathematica’s advanced model development features with LGO functionality. This leads to a numerical performance that—in terms of solution quality and solver speed—is comparable to compiler-based or optimization modeling language related LGO implementations. These advantages can become significant in comparison to the available native Mathematica solvers, when the models to handle are difficult and/or large-scale. The key operations and functionality of MOP are summarized by the following steps: • Optimization model formulation and optional model analysis in a Mathematica (notebook) document • Automatic conversion of the optimization model into C or FORTRAN code form, generation of an external model function file • Generation of a solver input file that includes all static model information as well as several key LGO solver options • Compilation of the generated model code into a dynamic link library (DLL) • Call to the external LGO solver engine, provided as an executable program • Solution of the optimization model by LGO • Report generation by LGO • Import of LGO results into the calling Mathematica notebook We have made available MOP versions with connectivity to a range of C/C++/C# and FORTRAN compilers: this requires only minor changes in the MOP-LGO interface. As mentioned earlier, the Lahey/Fujitsu FORTRAN 95 compiler (LF95) is used for code generation in the study presented here. (It is also worth to mention that in our experience the usage of a high-quality FORTRAN compiler leads to enhanced numerical results and some other advantages over using C/C++/C# compilers.) 3.3 MathOptimizer Professional activation and options Evidently, this article cannot serve even as a cursory introduction to Mathematica or as a user’s manual to MOP itself. Instead, we will briefly explain on the spot the specific Mathematica and MOP related commands that will be used in the subsequent discussion. Following the standard notational conventions of Mathematica, we will use Courier New Bold fonts to display Mathematica input commands; Mathematica’s output will be typeset in standard (Times New Roman) fonts. We will apply
Author's personal copy Benchmarking in technical computing environments
sometimes a minor editing of Mathematica input and output lines, exclusively for better readability. Assuming that the recommended default MOP installation directory structure is followed, MathOptimizer Professional is invoked by the Mathematica statement shown below. Needs[”MathOptimizerPro`callLGO`”]
Here MathOptimizerPro is the name of the Mathematica application directory for MOP, and callLGO is a Mathematica package that (together with several auxiliary files not discussed here) supports the communication between Mathematica and LGO. The usage of callLGO is summarized by the following structure. callLGO[objective, constraints_List, variableswithbounds_List, options]
(3)
In accordance with the model formulation (1)–(2), in the callLGO argument list the following entries appear. objective is the objective function (denoted earlier as f (x)); constraints_List is the possibly empty list of general constraints g(x); variableswithbounds_List is the list of lists of the variables and their bounds
(x, l, u), with the optional inclusion of initial variable values; options are callLGO options, set explicitly to override their default settings.
To solve a given optimization model, callLGO is invoked with the corresponding argument list. After its execution, callLGO returns the numerically optimized value of the objective function, a list of Mathematica rules that assign the corresponding solution arguments, the maximum constraint violation, and the total number of model function evaluations. MathOptimizer Professional can be used in a flexible variety of operational modes, controlled by its option settings. These options can be queried by the command Options[callLGO] which returns the list of default option settings. For brevity, we will discuss here only several of these options as needed, illustrated by examples. We refer to Pintér and Kampas (2010b) for a detailed description of all MathOptimizer Professional features, with many illustrative examples. 3.4 Getting started To start with a very simple instance of the model (1)–(2), consider the NLO problem min x12 + x22 x ∈ D := −2 ≤ x1 ≤ 3, −3 ≤ x2 ≤ 2, x1 + x2 − 1 = 0, x12 + 3x2 ≤ 2 .
(4) (5)
If we also use the optionally given initial values x1 = 1, x2 = 2, then this (evidently convex) model can be solved by the command below that—as an instance of (3)—
Author's personal copy J.D. Pintér, F.J. Kampas
is shown in the format below.
result = callLGO x1 ˆ 2+x2 ˆ 2, {x1+x2-1==0, x1 ˆ 2+3*x2
