Understanding Systems of Linear Equations and Programming through lpSolve and R Language Kamakshaiah Musunuru
[email protected]
Contents 1 Introduction
2
2 lpSolve 2.1 What is lp solve? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2 2
3 Methods in lp solve 3.1 Basic model . . . . . . . . . . . . . . . 3.1.1 The matrix form of the model . 3.1.2 Bounds & Ranges . . . . . . . 3.2 Variables . . . . . . . . . . . . . . . . 3.2.1 Integer & binary variables . . . 3.2.2 Semi-continuous variables . . . 3.3 Special ordered sets . . . . . . . . . . 3.3.1 Absolute values on constraints 3.4 Sensivity . . . . . . . . . . . . . . . . . 3.4.1 Dualities . . . . . . . . . . . . . 3.5 Integer programming . . . . . . . . . . 3.6 Presolve . . . . . . . . . . . . . . . . . 3.7 Infeasible solutions . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
4 How to solve equations in R 4.1 What is R? . . . . . . . . . . . . . . . . . 4.2 How do we implement linear programming 4.3 How to install lpSolve? . . . . . . . . . . . 4.4 Example . . . . . . . . . . . . . . . . . . . 4.4.1 Integer programming . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
3 3 4 4 4 4 4 5 5 6 6 6 7 7
. . . . in R? . . . . . . . . . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
8 8 8 9 9 11
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
5 The other free or open source packages for LP problems
1
12
1
Introduction
R has good stamina to solve systems of linear equations and also rich in optimization algorithms. There are numerous alternatives to take care of problems of optimization, viz. optimx, Rcgmin, Rvmmin, ROI etc.. However, lpSolve, lpSolveAPI appears to be strait and simple to deal with systems of linear equations. These packages are not available as part of R-base, rather as open (or free) projects at their respective sites. This paper describes as how to solve LP problems by using these packages. 1 Linear programming (LP; also called linear optimization) is a method to achieve the best outcome (such as maximum profit or lowest cost) in a mathematical model whose requirements are represented by linear relationships. Linear programming is a special case of mathematical programming (mathematical optimization). More formally, linear programming is a technique for the optimization of a linear objective function, subject to linear equality and linear inequality constraints. Its feasible region is a convex polytope, which is a set defined as the intersection of finitely many half spaces, each of which is defined by a linear inequality. Its objective function is a real-valued affine function defined on this polyhedron. A linear programming algorithm finds a point in the polyhedron where this function has the smallest (or largest) value if such a point exists. [1]
2
lpSolve The following information was taken from lp solve project page at sourceforge.net. Sourceforce.net is a community repository renowned for free and open source projects and software. Right now (at the time of this manual) there are 430, 000 projects; hosting over 3.7 million registered users. Serving 41.8 million customers with all of these open source projects with more than 4, 800, 000 downloads a day.[2]
2.1
What is lp solve?
What is lp solve and what is it not? The simple answer is, lp solve is a Mixed Integer Linear Programming (MILP) solver. lp solve is a free linear (integer) programming solver based on the revised simplex method and the Branch-and-bound method for the integers. It contains full source, examples and manuals. lp solve solves pure linear, (mixed) integer/binary, semi-continuous and special ordered sets (SOS) models. Note the word linear. This means that equations must be of the first order. 5 ∗ x − 3 ∗ y is an example. However x ∗ y is not linear and cannot be handled by lp solve. Both the objective function and the constraints have this restriction. Via the Branch-and-bound algorithm, it can handle integer variables, semi-continuous variables and Special Ordered Sets(SOS). lp solve has no limit on model size and accepts standard both lp or mps input files, but even that can be extended. Note however that some models could give lp solve a hard time and will 1 There are other proprietary solutions. Namely, IBM-Decision Optimization a sibling to SPSS. SAS has optimization algorithms through OR module. But the pricing mechanisms are excruciating. SAS Analytics Pro cost around Rs. 5, 30, 700 for an individual in India. You may read the pangs of acquiring SAS from here
2
even fail to solve. The larger the model the likely the chance for that. But even commercial solvers have problems with that. It can also be called as a library from different languages like C, VB, .NET, Delphi, Excel, Java, ... It can also be called from AMPL, MATLAB, O-Matrix, Scilab, Octave, R via a driver program. lp solve is written in ANSI C and can be compiled on many different platforms like linux and WINDOWS. lp solve has its own community via the Yahoo group http://groups.yahoo.com/group/lp_ solve/. There you can find the latest sources, executables for the common platforms, examples, manuals and a message board where people can share their thoughts on lp solve. lp solve was originally developed by Michel Berkelaar at Eindhoven University of Technology. The work of Jeroen Dirks made the transition from the basic version 1.5 to the full version 2.0 possible. He contributed the procedural interface, a built-in MPS reader, and many fixes and enhancements to the code. At that point there was also a Java port of lp solve 2.0. This was not a Java native interface (JNI) to the C library, but rather a translation of the algorithm (ver 2.0) from C into Java. It was also very limited This meant that it did not keep up with lp solve as it evolved. Starting at version 3.0, lp solve is released under the LGPL license. Before the code could only be used for non-commercial purposes. Many other people also contributed to lp solve, but there was no track of them. Sorry if you are not mentioned. Development was stagnated for some time at version 3.2, but now it is again alive and well via the new developers Kjell Eikland and Peter Notebaert. But also other people help to improve the product. For example the new Java interface to lp solve was made by Juergen Ebert. It is a JNI interface that supports the full functionality of lp solve. He did a great job. We encourage other people to participate in the development of lp solve. First a version 4 was introduced that had already several enhancements and improvements and now with version 5 this enhancement continues resulting is faster solving times, more stability and able to solver larger models and new functionality. See Changes from version 4 to version 5.1 for the changes done to version 5 and Changes from version 5.1 to version 5.5 for the changes done to version 5.5. Basically, lp solve is a library, a set of routines, called the API that can be called from almost any programming language to solve MILP problems. There are several ways to pass the data to the library. For instance: • Via the API • Via input files • Via an IDE [3]
3
Methods in lp solve
3.1
Basic model
• A linear function to be maximized or minimized. e.g. max:c1 x1 + c2 x2 (linear objective function) 3
• Problem constraints. e.g.
a_11x_1+a_12x_2; a_21x_1+a_22x_2; a_31x_1+a_32x_2 • Bounds (default lower bounds of zero on variables)) 3.1.1
The matrix form of the model
max: cT x subject to : Ax = 0 (implicit lower bound) 3.1.2
Bounds & Ranges
• Bounds (lower bound & upper bound) – implicit (lower) bound – explicit bound • Ranges – Restrictions (less than/more than) – equalities are more convenient compared to inequalities.
3.2 3.2.1
Variables Integer & binary variables
• By default all variables are real sometimes it requires one/more variables must be integers. • Problems with integer variables are called integer/discrete programming problems (IP). • If all variables are integers it is called a pure integer programming problem, else Mixed Integer Programming Problems (MIP). • The disadvantages with integer programming: – Reduced processing speed. – Inaccurate sensitivity analysis. 3.2.2
Semi-continuous variables
• The variables that must take values between crtain minimum & maximum limits. 4
3.3
Special ordered sets
• A special order set of degree N is a collection of variables where at most N variables may be non-zero. • The non-zero variables must be contiguous (neighbors) sorted by the ascending value of their respective unique neighbors. • lps olvemethodologyunderSOSengenderthreeequations, viz. • function rows • reference rows • convexity rows e.g. min: -x1 − x2 − 3.x3 − 2x4 − 2.x5 c1 : −x1 − x2 − x3 − x4 rhs = c(9,15) 9
Now we finished the basic requirements to compute lp model. The following are statements which we need to execute to set the lp model. # for solution # we compute the solution and parse it to an object with a name ’my.lp’ > my.lp = lp("max", obj, con, rel, rhs) # now the solution is ready, we might get the solution with the below command > my.lp$solution # for rest of the attribute we might execute the following command > names(my.lp) [1] "direction" "x.count" "objective" [4] "const.count" "constraints" "int.count" [7] "int.vec" "bin.count" "binary.vec" [10] "num.bin.solns" "objval" "solution" [13] "presolve" "compute.sens" "sens.coef.from" [16] "sens.coef.to" "duals" "duals.from" [19] "duals.to" "scale" "use.dense" [22] "dense.col" "dense.val" "dense.const.nrow" [25] "dense.ctr" "use.rw" "tmp" [28] "status" From the above list of attributes most important attributes are objval and solution. We can check our values with the help of following commands. > my.lp$objval [1] 40.5 > my.lp$solution [1] 0.0 4.5 0.0 From the above output it is clear that our objective function has a value 40.5 for variables x1 = 0; x2 = 4.5, x3 = 0. Very nice solution. This solution is of type SOS1. We can achieve the objective (the value mentioned before) with only one variable, it shows that rest of the variables are redundant. As discussed earlier in lp solve methodology. We may enquire further by exercising “sensitivities” and “dualities” in order to know the nature of variables and constants and their influence on value of the objective function. > my.lp.sense = lp("max", obj, con, rel, rhs, compute.sens = TRUE) > names(my.lp.sense) [1] "direction" "x.count" "objective" [4] "const.count" "constraints" "int.count" [7] "int.vec" "bin.count" "binary.vec" [10] "num.bin.solns" "objval" "solution" [13] "presolve" "compute.sens" "sens.coef.from" [16] "sens.coef.to" "duals" "duals.from" [19] "duals.to" "scale" "use.dense" [22] "dense.col" "dense.val" "dense.const.nrow" 10
[25] "dense.ctr" [28] "status
"use.rw"
"tmp"
From the above output the most important attributes are “sens.coef.from” and “sens.coef.to”; based on these we might judge the impact of coefficients in objective function on the value of objective function. > my.lp.sense$sens.coef.from [1] -1e+30 2e+00 -1e+30 > my.lp.sense$sens.coef.to [1] 4.50e+00 1.00e+30 1.35e+01 From the above output it is clear that the x1 coefficient in objective function i.e. 1 need to be between −1e + 30 and 4.50e + 00 in order to achieve the maximum value 40.5 for said variables. x2 value as a matter of feasibility is 4.5, it should be at least 2e + 00 to keep the value as best as 40.5. Now let us look at dualities by executing following set of commands. > my.lp.sense$duals [1] 4.5 0.0 -3.5 0.0 -12.5 > my.lp.sense$duals.from [1] 0e+00 -1e+30 -1e+30 -1e+30 -6e+00 > my.lp.sense$duals.to [1] 1.5e+01 1.0e+30 3.0e+00 1.0e+30 3.0e+00 From the above output it seems that there are three non-zero variables (known as non-basic variables). Constraint one is active in the solution. So, it might be possible to solve the system with first constraint alone. The constant for first constraint is 9 and from the input it is clear that it might range from 0 to 15. These numbers explain as how the value of objective function is subjected to change for a unit change in the value of RHS i.e. my.int.lp=lp("max", obj, con, rel, rhs, int.vec = 1:3) > my.int.lp$solution [1] 1 4 0 > my.int.lp$objval [1] 37 > 11
The integer solution for integers 1, 2, 3 for x1 , x2 and x3 is going to be 1, 4, 0 for maximum value 37. There are two non-zero values for variables. The simplex solution appears to be more feasible as compared to the second.
5
The other free or open source packages for LP problems
As mentioned in introduction lpSolve is not unique solution for LP problems. One of the advantages of using R is that there are millions of individuals working incessantly on and around R as by being community members. There are thousands of packages with not less than a million functions to take care of all most all (analytical) problems of different domains of knowledge. There are umpteen number of packages on CRAN to take care of optimization techniques. The following are only few that might be best but rest are not despicable. Each package is great in its way of solving the problem in a very unique way. These packages are available under CRAN Task View: Optimization and Mathematical Programming. The following is the list with description. – optimx: helps to proper specify the (nonlinear) optimization problem including objective function, gradient function, and scaling. This package supports the (local) optimization of smooth, nonlinear functions with at most box constraints (bounds). – Rcgmin and Rvmmin: ”pure R” implementations of conjugate gradient minimization and variable metric nonlinear function minimization algorithms, respectively. – The R Optimization Infrastructure (ROI): provides a framework for handling optimization problems in R. It uses an object oriented approach to define and solve various optimization tasks in R which can be from different problem classes (e.g., linear, quadratic, non-linear programming problems). – stats: offers several general purpose optimization routines. First, function optim() provides an implementation of the Broyden-Fletcher-Goldfarb-Shanno (BFGS) method, bounded BFGS, conjugate gradient, Nelder-Mead, and simulated annealing (SANN) optimization methods. It utilizes gradients, if provided, for faster convergence. Typically it is used for unconstrained optimization but includes an option for box-constrained optimization. Additionally, for minimizing a function subject to linear inequality constraints stats contains the routine constrOptim(). For one-dimensional unconstrained function optimization there is optimize() which searches an interval for a minimum or maximum. Then there is nlm which is used for solving nonlinear unconstrained minimization problems. Eventually, nlminb() offers unconstrained and constrained optimization using PORT routines. – For other packages you may visit http://cran.r-project.org/web/views/Optimization. html.
References [1] Wikipedia. Retrieved from http://en.wikipedia.org/wiki/Linear_programming
12
[2] Retrieved from http://sourceforge.net/about [3] Retrieved from http://lpsolve.sourceforge.net/5.5/ [4] Retrieved from http://en.wikipedia.org/wiki/R_%28programming_language%29 [5] Retrieved from http://cran.r-project.org/bin/windows/base/ [6] Retrieved from http://www.gnome.org/ [7] Retrieved from http://en.wikipedia.org/wiki/S_%28programming_language%29 [8] Retrieved from http://cran.r-project.org/web/views/
13
