of the chapters in a standard numerical analysis text or an early edition of IMSL or NAG. Later IMSL or NAG libraries have FAR too many chapters for us to.
ISL Working Note 1
Toward an Interval Subroutine Library George F. Corliss R. Baker Kearfott Ned Nedialkov Spencer Smith John D. Pryce Others are welcome DRAFT 23 Mar. 2005 Frozen 11 Sep. 2005 Abstract We propose the development of a full-featured, production quality library of validating routines for use by the wide community of applications developers. What would it take to develop such a library? How could it be done?
1
Disclaimer
Do not take this paper too seriously. It should have about the weight of several friends talking over a few beers. This is just something I’ve been thinking about and thought I’d write some things down for reaction. Pure speculation, OK? In particular, no one whose name is mentioned has agreed to anything. “We” = “members of the interval research community.” “Interval” refers to the validating, rigorous, or reliable algorithms generally. If you care to add, I suggest you add your name to the \author{ } field, add a \newcommand{\QW}{\textbf QW:} (where QW are your initials), and label each paragraph you add by \QW Whatever you want to say, as Baker has done. Paragraphs not otherwise attributed are Corliss’s.
2
Mission
Premise 1: The interval community would like to see people using our work. Premise 2: Academic members, at least, welcome a flow of papers and funding. 1
Mission: Provide a full-featured, production quality library of validating routines for use by the wide community of applications developers.
3
Challenge
We all know there are situations in which approximate algorithms fail to give correct results. Most of us admit that approximate algorithms usually do quite well. Interval algorithms certainly have limitations, especially in the size of problems we can currently handle. There are theoretical and practical challenges. Most of us believe scientists and engineers using scientific computing, and especially people who develop tools for them, should often (opinions differ as to degree) use interval techniques in some sense. We have had some successes, but interval algorithms are certainly not considered as the standard way that everyone does scientific computing. We have enormous up-side potential. Interval algorithms will perform better with better hardware support. Hardware support is unlikely before customers demand it. Customer demand is unlikely before there is lots of good interval software. Approximate scientific computing matured with general acceptance and use of IMSL, NAG, and similar libraries. Today, environments such as Matlab, Maple, and Mathematica bring high-quality routines for approximate scientific computing almost invisibly to the masses. I believe that wide-spread adoption of interval approaches by applications folks who would be well-served by them will not come without a full-featured, production quality library of validating routines. Such a library is beyond the means of any single research group to produce. The days of Bill Gear’s group writing a state-of-the-art ODE solver are pretty much gone, and that is only one chapter in the library we want. We must collaborate. In subsequent sections, I outline some aspects, challenges, and possible solutions for wide-spread collaboration within our community, including • Models • Inventory • Leadership and Organization • Who Does the Work? • Architecture • Programming Environment • Standards
2
• Quality Assurance • Technology Transfer • Funding • How Can We Work Together? • Plan • Other Issues GC: [Added 040917] Another view is expressed by Arnold Neumaier and often by Bill Walster and others. It holds that what we most need now are “killer applications” or “addictive applications” that practicing scientists and engineers come to depend on critically to get their work done. That will speed the acceptance of interval technologies and generate customers, which in turn, provide funds to develop the library envisioned here. The authors are very sympathetic to that view. It may be that the best strategy for the construction of a comprehensive library is to do the high level architecture, and then populate it with components that need to be developed anyway en route to developing addictive applications. The library justifies a little extra work and provides some of the building blocks necessary for building addictive applications, but it is not an end in its own right.
4
Models
How can such a broad-based effort be done? We do not have to invent it. Here are a few models of which I am aware for distributed development of scientific software. We should be able to learn some lessons. NAG and IMSL practice an interesting model of commercial plus academic projects. The general Open Source movement produces some very impressive projects (and far more less than impressive ones). LAPACK [1], its successors, and similar packages were the result of extensive, widely distributed collaborations. GSL - GNU Scientific Library [3], http://www.gnu.org/software/gsl/, is an approximate numerical library for C and C++ programmers. The library provides a wide range of approximate mathematical routines such as random number generators, special functions and least-squares fitting. There are over 1000 functions in total. The complete range of subject areas covered by the library includes
3
Complex Numbers Vectors and Matrices BLAS Support Fast Fourier Transforms Quasi-Random Sequences Histograms Simulated Annealing Numerical Differentiation Discrete Hankel Transforms Least-Squares Fitting
Roots of Polynomials Permutations Linear Algebra Quadrature Random Distributions N-Tuples Differential Equations Chebyshev Approximation Root-Finding Physical Constants
Special Functions Sorting Eigensystems Random Numbers Statistics Monte Carlo Integration Interpolation Series Acceleration Minimization IEEE Floating-Point
NN: PetSc aimed for high portability. Much software development in industry is globally distributed. For example, General Electric Medical Systems is headquartered near me. Very few of their development teams over four people work in the same country. Surely many more. These efforts probably can help us learn how to organize and successfully manage an effort of the scope necessary.
5
Inventory
Building an Interval Subroutine Library is hardly a new idea. Here are a few of the previous efforts which come to my mind: • Karlsruhe XSC Toolbox books [4] • Rump’s INTLAB, http://www.ti3.tu-harburg.de/~rump/intlab/ • Sun’s F95 library • Neumaier et al. COCONUT project for global optimization, http://www. mat.univie.ac.at/~neum/glopt/coconut/ • FILIB++, by Wolff von Gudenberg et al. [6], http://www.math.uni-wuppertal. de/wrswt/software/filib.html • Library from Brazil? • RBK: Wolfgang Walter’s Fortran-XSC work, which includes vector and matrix operations • COSY, by Berz et al., http://www.bt.pa.msu.edu/index_files/page0001. htm • surely several others Besides those, there are many single- (or few-) purpose packages such as Kearfott’s GlobSol or Nedialkov’s ODE solver which are state-of-the-art in their own right and include “lower level” components.
4
RBK: Some of these “lower level components” (such as INTBIS and INTLIB, each independently a TOMS algorithm but also in GlobSol) have been independently reviewed and are widely used in their own right. NN: My problem in C++ is to have an interval package that compiles on any, or many, platforms without errors and warnings. FILIB is not perfect in terms of installation. RBK: Much of the work on the present project might be expedited by identifying appropriate parts of the above packages, securing the cooperation of the authors, and including those parts as design specifications for our package. (We may not be able to include them without modification, since we presumably will be building on a common low-level infrastructure, such as the level 3 interval BLAS. Some interval Open Source projects: Rascal, the Advanced Scientific CALculator: http://rascal.sourceforge.net/ interval.sourceforge.net/interval: http://interval.sourceforge.net/interval/ Interval Analysis: http://sourceforge.net/projects/intervals/
6
Leadership and Organization
Leadership and organization are important. We need a strong, enthusiastic champion and leader. Leadership is not the boss not the funding source. The leader is the catalyst and holder-together-er of a widely distributed volunteer effort. LAPACK would not exist without Jack Dongarra. COCONUT would not exist without Arnold Neumaier. Most successful Open Source projects seem to have one person in charge. Possible candidates (remember the “over a few beers” disclaimer)? I think it would be wonderful to enlist Jack Dongarra or Chris Bischof from the LAPACK effort. No, I have ZERO indication either would do it. Arnold has demonstrated success in COCONUT. I’d nominate Bill Walster for enthusiasm, assuming the issues over the patent applications can be put behind us. Sun, IBM, and others have provided strong leadership to certain Open Source projects. NN: See also at http://www-124.ibm.com/developerworks/opensource/ coin/.
7
Who Does the Work?
Who does the work is important.
5
This has to be primarily a volunteer effort. I discuss funding later, but realistically, the project will have little. If the work gets done, it will be done by volunteer research groups who can be convinced by the leader that their best interests have a component in the same direction as this project. That seems to be how LAPACK and Open Source projects work. NN: Is there some brief history on how LAPACK started? This would be useful to know. JDP: The automatic differentiation chapter(s) would be most up my street, I guess, but I am willing to take other suggestions.
8
Architecture
Architecture is important. Perhaps the leader is the lead architect. Perhaps someone else is the lead architect. The overall and detailed architecture is distributed, too, but there must be one person primarily responsible for a combination of doing and of coordinating with others do. The architecture must be highly modular, with low coupling between different components. That is good software design, and it is essential for development that is distributed in time, personnel, and space. There must be a sound plan, and that plan must be enforced. In my view, there is external architecture, and there is internal architecture. The external architecture is the Application Program Interface (API) exported for use by applications programmers. The external API should be driven by client needs. That is, we ask, “What services do clients request?” rather than, “What services do we want to provide?” Getting the API right is a major project in its own right. The external architecture might be organized into Chapters, along the lines of the chapters in a standard numerical analysis text or an early edition of IMSL or NAG. Later IMSL or NAG libraries have FAR too many chapters for us to consider in the beginning. There should be as much consistency as possible in API’s of different chapters and different routines in each chapter. We could do worse than taking the API of a popular approximate library as our starting point. Our first release would be a very small subset of the eventual target. The internal architecture is how we organize the code to support the external architecture. Most modern software is organized in layers. I’d start with an interval BLAS: Level 0 “Basic Interval Arithmetic Subroutines.” Here is where the fundamental interval arithmetic goes. This should be (or be built on) existing interval libraries (see next section). Level 1 BLAS – Vector-vector operations. Eventually, I suspect we want interval BLAS to match floating-point BLAS as closely as possible, but I doubt we need every routine in our first release. 6
Level 1A Non-basic interval arithmetic. In the block diagram, along side the Level 1 BLAS, is a block of whatever interval arithmetic you do not consider “basic.” Some might put elementary functions here, for example. We might put into Level 0 operations that might be supported in hardware, and all else in Level 1. Level 2 BLAS – Matrix-vector operations. Again we want interval BLAS to match floating-point BLAS as closely as possible, but I doubt we need every routine in our first release. Level 3 BLAS – Matrix-matrix operations. Again we want interval BLAS to match floating-point BLAS as closely as possible, but I doubt we need every routine in our first release. We may not want Levels 2 & 3 in our first release. RBK: Chenyi Hu has essentially completed the design work for us for this phase. Several years ago, he stewarded a document through a fairly rigorous review process that included constructive criticism, guidance, and final acceptance (except for a couple of relatively small issues upon which the interval community itself could not agree) from Jack Dongarra and other principles from the original BLAS, LINPACK, and LAPACK projects. GC: On top of the BLAS, I’d build a Linear Algebra chapter and an Automatic Differentiation chapter. I’d make those the first chapters released because most of the other chapters will want to use some of their routines. NN: I agree. AD should be done as early as possible. An initial list of chapters might be Linear algebra Especially sparse. Including eigensystems. Possible chapter leaders/architects include: Siegfried Rump? A list of modern approximate solvers can be found at http://www.netlib.org/utk/people/ JackDongarra/la-sw.html. Automatic Differentiation Possible chapter leaders/architects include: Ned Nedialkov? Quadrature Possible chapter leaders/architects include: George Corliss? Nonlinear systems Possible chapter leaders/architects include: Baker Kearfott? Optimization Linear and nonlinear. Possible chapter leaders/architects include: Arnold Neumaier, Baker Kearfott? Probability and statistics Possible chapter leaders/architects include: Differential equations ODE’s and DAE’s. Possible chapter leaders/architects include: Ned Nedialkov? Partial differential equations Possible chapter leaders/architects include:
7
Eventually, the list of chapters might be as rich as that of GNU Scientific Library [3] given above, the NAG library, or Numerical Recipes. I would expect many routines in each chapter for different problem types. NN: If leader includes just me, it will be difficult. I assume I can involve more than one person.
9
Programming Environment
Choice of programming environment (language plus) is important. There is no religious war like, “What’s the best programming language?” If our mission is to “provide a full-featured, production quality library of validating routines for use by the wide community of applications developers,” I think the correct answer is, “All of them.” Since this paper is fantasy anyway, if a developer comes and says, “I work in language X. Can I use your routines?” I want to say, “Yes.” Similarly, if a developer asks, “Do I have to purchase anything to use your routines?” I want to say, “No.” There is some sentiment that we should rely on other open tools, e.g., GNU compilers. There is some sentiment that we should support vendors that support us, e.g., Sun’s F95 and C++. There is some sentiment that we should not support vendors who do not support us, e.g., Sun with patent applications. There is some sentiment that we not support single vendor solutions. In that connection, I observe that in the environment where I work, Matlab is not perceived as a single-vendor solution. It is just assumed to be part of the “system” of any reasonable machine. RBK: Although taking account of political considerations is useful to get as many people as possible “on-board,” and to assure there is no outright opposition to the project, our primary consideration should be fulfilling the needs of the scientific computing community as a whole, and maximizing the likelihood that use of the product will become and remain widespread. Thus, in case this conflicts with a particular political consideration, we should judge in favor of the usefulness of the package. If we set up an open-source environment well, then this “conflict of interest” could take care of itself. GC: Since this paper is fantasy anyway, I’d consider multiple language support. The features we need are a small subset of C++, F95, Matlab, Java, or C#. A few of us remember programming in a small subset of F66 in order to pass PFORT as required for publication in TOMS. If we agreed on a suitable subset of the intersection of a set of target languages, I suspect it would not be hard to use the C preprocessor or a collection of Perl scripts to produce various language sources from a single source file. I’d be surprised if there is not someone somewhere who has such a tool already. If not, that is certainly publishable computer science research. NN: This may contain useful information: http://www.llnl.gov/CASC/ components/babel.html. GC: If we can’t do that, I suspect that if we plan from the beginning to port
8
every routine to a total of two or three languages, and program appropriately, the job of porting would not be too hard. RBK: However, such tools should be viewed with skepticism. Manual translation results in tools that are: • significantly easier for humans to read • more efficient • easier to independently maintain Furthermore, the major effort will be the design, implementation, and testing of a reference version (which can be in any particular language), with manual translation being a relatively small effort. GC: I would rather NOT rely on, “You can call libraries written in X from programs written in Y,” because they do not always work as expected. RBK: Although generally true, one exception to this is the standardized interface to “C” that will definitely be in the next Fortran standard. However, implementation of that standard is still some time away, and it will not help with C++ code that is not also standard-conforming “C” code. GC: Within one language, I prefer to work in a subset with ZERO dependencies on compiler or operating system. Remember PFORT? A program that passed PFORT ran ANYWHERE. RBK: Some decisions need to be made along these lines. For example, C++ and C have been evolving fairly rapidly and less conservatively than Fortran. Although “C” has a clear 1998 standard, much older code does not conform to it, and much code relies on headers and libraries that are not standardized. For our work to be maximally flexible, we may want to use constructs from a recent standard that is not yet fully implemented in all compilers. How do we define the “lowest common denominator?” NN: Here we may want talk to the people that started PETSc. They chose C, and possibly some subset of it. I think portability was one of their major concerns. GC: Within one language, we can support different interval libraries with bridge classes, if necessary, to make them look like whatever API we have assumed. Again, rather than take advantage of special features, we program in the interior of the intersection. Of course, these are fantasies, but it would be interesting so see how close we can come. SS: The position that many programming languages should be supported seems like the most conservative approach to me. By using techniques from code generation, it should be possible to automatically support many languages. The original program can be written in one language and then that original program can be compiled into any target language of choice, for which a compiler backend has been implemented. Although the target of the compiler would be a high level language, as opposed to assembly code, the idea would be the same. The backend of the high-level-code generating compiler can be changed 9
to target different high level languages, just as the assembly level instructions generated by machine-code generating compilers can be modified so as to port the code between different machine architectures. Maple provides an example of a language that can be used to automatically generate C and FORTRAN code. If the code generation approach is adopted then there is the potential for other benefits. Portions of the code, especially the interval BLAS portion, could be automatically tuned for efficiency on different hardware platforms, as is currently done for approximate BLAS using ATLAS (http://math-atlas. sourceforge.net/). JDP: As Ned told you, we had a useful meeting with Rob Meyer, the new CEO of NAg after Brian Ford retired last summer, and Steve Hague his no. 2. Here are 2 main points I got from the meeting: 1. Steve said that the NAg “library engine” - the software that handles their version control, automatic generation of compiled libraries, documentation, etc. - is not private, and that they would be willing to let us use it, and adapt it to our needs, for a genuine puiblic-domain research project. 2. Rob said that a major aspect that will determine whether a company such as NAg will become involved, was the details of the license. That there were many public domain license models around, and it was essential to decide this at the start of the project. A license, for instance, that forces any involved group to make public the source code of all changes they make, would kill NAg’s interest at once. I assume ISL would want to promote a situation where an “unsupported” public open source version is created, but it is free for an organization like NAg to make it the basis of their own version, and make money from the added value of support, better documentation, expected longevity etc. NN: PROFIL/BIAS could be a good starting point, but this package has to be updated. It is very well designed. filib++ is another candidate, but it seems to me that it is too “heavy” with templates and advanced C++ features. I think basic operations should be in plain ANSI C or Fortran so they can compile without problems in the future. SUN has an interval library, but I do not know if it will be available for public use.
10
Standards
Standards are important. There are international standards for software development. We probably should follow them. RBK: Along the lines of arguments in the previous section, language standards are most important. Careful software development, especially the modular design and the testing phase, are clearly important. However, it is unclear to this author absent further study how universally applicable general international standards for development are. Perhaps this aspect should be taken with a grain of salt. GC: Yup. And they bring salt into Milwaukee by the lake freighter load . Standards for software development may not turn out to be help-
10
ful. For a project of the scope envisioned here, some systematic approach is essential, and it is probably not the best course for us to invent our own. We certainly will not be successful if we bury our project in paper, but we will also not be successful without effective communication among sub-teams. Standards for software development deserve serious consideration. NN: Very important. I have been looking for a systematic approach for documenting scientific codes, such that a program can be readily verified. I think describing a systematic approach for design, development, verification, and testing of scientific software is needed. We should have some guidelines. RBK: Do you have a reference here, i.e. a suggestion of a writeup or particular international standard for software development we should examine first? GC: I need to be more helpful than these, but here is a start: ISO 9000: http://www.iso.org/iso/en/iso9000-14000/iso9000/qmp.html ISO/IEC 15288: http://www.15288.com/ IEEE: http://standards.ieee.org/software/ CMU SEI: http://www.sei.cmu.edu/ SS: I agree with the position that standards for software development are necessary on a project as ambitious as this one. Given the large number of people that would be involved and that fact that the people will change over time, it is necessary to systematically document the project. Although there is no universally accepted standard for software development, there are many ideas that could be applied to the development of an interval subroutine library. One successful approach that has been used for software development is to produce specific documentation corresponding to each stage of the typical waterfall model. Thus there would be documentation corresponding to requirements (system requirements specification), high level design (system architecture or module guide), low level design (module interface specification and module internal design documentation), implementation (project plan and test plan) and testing and validation (test report and inspection report). The development process itself will likely not be as rationale as the waterfall model suggests, but as Parnas [7] argues there are reasons why it is a good idea for the documentation to “fake it”. SS: Rather than adopt the same documentation standard for all the documents, the selection of the standard could be varied to suit the particular phase of the software lifecycle that is being documented. For instance, for the requirements stage a requirements template could be constructed that borrows and modifies sections from the IEEE template Standard 830-1998 [5] and the Volere Requirements Specification Template [9]. The system architecture could be documented using a module guide, which classifies the decomposition of the system into modules based on the information hidden in each module. The low level design could be documented using displays, as mentioned by Ned, or
11
maybe the documentation could be done using module state machines. As another example, the implementation stage could involve adopting and modifying an existing coding standard.
11
Quality Assurance
Quality assurance is important. I’d propose an SQA research group separate from development groups. The QA group is responsible for ensuring that external and internal architecture designs are clear, and that they are followed. A set of acceptance tests prepared before the routine is written can go a long way toward ensuring that everyone agrees on what the routine will really do. RBK: That’s a very good idea. NN: [Added 040913] I have been thinking for several years that a template for specifying and documenting numerical programs is needed. I do not think such exists. Are you aware of any? If we pursue a project on an interval library, this should be one of the first things to be done. I wonder if a paper on specifying and documenting scientific code will be a good contribution on its own. I am sure there will be many people with good ideas and comments on content. GC: Interesting question. We teach about the importance of well-specified project specifications in software engineering classes. It would be an interesting challenge to write an automated acceptance test (or some other executable, formal specification) for a linear equation solver, a global optimizer, or an ODE solver, even for exact solvers, not to mention enclosure solvers. We all think we know what an A x = b solver should do, so it should not be too hard to write our specifications down? Sure, until we try to do so. On the other hand, it is my perception that it is too easy to get caught in a morass of every-deepening formalism. I wonder about its usefulness for producing code engineers will use. I could imagine a paper on a template and a separate paper with specifications for each chapter. One might get to that by writing formal specifications for 3 or 4 chapters and generalizing what is common into a template, publish the template, then publish the 3-4 as instances. How are specifications related to API’s? It would seem to me that some specifications are more general, but most are specific to each routine call. A set of formal specifications should lead to (be?) the acceptance test. GC: [Added 040917] Bill Walster maintains there is but one requirement for interval codes: THOU SHALT NOT LIE. All else are quality of implementation issues. Sure, that is overly simplistic, but not as far as you might think. For linear systems Ax = b, the program has two returns: 1. N -dimensional interval vector X, which asserts (a) Ax = b has a solution, and (b) the solution is contained in X; OR
12
2. “I can’t do the problem,” or something to that effect. An implementation can compete on the tightness of the bounds, speed, and the size of the domain for which it returns #1. Yes, requirements for an implementation might try to specify tightness achieved, characterize problems for which return #1 is assured, more precise messages for return #2, speed, etc. There are also nicities of describing what is a “solution” to an interval matrix problem, and a few other things, but relying in THOU SHALT NOT LIE is not a bad starting point, and it has the advantage of being understandable to users. GC: [Added 040917] Alternatively, consider a requirement corresponding to an efficient investment frontier in portfolio optimization theory. There, one imagines drawing a graph of expected return vs. risk. For a given level of risk, one wishes to maximize expected return. For a given level of expected return, one wishes to minimize risk. The result is an envelope of achievable (risk, expected return) pairs. That envelope is called the efficient investment frontier. In some classes of scientific computation, there is a similar accuracy vs. CPU time (or other cost) curve. For a given cost, one probably wishes as much accuracy as possible. Alternatively, for a given accuracy (width of interval enclosure), one wishes to achieve it as quickly as possible. If you are thinking of a linear solver, for example, being called within an interval Newton code, one could imagine accepting a (low cost, low accuracy) solution early in the Newton iteration when Newton is far from convergence, and demanding (high cost, high accuracy) near convergence of Newton. We are not prepared to meet such a requirement at this time. SS: To have quality assurance, it is necessary to have a specification of the system requirements. Unfortunately, as the discussion above demonstrates, there are challenges for stating requirements for scientific software. It seems to me that further research is necessary to find a methodology for documenting the System Requirements Specification (SRS) for scientific software. Any candidate methodology should be evaluated against the criteria for a well-written requirements specification, as described in reference [2]: Correct: “An SRS is correct if and only if every requirement stated therein represents something required of the system to be built” [2]. To be successful here requires sensitivity to the user’s needs. The discussion provided in the Challenge section above and in other sections of this document suggest that the work has already begun toward determining who the users are and toward determining their needs. Unambiguous: The SRS is unambiguous if and only if every requirement stated therein has only one interpretation [5]. Due to the mathematical nature of the system being built, it seems that it relatively straightforward to be unambiguous. Complete: The document is complete if everything that it is supposed to do is included; the responses to all realizable input are given; all pages, figures 13
etc. are numbered; and no sections are marked “To be determined”. This criterion may be difficult to judge, but a rigorous inspection process should help. Verifiable: An SRS is verifiable if and only if every requirement stated therein is verifiable. A requirement is verifiable if and only if there exists some finite cost effective process with which a person or machine can check that the actual as built software product meets the requirement [5]. This is likely the most difficult quality criteria to meet for any scientific computing software. For instance, the “Thou shalt not lie” requirement fails this test. No matter how many tests are done on the system to demonstrate that it is telling the truth, the next test could be the one where it lies. Consistent: There should be no conflicts within the document between behaviour, terms, and characteristics. Understandable by the Customer: In the current case the customer would be considered to be the people targeted as potential users of the system. This item is a challenge to meet, because it conflicts with some of the other criteria. For instance, unambiguous requirements are often formal, and formal requirements are seldom easy to understand. Modifiable: Given the large number of people working on the project and the likely long life of the project, the documentation should support change. Traceable: A traceable document means that the origin of each requirement will be clear and when it comes to design it should be possible to trace the design back to the requirements document. Design Independent (Abstract): The SRS should document “What” is required and not “How” to implement it. In scientific computing this can be a tough line to walk because sometimes a likely requirement might be to implement a specific algorithm. If possible, it would be better to put off the specification of the algorithm to the design stage. SS: When it comes to documenting the requirements it seems important to not forget the nonfunctional requirements, which represent the qualities that the system should have. For instance, what are the requirements for efficiency, portability, maintainability, reusability, etc.? The issue of portability has already been discussed in this current document, which means that some of the nonfunctional requirements are already being implicitly gathered. SS: A colleague (Jacques Carrette) who worked for Maple software explained to me part of the systematic approach to Quality Assurance that was adopted at Maple. Some of the ideas that he suggested seem like they would assist with QA for an interval subroutine library. The nice thing about Maple’s approach is that it was 100% automated. Each night 2 million test cases are run on 14 different platforms and the results are automatically reported. Test failures are tracked to the source code and the author and e-mails are automatically sent to 14
inform the author, and his/her manager. A database of timings and resources consumed is also maintained, so that if the system becomes slower (or faster) or uses more (or less) resources, the developers can be informed. SS: When it comes to QA, inspection can be used as well as testing. Some inspection techniques, like code walkthroughs, may be too much effort to be worthwhile, but simple things like checklists will likely improve the overall quality of the library. When there are many programmers involved a checklist can help to make sure that they each maintain a consistent standard.
12
Technology Transfer
For most funding agencies, “contribution to the scientific infrastructure” (or something to that effect) is a major criterion for funding, so technology transfer is important. I would expect a stream of papers and a drum beat of conference presentations, especially at conferences in applications areas. RBK: For this to happen, we should make sure we “add value,” and make it clear to all what value we are actually adding. Since many interval libraries have already appeared, it seems that the umbrella nature of this project would be the main value we are adding. GC: Our added value: This project should be more comprehensive, more portable, better documented, more uniform in its API, and higher quality than previous packages. It should be comparable to early IMSL or NAG libraries. It should have coverage comparable to Numerical Recipes. It should subsume all interval competition.
13
Funding
Funding is important. While it would be great if someone wanted to put up large funding, that is very unlikely. If we look at the models of LAPACK or most Open Source projects, there might be a little bit of funding somewhere for overall leadership and organization, but the developers are on their own to secure their own funding as always. One would hope that contributing to a large, well-organized, wellpublicized international effort might help each of us get our own work funded. In the fantasy world of this paper, it would be great if Sun (or IBM, or even Microsoft) would provide the leadership and modest funding for the champion to lead an Open Source project. In other fields of study, with more obvious customers, each of these firms has seen fit to make significant contributions to various Open Source projects. I have had very preliminary discussions with Ruud van der Pas of Sun. He was somewhat encouraging and thought it not impossible for Sun to participate. RBK: Sun also contributed to development of an interval data type in the Fortran 77 compiler, although the policies within Sun and other commercial vendors shift over time.
15
GC: It would be wonderful to involve a team or two from one of the national labs of any of the participating countries. GC: I suspect that Sun involvement in the interval library as I suggest here might doom it. I suspect many interval researchers would decline to participate, especially with little money. Two possible reservations: 1. Dependency on Sun’s products 2. Fear of Sun patenting results of the project. NN: Dependency on Sun’s products is an obstacle. I hope they make their interval libraries public and portable. GC: If we choose ANY one product, we are dependent on that product. The two safe paths seem to be 1. Have NO dependencies. The project is entirely self-contained. It might get that way by an agreement to subsume (?) your interval library, PROFIL, FILIB, or something else and make it part of an Open Source project. 2. Support MANY platforms, which could include vendor-specific tools without risk of entrapment. NN: If we choose FILIB, for example, we have to have some confidence that this package would compile and work at least 5 years from now.
14
How Can We Work Together?
“Whenever two or three are gathered together in my name, there am I in the midst of them.” Much of the leadership, organization, high level architecture, and enthusiasmbuilding is likely to happen at conferences whenever some of the participants in this project happen to gather. There might be some formal sessions on this project, but there certainly need to be informal discussions or formal discussions limited to a small set of participants. Much collaboration can be done electronically, of course. Most of the real work is done by decomposing the entire project into components small enough that one (or two) research group can complete it in reasonable time. Compared with many large projects, this one seems to decompose rather cleanly, once high-level API’s are agreed upon. Some roles: Leader Provides: Vision Recruitment of volunteers Constant visibility and encouragement Ensure everyone talks to everyone Should have a travel budget for frequent conference appearances and trips to participating labs 16
Qualifications: Respect, enthusiasm, people will follow Candidates: Arnold Neumaier? Baker Kearfott? Head Architect Provides: Vision Primary formulator of external architecture (API’s) Primary formulator of internal architecture (chapters and their contents) Facilitate community participation in architecture Coordinate details of architectures of various components Some QA responsibilities to ensure architecture is followed Should have a travel budget for frequent conference appearances and trips to participating labs Qualifications: Engineering of numerical software Candidates: Ned Nedialkov? Librarian There are wonderful tools for electronic collaboration. Many Open Source projects use SourceForge (http://sourceforge.net). My experience is that even small projects need a librarian to set up and continually maintain the mechanisms and to sort out versioning issues the tools cannot. The librarian makes sure the tools work as expected. That is NOT trivial. Software Quality Assurance Often, a good SQA group does most of the project management. SQA does testing, but their role is FAR more fundamental. They specify all standards the project will follow and then ensure that the standards are followed. They take the architect’s API and develop acceptance tests. Often, it is the acceptance tests to which the developers build rather than a specifications document (hence, you might not write a specifications document). One view of good software engineering is, “Say what you do. Do what you say. Be able to prove it.” SQA is responsible for saying what we do, and they are the authority to which participants are responsible for proving they did it. Chapter Leader Similar to overall project leader, except for an individual chapter in the library. Chapter Architect Similar to Head Architect, except for an individual chapter in the library. . . . Each chapter is a significant project in its own right, probably with its own sub-organization. RBK: With travel money, time, and a belief it is the most important thing we should be doing, we have the expertise and leadership within the community to do it without Sun. GC: I agree we have people who COULD do it on a volunteer basis or with funding they find for themselves, but I doubt we have anyone who WILL do so. The reader may fill in a list of suspects. 17
This is a BIG project. I think we need AT LEAST 50 routines completed in an integrated framework to launch for public use. Without critical mass, acceptance is on a case-by-case basis. How many routines are in Numerical Recipes? Nearly 1000, I think. To get people to use the library, it must be nearly true that if they have a problem, we have a solution. I think it took LAPACK and MINPACK each about 5 years and a dozen major researchers. They started with at least as much existing code as we have. Each of those is just one chapter in our library, and we all know good interval code is harder to write than good approximate code. That is why I am not optimistic. In my mind, the scenario most likely to succeed would be for Sun to make it entirely an Open Source project (to break skepticism of vendor-dependencies), but assign about two of their people full time to make it go, one leader and one architect. It would work if one of those people were one of “us” they could hire full time for a year on a sabbatical, and rotate that person. I don’t see Sun doing that. RBK: I am willing to help spearhead an effort, but I would need the following: 1. Adequate travel funds 2. Summer salary 3. Expectation that the deliverable will be the library. (That is, I don’t think I could do it on a research grant for another purpose.) 4. At least one other partner in the leadership. Although I’ve been mostly interested in basic research in the past, I’ve developed a taste for the pride in well-crafted libraries that are widely adopted. Along these lines, I view our INTLIB and INTERVAL ARITHMETIC module as success, while GlobSol, much more complicated and motivated by basic research, has not been polished to the same standard. Interval libraries, making a claim of validated computation, need to be at a higher standard of structure, simplicity, and readability. In particular, users should be able, by looking at the code, to become convinced that the code is correct. I’m thinking of the same standard as a mathematician would have for a mathematical proof under review for publication. NN: I have been thinking about “looking at the code, to become convinced that the code is correct” for about 3 years. I like the idea of the display method of David Parnas [8]. I do not think it is widespread. RBK: Do you see possibilities for public funding of two or more people, as I outlined above, for this effort? I note that COCONUT, funded by the EU, was a sizeable project, perhaps not on quite the scale we propose, but near it. Surely, shouldn’t the US have similar funding opportunities? GC: I doubt it. EU, in my opinion, spreads quite a bit of money around for the primary purpose of forcing people from various countries to talk to each other, building political sense of one Europe. I doubt NSF would fund a 18
COCONUT-style project, although I might be wrong. NSF has been reluctant to fund international travel, which would be essential here. Nor would I expect NSF to fund development that would be commercialized by NAG. We need a strong US industrial partner. Suspects include: Sun, Visual Numerics, IBM, Microsoft, Frontline Systems, perhaps HP, Oracle, Mathematica, Matlab, or Maple. An alternative model to the widespread collaboration might be an army of PhD students. Consider what Ned did for his PhD. Suppose Baker, Corliss, possibly Weldon and Ned each had money for about four PhD students each, with an initial allocation of two students per chapter. The model is to study existing work, replicate it and extend it in one common environment. Rough budget for one researcher + 4 PhD students for one year: Summer Fringe Student @ 12 x $2,000 Fringe Tuition @ 18 x $700 Travel 13 @ $1,200 Equipment & software Subtotal Indirects @ 50% Total
$20,000 6,000 100,000 15,000 50,000 15,000 15,000 220,000 110,000 330,000
Arithmetic is intentionally approximate, and I’ve use Marquette’s tuition, overhead rates, etc. Multiply by 4 PI’s and 5 years, this is a $6M project. We might try to do it for less, but that is close to actual cost. I doubt the standing of any of us in our university would not be hurt by bringing home a five year, $1.5M NSF (Canadian counterpart, for Ned) grant. NN: In the Canadian system, I do not need so much money. At the moment, 15,000CDN/year (≈ 10,000USD) for a PhD student is reasonable. (GC: Do not under-ask at this stage. You do not want to put yourself in a position you later regret.)
15
Plan [Added 040913]
Here are decisions I think need to be made quite early in such a project: • Name for the project • Inventory of approximate libraries to suggest coverage, API’s for ours, and models for such collaborative projects (see § 4) • Inventory of interval systems to suggest possible development environments and code bases we might subsume (see § 5) • Development environment and language (see § 9) 19
• Chapter outline and rough order of implementation (see § 8) • Quality assurance plan, precise specifications (Ned) (see § 11, 10) • High level architecture, both external and internal (see § 8) • Staffing (see § 6, 7, 14) • Funding (see § 13) • Communication • Deliverables: Software, papers, and conference presentations I would like to consider a small (3-6 person) WORKshop in December/January by a core working group to decide what we can and to develop a plan for deciding what we cannot.
16
Other Issues
Feel free to add here anything that does not fit in an earlier section. NN: I have a feeling that it will be difficult to start a project of this sort by e-mail. I am going to SCAN. We may discuss this subject there. I am wondering if a project proposal by several people in different countries would help rising money from NSF and here from NSERC. In Canada, if a company gives X dollars, there is a good possibility the government will match it.
17
Call to Action
Let’s DO it! Any volunteers?
References [1] E. Anderson, Z. Bai, Christian Bischof, S. Blackford, John Demmel, Jack Dongarra, J. Du Croz, A. Greenbaum, Sven Hammarling, A. McKenney, and Danny Sorensen. LAPACK User’s Guide, 3rd edn. SIAM, Philadelphia, PA, 1999. [2] Alan M. Davis. Software Requirements Objects, Functions, and States, Revision. Prentice Hall, Englewood Cliffs, NJ, 1990. [3] GSL. Gnu scientific library, 1996 - June 2004. [4] R. Hammer, M. Hocks, Ulrich Kulisch, and Dietmar Ratz. Numerical Toolbox for Verified Computing I — Basic Numerical Problems. Springer-Verlag, Heidelberg, 1993.
20
[5] IEEE. Recommended Practice for Software Requirements Specifications, IEEE Std. 830. IEEE, 1998. [6] M. Lerch, G. Tischler, J. Wolff von Gudenberg, W. Hofschuster, and W. Kr¨amer. The interval library filib++ 2.0 - design, features and sample programs. Preprint 2001/4, Universit¨at Wuppertal, 2001. [7] D. L. Parnas and P. C. Clements. A rational design process: How and why to fake it. IEEE Trans. Softw. Eng., 12(2):251–257, 1986. [8] D. L. Parnas, J. Madey, and M. Iglewski. Precise documentation of wellstructured programs. IEEE Trans. Softw. Eng., 20(12):948–976, 1994. [9] S. Robertson and J. Robertson. Volere requirements specification template. In Mastering the Requirements Process, pages 353–391. Addison-Wesley Educ. Publ. Inc., 1999.
21
