IEEE TRANSACTIONS ON SOFTWARE ENGINEERING,
VOL. 26,
NO. 11,
NOVEMBER 2000
1025
Guest Editors' Introduction: Workshop on Software and Performance Albert Mo Kim Cheng, Senior Member, IEEE, Paul Clements, and Murray Woodside
æ 1
T
INTRODUCTION
HE first Workshop on Software and Performance (WOSP '98) was motivated by a desire to pull together the scattered efforts being made, from many different starting points, to create a methodology for this problem. Performance is a problem in many software development projects and anecdotal evidence suggests that it is one of the principal reasons behind cases where projects fail totally. There is a disconnect between the techniques being developed for software analysis and design and the techniques that are available for performance analysis. It is the hope of the workshop organizers that a community will emerge that will tackle this problem and bridge this gap. The performance engineering work of the workshop general chair, Connie Smith, is an inspiration to this effort. There were 26 contributions accepted for the workshop out of 41 submissions. A number were nominated by the program committee as representing especially innovative contributions and the authors were invited to extend them for transactions publication. Most of the papers were very considerably extended beyond the workshop version. These were subjected to a new review process and five were selected for these special sections. A sixth paper (the one by Ramesh and Perros), which had been independently revised by its authors and had been accepted for publication, was incorporated in these special sections because it had originated at the workshop. We would like to acknowledge the efforts of associate editor Bill Sanders in this regard. For space reasons, the special issue has been broken into two parts, with four papers in this issue and two in the December issue.
2
THE PAPERS
The paper by Vikram Adve, Rajive Bagrodia, James Browne, Ewa Dellman, Aditya Dube, Elias Houstis, John Rice, Rizos Sakellariou, David Sundaram-Stukel, Patricia Teller, and Mary Vernon describes the POEMS project, a comprehensive environment for the end-to-end modeling of . A.M.K. Cheng is with the Department of Computer Science, University of Houston, Houston, TX 77204-3475. E-mail:
[email protected]. . P. Clements is with the Software Engineering Institute, Carnegie Mellon University, Pittsburgh, PA 15213-3890. E-mail:
[email protected]. . M. Woodside is with the Real-Time and Distributed Systems Group, Department of Systems and Computer Engineering, Carleton University, 1125 Colonel By Drive, Ottawa, Ontario, K1S 5B6 Canada. E-mail:
[email protected]. For information on obtaining reprints of this article, please send e-mail to:
[email protected], and reference IEEECS Log Number 111944.
large and complex parallel and distributed computer systems. POEMS can model every component of a parallel system, including the application software, the operating system, and the hardware architecture. It provides a choice of modeling techniques as well as different levels of modeling details. Of particular importance is the use of a graph-based specification for describing parallel computation that is automatically generated by a parallelizing compiler. The paper discusses the key components of POEMS and demonstrates its capabilities by applying it to predict the performance of a complex benchmark (Sweep3D). In ªArchitecture Based Performance Analysis Applied to a Telecommunication System,º Dorina Petriu, Christiane Shousha, and Anant Jalnapurkar attack the problem of carrying out performance engineering before you have detailed designs or even implementations set in stone. By showing how to predict and measure throughput and latency when all you have is the software architecture, they bring performance engineering to bear at precisely the time when it can do the most goodÐbefore the designs are frozen and irrevocable. Their approach is based on Layered Queuing Networks, a form of queuing models developed especially for distributed systems. LQN models can be built straightforwardly from architectural component-and-connector views and can illuminate bottlenecks and problematic coarse-grain design decisions early, when there is still time to correct them. The paper by Daniel Menasce and Hassan Gomaa addresses performance at an early stage of design for client-server systems. It addresses systems in which database behavior is an important part. Well-known object-oriented analysis techniques based on use cases and object modeling are used to define a model for the planned system, including its behavior, architecture, and deployment configuration. The information is captured in a special-purpose language, CLISSPE, which has features of a scenario specification language (for the transaction behavior), of a data definition language for the database, and of an architecture definition language for the software and hardware architecture and configuration. The language provides quantitative parameters for workload properties, including both the amounts of work to be done and the device capacities, and it also includes facilities for defining typical details of database storage and transaction execution. The compiler for CLISSPE reduces the combined definitions to a queuing network model, which is used to calculate the performance. The alternative to this kind of
0098-5589/00/$10.00 ß 2000 IEEE
1026
IEEE TRANSACTIONS ON SOFTWARE ENGINEERING,
special-purpose language for performance engineering would be integration of performance concerns into a software design language, which is an open question. The layered queuing model was introduced some years ago to represent contention for software servers in deeply layered server systems or for other logical resources. The special feature of a layered server is that its service time (or holding time) includes contention delays at lower layer servers, or at devices, and is therefore not a known quantity. A search continues for robust approximate solutions, usually based on a set of submodels constructed by layer that match conditions met in practice. In particular, software servers often have general (nonexponential) service times, multiple classes of service, and first-comefirst-serve queuing of request messages. The paper by Sridhar Ramesh and Harry Perros goes a considerable distance toward meeting this requirement. It applies a powerful queuing approximation to the layer submodels for systems with synchronous interaction of processes (for example, systems that communicate by remote procedure calls) or with asynchronous messages. There are still some limitations to the generality of the model, but within those limitations they have achieved excellent accuracy. It seems probable that these layered queuing approximations could be used in the future to model client-server system, like those studied by Daniel Menasce and Hassan Gomaa in the previous paper. The two papers still to come are by B. Nixon, on requirements, and by E. Weyuker and F. Vokolos, on testing. The papers in this special section and those in the Proceedings of the First International Workshop on Software and Performance, (ACM order number 488983, 1998) give important insights into the status and difficulties of analyzing software performance. We invite you to join in this exciting enterprise.
VOL. 26,
NO. 11,
NOVEMBER 2000
Albert Mo Kim Cheng received the BA with highest honors in computer science, graduating Phi Beta Kappa, the MS in computer science with a minor in electrical engineering, and the PhD in computer science, all from the University of Texas at Austin, where he also held a GTE Foundation Doctoral Fellowship. He is currently a tenured associate professor in the Department of Computer Science at the University of Houston, University Park, where he is the founding director of the Real-Time Systems Laboratory. He has served as a technical consultant for several organizations, including IBM. During the summer of 1995, he was a visiting faculty member in the Department of Computer Science at the City University of Hong Kong. Currently, he is a visiting faculty member in the Department of Computer Science at Rice University. His research interests include real-time systems, rule-based expert systems, reliable software systems, multimedia tools, and fault-tolerant distributed and parallel systems. One of his recent works presents a timing analysis of the X-38 Space Station Crew Return Vehicle Avionics, which contains a fault-tolerant distributed system. He is the author/coauthor of more than 50 refereed publications; he is serving and has served on the program committees of many conferences in his areas of research. He is a frequent reviewer for the IEEE Computer Society, as well as for many journals and conferences. He has received numerous awards, including the US National Science Foundation Research Initiation Award (now known as the NSF CAREER award), the Texas Higher Education Coordinating Board Advanced Research Program Award, and the University of Houston Research Initiation Grant. He has been nominated for the University of Houston Teaching Excellence Award by his students and colleagues for his outstanding teaching. He is a member of the honor societies of Phi Beta Kappa, Phi Kappa Phi, Upsilon Pi Epsilon, Beta Alpha Phi, and Golden Key. He has been invited to present seminars and tutorials at many conferences including IEEE CAIA, IEEE COMPASS, IEEE PDIS, IEEE SAST, IEA/AIE, SEKE, SEA, DAIS, IEEE CBMS, IEEE IC3N, ICCIMA, EIS, ICPDCS, IEEE ICECCS, IEEE IPCCC, IEEE MASCOT, ACM SAC, ICEIS, IEEE ICMCS, IEEE ISSRE, ACM CIKM, and IEEE IECON; and has given invited seminars/keynotes at many universities and organizations. He is the invited special session/panel chair for the software engineering for multimedia session at the 1999 IEEE International Conference on Multimedia Computing Systems (ICMCS), and is the work-in-progress program chair of the 2001 IEEE Real-Time Technology and Applications (RTAS) Symposium. He is an associate editor for the IEEE Transactions on Software Engineering and the International Journal of Computer and Information Science. He is a senior member of the IEEE. Paul Clements is a senior member of the technical staff at Carnegie Mellon University's Software Engineering Institute. His interests include software architecture and design, software product line methods and practices, software specifications, and real-time embedded systems. He has published more than 30 papers in these areas and has coauthored the book Software Architecture in Practice (Addison Wesley, 1998). He is currently working on books about software architecture evaluation, software architecture documentation, and software product line practices. Murray Woodside received the PhD degree in control engineering from Cambridge University, England. He has taught and done research in stochastic control, optimization, queuing theory, and performance modeling of communications and computer systems, with more than 100 articles in these subjects. His current interests are software engineering and performance engineering of distributed systems and telecommunications software. He currently holds the OCRI/NSERC Industrial Research Chair in Performance Engineering of Real-Time Software at Carleton University, Ottawa, Canada, where he has taught since 1970.
