Introduction to: Influence of Programming Techniques on the Design of ...

Introduction to “Influence of Programming Techniques on the Design of Computers” MICHAEL J. FLYNN, FELLOW, IEEE Invited Paper

In the prolog “summary” to this paper, there is a dramatically understated sentence: It is quite conceivable that these [programming] techniques will exert a strong influence on computer design in the future. That sentence summarizes the goal of computer architects and computer systems designers over the past 45 years. The two authors were pioneers in computer systems design. Grace Murray Hopper was the visionary leader who helped define software as a profession. During World War II she was a Naval officer assigned to Harvard to program the famous Mark series of computers, where she lead efforts in applications programming for Ordinance development. She joined the Eckert–Mauchly Corporation as that firm was developing the Univac-I, the first large-scale commercial electronic computer. While there, she conceived of and developed the first compiler program. She became a leader in the formalization of high-level programming languages, best known for her work in developing the language Cobol. She was rejoined the Navy in 1967 and continued her work on programming languages until she retired as Rear Admiral in 1985. John William Mauchly was also a visionary pioneer of the electronic digital computer era. He, together with J. Presper Eckert, conceived ENIAC while at the University of Pennsylvania’s Moore School of Electrical Engineering. This machine was later commissioned at the Aberdeen Proving Grounds, MD, in February 1946. By achieving a speedup factor of 1000 over earlier mechanical devices, the electronic ENIAC opened the door for the modern age of computer hardware. Mauchly and Eckert later formed their own company, which was to become a division of the Remington–Rand Corporation. The paper clearly outlined what was to come for the following three decades of software development and made several points that remain generally applicable today. Manuscript received January 10, 1997. The author is with the Electrical Engineering Department, Stanford University, Stanford, CA 94305 USA. Publisher Item Identifier S 0018-9219(97)02483-3.

1) The design of a computer is a tradeoff between hardware cost-performance on the one hand and programming functionality on the other. 2) The programmer has an important relationship to the functional attributes of a computer and should understand and make best use of these attributes. 3) At the time the paper was written, there were a number of emerging programming techniques or styles which promised enhanced programmer productivity. They anticipated “simple English” as a high-level programming language. I. DESIGN TRADEOFFS Tradeoffs between computer design cost-performance and programmer accessible functionality are as current a problem today as they were in 1953. A primary concern of the hardware designer is with the functionality provided to the software and programming user. The issues have become significantly more general than those anticipated by the authors. The issues are not ones simply of selecting new instructions and the ramifications thereof, but rather more subtle issues such as compatibility with older instruction sets no longer useful in the context of new applications, but whose functionality must be available to run old (so-called “heritage”) programs. While the authors’ concern about instruction sets may appear quaint to a modern reader, in the intervening decades instruction sets have occupied an enormous amount of technical discussion, frequently for the wrong reasons. The ease of use of an instruction set by a “programmer” is a valid concern both then and now, especially if we think of the term “programmer” in a very broad sense—old applications programs, operating systems, memory size restrictions, etc. Today’s programmers see high level language constructs; the instruction set efficiency is determined by its usefulness to the compiler code generator and hardware implementation considerations. The optimality of an instruction set in a hardware cost—performance sense was a legitimate concern in 1953, but with modern

0018–9219/97$10.00  1997 IEEE PROCEEDINGS OF THE IEEE, VOL. 85, NO. 3, MARCH 1997

467

technology it is much less of a concern. Actual performance differences in instruction set efficiency are slight, but these differences still stir passions among hardware designers. Within the past few years, there has been a continuing (and generally unproductive) debate over the cost—performance benefits of the so-called RISC instruction sets over earlier instruction sets labeled CISC. II. A HIERARCHY

OF MACHINES The authors refer at several points to the relationship or adaptability of the human programmer to the instruction code of a particular machine. While they were aware of compilers, they did not foresee the pervasive use of compilers. They saw an intimate relationship between the hardware and the human programmer. It is probably in this area that some of the greatest changes have occurred. Today’s users see a hierarchy of virtual machines, each interpreted by a lower level of machine, until finally the lowest level, the hardware, actually executes the required actions. These levels of abstractions are an integral part of the modern programming paradigm. In general, there is no machine specificity except for the final, bottom-most layer of programming abstraction. Applications and higher-level language programs sit on top of operating systems such as UNIX, which (at least ideally) appear the same to each application across a variety of platforms, each potentially with different instruction sets. This realization that an abstract machine can exist over many implementations has been an important development in both hardware and software. The initial impetus for a compatible family of machines came from designs such as IBM’s System 360, which offered many cost-performance design points all sharing a common instruction set. Since the software was compatible across all implementations, this enormously improved the accessibility of users to shared programs. A subsequent and significant development was the introduction of instruction set-independent operating systems that were designed to be “ported” to or implemented on different instruction set platforms. Abstract machines such as IBM’s MVS or UNIX provide a basis for higher levels of abstraction—various higher level languages, i.e., Fortran, C, and C —whose users see abstract machines even more independent of the physical hardware. The final abstraction is the sequence of actions that the ultimate user must take to specify the desired functions that are to be performed on specific data sets.

III. PROGRAMMING TECHNIQUES Hopper and Mauchly present, at least in vestigial form, many of the main themes in programming techniques that would be in use through the coming decades. It describes the use of interpreters; programs that interpret one of a specified series of actions that produce the particular result as soon as the data is presented. The contrasting technique is defined by the compiler or program translator. In the language of the day, the compiler input is pseudocode, which the compiler translates into the output ma468

chine code. The authors recognize the importance of tools such as compilers in improving programmer, and indeed machine, productivity. They make the cogent argument that an improvement in programmer productivity improves overall machine efficiency by reducing reruns, etc. Today’s compilers fulfill largely the same role as envisioned by Hopper and Mauchly and significantly improve programmer productivity. The levels of abstraction they operate on, however, are different. Today’s compilers translate highlevel language program representations into intermediate representations which are then manipulated, optimized, and translated into machine code. While interpretive, oriented and compiler oriented computing have been with us through these decades, it is interesting to reflect on the relative emphasis that they have had on machines through these years. When machine efficiency was a primary consideration, much of the computing was done by submitting a program representation that would be compiled and executed at some later time on some remote hardware. There has been a continuing need for user access to machines simply to debug various programs or simply to more readily access the computer hardware. The great movement away from the batch oriented computer system occurred with the introduction of the time shared system of terminal users, and later the personal computer and personal workstation. The emphasis on programming techniques then also changed towards interpretation, especially of icons, and the immediate response to program requests. The ultimate extension of this to date is networked web access to data and functions. Hopper and Mauchly make an interesting observation about programs that use and manipulate symbols. They point to a particular prototype program which correctly differentiates mathematical equations. In this, of course, they anticipate the genesis of generations of formula manipulating programs which come down to us today as programs such as Mathematica.

IV. A REFLECTION In reading this classic paper, it is natural to reflect on the evaluation of hardware programming techniques and the software that has enabled the computing industry as we know it today. It is interesting to consider which aspects of hardware and software development have accounted for the enormous growth in the field over the ensuing four decades. On the hardware side, it is easy to cite technology, in the broad sense, as being the enabling force. This is unsatisfactory, however, since some aspects of hardware technology have clearly played a lesser role than others. For example, the concern with the instruction set costperformance trade-off mentioned in the Hopper–Mauchly paper is now misplaced. Probably the most important technological development is the extraordinary advances in the cost and availability of storage. From the late 1950’s until the present, the cost per bit of memory has decreased by 10 . This enabled enormous PROCEEDINGS OF THE IEEE, VOL. 85, NO. 3, MARCH 1997

memory systems at low cost. Interestingly enough, the performance of memory during this same period has advanced by perhaps a modest factor of 20 in main memory cycle time. Of course, the same technology that enabled the extraordinary advance in storage also provided for a similar advance in the cost of computation, but it is storage that holds programs. From a computer architecture point of view, the important advances in the hardware were those that enabled transparent access to large virtual memory spaces. This enabled combinations of very large software programs to be available on a single processor platform. This ability of the hardware to reference very large address spaces contained on disk at the access time of main memory involves a process of capturing the working set—those areas of the program active in main memory—in a manner that is transparent to the user. This virtual memory management is a significant accomplishment of both hardware and software designers. It is extended one step further with the concept of a cache, a very high speed buffer that contains the refined working sets of the program in main memory. The cache enables the processor to access data items more than an order of magnitude faster than direct main memory access. While it is easy to see the DRAM and semiconductor technology playing pivotal roles in the advance of the computing system, a comparable though sometimes less heralded role has been accomplished in the area of mass storage by extraordinary advances by magnetic technologists. Just over the recent decade, the cost per bit of disk has decreased by 10 , again enabling extraordinary advances in systems cost performance and enabling the user to aggregate and execute huge programs and data sets. The software techniques have developed at a similar pace. Earlier developments focused on high level languages tailored to scientific or commercial applications. Greater understanding of programmer productivity and program correctness enabled the development and maintenance of large user applications. Operating systems were developed to efficiently maintain the flow of work through the processor, to manage memory and provide interprocessor access, and to share resources among many users or tasks. Ultimately, it is the operating system and the network access software that enables access to the latest element in the computer revolution, the network. With the availability of computation in an immediate and interactive manner, the whole style of computation changes from an emphasis on formal languages and formal specifications to an emphasis on interaction. The development of icon-based computation and scripts have played an enormous role in enabling the widespread use of computation to a myriad of less expert users.

PROCEEDINGS OF THE IEEE, VOL. 85, NO. 3, MARCH 1997

V. CONCLUSION With the increasing use of high-level abstractions, today’s programmer usually is far removed from hardware details. This can be advantageous, insofar as it increases productivity, but it also has a downside. The issues of performance, physical memory use, and error recovery can be seriously compromised by programmers’ unfamiliarity with hardware functionality. Indeed, many of us have endured personal computers with operating systems that allow erroneous application programs to “crash” the system. This occurs despite hardware state control facilities that, if properly used, easily identify and prohibit “erroneous” access attempts. The most productive software makes best use of all hardware facilities. The best hardware implementations readily recognize the way and the frequency of use of the various facilities. This awareness of the total system has not always been with us since 1953; indeed, we refocus on this question as the topic of this special issue of these PROCEEDINGS on “Hardware-Software Co-Design.” The computer revolution continues, and each generation sees a series of trade-offs between hardware and software in a peculiar way that adapts to the parameters of the day. Truly, the tradeoffs between hardware functionality and software efficiency will be there in decades to come. The nature of the tradeoffs described by Hopper and Mauchly will be with us in the future, even as the details of those trade-offs change.

Michael J. Flynn (Fellow, IEEE) is currently Professor of Electrical Engineering at Stanford University, Stanford, CA. He was Director of Stanford’s Computer Systems Laboratory from 1977 to 1983. His work experience includes ten years at IBM Corporation, where he worked on computer organization and design. He was also a faculty member at Northwestern University, Evanston, IL, and Johns Hopkins University, Baltimore, MD. His research interests include parallel processors and computer arithmetic, in which he has published widely. Dr. Flynn served as Vice President of the Computer Society and was Founding Chairman of CS’s Technical Committee on Computer Architecture and ACM’s Special Interest Group on Computer Architecture. He received the 1992 ACM/IEEE Eckert–Mauchly Award for his work in computer architecture, and the 1995 IEEE-CS Harry Goode Memorial Award for his outstanding contribution to the design and classification of computer architecture. He is a Fellow of the ACM.

469