TFS-17336; No of Pages 8 Technological Forecasting & Social Change xxx (2010) xxx–xxx
Contents lists available at ScienceDirect
Technological Forecasting & Social Change
Delphi: A brief look backward and forward Harold A. Linstone a,⁎, Murray Turoff b a b
Portland State University, P.O. Box 751, Portland, OR 97207, United States New Jersey Institute of Technology, Newark, NJ 07102, United States
a r t i c l e
i n f o
Article history: Received 29 September 2010 Accepted 29 September 2010 Available online xxxx Keywords: Delphi Social networks Participative planning Communities of practice Policy Delphi Delphi conferencing
a b s t r a c t In response to a request by the guest editors, we have set down our thoughts regarding the evolution of Delphi, beginning with our immersion in the subject in the late 1960s and concluding with some rumination about its future. Our focus is on the changing roles of Delphi. Most importantly, with the profound impact of the internet on organizational and community planning systems, it will foster a new age of participation through communication, coordination, and collaboration. © 2010 Published by Elsevier Inc.
1. The Past Delphi was developed at the RAND Corporation in the 1950s by Olaf Helmer, Norman Dalkey, Ted Gordon and associates under the auspices of the U.S. Air Force as a technique to apply expert input in a systematic manner using a series of questionnaires with controlled opinion feedback. Key features were preservation of anonymity in the expert panel's responses and iteration of the questionnaires. A key benefit of participation was the ability of individuals to participate in a group communication process asynchronously at times and places convenient to them. The initial applications were in the area of national defense, with the first unclassified use a RAND report [1] on long-range forecasting. An offspring was the Trend Delphi, in which a group extrapolates a trend curve into the future. We both encountered Delphi first in 1968: Turoff in the work for the Office of Emergency Preparedness (OEP) and Linstone in the preparation of the very first issue as editor of the new journal Technological Forecasting and Social Change (TFSC). That issue already included an article on a U.S. Navy technological forecasting procedure entitled “SEER: a Delphic Approach Applied to Information Processing” by G. Bernstein and M. Cetron. Our two paths soon crossed as the second and third TFSC volumes (1970– 1972) featured Turoff's articles on “Design of a Policy Delphi” [2] and “Delphi Conferencing” [3], respectively. A Policy Delphi seeks to expose different policy options and the most important pro and con evidence or arguments to support each policy resolution. Delphi Conferencing refers to an online computer-mediated asynchronous conference system with anonymity. Our Delphi book [4] featuring many articles from TFSC was published in 1975. In a foreword Olaf Helmer wrote “Delphi has come a long way in its brief history, and it has a long way to go”. He called for the solidification of the Delphi technique and foresaw its potential importance in supplying ‘soft’ data in the social sciences. Now a centenarian, he can view his comments more than thirty years ago with eminent satisfaction.
⁎ Corresponding author. E-mail addresses:
[email protected] (H.A. Linstone),
[email protected] (M. Turoff). URL: http://is.njit.edu/turoff (M. Turoff). 0040-1625/$ – see front matter © 2010 Published by Elsevier Inc. doi:10.1016/j.techfore.2010.09.011
Please cite this article as: H.A. Linstone, M. Turoff, Delphi: A brief look backward and forward, Technol. Forecast. Soc. Change (2010), doi:10.1016/j.techfore.2010.09.011
2
H.A. Linstone, M. Turoff / Technological Forecasting & Social Change xxx (2010) xxx–xxx
A recent development is the Problem Solving Delphi, a system for providing collaborative judgment [5]. It collects participants’ rankings or paired comparisons. Using Thurstone's Law of Comparative Judgment the votes of the N participants are converted to a single group interval scale where the distance between objects in the scale is proportional to the amount of agreement on preference order. Two solutions or components of solutions at the same point on the scale would be due to a 50–50% vote on whether A was preferred to B or B was preferred to A. Participants are encouraged to focus their discussion on resolving the major disagreements [6,7]. Over the past four decades more articles in TFSC have been devoted to Delphi than to any other technique in the domain covered by the Journal. A recent featured article is “RT Delphi: an efficient “round-less” almost real time Delphi method” by Ted Gordon and Adam Pease [8]. It exemplifies how computers and the Internet have enhanced the original concept by allowing for computer-mediated asynchronous communication that is now accessible globally by any group member. This flexibility makes it possible for the user to participate in any phase of a decision process at any time. The popularity and proliferation of Delphi literature over half a century inevitably means that more than one generation of analysts is involved. The Linstone–Turoff book [4] already included a bibliography of 670 Delphi-related items and by now the number of papers must be in the thousands. Since the book was made available free online, the number of citations has more than doubled to over 2200 (Google scholar: 8/15/2010). As a new generation enters the field it is oblivious of some of the work done by a previous generation with the result that it “reinvents the wheel.” Computer Mediated Communications (CMC) is particularly subject to this problem. The computer field is driven more by industry than by academics and every company seeks a new name for its “new” system to reflect its alleged uniqueness. This is a continuing problem in the areas of Computer and Information Systems where every vendor wants to make his system sound new and rarely points out the evolution based upon earlier systems. Derivatives of Delphi have emerged under names such as prediction markets, collaborative tagging, recommender systems (like Netflix), and social networks that usually serve a commercial objective [7]. But a new name does not necessarily imply a new field. In the Delphi area the specific topics of Group Decision Support Systems (GDSS), Collaborative Systems, and Collective Intelligence, and CMC are areas employing many of the ideas from the Delphi area [9,10]. These papers categorize hundreds of experiments and field studies in Group Decision Support Systems based in part upon common Delphi factors such as anonymity, feedback structures, pen names, communication process structures, and voting. In the 1978 and 1993 editions of The Network Nation [11], Hiltz and Turoff extrapolated the concept of Delphi in the form of Computer Mediated Communications as attempts to produce Collective or Collaborative Intelligence among groups. This is the seemingly "simple" concept that an effective group communication process should allow the group to reach a "better" result than any member of the group would have determined acting alone. One of the most important aspects of Delphi we attempted to illustrate in the Delphi book [4], by the wide variety of examples we chose, was that every communication structure presented in the different applications was specifically tailored to the “nature of the application and the nature of the participating group”. We focused on the philosophy C. W. Churchman in his book on The Design of Inquiry Systems [12] and that of Heidegger's “negotiated reality” as not only templates for the design of information systems but also for the design of Delphi processes [4 (Chap. 2)]. This concept has been extended in a number of ways in that the later work by Dr. Turoff applied the same approach to Computer Mediated Communications and to online versions of Delphi [11, (Chap. 14); 13,14]. Today when “collaborative structures or knowledge structures” are referred to in the literature , they are another set of names for exactly what was meant in the 1975 book by us as Delphi or group communication structures. A major concern in designing a Delphi, particularly online versions, is that the structure allows the participants to classify easily their inputs, particularly qualitative ones, in a morphological structure they can mutually understand. The resulting knowledge structure and its evolving content can prevent information overload [15]. In the future, we hope that participating groups will be able to collaboratively build and evolve the knowledge structure that they desire for their ongoing deliberations. Evolving structures with user participation was an essential part of the EIES (Electronic Information Exchange System) from 1976 to the mid-nineties [11,16]. Recent experiments and field trials have confirmed the efficacy of using graduate online learning groups in upper division courses oriented to collaborative learning for the total class [17,18]. In the 1975 book we specifically recommended research into the use of Multi-Dimensional Scaling (MDS) as a feedback structure by including an article on that topic. To date there has not been any meaningful use of that method except for the one dimensional approach of Thurstone to expose the subjective group linear scale from individual ranking or paired comparison input. The fact that MDS can now be run on one's personal computer might change that in the future. Also MDS is a perfect method for trying to aid a group to find the underlying dimensions or factors they are using to represent knowledge in their topic. The “Tower of Babel” we often find in the professional literature when crossing disciplinary lines is by no means unique to Delphi, as demonstrated in a special TFSC issue on Strategic Foresight developed by Coates, Durance, and Godet [19]. There is much discussion of Technology Foresight as a new concept developed in the UK in the 1980s by SPRU (Science and Technology Policy Research at the University of Sussex, UK) and others. Actually work that is now defined as technology foresight was done in the 1960s but not under that name. Not surprisingly, the new generation of researchers is often unfamiliar with their predecessors’ work, which may have been done in a different institution or country or in the context of classified national security projects. An historical appreciation of the evolution of a field, as well as codification of terminology, usually comes at a later stage, when maturity sets in and textbooks are compiled. Before proceeding, we should address a persistent misperception about Delphi. Over the years as a reviewer and editor-in-chief of TFSC it has often been necessary to correct the mistaken impression that the aim of Delphi is consensus. Our 1975 book clearly Please cite this article as: H.A. Linstone, M. Turoff, Delphi: A brief look backward and forward, Technol. Forecast. Soc. Change (2010), doi:10.1016/j.techfore.2010.09.011
H.A. Linstone, M. Turoff / Technological Forecasting & Social Change xxx (2010) xxx–xxx
3
states that Delphi is “a method for structuring a group communication process”, not a method aimed to produce consensus. The number of rounds should be based on when stability in the responses is attained, not when consensus is achieved. In fact, a bipolar distribution may be a result and a very significant one indeed. This shows a crucial difference between Delphi and a traditional panel, where consensus is desired and may even be forced. Futurist Joe Coates [20] put the aim of the forecast Delphi as follows: “The value of the Delphi is not in reporting high reliability consensus data, but rather in alerting the participants to the complexity of issues, by forcing, cajoling, urging, luring them to think, by having them challenge their assumptions… [O]ne deficiency I see characteristic of many Delphis is the failure to push hard enough on the challenge to concepts and underlying assumptions. More attention should go into the basis of divergence rather than the basis of convergence.” In the late 1960s the director of OEP, General Lincoln (Eisenhower's logistic general in World War II and a West Point Professor), having heard about ongoing work in using the Delphi Method to examine the future of the U.S. stockpile of strategic materials expressed concern about the fact that bureaucracy often tried to guess what his policy decision would be and to bring him evidence to support that particular view. He explained that what he really wanted to know as a policy maker was: What were the strongest arguments against whatever decision he was likely to choose? He believed this would allow him to better defend his final decision. The result of this was Dr. Turoff designing a Policy Delphi structure to produce the strongest opposing arguments about resolutions to a policy issue, using the properties of a Hegelian Inquiry process as defined by C. W. Churchman [12]. There are three types of measurements of human judgments that are treated as an academic undertaking: 1. Measurements of judgments that are based upon static unchanging psychological variables so that one can understand what the variables are. 2. Measurements of judgments to understand how to influence the marketing of products and services. 3. Measurements of human understanding and judgment to try to determine how to promote a group understanding of mental cognitions by the individuals in the group. These are very different undertakings and objectives and one cannot say that the knowledge and findings of one of these areas automatically apply to the other areas. This has often led to confusion about Delphi and unwarranted criticisms of it. The third objective is clearly what one tries to measure as part of the Delphi process and the most relevant work is in the areas of visualization and information representation design. We are not trying to get a quick subconscious response to a question we ask but to stimulate thought and consideration about what is usually a very complex question. We are also trying to make people aware of the pooling of knowledge and different viewpoints among the participants. 2. The future The Delphi concept with its various derivatives continues to evolve. Online collaborative learning systems constitute an emerging area that can make effective use of Delphi [17,18]. They can be done with a class as a whole using any Computer Mediated Communication System where there is a discussion thread conference system that allows the instructor to assign pen names to each student to participate in the Delphi process. This might also include role-playing “games” as well as enactments of the views of stakeholders represented by the students’ pen names for any multi-sided situation. The instructor, or even one of the students, can collect individual items to be voted on and put them into an online survey system to allow the class to vote every few days on differing options they are considering. Recently some discussion board systems have started to add voting options that would present a more integrated process for carrying out a limited Delphi process. However, the growing number of facilities to carry out a Delphi on the Web has definitely led to the start of a new growth curve in the use of these methods for the support of large collaborative groups. The fact that most professionals and just about all college students do have Web access and their own computers makes possible many applications of Delphi where the computer can handle the organization of the material into a meaningful structure in a dynamic manner without the round structures that characterized the paper and pencil Delphi exercises. Large numbers of proprietary Delphis in organizations have been done with the merger of a discussion based bulletin board system and a survey system. An outstanding challenge is to extend the Delphi concepts to allow large groups to build collaborative group models. An approach that is promising for this is the incorporation of the concept of “structural modeling” [21–23] to allow the users and the group to: 1. Reach consistency in specifying the inputs to a complex modeling situation. 2. Produce outputs from each individual's model, once consistent, which allows the results to be placed into the formulation of a group model. 3. Highlight disagreements or differences of views about the model and use that to ask participants to consider disagreements about such things as the direction of influence between two variables in a relationship within the model. The concept of structural modeling is to provide the user with the ability to make subjective estimates about a problem that can be used in a computer program to build a working model of the person's cognitive model. Equally a group can express comparative inputs about a problem and build a collaborative model. This Delphi process for collaborative model building can be used to evolve the model on a continuous basis for the update of the model as better data becomes available [24]. As with other online Delphis any Please cite this article as: H.A. Linstone, M. Turoff, Delphi: A brief look backward and forward, Technol. Forecast. Soc. Change (2010), doi:10.1016/j.techfore.2010.09.011
4
H.A. Linstone, M. Turoff / Technological Forecasting & Social Change xxx (2010) xxx–xxx
one at any time can input new material or change earlier views to any part of the knowledge structure that defines the scope of the group activity and the process may be integrated into a continuous planning process rather than single plan for X years. It is amusing that one of the first classified Delphis at RAND in the 1960's was specifically designed to replace a computer based simulation model. At the time the U.S. census tapes providing factory floor space (i.e., square feet) as the only measure of production potential by industry type were used to determine the hypothesized targeting of U.S. locations based upon the Russian nuclear threat. A computer calculated the reduction of floor space in a given U.S. industry to arrive at a picture of how bad the results would be. It was felt by some at RAND that this approach was very misleading given examples of, say, only one plant in the U.S. that made huge landing gears for large transport planes. Only one target was needed to make all that manufacturing floor space for large transport planes around the country useless. The famous expression of “garbage in, garbage out” came from many examples of computer models that ignored the knowledge of professionals or their subjective inputs and instead built models around quantitative data sources even when the assumptions behind the available data were not really relevant to the problem being addressed. The original RAND effort above sent a Delphi to 5 to 15 experts in a given industry and asked them to rank order the most critical top ten targets in their industry and explain why each was so important. This turned out much more insightful results than the use of the Census tapes providing only floor space data and much less expensive in those days (e.g. computer processing costs versus the time of people). In most areas of foresight and future forecasting what is important are the details of conditional (IF x THEN y) forecasts which clarify the potential relationships among future events. This yields the information necessary for real planning and the integration of many aspects of planning into one system that provides relationship modeling to determine the most important options for the future [23,24]. The design of the Steel and Ferroalloy Delphi application (4, (chapter III.C.3)) illustrated the challenge of model building. Three experts in this industry were asked to create a commodity flow model of materials. This led to a network of nodes having 45 links. Of these fundamental variables, only nine were collected on a yearly basis. Finding data to make meaningful models is harder than most model builders realize. So we decided to ask the other forty-one industrial experts to take this flow model and fill in the missing estimates for the prior year. Twenty-five of the experts actually made independent changes to the flow model as they did not agree with the three that built the original simplified model of the industry. Dr. Linstone has spent much time since 1977 on the concept of multiple perspectives in addressing complex systems. Two books published in 1971, Churchman's The Design of Inquiring Systems [12] and Allison's Essence of Decision: Explaining the Cuban Missile Crisis [25] as well as personal experience in corporate planning, suggested a means to bridge the glaring gap between traditional systems analysis and the real world. Churchman developed what he called a Singerian Inquiring System.1 Allison examined the Cuban missile crisis from three points of view: rational actor, organizational process, and bureaucratic politics. Each yielded unique insights. In the corporate environment it was found exceedingly valuable to introduce two perspectives to augment the systems analysis approach in examining a complex system [26,27]. A system can be viewed through distinct lenses: the engineer “sees” a proposed new venture very differently than does the CEO in the same corporation. The former focuses on the technical aspects while the latter must be concerned with organizational questions, such as the rivalry between departments when the inevitable reallocation of budgets to support the new system impinges on current activities. Then there are also considerations that reflect the personal career plans for those involved with the new system—who will gain and who will lose? We label the traditional approach the technical or T perspective; the added ones are the organizational/institutional (O) and the personal/individual (P). Each uses distinct paradigms and sweeps in insights not attainable with the others. (For typical perspective characteristics see Table 1)2 Multiple perspectives provide a much more meaningful basis for planning than does the T perspective alone. They are particularly valuable in identifying the differences in assumptions underlying forecasts and that frequently bias them. The categories in Table 1 are general aspects of what you would like participants in a Delphi to be aware of as triggers to stimulate new contributions. Attention to the O perspective will also make it much more likely that the institutional changes that must often accompany technological changes to make them effective will not be ignored. The T-O-P concept underscores an insight derived in complexity science about adaptive systems, namely, that each system element does not see the whole picture but must rely on its own internal models, that is, its own perspective. In other words, ‘rational’ organizational (O) or individual (P) behavior is by no means equivalent to the rational technical perception (T) of the system (and its optimization). Below we discuss two significant considerations that relate this concept to Delphi, experts’ T-focused bias and creating a shared reality: a. Experts’ T-focused bias Engineers and scientists are prone to be too optimistic in the short term and too pessimistic in the long term. This bias is ingrained in their T-focused training. In the short term they feel a new technical solution or concept is feasible, therefore it will be implemented. In other words, if it can be done it will be done. They tend to underestimate organizational or other nontechnical difficulties (funding, political obstruction, etc.) that may impede any realization of the technological concept. In the long term they may not be able to envision a technological solution and assume therefore that there will not be one, reflecting a lack of imagination. The same applies to software developers trying to estimate efforts needed to produce a piece of
1 It is in essence a pragmatic meta-inquiring system that considers the system designer a fundamental part of the system and takes holistic thinking seriously by constantly sweeping in new components. 2 Later a fourth perspective, the religious/mythological (R), was added.
Please cite this article as: H.A. Linstone, M. Turoff, Delphi: A brief look backward and forward, Technol. Forecast. Soc. Change (2010), doi:10.1016/j.techfore.2010.09.011
H.A. Linstone, M. Turoff / Technological Forecasting & Social Change xxx (2010) xxx–xxx
5
Table 1 Typical characteristics of multiple perspective types.
Worldview Objective System focus Mode of inquiry Ethical basis Planning horizon Other descriptors
Technical (T)
Organizational (O)
Personal (P)
Science-technology Problem solving, product Artificial construct Observation, analysis: data and models Logic, rationality Far (low discounting) Cause and effect Optimization, cost-benefit analysis Quantification, trade-offs Use of probabilities, averages, statistical analysis, expected value Problem simplified, idealized
Unique group or institutional view Action, process, stability Social Consensual, adversary bargaining and compromise Justice, fairness Intermediate (moderate discounting) Agenda (problem of the moment) Satisficing Incremental change Reliance on experts, internal training of practitioners Problem delegated and factored, issues and crisis management Need for standard operating procedures, routinization Reasonableness Uncertainty used for organizational self-preservation Institutional compatibility, political acceptability, practicality Preferable value (explicative) normative (prescriptive) participative stakeholders Insider language, outsiders’ assumptions often misperceived
Individual, the self Power, influence, prestige Genetic, psychological Intuition, learning, experience
Need for validation, replicability Conceptualization, theories Uncertainties noted Criteria for “acceptable risk” Logical soundness, openness to evaluation Scenario typology Probable • Criterion analysis (reproducible) • Orientation exploratory (extrapolative) • Mode structural • Creator think-tank teams Communications Technical report, briefing
Morality Short for most (high discounting for most) Challenge and response, leaders and followers Ability to cope with only a few alternatives Fear of change Need for beliefs, illusions, misperception of probabilities Hierarchy of individual needs (survival to self-fulfillment) Need to filter out inconsistent images Creativity and vision by the few, improvisation Need for certainty Risk aversion Possible image (plausible) visionary perceptual individuals Personality, charisma desirable
software. Most experienced managers know this and they then increase the effort needed for short term jobs and leave the long term ones alone! In theory, there is a crossover point for every individual where they predict the effort with complete accuracy. However, it is very likely the location of the crossover point, at which this occurs, may be different for each individual. b. Creating a shared reality T-focused scientists and engineers and their forecasts are not the only justification for drawing on multiple perspectives to assure realism. Policy and decision Delphis must deal with organizations and individuals. A look at Table 1 suggests that the O perspective recognizes factors quite different from those that concern T-oriented analysts, such as an organization's agenda, preference for satisficing, standard operating procedures, political acceptability, problem delegation, and more discounting than is the norm with the T perspective. The P perspective in turn focuses on power, influence, a need for certainty, risk aversion, the counterintuitive nature of probabilities, and long-held beliefs. These perspectives sweep in considerations that underlie Delphi responses. Personal interviews are often the only way to elicit these considerations. They require unique subtlety and sensitivity not generally found in computer processes or T-trained persons. The detailed recommendations for such interviewing in Linstone [27. pp. 290-300] make this very clear. Some sample questions for an astute interviewer: For organizational actors –What are the unstated aims of the group? When and how are standard operating procedures circumvented? Who are the organization's strongest friends and strongest enemies? For individual actors –What are the career objectives of each key actor? Who is a risk taker? Who is the most effective action implementer not shown on the company's organization chart? Answers to such questions may prove exceedingly valuable in determining the makeup of participants in policy or decision Delphis. An example is the case of Perinatal Regionalization in Los Angeles County hospitals, where the multiple perspective analysis was a precursor to a Delphi [27, pp. 284–289]. The aim was to design a regional network organization based on the different perspectives through participative action. Some Delphis have broken down the voting by the participants to show how different stakeholders have voted on the given issues. This can prove to be very significant when dealing with potential actions, decisions, and policies. This is also a useful approach to recommender systems based upon Delphi concepts [7]. In recent years one area of extreme future concerns has become that of Emergency Preparedness and Management. Our success in this area is directly dependent on the behavior of organizations and the individuals in all the parts and levels of the organizations. It cannot be accomplished by a small segment of the organization working alone and it has become apparent that many current assumptions about organizational behavior are not up to current threats [28,29]. In recent years the growing scale and frequency of crisis and emergency situations have shown a complete lack of adequate foresight and planning by all sorts of organizations that should have known better. There are fundamental changes needed to allow approaches like multiple perspective theory, creative risk assessment, future planning, emergency preparedness, and resilient emergency response to take Please cite this article as: H.A. Linstone, M. Turoff, Delphi: A brief look backward and forward, Technol. Forecast. Soc. Change (2010), doi:10.1016/j.techfore.2010.09.011
6
H.A. Linstone, M. Turoff / Technological Forecasting & Social Change xxx (2010) xxx–xxx
place. In one of many recent examples at evacuation civic employees did not show up to drive buses because they could not take their families on the same buses and most highway services were closed leading to clogged highways because there were no provisions made to keep them open. In emergency management the concept of High Reliability Organizations was the subject of LaPorte's memorandum in the Department of Political Science at UC Berkeley in 1989 [27, pp. 184–188). It was also developed in quite some detail by Weick's work with the Navy and popularized in his book [30]. This is the sort of behavioral process that must not only work in organizations but also across organizations. We have not overcome the fact that crises and emergencies do not recognize any organizational, governmental, or geographical boundaries. The response to any large scale situation must cut across all such boundaries and take on the property of Virtuality [14]. This is where the real world changes as a response to what is happening in the virtual world. Social networks are a good example of a system becoming as real to the participants online as it once was to them offline. Even though physical command and control centers failed in 9/11 and in Katrina, we have not learned about the need for integrated systems across all the organizations and individuals that need to be involved. Current disaster real time command and control systems as well as planning systems do not involve most community organizations, volunteer individuals, the press, and volunteer organizations [31]. Conservatively, thousands cooperating and/or collaborating in a Katrina type of crisis in a very timely and urgent manner are what is needed. Current efforts of social networks trying to respond to a disaster situation illustrate the need and the problems that could benefit from the Delphi-like concepts we have been talking about for many years. Social networks only recently introduced groups as a formal entity in their systems but they seem completely ignorant of what large groups need to really collaborate effectively. It is also ironic that in this area we stumble across the same sort of phenomena of rediscovery of what has been understood in the past. The principles of High Reliability Organizations or High Reliability Theory were made very explicit in the 1959 and 1979 papers [32,33] by Lindblom on "The Science of Muddling Through." In the 1979 paper [33], he details the components of one form of muddling through as “disjointed incrementalism.” This matches up point by point with what was later termed as HRO or HRT. He further pointed out in that paper: “A fast-moving sequence of small changes can more speedily accomplish a drastic alteration of the status quo than can an only infrequent major policy change.” (Page 520) What Lindblom believed was that it is fundamental to discover from understanding the past and current situation to determine anything that is wrong. The organization should have a principal goal to understand even the smallest mistakes and to improve the situation. If this is not done, such difficulties would grow and compound one another to bring about bigger difficulties that will no longer be so easy to correct. This also is one of the most fundamental expressions of High Reliability Theory. Both views believe that to accomplish this, determining and expressing the risks and current mistakes should be accomplished by every employee at any level. Their viewpoints have to be taken seriously both for reporting problems and suggesting solutions. Unfortunately for Lindblom the expression “the science of muddling through” was not as acceptable to sponsors from corporations and government as the term “High Reliability Organizations” and represents another factor in the many new names for Delphi that have evolved over time. As with many new paradigms there are always strong negative reactions from those objecting or a paradigm shift. It is still worth reviewing two earlier defenses of Delphi against earlier attacks [20,34]. Today there are many inhibitions in organizations against a free flow of information, especially where bad news is involved. The suggestion box never works because there is no ability of one participant to support the views of other participants and they are hardly ever considered by those that have real authority to make changes. This is why identifying risks and current errors are not done well today in most organizations [29]. There is a real role to be played by a modern online and continuous Delphi system that allows anonymous contributions (or a meaningful choice to be anonymous or use a real name) by anyone in the organizations and the ability for all to reinforce via both voting and comments the views of others. This should feed into a process to improve both current operations and evolving plans for the organization. Ultimately, this will encourage resiliency at all levels in the organization and a far better ability to deal with the unexpected by the organization and its members [7]. For example, there was a story told over drinks by employees of a major computer firm in the early days of message systems. Top management issued a white paper explaining the choice of the technology to be used in their next generation products. Many professionals throughout the organization strongly disagreed with this and a subgroup used the company's internal worldwide message system to write collaboratively a counter document which they circulated to everyone who was issued the original policy document. Top management was so outraged they started to demand that the message system be removed from the network. It took months to convince management that this would completely disrupt many current dispersed efforts. About six months later the original white paper was withdrawn and a new plan incorporating of the ideas in the counter document was issued. There was at least one more company where a where similar example occurred that Dr. Turoff is aware of. However, one can be certain this occurred in many companies after the first introduction of organizational e-mail systems. In the early days electronic message systems were first installed in organizations by the computing services department for use by the computer professionals. Such free software came with the early networking software for ARPA net. Other professionals and managers in other parts of the company learned about this new way of sending memos and messages and some started to use it. It spread by a process of technological diffusion of a useful service within the organization. There was never any really awareness or concern by higher level of management nor was there any consideration of many possible implications and policy issues. The above problem example occurred in slightly different ways in many companies and usually went undocumented. By the time Please cite this article as: H.A. Linstone, M. Turoff, Delphi: A brief look backward and forward, Technol. Forecast. Soc. Change (2010), doi:10.1016/j.techfore.2010.09.011
H.A. Linstone, M. Turoff / Technological Forecasting & Social Change xxx (2010) xxx–xxx
7
top management discovered the problems, the electronic message systems had become too valuable to remove. Some early commercial products limited message sending privileges to reflect the organization structure and eliminated lateral communication sending options. Fortunately, these software versions never became very popular and died out relatively fast [11]. It was the "good old days" when any technological advance was considered good. 3. Conclusions and summary observations In conclusion, the future of Delphi will be in collaborative organizational and community planning systems that are continuous, dispersed, and asynchronous. It will replace the impact of controlled surveys as a mechanism of influencing various organizational and community decision processes. These will allow many thousands to participate or observe an ongoing planning process. However, given the history of this field it is unlikely these systems will be referred to as Delphi systems. One would hope those that develop these systems will pay attention to what we already know about large scale collaboration and will not be relearning and stumbling over the same problems many of us have had to deal with in the past. The greatest impact the Web is having is the fostering a new age: The Age of Participation. Social networks and their growth are one manifestation of that age where the public that has adapted to the internet and the younger generations want to voice their views and feelings about all the matters that concern them. Increasingly, they want to provide the content rather than passively receive content from select organizations and an imposed leadership. Opinion leaders will evolve as part of the recognition of their contributions to the community. The ability to communicate, coordinate, collaborate and fully participate with others is the new online order. Internet-based collaboration is now occurring even in the most abstract areas of science and mathematics. A recent case is the proposed proof of a classic mathematical conjecture, known as “P versus NP”. It states that the set of problems that can be easily solved (P) does not equal NP, the problems that are impossible for computers to solve, but for which solutions are easily recognizable.3 This is of practical importance as modern cryptography is based on the assumption that P does not equal NP. In August 2010 a Hewlett-Packard mathematician circulated his “proof” to a number of complexity theorists. A discussion via blogs and a wiki was quickly set up and discussion and collective analysis was carried out in real time. Web-connected software programs make such collaborative work now feasible among top researchers on a continuous basis. [35]. Today, the use of the Web by professional groups is commonly referred to as "Communities of Practice." Even the academic process of peer review is being challenged. It has been suggested that the internet can serve to extend the review process to a much broader interested audience. In a recent test case, the Shakespeare Quarterly posted four essays not yet accepted on a scholarly web site and 41 people offered more than 350 comments which were transmitted to the authors. The resulting revision was subsequently published. [36] It is our hope that large groups of participants, in the range from social networks to communities of practice, will be able to negotiate and evolve their own virtual reality that they find satisfies their requirements and needs. What they will demonstrate will gradually penetrate more formal organizations. The evolution of both Wikis and Blogs appear to be an early demonstration or trend indicator of what is possible on a much wider scale. The contrast between early articles in the Wall Street Journal on why managers should never be seen using a keyboard, to most managers (and school children) today using their thumbs to key things into their phone almost anywhere, at anytime, gives us some hope we are on the way to the future age of participation. References [1] RAND Report P-2982 (Sept.1964) Report on a Long-Range Forecasting Study by Ted Gordon and Olaf Helmer. (For a sample of the results described in this report, see Linstone (1999), pp. 286-287.} [2] M. Turoff, The design of a policy Delphi, Technological Forecasting Social Change vol. 2 (1970) 149–171. [3] M. Turoff, Delphi conferencing: computer-based conferencing with anonymity, Technological Forecasting Social Change vol. 3 (1971-1972) 159–204. [4] H.A. Linstone, M. Turoff (Eds.), The Delphi method: techniques and applications, Addison-Wesley, Reading, Mass, 1975, (available online at http://is.njit.edu/ turoff). [5] C. White, et al., A dynamic Delphi process utilizing a modified Thurstone scaling method: collaborative judgment in emergency response, Proceedings of ISCRAM 2007, 4th International Conference on Information Systems for Crisis Response and Management., Delft, the Netherlands, Brussels University Press, May 13-16 2007. [6] Murray Turoff, The past, present, and future of Delphi, FUTURA, The Quarterly Journal of the Finnish Society for Futures Studies, Helsinki, 2009, pp. 32–44, 0785-5494. [7] M. Turoff, S.R. Hiltz, The future of professional communities of practice, in: C. Weinhardt, S. Luckner, J. Stößer (Eds.), WeB 2008. LNBIP, vol. 22, SpringerVerlag, Berlin Heidelberg, 2009, pp. 144–158. [8] T. Gordon, A. Pease, An efficient, “round-less” almost real-time Delphi method, Technological Forecasting Social Change vol. 73 (2006) 321–333. [9] J. Fjermestad, S.R. Hiltz, An assessment of group support systems experimental research: methodology and results, Journal of Management Information Systems (JMIS) 15 (3) (Winter 1998-99) 7–150. [10] J. Fjermestad, S.R. Hiltz, Group support systems: a descriptive evaluation of case and field studies, Journal of Management Information Systems 17 (3) (Winter 2000) 112–157. [11] S.R. Hiltz, Murray Turoff, The Network Nation: Human Communication via Computer, Revised EditionMIT Press, 1993 (original edition 1978, Addison Wesley). [12] C.W. Churchman, The Design of Inquiring Systems, Basic Books, New York, NY, 1971. [13] Murray Turoff, Computer mediated communication requirements for group support, Journal of Organizational Computing vol. 1 (1) (1990). [14] Murray Turoff, Virtuality, CACM vol. 40 (9) (September 1997) 38–43.
3
It is one of the “Millennium Problems” chosen by the Clay Mathematics Institute in 2000 as the seven greatest unsolved mathematics problems.
Please cite this article as: H.A. Linstone, M. Turoff, Delphi: A brief look backward and forward, Technol. Forecast. Soc. Change (2010), doi:10.1016/j.techfore.2010.09.011
8
H.A. Linstone, M. Turoff / Technological Forecasting & Social Change xxx (2010) xxx–xxx
[15] S.R. Hiltz, Murray Turoff, Structuring computer-mediated communication systems to avoid information overload, Communications of the ACM July 28 (7) (1985) 680–689. [16] Murray Turoff, S.R. Hiltz, A.N.F. Bahgat, Ajaz Rana, Distributed group support systems, Management Information Systems Quarterly (MISQ) (December 1993) 399–417. [17] Turoff, M., Hiltz, S.R., Li, Z., Wang, Y., & Cho, H., “The Delphi Process as a Collaborative Learning Method,” in Moore, J.C. (ed.): (edited by J. C. Moore) Elements of Quality Online Education: Into the Mainstream: Wisdom from the Sloan Consortium, 121-134. Needham, MA: Sloan-C, Needham, MA September 2004, pp 121-134. [18] M. Turoff, Hiltz S.R. Hiltz, Yao X. Yao, Li.Z. Li, Wang Y. Wang, H.K. Cho, Online Collaborative Learning Enhancement Through the Delphi Method, 2006 available at:, http://tojde.anadolu.edu.tr/index.htm. [19] J.F. Coates, P. Durance, Godet, “Strategic Foresight” issue, Technological Forecasting Social Change vol. 77 (9) (2010). [20] J.F. Coates, “In Defense of Delphi: A Review of Delphi Assessment: Expert Opinion, Forecasting, and Group Process” by H. Sackman, Technological Forecasting Social Change 7 (1975) 193–194. [21] H.A. Linstone, et al., The use of structural modeling for technology assessment, Technological Forecasting Social Change vol. 14 (1979) 291–327. [22] G.G. Lendaris, Structural modeling—a tutorial guide, IEEE Transactions on Systems, Man, and Cybernetics SMC-10 (12) (1980) 34. [23] Murray Turoff, Delphi and its potential impact on information systems, AFIPS Conference Proceedings, vol. 39, 1971. [24] Bañuls, V. A., Turoff, M., and Lopez, J., Clustering Scenarios using Cross-Impact Analysis, Published in the proceedings of ISCRAM 2010, Seattle, May (available at iscram.org). [25] G. Allison, The Essence of Decision: Explaining the Cuban Missile Crisis, Little, Brown & Co., Boston, MA, 1971. [26] H.A. Linstone, Multiple Perspectives for Decision Making, , 1984 New York: North-Holland. [27] H.A. Linstone, Decision Making for Technology Executives: Using Multiple Perspectives to Improve Performance, Artech House, Norwood, MA, 1999. [28] M. Turoff, S.R. Hiltz, C. White, L. Plotnick, A. Hendela, Y. Xiang, The past as the future of emergency preparedness and management, Journal of Information Systems for Crisis Response and Management 1 (1) (January-March 2009) 12–28. [29] B. Van de Walle, M. Turoff, Decision support for emergency situations, in: F. Burstein, C. Holsapple (Eds.), Handbook on Decision Support Systems, International Handbook on Information Systems Series, Springer-Verlag, 2008, (This chapter is open on the Web). [30] K. Weick, K. Sutcliffe, Managing the Unexpected: Assuring High Performance in an Age of Complexity, Jossey-Bass Publishers, 2001. [31] Information systems for emergency management, in: B. Van de Walle, M. Turoff, S.R. Hiltz (Eds.), Advances in Management Information Systems monograph series (Editor-in-Chief: Vladimir Zwass), M.E. Sharpe Inc, Armonk, NY, 2010, Chapter 16. [32] C. Lindblom, The science of muddling through, Public Administration Review (1959) 79–88. [33] C. Lindblom, Still muddling, not yet through, Public Administration Review (November/December 1979) 517–526. [34] P.G. Goldschmidt, Scientific Inquiry or Political Critique: Review of the Sackman Report, Technological Forecasting and Social Change 7 (1972) 195–213. [35] New York Times, August 17, 2010, p. D-2. [36] New York Times, August 24, 2010. Harold Linstone is University Professor Emeritus at Portland State University and founding editor-in-chief of this Journal. His most recent book is Decision Making for Technology Executives: Using Multiple Perspectives to Improve Performance (Artech House, 1999). Prior to his academic position he was Associate Director of Corporate Planning—Systems Analysis at the Lockheed Corporation. His M.A. and Ph.D. degrees are in Mathematics. Murray Turoff is a Distinguished Professor Emeritus at the New Jersey Institute of Technology. He is a co editor of a recent book on Emergency Management Information Systems (M.E. SharpE 2010). Besides his early and continuing work with the Delphi Method, he spent most of academic research career in the design and evaluation of Computer Mediated Communication systems. After 9/11 he turned his attention back to his early work in Emergency Management and in 2004, he was a cofounder of the international organization ISCRAM (Information Systems for Crisis Response and Management).
Please cite this article as: H.A. Linstone, M. Turoff, Delphi: A brief look backward and forward, Technol. Forecast. Soc. Change (2010), doi:10.1016/j.techfore.2010.09.011
