Decision-making processes evaluation using two methodologies: field

Theoretical Issues in Ergonomics Science Vol. 6, No. 1, January–February 2005, 35–48

Decision-making processes evaluation using two methodologies: field and simulation techniques G. ROLO* and D. DIÁZ-CABRERA Dpto. de Psicologı´ a Cognitiva, Social y Organizacional, Universidad de La Laguna, Facultad de Psicologı´ a, Campus de Guajara 38205, Tenerife, Spain

The study of information processing and human actions in complex environments requires the use of methodological tools that enable one to comprehend the complexity of context without sacrificing methodological accuracy, control of variables or results generalization. Research methods that allow the study of strategies that people use when they make decisions are, among others, field studies and simulated micro-worlds. The aim of this paper is to present the main theoretical and methodological conclusions obtained in two studies centred on decision-making processes. The authors were interested in evaluating the results obtained both in a previous field study developed with experts in process control tasks and in a simulated study with inexperienced people. In the field study, a verbal protocol technique was used, while in the simulated study a simulated dynamic task was developed which contained the main features of control process tasks. Among other factors, the authors are interested in exploring the effect of expertise with the task, the learning strategies and the ways of task solving. The results of these two studies indicate some characteristics of a good performance such as the importance of information searches; the taking into account of delays in the effects of the actions; the anticipation of possible system changes. Keywords: Decision-making processes; Simulation technique; Field study; Research methodology.

1. The study of dynamic decision-making The knowledge of how people process information, solve problems, make decisions or cope with goals in real work environments requires the use of diverse and complex technological tools. This is one of the most interesting subjects in cognitive ergonomics as well as in work and organizational psychology. Decision-making is a complex psychological activity stemming from complex cognitive, motivational and voluntary processes and it is influenced by multiple physical, psychological and social factors. The decision-making theory has been mainly developed around three disciplines: Economics, Mathematics and, more recently, Psychology. There are three significant stages in decision-making research. In the first stage, the decision studies are characterized by an economic and mathematical approach. In the second stage, *Corresponding author. Email: [email protected] Theoretical Issues in Ergonomics Science ISSN 1463–922X print/ISSN 1464–536X online # 2005 Taylor & Francis Ltd http://www.tandf.co.uk/journals DOI 10.1080/14639220512331311544

36

G. Rolo and D. Dıáz-Cabrera

there is a clear influence of cognitive orientations related to the development of Information Processing Theory. The last stage is centred upon decision-making in naturalistic contexts. In the same way, Cohen’s (1993) classification groups the decision-making studies into three categories: (1) formal-empirical paradigm; (2) rationalist paradigm and (3) naturalistic paradigm. The studies presented in this paper are framed in the naturalistic decision-making approach. This is a recent approach and its aim is to obtain a higher quality of decision through a comprehension of decision-making processes in natural contexts. The first critical studies in the context of this approach were developed in the 1980s. The results of these studies demonstrated that decision-making is not only selecting an action line, but it implies a global cycle of decision. In order for decision strategies to work in natural contexts, the intuitive process is important in the decision process and it is essential to take into account the evaluation of the decision-makers about the whole situation (Cohen 1993, Klein 1993b). Some theories have been developed from these assumptions, such as, for instance, Recognition-Primed Decisions (RPD) Model (Klein 1993a); Image Theory (Beach and Mitchell 1987); and the Cognitive Control of Decision-Making Model (Rasmussen 1981). These authors pointed out the limitations of the classic decision-making models and the new models that were developed demonstrated how the decision-makers use their experience to cope with natural context. An important starting point, as a relevant difference from the classical studies (centred on simple tasks), was the consideration of the naturalistic context or environmental characteristics. The realistic settings incorporate characteristics such as uncertain situations associated with incomplete and, frequently, contradictory information; decision-taking time pressures; feedback delays after the actions; shifting goals; dynamic continually changing conditions; competitive goals (Cannon-Bowers et al. 1996). Some theoretical notions of this naturalistic approach are in contrast to classical decision theory notions (see for revision, Orasanu and Connolly 1993). In the first place, there is a notion that people in real situations generate a single response with a high probability of solving the problem. This notion is in contrast to the person’s rational image or concept that tries to look for the best course of action. In many complex and ill-defined situations there is more than one correct response and it is difficult to find the best action. Secondly, another notion is concerned with the distinction between the situation diagnosis function and the course of action selected function. That is, when decisionmakers are confronted with a specific situation, they must first evaluate what is happening, which kind of problem has to be dealt with, what consequences could be derived from this situation and, finally, what action will be performed. The context that surrounded this situation, task or problem to be solved plays a very relevant role. People must evaluate and bear in mind the different and multiple aspects (social, organizational, technical, etc.) of these contexts during the whole decision process. The development of planning helps and guides operators in the search for essential information, in the identification of potential activities and in error prevention, as well as in the management of new or unexpected situations (Xiao et al. 1997). A third notion implies that reasoning and acting are intermingled. Such a type of decision cycle would, therefore, suppose that the decision-maker thinks a little, acts a little and then evaluates the outcomes and then thinks and acts again. The decision cycle or sequence embraces the decision events developed by the individual to

Decision-making processes evaluation

37

cope with the task or situation. More specifically, it implies the search for information, the determination of cause–effect relationships, ongoing evaluation, actions performed, etc. In the framework of this naturalistic approach, the objective of this paper is to present the main conclusions and theoretical and methodological reflections that stem from research in the decision-making area. Specifically, this paper refers to two studies on this topic: a field and a simulated study (Dı´ az-Cabrera and Rolo 2000, Rolo and Cabrera 2000), contrasting the two. Concretely, an objective of the simulated study presented in this paper was to evaluate decision-making processes by participants with no previous experience of these specific systems and, consequently, without a mental representation of the tasks’ internal structures. The authors were interested in comparing the results obtained in a previous study with experts in process control tasks in a refinery (Rolo 1996). In such real-world settings, the same task will vary from situation-tosituation because of internal and/or external factors (climate conditions, production pressures, technical aspects, relationships between different technical sub-systems, etc.). As a consequence, the studied situation will be somewhat different for each participant and, therefore, the strategies used by each of them. This situation created for the refinery context pointed out some questions: Can these characteristics of the real world influence the generalization of the findings? Are the individual differences the causes of the obtained variations about the strategies and decision sequences developed by the participants? Or are there effects of task conditions in each situation? How does the experience affect the task performance? Can one obtain similar patterns in decision-making processes in a participant without experience? Mainly for this reason, in the study presented here, a simulated situation was designed with the aim of ensuring that all the participants had to deal with the same variables and task conditions. The first study, the field study, was carried out in an automated process control organization, specifically a petrol refinery. One was mainly interested in evaluating three objectives of decision-making in process control using a qualitative methodology: first, the influence of the context and the associated demands required of the operators; secondly, the identification of a global planning pattern developed by the operators; and thirdly, the analysis of the decision sequence patterns in the studied tasks. Two tasks were selected from the distillation console in the petrol refinery. In the second study, a simulated study, the main objective was to analyse decision-making processes by participants with no previous experience of these specific systems. One was interested in comparing the results obtained in the field study and in determining the personal strategies that lead to a successful outcome in the task (Dı´ az-Cabrera and Rolo 2000).

2. The field study Perhaps one of the work environments that reflects the importance of the operators performance in a high technological complexity is the process control industry, for example: power generating plants, chemical industries, dairy companies, etc. Process control involves the supervision of a partly automated and continuous production process, as well as the making of necessary adjustments and the rectification of faults. There are significant cognitive demands made on the

38


operators. They have to supervise the temporal tendency of broad groups of variables and their inter-relationships with the aim of inferring the present and future state of the system. They must also take into account the characteristics of the product, the technical system and the organizational goals. Using the above data, they have to review potential actions and their effects to select appropriate actions and make the response. Process control organizations are useful contexts in which to study diverse cognitive processes because the tasks are performed in complex feedback systems which incorporate some relevant characteristics like: (a) dynamic tasks and situations requiring fast changes during the decision process; (b) feedback loops, so that every action can give a new alternative; (c) the possibility of diverse satisfactory solutions; (d) temporal pressures; and (e) antagonistic goals (Orasanu and Connolly 1993, Kaempf and Klein 1994). With the aim to study the significant characteristics of the decision-making process in process control tasks, a first study was developed on decision-making processes in continuous process control systems’ operators (Rolo 1996). By the process complexity that workers developed (simultaneous vigilance, control and maintenance of the automatized process of petrol distillation), the concurrent verbal protocol technique was used in this first investigation, similar to that proposed by Bainbridge (1974, 1979), with the aim of extracting a possible higher quantity of information during the operators’ execution task. Then, the analysis of verbal protocols enabled one to identify the main plans and tactics developed by the control operators studied and to identify some decision sequences (Rolo and Cabrera 2000). Verbal protocols were categorized by 12 raters, all lecturers in social and cognitive psychology based on a list of categories referring to several cognitive processes. Flow diagrams taken from the list of cognitive categories were made for each verbal protocol. These diagrams reflect the plans and tactics developed by operators, the specific steps in the sequence of task performance defined by the implied process (evaluation, prediction, action, etc.) and the order of the steps in these sequences. The verbalizations show two types of global strategies: ‘step-by-step sequence’ and ‘global sequence’. As the results show, most operators agreed on the strategy to be used in every task. Task 1 (To turn a unit furnace temperature down 2 C and maintain the stability and production level of the unit) was performed using the ‘step-by-step strategy’, while in Task 2 (To switch off two fuel burners of a unit furnace, while maintaining furnace, gas and fuel consumption and pressure, as well as stability and production level of the unit) they used the ‘global strategy’. In every task only one operator (a different person in each case) chose another strategy from his co-workers. An initial global situation assessment was the starting point that determined the choice of whether to cope with the task through one or another strategy and this assessment is mainly influenced by the type of operation or task and its context. Another finding is related to the two types of decision sequences obtained from the verbal protocols. The first one stems from the necessity to solve a problem (e.g. increase of pressure), while the second responds to the prediction and anticipation requirements of these systems. The first sequence was developed where the unit operator had to cope with a problem: when they begin to perform a specific operation in the system and/or when they change the alarm levels. Here the diagnosis of


39

the problem is at a crucial stage. The second decision sequence is more typical of normal adjustment situations; that is, situations in which the operators do not have to cope with substantial alterations in the system variables and their main task consists of performing small actions in order to maintain the system within normal given parameters. Anticipation and prediction are the most critical aspects of this decision sequence. The most relevant results of the research reflected some key aspects. First, the importance of familiarity with the task and the operators’ experience in the cognitive representation development as well as in the individual’s ability to anticipate changes in the system. Second, the difficulty to analyse the consequences of decision-action because of the multiple interaction between variables that make it difficult to estimate the secondary or collateral effects produced by the actions made. Third, the context influence in real situations of the tasks executed that comprehend a diversity of aspects like the task to be completed, technical systems characteristics, other parallel functions or production operations, as well as different organizational and extraorganizational factors. Fourth, the relevance of individual characteristics of operators to establish the optimal level of each parameter after his assessment of the situation and, at the same time, with respect to the kind of operators’ global planning of the task. In spite of the difficulty of studying the decision-action sequences in complex areas, this first investigation permitted one to extract a number of basic characteristics of decision-making processes in automatized process control tasks. These basic characteristics made it possible to develop different questions and hypotheses about the decision processes in complex and dynamic environments. Likewise, central aspects in any research are considered. On the one hand, is it possible to make generalizations from the results obtained in this specific area of analysis? On the other hand, do the obtained results represent general laws of decision-making behaviour? At the same time, the findings obtained in the field study indicated significant differences between operators in the problem-solving strategies. Specifically, individual differences were found in the type of global planning pattern used by the operators in the task performance, but also, and very significantly, in the strategies developed to cope with the alarms. In that sense, a question arose: what type of personal strategies facilitate a successful outcome in the simulated task? The answers to these questions let one take into account the necessity to develop a second study. At this time, the task simulation technique was used that allows one to analyse in detail the psychological processes implied in decision-making and the influences of individual differences in the task performance in a large number of people. Moreover, the simulation permits one to develop a more difficult task where an unsuccessful outcome could be possible without deriving important risks and accidents in the organization operations, as is the case in field studies.

3. The simulation study A simulated task was developed using the facilities of laboratory computers to create a situation that reflected some of the main features of the process control environments. This ‘micro-world’, as Brehmer (1992) and Do¨rner (1987) termed it, would permit one to study, in more controlled conditions, some cognitive processes used by

40


subjects when they are confronted with complex and dynamic tasks. The microworld refers to dynamic computer-generated environments that simulate some of the essential features, but not all the details of a dynamic system. This methodological tool permits one to analyse how subjects interact within the laboratory and within simulated conditions encountered in the field (Brehmer 1992, DiFonzo et al. 1998). A simulated micro-world facilitates, as Brehmer (1993) pointed out, not only the control of the situation task but also enable one to (a) give the possibility of designing task-specific characteristics in accordance with the research objectives; (b) eliminate risks produced by errors and the learning process; (c) collect more types of data; (d) use a bigger sample of participants with less time limitations; and (e) have the potential of a higher capacity of generalization than other situations. 3.1 Objectives The design of the simulated task attempted to respond to a main objective: to evaluate the decision-making process in a more controlled context, specifically, to analyse how people generate a decision–action cycle to solve a task in a particular situation. In this respect, the importance of the number of steps that they made during the task execution was considered. At the same time, one tried to study the relevance of diverse characteristics that appear to be in the base of an optimal execution such as: (1) the relevance of the feedback system, as much as the information delays, in relation to the quantity of information used and the number of actions made; (2) the role of the anticipation of the problem, taking action before the alarms appear that indicated the instability of variables to control; and (3) the search for relevant information, when the alarms appear, with the object of recovering the system’s stability. Moreover, it was planned to study how the decision-making process would be in people that had no previous knowledge of the system and, therefore, had no knowledge of an internal structure representation of the simulated system. This implied that all people would have the same condition (no-knowledge) and they would have acquired the necessary information at the same time that they had solved the task. It was important to develop a clear representation of the situation and to know their characteristics in order to act with some efficiency. 3.2 The simulated task In this study, a task was designed that simulated a vigilance and control system, in which the real environment was reflected in the most precise way. In the global planning of this task the words used by Brehmer (1993) were useful in how microworlds should be defined to reflect a simulation: opaque, complex and dynamic. Following these guidelines, a task was designed that simulated a dynamic system of control and vigilance of a patient with health problems (Dı´ az-Cabrera and Rolo 2000). Since the specifics of the process control task of petrol distillation were analysed in the first study, it was decided to design a task that would imply the control of familiar variables to all people, like temperature, arterial pressure and red blood cells. All of these factors are ‘common’ in everyday lives. Moreover, to design the task, some specifications were taken into account such as: inter-relationships between variables, delayed effect of actions and autonomous evolution of the


41

system, as well as generated changes by the operators’ actions. The alarms indicated unstable variables and important consequences of the operators’ decision-action process. The object of these design specifications was that the simulated system would respond to the main characteristics of process control. A dynamic system of control and supervision of a patient with health problems was simulated. The patient experienced fever, hypertension and red blood cell deficiency; all conditions that needed to be controlled. There were four types of medication to control these variables: antibiotic and anti-thermic to reduce temperature; iron supplements to increase red blood cells; and a diuretic to reduce arterial pressure. These medications interacted producing joint effects. Thus, antibiotics reduced the fever but also reduced the level of red blood cells; the anti-thermic not only reduced the temperature, but also produced an increase in arterial pressure. The simulated system responded slowly, that is, after medication was administered there was a delay of the control variables. However, when various doses of any medication were given at the same time, or over a short period of time, the capacity and speed on critical variables increased. Table 1 shows the dependent variables used in the simulated task. The system also had visual alarms in order to warn of critical situations when the variables exceeded the normal range. Nevertheless, information was not automatically provided regarding which variable was acting abnormally and, therefore, the subject had to look for the problem variable when an alarm was perceived. The task took 9 min and it ended with an indication of the patient in critical condition or otherwise, depending on whether subjects had been able to get and control stability of temperature, arterial pressure and red blood cells during the task. The simulated task and questionnaire were individually presented by computer. It permitted the subjects sufficient time to read the task instructions and to work at their own speed on the questionnaire. The order of presentation was as follows: the screen showed a brief study presentation and description; and, afterwards, the general information questionnaire. Once the subjects had answered this questionnaire,

Table 1. Information request

Actions

Results

Description of measured variables in simulated task. Number of information requests: number of times that the subject asked for information about the state of variables Number of actions: total number of doses of each type of medication given to the patient Average of doses: number of doses of each medication divided by the total number of actions Total number of steps: sum of the number of information requests and number of actions Time of first alarm: time of first alarm appearing from task starting Number of alarms: total number of alarms during the system regulation Number of controlled alarms: number of alarms that the subject controlled System stability: total time without alarms in which the system was regulated Performance: final result of the simulated task, success or failure.

42


Figure 1.

Example of simulated task report.

the instructions of the simulated task were presented and the subjects then began the task. Figure 1 is an example of a task report including the dependent variables used in this study as generated by the computer program. The order of presentation of the data in this report is as follows: first, personal data and level of computer experience; second, a description of every step taken by the participant; third, result obtained; and fourth, global data of the simulated task.


43

3.3 The findings The most interesting results obtained in this research are related to those characteristics of the decision-making process that enable one to discriminate between subjects that end the task with success to those that fail in the task execution. It is important to point out that all the study participants began the task in the same condition of no-knowledge of the system. This absence of knowledge implied the necessity to obtain information about the system and its possible development and, afterwards, to generate an hypothesis about the internal structure and elaborate causal relationships of the situation. The internal representation development should be made at the initial moment of the task, that is to say, the subjects must be learning through their execution of the task. The capacity to create and maintain this mental representation of the system, and from here to move through the variables network is a requirement in the decision-making process. This system internal structure is an essential condition of a good performance, especially in this type of control process task. It permits one to foresee the side effects and to determine what actions can have other effects, moreover, of those one is trying to find. For this reason, the subjects needed to know the specific interactions between variables. The task instructions of the study indicated that the variables could have side effects, although the necessary quantity of each medication that produces these side effects is not specified. A qualitative analysis of the protocols generated by computers as well as bivariate correlations of the variables were carried out. A first result indicates there is no best way of developing the task. In contrast, the protocols show different task procedures, some successful and others unsuccessful. In fact, some participants maintained the stability of the system with a low number of information and actions, while others needed more numbers of steps. The results obtained in this second study allow one to identify a set of characteristics that enable one to distinguish between ‘good’ or ‘bad’ decision-makers. The successful decision-makers: (a) looked for information about the state of variables (they acted less and requested more information), (b) they conducted necessary actions slowly, (c) they waited for the effects of their actions (including side-effects), and (d) they anticipated system changes for maintaining system stability in order to avoid or control the alarm system quickly. More specifically, common patterns of behaviours in the participants who had success were: (1) starting the task looking for information from the three variable states before administering medication; (2) spacing the administration of medication and no more than two doses each time; (3) looking for information about the critical variable immediately before and/or after to give the medication; (4) assuming feedback delays, that is, they took time in order to observe the evolution of the variables before administering new doses of medication; and (5) in the case of alarms, the participants searched for information of the specific variable, then they administered the correct medication and continuously supervised the critical variable until the alarm disappeared. These results point out the relevant role played by the information acquisition and the capacity of waiting for the system feedback in the successful participants. In contrast, the performance of subjects without task success moved away from the above characteristics: (a) more actions and less information searches; (b) nonanticipation of alarms; (c) not waiting for the effects; and (d) not taking into account

44


side effects. Most of these characteristics can be interpreted in terms of the ‘faulty behaviour classification’ of Do¨rner (1990). Some of these subjects seemed to act in a random way. So, one can explain how some subjects, when the alarm appeared on the display, were given doses of medication without looking for information in the system to check what variable was outside of its normal range. Perhaps, as Do¨rner and Schaub (1994) affirmed, being capable of doing something in difficult situations, although it is not the correct action, produces good feelings—in Do¨rner’s (1990) words, an illusion of competence. There are some limitations implicit to the study of the decision-making process. The capacity to obtain a global idea (or an image of strategies developed by subjects) requires a collection and analyses of data as related sequences of steps, instead of discrete units. In this type of process, there is a predominant sequence built by the subjects and every step produces a different combination of possible alternatives, similar to a chess game. However, until now, one does not have a well-developed statistical analysis that permits one to evaluate these kinds of sequences. Perhaps, joined with the qualitative methods normally used in these cases, the methods proposed by Howie and Vicente (1998) represent an advance in the analysis and interpretation of the data obtained in these studies.

4. Conclusions These results enable one to extract a set of conceptual and methodological conclusions. From a conceptual perspective, first, decision-makers have some specific characteristics in dynamic environment task solving: (1) they have a good cognitive representation of the problem or task; (2) they have the ability to predict changes that would be produced in the system; (3) they anticipate the future effects of their own actions; (4) they know the feedback delay in this kind of process control; and (5) they have developed familiarity and experience with the situation and the task that they are trying to solve. Second, it is important to emphasize that the decision cycle is modified during the task solving process. In that sense, decision-makers continuously assess their actions and the system state. Third, another relevant aspect in the decision-making processes is the role of the context. Context is important in a dynamic environment because multiple feedback loops back and forth between the decision-maker and the system is developed. Most of these conceptual aspects can be observed in both field and simulated studies. Specifically, the two decision sequences were obtained in the two studies. A first decision sequence is developed when the participants have to cope with a problem in the task or in the system’s instability. In that decision cycle, the diagnosis of the problem is the most important requirement and strongly demands the search for information about the variables states. The participants who had success in the simulated task looked actively for information about the system state or the type of alarm presented at that time. Similarly, the information acquisition from the operators of the refinery represented a central aspect of their task performance. The second decision sequence is developed in a normal adjustment situation and its main goals are the anticipation and prediction with the aim to maintain the system within the normal parameters. In this research, the ‘good’ decision-makers in the simulation study and the operators of the refinery emphasized the system stability in an attempt to avoid the presence of alarms.


45

Another conceptual aspect observed in both studies is related to the need of assuming the feedback delays typical of dynamic systems. The ‘good’ decisionmakers of the simulation study adequately assumed the delays, that is they waited for the effects of the doses of medication and they were not, apparently, very affected by the anxiety produced by the alarm sound. In the same way, the refinery operators took on the delays of the effects after specific actions. Thus, for example, they pointed out: ‘The temperature will rise again, but it will go down’. These delays set important cognitive demands such as recall and remembering: (a) past actions in order to supervise and interpret the system’s instability; (b) to decide a time interval in which they assume the effect will be produced; and (c) to distinguish provisional from final effects (see the above example). Likewise, from the results obtained in these studies, one can extract some methodological conclusions. The verbal protocol technique used in the field study permits one to obtain a rich and great number of qualitative data. This was important in reaching the main results and conclusions, but evidently this is a technique that requires a great quantity of time for the verbal reports transcription, categorization of phrases, flow diagram development and results evaluation (Howie and Vicente 1998). Perhaps the time consumed is bigger than the quantity of data obtained that can be generalized and/or can be refused in other similar studies (Brehmer 1992). Nevertheless, field research offers, moreover, a large amount of information, a level of complexity linked to the personal, organizational and technological aspects of the context that are difficult to apprehend in a simulated task. In this sense, complexity is a basic aspect in the working conditions based on human–technology interactions like in process control operations. As Meister (2000) affirms, assuming that change in the complexity of human–technology relationships is important, it is not known how these changes are manifested or how they affect both humans or technological tools. Complementing field studies with more laboratory-based methods, such as the simulated research, may be useful in finding a global comprehension of this phenomena. The simulated task used in the research is perhaps limited in respect to reliably reflecting some complex environmental characteristics. In that sense, the field in contrast to the simulation study generates relevant differences related to the demands produced by the huge number of variables to be controlled, the time pressure due to having to deal with other tasks at the same time, the high inter-dependence among variables and the important consequences for the worker’s safety and well-being, the optimum level of technical system and the organizational performance. A difference to take into account is the high level of expertize of refinery operators in contrast to the participants in the simulation. In future simulations research it would be convenient to increase the number of essays to give the participants the opportunity to acquire more task familiarization and experience. Another distinction between both studies is the time limitation (9 min) to perform the task in the simulation. Nevertheless, the refinery operators also had time pressures produced by the demands of concentrating as soon as possible in other operation tasks. In spite of the difficulties of a task simulated process design, the richness, the amount of data, the data reliability and the validity that is possible to obtain with a simulation technique compensates for all previous disadvantages. A future stage in the simulation studies would be to continue to progress so that the computer task simulation would enable one to represent more complex and dynamic environments in experimental conditions without losing the nature of real environments.

46


The micro-world is a broad and powerful methodology that can be used for different purposes in diverse research areas, for example: (1) in complex and dynamic phenomena (Brehmer et al. 1991); (2) in training (Hukki and Norros 1998); (3) in a diagnostic medical domain (Guerlain et al. 1999); etc. One of the advantages of micro-worlds, following the words of DiFonzo et al. (1998), is to offer superior levels of experimental realism (i.e. subject involvement) and mundane realism (correspondence with the field experience). The second advantage in using this simulated methodology is that it is a moderate means of gathering high-quality data. A third advantage of the simulated micro-world is that the designers are compelled to describe in detail all the mechanisms and, when the simulation is developed, it can be renewed and adapted to its applications in different areas of research (Roth et al. 1992). In spite of all the advantages mentioned above, the simulated micro-world needs more development to help to improve this methodology. In that sense, it is necessary to take into account variables that are present in real dynamic environments that, perhaps, increase the levels of realism and at the same time promote the internal and external validity of the methodology, as DiFonzo et al. (1998) pointed out. In actual situations in work dynamic environments, many internal and external factors such as emotion, personality, human relations, shared knowledge and responsibility, arousal level, working environment, etc., may influence the cognitive process and finally the decision-making and the resultant behaviour of an operator (Hasegawa and Yoshimura 1999). As Dubey (2001) said, many times the success or failure of a simulation may depend on social issues that are presented in the research environment. The authors think that, for a comprehensive and wide explanation of the operators’ behaviour, it is necessary to introduce and control these socioemotional variables in the future developments of micro-world methodology. At the same time, this would contribute to reaching the necessary ecological validity to transfer findings from these experimental situations to real work situations, as Hoc (2001) pointed out. Finally, future research should consider the above type of approach at the moment of designing the simulated micro-worlds with the aim of improving the validity and generalization of the study findings obtained that permit the reflection of natural contexts with maximum realism.

References BAINBRIDGE, L., 1974, Analysis of verbal protocols from a process control tasks, In The Human Operator in Process Control, edited by E. Edwards and F.P. Lees, pp. 146– 158 (London: Taylor & Francis). BAINBRIDGE, L., 1979, Verbal reports as evidence of the process operator’s knowledge. International Journal of Man-Machine Studies, 11, 411–436. BEACH, L.R. and MITCHELL, T.R., 1997, Image theory: principles, goals and plans. Acta Psychologica, 66, 201–220. BREHMER, B., 1992, Dynamic decision-making: human control of complex systems. Acta Psychologica, 81, 211–241. BREHMER, B., 1993, Cognitive aspects of safety. In Reliability and Safety Hazardous Work Systems. Approaches to Analysis and Design, edited by B. Wilpert and T. Quale, pp. 23– 42 (Hove: Lawrence Erlbaum Associates Publishers). BREHMER, B., LEPLAT, J. and RASMUSSEN, J., 1991, Use of simulation in the study of complex decision, In Distributed Decision Making. Cognitive Models for Cooperative Work,


47

edited by J. Rasmussen, B. Brehmer and J. Leplat, pp. 373–386 (New York: John Wiley & Sons). CANNON-BOWERS, J.A., SALAS, E. and PRUIT, J.S., 1996, Establishing the boundaries of a paradigm for decision making research. Human Factors, 38, 193–205. COHEN, M.S., 1993, Three paradigms for viewing decision biases, In Decision Making in Action: Models and Methods, edited by G.A. Klein, J. Orasanu, R. Calderwood and C.E. Zsambock, pp. 36–50 (New Jersey: Ablex Publishing Co). DIÁZ-CABRERA, D. and ROLO, G., 2000, Toma de decisioń en una tarea simulada de control de procesos. Revista de Psicologıá del Trabajo y de las Organizaciones, 16, 23–44. DIFONZO, N., HANTULA, D.A. and BORDIA, P., 1998, Microworlds for experimental research: having your (control and collection) cake, and realism too. Behavior Research Methods, Instruments & Computers, 30, 278–286. DO¨RNER, D., 1987, On the difficulties people have in dealing with complexity, In New Technology and Human Error, edited by J. Rasmussen, K. Duncan, and J. Leplat, pp. 97–110 (New York: Wiley and Sons Ltd). DO¨RNER, D., 1990, The logic of failure, In Human Factors in Hazardous Situations, edited by D.E. Broadbent, A. Baddeley and J.T. Reason, pp. 463–473 (Oxford: Clarendon Press). DO¨RNER, D. and SCHAUB, H., 1994, Errors planning and decision-making and the nature of human information processing. Applied Psychology, 43, 433–453. DUBEY, G., 2001, La simulation a` l’epreuve du lien social. Le Travail Humain, 64, 3–28. GUERLAIN, S.A., SMITH, P.J., OBRADOVICH, J.H., RUDMAN, S., STROHM, P., SMITH, J.W., SVIRBELY, J. and SACHS, L., 1999, Interactive critiquing as a form of decision support: an empirical evaluation. Human Factors, 41, 72–89. HASEGAWA, N. and YOSHIMURA, S., 1999, Human–machine cooperation for emotion support. Study by using a simulation model. Le Travail Humain, 62, 247–272. HOC, J.-M., 2001, Towards ecological validity of research in cognitive ergonomics. Theoretical Issues in Ergonomics Science, 2, 278–288. HOWIE, D.E. and VICENTE, K.J., 1998, Measures of operator performance in complex, dynamic microworlds: advancing the state of the art. Ergonomics, 41, 485–500. HUKKI, K. and NORROS, L., 1998, Subject-centred and systemic conceptualisation as a tool of simulator training. Le Travail Humain, 61, 313–331. KAEMPF, G.L. and KLEIN, G., 1994, Aeronautical decision-making: the next generation. In Aviation Psychology in Practice, edited by N. Johnston, N. McDonald and R. Fuller, pp. 223–254 (London: Avebury Technical). KLEIN, G.A., 1993a, A recognition-primed decision (RPD) model of rapid decision making. In Decision Making in Action: Models and Methods, edited by G.A. Klein, J. Orasanu, R. Calderwood and C. Zsambok, pp. 138–147 (Ablex Publishing Co: New Jersey). KLEIN, G.A., 1993b, Twenty questions—suggestions for research in naturalistic decision making, In Decision Making in Action: Models and Methods, edited by G.A. Klein, J. Orasanu, R. Calderwood and C.E. Zsambock, pp. 389–403 (New Jersey: Ablex Publishing Co). MEISTER, D., 2000, Theoretical issues in general and developmental ergonomics. Theoretical Issues in Ergonomics Science, 1, 13–21. ORASANU, J. and CONNOLLY, T., 1993, The reinvention of decision making. In Decision Making in Action: Models and Methods, edited by G.A. Klein, J. Orasanu, R. Calderwood and C.E. Zsambock, pp. 3–20 (New Jersey: Ablex Publishing Co). RASMUSSEN, J., 1981, Models of mental strategies in process plant diagnosis. In Human Detection and Diagnosis of System Failures, edited by J. Rasmussen and W.B. Rouse, pp. 241–258 (New York: Plenum Press). ROLO, G., 1996, Toma de decisioń en control de procesos. Influencia del contexto y de factores individuales, Unpublished doctoral dissertation, University of La Laguna, Tenerife, Spain. ROLO, G. and CABRERA, D., 2000, Features of decision-making in process control tasks: the relevance of the work context. Cognition, Technology and Work, 2, 154–163. ROTH, E.M., WOODS, D.D. and POPLE, H.E., 1992, Cognitive simulation as a tool for cognitive task analysis. Ergonomics, 3, 1163–1198.

48


XIAO, Y., MILGRAM, P. and DOYLE, D.J., 1997, Planning behavior and its functional role in interactions with complex systems. IEEE Transactions on Systems, Man and Cybernetics—part A: Systems and Humans, 27, 313–323.

About the authors Gladys Rolo is Senior Professor of Ergonomics and Organizational Psychology at the University of La Laguna, Tenerife, Canary Islands, Spain. She received her degree in Psychology at the University of La Laguna in 1987. She holds a PhD degree in Psychology from the University of La Laguna in 1996. Her main research interests have focused on decision-making process, mental workload, human error and risk prevention. Nowadays, she is participating in the European Project Adams-2 developed in maintenance departments of aviation. Her main interest in this project is related to the development of causal attributions and a safety culture model in organizations. She is an author and co-author of papers published in international journals and conference proceedings. She is a member of the European Association of Work and Organizational Psychology. She may be contacted at [email protected]. Dolores Diaz-Cabrera is Professor of Work and Organizational Psychology at the University of La Laguna, Tenerife, Canary Islands, Spain. She received her degree in Psychology at the University of La Laguna in 1980. She holds a PhD degree in Psychology from the University of La Laguna in 1985. She has been director and co-ordinator of several national and European research projects related to decision-making, new technology at work and organizational safety. Now, she is the Spanish co-ordinator of the European Project Adams-2 developed in maintenance departments of aviation. Her main research interests focus on decisionmaking, human error and risk prevention. More specifically, at the moment her current concerns are related to causal attribution, organizational and safety culture and organizational change and innovation directed toward the improvement of safety in organizations. She is an author and co-author of papers published in international journals and conference proceedings. She can be reached at [email protected].