conduct their research in laboratory settings? For .... laboratory research have been based on assump- ..... Milgram (1974) did in his obedience research.
Research Settings in Industrial and Organizational Psychology Are Findings in the Field More Generalizable Than in the Laboratory? ROBERT L. DIPBOYE MICHAEL F. FLANAGAN
ABSTRACT: The authors analyzed for content all the empirical articles from the 1966, 1970, and 1974 volumes of the Journal of Applied Psychology, Organizational Behavior and Human Performance, and Personnel Psychology to determine the types of organizations, subjects, and dependent measures studied. Contrary to the common belief that field settings provide for more generalisation of research findings than laboratory settings do, field research appeared as narrow as laboratory research in the actors, settings, and behaviors sampled. Indeed, industrial-organisational psychology seems to be developing in the laboratory a psychology of the college student, and in the field, a psychology of the self-report of male, professional, technical, and managerial employees in productiveeconomic organizations. The authors suggest that coordinated strategies of research in both laboratory and field settings are needed to construct an externally valid industrial and organizational psychology.
Should industrial and organizational psychologists conduct their research in laboratory settings? For many industrial and organizational psychologists, the answer to this question would be an emphatic no, but others would defend the laboratory as a legitimate setting for research (Fromkin & Streufert, 1976; Weick, 196S). The relative merits of laboratory and field settings have been debated within many areas of psychology, including comparative psychology (Miller, 1977), environmental psychology (Proshansky, 1976), social psychology (Ellsworth, 1977; McGuire, 1967; Ring, 1967), race relations (Fromkin & Ostrom, 1974), and perception (Gibson, 1966). Two common criticisms of laboratory research are that laboratory settings are susceptible to experimental artifacts Vol. 34, No. 2, 141-150
Rice University Florida State University
(Rosenthai & Rosnow, 1969; Silverman, 1977) and .serious ethical problems (Kelman, 1967). Although these two criticisms have been debated in numerous articles and have not been resolved entirely, Fromkin and Streufert have argued persuasively that artifacts and ethical problems also exist in field settings and are not unique to the laboratory. A third criticism of laboratory research is that the artificiality of the laboratory imposes severe restraints on the external validity of findings. Even those critics who concede that laboratory settings provide for more elimination of alternative interpretations of results than the typical field settings do are often doubtful that one can generalize findings from the laboratory to predict, understand, or control real-world phenomena. A perusal of the recent Handbook of Industrial and Organizational Psychology (Dunnette, 1976) reveals the differences of opinion that exist among industrial and organizational psychologists with regard to the generalizability of laboratory research. Ohapanis (1976) observed that "most laboratory experiments in psychology have only very limited relevance for the solution of practical Portions of this article were presented at the meeting of the Midwestern Psychological Association, Chicago, May 1977. The authors' names are listed in alphabetical order, reflecting equal contributions by both to the design, implementation, analysis, and writing of this study. The authors thank John Dzamba, Claude Mattox, Brian Robinson, and William Wratten for their assistance with the data analysis. Requests for reprints should be sent to Robert L. Dipboye, Rice University, Department of Psychology, Houston, Texas 77001.
AMERICAN PSYCHOLOGIST • FEBRUARY 1979 • 141 Copyright 1979 by the American Psychological Association, Inc. 0003-066X/79/3402-0141$00.75
problems. . . . Unsuspected interactions in real life may nullify or even reverse conclusions reached in the laboratory" (p. 730). Similarly, Bouchard (1976) observed that "laboratory experiments seldom deal adequately with boundary conditions or context factors and therefore, lend themselves to unjustified and often erroneous extrapolations" (p. 364). In contrast with these arguments, Fromkin and Streufert (1976) argued that "the artificiality of laboratories is being unjustly elevated to the status of a fatal flaw" (p. 433). Furthermore, they asserted that instead of reinforcing the myth that laboratory settings seldom yield data which are relevant to real world problems, it is proposed that laboratory settings merely impose identifiable limitations upon the range of criterion situations to which a particular set of laboratory findings may be practically applied, (p. 442)
Despite the firm convictions held by proponents and critics, many of the arguments for and against the external validity of laboratory research are based on stereotypes rather than data. One such stereotype has been that a field setting, because it is natural to the subject and not contrived, automatically provides for more generalization of results than an artificial laboratory setting does. However, the problem of external validity is one of making inferences not only from settings but from actors and the behaviors of these actors (Runkel & McGrath, 1972). Critics of laboratory research often place inordinate weight on the setting, in discussions of external validity, to the neglect of actors and their behavior. For instance, some have assumed that because the setting in field research is typically an actual organization, data collected in such a setting must be more generalizable to other organizations than are data collected in the laboratory. However, there are differences among organizational settings that are as large as the differences that exist between an organizational setting and a laboratory. Rather than assuming that field research in the abstract is more generalizable, there is a need to determine the types of organizations with which field researchers have been concerned. The belief that laboratory research in industrial and organizational psychology is less externally valid than field research also results from the tendency to infer from the setting characteristics of the actors and their behaviors. Such inferences are often without empirical support. For example, the assumption has been made that field research involves subjects who are more representative of 142 • FEBRUARY 1979 • AMERICAN PSYCHOLOGIST
the working population than the college students typically used in the laboratory. But is this assumption correct? Instead of rejecting laboratory research on the basis of an untested stereotype, we need to examine more carefully who the participants are in laboratory and field research. Another assumption has been that research in field settings has as the focus of investigation natural behavior, defined by Tunnell (1977) as behavior that is not established or maintained for the sole or primary purpose of conducting research; the behavior is part of the person's existing response repertoire. Whether the procedure used to record the behavior is human or mechanical (e.g., videotapes, audiotapes, explicit behavioral checklists) the essential naturalness of the observation is maintained. . . . Self-reports should not be considered natural behaviors unless they are made by the person in real life. (pp. 426-427)
The critics of laboratory research often seem to assume that the dependent variables examined in the field are more natural and thus more externally valid than the dependent variables examined in the laboratory. But what are the dependent variables used in laboratory and field research? Again, arguments for and against the external validity of laboratory research have been based on assumptions rather than data. The issue of the relative external validity of laboratory and field settings is an important one not only to industrial and organizational psychologists but to all psychologists conducting laboratory and field research or utilizing the findings of such research. A more objective analysis of the issue of external validity is needed than has appeared in previous literature. In this article, our intent was to test empirically some of the assumptions underlying the belief that findings from field research are more generalizable to other settings, behaviors, and actors than are the findings of laboratory research. Of course, one cannot conclude that any one study or set of studies possesses complete external validity (Campbell & Stanley, 1963, p. 17), but one can discuss the relative limits that seem to exist for generalizing findings from research using different strategies. The intent of this article is to assess empirically the limits that seem to exist for recent laboratory and field research in industrial and organizational psychology. In order to achieve this objective, articles in three major journals were analyzed for content to determine the types of organizations, persons, and behaviors studied.
Method
TABLE 1
Selection oj Journals and Years
Percentage of Articles, for Each of the Dimensions, in Which There Was Agreement Between Coders
In order to assess the characteristics of laboratory and field research in industrial-organizational psychology, all articles published in the 1966, 1970, and 1974 volumes of the Journal oj Applied Psychology (JAP), Personnel Psychology (PP), and Organizational Behavior and Human Performance (OBHP) were analyzed for content. These three journals seemed to us to be representative of the published literature in industrial-organizational psychology. In support of our attention to these journals, Meltzer (1978) found that during the 1970s JAP, PP, and OBHP were the most frequently cited journals appearing in Annual Review of Psychology articles on industrial-organizational psychology topics. The years 1966, 1970, and 1974 were chosen because of the large increase in the amount of laboratory research appearing in the literature during this interval, reflecting a movement of the field away from an emphasis on personnel issues toward more diverse research interests and methodologies. Thus these 3 years seemed representative of the current literature in industrial and organizational psychology and provided a sufficiently large sample of laboratory research with which to make comparisons. Coding Scheme A trained assistant coded 561 articles for content. Of these articles, 490 contained empirical research and constituted the pool of studies used in this article. These empirical studies were coded as having been conducted in a laboratory or in a field setting. A laboratory setting was defined as one created for research purposes, and a field setting was defined as a natural setting created for purposes other than research (Tunnell, 1977, p. 427). Nine coding dimensions were selected for comparison of the laboratory and field articles. Four of these dimensions related to the type of subject, one to the type of organization, and two to the type of dependent measure used. The choice of these categories was consistent with Runkel and McGrath's (1972) statement that the limits of external validity are set by the actors, behaviors, and settings that are sampled in a body of research. To estimate the reliability of the coding, a sample of S3 articles was coded by a second person. The percentage of articles on which the two coders agreed is reported for each of the dimensions in Table 1.
Dimension
Laboratory vs. field setting Sex of subject College student vs. other Volunteers vs. nonvolunteers Inducement Sampling Type of organization Self-report vs. observation Performance vs. satisfaction
Percentage agreement
94 100 98 85 85 81 83 87 89
SUBJECTS
Five of the coding dimensions pertained to the characteristics of the subjects participating in the research or to the sampling procedures used to select ithem. Each of these five dimensions has been discussed as a possible limiting factor to external validity. Sex, Subjects were classified as either male or female. Carlson (1971) criticized the personality and social psychology literature for oversampling males to the exclusion of females. As a consequence of this neglect of the female subject, findings in this area of research may be more generalizable 'to males. Whether or not this is a valid criticism of the industrial-organizational psychology literature has never been addressed. In order to determine whether subject sex acts as a possible limit to generalization, articles were categorized as using only males, only females, or a combination of males and females. College student versus other. Subjects were classified as either college students or other. If they were college students, they were identified as having been enrolled in business, psychology, or some other course when they participated in the study. Inducement. Another subject variable important to the issue of whether or not findings are generalizable is the type of inducement used to solicit subject participation. Research suggests that in psychology experiments, volunteers behave differently and have different characteristics from nonvolunteers (Rosenthai & Rosnow, 1969). The conditions under which subjects in laboratory and field research in industrial and organizational psychology are solicited have never been examined empirically (to our knowledge). In the present study, articles were classified as using volunteers or
AMERICAN PSYCHOLOGIST • FEBRUARY 1979 • 143
nonvolunteers. If subjects freely volunteered to participate under no external inducement, they were considered volunteers. Nonvolunteers participated in research because of some external inducement. The type of inducement was coded as course credit, pay, or other. Sampling. This category refers to the sampling strategy of the authors and to whether or not there was an attempt to generalize the results to other types of populations. A sample was coded as representative if it was similar to the target population to which the investigator wished to generalize or if the sample was gathered in such a way as to be part of the target population. If two investigators used a random sample of freshmen from their university and generalized their findings only to college freshmen at universities with characteristics similar to theirs, it was coded as a representative sample. Likewise, if two investigators conducted a survey of college freshmen in the United States by means of a single-stage sampling technique, the study was coded as representative. If a researcher attempted to generalize beyond a study to samples that were not represented in the study, the article was coded as nonrepresentative. TYPE OF ORGANIZATION
Studies that directly investigated a particular kind of organization (e.g., fast food restaurants) or that had important implications for a particular kind of organization (e.g., a laboratory simulation of a manufacturing firm) were coded as having something to say concerning particular types of organizations. Studies were categorized as making statements concerning productive-economic, mainTABLE 2
Percentage of Studies in Laboratory and Field Settings Pertaining to Specific Types of Organizations Setting Type of organization
Prod ucti ve-economic Maintenance Adaptive Managerial-political Combination Nonorganizational Total
Lab»
l.S .4 .0 .0 .0
Field*1
23.9
8.3
98.1
2.3 6.0 1.0 58.5
100.0
100.0
• n = 189. b n =301.
144 • FEBRUARY 1979 • AMERICAN PSYCHOLOGIST
tenance, adaptive, managerial-political, or some combination of these four types. Katz and Kahn (1966, pp. 112-113) have provided a description of these organizational types. TYPE OF DEPENDENT MEASURE
A common limit to generalization is to use subjective self-report measures and to neglect the more objective observations of behavior, performance, and choice. In order to gather data on the extent to which laboratory and field research make use of one type of dependent measure to the exclusion of others, articles were classified on the basis of who provided the measures of the dependent variable and what constituted the dependent variable. Self-report versus observation. Articles were coded as using self-report, external observation, or some combination of self-report and external observation. The self-report category consisted of current or retrospective accounts by a person of his or her own behavior, performance, or attitudes. External observations consisted of observations, ratings, and other measures originating from sources external to the subject of the research. The external category included objective measures of ongoing or completed activity (e.g., performance, choice, or effort). For example, if a subject rated his or her own preference for various brands of a product, this was coded as self-report. If preference was measured by observing actual choices made between different brands, this was coded as external observation. Performance versus satisfaction. Dependent measures were categorized as measures of individual, group, or organizational performance if they dealt with effort, choice, or the consequences of effort and choice. Performance dimensions included quantity, quality, sales, profit, accidents, sickness rate, absenteeism, and turnover. Satisfaction measures included indices of individual and group attitudes toward pay, co-workers, and other facets of the job or task. These were typically self-report measures. When some combination of performance and satisfaction was measured, the article was coded as such.
Results TYPE OF ORGANIZATION
One issue concerned the types of organizations that researchers have studied. Table 2 summarizes
the percentage of laboratory and field studies that dealt with each of Katz and Kahn's (1966) organizational types. In more than one half of the field studies (58.5%), the study was not directly relevant to organizations or organizational processes. Of those studies that were concerned with organizations, the productive-economic type (23.9%) was the most commonly studied, followed by the maintenance (8.3%), managerialpolitical (6.0%), and adaptive (2.3%) types of organizations. Only a few of the laboratory studies (1.9%) were directly relevant to specific organizational types. SUBJECTS Occupation of subjects. Table 3 summarizes the occupations of subjects in laboratory and field research. Over 75% of the laboratory research articles, compared with 13% of the field research articles, used students as subjects. Of the laboratory articles using college students, 36.7% used students enrolled in a psychology course, 12.2% used students enrolled in a business course, and 40.8% did not report the course in which the students were enrolled when participating in the study. The remaining occupation groupings were taken from the Dictionary of Occupational Titles (U.S. Department of Labor, 1965). Most of the subjects in field research were in the professional,
TABLE 4
Sample Characteristics of Research in Laboratory and Field Settings Setting Characteristic
Lab (%)« Field (%)b
Representativeness of sample Representative Nonrepresentative Other Total
48.1 49.2 2.7 100.0
42.5 54.8 2.7 100.0
Sex of subjects Males only Females only Both males and females Not known Total
31.7 4.2 23.8 40.3 100.0
27.2 10.0 19.9 42.9 100.0
8.5
13.6
22.8 17.5 6.9 2.1 42.2
2.7 .0 12.5 .7 70.5
100.0
100.0
Volunteer status Volunteers Nonvolunteers Inducement with money Inducement with course credit Other inducement Course credit and money Not known Total » n = 189. b n = 301.
technical, managerial category (36.9%). The second largest single category was service (8.0%). Of the multiple occupations studies, 88.4% involved subjects who could be included in both the professional, technical, managerial category and one of the other occupational groupings. Thus, TABLE 3 52.2% of subjects in field research were in some Percentage of Studies in Laboratory and Field respect associated with the professional, technical, Settings Using Subjects From Specific Occupations managerial category. Sampling of subjects. Table 4 reports the perSetting centages of laboratory and field studies using FieldLab" Occupation representative and nonrepresentative sampling pro6.9 36.9 cedures. More than one half of the field studies Professional, technical, managerial .0 3.0 (54.8%) and nearly one half of the laboratory Clerical, sales 6.4 8.0 studies (49.2%) employed sampling procedures Service .0 .3 Farm, fish, forest that were classified as nonrepresentative of the .0 .7 Processing target population. .0 1.0 Machine trades Sex of subjects. A surprising number of both .0 3.7 Bench work .0 1.6. laboratory (40.3%) and field (42.9%) studies Structural work 3.9 17.2 Multiple occupations contained no specification of the sex of partici1.2 1.6 Heterogeneous pants, as may be seen in Table 4. Males tended 10.0 3.3 Other 13.0 to be used more frequently than females in both 75.1 Student 3.0 field (27.2% vs. 10.0%) and laboratory (31.7% 3.2 Not known 100.0 100.0 vs. 4.2%) studies. A similar percentage of laboraTotal tory (23.8%) and field (19.9%) research used » n - 189. both males and females. However, of those studies » n - 301. AMERICAN PSYCHOLOGIST • FEBRUARY 1979 • 145
using both males and females, only 6.9% of the laboratory studies and 6.0% of the field studies contained tests for sex differences. Volunteers versus non-volunteers. The percentage of studies in field settings that involved volunteer subjects (13.6%) was somewhat larger than the percentage of studies in the laboratory that involved volunteers (8.5%). As may be seen in Table 4, more laboratory studies (49.3%) than field studies (15.9%) involved nonvolunteers. However, the field research more frequently (70.5% vs. 42.2%) contained no information on the volunteer status of the subjects. Inducement of subjects. Table 4 summarizes the types of inducements used in laboratory and field research. In laboratory research, the most frequent inducements were money (22.8%) and course credit (17.5%). When the mode of inducement was stated in field research, it tended to be something other than money or course credit (12.5%). DEPENDENT MEASURES
Observation versus self-report. Table 5 contains the percentages of laboratory and field research using observational or self-report measures. Laboratory research (69.4%) was more likely than field research (24.3%) to use only observational measures, but field research (49.8%) employed only self-report measures more frequently than laboratory research (16.9%). Also, field research (22.6%)) was somewhat more likely than laboraTABLE 5
Characteristics of Dependent Measures in Laboratory and Field Research Setting Dependent measure
Observation vs. self-report Observation Self-report Other Some combination Total Performance vs. satisfaction Performance only Satisfaction only Other Some combination Total
Lab(%)'
Field (%)b
69.4 16.9 1.0 12.7 100.0
24.3 49.8 3.3 22.6 100.0
72.0 1.1
14.8 12.1
24.6 11.3 30.2 33.9
100.0
100.0
•n = 189. b n = 301.
146 • FEBRUARY 1979 • AMERICAN PSYCHOLOGIST
tory research (12.7%) to contain a combination of observational and self-report measures. Performance versus satisfaction. In Table S, laboratory and field research are classified as using only performance, only satisfaction, some other variable, or some combination of variables as the dependent measure. Laboratory research (72.0%) made more frequent use of performance measures than field research (24.6%) did, and field research (11.3%) made more frequent use of only satisfaction than laboratory research (1.1%) did. Also, field research made more frequent use of some combination of variables or a variable other than satisfaction or performance.
Discussion A typical assumption made by researchers in industrial-organizational psychology, as well as in other areas within psychology, is that research in field settings is inherently more generalizable than research in laboratory settings. The findings of the present article suggest that this assumption is oversimplified at best and in several respects is erroneous. The question of external validity must begin with a delineation of the actors, settings, and behaviors to which one wishes to generalize (Runkel & McGrath, 1972). A safe assumption would be that most industrial-organizational psychologists aspire to a science of individual and group behavior relevant to the general working population and to all types of organizations. If the findings of this article are examined with reference to these aspirations, field research in industrial-organizational psychology has dealt with a rather narrow subset of settings, actors, and behaviors. Indeed, if laboratory research can be described as having developed a psychology of the college sophomore, then field research can be described as having produced a psychology of selfreport by male, professional, technical, and managerial personnel in productive-economic organizations. There are several explanations for the narrow! / sampling of subjects, settings, and behaviors. Be/ cause behavior is typically less accessible in field settings than in laboratory settings, self-reports must often be used as the dependent measures. Also, professional, technical, and managerial personnel are often more amenable to and capable of completing the questionnaires used in research than are blue-collar workers. Finally, the majority of subjects are male because professional,
technical, and managerial personnel are predominantly male. There is nothing inherently undesirable about a narrow psychology if it is a valid one. However, most industrial and organizational psychologists aspire to a psychology of a working population that has a larger proportion of females and a smaller proportion of professional, technical, and managerial personnel than is found in our sample of the literature. THE MYTH OF THE FIELD'S INHERENT EXTERNAL VALIDITY
One should not infer from the data that the findings of laboratory research are more externally valid than the findings of field research. Not only have laboratory researchers sampled primarily male college students under external inducement, laboratory researchers too readily have generalized their findings to populations unrepresented in their samples. The present article, in combination with a previous report (Flanagan & Dipboye, Note 1), suggests that laboratory research in industrial-organizational psychology suffers from many of the same problems that exist with laboratory research in other disciplines within psychology. Despite the problems with laboratory research, the findings indicate that there is no empirical basis for a belief in the inherent external validity of field research. Not only is field research limited in its sampling of actors, behaviors, and settings, in some respects laboratory research provides as firm a basis for generalization to the general population of working people and organizations as does field research. For instance, college students may constitute a heterogeneous sample as representative of the general working working population as do the samples of professional, technical, and managerial employees so often utilized in field research. In addition to having sampled a narrow range of subjects, field researchers appear to have relied on self-report and satisfaction as dependent measures more frequently than have laboratory researchers. Self-report measures are seriously limited by response biases, such as halo. Moreover, they are not of primary interest to industrial and organizational psychologists, who most often seek to understand, predict, and control actual behavior, not self-reported behavior. Indeed, if one employs Tunnell's definition of natural behavior, laboratory researchers seem to have examined natural behavior at least as frequently as have field researchers.
SUGGESTIONS FOR IMPROVING EXTERNAL VALIDITY
On the basis of the articles sampled, the best conclusion appears to be that both laboratory and field research have been narrow in their sampling of actors, settings, and behaviors, and both provide a limited basis for generalization of results to the general working population. What can be done to improve the external validity of research in the laboratory and the field? One approach to the problem would be to determine the variables distinguishing the setting, subjects, and behaviors in a particular research study from those settings, subjects, and behaviors to which generalization is desired. Through programmatic research we may manipulate these variables to determine the limits or boundaries of our findings. For instance, Weick (1965) believed that the laboratory differs from an organization on several dimensions, such as size, duration of membership, and career-orientation, feedback, and task interdependencies. However, he denied that these are insurmountable barriers to relevance. According to Weick, the effects of these variables, if not the variables themselves, may be preserved in a laboratory setting. For instance, a large group size may not be possible in the laboratory as it is in the organization, but subgroup formation, barriers to communication, and formalization, all consequences of group size, may be manipulated to establish correspondence between the laboratory and field. Similar to Weick (196S), Fromkin and Streufert (1976) proposed that we search for the critical differences between the field or laboratory settings in which we collect our data and the nonresearch settings that are the targets of generalization. Awareness of being a research subject, unidimensionality of dependent measures, shortened time perspective, and low expectations of accountability for action are among the ways that laboratory and field settings may differ from a target setting. Once these are identified, the experimenter can hold the boundary variable constant at some level which represents its occurrence in the criterion setting.. However, this procedure does not yield information about the effects of this variable under different levels of the variable. It remains a potential boundary variable when generalizations are made to other settings. Alternatively, the experimenter can treat the potential boundary variable as an additional independent variable and manipulate across several levels which are believed to encompass its operation in several criterion settings. (Fromkin & Streufert, 1976, p. 442)
Research in which we empirically determine the extent to which findings in the laboratory generalAMERICAN PSYCHOLOGIST • FEBRUARY 1979 • 147
ize to the field may reveal that some of our anxieties concerning the external validity of specific laboratory studies are unjustified. For instance, Bernstein, Hakel, and Harlan (197S) reviewed laboratory and field research in which subjects evaluated qualifications of hypothetical candidates on the basis of bogus r6sum6s. They concluded that the findings from laboratory research on employment decision making by students are very similar to the findings from research with professional interviewers. Not only can one argue that students and professionals do not differ substantially in their responses, the experimental task is in several respects similar to that performed in the real world. Many hiring decisions are made on the basis of very limited information, as is illustrated by the "paper people" constituted by a set of resumes. Another approach to improving external validity would be to broaden our samples of actors, settings, and behaviors. Cook and Campbell (1976) suggested that the most powerful but least feasible means of increasing external validity is to take random samples from the universe to which we wish to generalize. However, they indicated that a more practical model is to use within a single design several heterogeneous groups of persons, settings, and times and "test the range of variables across which a causal relationship could be inferred. Thus, generalizing 'across' would replace generalizing 'to,' and 'quasi-representativeness' would replace more 'formal representativeness' of the random sampling model" (p. 236). This second strategy seems applicable to both laboratory and field research in industrial-organizational psychology. For instance, researchers should consider bringing people from outside the college setting into the laboratory or taking the laboratory to subjects outside the college environment, as Milgram (1974) did in his obedience research. Perloff and Perloff (1977) .recently suggested the fair as a setting in which research could be conducted with samples more representative of the general population than samples typical of current laboratory and field research. A final approach to improving external validity would be to employ coordinated strategies of research in which theories in industrial-organizational psychology are tested in both laboratory and field settings. Evan (1971, p. 4) and Runkel and McGrath (1972) came to similar conclusions when they suggested that the laboratory and the field provide more powerful vehicles for research on 148 • FEBRUARY 1979 • AMERICAN PSYCHOLOGIST
organizational behavior when used in coordination than when used separately. For instance, one approach to studying the relative effectiveness of conflict-resolution methods has been to ask managers to rate the extent to which they and their own supervisors deal constructively with conflict and how frequently various methods are employed (Burke, 1970). The use of self-report in this case certainly provided useful information, but the findings of such research may be limited in generalizability to self-reported conflict among male managerial personnel. One solution would be to conduct field research in which actual, ongoing behavior is observed in response to naturally occurring or manipulated conflict resolution. However, behavior is simply not as accessible in most field settings as it is in the laboratory, and with the current concern over privacy and confidentiality, observations of actual behavior may be conducted even less frequently in future field research. The most rigorous and controlled tests of hypotheses will continue to be done in laboratory settings. However, the use of both types of research in coordination could provide a powerful means for overcoming weaknesses found in both laboratory and field research. Thus, one could follow Burke's research with laboratory operationalizations of his pencil-and-paper conflict-resolution modes and could observe the effects of conflict resolution on behavior, attitudes, and a variety of objective and subjective events. Despite the wisdom of coordinated strategies, very few research programs in industrial and organizational psychology exist that may be held up as examples. One could point to recent research by Kipnis and his associates on power (Kipnis & Cosentino, 1969; Goodstadt & Kipnis, 1970) or by Zander (1971) on goal setting. In addition, there, are isolated studies constituting replications of field research in laboratory settings and replications of laboratory research in the field. However, there is a notable lack of programmatic research in which investigators have moved between laboratory and field settings in a systematic attempt to establish the external validity of research findings.
Conclusions There are other issues related to generalizability that we have not examined. For instance, one important consideration in generalizing from the research setting to the target setting is whether the
events we create in the laboratory or field follow the same laws as those that govern similar events in the target setting (Zelditch & Hopkins, 1969). Another important consideration is whether or not our treatment conditions are restricted in the range of effects they can create compared with the range of effects that exist in the target setting (Ellsworth, 1977, p. 607). Related to this question is the extent to which the independent variables are representative of stimuli as they exist in the environment (Brunswik, 1949). Despite these limitations, we believe that the content analysis in the present article accurately reflects several important characteristics of current laboratory and field research in industrialorganizational psychology. The results suggest that blanket statements concerning the inherent external validity of field research are not only inaccurate but serve to hinder the development of industrial and organizational psychology as a field of study. Rather than rejecting or accepting a study because of its setting, a careful examination of the organizations, people, and responses sampled in each individual study is needed to determine the possible limits on external validity. Rather than asking, Is this study externally valid?, a more appropriate question is, To what actors, settings, and behaviors may we generalize the findings of this study? Approached from the latter view, laboratory and field research may be viewed as complementary rather than conflicting strategies. REFERENCE NOTE 1. Flanagan, M. F., & Dipboye, R. L. Research settings in industrial and organizational psychology: I. Characteristics of research in laboratory and field settings (Paper No. 645). West Lafayette, Ind.: Purdue University, Institute for Research in the Behavioral, Economic, and Management Sciences, December 1977. REFERENCES Bernstein, V., Hakel, M. D., & Harlan, A. The college student as interviewer: A threat to generalizability? Journal of Applied Psychology, 1975, 60, 266-268. Bouchard, T. J. Field research methods: Interviewing, questionnaires, participant observation, systematic observation, unobtrusive measures. In M. D. Dunnette (Ed.), Handbook of industrial and organizational psychology. Chicago: Rand McNally, 1976. Brunswik, E. Systematic and representative design of psychological experiments. Berkeley: University of California Press, 1949. Burke, R. J. Methods of resolving superior-subordinate conflict: The constructive use of subordinate differences and disagreements. Organizational Behavior and Human Performance, 1970, J, 393-411. Campbell, D. T., & Stanley, J. C. Experimental and
quasi-experimental designs for research. Chicago: Rand McNally, 1963. Carlson, R. Where is the person in personality research? Psychological Bulletin, 1971, 75, 203-219. Chapanis, A. Engineering psychology. In M. D. Dunnette (Ed.), Handbook of industrial and organizational psychology. Chicago: Rand McNally, 19'76. Cook, T. D., & Campbell, D. T. The design and conduct of quasi-experiments and true experiments in field settings. In M. D. Dunnette (Ed.), Handbook of industrial and organizational psychology. Chicago: Rand McNally, 1976. Dunnette, M. D. (Ed.). Handbook of industrial and organizational psychology. Chicago: Rand McNally, 1976. Ellsworth, P. C. From abstract ideas to concrete instances: Some guidelines for choosing natural research settings. American Psychologist, 1977, 32, 604-615. Evan, W. M. (Ed.). Organizational experiments: Laboratory and field research. New York: Harper & Row, 1971. Fromkin, H. L., & Ostrom, T. M. Laboratory research and the organization: Generalizing from lab to life. In H. L. Fromkin & ]. Sherwood (Eds.), Integrating the organization: A social psychological analysis. New York: Free Press, 1974. Fromkin, H. L., & Streufert, S. Laboratory experimentation. In M. D. Dunnette (Ed.), Handbook of industrial and organizational psychology. Chicago: Rand McNally, 1976. Gibson, J. J. The senses considered as perceptual systems. London: Allen & Unwin, 1966. Goodstadt, B., & Kipnis, D. Situational influences on the use of power. Journal of Applied Psychology, 1970, 54, 201-207. Katz, D., & Kahn, R. L. The social psychology of organizations. New York: Wiley, 1966. Kelman, H. Human use of human subjects: The problem of deception in social psychological experiments. Psychological Bulletin, 1967, 67, 1-11. Kipnis, D., & Cosentino, J. Use of leadership powers in industry. Journal of Applied Psychology, 1969, 53, 460466. McGuire, W. J. Some impending reorientations in social psychology: Some thoughts provoked by Kenneth Ring. Journal of Experimental Social Psychology, 1967, 3, 124139. Meltzer, H. Ratings of psychological journals by and for industrial-organizational psychologists. Professional Psychology, 1978, 9, 290-300. Milgram, S. Obedience to authority: An experimental view. New York: Harper & Row, 1974. Miller, D. B. Roles of naturalistic observation in comparative psychology. American Psychologist, 19'77, 32, 211-219. Perloff, R., & Perloff, L. S. The fair—An opportunity for depicting psychology and for conducting behavioral research. American Psychologist, 1977, 32, 220-229. Proshansky, H. M. Environmental psychology and the real world. American Psychologist, 1976, 31, 303-310. Ring, K. R. Experimental social .psychology: Some sober questions about some frivolous values. Journal of Experimental Social Psychology, 1967, 3, 113-123. Rosenthal, R., & Rosnow, R. L. (Eds.). Artifact in behavioral research. New York: Academic Press, 1969. Runkel, P. J., & McGrath, J. E. Research on human behavior: A systematic guide to method. New York: Holt, Rinehart & Winston, 1972. Silverman, I. The human subject in the psychological laboratory. New York: Pergamon Press, 1977.
AMERICAN PSYCHOLOGIST • FEBRUARY 1979 • 149
Tunnell, G. B. Three dimensions of naturalness: An expanded definition of field research. Psychological Bulletin, 1977, W, 426-437. U.S. Department of Labor. Dictionary of occupational titles: Occupational classification (Vol. 1, 3rd ed.). Washington, D.C.: U.S. Government Printing Office, 1965. Weick, K. E. Laboratory experiments with organizations.
In J. G. March (Ed.), Handbook of organizations. Chicago: Rand McNally, 1965. Zander, A. Motives and goals in groups. New York: Academic Press, 1971. Zelditch, M. J., & Hopkins, T. K. Can you really study an army in the laboratory? In A. Etzioni (Ed.), A sociological reader on complex organizations. New York: Holt, Rinehart & Winston, 1969.
Awards for Distinguished Professional
Contributions
The Board of Professional Affairs is proud to announce its 1979 Professional Awards Program. Beginning in 1979, the Board will present annually three Awards for Distinguished Professional Contributions. The $1,000 awards will be presented in the following three areas: 1. The Award for Distinguished Professional Contributions to Knowledge—to a psychologist deemed to have made a significant contribution to professional psychology through scholarly research efforts. 2. The Award for Distinguished Contributions to Applied Psychology as a Professional Practice—to a psychologist whose application of psychology to professional practice is considered to be outstanding. Special emphasis should be placed on skills or technique as taught or practiced. 3. The Award for Distinguished Professional Contributions to Public Service— to a psychologist whose contributions to public service are considered to be outstanding. Such activities may occur within, but are not limited to, services to federal, state, and/or local governmental units. Those wishing to submit a nomination are asked to write for nomination forms to the Office of Professional Affairs, American Psychological Association, 1200 Seventeenth Street, N.W., Washington, D.C. 20036. The deadline for receiving completed application materials is April 15, 1979.
ISO • FEBRUARY 1979 • AMERICAN PSYCHOLOGIST
