KARLENE H. ROBERTS 2 AND CHARLES A. O'REILLY III. University of ... appear to be different variables (Porter & ... (Lawler, Porter, & Tenenbaum, 1968), and.
Journal of Applied Psychology 1974, Vol. 59, No. 3, 321-326
MEASURING ORGANIZATIONAL COMMUNICATION KARLENE H. ROBERTS
2
AND CHARLES A. O'REILLY III
University of California, Berkeley Past research on organizational communication has not included systematic development of a measurement method that can be used to compare dimensions of communication within and across organizations. This research reports on initial attempts to develop such instrumentation. Seven samples, with a total number of over 1,200 respondents, were used to develop a 35-item questionnaire measuring 16 facets of communication.
Communication has been the subject of considerable attention in organizational research. However, investigators often attach diverse labels to seemingly similar communication variables and similar labels to what appear to be different variables (Porter & Roberts, in press; Roberts, O'Reilly, Bretton, & Porter, in press). There has been no systematic development of instrumentation to measure communication variables in organizations. The present research begins to develop a standardized questionnaire that can be used to compare communication across organizations. A review of the organizational literature reveals that among the communication facets most frequently discussed and investigated are directionality of information flow (e.g., Graves, 1972), accuracy and distortion of information (Read, 1962; Wilensky, 1967), modalities used in transmitting information (Lawler, Porter, & Tenenbaum, 1968), and gatekeeping of information (Davis, 1968; O'Reilly & Roberts, in press; Rosen & Tesser, 1970). The communication literature not directly concerned with human behavior in organizations suggests additional aspects of communication that should be given greater attention in behavioral organizational research. These include overload (Porat & Haas, 1969) and satisfaction with one's communication (Lawler, Porter, & Tenenbaum, 1968; Lawson, 1965). Desire to interact with others iThis study was partially supported by Public Health Service Grant MH220S4-01 and Office of Naval Research Grant N000314-69-A-0200-10S4. 2 Requests for reprints should be sent to Karlene H. Roberts, School of Business Administration, University of California, Berkeley, 3SO Barrows Hall, Berkeley, California 94720.
is suggested by the interpersonal literature to be an important progenitor of actual interaction behavior (Rogers & Bhowmik, 1970). Finally, three noncommunication variables—• trust in superior, perceived influence of superior, and mobility aspirations of the respondent—have been repeatedly shown to influence individual communication in organizations (e.g., Cohen, 1958; O'Reilly & Roberts, in press; Read, 1962). Because communication is a process rather than a static variable it is one of the most difficult organizational variables to measure. Behavioral observations are often impossible to make and self-report forms are usually phrased in such a way that they take snapshots of process variables. The attempt in this study was to develop a measurement device that allows respondents to summarize their own behavior over time. Aggregation of individual responses should minimize many of the inherent individual response errors. METHOD Based on a literature review, 189 Likert-type items were created to assess respondent perceptions of the communication dimensions previously indicated. Communication purpose, content, importance, and speed, which are also frequently mentioned in the organizational literature, were considered in initial instrument development. They were later discarded because of the difficulty respondents had answering items relevant to them. Items were also developed to measure desire for interaction and to assess three noncommunication variables: trust in superior, perceived influence of the superior, and respondent mobility aspirations (Likert, 1967; Read, 1962). While the number of researchers who have measured the latter three dimensions is extensive, only Read (1962) reports intercorrelations of items supposedly assessing the same facets. The 189 items in the pool were administered to 70 graduate students, all of whom had prior work
321
KAELENE H. ROBERTS AND CHARLES A. O'REILLY III
322
TABLE 1 VARIMAX FACTOR LOADINGS FOR ITEMS ASSESSING TRUST IN SUPERIOR, PERCEIVED INFLUENCE or SUPERIOR, AND SUBORDINATE'S MOBILITY ASPIRATIONS FOR A SAMPLE OF 95 MILITARY SUBJECTS
Item
1. Trust 2. Trust 3. Trust 4. Perceived influence S. Perceived influence 6. Mobility aspirations 7. Mobility aspirations Proportion of common factor variance
Varimax factor loadings (» = 95)
1
2
3
.68 .76 .79 .28 .14 .12 .23
.22 .08 .23 .30 .16 .90 .88
.13 .41 .01 .81 .87 .21 .20
.26
.26
.24
experience. These students were asked to respond to the questions based on their experience in some past or current job, considering their communication behavior in a typical work week. Items were retained that had high intercorrelations with other items ostensibly measuring the same facets and low intercorrelations with the rest of the item pool. A 60item questionnaire resulted and was further tested. This version of the questionnaire (51 communication and 9 noncommunication items) was administered to 86 mental health workers, at all job levels, in five health-care-delivery teams working in a large medical center. Their responses were subjected to a V-type cluster analysis to further reduce the number of questionnaire items. This technique does not seek orthogonal item groups. Instead, using an objective index of collinearity (P2) and the correlation matrix among variables, the pattern of correlations is compared, and subsets or clusters of variables are identified that have collinear correlation profiles. Each cluster of variables may be thought of as occupying generally the same space, that is, are measuring the same construct (Tryon & Bailey, 1970). Two single communication items were included with the clusters (one measuring overload and the other measuring general satisfaction with communication) after it was determined that these questions behaved appropriately with regard to the rest. The question concerned with satisfaction is a modification of the GM faces scale (Kunin, 1955). When this question was related to the clusters it showed relationships to each cluster without loading heavily on any one, suggesting that satisfaction with communication in an organization is perceived to be partly related to (a) interaction, (b) accuracy, (c) directionality of information flow, etc. Conceptually, trust, influence, and mobility should be more independent of one another than are the
communication items. Consequently these nine noncommunication items were subjected first to a principal-components analysis with initial communality estimates in the diagonal provided by squared multiple correlations (/J 2 ) and then a varimax rotation. Three separate factors emerged. These items were included in a test battery completed by 95 officers and enlisted men in a United States military unit. The results are reported in Table 1. Based on these analyses the total instrument was reduced to 35 items which were then scored on 10 multi-item indexes and 6 single questions. As previously mentioned, questions concerned with overload, satisfaction, and modality use are reported as individual items.
RESULTS Reliability The 35-item version of the instrument was administered twice to 24 graduate business students (all with previous job experience) and to 42 managers for test-retest reliability. In both cases the administrations were three weeks apart. The test-retest reliabilities for the 10 indexes and 6 individual items and a sample question for each dimension are reported in Table 2; 9 of the 10 indexes have reasonable test-retest reliabilities. However, the small sample sizes involved should be kept in mind. In addition to the mental health workers, the 35-item version of the questionnaire was administered to the military sample previously referenced, 54 emergency room and outpatient hospital personnel, 52 managers from various firms, 179 respondents at various job levels in six United Kingdom locations of a large financial institution, 148 subjects from six comparable United States offices of the same organization, and 579 personnel in another military organization. Differences among the means and standard deviations across the seven samples (n = 1,218) reveal that within-group variance is less than between-group variance. This suggests that it is feasible to discriminate among organizations using these communication indexes. Maximum, minimum, and median Cronbach alphas (a measure of the internal consistency of the items measuring a dimension) are reported in Table 3 for each of the seven samples on the 10 multi-item indexes. The alphas appear relatively consistent for all except the last index.
323
MEASURING ORGANIZATIONAL COMMUNICATION TABLE 2 COMMUNICATION DIMENSIONS AND TEST-RETESX RELIABILITIES
Index
1. Trust
2. Influence 3. Mobility 4. Desire for interaction 5. Directionality —upward 6. Directionality —downward 7. Directionality —lateral 8. Accuracy 9. Summarization 10. Gatekeeping 11. Overload 12. Satisfaction
Modalities 13. Written 14. Face-to-face 15. Telephone 16. Other
No. items
Definer
How free do you feel to discuss with your immediate superior the problems and difficulties you have in your job without jeopardizing your position or having it "held against" you later? In general, how much do you feel that your immediate superior can do to further your career in this organization? How important is it for you to progress upward in your present organization? How desirable do you feel it is in your organization to be in contact frequently with others at the same job level? While working, what percentage of the time do you spend in contact with superiors? While working, what percentage of the time do you spend in contact with subordinates? While working, what percentage of the time do you spend in contact with others at the same job level? When receiving information from the sources listed below (superior, subordinate, peers), how accurate would you estimate it usually is? When transmitting information to your immediate superiors, how often do you summarize by emphasizing aspects that are important and minimizing those aspects that are unimportant? Of the total amount of information you receive at work, how much do you pass on to your immediate superior? Do you ever feel that you receive more information than you can efficiently use? Put a check under the face that expresses how you feel about communication in general, including the amount of information you receive, contacts with your superiors and others, the accuracy of information available, etc. ? Of the total time you engage in communications while on the job, about what percentage of the time do you use the following methods : Written' Face-to-face Telephone Other
Test-retest correlation Students Managers (« = 24) (n = 42)
3
.71
.69
2
.49
.62
2
.77
.80
3
.66
.76
3
.84
.87
3
.82
.83
3
.82
.76
3
.82
.61
3
.35
.37
3
.83
.78
1
.46
.69
1
.60
.73
1 1 1 1
.44 .86 .39
.63 .41 .58 .83
Note. A revision of the questionnaire including additional items and indexes is available on request. Items are scored on 7-point scales, except for indexes 5, 6, and 7 which use 10-point scales.
Validity Objective criterion measures might be devised for comparison with each of the communication facets discussed in this study. However, difficulty in obtaining objective criterion measures and their potential inappropriateness once obtained warrant consideration of more indirect validity assessments:
construct, face, and a very limited attempt at convergent-discriminant validity. First, evidence for the construct validity of the 10 dimensions is provided by the results of the cluster analyses previously discussed and increases confidence that even similar appearing facets such as trust and influence, or interaction and desire for interaction, can be
KARLENE H. ROBERTS AND CHARLES A. O'REILLY III
324
TABLE 3 MAXIMUM, MINIMUM, AND MEDIAN CRONBACH ALPHAS FOR THE 10 MULTI-ITEM INDEXES ACROSS SEVEN SAMPLES (n = 1,218) Cronbach alpha Index Median
1. Trust 2. Influence 3. Mobility 4. Desire for interaction 5. Directionality—upward 6. Directionality—downward 7. Directionality—lateral 8. Accuracy 9. Summarization 10. Gatekeeping
.68 .69 .82 .68 .65 .84 .71 .62 .73 .53
Maxi- Minimum mum
.86 .77 .92 .74 .76 .88 .77 .71 .88 .64
1 1
.62
.53 .74 .29 .21 .41 .41 .37 .16 .16
successfully differentiated from each other. The intercorrelations among indexes from a representative sample, shown in Table 4, confirm the notion that communication dimensions are differentiable. Second, individual questionnaire items appear to have face validity. During the item development respondents indicated which questions were confusing or not differentiable in their minds. Thus, questions attempting to measure task versus social information, speed of transmission, and the purpose of the communication were eliminated. Third, in the spirit of convergent-discriminant validity, intercorrelations of nine communication and three noncommunication dimensions with other similar and dissimilar measures are presented in Table 5 for the first military sample. The other measures include assessments of overall job satisfaction (Kunin, 1955), supervisor's considerate leadership style (Stogdill & Coons, 1957), organizational competence and flexibility (Campbell & Beaty, 1971), organizational commitment (Porter & Smith, 1970), and respondent's age and tenure. Examination of Table S reveals some of the expected convergence and discrimination. Education and tenure may be surrogates for rank in military units. Thus, it was hypothesized that only communication facets that change as a result of rank are related to edu-
J .1
js
I
I
i
ii
1
^ OOcs O
I
1
1
J3 I
i
ii
.„
-
-pi! gffrr
S'_*-qOq "JVVV
325
MEASURING ORGANIZATIONAL COMMUNICATION TABLE 5 INTERCORSELATIONS OP NlNE COMMUNCIATION AND THREE NONCOMMUNICATION INDEXES WITH OTHER INSTRUMENTS FOR 95 MILITARY RESPONDENTS
Index 1. Trust 2. Influence 3. Mobility 4. Desire for interaction 5. Directionality of contact — upward 6. Directionality of contact — downward 7. Directionality of contact —lateral 8. Accuracy 9. Summarization 10. Amount of information passed 11. Overload 12. Satisfaction with communication
Overall job satisfaction (Kunin, 1955)
.38* .35* .53* .46*
-.22*
Leadershipconsideration (Stogdill & Coons, 1957)
Organizational competence and flexibility (Campbell & Beaty, 1971)
.61* .36* .40* .21* .04
Organizational Commitment (Porter & Smith, 1970)
.31* .31* .31* .26* -.09
.25* .50* .57* .37* -.23*
Respondent job tenure -.12 -.29* -.10 -.04 -.20
Respondent formal education
.20 .13 .19 .31*
-.11
.33*
.12
.17
.34*
-.06 .29* .27*
.04 .11 .19
-.01 .26* .15
-.04 .21* .16
-.35* -.01
.32* -.22*
.27* .01
.13 .08
.20 -.05
-.14
-.17
.37*
.34*
.32*
.33
-.04
-.01
.49*
.01 .14
-.05 .16 .14 .14 .09
NateMl indexes scored so that a high score represents high values for the measure. * p < .05.
cation and tenure. Of the 18 correlation coefficients involving education and tenure, only the three most logical ones are significant. Other relationships are also of the type expected. Trust in superior, for example, is related to the respondent's overall job satisfaction, perceived considerate leadership style, perceived organizational competence and flexibilty, and commitment to the organization, but not to tenure and education. The amount of information an individual passes is significantly related to his overall job satisfaction and to his perception of his superior as being considerate, but not to other variables. Desire for interaction also shows appropriate relationships with responses to other perceptions about the organization. The negative relationship between information overload and overall job satisfaction and the positive correlation between communication satisfaction and considerate leadership are also understandable. Finally, the large number of significant relationships among overall job satisfaction and communication and noncommunication indexes seem logical. In sum, the results presented in this study are in no way definitive, but provide a crude approximation of convergent-discriminant validity and increase confidence in the indexes. Further examples of convergence and discrimination are obtainable by comparing perceptions about communication by people in
jobs dependent on information transmission (e.g., dispatchers and operations officers in the military units) with those of people occupying jobs not as sensitive to information transmission (e.g., mechanics). The results of these comparisons (not reported in this study) suggest that communication facets measured by this questionnaire can, in general, discriminate perceptions about communications for occupants of jobs dependent on information from those not so dependent. The former are more discriminating among the communication dimensions (they think more frequently about them). DISCUSSION The questionnaire can be considered from three perspectives: comparing organizations, work groups, or homogeneous groups of individuals. Our data suggest organizations can be differentiated on the communication dimensions. This implies that organizations might be taxonomically described in communication terms, for example, a taxonomy based on the openness and accuracy of communication. Work groups can also be contrasted similarly. Groups of individuals (e.g., workers and first-line supervisors) may also display unique communication behaviors. For even greater differentiation of communication problems, practitioners might find it more
326
KARLENE H. ROBERTS AND CHARLES A. O'REILLY III
beneficial to use individual items rather than communication indexes. The complexities of human behavior in organizations, including communications, should be better understood when they are more rigorously related to other organizational variables. For example, if there is a strong relationship between the degree to which organizational members perceive that the information transmitted at work is accurate and their performance, it should be advantageous for organizations to devote more time to correcting problems of accuracy. This suggests the obvious need to relate communication measures to performance criteria. Determining the relationships of communication variables to objective and behavioral criteria should aid in developing organizational communication models. Obviously much work remains to be done in developing a standardized measure that can describe various facets of communication across a variety of organizations. The data reported in the present study are exploratory and an initial attempt in this direction. The value of an easily completed descriptive measure of organizational communication seems obvious. If the case often made by organizational theorists is correct—that "good communication" makes a difference—then an understanding of what is "good communication" and its correlates should increase our knowledge of organizational behavior. REFERENCES CAMPBELL, J. P., & BEATY, E. Organizational climate: Its measurement and relationship to work group performance. Paper presented at the meeting of the American Psychological Association, Washington, B.C., September 1971. COHEN, A. Upward communication in experimentally created hierarchies. Sociological Review, 1958, 15, 342-351. DAVIS, K. Success of chain-of-command oral communication in a manufacturing management group. Academy of Management Journal, 1968, 11, 379387.
GRAVES, D. Reported communication ratios and informal status in managerial work groups. Human Relations, 1972, 25, 159-170. KUNIN, T. The construction of a new type of attitude measure. Personnel Psychology, 1955, 8, 65-78. LAWLER, E., PORTER, L., & TENENBATJM, A. Manager's attitudes toward interaction episodes. Journal of Applied Psychology, 1968, 52, 432-439. LAWSON, E. Change in communication nets, performance, and morale. Human Relations, 1965, 18, 139-147. LIKERT, R. The human organization. New York: McGraw-Hill, 1967. O'REILLY, C., & ROBERTS, K. Information filtration in organizations: Three experiments. Organizational Behavior and Human Performance, in press. PORAT, A., & HAAS, J. Information effects on decision-making. Behavioral Science, 1969, 14, 98-104. PORTER, L. W., & ROBERTS, K. Organizational communication. In M. Dunnette (Ed.), Handbook of industrial and organizational psychology. Chicago: Rand-McNally, in press. PORTER, L. W., & SMITH, F. E. The etiology of organizational commitment: A longitudinal study of the initial stages of employee-organization reactions. Unpublished manuscript, University of California, Irvine, 1970. READ, W. Upward communication in industrial hierarchies. Human Relations, 1962, 15, 3-16. ROBERTS, K., O'REILLY, C., BRETTON, G., & PORTER, L. Organizational theory and organizational communication: A communication failure? In L. W. Porter & K. H. Roberts (Ed.), Communication in organizations. Middlesex, England: Penguin, in press. (Human Relations, in press.) ROGERS, E., & BHOWMIK, D. Homophily-heterophily: Relational concepts for communication research. Public Opinion Quarterly, 1970, 34, 523-538. ROSEN, S., & TESSER, A. On reluctance to communicate undesirable information: The MUM effect. Sociometry, 1970, 33, 253-263. STOODILL, R., & COONS, A. Leader behavior: Its description and measurement. Columbus: Ohio State University, Bureau of Business Research, 1957. TRYON, R. C., & BAILEY, D. E. Cluster analysis. New York: McGraw-Hill, 1970. WILENSKY, H. Organizational intelligence. New York: Free Press, 1967. (Received April 30, 1973)
