Believe it or not! longer questionnaires have lower ... - Springer Link

J O U R N A L OF B U S I N E S S AND PSYCHOLOGY

Volume 4, No. 4, Summer 1990

BELIEVE IT OR NOT! LONGER Q U E S T I O N N A I R E S H A V E LOWER R E S P O N S E RATES Michael J. Roszkowski American College

Andrew G. Bean La Salle University

ABSTRACT: The effects of questionnaire length on response rate and response bias were examined by conducting a mail survey of 8534 persons taking "distance education" courses in insurance and financial planning. In 14 replications, persons were randomly assigned to receive a long or short course evaluation questionnaire. Response rate for the short form averaged about 28% higher than for the long form, and was significantly higher in all 14 replications. A measure of course satisfaction appearing on both questionnaires showed no significant differences between the long and short form in 12 of the 14 replications. The results suggested that biased measurement of consumer satisfaction does not necessarily occur on a long questionnaire with a relatively low response rate. S T A T E M E N T OF T H E PROBLEM In books and articles directed at practitioners, an often cited "rule of t h u m b " regarding the appropriate length for a questionnaire intended for use in a mail survey is t h a t the fewer the pages or items, the hi gher th e percentage of r e t u r n s (e.g. Erdos, 1974). This advice seems so sound and intuitively obvious t h a t research conducted to prove or disprove it mig h t appear to be a perfect candidate for Senator Proxmire's "Golden Fleece" award. Surprisingly, however, the results of studies t h a t have actually considered the relationship between a questionnaire's length and its r a t e of r e t u r n are not as conclusive as one might expect. In fact, much of t he empirical evidence summarized below suggests a counterintuitive conclusion: questionnaire length does not influence r e t u r n rates to a ny great extent. Some studies even imply t h a t to maximize r etu r n , one should m a k e the questionnaire longer! In this paper, we will review the l i t e r a t u r e concerning the effects of questionnaire length. T he n questionnaire length effects on response 495

9 1990 Human SciencesPress

496

JOURNAL OF BUSINESSAND PSYCHOLOGY

rate and response bias will be examined using an experimental design with 14 replications in an attempt to clarify some currently unresolved issues.

THE LITERATURE

Questionnaire Length and Response Rate Kanuk and Berenson (1975) published a literature review of available empirical studies concerned with the relation between response rate and various factors (length, monetary incentive, follow-up, etc.). They identified seven studies dealing with questionnaire length and found that only one of the seven reported a significant relationship between length and response rate. These reviewers concluded that "the evidence does not indicate that short mail questionnaires are more likely to receive higher response rates than long questionnaires" (p.443). A similar literature review (Linsky, 1975) identified seven studies that addressed length of questionnaire and response rate. Four of these studies were the same as those considered by Kanuk and Berenson (1975). Linsky noted that of this total, four found no differences, two reported higher returns for the longer questionnaire, and only one found a difference in favor of the short questionnaire. Thus, Linsky likewise decided that "the evidence to date provides little support for a blanket endorsement of shorter forms" (p.90). A more quantitative review of the literature on research design and response rates was conducted by Yu and Cooper (1983). Contending that response tendencies have changed over time, they limited themselves to data published between 1965 and 1981. For the analysis of length and response rate, only studies that reported questionnaire length in terms of number of items were considered. Studies representing 116 different reported response rates were examined. It is important to realize, however, that these were not 116 experiments in which both a short form and a long form were administered together. Rather, these were simply studies that happened to report both a response and length of questionnaire so that a correlation between the two could be computed. Finding a correlation coefficient of only - .06 between a survey's response rate and questionnaire length, Yu and Cooper concluded that "questionnaire length is nearly uncorrelated with the weighted average response rate associated with that length" (p.39). Another quantitative review of the literature that studied response rate in relation to questionnaire length was carried out by Heberlein and Baumgartner (1978). A survey's response rate was correlated with

MICHAELJ. ROSZKOWSKIAND ANDREWG. BEAN

497

each of 71 charactristics describing the survey. Questionnaire length, one of the 71 features examined for its possible influence on response rate, was measured three ways: (1) number of questions, (2) number of pages, and (3) estimated time required to complete the questionnaire. The data were extracted from 98 methodological studies reporting 214 separate response rates. Only methodological studies were considered because they reported enough detail to allow for this type of analysis. Like the other reviewers of this topic, these authors failed to find a significant zero-order correlation between length of the questionnaire and its rate of return, even though three different measures of length were used. However, they did discover that when questionnaire salience and the number of contacts were controlled, "instruments with more items got lower returns" (p. 453). Salience, as used by these authors, refers to the questionnaire's importance and timeliness to the potential respondent. It has been suggested that interest in the topic is an especially important variable when professionals are surveyed (Sudman, 1985). Goyder (1982), employing similar methodology and an overlapping sample of studies, attempted to reproduce Heberlein and Baumgartner's results. With regard to questionnaire length, he noted that the "replication reaffirms the small negative impact of length of questionnaire upon returns" (p. 551). However, a similar replication by Eichner and Habermehl (1981), based on European mail surveys, found that length was positively related to cooperation rate. In other words, longer questionnaires yielded higher response rates. In 1984, Baumgartner and Heberlein provided an update to their earlier review of the literature dealing with response rate. On the topic of length, they found two additional studies. The first investigated response rates to a one and two page questionnaire, presented on both 8 1/2 • 11" and 8 1/2 • 14" sized paper. The number of pages was not related to response rate, but the non-standard sized paper (8 1/2 • 14") had a negative impact on response. The second study dealt with time cues in the cover letter. Telling the potential respondents that it will take less time to complete the questionnaire produced a larger response rate. Given these contradictory results, Baumgartner and Heberlein (1984) concluded by advising that "the effect of a wide range of questionnaire lengths on response rate needs to be examined" (p. 70). One study not mentioned in Baumgartner and Heberlein's review was conducted by Adams and Gale (1982), who found a non-linear relationship between length and response rate. Mailing 1 page, 3 page, and 5 page questionnaires, they obtained the following respective response rates: 41%, 47%, and 22%. Also noteworthy is a study conducted by Powers and Alderman (1982) in which a long questionnaire (7 pages, 83 responses) was compared to a shorter questionnaire (6 pages, 69 responses). Here too, there was a significant difference in response rate

498


with 52% and 44% responding to the short and long questionnaires, respectively. An uncritical overview of the literature relating questionnaire length to response rate could easily lead one to conclude that this issue has been explored thoroughly. Upon closer examination, however, it becomes apparent that only a limited number of the investigations dealing with this topic can be considered to be controlled studies. Meta-analytic type reviews (such as the one conducted by Yu and Cooper, 1983), seem to be based on a large number of cases but they included studies that did not compare long and short versions of the same questionnaire. The issue of length and response rate was addressed in these meta-analytic reviews by searching for published surveys that listed both a questionnaire size and response rate thereby enabling one to correlate these two variables. While this type of study provides one source of useful evidence, analyses that pool response rates from entirely different surveys allow for the potential intrusion of confounding variables. As Adams and Gale (1982) note, recent conclusions on the relationship between questionnaire length and cooperation rate were based on the aggregation of cases (studies) that used survey forms constructed by different authors, dealt with a variety of topics, surveyed different populations, and used a number of procedures to distribute and collect the survey forms. It is possible that the aggregation process could have allowed for these differences to cancel each other out, but one cannot be certain that this actually happened. Controlled experimental studies using questionnaires of different lengths could avoid many of the problems in interpretation that arise from many of the above extraneous variables. Unfortunately, the few experimental studies that have been published do not lead to any conclusive answer to the question of whether, in a mail survey, there exists a relationship between questionnaire length and willingness to complete and return it. A major problem is that the operational definitions of "short" and "long" vary dramatically across experimental studies. In some cases the difference between long and short versions of a questionnaire may be insufficient to create a significant effect. Clearly, more studies of an experimental design are needed in which there is a realistic but substantial difference in the size of the two questionnaires.

Questionnaire Length and Response Bias Houston and Ford (1976) contend that the scope of research on surveys needs to be broadened beyond response rate to include "response quality" issues such as response error and response bias. Presently, the literature on this subject is relatively sparse. This is surprising since the practitioner's major concern with low response is that biased informa-


499

tion will be collected. In one of the few studies addressing the issue of response quality as it relates to questionnaire length, it was found that "straight line" (sterotypical) responding is more likely to occur on long questionnaires (Herzog & Bachman, 1981). It has been suggested that another possible bias is that long questionnaires with low response rates are returned to a disproportionate degree by individuals who have very strong opinions about an issue, either positive or negative (Sudman & Bradburn, 1984), but further data on this matter would be welcome. A few other studies support the idea that an increase in response rate does not automatically mean a reduction in bias. Hutchison, Tollefson, and Wigington (1987) elicited attitudes toward the college environment using (1) an on-site classroom survey with a 100% rate of cooperation, (2) a mail survey with only a 35% response rate and (3) a telephone follow-up of the non-respondents to the mail survey. Their findings indicate no significant differences in the answers obtained through these three methods, despite the dramatic difference in percentage of returns. Similarly, Denton, Tsai, and Cheverette (1988) report that while offering an incentive to respond (i.e., newsletter) or making multiple mailings produced a better response rate, the answers given under these circumstances were not markedly different from ones obtained from subjects not receiving these additional treatments. In fact, according to Ray and Still (1986) successful attempts to induce greater cooperation in a postal survey may actually produce biased responding in some instances. In their study, a follow-up letter to non-respondents increased the return rate from 25% to 47% but this apparent improvement may have been counterproductive in reality since the individuals responding to the follow-up answered the questionnaire in an acquiescent response set (i.e., a tendency to agree with all statements on the form, even contradictory ones). In the study to be presented here, the effects of two substantially different questionnaire lengths on both return rate and response bias are considered. The design is experimental in nature. An additional strength of this study is that it involves replications of the same two surveys across 14 different samples drawn from the same population. Replications or near replications, which are rare in survey research (Glenn, 1983), can lead to greater confidence in one's conclusions.

METHOD

Subjects The data were gathered from 8534 persons taking courses developed by The American College, an accredited institution providing profes-

500


sional education in life insurance and related financial services. Programs offered by the College include the Chartered Life Underwriter (CLU) designation, the Chartered Financial Consultant (ChFC) designation, as well as a Master's degree in financial services and management. The courses in each program are created by the resident faculty at The American College and are made available nationally through "distance education". This means that individuals taking American College courses are free to study either independently or by participating in organized study groups or classes held in their local community. Irrespective of method of preparation, credit toward an American College designation or degree can only be earned by passing a proctored, standardized examination administered nationally at various centers. A majority of the college's students are already employed in the financial services industry and undertake the courses independently without the benefit of classroom instruction.

Instrumentation In view of the College's method of delivery of instruction, course evaluations must be conducted through mail surveys. Unlike traditional course evaluations, in which the instructor is the focus of evaluation, American College evaluations focus exclusively on the course. That is, instead of asking for feedback about the instructor, feedback is sought about the adequacy of the study materials. Determining student sarisfaction with courses is a central concept in the marketing of higher education (Lundsten, 1986). This process is analagous to the measurement of post-purchase satisfaction with a product, a key issue in marketing. A short and a long questionnaire were used to evaluate each of 14 courses. The short questionnaire was a postcard measuring 8 1/2 by 5 1/2 inches with questions on one side and return address on the other. The long questionnaire consisted of a continuous form measuring 8 1/2 by 22 inches (two 8 1/2 by 11 inch pages joined together) with printing on both sides of each page. Respondents returned the long questionnaire by simply refolding it to expose the return address and prepaid postage stamp. The short questionnaire was 25% of the long questionnaire in size. The long and short forms were designed for different purposes. The short form, which only solicits global impressions about a course, was intended to be used to track general summative opinions. The long form, on the other hand, was meant to be used as a diagnostic device for determining why a course failed to meet certain students' expectations. However, the long form provided summative information identical to that obtained from the short form. Specifically, the long questionnaire was divided into four sections: (1) self-descriptive demographic data, (2) evaluations of course assign-

MICHAEL J. ROSZKOWSKIAND ANDREWG. BEAN

501

ments, (3) evaluations of the course study guide, and (4) overall course impressions. The short questionnaire consisted of all of the questions in section four of the long form plus the major demographic questions of section one. The response format for all four sections was similar, consisting primarily of questions requiring either a multiple choice response or the selection of a point on a rating scale. In the few cases where an openended question was used, it was primarily to allow the respondents to elaborate on their answers to the standardized questions. The diagnostic ratings called for by sections 2 and 3 of the long questionnaire were quite detailed, requiring the respondent to rate course materials on a chapter by chapter basis. Thus, the long form of the questionnaire placed a substantially greater burden on the respondent than a simple comparison of physical sizes would imply. For example, a series of specific assignments were rated on a five point scale ranging from "very good" to "very poor". If the assignment was rated "poor" or "very poor", the respondent was asked to check one or more of a standardized set of reasons (e.g. too technical", "too elementary", "too detailed", "not enough examples", etc.). The final item on both the short and the long survey form requested a rating of overall satisfaction with the course on a three point scale. We focused on this last "bottom line" question as the basis of our analysis of bias because it best summarizes the evalution of the course by the respondent.

Survey Design The surveys presented in this article consist of all available studies in which both a short form and a long form were used at the same time. The course evaluations were timed to be received by the potential respondent before the scheduled examination date. To control survey costs, students enrolled in each course were randomly sampled from the population. Recipients were randomly assigned to be mailed either the long or the short version of the questionnaire. Although the number of persons who received each form varied from one replication to the next, in all cases respondents were randomly assigned to the long or the short form. In most cases, the number of persons who received the long form was larger than the number who received the short form because of the anticipated lower response rate in the long form condition and the desire to keep the number of respondents in each group approximately equal. Thus, the survey procedures in effect were a true experimental design in which assignment to the long vs. the short form constituted an independent variable and differences in response rate and satisfaction

502

JOURNAL OF BUSINESS AND PSYCHOLOGY

were dependent variables. Repeated surveys across semesters served as independent replications of the experiment. Some potentially important extraneous variables were controlled by having several features common to both questionnaires: (1) The summative evaluation questions on the post card were identical in content and format to those included on the longer version; (2) insertion into an envelope was not required to return the questionnaire; (3) postage was prepaid; (4) the content of introductory cover letters was identical; (5) the forms were professionally typeset and self-addressed; (6) a followup questionnaire, identical to the first, was mailed to non-respondents to the first mailing. RESULTS Response Rate

Table 1 presents return rates for long and short versions of the questionnaire across 14 different mailings. In every case, return rates for the short form were significantly greater than the long form, based on a Chi-square test of significance (p < .01). Although the return rates varied considerably across replications, the differences observed between long and short forms were consistently in the same direction. The difference between long and short form response rates averaged about 28%. Even though the physical dimensions of the long questionnaire was identical in all 14 replications, the amount of effort required to complete it varied from course to course depending on the number of chapters to be rated. This amount of effort can be quantified by counting the number of strokes (marks) that a person would have to make if all questions were answered (see Table 1). The mean number of strokes was 240 ( S D = 97) on the long form, and 15 ( S D = 1) on the short form. Due to variations in the number of strokes within the long and short forms, an additional analysis was conducted to determine the relationship between the number of strokes and response rate. The following three steps were used to obtain a correlation: (1) For each of the 14 replications, the number of strokes on the short form was subtracted from the number of strokes on the long form; (2) For each of the 14 replications, the response rate (%) for the long form was subtracted from the response rate (%) for the short form; (3) The 14 pairs of scores derived above were correlated. The resulting Pearson r indicates the extent to which a difference in response rate is a function of the increase in strokes required to complete the long questionnaire. A Pearson r of .50 (p < .05) was obtained, providing further evidence that length is significantly related to questionnaire response rate.

Table 1 Response Rates to Short and Long Survey Forms

Strokes Course

Number Sent

Percent Returned

Short Long Short Long Short Long

HS 329 Wealth Accumulation Planning

17

209

500

1000

88

62

HS 320 Financial Services: Environment & Professions

15

212

300

500

77

46

HS 330 Estate & Gift Tax Planning

14

377

200

300

84

46

HS 329 Wealth Accumulation Planning

15

208

200

300

80

47

HS 322 Economics

14

281

200

300

78

39

HS 326 Pensions and Other Retirement Plans

14

383

200

300

87

39

GS 511 Advanced Estate Planning

14

53

200

234

92

60

HS 323 Financial Statement Analysis

16

259

200

300

81

51

HS 325 Group Benefits and Social Insurance

16

233

200

300

77

48

HS 331 Planning for Business Owners and Professionals

16

276

200

300

77

51

HS 323 Financial Statement Analysis

17

373

300

300

69

47

HS 332 Financial and Estate Planning

14

129

200

300

90

74

HS 332 Financial and Estate Planning

15

129

300

300

75

57

HS 325 Group Benefits and Social Insurance

17

240

300

300

76

52

3500 5034

81

53

Pooled Data

N o t e - - A l l r e t u r n r a t e s are significantly different at p < .001.

504


Another w a y to study this issue is to simply look at the correlation between strokes and response rate within the long form. The obtained correlation between the number of strokes and response rate was - . 7 1 , (p < .05).

Response Bias The levels of satisfaction expressed by the respondents on the long and short forms are shown in Table 2. In 12 of 14 replications, no statistically significant differences in satisfaction were noted between long and short versions of the questionnaire. When the satisfaction data were pooled across replications, the frequency distributions were quite similar for the long and short forms (See Table 2). In the two significant differences that were obtained between long and short versions, the magnitude of the bias appears to be small. The obtained values of Cramer's V were .10 and .18. (Cramer's V is a measure of association based on Chi-square, and is analogous to the correlation coeficient.) The Cramer's V of .10 is statistically significant but the differences in satisfaction levels between the long and short forms are too small to be of practical concern (See Table 2). For the case with Cramer's V = .18, the difference in satisfaction ratings between the long and short forms might be viewed by some as being large enough to have practical importance; short form responses had a higher percentage of cases in the middle rating point as compared to the long form. This relationship could be interpreted as evidence for the commonly held notion that longer questionnaires are likely to be returned by people who hold extreme attitudes. However, one needs to bear in mind that a relationship of this magnitude was observed on only one of the 14 replications. The pooled data in Table 2 suggest that the degree of bias resulting from the lower response rate of the long form is either very small or non-existant (Cramer's V = .03). It is especially important to note that this freedom from bias occured even though the long form caused a substantially lower response rate. The data shown in Table 2 were also subjected to a two-way analysis of variance. The independent variables were form (long vs. short) and course; the dependent variable was level of satisfaction. A significant difference in satisfaction level between forms would indicate that response bias was occuring due to the lower response rate obtained with the long form. This analysis has the following three advantages over a Chi-square analysis: (1) all of the data are tested in a single analysis; (2) mean differences between forms and replications are tested; and (3) statistical power is increased. The results of the analysis are straightforward. No significant difference in satisfaction was found across forms; significant differences

Table 2 Reported Level of Satisfaction as a F u n c t i o n of Questionnaire Length

Course

Number Form Returned

Very Satisfied

Satisfied Dissatisfied

Cramer's V

HS 329 Short Long

434 579

30% 34%

52% 55%

18% 11%

.10"

HS 320 Short Long

223 216

28% 27%

56% 53%

16% 19%

.04

HS 330 Short Long

162 130

47% 47%

44% 41%

9% 12%

.06

HS 329 Short Long

156 137

42% 48%

51% 46%

6% 6%

.06

HS 322 Short Long

151 108

27% 30%

58% 48%

15% 22%

.11

HS 326 Short Long

169 102

28% 21%

58% 66%

14% 14%

.09

GS 511 Short Long

180 135

58% 58%

37% 38%

5% 4%

.01

HS 323 Short Long

155 135

11% 7%

79% 80%

10% 13%

.07

HS 325 Short Long

148 137

21% 22%

69% 73%

10% 5%

.09

HS 331 Short Long

149 136

32% 37%

66% 60%

2% 4%

.08

HS 323 Short Long

202 130

13% 13%

68% 67%

19% 20%

.02

HS 332 Short Long

173 213

18% 18%

58% 58%

24% 24%

.01

HS 332 Short Long

225 167

19% 31%

72% 55%

9% 14%

.18"

HS 325 Short Long

226 152

20% 22%

74% 71%

6% 7%

.03

2753 2477

28% 30%

60% 57%

12% 13%

.03

Pooled Data

Short Long

* S i g n i f i c a n t a t p < .01.

506


were indicated between courses F (13, 5202) = 9.94, p < .001. These differences would be expected in that respondents would ordinarily show different satisfaction levels with different "products". No significant interaction between forms and courses was found. Thus, the results of the ANOVA also failed to provide evidence of response bias due to form length.

DISCUSSION Mail surveys provide a means for collecting information within a short time-span from a large number of people at a relatively low cost. Invariably, cooperation is obtained from fewer than 100% of those contacted, thereby creating concerns about potential bias in the answers to survey questions. A variety of methods for increasing motivation to participate in mail surveys have been suggested. One seemingly obvious method for increasing response is to decrease the demand placed on the potential respondent by shortening the questionnaire. Surprisingly however, the limited research dealing with questionnaire length and willingness to respond has been rather equivocal. We believe that due to its experimental design incorporating a series of replications, the analysis reported here is less subject to the influence of extraneous variables and the vagaries of chance, thus offering more conclusive evidence on this issue than previously available. Our results support the common sense (but empirically disputed) notion that questionnaire length is inversely related to response rate. It would not be reasonable to believe that the lower response rates for the longer questionnaires in this study were mere chance occurances given that they were consistently observed across 14 different course-semester combinations and that the sample sizes upon which these results are based were more than respectable in all instances. Moreover, the finding that response rate was negatively correlated with the number of questions on the long questionnaire indicates that questionnaire length is not just a matter of physical size. Even within a questionnaire of the same physical size, the presence of additional questions lowered the response rate. If the findings of the studies in the literature with results contrary to those reported here are similarly not to be dismissed as instances of Type II error, then it becomes necessary to reconcile the findings from the different investigations. We believe that our data support those who contend that the two crucial factors moderating the relationship between questionnaire length and willingness to respond to it are (1) salience of the issues addressed in the questionnaire and (2) the exact operational definitions of "long" and "short" questionnaires. Our find-


507

ings suggest that increased length will be detrimental to return rate when (1) the difference between short and long forms becomes sufficient to place a significantly greater burden on the respondent and (2) questionnaire salience is not high to the potential respondent. The reason why we believe our questionnaires met the above two criteria is elaborated below. In our study, the long questionnaire was four times as long in terms of physical dimensions, but the time and effort required to complete the long questionnaire were considerably greater than the physical size differences would suggest. The short questionnaire required only global judgments about satisfaction, and these ratings could be made quickly by the respondent without much effort or thought. The longer questionnaire required the respondent to provide detailed ratings of specific parts of the course. Ratings of this type require careful analysis and perhaps even a need to actually examine the course materials in order to respond. Thus, the longer questionnaire placed enough additional burden to deter some respondents who might have answered the shorter, simpler form. The salience of the questionnaires to the respondents is, in our opinion, moderate. At the point that the questionnaires were being filled out, the students had already completed study of the material. Cerisequently, persons completing the questionnaire would not benefit directly from any improvements in the course that resulted from their feedback. On the other hand, they were still involved with the course since they had not yet taken the final examination. Moreover, they were individuals who undertook study in the programs offered by the American College and must therefore have some commitment to the institution and the industry. If salience were very high, differences in response rate as a function of questionnaire length might have been less dramatic than those observed in this study. From a practical standpoint, both positive and negative implications may be drawn from our findings. The "bad news" conveyed in these results is that one cannot burden respondents with a long questionnaire without paying the penalty of a lower response rate. The "good news" suggested by these results is that lower response rates may not necessarily yield biased measures of consumer satisfaction. While one might intuitively expect dissatisfied persons to be more likely to make the extra effort to express their opinions on a long questionnaire, there is little in our data to support this point of view. Neither were very satisfied persons likely to put up with the added imposition. Naturally, the generality of these findings needs to be established across different subjects, issues, and questionnaire lengths using controlled experiments. The data presented here add to the body of evidence suggesting that

508


m a n y of the methods m e a n t to increase response rate, even when successful, do not necessarily improve the quality of the information that one obtains through a mail survey (Denton, Tsai, and Cheverette, 1988; Hutchinson, Tollefson, and Wigington, 1987; Ray and Still, 1986). In the 14 surveys discussed in this report, shortening the questionnaire did improve the rate of cooperation, but the nature of the answers on the issue of product satisfaction were remarkably similar on the two forms.

CONCLUSIONS When a body of research leads to results so counter-intuitive that they seem to defy common sense, a reexamination of the phenomenon m a y be required. Our data suggest that, in the relationship between questionnaire length and response rate, common sense does indeed prevail; substantially longer questionnaires have lower response rates. An equally important finding is that a significantly lower response rate does not necessarily lead to biased results on a measure of consumer satisfaction.

REFERENCES Adams, L. and Gale, D. (1982). Solving the quandary between questionnaire length and response rate in educational research. Research in Higher Education, 17, 231-240. Baumgartner, R.M. & Heberlein, T.A. (1984). Recent research on mailed questionnaire response rates. In D.C. Lockhart (Ed.), Making effective use of mailed questionnaires (pp 65-75). San Francisco: Jossey-Bass. Denton, J. J. Tsai, C. Y. and Chevrette, P. (1988). Effects on survey responses of subjects, incentives and multiple mailings. Journal of Experimental Education, 56, 77-82. Eichner, K. & Habermehl, W. (1981). Predicting response rates to mailed questionnaires (comments on Heberlein and Baumgartner). American Sociological Review, 46, 361-363. Erdos, P.L. (1974). "Data collection methods: mail surveys". In R. Ferber (Ed.), Handbook of marketing research (pp. 2-90, 101-104). NY: McGraw-Hill. Glenn, N.D. (1983). Replications, significance tests and confidence in survey research. Public Opinion Quarterly, 47, 261-269. Goyder, J.C. (1982). Further evidence on factors affecting response rates to mailed questionnaires. American Sociological Review, 47, 550-553. Heberlein, T.A. and Baumgartner, R. (1978). Factors affecting response rates to mailed questionnaires: A quantitative analysis of the published literature. American Sociological Review, 43, 447-462. Herzog, A. R. & Bachman, J.G. (1981). Effects of questionnaire length on response quality. Public Opinion Quarterly, 45, 549-559. Houston, M.J. & Ford, N.M. (1976). Broadening the scope of methodological research on mail surveys. Journal of Marketing Research, 13, 397-403. Hutchison, J., Tollefson, N. and Wigington, H. (1987). Response bias in college freshmen's response to mail surveys. Research in Higher Education, 26, 99-106. Kanuk, L. & Berenson, C. (1975). Mail surveys and response rates: A literature review. Journal of Marketing Research, 12, 440-453.

MICHAEL J. ROSZKOWSKI AND ANDREW G. BEAN

509

Linsky, A.S. (1975). Stimulating responses to mailed questionnaires: A review. Public Opinion Quarterly, 39, 82-101. Lundsten, L.L. (1986). Student evaluation in a business administration curriculum. Developments in Marketing Science, Vol. IX (pp. 77-87). Proceedings of the Tenth Annual Conference of the Academy of Marketing Science. Anaheim, California, April 30-May 3, 1986. Powers, D.E. and Alderman D.L. (1982). Feedback as an incentive for responding to a mail questionnaire. Research in Higher Education, 17, 207-211. Ray, J. J. & Still, L. V. (1986). Maximizing the response rate in a survey may be a mistake. Personality & Individual Differences, 8, 571-573. Sudman, S. (1985). Mail surveys of reluctant professionals. Evaluation Review, 9, 349-360. Sudman, S. & Bradburn, N. (1984). Improving mailed questionnaire design. In D.C. Lockhart (Ed.), Making effective use of mailed questionnaires (pp. 33-47). San Francisco: Jossey-Bass, Inc. Yu, J. & Cooper, H. (1983). A quantitative review of research design effects on response rates to questionnaires. Journal of Marketing Research, 20, 36-44.