The Influence of Academic Rigor on Factors ... - Wiley Online Library

PART III. IMPLICATIONS REFRAMED

OF

RIGOR

This part includes chapters that describe the importance of the reframed notion of rigor for student success, equity, and institutional and federal policy.

6

This chapter describes research findings that show a positive relationship between higher order examination questions and core concepts of empirically supported theories of college student persistence for both residential and commuter colleges and universities.

The Influence of Academic Rigor on Factors Related to College Student Persistence John M. Braxton, Clay H. Francis Student success in college requires persistence to graduation. Without persistence to graduation, other markers of college student success become less likely to achieve. These markers include acquisition of general education, development of academic competence, development of cognitive skills and intellectual dispositions, preparation for adulthood and citizenship, and personal development (Braxton, 2008). First-year persistence is integral to persistence to graduation. However, first-year to second-year student retention hovers around 70% for baccalaureate private colleges and universities and 64.9% baccalaureate public colleges and universities (American College Testing Program, 2016). Retention rates hold significance for colleges and universities given the role they play in institutional budgeting and as an index of the college student experience for the lay public. Colleges and universities seeking improvement in their institutional retention rates may implement policies and practices designed to achieve such improvements. However, scholars such as Arum and Roksa (2011) raise substantive concerns about the consequences of institutional efforts to enhance their likelihood of persistence in college. The crux of their argument is that “policy makers and practitioners alike have focused on keeping students in college assuming that they will learn. The simple act of staying enrolled does not ensure that students are learning much” (Arum & Roksa, 2011, p. 136). They contend enhanced learning transpires through increased academic rigor. Their contention raises an important issue about the role of academic rigor in college student persistence. Does academic NEW DIRECTIONS FOR HIGHER EDUCATION, no. 181, Spring 2018 © 2018 Wiley Periodicals, Inc. Published online in Wiley Online Library (wileyonlinelibrary.com) • DOI: 10.1002/he.20272

73

74

NEW UNDERSTANDINGS FOR EQUITY AND STUDENT SUCCESS

rigor lead to student departure or to student persistence in college? We address the significance of this question in the formulations of our conceptual framework. The types of questions faculty ask students during class, the nature of term papers or other written assignments, and course examination questions written by faculty members constitute course processes through which faculty convey their academic demands for student course achievement. More specifically, Nordvall and Braxton (1996) define academic demands or course rigor as the level of understanding of course content required by such course processes. Levels of understanding range from a knowledge level of understanding (e.g., recall and recognition) to higher order levels of understanding (e.g., analysis and synthesis). The higher the level of understanding required, the greater the level of academic rigor expected. Greater levels of academic rigor challenge students to achieve their full potential for academic learning and broader development. In this chapter, we focus on the role that course examination questions play in the process of student persistence in private residential colleges and universities and in publicly supported commuter colleges and universities. These examination questions include a students’ knowledge of facts (recall and recognition/comprehension) as well as three types of higher order questions that require students to: compare and contrast course content, point out the strengths and weaknesses of particular arguments or points of view, and to argue for or against a particular point of view and defend their argument. These higher order questions also fit conceptions of critical thinking (Furedy & Furedy, 1985). We used first-year college students enrolled in residential and commuter colleges and universities to determine the role that these four types of examination questions play in the process of college student persistence. Residential and commuter colleges and universities provide an opportunity to ascertain whether the role of these four types of examination questions in the student persistence process transpires across distinctly different types of college and universities or if this role depends on the institutional context. Residential and commuter institutions differ in the residential arrangement of first-year students, as students in residential institutions live in oncampus housing facilities whereas students in commuter institutions live off campus either with their parents, by themselves, or with a significant other. Institutional control stands as another point of distinction. The residential institutions included in this study are all privately affiliated four-year colleges and universities whereas all the commuter institutions are statesupported four-year colleges and universities. To ascertain the part that the four types of examination questions play in college student persistence in residential and commuter colleges and universities, we used two different theories of college student persistence: the revised theory of student persistence in residential colleges and universities and the theory of student persistence in commuter colleges and universities. NEW DIRECTIONS FOR HIGHER EDUCATION • DOI: 10.1002/he

THE INFLUENCE OF ACADEMIC RIGOR ON COLLEGE STUDENT PERSISTENCE

75

These two theories and empirical tests of them are described in Rethinking College Student Retention (Braxton et al., 2014). We briefly describe the major components of these two theories in the following two sections.

The Revised Theory of Student Persistence in Residential Colleges and Universities This theory regards student persistence as a longitudinal process (Braxton et al., 2014; Tinto, 1975). Social integration stands as the core concept of this theory because of its position between its antecedents on one hand and subsequent institutional commitment on the other hand. Social integration refers to the student’s perception of their degree of social affiliation with other students and their degree of congruency with the attitudes, norms, and values of the social communities of their college or university (Tinto, 1975). Six antecedents positively influence social integration and social integration influences subsequent institutional commitment in a positive manner. Subsequent intuitional commitment, in turn, positively influences student decisions to persist in a residential college or university (Braxton et al., 2014). Ability to pay, communal potential, institutional commitment to the welfare of students, institutional integrity, proactive social adjustment, and psychosocial engagement constitute these six antecedents (Braxton et al., 2014). Only institutional commitment to the welfare of students and institutional integrity hold relevance to the current study. We subsequently define these two antecedents of social integration.

The Theory of Student Persistence in Commuter Colleges and Universities This theory also views student persistence as a longitudinal process (Braxton et al., 2014). Academic and intellectual development plays a pivotal role in this theory as it stands between its antecedents and subsequent institutional commitment. The antecedents to academic and intellectual development include the organizational attributes of commitment of the institution to student welfare and institutional integrity (Braxton et al., 2014). These two organizational attributes influence academic and intellectual development in a positive manner which, in turn, positively affects subsequent institutional commitment (Braxton et al., 2014). Subsequent institutional commitment positively influences student decisions to persist in a commuter college or university (Braxton et al., 2014).

Common Concepts of Both Theories Commitment of the Institution to Student Welfare and Institutional Integrity constitute the concepts common to these two theories (Braxton NEW DIRECTIONS FOR HIGHER EDUCATION • DOI: 10.1002/he

76


et al., 2014). Empirical tests of these theories provide support for their role as antecedents to the core dimensions of these theories: social integration for residential colleges and universities and academic and intellectual development for commuter colleges and universities (Braxton et al., 2014). We describe these two empirically backed concepts in the following paragraphs. Commitment of the Institution to Student Welfare. The Commitment of the Institution to Student Welfare manifests itself as an institution’s abiding concern for the growth and development of its students (Braxton et al., 2014). An institution committed to the welfare of its students also clearly communicates the high value it places on students in groups as well as individuals (Braxton et al., 2014). The equitable treatment of students and respect for them as individuals constitute additional aspects of this organizational attribute. Institutional Integrity. Institutional Integrity is the extent to which a college or university is true to its espoused mission and goals (Braxton et al., 2014). Institutional integrity demonstrates itself when the actions of a college or university’s administrators, faculty, and staff are compatible with the mission and goals proclaimed by a given college or university (Braxton et al., 2014). Returning to our focus on the role of course examination questions in the student persistence process, we posit the four types of examination questions as possible sources of influence on student perceptions of the institutional integrity of their college or university and the commitment of the institution to student welfare in both commuter colleges and universities and residential colleges and universities. The formulations of the conceptual framework contained in the next section of this chapter provide a rationale for the possible indirect and direct influence of these four types of examination questions on college student persistence in both commuter and residential colleges and universities.

Conceptual Framework The examination process stands as a pivotal moment in the learning process. Through graded and ungraded course assignments, professors gauge how much students have learned and how well they have learned that information. The types of exam questions that professors write dictate how students will study (Scouller, 1998). More specifically, students’ studying processes reflect both the way that professors have taught and what professors expect from students on examinations (Biggs, 1979; Fransson, 1977; Gibbs & Simpson, 2005). For example, questions that require rote memorization—usually multiple choice questions or lower order thinking questions related to recall or comprehension (Bloom, Engelhart, Furst, Hill, & Krathwohl, 1956)—prepare students for surface level learning (Biggs, 1979). Tang (1992), framed this type of examination as one that assesses the NEW DIRECTIONS FOR HIGHER EDUCATION • DOI: 10.1002/he


77

quantity and not the quality of information that a student learns in a course. Similarly, Maier (1933) highlighted this type of examination as largely reproducing knowledge rather than reasoning through it. Since students typically understand which types of examinations demand the higher order thinking skills that lead to deeper learning, they often do not engage in rigorous study processes for multiple choice exams (Gibbs & Simpson, 2005). Simply put, multiple choice question examinations lead to limited student learning (Newble & Jaeger, 1983; Ramsden, 1988; Scouller, 1998; Thomas & Bain, 1982; Watkins, 1982). Examination questions also influence student perceptions of how well they have learned the information being tested—when they believe, the assignment requires lower order thinking, they will report learning less from that part of the curriculum (Scouller, 1998). Moreover, recent research demonstrates that answering higher order questions in coursework enhances the critical thinking abilities of students. Based on the three studies they conducted, Renaud and Murray (2007) conclude “the findings of this research clearly indicate that students are more likely to improve their critical thinking skills when they have answered higher-order questions in their coursework” (p. 345). Students who frequently encounter in their courses the three higher order examination questions and infrequently encounter examination questions that are limited to a student’s knowledge of facts also experience increased learning in their courses and further development of their critical thinking skills. Student perceptions of their increased course learning and increased development of their critical thinking skills leads, in turn, to their belief that their college or university holds an abiding concern of their growth and development. As a result, students come to perceive that their college or university is highly committed to the welfare of its students because of their experience taking course examinations that require higher order thinking skills. Moreover, students also form positive perceptions of the institutional integrity of their college or university given that most colleges and universities espouse student learning and the development of critical thinking skills as institutional goals. Student perceptions of their increased course learning and development of their critical thinking skills lead students to believe that their college or university is true to its espoused institutional mission and goals. Put differently, students who frequently experience course examinations that require higher order thinking skills perceive their college or university as being true to its espoused mission of the development of student critical thinking skills. These formulations lead to the following two hypotheses: (1) the more frequently students encounter each of the three higher order course examination questions, the more positive their perception of commitment of their college or university to the welfare of its students and of its institutional integrity; (2) the less frequently students encounter course examination questions that are NEW DIRECTIONS FOR HIGHER EDUCATION • DOI: 10.1002/he

78


limited to a student’s knowledge of facts, the more positive their perception of commitment of their college or university to the welfare of its students and of its institutional integrity.

Data Sources We tested the above two hypotheses as a secondary analysis of the data collected and described in Rethinking College Student Retention (Braxton et al., 2014). We tested these two hypotheses in religiously affiliated residential colleges and universities and in commuter colleges and universities. The rationale and selection process for these two samples are described in Rethinking College Student Retention (Braxton et al., 2014). For the test of these two hypotheses in religiously affiliated residential colleges and universities, we used the sample of 408 first-time, full-time undergraduate students at eight religiously affiliated private residential colleges and universities. We used the sample of 714 students enrolled at five state-supported commuter colleges and universities who have completed two or fewer semesters at their college or university and who live at home with their parents, their spouse, or partner, or by themselves to test the above two hypotheses in commuter colleges and universities. A response rate of 53.2% was attained across the five supported commuter institutions. Such a response rate raises questions about the extent of bias that may exist in this sample. However, Braxton et al. (2014) found that the demographic characteristics of the populations sampled in the commuter institutions were generally similar to the demographic profile of the entire institution. With regard to the sample of religiously affiliated private residential colleges and universities, Braxton et al. (2014) obtained a response rate of 28.4% across the eight institutions. This response rate raises questions about the extent of bias that may exist in this sample. However, cases from two of the eight institutions were weighted to ensure some degree of representativeness to their respective campus populations on gender. Rethinking College Student Retention (2014) provides additional detail about these two samples.

Measures of Course Examination Questions, Commitment of the Institution to the Welfare of Its Students, and Institutional Integrity In this section, we delineate the four survey items used to measure the frequency in which students encountered each of the four types of examination questions used in this inquiry. We derived these four items from survey instruments used in the development of the two sources of data described in the above section of this chapter. We also provide examples of survey items used to measure commitment of the institution to student welfare and institutional integrity. NEW DIRECTIONS FOR HIGHER EDUCATION • DOI: 10.1002/he


79

Course Examination Questions. Although the instructions for responding to these four items differ between the two surveys, the items are identical in their wording. Respondents were asked to indicate the frequency in which they encountered each of the four types of examination questions since entering their college or university in the case of residential college and university survey respondents and classes that you have taken or are currently taking this academic year in the case of commuter college survey respondents. Respondents used a four-point scale to record the frequency of their encounters (1 = never, 2 = occasionally, 3 = often, and 4 = very often). These survey items depicting each of the four focal types of examination questions are as follows:

• “Most exams require me to compare or contrast dimensions of course

content.”

• “Most exams require me to point out the strengths and weaknesses of a

particular argument or point of view.”

• “Few exams require me to argue for or against a particular point of view

and defend my argument.” We reverse scored this item for our regression analysis so that a score of four equals one or never. Put differently, the higher the score, the less frequent the occurrence of this type of examination question. • “Most exam questions are limited to my knowledge of facts.” We reverse scored this item for our regression analysis so that a score of four equals one or never. Thus, the higher the score, the less frequent the occurrence of this type of examination question.

Commitment of the Institution to the Welfare of Its Students. We measured this concept using a composite scale comprised of ten items measuring student perceptions that the institution is committed to the welfare of students. These items include: “most student services staff (e.g., dean of students’ office, student activities, housing, etc.) are genuinely interested in students,” faculty members treat students with respect, student services staff treat students with respect, other college/university staff treat students with respect. Respondents used a four-point scale to respond (strongly disagree = 1; strongly agree = 4) (Braxton et al., 2014). Institutional Integrity. We used a composite scale comprised of five items measuring student perceptions that the institution exhibits integrity. Examples of these items include: “the actions of the administration are consistent with the stated mission of this institution,” “the rules of this institution appear in harmony with the values the institution espouses,” and the decisions made at this institution rarely conflict with the values it espouses (strongly disagree = 1; strongly agree = 4) (Braxton et al., 2014). NEW DIRECTIONS FOR HIGHER EDUCATION • DOI: 10.1002/he

80


Statistical Analyses To test the two hypotheses of this study, we used ordinary least squares multiple linear regression with fixed effects for the specific institutions represented in these two data sources. We also controlled for such student background characteristics such as gender, race/ethnicity, and parental educational level and high school grades. More specifically, we carried out two regression analyses with the dependent variables being commitment of the institution to student welfare and institutional integrity for both residential colleges and universities and commuter colleges and universities. We refer readers to tables A.2 and A.6 in Rethinking College Student Retention (Braxton et al., 2014) for the means and standard deviations of these variables.

Findings: The Influence of Examination Questions on Commitment of the Institution to the Welfare of Its Students and Institutional Integrity We array our findings by the type of colleges and universities included in this study. We derive our findings from the regression analyses displayed in Tables 6.1 and 6.2. Commuter Colleges and Universities. The results of the regression analyses shown in Table 6.1 indicate that two types of examinations positively influence student perceptions of the commitment of the institution to student welfare and institutional integrity. More specifically, the more frequently students report that “most examination questions require a student to point out strengths and weaknesses of a particular argument or point of view,” the greater their perception that their college or university demonstrates institutional integrity and embraces a commitment to the welfare of it students. Likewise, the more frequently students indicate, “most examination questions require me to compare or contrast dimensions of course content,” the greater their perception that their college or university demonstrates institutional integrity and embraces a commitment to the welfare of it students. These findings support the first hypothesis of this study. As shown in Model 3 of Table 6.1, examination questions that require students to point out strengths and weaknesses of a particular argument or point of view also positively influences student perceptions of their degree of academic and intellectual development, one of the core concepts of the theory of college student persistence in commuter colleges and universities. Through their empirical tests of the theory of commuter student persistence, Braxton et al. (2014) provide a chain of findings that indirectly link academic and intellectual development to first-year student persistence. Specifically, they report that student’s perception of their level of academic and intellectual development positively influences their level of subsequent commitment to their college or university of enrollment, which, in turn increases their likelihood of first-year persistence at their college or NEW DIRECTIONS FOR HIGHER EDUCATION • DOI: 10.1002/he


81

Table 6.1. Results of the regression analyses to test the influence of course examination questions on key concepts in the theory of student persistence in commuter colleges and universities DV: Institutional commitment to student welfare

DV: Institutional integrity

DV: Academic and intellectual development

0.084∗ (0.078) −0.068 (−0.083) −0.002 (0.000) −0.014 (−0.018) 0.050 (0.011)

0.158∗∗∗ (0.155) 0.047 (0.061) 0.014 (0.002) 0.027 (0.036) −0.006 (−0.001)

−0.037 (−0.040) 0.037 (0.054) −0.028 (0.005) −0.100∗∗ (−0.146) −0.034 (−0.009)

Exam questions are limited to knowledge of facts Most exams require me to compare and contrast Most exams require me to point out strengths/ weaknesses Few exams require me to argue points of view Institutional commitment to student welfare Institutional integrity

0.063 (0.037)

0.067 (0.041)

0.024 (0.016)

0.105∗∗ (0.067) 0.117∗∗ (0.069)

0.143∗∗∗ (0.096) 0.133∗∗∗ (0.084)

0.039 (0.029) 0.133∗∗∗ (0.093)

−0.031 (−0.018) —

−0.010 (−0.006) —

0.059 (0.040) 0.328∗∗∗ (0.387)

—

—

Institutional fixed effects Constant Adjusted R-squared N

Yes 2.962∗∗∗ 0.113 709

Yes 2.256∗∗∗ 0.116 709

0.207∗∗∗ (0.231) Yes 1.043∗∗∗ 0.288 709

Gender, female = 1 White Parental education Fulltime status Average grades in high school

Note: Unstandardized regression coefficients in parenthesis; ∗ p < 0.05, ∗∗ p < 0.01, ∗∗∗ p < 0.001.

university. By extension, this pattern of findings suggests the indirect role that course examination questions play in the first-year persistence of students in commuter colleges and universities. Residential Colleges and Universities. We learn from the results of the regression analyses exhibited in Table 6.2 that examination questions that require students to point out strengths and weaknesses of a particular argument or point of view also shape student perceptions of institutional integrity and the commitment of the institution to student welfare in residential colleges and universities. In contrast to commuter colleges and universities, examination questions that require students to compare or NEW DIRECTIONS FOR HIGHER EDUCATION • DOI: 10.1002/he

82


Table 6.2. Results of the regression analyses to test the influence of course examination questions on key concepts in the theory of student persistence in residential colleges and universities

Gender, female = 1 White Parental education Average grades in high school Initial institutional commitment Exam questions are limited to knowledge of facts (reverse scored) Most exams require me to compare and contrast Most exams require me to point out strengths/ weaknesses Few exams require me to argue points of view Institutional commitment to student welfare Institutional integrity Psychosocial engagementa Institutional fixed effects Constant Adjusted R-squared N

DV: Institutional commitment to student welfare

DV: Institutional integrity

DV: Social integration

0.094 (0.090) 0.047 (0.057) 0.166∗∗ (0.023) 0.032 (0.009) 0.029 (0.017) 0.050 (0.031)

0.070 (0.074) 0.035 (0.047) −0.012 (−0.002) 0.014 (0.004) 0.039 (0.025) 0.114∗ (0.079)

0.030 (0.029) 0.104∗ (0.125) 0.028 (0.004) −0.014 (−0.004) 0.066 (0.038) 0.026 (0.016)

0.039 (0.025) 0.132∗ (0.079)

−0.015 (−0.011) 0.191∗∗∗ (0.127)

0.024 (0.015) −0.067 (−0.040)

0.026 (0.016) —

0.003 (0.002) —

—

—

— Yes 2.324∗∗∗ 0.076 404

— Yes 2.321∗∗∗ 0.037 404

0.054 (0.033) 0.287∗∗∗ (0.287) 0.232∗∗∗ (0.208) 0.306∗∗∗ (0.307) Yes 0.388 0.406 404

Note: Unstandardized regression coefficients in parenthesis; ∗ p < 0.05, ∗∗ p < 0.01, ∗∗∗ p < 0.001. a Psychosocial engagement included in this regression to control for its influence of social integration as it positively influences social integration (Braxton et al., 2014).

contrast dimensions of course content exert little or no influence on either of these core concepts in residential colleges and universities. Nevertheless, the less frequently students encounter examination questions limited to a knowledge of facts, the more positive the perceptions of the institutional integrity of their college or university. The results of the regression analyses shown in Table 6.2 provide an indication of the indirect role course examination questions play in the persistence decisions of first-year students enrolled in religiously affiliated NEW DIRECTIONS FOR HIGHER EDUCATION • DOI: 10.1002/he


83

residential colleges and universities. More specifically, Model 3 of Table 6.2 shows that the more a student perceives that the institution is committed to the welfare of its students, the greater the student’s level of social integration. Likewise, the more a student perceives that the institution exhibits institutional integrity, the greater the student’s level of social integration. By extension, Braxton et al. (2014) develop an indirect linkage of course examination questions to persistence through their finding that student perception of their level of social integration influences their level of subsequent commitment to their college or university of enrollment. They also found that the greater the subsequent level of commitment to their college or university espoused by a student, the greater their likelihood of persisting from the first to the second year of their enrollment at the focal residential college or university (Braxton et al., 2014).

Limitations We note limitations to both of the data sources used in this study. These limitations temper our conclusions derived from our findings as well as our recommendations for practice. The three limitations we identify are common to both the data source for commuter colleges and universities and the data source of religiously affiliated private residential colleges and universities. We obtained these limitations from Rethinking College Student Retention (Braxton et al., 2014): 1. The samples for both state-supported commuter college and universities and religiously affiliated private residential colleges and universities are nonprobability samples of convenience. Consequently, inferences to these two populations of colleges and universities are limited. Nevertheless, these two samples contain specific institutions that bear a resemblance to the institutions that comprise their respective populations. Hence, we do have confidence that these institutions share sufficient similarities with other institutions of their type so that we can expect the results of this current study to apply to these two settings. 2. As previously stated, somewhat low survey, response rates were attained for both samples of colleges and universities: 53.2% for the commuter colleges and universities sample and 28.4% for the sample of religiously affiliated private residential colleges and universities. Nevertheless, Braxton et al. (2014) assess the extent of bias in these samples to be minimal. 3. Course examination questions, commitment of the institution to student welfare, institutional integrity, social integration, and academic and intellectual development were measured using student self-reports or student perceptions of these concepts. Different findings might emerge if objective measures of these concepts are used. NEW DIRECTIONS FOR HIGHER EDUCATION • DOI: 10.1002/he

84


Conclusion and Recommendations for Institutional Practice Academic rigor in the form of higher order examination questions matters for factors important to college student persistence, such as the Commitment of the Institution to the Welfare of Its Students and Institutional Integrity. However, complex best depicts the relationship between these two key concepts, the type of higher order examination question and institutional type. Put differently, this relationship depends on institutional context. To elaborate, higher order examination questions such as those that require students to point out strengths and weaknesses of a particular argument or point of view play a direct part in positively influencing student perceptions of institutional integrity and the commitment of their college or university to the welfare of its students in both state-supported commuter colleges and universities and in private religiously affiliated residential colleges and universities. However, higher order questions requiring students to compare and contrast dimensions of course content positively influence student perceptions of the commitment of their institution to the welfare of its students and of institutional integrity in commuter colleges and universities but not in religiously affiliated residential colleges and universities. Although a lower order examination question, questions that infrequently require knowledge of facts positively influence student perceptions of institutional integrity in religiously affiliated residential colleges and universities but not in state-supported commuter colleges and universities. We derive this conclusion from the pattern of findings of this study. Our conclusion resonates with Tinto’s call for institution action for student retention centered on the setting of high expectations for students. Tinto (2012) delineates four research-based conditions for student retention. The setting of expectations stands as one of these four conditions. The setting of expectations for success in the classroom constitutes a key aspect of this particular condition. Tinto (2012) asserts that faculty play a critical role by espousing their high expectations for student academic performance through course syllabi, communication with the class, grading standards, and feedback on assignments and course examinations. We add faculty use of higher order examination questions to this list of ways for faculty to set high expectations for student academic success in their classroom.

Implications for Practice We offer the following recommendations for practice at both residential and commuter colleges and universities. 1. Centers for teaching or individuals responsible for faculty development should sponsor workshops or seminars focused on assisting faculty members in the development of skills in the writing of higher order examination questions used in this study. Questions that NEW DIRECTIONS FOR HIGHER EDUCATION • DOI: 10.1002/he


85

require students to point out strengths and weaknesses of a particular argument or point of view could be the focus of workshops or seminars in both private residential colleges and universities and in state-supported commuter colleges and universities. Workshops or seminars in commuter colleges and universities might also stress questions that require students to compare and contrast dimensions of course content. Faculty development workshops or seminars in private religiously affiliated residential colleges and universities might emphasize limiting examination questions that require only recall or recognition of facts. Such workshops or seminars should also extend the notion of academic rigor as the level of understanding of course content to other course processes such as the types of questions faculty ask students during class, the nature of term papers, and other written assignments. Accordingly, workshops or seminars could also focus on faculty acquisition of skills in asking higher order question of students in class as well as directions for term papers and other written assignments. 2. Student course rating instruments should include items that ask students how frequently faculty ask higher order questions on their course examinations. The four types of examination questions used in this study suggest the type of items to be included in such course rating instruments. Colleges and universities should collect such information from course rating instruments for individual faculty members and make such information available to academic advisors for their advising session with their first-year advisees both during the orientation period before the start of the academic year and at regular intervals during the academic year (Braxton, Hartley, & LykenSegosebe, 2015). The following recommendation discusses the use for such information. 3. Academic advisors such as faculty members, professional advisors, and peer advisors should encourage their first-year advisees to take courses from faculty who frequently ask higher order questions on their examinations. By taking such courses, first-year students would likely form positive assessments about the commitment of their institution to student welfare and its institutional integrity. Moreover, academic advisors could discuss with their advisees the educational benefits of taking courses from faculty who frequently ask higher order questions on their examinations, as taking such courses may lead to the sharpening of their critical thinking skills.

Closing Thoughts Academic rigor manifested in course examination questions matters to factors important to the persistence of first-year college students in two distinctly different types of colleges and universities: state-supported NEW DIRECTIONS FOR HIGHER EDUCATION • DOI: 10.1002/he

86


commuter colleges and universities and private religiously affiliated residential colleges and universities. Does academic rigor matter for student persistence in other types of colleges and universities such as research universities and two-year community colleges? We urge scholars to provide an empirical treatment of this question. We also encourage scholars to firm-up the linkages between course examination questions and student persistence using Path Analysis or Structural Equation Modeling statistical techniques that test for the statistical significance of possible indirect effects. In the interim, colleges and universities can assure some degree of academic rigor by implementing one or more of the above recommendations. Academic rigor in the form of course examinations, however, ultimately depends on individual faculty members to use in the construction of their examinations the types of higher order examination questions included in this study. Institutional implementation of our recommendations would work to buttress and reward such faculty efforts. References American College Testing Program. (2016). National collegiate retention and persistence to degree rates. Iowa, IA: Author. Arum, R., & Roksa, J. (2011). Academically adrift: Limited learning on college campuses. Chicago, IL: University of Chicago Press. Biggs, J. (1979). Individual differences in study processes and the quality of learning outcomes. Higher Education, 8(4), 381–394. Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy of educational objectives. Handbook I: Cognitive domain. London, England: Longman. Braxton, J. M. (2008). Toward a theory of faculty professional choices in teaching that foster college student success. In J. C. Smart (Ed.), Higher education: Handbook of theory and research (Vol. 23, pp. 181–207). Dordrecht, the Netherlands: Springer. Braxton, J., Doyle, W., Hartley, H., Hirschy, A., Jones, W., & McLendon, M. (2014). Rethinking college student retention. San Francisco, CA: Jossey-Bass. Braxton, J., Hartley, H., & Lyken-Segosebe, D. (2015). Students at risk in residential and commuter colleges and universities. In D. Hossler & B. Bontrager (Eds.), Handbook of strategic enrollment management (pp. 289–310). San Francisco, CA: Jossey-Bass. Fransson, A. (1977). On qualitative differences in learning: IV—Effects of intrinsic motivation and extrinsic test anxiety on process and outcome. British Journal of Educational Psychology, 47(3), 244–257. Furedy, C., & Furedy, J. (1985). Critical thinking: Toward research and dialogue. In J. Donald & A. Sullivan (Eds.), New direction for teaching and learning (Using research to improve teaching) (Vol. 23, pp. 51–69). San Francisco, CA: Jossey-Bass. Gibbs, G., & Simpson, C. (2005). Conditions under which assessment supports students’ learning. Learning and Teaching in Higher Education, 1, 3–31. Maier, N. R. (1933). An aspect of human reasoning. British Journal of Psychology. General Section, 24(2), 144–155. Newble, D. I., & Jaeger, K. (1983). The effect of assessments and examinations on the learning of medical students. Medical Education, 17(3), 165–171. Nordvall, R. C., & Braxton, J. M. (1996). An alternative definition of quality of undergraduate college education: Toward usable knowledge for improvement. Journal of Higher Education, 67, 483–497. NEW DIRECTIONS FOR HIGHER EDUCATION • DOI: 10.1002/he


87

Ramsden, P. (1988). Context and strategy. In R. R. Schmeck (Ed.) Learning strategies and learning styles (pp. 159–184). Boston, MA: Springer. Renaud, R. D., & Murray, H. G. (2007). The validity of higher-order questions as a process indicator of educational quality. Research in Higher Education, 48, 319–351. Scouller, K. (1998). The influence of assessment method on students’ learning approaches: Multiple choice question examination versus assignment essay. Higher Education, 35(4), 453–472. Tang, K. C. C. (1992). Perceptions of task demand, strategy attributions and student learning. Research and Development in Higher Education, 15, 474–481. Thomas, P. R., & Bain, J. D. (1982). Consistency in learning strategies. Higher Education, 11(3), 249–259. Tinto, V. (1975). Dropout from higher education: A theoretical synthesis of recent research. Review of Educational Research, 45(1), 89–125. Tinto, V. (2012). Completing college: Rethinking institutional action. Chicago, IL: University of Chicago Press. Watkins, D. (1982). Factors influencing the study methods of Australian tertiary students. Higher Education, 11(4), 369–380.

JOHN M. BRAXTON is a professor of leadership, policy, and organizations emeritus of the Higher Education Leadership and Policy Program of Peabody College of Vanderbilt University. CLAY H. FRANCIS is a faculty member at Hutchison School, a PK–12 independent girls’ school in Memphis, TN, and the University of Memphis. He earned an MS degree in Educational Leadership from Vanderbilt University where he studied higher education policy. NEW DIRECTIONS FOR HIGHER EDUCATION • DOI: 10.1002/he