Do Clickers 'Click' in the Finance Classroom? - CiteSeerX

Do Clickers ‘Click’ in the Finance Classroom?

Kam C. Chan Jean C. Snavely

Department of Finance Western Kentucky University Bowling Green, KY 42101

September 2008

Do Clickers ‘Click’ in the Finance Classroom? 1. Introduction Adoption of innovative classroom technologies should be considered when there are demonstrable net benefits from adoption. Benefits may be quantitative or qualitative. The purpose of this research is to examine whether the use of Student Response Systems (Clickers) results in improved student performance in principles of finance classes.1 While we specifically examine quantitative measures of performance, we consider some qualitative aspects from clicker usage as well. Previous research in financial education shows the beneficial application of other innovations. Among these are the use of finance software, classroom games, collaborative learning, entertaining metaphors, and movies (among others) in finance classrooms (see for example, Dyl (1991), Chan, Shum, and Lai (1996), Marks (1998), Philpot and Peterson (1998), and Ardalan (1998)). A common theme of these studies is to move a traditional static lecture learning style finance classroom to a more interactive dynamic teaching-learning environment. The literature generally suggests that students learn better in terms of their perceived classroom experience, and in many cases they perform better in terms of mastering the finance concepts as reflected in course grades. Clickers are electronic keypads that allow individuals to spontaneously reveal answers to objective question sets in the classroom. The keypads transmit student responses via radio frequency to a computer-interfaced receiver. The system software then processes, records,

1

Engaging students in active learning is one goal when using clickers. Other beneficial goals include: promote student discussion, provide feedback to the instructor, and use as a formative assessment to guide teaching, and reaffirm learning. For an extended list see ―Clicker Resource Guide: An Instructor’s Guide to Effective Use of Personal Response Systems (Clickers) in Teaching‖ prepared by the staff of the University of Colorado Science Education Initiative and the University of British Columbia Carl Weiman Science Education Initiative (2008).

reports, and instantly displays a histogram showing the percentage responses for each question. Clicker software allows instructors to save results for later review of student responses by question. According to the emerging clicker evaluation literature and clicker vendors, clickers, in principle, allow students to participate in the classroom quickly and effectively. In addition, clickers also enable instructors to get timely feedback regarding particular chapters or concepts. Theoretically, instructors can then adjust classroom presentations and subsequently improve student performance in the classroom. The primary objective of this paper is to extend the clicker research by examining usage in the principles of finance classroom. Most of the relevant literature concentrates on large sections of lecture classes concentrated in the applied and social sciences. There is little research into usage in business classes, and we find no examination of clicker usage in finance classes. Research in the business curriculum is limited to the use of clickers in the areas of accounting and management information systems. This research differs and adds to the literature by examining the result of using clickers in smallto medium-sized (maximum enrollment of 45) principles of finance classes. These classes are junior level with prerequisites for admission.2 Given the nature of the differences in science and business education and the difference in student experience, we do not have a priori knowledge whether the performance of business students in finance classes would be the same as science students. Thus, it would be informative to provide an empirical study of the effectiveness of using clickers in the finance classroom. Second, we include a number of confounding factors in our analysis and thus are able to isolate the effect of using clickers in the finance classroom. Our findings should be useful for instructors and administrators in making decisions regarding clicker technology adoption. 2

Students must have successfully completed a college algebra course and the first financial accounting course.

1

Looking at quantitative measures, our results suggest that, after controlling for other confounding factors, there is no statistically significant difference in student performance between clicker users and non-users. Our conclusion is robust to different research methods. There may be qualitative benefits from clicker adoption since students report that they enjoy their classroom experience in a class using the clicker technology.

2. Literature review There is a large volume of literature on using clickers in the context of science education. See Fies and Marshall (2006) for a comprehensive review of using clickers in science education. We highlight only a few important, recent studies here. The majority of studies examine the use of clickers in large, lecture classes comprised primarily of lower classmen requiring no prerequisites for admission. Hake (1998) offers an early study in using clickers in the classroom. For a large sample of physics students, Hake finds that, on average, clicker students perform 25% better than students who do not use clickers. Similarly, in a study of biology students, Knight and Wood (2005) suggest that significantly more students in the clicker class score B grades or higher than the non-clicker users.

Preszler et al (2007) and Freeman et al (2007)

report that an increased use of clickers during lectures is positively correlated with student performance on examination questions across various biology courses. The literature in business is very limited and less conclusive. Contrary to the results from the science literature, Carnaghan and Webb (2007) do not find evidence to suggest using clickers enhance student performance in introductory managerial accounting classes.3 Trenholm and

3

Course grades are used to measure student performance.

2

Dunnett (2007) replicate the Carnaghan and Webb study with a group of financial accounting students and find that using clickers actually hurt student performance. In summary, clickers appear to enhance student performance in large, lecture-based, science classes. The literature for clicker usage in introductory business classes is more limited, and the results do not support the conclusion that clicker usage in business classes improves student performance. A major weakness of the literature is that comparisons are primarily of clicker user group performance (in terms of course grade or examination performance) vis-à-vis the non-clicker group without controlling for other performance contributing factors. That is, the literature seldom uses multivariate analysis.

Prior financial education research, suggests there are

numerous factors contributing to student performance in the finance classroom. Some of the contributing factors include student attendance, mathematics preparation, accounting preparation, and maturity of students in terms of credit hours completed.

See Chan and Li

(2003) for a summary of these performance-enhancing factors in the finance classroom. To simply compare the performance of a control class (no clickers) to a treatment class (using clickers) without considering other confounding factors does not provide a clear assessment of the effectiveness of clicker usage in the classroom. Our study helps fill this void.

3. Methods The research subjects are students in two principles of finance courses in Fall 2007 in a medium size comprehensive university in the South. One course uses clickers and the other course uses the traditional lecture style. Both courses have the same instructor who had more than ten years of experience in teaching principles of finance. The instructor uses the same text, syllabi,

3

material, and examination questions in both courses. There are a total of 48 clicker questions for the clicker class evenly spread over the semester. Both courses are on Tuesday and Thursday; hence, there is no compressed learning issue as suggested in Chan and Li (2003). In addition, students do not know which course will use clickers before registration; therefore, there is no selectivity bias among students. There are 34 and 36 students in the clicker and the control classes, respectively. During the semester, three students dropped from each class (a total of six students). We also exclude one exchange student whose background and motivation differs from the rest of the class. The final sample has 63 students in both classes, N=30 for the control group and N=33 (after student withdrawals). Student demographic, academic, and leisure information is collected through the university registrar’s office or from students’ self-reported questionnaire. We also record student attendance and examination scores for both classes for the analysis. We use the following multiple regression model to examine the effectiveness of clickers in the classroom: Course average = β0 + β1*Clicker_avg (or Clicker) + β2*Sex + β3*Major + β4*Accounting + β5*Math + β6*Attendance + β7*Age + β8*Enrolled + β9*GPA + β10*Work + β11*Completed + ε (1) Where Course average = the average score on three examinations; Clicker_avg = a student’s percentage score in the clicker exercise; for no clicker class, value=0) Clicker = a dummy variable; for clicker-user, value=1; else=0; Sex = a dummy variable for gender; for male, value=1; else=0; Major = a dummy variable for student major; if a student major is finance or accounting then value=1; else=0; Accounting = a dummy variable for accounting course performance; if a student

4

scored A or B in accounting, then value=1; else=0;4 Math = a dummy variable for mathematical course performance; if a student scored A or B in the mathematics course, then value=1; else=0;5 Attendance = the percentage of student attendance in %; Age = the age of a student; Enrolled = a student’s total credit hours enrolled for the semester; GPA = a student’s GPA; Work = a student’s weekly work hours; Completed = a student’s completed credit hours. ε = a random error term

We estimate Equation (1) by ordinary least squares using the information for the 63 students. Because Chan, Shum, and Lai (1996) show the potential impact of survival bias on results due to student withdrawal, we follow Chan et al (1996) in using TOBIT estimation to mitigate the potential survival bias. In the TOBIT analysis, we are able to include the six students who withdrew from the course in the analysis. We also conduct a survey at the end of the semester in the clicker class to gauge the students’ perceptions regarding their clicker experience. The survey uses a five-point scale from 1 = strongly agree to 5 = strongly disagree.

4. Results and discussion The summary statistics are presented in Table 1, Panels A and B. We partition the summary statistics into clicker vis-à-vis non-clicker classes for comparison. 4 5

For Panel A, students in both

Performance is measured for the prerequisite accounting class. Performance is measured for the prerequisite mathematics class.

5

classes share similar profiles. On average, a student has a 2.9 GPA, works about 17 hours per week, is enrolled in about 14 credit hours per semester, and averaged around 74% on test grades. The two sample t-tests do not show significant differences between the two groups of students. Similarly, in terms of the dummy variables (proportion of male students, accounting or finance major, good accounting background, and good mathematics background); there are no statistical differences between the students in both classes. We present two multivariate models. Model 1 uses the students’ percentage scores on clicker questions to examine the effectiveness of clickers while Model 2 uses a conventional (1, 0) dummy coding for using clickers. We estimate both Models using an ordinary least squares method. The variance inflation factors in all variables have values less than four, which suggest that multicollinearity is not a serious problem. In both Models, the results are consistent in that percentage scores on clicker questions and clicker dummy variables do not show statistically significant coefficients. The results suggest that while clickers do not hurt student performance, they do not enhance student performance either. Consistent with other financial education research literature, we find that students’ GPA, accounting course performance, and accounting or finance major variables are statistically significant in explaining student performance in the principles of finance class. To obtain robust results, we estimate Models 1 and 2 again with TOBIT analysis. TOBIT analysis enables us to mitigate a potential survival bias. Six students withdrew during the semester. We have their information (explanatory variables) but not an overall course average. In TOBIT analysis, we use the known information to estimate the students’ scores had they completed the course. We then use the withdrawn students’ predicted scores to represent their scores in the analysis. Therefore, we are able to use all 69 observations. The results are in Table

6

3.

The clicker question average scores and clicker dummy variables are not statistically

significant. The findings are similar to Table 2.

Therefore, our conclusion regarding the

effectiveness of using clickers in the principles of finance are robust to different estimation methods. We conduct a survey at the end of the semester in the clicker class. The objective of the survey is to obtain some qualitative elements of clickers that may not be captured in a quantitative analysis. We apply a simple seven statement questionnaire with a five-point scale (1 is strongly agree, 2 is agree, 3 is neutral, 4 is disagree, and 5 is strongly disagree). In the questionnaire, we deliberately put Statements 1 and 6 in negative tones so that a positive response to the use of clickers means ―disagree‖ or ―strongly disagree‖ to the two statements. In order to observe the rules of institutional research at the university, survey participation is purely voluntary. All of the students who completed the course participated in the questionnaire; therefore, we have 30 usable responses.

The survey results are in Table 4.

Overall, students exhibit positive

perceptions of clicker usage. Statements 2, 3, 4, 5, and 7 all show modes in ―Agree‖ columns with averages less than 3.0. For the negative tone statements (1 and 6), the modes are in ―Disagree‖ columns.

5. Summary We study the effectiveness of using clickers in the principles of finance classroom. A careful experimental design of the study enables us to isolate the performance of clicker users vis-à-vis non-users. After controlling for other confounding factors, we do not find statistically significant differences between the two groups of students. Nonetheless, students, in general, respond positively to their clicker experience. Given the mixed results in literature (positive in science

7

and negative in accounting), and the neutral finding from the current study, the effectiveness of using clickers appears to be discipline specific. Further research is needed to explore a larger sample, increased use of clickers throughout the semester as well as a longitudinal study of clicker technology in the classroom.

8

References

Burnstein, R. and L. Lederman, 2001. Using wireless keypads in lecture classes, the Physics Teacher, 39, 8-11. Carnaghan, C. and A. Webb, 2007. Investigating the effects of group response systems on student satisfaction, learning, and engagement in accounting education, Issues in Accounting Education, 22, 391-409. Chan, Kam C. and Joanne Li, 2003. Decaf or regular? A study of the impact of time-compressed learning on student performance of the MBA Investments course, Journal of Financial Education, 29 (Spring), 12-25. Chan, Kam C., C. Shum, and P. Lai, 1996. An empirical study of cooperative instructional environment on student achievement in principles of finance, Journal of Financial Education 22, 21-28. Ardalan, Kavous, 1998. On the Use of Entertaining Metaphors in the Introductory Finance Course, Financial Practice & Education 8, 108-119. Dyl, Edward A., 1991. Wall Street: a case in ethics, Financial Practice and Education 1, 49-51. Hake, R., 1998. Interactive-engagement versus traditional methods: a six thousand-student survey of mechanics test data for introductory physics courses, American Journal of Physics, 66, 64-74. Knight, J. and W. Wood, 2005. Teaching more by lecturing less, Cell Biology Education, 4, 298-310. Trenholm, B. and J. Dunnett, 2007. When it all ―clicks‖—the effectiveness of using game show technology in the classroom, paper presented at the Financial Education Association, Bermuda. Fies, Carmen and Jim Marshall, 2006. Classroom response systems: a review of the literature, Journal of Science Education and Technology, 15, 1573-1839. Preszler, Ralph W. Angus Dawe, Charles B. Shuster, and Michèle Shuster, 2007. Assessment of the effects of student response systems on student learning and attitudes over a broad range of biology courses, CBE Life Science Education 6, 29-41. Freeman, Scott, Eileen O'Connor, John W. Parks, Matthew Cunningham, David Hurley, David Haak, Clarissa Dirks, and Mary Pat Wenderoth, 2007. Prescribed Active Learning Increases Performance in Introductory Biology, CBE Life Science Education 6, 132-139.

9

Marks, Barry R., 1998. An examination of the effectiveness of a computerized learning aid in the introductory graduate finance course, Financial Practice and Education, 127-132. Philpot, James and Craig Peterson, 1998. Improving the Investments or Capital Markets Course with Stock Market Specialist, Financial Practice and Education 8, 118-124.

10

Table 1. Summary statistics Panel A: continuous variables

Variables Clicker average score Attendance Age (in years) Student enrolled credit hours in the current semester GPA Student working hours per week Student completed credit hours Course overall exam score

Control class (N=33)

Clicker class (N=30)

t-statistics for equal means [(1) – (3)]

Mean (1) 0 84.6% 23.1 13.1

Std deviation 0 21.5% 4.4 3.1

Mean (3) 63.7% 90.9% 22.2 14.0

Std deviation 15.4% 17.9% 4.5 2.6

Not meaningful -1.33 0.76 -1.40

2.9 17.5

0.5 13.6

2.9 17.0

0.4 15.8

-0.03 0.14

94.6

26.2

86.3

21.4

1.44

73.7 (N=33)

10.5

74.1 (N=30)

10.0

-0.15

Panel B: dummy variables Variables

Control class (N=33) Gender (proportion of male) 44.4% Major (proportion of accounting 8.3% and finance majors) Accounting (proportion of 47.2% students scored A or B) Mathematics (proportion of 66.7% students scored A or B)

Clicker class (N=30) 60.1% 6.1%

Z-statistics for equal proportion [(1) – (3)] -1.32 0.35

39.4%

0.66

72.7%

-0.54

11

Table 2. A multivariate model of using clickers in the finance classroom

Variables Intercept Clicker_avg (the percentage score of clicker answer; for no clicker class, value=0) Clicker (for clicker-user, value=1; else=0) Sex (for male, value=1; else=0) Major (if major is finance or accounting then value=1; else=0) Accounting (if score A or B in accounting, then value=1; else=0) Math (if score A or B in math then value=1; else=0) Attendance (in %) Age Credit hours enrolled GPA Work hours per week Credit hours completed Adjusted R2 F statistics N

Model 1 Estimated coefficient 38.5693 3.9247

t-statistics 2.51** 1.23

Model 2 Estimated coefficient 36.8004

t-statistics 2.38**

1.4043

0.66

0.3461

0.15

0.5114

0.22

13.4069

3.55***

13.4086

3.52***

7.1723

2.50**

6.9901

2.40**

-1.7889

-0.77

-1.5963

-0.68

12.6492 0.0320 -0.4550 7.0224 -0.0614 0.0529 0.4307 5.26*** 63

1.45 0.12 -1.12 2.25** -0.85 1.13

14.3123 0.0337 -0.4105 7.1345 -0.0612 0.0493 0.4186 5.06 63

1.63 0.12 -1.00 2.26** -0.84 1.04

12

Table 3. A TOBIT model of clicker use in the finance classroom

Variables Intercept Clicker_avg (the percentage score of clicker answer; for no clicker class, value=0) Clicker (for clicker-user, value=1; else=0) Sex (for male, value=1; else=0) Major (if major is finance or accounting then value=1; else=0) Accounting (if score A or B in accounting, then value=1; else=0) Math (if score A or B in math then value=1; else=0) Attendance (in %) Age Credit hours enrolled GPA Work hours per week Credit hours completed Log likelihood value N

Model 1 Estimated coefficient -44.8752 -3.7957

χ2 statistics 4.21** 0.54

Model 2 Estimated coefficient -45.0844

χ2 statistics 4.40**

-5.0845

2.40

2.5655

0.50

2.7700

0.60

21.8014

12.47***

21.7038

12.69***

3.1881

0.47

2.4380

0.28

0.8542

0.05

0.9401

0.07

71.36*** 0.02 0.26 5.33** 0.01 0.07

86.8445 0.0787 -0.2526 11.7563 -0.0111 0.0128 -255.78 69

76.40*** 0.03 0.15 5.65** 0.01 0.03

86.6295 0.0561 -0.3346 11.5711 -0.0122 0.0205 -256.70 69

13

Table 4. A survey on the perceptions of using clickers by students (N=30)

Statements 1. The clicker caused me to underestimate the difficulty of the exams. 2. The clicker exercises increased my understanding of the course material. 3. The clicker increased my understanding of how I earned my final course grade. 4. I spent more time on this class because of the clicker. 5. I believe that I earned a better grade than expected because of the clicker. 6. I believe that my grade was worse than expected because of the clicker. 7. I believe that the clicker provided me more control over my learning than in classes that do not use a clicker.

Strongly agree (1) 2

Agree

Neutral

Disagree

Average

(4) 12

Strongly disagree (5) 1

(2) 8

(3) 7

2

21

3

3

1

2.33

1

12

9

6

2

2.87

1

10

8

8

3

3.07

2

15

6

3

4

2.73

3

4

7

14

2

3.27

0

15

8

6

1

2.77

3.07

14