Multiple Choice Questions (MCQ) me often the assessment technique of last resort. They are the refuge of academics who are over-whelmed by student n-tubers ...
On Blooming First Year Programming, and its Blooming Assessment Raymond Lister School of Computing and Information Technology, University of Western Sydney, PO Box 10, Kingswood NSW 2747 r.liste~,uws.edu.au
Abstract The wad.itlonal approach to introductory programming has students wri~n~ complete progrRm% as early as possible. Also, the traditional emphasis is on the technology, not the explicit cognitive development of the student. This approach jmT,ps to the fifth and sixth levels of Bloom's Taxonomy of Educational Objectives, when these last two levels depend upon competence in the first four levels. I descn'be my alternative approach that addresses the six stages in sequence, emphasising effective assessment processes for the first four levels.
after two introductory chapters, the sequence of chapter headings of a popular Java textbook [5] are "Methods, Statements and Expressions", "Program Design with Methods and Graphics", "Selection Stractores ~, "Repetition Structures", "Arrays and Vectors", etc. The traditional approach has worked well for the gifted studemt, but mediocre students gain few genuine program writing ~h'll~ ... luld yet we pass them. This paper descries alternative approaches to teaching ~,a asses~in~ first year programming. While undergraduates must write large complete programs at some point in their education, I argue that they need not do so from the very start. The sequencing of material should be less clearly tied to the technology. Instead, the sequencing should be based explicitly upon a cognitive model of how students learn. The model I use is a well-known taxonomy of learning stages, descn'bed in the next section. This model leads to an early emphasis on "mastery" rather than "development", "The instrucffonal emphasis with mastery objectives is to bring all pupils to a uniform level of performance on the minimum essentials ... whereas the emphasis with development objectives is to assist pupils to achieve their maximum development" [4]. In first year programming. mastery should be sufficient for a pass or credit grade, and satisfy the prerequisites for the following subjects. More gifted students should continue to be challellged and assessed in the traditional way, and be rewarded with the
1 Introduction In many Australian universities, the failure rates for first year programming subjecls are among the worst across the university. Many students become disencha-ted, and either drop out or resort to plagiarism In the coffee room, the first year progra~r~ning subjects are the staple of office gossip, with staff who teach "down stream" complaininE that students lack elementary programml.e ~k'ill~. Under the c;.camstances, we should be prepared to question our most
cherished as.~-,~tions. Traditiomdly, the teaching of introductory programming has emphssised the writing of complete progrAm~ fi'om the earliest posE'hie stage. The week-by-week breakdown of topics is explicitly organised accorai-~ to the technology, not the cognitive development oftbe student. For example,
higber grades. 1.1 Bloom's Taxonomy of Educational Objectives Bloom's taxonomy [2] is venerated by educationalists, but has received only lira/ted recognition in computer science education [3]. The taxonomy contains six levels. Each successive level depends upon belmvionrs acquired at the earlier levels. When descn'bing each of these levels, it is ~ . , - - o n practise to include a list of words Rlustrafing the behaviours that a student is expected to exhibit The following description is generic, not specific to computer progr~-...~;n~. The levels, from lowest to highest are: Knowledge: the recall of specifics and universals, the recall of methods, and processes, or the recall of a pattern, structure or setting. A word list of activities is ... arrange,
Pem~mion to n~ke digital or hard ¢opim of all or part of this work for l a a m m l or clasmmm m e b granted withoat ~ e laOVided that ¢opim are m t nmie or dim.~mted fro-profit or ¢onmm~ial advammge. ~0d that copies b m r this notice m~dthe fall cimiou on the firn Imge. To copy otberwh;¢, m republish, m post on ~ or m redigW~u~¢m
rcq~espriorspecie ~r~ufiooand/or• fee.
ACE 2000 12/00 M e [ a ~ e , Atmralia O 2000 ACM l-$8113-271-9/00/0012 ... $5.00
158
define, duplicate, label, list, memorise, name, order, recognise, relate, recall, repeat, reproduce, state.
frazments of that language. Why should it not be the ~ m e when learning to program?
Comprehension: a type of understanding when the individual knows what is being commanicated and can make use of the materials or idea being co ....... mlcated without necessarily relating it to other material or seeing its fullest implications ... Classify, describe, discuss, explain,
2
AssessingKnowledge and C o m p m h e n s l o n
ideas, and theories that must be remembered and applied ...
Multiple Choice Questions (MCQ) me often the assessment technique of last resort. They are the refuge of academics who are over-whelmed by student n-tubers, and also those academics who are merely lazy or incompetent. To admit to using MCQs is to invite disapproval fi'om one's peers, without those peers feeling any need to actually look at the questions themselves. Most nndtiple choice exnm~ are indeed poorly designed. Consider the following example of a poor MCQ, taken fi'om a formative assessment task at an Aus~lJAn university:
Calculate, choose, demonstrate dramatise, employ, illustrate, interpret, operate, practice, prepare, schedule, sketch, solve, use, write.
When passing an array to a function, why is it necessary to pass the length of the array as well?
express, identi.~, indicate, locate, recognise, report, restate, review, select, summarise, translate. Application: is the use of abstractions in particular and concrete situations and may include general ideas, rules of
procedures, zenera[ised methods, tedmic~ prtncip]~
Analysis: en~phasises the breakdown of the material into its constituent parts and the detection of the relationships of the parts and of the way they are orgAni~d ... Appraise,
calculate, categorise, compare, contrast, differentiate, discriminate, distinguish, experiment question, test.
criticise, examine,
Synthesis: the putting together of elements and parts so as to form a whole ... Arrange, assemble, collect, compose, construct, create, design, develop, formulate, manage, organise, plan, prepare, propose, set up, write. Evaluation: the mA~.~ of judgments about the value, for some purpose, of ideas, works, solutions, methods and materials. It involves the use of criteria as well as standards for appraising the extent of which particulars are ~curate, effective, economical, or satisfying ... appraise, argue,
assess, attach, choose, compare, conclude, defend, estimate, evaluate, judge, predict, rate, score, select support, value. I take a p r a ~ t i c view of the taxonomy, ralher the. a strict ideological view. The taxonomy is a conceptual tool. It is a framework for lesson design, assessment design, and discourse with peers. There is little to be gained by attempting to pigeon hole every specific aspect of progwmmlng into a specific level. I merely make. the following broad observations, specific to the teaching of elementary progr~rmning: the lower two levels emphasise the skill of reading and comprehending code, the intermediate two levels emphasise the writing of small fragments of code, but within a well defined context, and the upper two levels emphasise the writing of complete non-trivial progrRm~ ... students should first be taught to read programs before they write programs. As chikh'en, in our early school years, the ability to read and tmdersmnd our native language outstripped our capacity to write
159
1. Because the function cannot deduce the size of the array. 2. Because this tutorial suggests this is done. 3. Because most progr~ ...... ers do it that way. 4. All of the above. This is poor because it does not require any knowledge of programm/~s, simply the application of irmAte omn/ng. I presented this question to an audience of six academics, from a non-computing discipline. None of the six had any programming experience. Despite that, all six academics were able to guess the correct answer. Options 2 and 3 are trite "game phying" [I].Eliminm£ng those two options el/m/-~tes the fourth option. The following MCQ has mere plaus~le distractors,~.a is more appropriate for assessing a student's knowledge, midway through a first semester programming subject: What output is produced by the following Java code: for (int iffiO; i ~ t ; i - ) if( y[i] < y[bestSoFar] ) bestSoFar = i; for (int i~fitst+l; ifirst; i - ) if( y[i]< bestSoFar )
It is t ~ to ~Heve t ~ t a s u e MCQ must be intzinsically easier than asking sRlden~ to wri~ the complete code. However, it is the nmrking scheme that mtly defines any exam Vague queslions, such as "Write a function to calculate N factorial", invite equally vague "stream of consciousness" answers, which all too often attract a "charity mark" for an appropr/ate reserved word. These charity marks, along with solid performance on some
bestSoFar = i; (d)
for (int i=last; i>first; i - ) if( y[i] > y[bestSoFar] ) bestSoFar ffii;
(e)
for (int i-fu~t+l; i
