Education Tech Research Dev (2012) 60:501–527 DOI 10.1007/s11423-012-9233-x DEVELOPMENT ARTICLE
Study to minimize learning progress differences in software learning class using PLITAZ system Jian-Jie Dong • Wu-Yuin Hwang
Published online: 7 February 2012 Association for Educational Communications and Technology 2012
Abstract This study developed a system using two-phased strategies called ‘‘Pause Lecture, Instant Tutor-Tutee Match, and Attention Zone’’ (PLITAZ). This system was used to help solve learning challenges and to minimize learning progress differences in a software learning class. During a teacher’s lecture time, students were encouraged to anonymously express their desire to pause the lecture, or to take a short break, in order to catch up with a teacher’s lecture. A simple proportion of one-third of the class was found to be a suitable pause-lecture threshold to prevent learning progress differences from becoming too great as well as to provide enough peer tutorial resources. During students’ practice time, an instant tutor-tutee match strategy extended tutorial resources, which took 60% workload from the teacher. Meanwhile, the attention zone (AZ) strategy helped the teacher to identify students with low levels of learning progress, as AZ students who needed more attention. It was found that AZ student numbers had a negative relation to overall learning achievement. Furthermore, 49% of the identified AZ students who received PLITAZ strategies experienced improved learning progress over identified nonAZ students. Overall learning progress differences were significantly minimized with the Instant Tutor-Tutee Match and Attention Zone strategies. Keywords Learning challenges Pause lecture intention Instant tutor-tutee match Attention zone Learning progress differences
J.-J. Dong Department of Computer Science & Information Engineering, National Central University, No. 300, Jhongda Rd., Jhongli City, Taoyuan County 32001, Taiwan, ROC e-mail:
[email protected] J.-J. Dong National Chia-Yi Girl’s Senior High School, NO. 243, Chuiyang Rd., Chiayi City 60043, Taiwan W.-Y. Hwang (&) Graduate School of Network Learning Technology, National Central University, No. 300, Jhongda Rd., Jhongli City, Taoyuan County 32001, Taiwan, ROC e-mail:
[email protected]
123
502
J.-J. Dong, W.-Y. Hwang
Introduction The computer classroom evolved from a traditional classroom setting into a new learning environment, consisting of individual workstations for students. Computer classrooms provide students with a ‘‘learning by doing’’ approach, which supplements ‘‘learning by seeing and hearing’’ (Harger 1996). In Taiwan, a high school class designed for learning a software package, like Microsoft Office, often consists of two phases: a teacher’s lecture phase and the students’ practice phase in a computer classroom (see Fig. 1). During the lecture phase, the teacher gives a lecture by presenting a sequence of steps on a screen, which is broadcast to all class screens. Within software learning classes, students need to memorize the teacher’s operating steps before engaging in practice exercises. If the teacher progresses too quickly in his or her lecture, the students can easily forget the step sequences. After a given lecture phase, students are allocated time to practice exercises on their own (practice phase). In this study, the researchers defined one learning session as one lecture phase. Commercial products such as SMART Technologies (2011), Codework System (2010) and Taiwan Good (2011) multimedia broadcast system are developed for a teacher to manage and facilitate student’ learning in a computer classroom. With those products, the teacher is able to broadcast teacher’s screen to whole class, to view student screens on teacher’s screen, to facilitate collaboration work by sharing screens, to lock specific application and website hyperlink in class; furthermore Taiwan Good also has a function called ‘‘Electronic Raise Hand’’, with which the students can answer by pressing Caps Lock key so as to increase interaction between teachers and students in class. In a software learning class, how much lecture time should a teacher allot before allowing students to practice? If the teacher provides too much lecture time during one lecture phase, some students will fail to catch up with the progress and they may possibly experience more problems during the practice phase. On the other hand, if the teacher provides too little lecture time during one lecture phase, more lecture phases will be necessary to make up for the lost lecture progress. This would also result in frequent screen-switching, resulting in practice times being interrupted, which may annoy the students. In a computer classroom, there must be some students who comprehend (understand) better of teacher’s lecture material, called ‘‘high-learning progress students’’ in this research; while there must be some students who understand (understand) fewer of teacher’s lecture material, called ‘‘low-learning progress students’’ in this research. According to McGrail’s (2007) research and various teachers’ experiences, the following challenges exist in the computer classroom, which lead to substantial differences in levels of learning progress among students, called ‘‘learning progress differences’’ in this research: limited communication between peers and the teacher; unknown learning conditions; off-task Fig. 1 Software learning in a computer classroom
Computer classroom learning flowchart in one class Teacher gives lecture with computer
123
Students practice with computers
Using PLITAZ system
503
Learning challenges-Social isolation Off-task behaviors Heavy workload on teacher Limited communication Learning condition unknown
PLITAZ strategies--
Learning progress differences enlarged
Learning progress differences reduced
Pause Lecture threshold Instant tutor-tutee match Attention zone treatments
Fig. 2 Effects of learning challenges and PLITAZ strategies on learning progress differences
behaviors such as instant messaging, gaming, etc.; heavy teacher workloads; and social isolation (Fig. 2). In Taiwan, the common number of students in a senior high school class is more than 40, sometimes 49. Therefore, *50 computer stations (including computers, monitors, desks, and chairs) are provided in a computer classroom to ensure that computer usage by every student is feasible. Also, due to classroom space limits, computer equipment is arranged in several rows. There are two common computer classroom arrangements in Taiwan. In the first arrangement all students sit facing the teacher. In the second setting, like in the Life Technology computer classroom in Chia-Yi Girls’ Senior High School, the computers are set in four rows: 12 seats are placed in lines 1, 3, and 4, respectively, while 14 seats are placed in line 2 (see Fig. 3). Students who sit in line 1 face line 2; line 3 faces line 4. Face-to-face communication is limited between the teacher and students because the students all watch the screens ahead of them in class. When students cannot understand the lecture, due to visibility being obstructed by computers, they cannot easily get the teacher’s attention to ask a question. This is especially true for the students who sit far away from the teacher. The teacher gives the lecture with the screen broadcast function to all students’ screens via a commercial product. Due to the limited face-to-face communication, the teacher is not aware of which students fall into the ‘‘low-learning progress’’ category; students in this category are given immediate support during the practice phase. Additionally, the teacher is not aware of
Fig. 3 Computer classroom arrangement used in this study
123
504
J.-J. Dong, W.-Y. Hwang
which students fall into the ‘‘high-learning progress’’ category; students in this category are instructed to support other students during the practice phase. Off-task behaviors such as surfing the Internet, playing flash games, among other nontask activities, not only distract a student’s attention from learning but they also impact other students’ learning during the practice phase. When the ‘‘fast-learning students’’ engage in off-task behaviors, their minds are not focused on the classroom when they could be helping the ‘‘low-learning progress students’’. Commercial products provide the teacher with a function to avoid off-task behavior such as ‘‘observing students’ screens’’. However, this feature is seldom implemented because students do not like to be observed and it may cause conflict between the teacher and students. The teacher’s workload is heavily weighted toward solving students’ problems. When the teacher offers help to one student with a problem, he or she is not available to help other students at the same time. The more time the teacher takes to solve an individual student’s problems, the less time the teacher has to give a lecture to the whole class. If the teacher does not help solve all the students’ problems and present a new lecture, learning progress differences will become great in the next progression. Social isolation exists because students often do not leave their seats out of respect for the teacher. This practice makes smooth communication among students difficult to accomplish. Furthermore, students who progress quickly do not know who needs help or what support to provide. From the other perspective, students who progress slowly do not know who can and will provide support. Finally, students may simply be afraid of being rejected when they ask for help from other students. In summary, to solve these challenges, the teacher needs information about learning conditions among students to control the progress of lectures, while students need alternative methods to express how their levels of understanding during the lecture phase. On the other hand, during the practice phase, the teacher needs a strategy to efficiently help students, and moreover, peer help among students needs to be strengthened in a computer classroom. Learning progress differences will be an indicator to help the teacher understand the learning conditions of the entire class. When the class’s learning progress differences became great, the teacher has a hard time moving the whole class on to a new lecture. This empirical research implemented a system that uses two-phased strategies called ‘‘Pause Lecture, Instant Tutor-Tutee Match, and Attention Zone’’ (PLITAZ). These strategies incorporate treatments to overcome computer classroom learning challenges as well as to find a feasible method to minimize learning progress differences for software learning in a computer classroom. Several questions needed to be solved in this study: 1. Is there a difference in solving learning challenges between lecture phase and practice phase with the use of PLITAZ? 2. Whether identified Attention Zone (AZ) students need more attention over non-AZ students? 3. Whether PLITAZ system help reduce the learning progress difference?
Literature review Objective of software learning Based on Bloom’s taxonomy (Bloom and Krathwohl 1956), learning objectives can be classified as: ‘‘knowledge, comprehension, application, analysis, synthesis, and
123
Using PLITAZ system
505
evaluation’’. Anderson and Krathwohl (2001) revised the taxonomy of cognitive domains to be: ‘‘remember, understand, apply, analyze, evaluate, and create’’, respectively. Falkner (2009) conducted a study with 147 computer science freshmen to investigate the factors required to successfully perform practical task within one class rather than a big project. This research found that the abilities to recall (remember) and comprehend (understand) previous knowledge are more important than the ability to apply material in practical task performance; application must be grounded in comprehension. Furthermore, a teacher often asks the students if they remember and understand lecture material during a lecture phase in order to ascertain the whole class’s learning status. The action of ‘‘inquiry’’ relates to the basic learning objectives: ‘‘remember and understand’’. Therefore, the first two fundamental learning objectives, remember and understand, are crucial for students to successfully learn computer software. Lecture progress and pause lecture Response systems have been recently introduced in a classroom; a teacher can poll students about their level of understanding and students can respond by pressing a numeric button on the system. However, studies found this type of system did not have a significant effect on learning efficiency and performance. Cain et al. (2009) conducted a study in physiological chemistry and molecular biology courses. In each 50-min lecture, 6–7 questions were prompted to students via an audience response system. 98% out of 109 first-year pharmacy students reported that questions via response system helped them maintain attention. Hoyt et al. (2010) integrated a response system in a human anatomy course over 2 years. The participant rate of using via smart phones, laptops ranged from 65 to 80% in the first lecture, but it decreased over the next four seasons. Moreover, the response system did not dramatically enhance overall student performance. Therefore, response systems are not enough to enhance learning performance in a classroom. Before giving a series of lectures, a teacher should call for the class’s attention to maximize learning efficiency (McKenzie 2008). However, according to the cognitive load theory, the lecture material should be designed to avoid overloading the learner’s working memory; when attention capacity is exceeded, memory capability is degraded (Chandler and Sweller 1992, 1996; Grimes 1990; Moreno and Mayer 2000, 2002). Stuart and Rutherford (1978) collected data from 1,353 questionnaires based on 12 lectures and found students’ attention during the lecture phase reach the maximum level at the 10–15 min mark and then fell steadily. The study suggested optimal time for a lecture is 30 rather than 60 min. With response systems, a teacher can only determine the whole class’s learning condition at a specific point in time; continuous class learning conditions cannot be ascertained. The teacher does not have an idea about whether or not the students’ attention capacity has been exceeded; therefore, lecture progress cannot be easily controlled and the difficulty of lecture material cannot be readily changed. In a software learning class, the teacher often gets the class’s attention by broadcasting his or her screen onto all computers during the lecture phase. The students must concentrate on each step the teacher operates on the screen; otherwise, students may not understand the teacher’s progress and they will have trouble during the subsequent practice phase. With less face-to-face communication between the teacher and students in a computer classroom, a teacher can easily miscalculate the students’ overall learning condition and proceed too quickly. This may result in some students being cognitively overloaded, leading to an increase in learning progress differences among students.
123
506
J.-J. Dong, W.-Y. Hwang
Listens to lecture
Cannot remember more steps or cannot understand lecture
Do nothing
Raises hand to interrupt lecture and asks teacher a question Presses 'Pause' key to express intention to pause lecture
Fig. 4 Pause lecture strategy in software learning based on revised Bloom’s Taxonomy
Ruhl et al. (2010) found a short pause (2-min breaks) during lecture time had significant effects on students’ attention and performance. Therefore, to avoid cognitive overload and to minimize learning progress differences, the teacher needs to understand the class’s overall learning condition in the computer classroom. Short breaks contribute to students’ attention and performance and they should be incorporated in computer learning environments. In this study, the researcher proposed the ‘‘Pause’’ strategy in a software learning class based on the revised learning objectives of Bloom’s taxonomy. When the students could not remember or could not understand the lecture (that is, they could not achieve the first two fundamental learning objectives of cognitive taxonomy), they could express their individual intention to pause the lecture at any time to minimize learning progress differences with their peers. Unlike raising one’s hand, this anonymous approach does not interrupt the lecture (the alternative option is the thick route and colored block in Fig. 4). With an understanding of the overall class’s learning condition, the teacher could determine a better time to pause the lecture and adjust the difficulty level of the material. Instant tutor allocation and potential scaffolding Vygotsky’s Zone of Proximal Development (ZPD) theory is the distance between a learner’s actual development and his or her potential development assisted by others’ scaffolding beyond the teacher’s help. With peer scaffolding, the student does not just concentrate on work to learn quickly; he or she also obtains an active learning attitude (Kim and Baylor 2006; Lai and Law 2006). Moreover, tutors benefit from having the opportunity to present what they have learned (their learning progress), while the ‘‘tutees’’ (those receiving the tutoring) benefit from seeing the problem from different perspectives. Tutees can also reflect on, monitor, and evaluate their learning (Galbraith and Winterbottom 2011; Nussbaum et al. 2009; Xun and Land 2004). Many computer-supported collaborative learning studies provide a medium of communication between peers, such as e-mail, chat, video conferencing, etc. (Stahl et al. 2006; Lin et al. 2010; Huang and Wu 2011). Furthermore, there are also several tutor allocation systems for distance learning such as the intelligent tutoring system, the reciprocal tutoring system, and teacher-assigned tutoring (Anderson et al. 1985; Wong et al. 2003; Chien 2008). Westera (2007) proposed a self-allocating peer tutoring mechanism, which estimated the expertise, past performance of a tutor candidate, and also included the workload of a tutor candidate to successfully allocate an appropriate tutor. Rosmalen et al. (2008) also proposed a model for selecting suitable tutors for online learning, based on the learners’ knowledge competency level and workload.
123
Using PLITAZ system
507
Attention zone strategy
In AZ Practices
Practice finished
Has a problem
Searches for help from immediate neighbor(s) or raises hand to call for teacher's help
Be a tutor voluntarily via system and welcomes call for help from others
Searches a tutor or calls for teacher's help via system
Waits for others or has off-task behaviors Fig. 5 Instant tutor-tutee match and AZ strategies during the practice phase
When a student encountered a practical problem in a computer classroom, the most accessible potential tutors are the student’s immediate neighbor(s) (other than the teacher). Although students in a computer classroom can interact face-to-face, they cannot easily reach students other than their immediate neighbors due to the seating arrangement. Moreover, one’s immediate neighbors are not always qualified to provide help; it was necessary to encourage students to search for help from classmates other than their immediate neighbors. According to one teacher’s experience in a computer classroom, some students who have finished practice exercises often feel bored waiting for others or they participate in off-task behaviors. On the other hand, some students who have finished practice exercises want to help others, but they do not know who needs help. At the same time, students who need help often do not know who can help or who wants to provide assistance. To provide instant potential scaffolding based on Vygotsky’s ZPD for low-learning progress students, the researchers of this study proposed an ‘‘instant tutor-tutee match’’ strategy to help the low-learning progress students find suitable tutors via the proposed system. This strategy is depicted as the thick route and colored block in the bottom part of Fig. 5. After a match was made, the tutor moved forward, to the tutee, and provided help. It was expected that the learning progress differences among students would be minimized via this strategy and the off-task behaviors would simultaneously be reduced. Time waiting for help and the effect on students’ AZ Hall et al. (1968) found that students experienced more productive learning after receiving the teacher’s attention. A computer-supported collaborative learning study conducted by Alavi et al. (2009) showed that learning productivity was only 38% when waiting for the teacher’s assistance; the majority of the waiting time was wasted. Nevertheless, Alavi et al. also had the following observations:
123
508
J.-J. Dong, W.-Y. Hwang
1. The teacher did not attend to all students with raised hands; only to those the teacher noticed. 2. The teacher’s order of providing help did not follow the sequence of the students’ requests. 3. The students devoted a great deal of attention observing if the teacher was available to help. The research implemented Lantern tools, which allowed the teacher to see the students’ working status (colors) and the urgency (blink rate) of searching for help. These tools overcame the above observations and increased learning productivity to 94% while waiting for help. The Lantern research pointed out that the teacher’s response time to a call for help was in contrast to the student’s learning efficiency. In other words, instant help from a teacher will enhance learning progress. Another problem exists during the practice phase of software learning: the teacher usually has no requests for help at the beginning of the phase, but the teacher is overwhelmed by many simultaneous requests at a later stage. Even helping a student with a simple practice problem can take much of the teacher’s time, sometimes more than 10 min. During the practice phase, if five to six students run into practical problems at the same time, the teacher will be overwhelmed and will not be able to efficiently help solve all problems. On the other hand, if the teacher could identify in advance which students fall into the low-learning progress category and which students might have trouble during the following practice phase, proper support can be provided. Consequently, the teacher’s workload helping solve problems can be dispersed to avoid being overwhelmed during the practice phase. To minimize learning progress differences as well as to avoid overwhelming the teacher with a heavy workload, the researchers of this study proposed an AZ strategy to help the teacher identify the students who progressed slowly and had little potential support from immediate neighbor(s). In other words, the strategy based on revised Bloom’s taxonomy and Vygotsky’s ZPD. Several treatments in this study were designed to help the AZ students quickly improve their learning progress. After the AZ students caught up with the teacher’s progress, they could be another potential tutorial resource in classroom.
PLITAZ system To help minimize learning progress differences, a proposed system (PLITAZ) with twophased strategies was developed using Java in a Client–Server architecture. PLITAZ includes several functions (see Table 1), which are not part of current commercial computer classroom management products. The interfaces and main functions, geared toward the students and teacher, are depicted in Figs. 6 and 7, respectively.
Research design Participants This study involved 126 sixteen-year-old female senior high school students in three classes (A, B, and C) in Taiwan. Of the participants, 42 students were in each class, respectively. The high school is well known for its academic achievement, and it is
123
Using PLITAZ system
509
Table 1 PLITAZ functions Lecture phase Teacher Functions
Switches the learning phases (between lecture and practice)
Practice phase Send text or graphic messages to students..
Sets a pause lecture threshold Actively poses a self-rated learning progress question to each student (Likert 1–5) Student functions
Expresses an intention to pause the lecture by pressing the ‘‘Pause’’ key Expresses learning progress by pressing a number key (1–5)
Server
Calculates the total number of ‘‘pause’’ selections and prompts a ‘‘pause lecture’’ message to the teacher if threshold is reached After the teacher stops the lecture phase, the system poses a self-rated learning progress question to each student (Likert 1–5)
Becomes a tutor or searches for a tutor Calls for teacher’s help Enrolls students in AZ or withdraws them from AZ
Shows each student’s learning status on the teacher’s interface Prompts match messages to a tutor and tutee, respectively, including the tutor’s and tutee’s names and seat places Sends ‘‘notification’’ message to the teacher if the mutual help task fails After each student presses the ‘‘Finish Practice’’ button, the system poses a selfrated learning progress question to each student (Likert 1 * 5)
Finish practice button and checkbox to be a tutor
Search tutor button and checkbox to call for teacher’s help
Whole class Information (tutor number/ Searching for tutor number/ calling for teacher’s help number)
Fig. 6 Student’s PLITAZ interface
virtually the first choice for junior high school female students in neighboring areas. Few participants were familiar with multimedia editing software before attending the high school. The software adopted in this study, Goldwave and VideoStudio, was new to all participants. The participating teacher in this study, also one of the researchers, was a Life Technology teacher at the high school.
123
510
J.-J. Dong, W.-Y. Hwang
Text and graph message broadcast buttons with broadcast range scroll box
Pause lecture threshold
Self-rated learning progress question button
Whole class information practicing number, finished practice number, searching help number, tutor number
Fig. 7 Teacher’s PLITAZ interface
PLITAZ training Participants received training on the use of the PLITAZ system within a 2-week period of lessons before formal research. The teacher guided the participants to follow two to three simulated learning sessions. During simulated lecture phase, participants were asked to use the ‘‘Pause’’ key to freely express their intention to pause the lecture and to know how their individual opinion had been calculated and anonymously displayed on the teacher’s screen. The students were excited to use the ‘‘Pause’’ key, but the teacher worried about whether the students would abuse this strategy. Therefore, students were told to press the key only when they were not able to remember or were not able to understand the lecture during a formal learning session. For the teacher, the greatest difficulty of using this ‘‘pause’’ strategy was to share the lecture progress control privilege with all students. During the training practice phase, students were freely able of being tutors or tutees via the PLITAZ system. After tutortutee match, designated tutors were asked to move to the tutees’ location. Procedures The Life Technology course is required for all senior high school students in Taiwan; the course has several course domains including ‘‘Technology in Life’’, ‘‘Creative Design’’, ‘‘Communication Technology’’, ‘‘Transportation Technology’’, ‘‘Manufacture Technology’’, ‘‘Energy Technology’’, and ‘‘Construction Technology’’. The pilot study was conducted with a ‘‘Creative Design’’ domain activity. Students had two consecutive Life Technology lessons per week, 50 min a lesson. In this study, a 3-week period was assigned for the students to learn Goldwave and VideoStudio software in a computer classroom (see Fig. 3). When in class, the teacher started the PLITAZ system and opened the ‘‘teacher’s interface’’ while students opened the ‘‘student’s interface’’ and logged into the system. Each learning session was composed of one lecture phase presented by the teacher and a practice phase for the students, as well as two self-rated learning progress questions.
123
Using PLITAZ system
511
Self-rated learning progress (1~5) after practice
Self-rated learning progress (1~5) after lecture
Lecture Phase
Q
Practice Phase
Q
Students' intentions to pause lecture Instant tutor-tutee match & attention zone treatments to minimize learning difference Fig. 8 Learning strategies and self-rated learning progress. Note: The figure includes a free icon set at: http://www.visual-blast.com/graphics/icons/girl-avatars-free-psd-vector-icon-set/
Although all students in the three study classes, A, B, C, were given the same lecture material with the same teacher over the 3-week period, neither the teacher’s speech speed nor the students’ practice speed of a lesson could be consistently controlled throughout the three classes. There were often varied learning sessions in a lesson depending on the length of the lecture and the practice time (see Fig. 8). During the lecture phase, the teacher used a commercial classroom management product to broadcast his screen to the students’ computers, and switched the PLITAZ into ‘‘lecture’’ phase. The students could express their intention to pause the lecture by pressing the ‘‘Pause’’ key. After the teacher stopped the lecture, PLITAZ posed a self-rated learning progress question to each student. After answering the question, the students began the practice session. During this practice phase, the ‘‘instant tutor-tutee match’’ and ‘‘AZ’’ strategies were implemented to help students learn. When a student finished a practice exercise, PLITAZ posed another self-rated learning progress question to the student. Then the class progressed to a new learning session. In this study, a learning session was composed of both the lecture and practice phases as well as two self-rated learning progress questions. Each student’s learning log, which included pausing the lecture, searching for a tutor, providing help, self-rated learning progress, and AZ enrollment status, etc., was recorded as one learning session profile. The study investigated strategies’ effect on learning progress differences among class rather than only individual’s learning behavior. Under this premise, this study analyzed all learning sessions in class A, B, C together. Self-rated learning progress question implementation To successfully embed a technology into classroom, some factors such as integration flexibility, promoting learning awareness, and minimal setting change in classroom should be concerned (Dillenbourg 2011; Dillenbourg and Jermann 2010). The research wanted to minimize learning progress differences with instant tutor-tutee match and AZ strategies;
123
512
J.-J. Dong, W.-Y. Hwang
the student’s learning progress must be measured instantly in class. This study adopted self-rated learning progress questions rather than achievement scales based on the teacher’s years of experience and the following reasons: • In a software learning class, it was not feasible to implement an achievement test for every student relating to each operating step right after the teacher presents a lecture. For example, after students listened to a lecture about Goldwave and how to trim two songs and merge them together, time was not available to measure each student’s achievement; it could take the entire class time. (flexibility and minimalism factors) • When the teacher asked the students, ‘‘Is it ok to move on to the next lecture?’’ students must answered after a self evaluation process. (awareness factor) • Without self evaluation, students could not decide whether provided help or searched for help. (awareness factor) Self-rated learning progress questions were more feasible and more suitable for an authentic learning environment. Self-rated learning progress questions, based on a fivepoint Likert scale, were used to measure each student’s learning progression after the lecture (shown in ‘‘Self-rated learning progress after lecture phase’’ column of Table 2) and after practice phases (shown in ‘‘Self-rated learning progress after practice phase’’ column of Table 2), respectively. The timing of the two questions is represented by the two ‘‘Q’’ in Fig. 8, while the question graph is shown in Fig. 9. Like the in-class self-rated learning progress question conducted by the teacher, the two learning progress questions Table 2 Learning session profile Session number
Class
Practice phase
Lecture phase
Self-rated learning progress after lecture phase
Number of paused-lecture students
Means (Likert 1–5)
Std. D
Number of tutors
Number of tutees
Self-rated learning progress after practice phase Number of AZ students
Means (Likert 1–5)
Std. D
1
A
18
3.22
1.30
28
3
5
4.23
1.04
2
A
22
3.86
0.92
15
1
10
4.14
0.79
3
A
18
3.74
1.07
19
4
32
3.68
1.16
4
A
18
3.98
0.95
22
1
15
4.17
0.87
5
B
26
3.84
1.02
23
5
3
4.31
0.71
6
B
11
3.97
0.97
13
3
21
4.50
0.76
7
B
0
4.13
0.89
14
1
20
4.29
0.84
8
B
9
3.79
0.99
7
0
17
3.91
0.54
9
C
8
3.21
0.97
1
0
3
3.50
0.80
10
C
3
3.33
0.91
2
4
3
3.81
1.17
11
C
17
3.32
1.32
2
1
29
3.53
1.21
12
C
14
3.53
1.39
4
3
21
3.96
1.11
13
C
16
3.18
1.19
2
4
31
3.40
1.45
Notes Std. D Standard Deviation Number of paused-lecture students: the number of students who at one point pressed the ‘‘Pause’’ key during the lecture phase in each session Number of AZ students: the number of students who were enrolled at one point in an AZ during the practice phase in each session
123
Using PLITAZ system
513
1)Totally not understanding 2)Not understanding 3)Normal 4)Understanding 5)Totally understanding
Confirm or cancel
Fig. 9 Prompted self-rated learning progress question
posed by the system focused on identifying the level at which students acquired the lecture material in each learning session. Pause lecture implementation During the lecture phase, each student was encouraged to express an intention to pause the lecture by pressing the ‘‘Pause’’ key on the keyboard when she was not able to memorize more steps or did not understand the material; that is, she did not meet the first two fundamental learning objectives of Bloom’s revised taxonomy. In this study, the teacher set the pause lecture threshold as 8–12 paused lectures by students in each lecture phase because the teacher estimated he could help 8–12 students during the following practice phase without becoming overwhelmed. When the ‘‘paused-lecture’’ student number reached the threshold set by the teacher, a reminder message popped up on the teacher’s screen, which was broadcast to the whole class during the lecture phase (see Fig. 10). The teacher could then stop the lecture immediately after receiving the message. Alternatively, the teacher could conduct an in-class self-rated learning progress question via the system for the teacher to confirm the class’s overall learning condition again. PLITAZ repeated sending reminder messages when the paused-lecture student number = threshold ? 5, ?10 …, if the teacher kept presenting the lecture. In this study, some lectures were finished before the threshold was reached, while some did not, depending on the teacher’s lecture progress and judgment. Instant tutor-tutee match implementation To foster peer scaffolding of Vygotsky’s ZPD in software learning, during the practice phase, students who exhibited quick learning progress during the lecture phase were Message to teacher in the right bottom corner of screen: 12 students have expressed intensions to pause lecture
Fig. 10 Prompted reminder message sent to teacher’s computer
123
514
J.-J. Dong, W.-Y. Hwang
Fig. 11 Notification messages to tutor and tutee, respectively
encouraged to volunteer to serve as tutors; these volunteers joined a tutor queue. When a student made a help request, the PLITAZ system automatically matched a tutor and tutee together. Information about the tutee and tutor, including name and seat location, was sent to tutor and tutee, respectively (see Fig. 11). The tutor was asked to move to the tutee’s place to provide face-to-face assistance. If there was no tutor candidate in the queue, the help request was forwarded to the teacher. After receiving the tutor’s help, the tutee could respond as to whether the assistance was successful or not via PLITAZ. If the assistance failed, a notification was sent to the teacher so the teacher could provide help if deemed necessary. If the tutor’s assistance was successful, the tutor got a bonus score. When a tutor was helping a tutee, she was removed from the tutor queue. After the help task, she was moved to the last place in the tutor queue so the workload could be balanced among tutors. In addition to requesting help from a tutor, help from the teacher was also directly available. On both the teacher’s and students’ interfaces, information pertaining to the numbers of tutor-and-tutee matches were instantly updated. This allowed both students and the teacher to make suitable decisions about either requesting or providing help. Attention zone implementation There were two assumptions according to the teacher’s observation. The first assumption was that in a computer classroom, students first searched for assistance from an immediate neighbor when encountering problem, and the assumption was proved later by Q15 of the learning behavior questionnaire (see Table 4). The second assumption was that when students were not in the mood to learn, they probably pressed the ‘‘Pause’’ key. This assumption was verified later by Q10 of the learning behavior questionnaire. Based on the above two verified assumptions, it was concluded that the ‘‘pause lecture’’ intention was not always equal to low learning progress during the lecture phase. Also, if immediate neighbors learned more quickly, they could serve as essential tutors. The researchers proposed an AZ algorithm to help the teacher identify the students who did not meet the first two fundamental learning objectives of Bloom’s revised taxonomy, and those who had low availability of scaffolding based on Vygotsky’s ZPD. The teacher could quickly help the AZ students before they made help requests during the practice phase. With regard to the AZ enrollment algorithm, three learning conditions (a, b, c) were taken into consideration. Condition a: If a student pressed the ‘‘Pause’’ key during the lecture phase, it meant she could not keep up with the teacher’s lecture progress or if she was not in a mood to learn.
123
Using PLITAZ system
515
Fig. 12 Learning status table on teacher’s interface. Note: *Font color—black [practicing], orange [searching for help], blue [involved in tutor queue], green [practice finished] *Background color—red [enrolled in AZ], white [withdrew from AZ]), gray [never in AZ] (Color figure online)
Condition b: If a student’s learning progress, as determined by the self-rated learning progress question after the lecture phase, was lower than average. Condition c: If a student’s immediate neighbor(s) was not capable of giving support. (Self-rated learning progress after the lecture phase was lower than 5.) AZ Enrollment = (a pause intention expressed [ b one’s learning progress \= whole class average learning) \ (c both immediate neighbors’ learning progresses during lecture phase \ 5). The PLITAZ ran the AZ enrollment procedure half a minute after the teacher stopped the lecture phase, and the students were expected to have finished the self-rated learning progress question. (‘‘Q’’ after the lecture phase in Fig. 7). The student enrolled in the AZ would withdraw from the AZ if her practice was finished—condition ‘d’, or if one of her neighbor’s practice exercises was finished so that an immediate tutor resource was available—condition ‘e’. AZ Withdraw = (d one’s practice finished) [ (e one of immediate neighbors’ practice finished). The students who sat in the first and last seats in each of the four columns in the computer classroom (see Fig. 12) would have a higher possibility to be enrolled in the AZ because they only had one immediate neighbor to assist them. With regard to privacy, the student’s learning status and AZ lists were only available to the teacher. Each student’s ID and name during the practice phase was coded with a specific font color and background color in the teacher’s interface of the PLITAZ system. The teacher could easily recognize each student’s learning status, such as ‘‘practicing, practice finished, searching for help, and tutoring’’. The teacher could also identify those who were part of the AZ, which was marked with a red background. Since students who enrolled in the AZ experienced lower learning progress and had less immediate neighbors’ support, the study gave AZ students special treatments to minimize a learning progress difference between AZ and non-AZ students:
123
516
J.-J. Dong, W.-Y. Hwang Potential peer support
Peer support
Teacher’s attention
Peer support
Fig. 13 AZ chain breakup with teacher’s attention
• Longest AZ chain breakup by teacher: since the readily available potential tutor was the immediate neighbor, a student inside an AZ chain, formed by consecutive AZ students, would find it more difficult to get immediate neighboring support. For example, in Fig. 13, there were five students, labeled as AZ1 to AZ5 side-by-side, forming the longest AZ chain at some moment during each practice phase. The teacher paid more attention to and actively helped the middle student, AZ3, of the longest AZ chain for two reasons: First, the edge of the AZ chain (AZ1 and AZ5) could get support from one immediate non-AZ neighbor. Second, once the learning progress of the middle student, AZ3, had been improved and withdrawn from the AZ, the longest chain was then broken up into two smaller AZ chains and the middle student, AZ3, could become a potential tutor of her immediate neighbors, AZ2 and AZ4. In other words, after the teacher prided help once, three AZ students benefited from the policy. On the other hand, if the teacher helped the edge student of the AZ chain, for example AZ1, only two AZ students, AZ1 and AZ2, benefited after the teacher helped once. In summary, with the longest AZ chain breakup conducted by the teacher’s policy, the learning progress differences could be minimized faster. • The top five of the ‘‘fast learning progress’’ tutor list via PLITAZ: The list was prompted via the PLITAZ system for AZ students in the beginning of each practice phase. AZ students were encouraged to search for the recommended tutors’ support directly. • High priority of support by the teacher: if more than two students had problems at the same time, the teacher addressed the needs of the AZ students first. • High priority of tutor-tutee match via PLITAZ: If there were more tutees than tutors, the tutor-tutee match was given priority to AZ tutees. Instrument: usefulness and learning behavior questionnaires The researchers developed a ‘‘system usefulness’’ questionnaire with seven questions (Q1 to Q7) as depicted in Table 3 as well as a ‘‘learning behavior’’ questionnaire with 11 questions (Q8 to Q18) as seen in Table 4. Cronbach’s a coefficient of the ‘‘system usefulness’’ questionnaire was 0.838, while the content validity of the ‘‘learning behavior’’ questionnaire was ensured by the lecture teacher and researchers in this study. Questionnaires were answered anonymously by 82 participants from classes B and C after 3 weeks of One-group Pretest–Posttest of Pre-experimental design study. Several participants in each class were interviewed by a researcher to get their responses concerning system usage.
123
Using PLITAZ system
517
Table 3 Survey of system usefulness Mean
Std. D
Q1. During the lecture phase, I could express my learning condition with the ‘‘pause function’’
3.90
1.03
Q2. During the lecture phase, I could express my learning condition by answering self-rated learning progress questions conducted by the teacher
4.04
0.94
Q3. During the practice phase, the ‘‘search tutor’’ function was useful when encountering problems
3.93
0.97
Q4. During the practice phase, the ‘‘call for help from teacher’’ function was useful when encountering problems
4.22
0.83
Q5. During the practice phase, the ‘‘key point’’ or ‘‘reminder messages’’ sent by the teacher were useful
4.13
0.83
Q6. After the lecture phase, I could express my learning condition through self-rated learning progress inquiry
3.72
1.01
Q7. After the practice phase, I could express my learning condition through self-rated learning progress inquiry
3.98
0.85
Mean
Std. D
Table 4 Survey of learning behavior
Q8. During the lecture phase, I would press ‘‘Pause’’ if I could not remember the teacher’s lecture steps
3.62
1.13
Q9. During the lecture phase, I would press ‘‘Pause’’ if I could not understand the lecture material
3.61
1.09
Q10. During the lecture phase, I would press ‘‘Pause’’ if I was not in the mood to listen to the lecture
2.21
1.33
Q11. At what percent (%) (of students who pressed the ‘‘Pause’’ key) should the teacher stop the lecture and let students practice?
34.18
17.72
Q12. If I understood the lecture material, I would like to be a tutor although I still practiced the exercises
2.85
1.11
Q13. If I understood the lecture material during lecture phase, I would like to be a tutor only if I had finished practice exercises
4.12
1.01
Q14. If I didn’t totally understand the lecture material, I would still like to be a tutor if I had finished practice
2.26
1.12
Q15. During the practice phase, the first person I asked for help from was my immediate neighbor(s)
4.78
.47
Q16. I would call for help from other classmates via the system if my problem could not be solved by immediate neighbor(s)
3.45
1.11
Q17. I would call for help from the teacher via the system if my problem could not be solved by immediate neighbors
3.88
1.06
Q18. If the system could automatically select a tutor from the following three subjects: best friend (B), teacher (T), or other classmates (O)—which priority would you prefer? (1) B, O, T 42 51.2% (2) B, T, O 23 28.1% (3) T, B, O 9 11.0% (4) O, B, T 5 6.1% (5) T, O, B 2 2.4% (6) O, T, B 1 1,2%
123
518
J.-J. Dong, W.-Y. Hwang
AZ students’ learning conditions estimate method The PLITAZ system implemented an AZ enrollment algorithm to detect students who needed more support and attention from the teacher during practice exercises, while a withdraw algorithm was used to detect when the AZ students could withdraw from the AZ. The researchers wanted to know whether the AZ enrollment algorithm correctly identified AZ students during the practice phase. First, immediate neighbors were the most readily available potential tutors for students in the computer classroom; it was assumed that more AZ students indicated less class learning progression during the practice phase and vice versa. Correlation analysis was conducted with AZ student numbers and the average learning progress improvement rate between two self-rated learning progress evaluations within 13 sessions. The researchers classified as three subgroups according to the AZ enrollment algorithm: (1) (2) (3)
Students only pressed the ‘‘Pause’’ key during the lecture phase, classified as ‘‘AZ-pause’’. Students only had low self-rated learning progress after the lecture phase, classified as ‘‘AZ-low’’. Students both pressed the ‘‘Pause’’ key during the lecture phase and had low selfrated learning progress after this phase, classified as ‘‘AZ-pause and low’’.
The possible learning behaviors of the three AZ subgroups according to the teacher’s responses and an interview with students after the study were assumed to be as follows: AZ-pause 1. Students rated themselves as not being able to catch up with lecture progress, and pressed the ‘‘Pause’’ key, but their learning progression was still above average. 2. Students thought they caught up with the teacher’s lecture progress but they were eager to practice so they pressed ‘‘Pause’’. 3. Students might not be in a mood to learn and they wanted more time to participate in off-task behaviors. AZ-low 1. Students who experienced low learning progress but did not actively express their individual learning condition. 2. Students who had low learning progress during the lecture phase but did not want to interrupt the teacher. AZ-pause and low 1. Students who had low learning progress during the lecture phase and they actively expressed an individual learning condition. 2. Students had low learning progress because they were not in a mood to learn and they wanted more time to participant in off-task behaviors. A v2 test for homogeneity and an independent t test to evaluate learning behaviors, including searching for a tutor, being a tutor between AZ, and subgroups of AZ and nonAZ students, were taken as references to verify the AZ enrollment algorithm. Learning progress differences minimized with PLITAZ estimate method In statistics, the Standard Deviation (Std. D) is normally used for measuring diversity of variables; a high standard deviation means each subject’s data scatter over a large range. In
123
Using PLITAZ system
519
this study, the standard deviation of students’ self-rated learning progresses after lecture phase and after practice phase was used to represent learning progress differences among students after lecture phase and after practice phase, respectively. Simple proportions of class student numbers such as one-fifth, one-fourth, one-third, and one-half, two-thirds, and three-fourths, etc., are often used in consensus decisionmaking of class affairs. Therefore, a simple proportion could be a feasible threshold for a teacher to decide when to stop the lecture for minimizing learning progress differences. Furthermore, an opinion was in the majority when the number was over one-half. Therefore, only lecture pause thresholds of one-fifth, one-fourth, one-third, and one-half were tested. For example, if there were 42 students in each class; the pause lecture threshold was set at a student number of 8.4 (or one-fifth), 10.5 (or one-fourth), 13.86 (or one-third) and 21 (or one-half), respectively. The researchers defined a learning session with a higher paused-lecture student number than the threshold as a ‘‘High-Pause’’ session, while a learning session with a lower paused-lecture student number than the threshold was referred to as a ‘‘Low-Pause’’ session. The researchers took one-fifth, one-fourth, one-third, and one-half of the pausedlecture student numbers as the pause lecture thresholds to classify the sessions as four different ‘‘High-Pause’’ and ‘‘Low-Pause’’ session combinations, respectively. The ‘‘independent t’’ test was conducted in each combination to examine if the HighPause sessions had larger learning progress differences than the Low-Pause sessions to estimate a suitable pause threshold after lecture phase. (See the third column of Table 6). This pilot study adopted One-group Pretest–Posttest of Pre-experimental design to examine whether the learning progress differences minimized after practice phase with instant tutor-tutee match and AZ strategies. It was assumed the High-Pause sessions had larger learning progress differences than Low-Pause sessions after lecture phase (Pretest: see the third column of Table 6). Independent t test was also conducted to compare HighPause and Low-Pause sessions’ learning progress differences scores after practice phase (Posttest: see the forth column of Table 6). If High-Pause and Low-Pause leaning sessions’ learning progress differences became similar after practice phase, it might support the efficiency of the instant tutor-tutee match and AZ strategies’ effect on minimizing learning progress differences during practice phase.
Data analysis and results In this pilot study, 13 learning sessions were recorded including four sessions from class A and B, respectively, but five sessions were recorded from class C (See Table 2). The log data and questionnaires were analyzed respect to research questions. Analysis of learning challenges solving with PLITAZ System As we stated, there were learning challenges in computer classroom learning such as limited communication between peers and the teacher, unknown learning conditions, offtask behaviors, heavy teacher workloads, and social isolation. With pause strategy during lecture phase, student almost expressed pause intention all sessions, except session 7, during lecture phase. During practice phase, the students who sat far away from teacher used PLITAZ to search help from teacher (14 times in Fig. 14) that increase communication between teacher and students and help teacher understood students’ learning condition.
123
520
J.-J. Dong, W.-Y. Hwang
Fig. 14 Search for help priority count
The instant tutor-tutee match strategy was implemented via the PLITAZ system to foster peer scaffolding as well as to reduce the teacher’s workload, to reduce off-task behaviors, and to reduce social isolation during the practice phase. From the questionnaire, students stated that they were willing to become tutors during the practice phase (Q12, Q13, and Q14). According to the log data, tutor number than tutees in all 13 learning sessions. Therefore, an instant tutor-tutee match was feasible in the computer classroom. The students would search for help from the teacher or tutors via PLITAZ when the problem could not be solved by immediate neighbor(s) (Q16 and Q17). Figure 14 showed that students preferred searching for help from immediate neighbors than from the teacher or tutors (from immediate neighbors then tutors ? from immediate neighbors then the teacher: total of 77% or 27 out of 35 times in 13 sessions). Immediate neighbors were the most readily available potential tutor resources. In further analysis, tutors mediated by PLITAZ took a 60% workload in helping others from the teacher (from tutor directly ? from immediate neighbor then tutor: in total 21 out of 35 times in 13 sessions). It meant the instant tutor-tutee match strategy took a 60% workload from the teacher during the practice phase. Analysis of AZ students’ learning conditions The students in the three participating classes created 546 records during the practice phases in 13 sessions, including 210 records classified as AZ records and 336 records categorized as non-AZ records. There were 59 AZ-pause records (28% in AZ), 103 AZ-low records (49% in AZ) and 48 AZ-pause and low records (23% in AZ). The v2 test for homogeneity and Independent t test were conducted to compare AZ, subgroups of AZ students with non-AZ students in self-rated improvement, intention to become tutor and to search tutor via PLITAZ (See Table 5). There was a significant difference in number of tutor between overall AZ and non-AZ (v2 = 11.74***) students; non-AZ students were more aggressive than AZ students to become a tutor during the practice phase, especially compared to AZ-pause (v2 = 9.04**) and AZ-low students (v2 = 8.05**). No significant result found on searching tutor via PLITAZ. Although overall AZ students did not significantly improve compared to non-AZ students (t = 1.913), AZ-low students (49% of AZ students) had a significant improvement compared to non-AZ students (t = 0.319*). However, it was noticeable that AZ-pause and low students (23% in AZ) had fewer improvements compared to non-AZ students (t = -0.378*), which counteracted the overall AZ students’ improvement.
123
Using PLITAZ system
521
Table 5 Comparison AZ, subgroups of AZ with non-AZ Self-rated improved
Number of tutors
AZ (n = 210)
AZ \ non-AZ (v2 = 11.74, p = 0.001***)
AZ-pause (n = 59)
AZ-pause \ non-AZ (v2 = 9.04, p = 0.003**)
AZ-low (n = 103)
AZ-low [ non-AZ (t = 0.319, p = 0.043*)
AZ-pause and low (n = 48)
AZ-pause and low \ non-AZ (t = -0.378, p = 0.046*)
AZ-low \ non-AZ (v2 = 8.05, p = 0.005**)
* p \ 0.05, ** p \ 0.01, *** p \ 0.001
To sum up, AZ students had a lower intention to become tutors than non-AZ students, and AZ student numbers influenced students’ improvement during the practice phase that verified the AZ enrollment algorithm effect.
Analysis of minimizing learning progress differences with PLITAZ system Students agreed that they used the designated ‘‘Pause’’ key when they could not remember more procedure steps or when they could not understand the lecture (see Q1, Q8, and Q9 of the questionnaire). However, some students stated that they sometimes pressed the ‘‘Pause’’ key if they felt bored or if they just wanted to do private work (see Q10 of the questionnaire). Students agreed that 34% is a good pause lecture threshold (Q11), which is close to onethird of the class. This means that the students expected the teacher to pause the lecture when one-third of the students expressed their intention by press ‘‘Pause’’. As for log data analysis, the result showed that, after lecture phase, ‘‘High-Pause sessions’’ had significantly larger learning progress differences than ‘‘Low-Pause sessions’’ when the lecture pause threshold was set as one-fifth (t = 3.15, p = 0.009**), one-fourth (t = 2.90, p = 0.016*) and one-third (t = 2.99, p = 0.017*), respectively. After the practice phase using the PLITAZ strategies, the result showed that the learning progress differences in High-Pause sessions became similar to those of the Low-Pause sessions after the practice phase. In other words, in the three combinations (the lecture phase threshold set at 1/5, 1/4, and 1/3 of the class), the High-Pause sessions had a transition from a larger learning progress difference (Third column of Table 6) to a lower learning progress difference than Low-Pause sessions (Forth column of Table 6).
Discussion Improvements on solving learning challenges with PLITAZ system Within a software learning class, a student learns by doing rather than just by and hearing. Based on literature review (McGrail 2007) and teacher experience, there are some challenges in computer classroom learning such as limited communication, social isolation, off-task behavior, unknown learning conditions, and heavy workload on the teacher that result in large learning progress differences among students. To solve the learning
123
522
J.-J. Dong, W.-Y. Hwang
Table 6 Independent t test of learning progress differences Lecture pause threshold
Session members
Learning progress differences (Std. D) after lecture phase
Learning progress differences (Std. D) after practice phase
1/5 class
H [1, 2, 3, 4, 5, 6, 8, 11, 12, 13] L [7, 9, 10]
t = 3.15, p = 0.009** (H [ L)
t = 0.157, p = 0.878
1/4 class
H [1, 2, 3, 4, 5, 6, 11, 12, 13] L [7, 8, 9, 10]
t = 2.90, p = 0.016* (H [ L)
t = 1.155, p = 0.272
1/3 class
H [1, 2, 3, 4, 5, 11, 12, 13] L [6, 7, 8, 9, 10]
t = 2.99, p = 0.017* (H [ L)
t = 0.163, p = 0.132
1/2 class
H [2, 5] L [1, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13]
t = -0.87, p = 0.403
Note H High-Pause sessions, L Low-Pause sessions * p \ 0.05, ** p \ 0.01
challenges as well as to minimize learning progress differences, the researchers developed PLITAZ strategies based on revised Blooms’ taxonomy and Vygotsky’s ZPD during the lecture phase and the practice phase, respectively. After 3 weeks study, the teacher found the learning challenges more or less been reduced. Before this study, the face-to-face communication between teachers and students was obstructed by computers so the teacher was not aware of students’ learning condition. In this study, with the ‘‘Pause’’ strategy, the students actively express their learning condition. Some students responded during the interview that ‘‘pause’’ was the most interesting function. Based on the teachers’ observation, with this function, the students participated in the lecture more aggressively and had more opinions during the lecture phase. The teacher also agreed that the ‘‘AZ’’ strategy helped him understand the class’s learning condition, which enabled him to quickly help the students who had low learning progress to minimize learning progress differences among class. Before this study, some students who sat far away from teacher often could not get a teacher’s attention when they had a practical problem. In this study, with PLITAZ, students could call the teacher very fast. This was especially useful when students encountered some exceptional problems, which were not included in the lecture progress; they would search for help from the teacher rather than from other students. In other words, ‘‘Pause’’ strategy provided alternative method for communication between the teacher and students as well as to prevent learning progress differences becoming large. Tutty and Klein (2008) found the students in a virtual environment asked more questions compared to students in a face-to-face environment with respect to collaboration learning. Before this study, the students neither knew who needed help nor who could and wanted to provide help during practice time. With the ‘‘Instant Tutor-Tutee Match’’ strategy, there were more interactions among the whole class, which reduced the incidence of social isolation. In each learning session, the number tutors was greater than the number of tutees. Students could show their intention to help and search for help from others so that learning progress differences could be minimized. Although, in this study, the off-task behaviors of students were not quantitatively measured, the teacher could infer that the learning atmosphere classroom was active. To get a bonus score, the students with high learning progress waited for opportunities to help others with low learning progress. This helped students avoid off-task behaviors in class.
123
Using PLITAZ system
523
Students agreed that the system was useful (from the system usefulness questionnaire) in assisting their learning. After data analysis, it was found that tutors mediated by PLITAZ took a 60% workload in helping others from the teacher. The teacher also responded that while using PLITAZ, the overall lesson progress was faster than before, while overall workload with helping students solve problems was reduced. AZ students need more attention and help The study showed that students had more productive learning after the teacher paid attention to them (Hall et al. 1968). The researchers proposed the ‘‘AZ’’ strategy based on revised Bloom’s taxonomy of learning objectives and Vygotsky’s ZPD, which can help a teacher identify those students who expressed low learning progress, had few immediate neighbors’ scaffolding, or those who might not be in a mood to learn. AZ students received the teacher’s attention and several treatments during the practice phase, including: the longest AZ chain breakup policy, the top five recommended tutor lists, and the highest priority when searching for help. AZ student numbers in each session had a significant negative relation with average self-rated learning progress improvement (r = -0.565, p = 0.044*). This meant that if there were more AZ students, the students’ average improvement during the practice phase would be decreased and vice versa. The result also corresponded to non-AZ students who were more willing to become tutors; more non-AZ students, and more tutors volunteering their availability to help increase overall improvement. The threshold set—as one-third of the class student number—also prevented large learning progress differences and enabled the following practice phase to have enough tutors. The teacher often responded to AZ students who had lower learning progress, or those who indicated problems when asked, ‘‘Any questions?’’ The longest AZ chain breakup policy was a good method to help low learning progress students. However, the top five recommend tutors list and high priority allocated to the tutor-tutee match treatments could not have had an obvious effect because AZ students did not actively search for help compared to non-AZ students. To sum up, according to comparisons between AZ and non-AZ students’ learning behaviors, the ‘‘AZ’’ strategy really identified the students who need the teacher’s attention and help. PLITAZ system help minimize learning progress differences Previous studies have found that students’ concentration declined after 10–15 min during classroom instruction, while giving students a brief pause had a significant effect on overall attention and performance. Furthermore, the first two fundamental learning objectives of Bloom’s revised taxomony—remembering and understanding—are crucial to successfully learn software (Stuart and Rutherford 1978; Falkner 2009; Ruhl et al. 2010). This study designed a ‘‘Pause’’ strategy in which students could express their intention to pause a lecture when they could not remember more steps or they could not understand the lecture content. Simple proportions of class student numbers were often adopted as a threshold for class affair decision making. In this study, the researchers tested several combinations of highpause lecture sessions and low-pause lecture sessions with different thresholds and found simple proportions of one-fifth, one-fourth, and one-third could be the threshold to minimize learning progress differences.
123
524
J.-J. Dong, W.-Y. Hwang
If the teacher wanted to strike a balance between increasing lecture progress, reducing screen switch frequency, and minimizing students’ learning progress differences as well as students’ expectations, a threshold of one-third would be more suitable than one-fifth and one-fourth. Briefly, one-third paused-lecture students would be a suitable threshold for teacher and students to have a consensus on the timing to stop lecture as well as to minimize learning progress differences. Students’ learning productivity when waiting for an assistant was only 38% (Alavi et al. 2009). The AZ algorithm helped the teacher identify who most needed support. AZ students received several treatments to enhance their learning during the practice phase, including the longest AZ chain breakup policy, being given high priority when searching for help, and given access to the top five tutors list. However, AZ students did not actively search for help compared to non-AZ students (number of tutee, v2 = 0.96), which led to the ‘‘high priority given to searching for help’’ and the ‘‘top five tutors list’’ treatments not having an obvious effect. However, instant Tutor-Tutee match strategy, in this study, still took a 60% workload from the teacher and helped reduce the waiting time for help. As for learning performance between AZ and non-AZ students, the AZ-low students (49% of AZ students) improved more than non-AZ students; however, AZ-pause and low students, in contrast, improved less compared to the non-AZ students. The reason why AZ-pause and low students had lower learning progress comparing to non-AZ students with AZ and instant tutor-tutee match strategies was still not clear in this study. The teacher also responded, with an instant tutor-tutee match and AZ strategy, the overall learning progress became faster than before. Besides, High-Pause sessions had a transition from a larger learning progress difference to a lower learning progress difference as Low-Pause sessions so that learning progress differences decreased significantly after the practice phase with PLITAZ that support the strategies’ effect on reducing learning progress differences during the practice phase. Limitation and future study After this study, some interesting issues arose for further investigation. First, the students responded that they preferred their best friends as tutors by questionnaire. In future, PLITAZ can add a ‘‘best friend’’ setting within the tutor-tutee match strategy to promote peer help. Second, the AZ student number had a significant negative relation with overall learning achievement during the practice phase. It can be a trial that after AZ algorithm identify the AZ students, system rearranges the students’ seats dynamically to minimize the AZ student number and to reduce social isolation as well as to increase overall learning achievement in further research. Third, there was no learning behavior times stamp recording with PLITAZ, except the start and the end of each student’s practice phase. However, it’s found many students didn’t press the button of Finish Practice button right after their practice finished that the practice time was not precisely. Finally, since students’ concentration in class declined after 10–15 min in a classroom learning environment, it will be worthwhile to record each time the ‘‘Pause’’ key is used to verify whether the teacher should make a short break after 10–15 min. This information can be used to further investigate the AZ-pause and AZ-pause and low students’ learning behaviors.
123
Using PLITAZ system
525
Conclusion and suggestion This purpose of this study is to investigate feasible strategies to minimize learning progress differences as well as to solve learning challenges with a newly devised system, PLITAZ, based on Bloom’s revised taxonomy of learning objectives and Vygotsky’s ZPD. With PLITAZ, learning challenges such as limited communication between peers and the teacher, unknown learning conditions, off-task behaviors, a heavy workload on the teacher, and social isolation, could be solved while learning progress differences could be minimized. A simple proportion of one-third of the class was found to be a suitable pause lecture threshold, which could minimize learning progress differences as well as to provide enough tutorial resources during the lecture phase in software learning. During practice phase, this study found the instant tutor-tutee match extended scaffolding for students and shared about 60% of workload on helping solving problems that the teacher could focus on helping the students whose problems are more difficult. The AZ strategy in this study also successfully identified the students who had low learning progress and low peer immediate neighbors’ support during practice time, so the teacher could focus on helping students who needed more help. The study suggests that teachers of subjects other than computer software can use the pause lecture strategy to control lecture progress, to avoid overloading students cognitively, and to minimize learning progress differences. Furthermore, the pause lecture strategy also helps the teacher adjust lecture progress and levels of difficulty. Although many classes do not have dedicated practice times like software learning classes, it is common and useful to have peer discussion times after lectures; in this kind of instructional design, with a tutor-tutee match strategy, those categorized as ‘‘low learning progress students’’ are able to learn from some other students to get potential scaffold and proximal development. Additionally, the AZ concept is feasible in other classes. By using specific AZ algorithms for other classes, the teacher can identify which students need more help, and he or she possibly dedicate more instruction during discussion time. Acknowledgment This work was supported in part by the National Science Council (NSC), Taiwan, ROC, under Grant NSC 98-2511-S-008-008-MY3, NSC and 98-2511-S-008-005-MY3. Besides, the researchers thanks to the editor Prof. J. Michael Spector and all the reviewers’ suggestions and comments that helped the researchers revise the paper.
References Alavi, H. S., Dillenbourg, P., & Kaplan, F. (2009). Distributed awareness for class orchestration. Lecture Notes in Computer Science, 5794, 211–225. Anderson, L., & Krathwohl, D. A. (2001). Taxonomy for learning, teaching and assessing: A revision of Bloom’s taxonomy of educational objectives. New York: Longmans. Anderson, J. R., Boyle, C. F., & Reiser, B. J. (1985). Intelligent tutoring systems. Science, 228, 456–462. Bloom, B. S., & Krathwohl, D. R. (1956). Taxonomy of educational objectives: The classification of educational goals handbook I: Cognitive domain. New York: Longmans. Cain, J., Black, E. P., & Rohr, J. (2009). An audience response system strategy to improve student motivation, attention, and feedback. American Journal of Pharmaceutical Education, 73(2), 21. Chandler, P., & Sweller, J. (1992). The split-attention effect as a factor in the design of instruction. British Journal of Educational Psychology, 62, 233–246. Chandler, P., & Sweller, J. (1996). Cognitive load while learning to use a computer program. Applied Cognitive Psychology, 10(2), 151–170.
123
526
J.-J. Dong, W.-Y. Hwang
Chien, C. M. (2008). Applying peer tutoring to programming languages instruction platform. Taiwan: Unpublished dissertation of Yunlin University. Codework Systems (2010). Netop classroom management system. Codework Systems. Retrieved July 11, 2011 from http://www.codework-systems.com/products/netop/netop-vision/. Dillenbourg, P. (2011). Stellar Deliverable 1.5. Retrieved Aug 22, 2011 from http://www.stellarnet.eu/kmi/ deliverables/20110818_stellar___d1.5___trends-in-orchestration.pdf. Dillenbourg, P., & Jermann, P. (2010). Technology for classroom orchestration. New science of learning cognition computers and collaboration in education. New York: Springer. Falkner, N. (2009). Is comprehension or application the more important skill for first-year computer science students? ERGO, 1(2), 15–26. Galbraith, J., & Winterbottom, M. (2011). Peer-tutoring: What’s in it for the tutor? Educational Studies, 37(3), 321–332. Grimes, T. (1990). Audio–video correspondence and its role in attention and memory. Educational Technology Research and Development, 38(3), 15–25. Hall, R. V., Lund, D., & Jackson, D. (1968). Effects of teacher attention on study behavior. Journal of Applied Behavior Analysis, 1, 1–12. Harger, R. O. (1996). Teaching in a computer classroom with a hyperlinked, interactive book. IEEE Transaction on Education, 39(3), 321. Hoyt, A., McNulty, J. A., Gruener, G., Chandrasekhar, A., Espiritu, B., Ensminger, D., et al. (2010). An audience response system may influence student performance on anatomy examination questions. Anatomical Science Journal, 3(6), 295–299. Huang, Y.-M., & Wu, T.-T. (2011). A systematic approach for learner group composition utilizing u-learning portfolio. Educational Technology & Society, 14(3), 102–117. Kim, Y., & Baylor, A. L. (2006). A social-cognitive framework for pedagogical agents as learning companions. Educational Technology Research and Development, 54(6), 569–596. Lai, M., & Law, N. (2006). Peer helping of knowledge building through collaborative groups with differential learning experiences. Journal of Educational Computing Research, 35(2), 123–144. Lin, Y.-T., Huang, Y.-M., & Cheng, S.-C. (2010). An automatic group composition system for composing collaborative learning groups using enhanced particle swarm optimization. Computers & Education, 55(4), 1483–1493. Mayer, R. E., & Moreno, R. (2002). Aids to computer-based multimedia learning. Learning and Instruction, 12, 107–119. Mcgrail, E. (2007). Laptop technology and pedagogy in the English language arts classroom. Journal of Technology and Teacher Education, 15(1), 59–85. McKenzie, J. (2008). Getting attention in the laptop classroom. From Now On, 18(2). Moreno, R., & Mayer, R. E. (2000). A coherence effect in multimedia learning: The case for minimizing irrelevant sounds in the design of multimedia instructional messages. Journal of Educational Psychology, 92(1), 117–125. Nussbaum, M., Alvarez, C., Mcfarlane, A., Gomez, F., Claro, S., & Radovic, D. (2009). Technology as small group face to face collaborative helping. Computer & Education, 52, 147–153. Rosmalen, P., Sloep, P., Kester, L., Brouns, F., Croock, M., Pannekeet, K., et al. (2008). A learner support model based on peer tutor selection. Journal of Computer Assisted Learning, 24, 74–86. Ruhl, K. L., Hughes, C. A., & Schloss, P. J. (2010). The effect of differing audience response system question types on student attention in the veterinary medical classroom. Journal of Veterinary Medical Education, 37, 145–155. Stahl, G., Koschmann, T., & Suthers, D. (2006). Computer-supported collaborative learning: An historical perspective. Cambridge handbook of the learning sciences, 409–426. Stuart, J., & Rutherford, R. J. D. (1978). Medical student concentration during lectures. The Lancet, 312(8088), 514–516. Smart Technologies. (2011). SMART Sync classroom management software. Smart Technologies. Retrieved July 12, 2011 from http://smarttech.com/us/Solutions/Education?Solutions/Products?for? education/Software/SMART?Sync. Taiwan Good (2011). TWG multimedia teaching broadcast system. Taiwan Good. Retrieved July 10, 2011 from http://www.twg.tw/index.php. Tutty, J. I., & Klein, J. D. (2008). Computer-mediated instruction: A comparison of online and face-to-face collaboration. Educational Technology Research and Development, 56(2), 101–124. Westera, W. (2007). Peer-allocated instant response (PAIR): Computational allocation of peer tutors in learning communities. Journal of Artificial Societies and Social Simulation, 10. Wong, W. K., Chan, T. W., Chou, C. Y., Heh, J. S., & Tung, S. H. (2003). Reciprocal tutoring using cognitive tools. Journal of Computer Assisted Learning, 19, 416–428.
123
Using PLITAZ system
527
Xun, G. E., & Land, S. M. (2004). A conceptual framework for helping III-structured problem-solving processes using question prompts and peer interactions. Educational Technology Research and Development, 52(2), 5–22.
Jian-Jie Dong is a technology teacher of National Chia-Yi Girl’s Senior High School and also a Ph.D student of Department of Computer Science & Information Engineering, National Central University, Taiwan. Wu-Yuin Hwang is an associate professor of Graduate School of Network Learning Technology, National Central University, Taiwan.
123
