Investigating Classroom Activities in English Conversation Lessons based on Activity Coding and Data Visualization Zilu Liang1, Satoshi Nishimura2, Takuichi Nishimura2, Mario Alberto Chapa-Martell3 1
2
Faculty of Engineering, The University of Tokyo, Japan AI Research Centre, National Institute of Advanced Science and Technology, Japan 3 CAC Corporation, Japan
[email protected], {satoshi.nishimura, takuichi.nishimura}@aist.go.jp
Abstract.
Reflective teaching has become dominant paradigm in second language teacher education, as critical reflection helps teachers achieve a better understanding of teaching and learning processes. Critical reflection begins from classroom investigation. Several methods such as questionnaire, lesson report, teaching journal and audio/video recordings are widely used for classroom investigation. However, these methods are either susceptible to memory bias or are hard to be continued on a day-to-day basis. In this study, we proposed an approach for effective classroom investigation in second language education using activity coding in combination with data visualization technology. The proposed method consists of three stages. In the first stage, a smartphone application was used to record the activities that happen in a class following a slightly modified experience sampling method. In the second stage, the activities were quantified using our proposed 2-level activity coding scheme, and each activity was assigned a colour code. In the third stage, a data visualization tool D3.js was used to create heat maps of the classroom activities. We applied the proposed method to investigating the classroom activities in five English conversation lessons given by native-speaking teachers. The visual feedback led to the answering of some key questions that critical reflection aims to address, including teachers’ time management, lesson structure, and the characteristics of teacher-student interaction. Based on the results obtained, we highlighted the potential of the proposed approach for involving different sectors in second language education and pointed out the directions for future research.
Keywords:
second language education, data visualization, d3.js, experience sampling method, reflective teaching.
1
Introduction
Nurturing the skills of critical reflection has become the underpinnings of most second language teacher education program worldwide [1-2]. Previous research indicated that critical reflection makes a significant impact on a teacher’s knowledge, skills and
effectiveness [3-5]. Critical reflection helps teachers understand their teaching practices and enables them to look at the underlying principles and beliefs that define the way they teach [6]. Critical reflection begins in gathering information about what happens in the class. Keeping tracking and reflecting on classroom activities can help teachers develop deeper understanding of their teaching practices and strategies. A number of approaches have been employed for classroom investigation. Surveys and questionnaires collect information on a particular aspect of teaching but tell little about the actual teaching procedures [6]. Teachers’ diary records the routine and conscious actions in the classroom for later reflection but is susceptible to memory bias. Previous studies suggest that people are not good at reconstructing their experience after the fact and they cannot provide reliable assessment of complex dimensions of their experiences [7-8]. Peer observation and audio/video recording capture the details of classroom activities. However, these approaches generate large amount of qualitative data. Reviewing these data requires a lot of time and therefore can never become an activity continued on a day-to-day basis [9]. In this study, we proposed a novel approach for classroom investigation using activity coding and data visualization technology. The proposed approach enables the recording of classroom activities on the spot to address the problem of memory bias in existing approaches. Additionally, data visualization technology helps to generate visual feedback that leads to insights on the teaching procedures. The proposed method consists of three stages: data collection, data processing, and data visualization. In the first stage, a smartphone application named aTimeLogger [10] was used to record all activities that occur in a class following a slightly modified procedure of experience sampling method. In the second stage, a 2-level activity coding scheme that we proposed is used to map each recorded classroom activities to a corresponding color code. This stage is essential for converting qualitative description of classroom activity to quantitative color code. In the third stage, the sequence of color code generated in the previous stage is used as input to render a heat map [11] of the classroom activities using a JavaScript library named D3.js. Employing the proposed approach, we investigated five English conversation classes given by native-speaking teachers. We plotted heat maps of the classroom activities recorded in the five classes and calculated several metrics that characterizes the teaching procedures. The visualization and quantitative results helped us answer important questions in critical reflection such as “was the class teacher dominated?”, “how was teacher’s time management?”, “how did teacher and students interact?”. Acknowledging the limitations of the proposed approach, we highlighted the opportunities for future research. Note that we do not intend to replace existing approaches with the proposed one. Instead, we suggest that the proposed approach can be used in combination with existing approaches to create synergistic effect. This approach has many potential applications to involve different sectors in second language education. It empowers student teachers as well as in-service teachers to move beyond the level of routinized responses to classroom situations or to investigate the effect of changes in their teaching. Educational institutions can apply this approach in their teacher education pro-
grams. Education authorities can employ and extend this approach to quantitatively compare and evaluate teaching practices.
2
Related Work
Critical reflection on ones’ own teaching procedures help teachers achieve a better understanding of teaching and learning processes [6]. Therefore, reflective teaching has become a dominant paradigm in second language teacher education in recent years [1-2]. Reflective teaching commences when one investigates the activities that occur in class. A number of simple procedures are widely used for classroom investigation. Depending on when the information is collected, existing approaches for classroom investigation can be classified into two categories: after-event approaches and inevent approaches. Typical after-event approaches include surveys and questionnaires, lesson reports [12] and teaching journal (also called teachers’ diary) [13]. Typical inevent approaches include peer observation [14-15] and audio/video recording [6]. Surveys and questionnaires are often used at the end of a class or a semester to gain general impression on certain aspects of teaching. Though this approach has been widely used to obtain feedback from students, it collects little information on the actual teaching procedures and activities in class [6]. Alternatively, teachers can use a diary to describe their recollections of what happened during a lesson for later reflection [13]. However, this approach dominantly relies on teachers’ recalling on the activities occurred in class and thus are susceptible to memory bias. Previous studies suggest that people are not good at reconstructing their experience after the fact and they cannot provide reliable assessment of complex dimensions of their experiences [7-8]. In addition, this approach can capture only recollections and interpretations of the activities happened during a lesson but not the actual activities themselves. In-event approaches such as peer observation and audio/video recording solve the problems of memory bias and enable the collection of detailed information on classroom activities. These approaches also have problems. For one thing, peer observation generates mostly qualitative data which is hard to analyze quantitatively. Comparing the observation notes of different lessons also imposes great challenges to teachers. For another, reviewing an audio or video recording requires large amount of time and therefore can never become an activity continued daily given the busy schedule of teachers [9]. In our study, we employed activity coding in combination with data visualization technology to address the limitations of existing approaches. Classroom activities are recorded in real time to avoid memory bias, and the captured data is coded and then visualized in heat maps that render intuitive visual cues for insights and for further analysis. We will present the proposal in detail in Section 3.
3
Proposed Method
The proposed approach follows a procedure that resembles individual’s reflection on personal data [18]: data collection following experience sampling method, data processing for classroom activity coding, and data visualization of classroom activities using JavaScript library D3.js. 3.1
Data Collection: Recording Activities using Experience Sampling Method
Critical reflection begins in gathering information on what happen in class. Therefore, the first stage of the proposed approach is to record all the classroom activities to form an archival information on the teaching procedure in a lesson. To avoid the problem of memory bias in existing approaches, we adopted a modified procedure of the Experience Sampling Method (ESM) [16-17] to capture teaching as it is delivered. Given that it is not feasible for teachers to provide self-reports during their teaching, we created logs from a third person’s point of view following the procedures recommended in [19-21]. This process can be done either on paper or digitally. In this study, we used a mobile time tracking app named aTimeLogger to log all the activities happened during a class, including the teacher’s and the students’ activities and the time stamp of those activities. The aTimeLogger was originally developed to help individuals manage their daily time usage and to enhance productivity. It was designed to record different type of activities. We used the activity type “work” and “study” to record the activities of teachers and students respectively. Considering each activity as an event, we selected event-based sampling strategy [22] to record the classroom activities. An entry was created when a new activity started, and the time stamp was recorded as the start time of this activity. When this activity was completed, the observer simply tapped the stop button to record the end time. The aTimeLogger also allowed for writing comments for each event. Therefore the observer logged the content of the activities and students’ reactions in detail. When the lesson finished, the recorded data was exported into a CSV file for further processing. 3.2
Data Processing: Classroom Activity Coding
In the CSV files exported from aTimeLogger app, each entry comprised the following information of an activity: activity type, duration of the activity in minutes, start time, end time, and observer’s comments. All activities were listed in chronological order starting with the most recent one. A screenshot of an exported CSV file is shown in Figure 1.
Figure 1. A screenshot of the CSV file exported from aTimeLogger app. Each data entry comprised information of an activity occurred in the class, including activity type, time stamps and observer’s comments. The raw data was qualitative and thus was not suitable for further analysis. We converted the qualitative data to quantitative through activity coding. Activity coding starts with the creation of a list of initial codes of classroom activities such as “having group discussions”, “giving presentations”, “explaining new words and phrases” and “checking answers with the whole class”. The activities occurred in second language classes are highly repetitive and share common features across classes. A list of common activities used in second language teaching was presented in [23]. However, this list served for a more general purpose in language teaching, which covered reading, writing, listening and grammar. Most of the activities mainly targeted at young learners rather than university students. Therefore, the activity coding list in [23] was not suitable for our purpose of understanding conversation classes for adults (i.e. university students and staff) in particular, and we thus developed our own activity list which is presented in section 4. 3.3
Data Visualization: Creating Heat Map of Classroom Activities
After activity coding, the sequence of color code that corresponds to the whole set of activities occurred in a class was used to create a heat map using a JavaScript library D3.js [11]. A heat map of a class illustrates the sequence of classroom activities in chronological order using colored spectrum. The level-1 activity coding scheme maps the activities of students and teachers into red and blue segments on the colored spectrum, while the level-2 activity coding scheme further differentiate activities under each category using different shades of red or blue. Base on the heat maps, we also deducted several quantitative features to characterized teaching styles, including the Ratio of Students’ Activities (RSA), the Ratio of
Teachers’ Activities (RTA), and the Normalized Class Dynamics (NCD). The metrics were calculated using Equation (1) ~ (3).
𝑅𝑆𝐴 =
𝑅𝑇𝐴 = 𝑁𝐶𝐷 =
∑𝐼𝑖=1 𝑡𝑖𝑠𝑡𝑢𝑑𝑒𝑛𝑡 𝑇 ∑𝐽𝑗=1 𝑡𝑗𝑡𝑒𝑎𝑐ℎ𝑒𝑟 𝑇
𝑁𝑠𝑡𝑢𝑑𝑒𝑛𝑡↔𝑡𝑒𝑎𝑐ℎ𝑒𝑟 𝑇
(1)
(2) (3)
where 𝑡𝑖𝑠𝑡𝑢𝑑𝑒𝑛𝑡 and 𝑡𝑗𝑡𝑒𝑎𝑐ℎ𝑒𝑟 represent the duration of the i-th students’ activity and the duration of the j-th teachers’ activity in minutes respectively. T is the total duration of the whole class in minutes. I and J are the total amount of students’ and teachers’ activities. 𝑁𝑠𝑡𝑢𝑑𝑒𝑛𝑡↔𝑡𝑒𝑎𝑐ℎ𝑒𝑟 is the total number of switches between students’ and teachers’ activities.
4
Results
4.1
Data Collection and Processing
Using the proposed method, we collected and analyzed the classroom activities in five English conversation classes given by five native-speaking teachers. The first author presented at the classes and logged all the activities using aTimeLogger in a manner as unobtrusive as possible. The second and the third author also presented in part of the classes. The first author then discussed with the second and the fourth author to develop the activity coding list which is summarized in Table 1. In this study, we adopted a hybrid approach inspired by thematic analysis [24] to create the activity coding list. Firstly, we read through all the comments in the classroom observation logs to get familiar with the classroom activities occurred in the observed lessons. Second, we identified initial codes by repeatedly going through the comments. Routine activities such as “having group discussions”, “giving presentations”, “explaining tasks and taking questions”, “giving pronunciation drill”, “correcting grammar mistakes” were first extracted and then ranked according to their perceived educational benefits. Third, all the initial codes were assigned a numerical color code, i.e. red for students activities and blue for teachers activities. Main activities are marked bold in the list. An activity was considered as main activity if its total duration last more than 10% of the total class time in at least one of the five classes observed. In line with the coding scheme of the d3.js library, the color codes assigned to students’ activities were within [50, 100], and higher values corresponded to darker red. The color codes assigned to teachers’ activities were within [0, 50], and lower values corresponded to darker blue. The visualization results are presented in following sections.
Table 1: Activity Coding List.
Category (Level-1 Coding) Students’ Activities
Teachers’ Activities
Activity Name (Level-2 Coding) #Having group discussions with teacher’s interaction #Having pair discussions with teacher’s interaction #Having group discussions without teacher’s interaction #Having pair discussion without teacher’s interaction #Having role plays or playing a game #Asking questions to teacher #Asking questions to presenter #Asking questions to other students #Having casual chat with teacher #Comparing answers in group #Comparing answers in pair #Giving a presentation in group #Giving a presentation alone #Brainstorming alone #Reading out #Listening to an audio and answering questions #Doing a quiz or an exercise #Preparing for the classes #Having a break #Checking attendance #Assigning homework #Explaining tasks and taking questions #Explaining new words and phrases #Explaining grammar points #Wrapping up a discussion #Checking answers with the whole class #Asking questions to students individually #Sharing techniques and tips #Correcting grammar mistakes #Giving comments to the whole class #Asking follow-up questions #Sharing personal/cultural experiences with students
Color Code 100 97.5 95 92.5 90 85 82.5 80 77.5 75 72.5 70 67.5 65 62.5 60 57.5 55 50 49 48 44 40 36 32 28 24 20 16 12 4 0
4.2
Heat Map Visualization of Classroom Activities
Heat maps of the five classes are shown in Figure 2. The color and length of a segment indicate the type and length of an activity. Red segments represent student activities such as having group discussions, giving presentations, comparing answers. Blue segments represent teacher activities such as explaining new words and phrases, explaining tasks, giving learning tips. As shown in Figure 2, the class of Teacher_3 had more blue segments than red ones, indicating that the teacher was dominating the class and the students were not given sufficient time to talk. As a contrast, student activities consumed more time than teacher activities in the classes of Teacher_4 and Teacher_5. Especially, the class of Teacher_5 manifested good rhythm as the activities of students and teachers were almost uniformly distributed. This matches our observation that Teacher_5 had very good time management in his classes.
Time elapsed (s)
Time elapsed (min)
Figure 2: Heat maps of the observed classes.
The Ratio of Students’ Activities (RSA) and the Ratio of Teachers’ Activities (RTA) are plotted in Figure 3. Given the objectives of the classes, it is expected that the teachers should not dominate the class and instead should give sufficient time for students to practice speaking. Figure 3 shows that the RSA of the five observed classes ranged from as low as 27.3% in the class of Teacher_3 to as high as 70.8% in the class of Teacher_4. The Normalized Class Dynamics (NCD) is plotted in Figure 4. The class of Teacher_3 peaked in the plot with one activity switch between students and the teacher every 3 minutes. The NCD of Teacher_5 was the lowest with approximately one switch every 20 minutes. Classes of the other teachers have NCD in between. Visualizing the nominal coding helped answer some of the key questions that critical reflection aims to address [6]. 1. How did the teacher structure the lesson? Different teachers have their own ways of structuring a class. For example, Teacher_2 started her lesson with a quiz exercise, whereas other teachers started their lessons by explaining the tasks and tak-
ing questions from students. In addition, Teacher_2 integrated listening practice into the conversation class. Ratio of Students' and Teachers' Activities 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%
Teachers' Activities
64.1%
70.8%
68.5%
Students' Activities
51.2% 27.3%
Figure 3: The ratio of students’ and teachers’ activities in five classes.
0.4
Normalized Class Dynamics
0.3 0.2 0.1
0 Teacher_1 Teacher_2 Teacher_3 Teacher_4 Teacher_5
Figure 4: Normalized frequency of switch between students’ and teachers’ activities.
2. What kind of teacher-student interaction occurred? The visualization of level-2 coding provided fine-grained information on the context in which teacher-student interaction happened. Teacher_2, Teacher_3 and Teacher_5 all interacted with students during pair discussions, whereas Teacher_1 and Teacher_4 observed students’ group discussions but did not have verbal interaction with them. The pattern of teacher-student interaction in Teacher_3’s class was characterized by teacher sharing personal experience and asking follow-up questions after the students summarized their group discussion. Indeed, our observation suggested that Teacher_3 asked many follow-up questions to help the students further develop their discussions. In four out of the five classes, teachers interacted with students by comparing answers with the whole class. 3. What are the key activities? Table 2 summarizes the top 3 activities in each class according to the total amount of time. The total amount of time spent on the top 3
activities occupied at least 40% of the total class time. Students having group discussion without teachers interaction was the dominant activity in the class of Teacher_1, Teacher_4 and Teacher_5. Teacher_2 allocated abundant time for students to practice listening, while Teacher_3 spent much time asking follow-up questions to students. Table 2: Top 3 Activities in Each Class
Instructor Teacher_1
Teacher_2
Teacher_3
Teacher_4
Teacher_5
Top 3 Activities 1. Students having group discussions without teacher’s interaction 2. Teacher asking questions and students answer one by one. 3. Students reading out 1. Students listening to an audio and answer questions. 2. Teacher checking answers with the whole class 3. Teacher explaining tasks and took questions 1. Teacher asking follow-up questions 2. Teacher sharing personal/cultural experience with students 3. Teacher checking answers with the whole class 1. Students having group discussions without teacher’s interaction 2. Students having role play/game 3. Students comparing answers in group 1. Students having group discussions without teacher’s interaction 2. Students giving a presentation alone 3. Teacher checking answers with the whole class
Ratio 23.1% 16.7% 12.8% 30.5% 18.3% 11.0% 21.6% 12.5% 9.1% 36.2% 9.2% 9.2% 25.0% 21.7% 15.2%
4. How did the classes differ from one another? To investigate inter-class difference as well as commonness, we plotted the main activities in Figure 5. An activity was considered as main activity if the total duration of this activity last more than 10% of the total class time in at least one of the five classes observed. Figure 5 shows that asking follow-up questions (21.6%) and sharing personal/cultural experience with students (12.5%) seemed to be the favorite of Teacher_3, which occupied approximately one third of the total class time. Teacher_2 spent 30% of the total class time doing listening practice and 18% of the time checking answers with the whole class, with only a small portion of time allocated to group or pair discussions. Teacher_5 sometimes interacted with the students during their discussions, whereas such interaction was absent in the classes of Teacher_1 and Teacher_4.
100% 90%
15.2%
80% 70% 60% 50%
16.7%
18.3%
30%
12.8%
20% 10%
21.7%
21.6%
40%
23.1%
30.5%
36.2%
25.0%
0%
Teacher_1 Teacher_2 Teacher_3 Teacher_4 Teacher_5 Others Teacher share personal/cultural experience with students Teacher asks follow-up questions Teacher ask questions and students answer one by one. Teacher checks answers/homework with the whole class Teacher explain new words and phrases Teacher explain tasks and took questions Students listen to an audio and answer questions. Students read out Students gave a presentation alone Students had group discussions without teacher’s interaction Students had pair discussions and teacher interacted Figure 5: Comparison of the ratio of main activities in each class
5
Discussions and Potential Applications
5.1
Principal Findings
We have shown that the heat maps of classroom activities provided useful visual feedback on teaching procedures and practices. The visualization of level-1 coding led to the answers to questions such as “Was the lesson teacher dominated?” “How often did teacher-student interaction occur?” “How was teacher’ time management?”. The visualization of level-2 coding rendered fine-grained information to help answer questions such as “How did the teacher structure the lesson?” “What kind of teacherstudent interaction occurred?” We found that group discussion was a commonly used strategy by all teachers to help students develop conversation skills. The visual cues deepened our understanding on how the classes differed from one teacher to another. Through analyzing and visualizing the lessons, we achieved a better understanding on the strong and weak points of each teacher. For example, the class of Teacher_3 was featured by quick pace switches between students and teach-
ers activities. Indeed, Teacher_3 frequently interrupted the students to ask follow-up questions during their group discussions. Though sharing personal and cultural experience with students was no doubt very beneficial for students’ English learning. Given that many Japanese students were shy and were not used to the western-style communication, however, it was likely that some students might feel overwhelmed by the dynamics of the class. This is supported by our observation that some students felt lost when Teacher_3 suddenly changed to a random new topic in the course of their discussions. As a contrast, the class of Teacher_4 had a steady pace, and he allocated abundant time for group discussions. However, our observations revealed that the students were usually left unattended in the second half of the class. They were asked to discuss on a certain topic, but the teacher didn’t interact with the students. Teacher_2 differentiated her class from others’ by integrating listening practice into conversation class. This strategy was supported by previous studies that aural and oral skills are close related [25-26] and that enhancing listening could support the improvement on speaking in English learning [27]. Yet, we noticed that students spent more time doing exercise or individual tasks rather than practicing conversation skills in pairs or groups. Visualizing classroom activities may also uncover interesting phenomenon that are otherwise not easy to perceive, which may eventually lead to new hypothesis in second language teaching. For example, the class of Teacher_5 demonstrated less dynamics and less frequent activity switches in comparison with other classes. However, all students of this class acknowledged that his teaching style was very impressive, and his classes had good rhythm. This challenges our stereotype that second language teachers should be entertaining and should bring fun to classes. Instead, we noticed that Teacher_5’s class had the traits that all activities were almost uniformly distributed. There was a clear goal for each of the students’ activity and the students were given sufficient time to practice English speaking in each activity. Based on this observation, we hypothesized that the rhythm of a class may be an important metric that impacts learning outcomes in second language teaching. The proposed approach can also be employed to compare different classes given by the same teacher. If a teacher keeps tracking his/her classes to collect longitude data spreading over months or even years, he/her could gain insightful information on other key questions in critical reflection such as “Did I do anything differently than usual in my class?”, “Did the change make things better or worse?”, “what else can I change to make my teaching better?” The heat maps of classroom activity rendered rich information that helped us gain insights on many important aspects of teaching in English conversation classes. However, we also found that the current design failed to manifest how students performed and reacted in each activity. The 2-level activity coding scheme only focused on classroom activities per se and provided little information on the performance and cognitive response of the students. The impact of these classroom activities on students’ learning process therefore remains unknown. For example, Teacher_4 and Teacher_5 allocated almost the same ratio of class time for students to discuss in groups, yet the effect differed. According to our observations, the students of Teacher_5 made full use of the time and actively participated in the discussions, whereas
the students of Teacher_4 were very shy and were not actually on the task for full time. Future research is needed to address this problem. 5.2
Limitations
In addition to lacking focus on students’ performance as explained above, the current study has several limitations presented below. First, capturing data on the activities happened during a class was done in a semi-automatic way. Whereas using smartphone applications significantly reduced the burden of data collection compared to paper-based approach, a complete automatic data collection scheme will further improve the scalability of the proposed method. Second, the current coding list only captures activities that are accompanied with verbal exchanges. Non-verbal activities such as teachers wandering around and observing group discussion were not considered in creating the activity coding list. These non-verbal activities may have educational benefit, but it is not possible to accommodate these activities following the existing line of how we created the activity coding list. A promising solution is to model English teaching using structured ontology engineering [28-29]. Furthermore, we only used activity logs for analysis and visualization. Although activity logs provided an archival report on what happened in a class, the effect and impact of these activities remained unknown. Therefore, it will be preferable to include multi-modal data from several sources ranging from the evaluation on students’ improvement, students’ evaluation of their teachers, to bio-metrics such as heart rate and cognitive load during study and teaching [30-31]. Third, the activity coding list that we extracted from the five observed classes did not exhaustively cover all possible classroom activities. This list should be extended as more classes are observed in the future. Last but not the least, it is important to investigate the impact of such visualization on teaching practices. The questions of interest include what changes have been trigger in teaching strategies and procedures and if such changes can eventually lead to positive outcomes in teaching and learning in second language classes. Future researches should focus on addressing these limitations. 5.3
Potential Applications
Classroom observation for critical reflection have immediate practical benefits for individual teachers as well as longer term benefits to second language teacher development programs. The method proposed in this study enables a lower-burden way for recording and analyzing classroom activities and can be employed in different sectors of second language education to support critical reflection. The proposed approach can help experienced teachers to move beyond the level of automatic or routinized responses to classroom situations and to investigate the effect of changes in their teaching strategies and procedures. New student teachers can use this approach to reflect their teaching practices to achieve a higher level of awareness of how they teach. Educational institutions can use this method in their training programs to foster critical reflection in student teachers. Education authorities can employ and extend this method to quantitatively compare and assess teachers’ classroom management
and teaching procedures. In the next step, we plan to conduct a large-scale study using the proposed approach to investigate how the visualization of classroom activities facilitates teachers’ reflection and teaching practices.
6
Conclusions
We have proposed a novel approach for low-burden classroom activity investigation based on experience sampling method and data visualization technology. We applied the proposed method to record and visualize the classroom activities of five English conversation classes given by native speakers. The visualization provided rich information on teachers’ time management, class structure, and teacher-student interaction patterns, which led to the answering of some important questions that critical reflection aims to address. We highlighted four limitations of the current study to guide future research on this topic. The proposed method has the potential for large-scale implementation in educational institutions and will benefit both teachers and students through facilitating critical reflection on teaching and learning.
References 1. M. Clark and D. Otaky. 2006. Reflection “on” and “in” teacher education in the United Arab Emirates. International Journal of Educational Development 26, 1:111-122. 2. P. Sze. 1999. Reflective teaching in second language teacher education: an overview. Educational Research Journal 14, 1:131-155. 3. P. Willis. 1999. Looking for what its’ really like: Phenomenology in reflective practice. Studies in Continuing Education 21, 1:91-112. 4. A. Harris. 1998. Effective teaching: a review of the literature. School Leadership & Management 18, 2:169-183. 5. P. Cranton. 1996. Professional Development as Transformative Learning: New perspectives for Teachers of Adults. San Francisco: Jossey Bass. 6. J. C. Richard and C. Lockhart. 2013. Reflective Teaching in Second Language Classrooms. Cambridge University Press, New York, USA. 7. D. Yarmey. 1979. The Psychology of Eyewitness Testimony. New York: Free Press. 8. D. Fiske. 1971. Measuring the concept of personality. Chicago: Aldine. 9. M. Schratz. 1992. Researching while teaching: an action research in higher education. Studies in Higher Education 17, 1:81-95. 10. http://www.atimelogger.com/ 11. Leland Wilkinson and Michael Friendly. 2009. The History of the Cluster Heat Map. The American Statistician 63 (2): 179–184. doi:10.1198/tas.2009.0033. 12. J. Pak. 1986. Find Out How You Teach. Adelaide, Australia: National Curriculum Resource Centre. 13. L. Bartlett. 1990. Teacher development through reflective teaching. In J. C. Richards and D. Nunan (eds.), Second Language Teacher Education. New York: Cambridge University Press. Pp. 202-14. 14. J. M. Murphy. 1991. An etiquette for t Edward he non-supervisory observation of L2 classrooms. In Proceedings of the 1st International Conference on Teacher Education, City Polytechnic of Hong Kong.
15. J. C. Richards and C. Lockhart. 1991-1992. Teacher development through peer observation. TESOL Journal 1, 2: 7-10. 16. Joel M Hektner, Jennifer A Schmidt and Mihaly Csikszentmihalyi. Experience Sampling Method: Measuring the Quality of Everyday Life. Sage, 2007. 17. Reed W. Larson and Mihaly Csikszentmihalyi. 1983. The experience sampling method. New Directions for Methodology of Social and Behavioral Science, 15, 41-56. 18. Z. Liang and M. A. Chapa Martell. Framing Self-quantification for Individual-level Preventive Health Care. In Proceedings of HEALTHINF 2015: 336-343. 19. M. Pennington and A. Young. 1989. Approaches to faculty evaluation for ESL. TESOL Quarterly, 23, 4:619-646. 20. T. Good and F. Brophy. 1987. Looking in Classrooms. New York: Harper and Row. 21. D. Freeman. 1982. Observing teachers: three approaches to in-service training and development. TESOL Quarterly, 16, 1:21-28. 22. L. Wheeler and H. T. Reis. 1991.Self-recording of everyday life events: origins, types, and uses. Journal of Personality 59, 3:339-354. 23. C, Edwards, T. Shortall, D. Willits, et al. 2000. Language Teaching Methodology. Birmingham: Center for English Language Studies. 24. J. Fereday, E. Muir-Cochrane, E. 2006. Demonstrating rigor using thematic analysis: a hybrid approach of inductive and deductive coding and theme development. International Journal of Qualitative Methods 5, 1:80-92. 25. L. Vandergrift. 2007. Recent development in second and foreign language listening comprehension research. Language Teaching 40, 191-210. 26. H. Bozorgian. 2012. The relationship between listening and other language skills in international English language testing system. Theory and Practice in Language Studies 2, 4: 657-663. 27. E. D. Astorga-Cabezas. 2015. The relationship between listening proficiency and speaking improvement in higher education: considerations in assessing speaking and listening. Higher Learning Research Communications 5, 2: 34-56. 28. R. Mizoguchi, J. Bourdeau. 2000. Using ontological engineering to overcome common AIED problem. Journal of Artificial Intelligence and Education 11, 107-121. 29. K. Munn, B. Smith. (Eds). 2008. Applied Ontology: an introduction. Volume 9. Walter de Gruyter. 30. C. Mills, I. Fridman, W. Soussou, et al. 2017. Put your thinking cap on: detecting cognitive load using EEG during learning. In Proc. of LAK 2017. 31. Di D. Mitri, M. Scheffel, H. Drachsler, et al. 2017. Learning Pulse: a machine learning approach for predicting performance in self-regularted learning using multimodal data. In Proc. of LAK 2017.
