Lewis & Harrap, The Development of a Problem-Based Learning and Teaching Strategy in an Aviation-Related Project at the Australian Defence Force Academy
The Development of a Problem-Based Learning and Teaching Strategy in an Aviation-Related Project at the Australian Defence Force Academy
Mr Raymond Lewis UNSW@ADFA, Canberra, Australia
[email protected] Dr Michael Harrap UNSW@ADFA, Canberra, Australia
[email protected] Abstract: Compared to the seven other Australian Universities that offer an Aviation degree, the aviation-related project is a major component of the Bachelor of Technology (Aviation) degree program curriculum at the University of New South Wales at the Australian Defence Force Academy. This paper describes the aviation-related project as a case study in the strategy of problem-based learning and teaching. Since the inception of the degree the project has evolved and expanded. This process has placed greater expectations on the learning outcomes of students which some students have had difficulty in fulfilling. Coursework has been introduced to ameliorate some of the perceived shortcomings of the aviation-related problem-based learning project. This paper describes an evaluation of the aviation project based on student learning and student self-assessment.
Introduction In 2001, in response to a statement of requirement issued by the Royal Australian Air Force Commander of Training, the Bachelor of Technology in Aviation [B’Tech(Av)] degree program was established at the University College at the Australian Defence Force Academy (ADFA). The University College at ADFA is a college of the University of New South Wales and has provided undergraduate and postgraduate degree programs for the Australian Defence Force (ADF) since ADFA’s commencement in 1986. The B’Tech(Av) degree program statement of requirement, communicated to the University College, specified that the academic program should be of two years duration and conducted at the University of New South Wales at the Australian Defence Force Academy (UNSW@ADFA). This academic program was to be followed by approximately eighteen months of flying training: the basic flying training course, conducted at Tamworth NSW by British Aerospace Australia; the advanced flying training course, conducted at Pearce WA by Royal Australian Air Force (RAAF) personnel. The feature of the separation of an academic program and a practical flying program makes the B’Tech(Av) degree program at UNSW@ADFA unique in Australia. There are seven Australian Universities that conduct an Aviation degree program where flying training is a component of the academic accreditation. However, all of these programs integrate the practical flying training with the academic program. The development of the B’Tech(Av) program at UNSW@ADFA, as reported by Harrap, Burdekin and Lewis (2007) mentioned that since 2001, the degree program has been twice reviewed by the ADF, (Given, 2002, 2003). Both reviews resulted in the program receiving a favourable review. In 2006, the B’Tech(Av) program was granted full Engineers Australia accreditation. This paper will discuss one of the features of the program – the B’Tech Aviation Project. This ‘capstone’ project has been briefly described in the Harrap et al (2007) paper. In a comparison of the curriculum content of the seven tertiary aviation degree programs in Australia, the UNSW@ADFA
Proceedings of the 2008 AaeE Conference, Yeppoon, Copyright © Lewis & Harrap, 2008 1
Lewis & Harrap, The Development of a Problem-Based Learning and Teaching Strategy in an Aviation-Related Project at the Australian Defence Force Academy
aviation-related project is a significant component of the program in terms of percentage of total program credit, (Harrap et al, 2007). The rationale behind this project and the evolutionary nature of its development as the ‘jewel in the crown’ of the B’Tech (Av) program will be described in terms of the value inherent in a problem-based learning and teaching paradigm. Finally, examples of student projects are given and student feedback is discussed as a metric of evaluating the learning objectives of the program.
Problem Based Learning Problem-based learning is a teaching and learning strategy where significant, contextualised, realworld situations and problems are posed and resources, guidance and instructions are offered to learners as they develop content knowledge and problem solving skill sets, (Jones, 1996). Problembased learning, as a pedagogical strategy can adopt different guides; model-centred instruction; situated learning; case-based learning; task-centred instructional strategy and so on, Merrill (2007). Fundamentally, problem-based learning is far removed from the situation where the teacher is a lecturer, delivering facts and rules for student assimilation; problem-based learning is a learning and teaching strategy designed to achieve optimum student learning outcomes and engagement. Problem-based learning usually involves students working in collaborative groups in order to solve a complex problem that does not have a single correct answer (Hmelo-Silver, 2004). In the case of the problem-based learning conducted by the BTech(Av) students, each student adopts a problem that is either “seeded” by a supervisor or is devised by the student. Capstone projects are common in engineering programs of education. One of the problems of the aviation-related project as a “capstone” course is that it occurs relatively early in the BTech(Av) program. Usually, a similar project is conducted at thesis level by fourth-year engineering students. Because of the requirement that the academic program be conducted before flying training, the students begin their aviation-related project mid-way during their second year at the Academy. In responding to the ADF statement of requirement and in constructing a program of education that fulfilled the Engineers Australia graduate attributes, the aviation-related project was included in the BTech(AV) program in order to engage the students in problematic aspects of their chosen field of endeavour. It is germane, at this juncture, to comment on the perceived need for “engagement” by the students enrolled in a B’Tech (Av) degree at UNSW@ADFA. Firstly, the academic program competes against concurrent military studies and duties as well as the aspirations of the students to become pilots. Their civilian counterparts on civilian campuses, where academic programs and flying training programs are intertwined, have their flying aspirations satisfied whilst being involved in academic pursuits. The BTech(Av) students at UNSW@ADFA are highly motivated to become pilots in the ADF and often display an impatience with the academic program, especially as the day of leaving the Academy to commence flying training approaches. These observations of BTech(Av) students regarding their behaviour and motives and their “surface approach” to learning are anecdotal but are supported by several colleagues at the Academy. The “surface approach” to learning is described by Biggs (1999a) who comments on the seminal work in student learning carried out by Marton and Säljö in Sweden. Marton and Säljö (1976a,b cited in Biggs, 1999a) developed the concept of surface and deep approaches to learning. In response to a given piece of text to read and the promise of later questions, the surface learner skimmed the surface of the text and remembered a list of disjointed facts; they did not comprehend the substance of the text. The deep learner read the text to interpret its meaning in order to understand the author’s perspective, (Marton and Säljö 1976a, b cited in Biggs 1999a). Biggs (1999a) takes a constructivist perspective of the Marton and Säljö (1976a, b) work and postulates “a two-way interaction between the degree of learning-related activity that a teaching method is likely to stimulate and the academic orientation of the students” (p. 58). Biggs (1999a) describes an academic student Susan who arrives at the lecture armed with relevant background information and questions to which she seeks answers. She reflects upon the significance of what she is learning – in short, Susan is the deep learner according to the Marton and Säljö (1976a, b cited in
Proceedings of the 2008 AaeE Conference, Yeppoon, Copyright © Lewis & Harrap, 2008 2
Lewis & Harrap, The Development of a Problem-Based Learning and Teaching Strategy in an Aviation-Related Project at the Australian Defence Force Academy
Biggs 1999a, b) definition. Susan is engaged by her studies independently of how they are taught. Robert, on the other hand, in being defined as a ‘non-academic’ surface learner, does not do as well in a passive learning and teaching environment such as the standard lecture. The difference in the level of the two students’ engagement, in a passive learning and teaching environment, is evident in figure one, taken from Biggs (1999b). However, as the teaching method becomes more “active” such as in a problem-based learning scenario, the “engagement” level of non-academic Robert approaches that of academic Susan.
Figure 1: Student orientation, teaching method and level of engagement, From Biggs (1999 p59). It has been argued that the Biggs (1999a) matrix of level of student engagement versus teaching method, (passive → active), is too simplistic in that not all Robert-type students are capable of responding to a problem-based learning strategy. Recent experiences of the aviation-related project at UNSW@ADFA indicate that some students need special nurturing and sometimes an extension of time to complete their work. Merrill (2007) makes the point that some reviewers of problem-based learning paradigms claim that “approaches that provide minimum guidance are often ineffective and inefficient (Kirschner, Sweller & Clark, 2006; Mayer, 2004 cited in Merrill, 2007 p. 3).
The B’Tech Aviation Project When the B’Tech (Av) program commenced in 2001, the program was designed so that the aviationrelated project would be accomplished when the students were at Tamworth undergoing their basic flying training. However, for a number of reasons this arrangement did not eventuate and it was necessary to repatriate the Aviation Project to ADFA. The Project as originally conceived, was to be six UoC, where under the tutelage of an individual supervisor the student would study and (hopefully) expand on a body of knowledge related to an aspect of aviation. In 2005, after a review of the School of ACME’s curriculum, the aviation project was expanded to nine UoC. Three UoC were to be assigned to the session or semester prior to the summer school and six UoC to be completed over the summer period, prior to the commencement of flying training. Accordingly, the project was expanded to give greater emphasis to the research component of the project. Recently the Avaition Project has been further refined in response to the following three factors: 1. In early 2007, a lecturer on exchange from the United States Air Force Academy participated in the marking of the submitted aviation project report. Millard (2007) offered a critique of the students’ reports which drew attention to a poor level of student understanding of any statistical analysis contained in the reports. Also, Millard (2007) reported that there were weaknesses in the student written discussions of methodological issues.
Proceedings of the 2008 AaeE Conference, Yeppoon, Copyright © Lewis & Harrap, 2008 3
Lewis & Harrap, The Development of a Problem-Based Learning and Teaching Strategy in an Aviation-Related Project at the Australian Defence Force Academy
2. One of the problems in carrying out a problem-based learning exercise in a school that has four disciplines – aviation, aeronautical, mechanical and civil engineering is that supervisors from different disciplines may have diverse methodologies and procedures. 3. In 2006 Engineers Australia conducted a review of the engineering programs carried out at the School of ACME. One of the suggested improvements to the degree programs was to incorporate greater student reflection into the programs. It was suggested that this student reflection could be achieved by having students complete learning journals. In response to these factors and the University management’s call for “greater teaching efficiency”, a more structured approach to the teaching of the aviation project was adopted. In 2007 a course was designed as an introduction to the aviation-related project. This aim of this course was to provide the student with a toolbox of skills which would equip him or her to carry out the project and to ameliorate those situations where the student became very reliant on the interventions of the respective supervisor. In the Merrill (2007) context of a task-centred instructional strategy, the initial presummer school aviation project course became a ‘form of direct instruction in the context of real world problems’, (p.3). The students were guided through modules, such as: applying to the University and Defence organisations for ethical approval to conduct trials, questionnaires and psychometric measurement experiments; basic inferential statistics and providing templates with regard to procedure and methodology. In response the the Engineer’s Australia suggestion to improve reflective learning, students are now required to a create a learning journal. The effectiveness of this will be evaluated at the conclusion of the project. Finally, during the coursework component of their project, students are required to make made a short presentation to their peers on their particular project. Students were asked to include in their presentations a brief description of their variables, both dependant and independent and how they proposed to measure these variables. Carried out over four weeks, these seminar-type sessions sparked a level of peer-peer; peer-teacher interaction that conforms to the relationships among students described by Webb (1982) where giving and receiving explanations are positively related to achievement. These presentations have proven to be most productive.
Current Experiences The current class of B’Tech(Av) students, who commenced the coursework component of the aviation-related project in July 2008, discussed the learning and teaching implications of the Biggs (1999a) paper in a classroom tutorial. As part of the first entry into their learning journals, the students were asked to write a brief description of their own learning style and relate that style to the Biggs (1999a) matrix of level of student engagement versus teaching method, (passive → active) as shown in figure 1. That is, the students were asked to self-assess their level of engagement thus far in the BTech(Av) program. This journal entry was later read by the lecturer who rated the student level of engagement on a Likert scale so that a score of 5 corresponded to “high level engagement” and a score of 1 corresponded to “low level engagement” At the commencement of the aviation-related project 75% of students scored less than 3 on the abovementioned Likert scale. At the conclusion of the project the students will again be asked to repeat the exercise of writing a brief description of their own learning style. The scores at the conclusion of the aviation-related program will be compared to the scores at the commencement of the program to determine if there is a significant difference in student self-assessed level of engagement.
Examples of Aviation-Related Project Problem-Based Learning As described in Harrap et al (2007), one of the advantages of the blending of engineering and aviation coursework is that this allows students to undertake research in a wide variety of topics, albeit related to an aviation activity. However, one of the disadvantages of this blend of coursework is that some of the aviation-specific coursework, such as aviation human factors and aviation safety – courses which contain social science and behavioural science issues, must compete for time on the curriculum with the engineering specific courses such as statics, dynamics and structures. It can be argued that more of
Proceedings of the 2008 AaeE Conference, Yeppoon, Copyright © Lewis & Harrap, 2008 4
Lewis & Harrap, The Development of a Problem-Based Learning and Teaching Strategy in an Aviation-Related Project at the Australian Defence Force Academy
the B’Tech (Av) curriculum should be devoted to behavioural science. This is because as BTech(Av) graduates the students will eventually serve as pilots, squadron safety specialists and crew resource management facilitators. A BTech(Av) graduate will never be allowed to perform any purely engineering function or contribute to an aircraft engineering process, (a pilot may not even change a sparkplug on an aero engine). This perspective on the evolution of the BTech(Av) degree program structure is elucidated in Lewis and Harrap (2008). Notwithstanding the preceding comments, one such recent aviation-related project concerned the use and pilot knowledge of major airport surface movement area guidance signs (MAGS). These signs are now mandated by the International Civil Aviation Organisation (ICAO) for capital city-type airports. The misinterpretation of these signs is one of the causal factors of unauthorised runway incursions. An unauthorised runway incursion is the entering of an active runway, without the proper air traffic control permission. Such occurrences have caused numerous aircraft accidents and incidents. A study at Newcastle University had demonstrated poor pilot knowledge of pictures of MAGS, (Carrick & Nicholas, 2003). In an attempt to test pilot knowledge in a more realistic scenario, the student and supervisor taxied the School aircraft over the surface movement areas of Canberra airport and photographed all the MAGS. After obtaining approval form the UNSW@ADFA ethics committee, the student used current B’Tech(Av) students as subjects and tested their knowledge of a pictorial presentation of the MAGS. These pictures were presented on a computer in three predetermined sequences. For example, one such sequence followed a path from the VIP parking area at Fairbairn RAAF Base across two runways, via the associated taxiways to the general aviation park. The student found (in this study) that novice pilots displayed a very good knowledge of MAGS when presented in a more realistic setting and sequence. The problem of inadvertent and unauthorised runway and taxiway incursions is one that the student will encounter during his or her career as a pilot. Therefore the problem of pilot education and knowledge of these signs is very relevant to the student. The student had to investigate the psychology of signage in general and the placement and interpretation of the signs on an airport in particular. The student actively investigated a problem domain and designed and conducted an experiment on a student cohort. The findings were ultimately reported at an Australian Aviation Psychology Association conference and the student, of course, received full acknowledgement. Another example of the problem-based learning and teaching strategy was an investigation into the turbulence created by buildings on a Canberra airport approach path. The owner of the now privatised airport had built office blocks adjacent to the threshold of the main north-south runway. The prevailing north-westerly winds pass over these buildings and could possibly cause turbulence for aircraft on final approach to land. The student built a scale model of the airport and the buildings in question. He then used a wind tunnel and laser Doppler velocimetry to measure the runway turbulence, (Nelson,2004). Winds and weather that effect the safe navigation of aircraft are of vital concern to pilots. Not only was the project highly relevant to the student’s aviation career but the quantifying of the turbulence effect impressed upon the student the social significance of such investigations. Significantly, the Civil Aviation Safety Authority (CASA) has recently requested the results of the student’s findings. These two examples of the aviation-related project as a problem-based learning strategy conform to the notion of (Jones, 1996) who maintains that significant, contextualised, real-world situations and problems are posed and resources, guidance and instructions are offered to learners as they develop content knowledge and problem solving skill sets.
Conclusion At present, the School of ACME is undergoing another program and curriculum review. In late 2007, another academic joined the aviation team in the School of ACME. This person was recruited because of his behavioural science background as, at the time of recruitment, this was seen to be the direction of the B’Tech (Av) degree. Whether the B’Tech (Av) degree program will evolve into a course of education that will produce a human factors specialist or will remain a quasi-engineering degree remains to be seen (Lewis and Harrap, 2008). Either way, the aviation-related problem-based learning
Proceedings of the 2008 AaeE Conference, Yeppoon, Copyright © Lewis & Harrap, 2008 5
Lewis & Harrap, The Development of a Problem-Based Learning and Teaching Strategy in an Aviation-Related Project at the Australian Defence Force Academy
instructional strategy will remain a cornerstone of the UNSW@ADFA B’Tech (Av) program learning and teaching outcomes.
References Biggs, J. (1999a). Teaching for quality learning. Buckingham: Open University Press. Biggs, J. (1999b). What the student does: teaching for enhanced learning. Higher Education Research and Development, 18(1), 57-75. Carrick, K. & Nicholas, C. (2003). Poster: Understanding of Airport Movement Signs. Newcastle: University of Newcastle. Given, K. C. (2002). Report on the Review of the Bachelor of Technology (Aviation) Degree. DGPERS - AF. Given, K. C. (2003). Update on the Bachelor of Technology (Aviation) Program. DGPERS – AF. Harrap, M. J., Burdekin, S. & Lewis, R. C. (2007). The development of the Bachelor of Technology in Aviation degree program at the Australian Defence Force Academy. Proceedings of the 2007 AaeE Conference, Melbourne (pp Hmelo-Silver, C. E. (2004). Problem-Based Learning: What and Hoe Do Students Learn? Educational Psychology Review, 16(3) 235-266 Jones, D. (1996). What is PBL? Accessed at http://edweb.sdsu.edu/clrit/learningresource/PBL/WhatisPBL.html on 28 July 2008. Lewis, R. C.& Harrap, M. J.(2008). Increasing Aviation Psychology Curriculum Content in an Aviation Degree at the Australian Defence Force Academy. Proceedings of 28th European Association for Aviation Psychology Conference, Valencia. Lewis, R. C. & Barclay, D. S. (2007). The effectiveness of airport movement area guidance signs. Proceedings of the 8th International Australian Aviation Psychology Symposium, Sydney. Merrill, D. (2007). A task-centred instructional strategy. Journal for Research on Technology in Education, 40(1), 33-50. Millard, R. (2007). United States Air Force Academy. Personal communication. June 2007. Nelson, D. (2004). Investigation of Building Induced Turbulence Over Runway 35 at Canberra Airport. B’Tech Av Project Report, SACME, ADFA, March 2004. Webb, N. M. (1982). Peer interaction and learning in cooperative small groups. Journal of Educational Psychology, 74(5) 642-655.
Copyright © 2008 Lewis & Harrap: The authors assign to AaeE and educational non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The authors also grant a non-exclusive licence to AaeE to publish this document in full on the World Wide Web (prime sites and mirrors) on CD-ROM and in printed form within the AaeE 2008 conference proceedings. Any other usage is prohibited without the express permission of the authors.
Proceedings of the 2008 AaeE Conference, Yeppoon, Copyright © Lewis & Harrap, 2008 6
