of differing types of programs at building key sustainability competencies defined ... integrate practice-based didactic approaches for building skills and methods via .... MacDonald and ... The study targets Master of Science (MSc) degree programs ... containing unclear answers, respondents were contacted for clarification.
Evaluating core competencies development in sustainability and environmental master’s programs: An empirical analysis Forthcoming in Journal of Cleaner Production https://doi.org/10.1016/j.jclepro.2018.01.164 Unedited Gregory Trencher1, Shirley Vincent2, Kyle Bahr3, Shogo Kudo4, Kate Markham5,6, Yasuhiro Yamanaka7 1. Graduate School of Environmental Studies, Tohoku University, Japan 2. National Council for Science and the Environment, Washington DC, USA 3. Graduate School of Environmental Studies, Tohoku University, Japan 4. Graduate School of Frontier Sciences, University of Tokyo, Japan 5. Human-Environment Systems Research Center, Boise State University, USA 6. Ecology, Evolution, and Behavior Program, Department of Biological Sciences, Boise State University, USA 7. Graduate Institute of Environmental Earth Sciences, University of Hokkaido, Japan Abstract Interest in and understanding of the various competencies that university sustainability and environmental graduate degree programs should aim for has increased in recent years. Yet empirical efforts that assess the effectiveness of programs from a competency building perspective have visibly lacked—particularly from a macro-perspective examining multiple cases. This study fills this gap in the literature by conducting a novel comparative assessment of different types of master’s degrees in a sample of 14 programs from top-performing universities in Europe, Asia and North America. Our study uses quantitative and qualitative approaches to assess the effectiveness of differing types of programs at building key sustainability competencies defined in the literature, and to understand the defining characteristics of programs, innovative competency building approaches, challenges encountered, and potential countermeasures. Using a typological methodology, this study classifies programs into three categories: research-oriented, neutrallyoriented and practice-oriented and then examined the competency building effectiveness of each category through questionnaires administered to faculty (n=40) and students (n = 205).
1
Results revealed low success in all program types at equipping graduates with anticipatory competencies. Statistically significant differences were also observed between research-oriented and practice-oriented programs, with the latter demonstrating higher success in building interpersonal, strategic and normative competencies. Qualitative questionnaire responses revealed a widespread student demand for more practice-orientated learning and collaborative projects with societal stakeholders. However they also highlight the important role of research-oriented programs at equipping students with theoretical and conceptual knowledge. From a sustainability competency building perspective, the findings point to a need for research-oriented programs to integrate practice-based didactic approaches for building skills and methods via real-world learning projects with external stakeholders. They also prompt a reconsideration of the special importance of research-oriented sustainability and environmental degree programs in a higher education landscape increasingly shaped by vocational and job market expectations.
Keywords: sustainability competencies; university; higher education; environmental; evaluation; assessment
Word count: 8,570 (including references and excluding tables)
2
1. Introduction The global proliferation and increase of university sustainability and environmental degree programs and enrollments is well documented (Vermeulen et al. 2014; Withycombe Keeler et al. 2016; O’Byrne et al. 2015; Vincent et al. 2013). Multiple factors are driving the expansion of this academic field. Initiatives such as the United Nations Decade of Education for Sustainable Development (2004-2014) and multiple global charters (e.g. the Talloires Declaration of 1990 and the Earth Charter from 2000) have spurred decision-maker support for increasing sustainability education in universities (Beynaghi et al. 2016; Lozano et al. 2013). In parallel, green job opportunities continue to expand globally (OECD 2014) while sustainability expertise is increasingly sought by traditional employers in non-green sectors (Vincent and Focht 2010). Yet several questions remain. Are graduates from these programs gaining the skills they need to support the global transformation towards sustainability? Which learning approaches and program types are effective in conveying these skills? While much scholarship exists on novel and effective pedagogical methods for increasing the ability of students to function as societal change agents for tackling sustainability issues (Ortega-Sánchez et al. 2018; Trencher et al. 2016; O'Brien and Sarkis 2014; Wiek and Kay 2015), such approaches are often marginal and confined to single courses or the initiatives of frontrunner faculty. Efforts to propagate or mainstream emerging and effective sustainability education practices is often hampered by lack of institutional support and a need for faculty training and support from diverse societal stakeholders and students (Disterheft et al. 2015). Even in top universities, despite promises of providing opportunities for transformational learning and effecting social change, environmental and sustainability degrees can fall short of student expectations (Ho and Ang 2016), often failing to provide key skills desired by prospective employers (MacDonald and Shriberg 2016; Thomas and Day 2014; Brundiers and Wiek 2017). As Sterling (2011) argued while recalling the words of educator Ernst Friedrich Schumacher, although the volume of education is increasing, so too is pollution, environmental destruction and dangers of a global ecological catastrophe. Similarly, Orr (2004) reminds us of the fundamental problem of education by asserting that destruction of the Earth’s environment is driven not by a lack of education but more so from “educated” graduates armed with university degrees. As such, environmental and sustainability programs are pressed with the need to break from traditional knowledge-transmission approaches and focus pedagogical efforts on building cognitive, behavioral and methodological competencies (Segalàs et al. 2009; Thomas and Day 2014) that increase the potential for students to function as societal change agents for sustainability (Wiek et al. 2011; Wiek et al. 2016; Barth et al. 2007; Lambrechts et al. 2013; Lambrechts and Petegem 2016). This points to a need for so-called constructivist approaches which, following Shuell (1986), prompt a shift in emphasis from “what the teacher does” in creating and delivering content,
3
to student engagement and “what the student does” in order to learn. In this context, interest continues to rise around the particular sets of competencies that environmental and sustainability degree programs should aim for (Glasser and Hirsh 2016; Vincent and Focht 2010; Wiek et al. 2016; Wiek et al. 2011; Barth et al. 2007; Lambrechts et al. 2013) and the relative importance of each (Rieckmann 2012; Missimer and Connell 2012). Yet despite the growth in this knowledge, graduate programs globally are still wrestling with attempts to translate these theoretical frameworks into explicit learning objectives, and how to then integrate these into curricula (Thomas and Day 2014; Wiek and Lang 2016). Moreover, empirical efforts to assess the competency building effectiveness of degree programs have visibly lacked (Glasser and Hirsh 2016)—particularly from a national or international perspective from which multiple cases are examined. Against this background, this study reviews 14 environmental and sustainability master programs from top-performing research universities around the world to examine: 1) the effectiveness of different program types at equipping students with sustainability competencies defined by (Wiek et al. 2011; Wiek et al. 2016), and 2) innovative practices for fostering competencies, challenges encountered and coping strategies. The sample of 14 programs encompasses one and two-year degrees in environmental science/management, sustainability science, and sustainability leadership. Both qualitative and quantitative data is examined from diverse sources including interviews, questionnaires, and document analysis. The small but comprehensive sample of 14 programs allows macro-level comparisons as well as the examination of finer details on faculty and student experiences in sustainability competency building and program development. This research builds on previous studies that have defined various sustainability competencies, examined the learning outcomes success of individual courses (San Carlos et al. 2017) or appraised the extent to which competency building is reflected in the curricula of a small set of bachelor programs (Lambrechts et al. 2013). It also contributes to scholarship that examines larger samples of environmental and sustainability programs (O’Byrne et al. 2015; Vermeulen et al. 2014; Vincent and Dutton 2016) to determine global trends in terms of curriculum content, program objectives, and the sustainability competencies valued by program leaders. While this study does not set out to compare the performance of one specific degree program to another, by classifying programs into three contrasting types it generates a first-of-its-kind comparative appraisal of sustainability competency building effectiveness across a sample of 14 internationally recognized programs. The findings generate insights to guide future development of new and existing sustainability and environmental graduate programs around the world. In parallel, this study’s typology-based assessment approach provides stakeholders, such as government and philanthropic funders, a basis for categorizing and appraising the didactic effectiveness of programs from a
4
sustainability competency building perspective.
5
2. Literature review In sustainably education the concept of “competency” refers to a holistic and interlinked set of knowledge, methods, practical skills, values and behaviors that are purported to enhance task performance and problem solving when dealing with real-world sustainability challenges (Rieckmann 2012; Wiek et al. 2011; UNESCO 2005; Barth et al. 2007). Distilled further, a competency can also conceived of as a triangular and integrated package of knowledge, skills and attitudes (Ploum et al. 2017). Despite agreement on this fundamental definition and the undeniable utility of the competency paradigm for steering degree programs towards more problem-based, transdisciplinary, experiential and student-centered learning approaches, a proliferation of studies and contrasting delineations has created “a sea of labels” (Glasser and Hirsh 2016: 132). This challenges any consensus regarding the particular quantity or type of competencies that sustainability and environmental education should aim for (Hesselbarth and Schaltegger 2014; Missimer and Connell 2012). Furthermore, scholars also point to a lack of commonly agreed upon tools or approaches to measure and assess the extent to which competency acquisition actually occurs in students (Glasser and Hirsh 2016). With regard to key literature, Wiek et al. (2016) build on earlier studies discussing differing types of competencies relevant to sustainability (Barth, Godemann, & Rieckmann, 2005; de Haan, 2006; Sharp, 2002; Wals, 2010), Wiek et al. (2011) to synthesize and propose a comprehensive framework of five key competencies (see Table 1) emphasized by sustainability experts in academic programs in eight universities: (i) systems thinking, (ii) anticipatory, (iii) normative, (iv) strategic, and (v) interpersonal. Although this framework is widely used across the sustainability education field, Glasser and Hirsh (2016: 132) argue it is not entirely comprehensive and that more “still needs to be done to develop and refine a complete and minimal set of relatively non-overlapping key or core sustainability competencies.” Based on a panel of sustainability education experts, the authors build on the work of Wiek et al. (2011) to propose a further set of five competencies: (i) affinity for all life, (ii) state-of-the-planet knowledge regarding life-supporting natural systems and processes, (iii) wise decision making in face of uncertainty and conflict, (iv) demonstration of sustainable behavior reflecting “the change that one wants to see in the world”, and (v) transformative social change to foster social learning and understand and apply first and second-order learning. These contrasting interpretations and articulations of key sustainability competencies can be partly explained by the objectives of each. While the set articulated by Wiek et al. (2011) concerns the need to foster the problem-solving ability of individual students relative to social sustainability challenges, the set proposed by Glasser and Hirsh (2016) has occurred in the wider ambition of prompting a fundamental repurposing of sustainability education itself. This repurposing objective encompasses not only education goals and approaches, but equally,
6
worldviews and conceptions of nature and sustainability that are shared with students. While other authors (Missimer and Connell 2012; Downs et al. 2017; Hesselbarth and Schaltegger 2014) also outline alternative or complementary conceptions of sustainability relevant competencies, some scholars have recently begun to examine sets of competencies with pertinence to the workplace and roles that students might assume in differing professional contexts upon graduation (Brundiers and Wiek 2017; Thomas et al. 2013). With some studies integrating entrepreneurial qualities and business world sensitivity into conceptions of desirable sustainability competencies (Ploum et al. 2017; Lans et al. 2014), increasingly explicit links are being made between market place expectations and the desired objectives of graduate sustainability education (Lambrechts and Petegem 2016). Although this development could prompt a normative discussion on the desirability for market expectations to shape university education goals for sustainability, scholarship suggests that workplace oriented conceptions of sustainability competencies are highly compatible with less vocationally oriented conceptions from earlier scholarship—inclusive of normative and value dimensions (Blok et al. 2015). For example, empirical work by Ploum et al. (2017) finds that a set of six theoretical competencies (1. strategic management and action, 2. embracing diversity and interdisciplinary, 3. systems thinking, 4. normative judgement, 5. foresighted thinking, 6. interpersonal) obtained from previous scholarship (e.g. Lans et al. 2014) correspond with actual competencies desired by young sustainability entrepreneurs. Their addition of the “embracing diversity and interdisciplinary” competency is noteworthy. This is not explicitly reflected in the Wiek et al. (2011) framework and demonstrates the importance of attitudinal competencies in professional practice and, in particular, the need to foster the ability of students to appreciate alternative viewpoints, values and types of knowledge. Despite rich theoretical discussions, few studies have empirically assessed the acquisition of sustainability competencies in students. San Carlos et al. (2017) applied the Wiek et al. (2011) framework to quantitatively evaluate the effectiveness of fieldwork-based courses. MacDonald and Shriberg (2016) conducted a more ambitious survey of alumni from 22 sustainability management and leadership programs. While this allowed an interesting comparison of general trends across academia, the voices of faculty are missing and many respondents had graduated several years before the questionnaire was administered. This significantly reduces this study’s value as a “realtime” evaluation of current practices in surveyed programs. It can consequently be argued that the effectiveness of environmental and sustainability graduate programs at equipping students with key competencies is still largely unverified in academic literature. In pursuit of this objective, this study employs the widely utilized core competencies framework proposed by Wiek et al. (2011). Since sustainability competencies encompass a knowledge/concepts (i.e. “understanding”) and a methods/skills (i.e. “doing”) dimension, as shown
7
in Table 1 the appraisal of competency-building effectiveness is conducted from both perspectives in this study. Admittedly, competency frameworks proposed by other scholars could have also been suitable for this study. Reasons for choosing the Wiek et al. (2011) framework mainly concern its: 1) explicit incorporation of the two dimensions knowledge/concepts and methods/skills, which facilitates the comparison of results for each; 2) relative simplicity for questionnaire respondents to understand due to the articulation of only five competencies; 3) widespread recognition in academia as shown by numerous citations (281 according to Scopus in January 2018); and finally, 4) overlap and consistency with other frameworks proposed in both academic and business contexts (Lans et al. 2014; Brundiers and Wiek 2017; Ploum et al. 2017; Hesselbarth and Schaltegger 2014; Rieckmann 2012).
8
Table 1. Framework for assessing effectiveness at fostering core sustainability competencies. After Wiek et al. (2011) and Wiek et al. (2016). Competence
Knowledge/concepts
Methods/skills
Systems thinking
Ability to understand how differing systems
Ability to generate and interpret results
interact across different domains and scales.
showing how differing systems interact across different domains and scales.
Anticipatory
Ability to understand differing future visions,
Ability to generate and interpret results
states and impacts related to sustainability.
showing differing future visions, states and impacts related to sustainability.
Normative
Ability to understand the (un)sustainability of
Ability to generate and interpret results
current/future states and required values,
about the (un)sustainability of
ethics, principles, lifestyles etc.
current/future states and required values, ethics, principles, lifestyles etc.
Strategic
Interpersonal
Ability to understand the design and
Ability to generate change and evaluate
implementation of interventions and
the design and implementation of
transformative governance strategies for
interventions and transformative
sustainability.
governance strategies for sustainability.
Ability to understand collaborative and
Ability to facilitate collaborative and
participatory sustainability research and
participatory sustainability research and
problem-solving.
problem-solving.
9
3. Methods 3.1 Sample selection The sample for this study comprises of 14 multi-disciplinary master programs from the fields of environmental science, environmental management, sustainability science, and sustainability leadership (see Table 2). The study targets Master of Science (MSc) degree programs conducted in English that encompass varying combinations of mandatory core courses, electives, specialization tracks, and requirements regarding capstone projects or master’s theses. To gain insight into best practices, research universities performing highly in the Academic Ranking of World Universities (ARWU) by Shanghai Jiao Tong University are examined. Nine of the surveyed programs are ranked in the top-100, three between 100 to 150, and two are non-ranked. Purposive sampling was employed to ensure a contrasting representation of the following variables: •
Disciplinary focus (environmental science, environmental management, sustainability science or sustainability leadership)
•
Orientation towards 1) practice or research, and 2) social sciences or natural/applied/technical sciences
•
Geographic location
•
Length of degree (one or two years)
The sample is confined to programs where at least one full-time teaching faculty agreed to cooperate for a semi-structured interview and authorize the administration of written surveys. Seventeen universities were initially contacted and fourteen responded favorably.
3.2 Data collection Primary data were obtained from semi-structured interviews and questionnaires containing both quantitative and open questions. Secondary data came from program websites, brochures, syllabi, curricula and internal documentation. The first author conducted 18 interviews via Skype or telephone from July 2016 to May 2017. Most interviewees were chosen through convenience or snowball sampling and were senior faculty (either associate- or full-professor rank) with several years of teaching, research and mentoring experience in that program. All conversations were recorded and then transcribed into detailed minutes. Interviews were conducted in two rounds. The first built preliminary understanding regarding: 1) basic program attributes (i.e. program history, objectives, orientation, number and background of enrollees); 2) unique approaches to governance, curriculum design, and education; and 3) challenges encountered during competency building efforts and effective countermeasures. The second interview round targeted four programs to examine signature approaches to fostering student sustainability competency acquisition (see Supplementary Material). These were chosen for
10
further analysis due to contrasting and well-developed signature didactic approaches to fostering sustainability competencies. Written questionnaires consisted of two sections. The first collected quantitative data on program orientations and learning objectives. The second collected both quantitative and qualitative data on the effectiveness of programs (inclusive of all core courses, electives and capstone options) at equipping students with sustainability competencies defined by Wiek et al. (2011) using a 1-5 Likert scale. Questionnaires provided detailed explanations and examples of each sustainability competency, also providing room for qualitative information on effective learning practices and barriers hampering competency acquisition. Questionnaires were pilot tested with colleagues at the University of Tokyo, then finetuned based on feedback. They were distributed as Word documents via email to faculty and students in each program from late September 2016 to early April 2017. Respondents were chosen via snowball sampling or random selection from publically or privately provided email lists. Roughly 120 faculty and 600 students were initially emailed to request permission to send questionnaires. Of these, approximately 60% in each group responded favorably. We then distributed questionnaires to 1) faculty affiliated principally with the program concerned and 2) students who were either in the last semester/year of studies or who had graduated within the last 12-15 months. This ensured respondents possessed sufficient and up-to-date understanding of program objectives, courses, and pedagogies, while also being enrolled recently enough for their perspectives to be relevant. In programs with specialization tracks, efforts were made to target students in contrasting specializations. Completed questionnaires were secured from 40 faculty and 205 students, achieving response rates of roughly 50% and 40% respectively. For questionnaire responses containing unclear answers, respondents were contacted for clarification. In cases where this proved unsuccessful, answers were rejected (n=9).
3.3 Data analysis Although student responses outweigh faculty roughly five-fold, this study considers faculty as “experts”. Accordingly, when calculating mean scores for each program faculty responses are weighted as 30% relative to 70% for students. Aggregate results for the three types of programs (research, neutral and practice) show the combined mean scores of faculty and student responses from individual programs. One-tailed t-tests were conducted to verify statistical discrepancies between aggregate results for research- and practice-programs. Qualitative data collected through questionnaires were integrated into Excel spreadsheets and analyzed through a qualitative data analysis software (MAXQDA) to identify recurring themes.
11
Table 2: Overview of Sample Code
Australia ANU-E
UM-E
Asia NUS-EM
UT-SS
TU-IELP
Europe LU-ESSS
BTH-SLS
1 2
University and host institution
Degree name
ARWU ranking (2016)
Year formed
Approx. enrolments in 20161
Specialization tracks
Length (full-time years)
Australian National University College of Medicine, Biology and Environment University of Melbourne Faculty of Science (Office for Environmental Programs)
(MSc) Master of Environment
77
20142
40
Yes
2
(MSc) Master of Environment
40
Early 2000
400
Yes
2
National University of Singapore School of Design and Environment University of Tokyo Graduate School of Frontier Sciences (Department of Environmental Studies) Tohoku University Graduate School of Environmental Studies
(MSc) Master of Environmental Management
83
2001
50
No
1
(MSc) Master of Sustainability Science
20
2007
30
No
2
(MSc) International Environmental Leadership Program
101-150
2015
25
No
2
Lund University Lund University Centre for Sustainability Studies Blekinge Institute of Technology Department of
(MSc) Master in Environmental Studies and Sustainability Science
101-150
1997
90
No
2
(MSc) Master in Strategic Leadership towards
n/a
2004
45
No
1
Includes part-time students or students in dual degrees. For the new 2-year program that builds on the older 1-year program.
12
Strategic Sustainable Development OU-NSEG Oxford University School of Geography and the Environment ETHZ-ES ETH Zurich Department of Environmental Systems Science North America ASU-SS Arizona State University School of Sustainability CU-ESP Clark University Department of International Development, Community, and Environment DU-EM Duke University Nicholas School of Environment UBC-RES University of British Columbia Institute for Resources, Environment and Sustainability UCSBUniversity of ESM California, Santa Barbara Bren School of Environmental Science and Management
Sustainability
(MSc) Master in Nature, Society and Environmental Governance
7
2002
22
No
1 (with 2year option)
(MSc) Master in Environmental Sciences
19
1987
300
Yes
2
(MSUS) Master of Sustainability Solutions
101-150
2010
50
Yes
2
(MSc) Master in Environmental Science and Policy
n/a
mid-1980s
35
No
2 (with 1year option)
(MSc) Master of Environmental Management
25
1991
300
Yes
2
(MSc) Master in Resources, Environmental and Sustainability
34
20032004
40
No
2
(MSc) Master of Environmental Science and Management
42
1996
160
Yes
2
13
4. Results 4.1 Program characteristics Figures 1a-c show the mean scores of questionnaire responses that characterize program orientations from three perspectives: 1) practice or research, 2) social sciences (implying a qualitative approach and policy/society focus) or natural/technical/applied sciences (implying a quantitative approach and environmental science focus); and 3) flexibility to change this orientation (shown as thin, solid lines) through different course electives, specialization tracks or capstone models. Program names corresponding to each code appear in Table 2. The first perspective concerns the overall learning objectives of programs and assumed career pathways of graduates (i.e. the x-axis) while the second perspective concerns mainly disciplinary content, methodologies and subject focuses (i.e. the y-axis). Results suggest that programs can be classified into three clusters: practice-oriented, neutrally-oriented, and research-oriented. This distinction is retained for the purpose of later assessing each type’s effectiveness at equipping graduates with core sustainability competencies. This categorization attaches greater emphasis to program objectives (i.e. practice versus research) rather than disciplinary content, methodologies and thematic focus. This decision is based on the assumption that effectiveness in building sustainability competencies would be affected by research/practice orientations more so than differing content and disciplinary focuses. Results show a slight tendency towards social science, qualitative and societal/policy aspects for all three program types. Even for programs where a higher orientation towards natural/technical/applied sciences (e.g. ETHZ-ES and DU-EM) was expected, integration of knowledge and methods from both ends of the disciplinary spectrum is apparent. Practice-oriented programs focus on meeting employer needs and fostering problem solvers for business, government and non-profit organizations as opposed to traditional scientists for academic research positions (Vincent and Focht 2009). They emphasize hands-on problem solving and team work, aim to foster methodological and intrapersonal skills, and actively integrate real-world learning opportunities into curricula. This often occurs through client projects collaboratively implemented with external partners in industry, government and non-profit organizations. These projects feature in both regular courses and capstone experiences, and programs have developed signatory pedagogical models in this area (see Supplementary Material). Practice-oriented programs also proactively seek to align curricula with employer demands by incorporating stakeholders from industry, government and non-profits into either faculty teams or department governance structures. Additionally, several practice-oriented programs reported that they principally target students with two or more years of practical experience to deepen ties with the workforce.
14
Research-oriented programs focus more on fostering the academic research skills of graduates for knowledge-producing careers in industry or government, or alternatively, entry into doctoral programs. These programs reported less formal measures to reflect employer needs in curricula design and instead place relatively greater emphasis on academic theory and producing scientifically rigorous knowledge. Practical projects with clients tend to feature less and capstone requirements generally involve traditional dissertations. Overall, less explicit efforts were reported to recruit practitioners into faculty teams and societal stakeholders into program governance bodies. Finally, neutrally-oriented programs contain characteristics of both these program types. Individual programs on Figures 1a-c show highly varying levels of flexibility. This has significantly challenged our attempts to classify programs. While some programs like OU-ESSS and NUS-EM provide relatively little flexibility due to a mostly fixed curriculum, students in much larger programs like UCSB-EM, ASU-SS, DU-EM and ETHZ-ES can heavily tailor their individual degrees. This can occur firstly by selecting various pre-fabricated specializations, which span both the natural sciences and social sciences and prescribe core and elective subjects, and secondly, by choosing different electives and capstone experiences. Degree customization can occur to such an extent that two students in the same program can acquire a completely different learning experience in terms of both teaching content and learning objectives.
15
Figure 1a Program orientation and range of flexibility (research-oriented)
16
Figure 1b Program orientation and range of flexibility (neutrally-oriented)
17
Figure 1c Program orientation and range of flexibility (practice-oriented)
18
4.2 Evaluation of effectiveness at fostering sustainability competencies The following sections present the results obtained from faculty and students regarding the question: “Based on the totality of your program’s vision, objectives, core curriculum, electives and capstone options, in your view how effective is your program at equipping students with the following sustainability competencies?”. Self-evaluation occurred from two dimensions: 1) concepts/theory and 2) methods/skills. The first surprising finding concerns the knowledge/concepts dimensions of each competency (Fig. 2a). It was initially assumed that research-oriented programs would perform the highest in this dimension due to a focus on knowledge production. However results show no statistical difference (at p
