Qual Quant (2011) 45:783–800 DOI 10.1007/s11135-010-9322-z RESEARCH PAPER
Revealing personal professional theories An explorative methodological study Pauline M. Huijts · Elly de Bruijn · Harmen Schaap
Published online: 2 April 2010 © Springer Science+Business Media B.V. 2010
Abstract This article introduces the concept of Personal Professional Theories to refer to knowledge development within the context of vocational education. They are personal interpretations of formal theories, work process knowledge and notions about professional attitude in a specific occupational domain, and are assumed to be frames of reference for professional experiences and guides in action simultaneously. To make detecting development in personal professional theories possible, strategies need to be validated for revealing them. Two strategies [i.e., concept mapping (CM) and a combination of CM with stimulated recall (SR)] are compared, by grading them on three aspects of adequacy (i.e., richness, coherence and content), to see which one reveals personal professional theories better. The sample consisted of 16 students of two training courses. Both strategies succeeded in revealing personal professional theories: The constructed maps displayed personal interpretations of formal theories, work process knowledge and notions about professional attitude. However, the concept maps of the combined strategy did worse on all aspects of adequacy. The CM strategy appeared better for revealing personal professional theories. Several explanations are offered, which deserve further investigation. Hence, despite the project’s small scale, the outcomes provide in interesting clues to reveal personal professional theories. Keywords Personal professional theories · Knowledge development · Professional development · Vocational education and training · Concept mapping · Stimulated recall Of course you have to keep up with developments, otherwise you will lag behind within weeks, and who will offer you a job then?! Student in vocational education The quote illustrates a frequent answer students came up with when asked what characterizes a good professional: The need to keep up with developments. In order to do this, one has to have basic knowledge and continue to learn. It is exactly this combination of theories,
P. M. Huijts · E. de Bruijn (B) · H. Schaap Department of Pedagogical and Educational Sciences, Faculty of Social and Behavioural Sciences, Utrecht University, P.O. Box 80140, 3508 TC Utrecht, The Netherlands e-mail:
[email protected]
123
784
P. M. Huijts et al.
knowledge, and attitude about one’s occupation, which constitutes a personal professional theory (PPT). PPTs are personal interpretations of formal theories, work process knowledge, and notions about professional attitude in a specific occupational field. PPTs are assumed to be frames of reference for professional experiences and guides for professional action simultaneously. Hence, students should form adequate (i.e. relevant for the professional domain they are trained for) PPTs during their educational training courses (Schaap et al. 2009). This article aims to detect strategies to reveal PPTs in an explorative way.
1 Context In the past decade, curriculum changes and reforms of vocational education and training have been introduced in the Netherlands, like in many European countries, in response to societal changes (i.e. lifelong learning) (Eurydice/Centre Européen pour le Développement de la Formation Professionnelle 2001; Nijhof and van Esch 2004; de Bruijn 2004). In the Netherlands, vocational education and training is the continuation of pre-vocational education with end qualification levels ranging from SEDOC-level II to IV, the latter of which gives entrance to higher education (de Bruijn 2004). The ages of its students range from 16 to 64 theoretically, but 95% of them are younger than 22. The move towards competence based vocational education asks for improvement of the ways in which learning participation and outcomes are defined, understood, and appreciated (Eurydice/CEDEFOP). One of the main issues for debate and experimentation is how to define, support and assess learning outcomes in which knowledge, skills, and attitudes are addressed as integrated wholes, while acknowledging the value of each of the components (Baartman et al. 2007). The PPT concept might be a relevant contribution in acknowledging the value of knowledge, without extracting it from the integrated concept of competence. If PPTs are revealed, personal interpretations about being a good professional can be assessed. If repeatedly revealed during education, development in these interpretations can be assessed too. The fundamental idea of this article is that vocational education and training students should develop adequate PPTs during their training (Schaap et al. 2009).
2 Revealing PPTs Personal professional theories are personal interpretations of formal theories, work process knowledge, and notions about professional attitude in an occupational field. Formal theories are discipline-based theories and concepts, derived from bodies of coherent, systematic knowledge (Eraut 1994). The experts who teach these theories may not even be members of the occupational field concerned (Eraut). Work process knowledge is an employee’s knowledge of the work processes in the enterprise as a whole, which goes beyond what is required to substantiate a single occupation (Boreham 2002). Notions about professional attitude include group notions about values, norms, and virtues (Barney et al. 1997) with regard to an occupation. Since PPTs are personal interpretations, they vary between individuals. However, they vary less between employees within an occupational field than between employees across occupational fields, because (a) occupations are based on systematic knowledge, acquired through prescribed training and experience, and (b) a professional adheres to a set of professional norms (see Wilensky 1964). This similarity in training and norms within an occupational field makes it possible to distinguish principles of formal theories, work process
123
Revealing PPTs
785
knowledge, and notions about professional attitude which are shared by employees within an occupational field. These shared principles are termed the collective professional theory. There is more to a PPT than its substance. For a PPT to be considered adequate, not only substance, but also structure matters. To cover both, the researchers propose three aspects of adequacy in line with what Buitink (2007) determined for working theories. The first is richness, which relates to the dispersal of a PPT over the domains formal theories, work process knowledge, and notions about professional attitude. The better (i.e. proportionately) dispersed a PPT is over these three domains, the more adequate it is. The second aspect of adequacy is coherence. As they are not innate, PPTs develop in time, just as knowledge bases are assumed to develop; from small and less coherently organized structures in the novice stage, into large and integrated structures in the expert stage (Shavelson et al. 2005). So to be an expert in a domain implies a highly coherent conceptual structure. Therefore, the more coherent a PPT is, the more adequate it is. The third aspect of adequacy is content itself. A PPTs’ content consists of the three domains formal theories, work process knowledge, and notions about professional attitude. However, whether these three domains are present in one’s PPT, and what they include differs per person. In determining adequacy with respect to content, a comparison of a PPT to an occupational field’s collective professional theory can be made. The more agreement there is with the collective professional theory principles, the more adequate a PPT is with respect to content. The three aspects together provide criteria for a PPTs’ adequacy; the richer, more coherent, and more in line with a collective professional theory a PPT is, the more adequate it is. Though PPTs have a practical function as they are assumed to guide action in practice, they are implicit in nature because their contents are outside one’s ordinary consciousness (cf. Reber 1993). Implicit constructs are hard to reveal, because people do not always realize what they know (Eraut 1994). Strategies to reveal implicit constructs are, amongst others, found in research examining teacher knowledge. Two strategies seem particularly suitable for revealing PPTs; concept mapping (CM) and stimulated recall(SR). Concept mapping is a research strategy with which the meaning one assigns to a concept can be derived from a graphical representation of the relations between concept and related propositions. Two approaches can be distinguished: a structured and a non-structured one. In the structured approach, participants get a fixed list of concepts (Zanting et al. 2003) to allocate around the central theme. In the non-structured approach only the central theme is given (Shavelson et al. 2005), and participants think of concepts to allocate themselves. Winitzky and Kauchak (1995) state that CM is a valid strategy, which is both accessible and usable, in revealing knowledge. Zanting et al. consider CM a valuable strategy to access practical knowledge, and Shavelson et al. state that CM provides representations of the structure of knowledge. The use of CM hence is appropriate, because besides being used to access teachers’ practical knowledge (Meijer et al. 2002; Schepens et al. 2007; Winitzky and Kauchak 1995; Zanting et al. 2001), assess students’ learning styles (Nickles 2003), and mental strategies (Edwards and Marland 1984), the use of CM also brings much information about in a relatively short time, without requiring intensive training (Zanting et al. 2001). Stimulated recall is an inspection procedure in which videotaped passages of behaviour are replayed to individuals to stimulate recall of their concurrent cognitive activity (Lyle 2003). Two approaches can be distinguished as well: The recall can be prompted by the researcher (i.e. cued recall) or be spontaneous response by the participant, (i.e. free recall). The use of SR is also appropriate, because besides being used to access practical knowledge (Zanting et al. 2001, 2003), reveal student teachers’ learning processes (Schepens et al. 2007), examine the instructional decision making of expert special educators (Stough and Palmer
123
786
P. M. Huijts et al.
2003), and reveal what students were thinking during class (Edwards and Marland 1984; Appleton 1995), it alerts participants to possible knowledge or actions that might otherwise go unnoticed for further analysis (Lyle 2003).
3 Problem definition The project described in this article is part of a larger study on collaborative learning in vocational education and training and the development of students’ PPTs (Schaap et al. 2009). To make detecting development in students’ PPTs possible in the larger study, strategies to reveal them need to be validated. Based on prior research, two strategies are chosen: CM and the combination of CM with SR. A non-structured approach to CM is favoured, because this provides opportunities to reveal students’ conceptual understanding more than the structured approach, as students explain the concepts to themselves more (Shavelson et al. 2005). The literature discussed previously suggests that adding SR to the CM strategy might be beneficial. While watching the video, participants will be asked questions. Hence, the cued recall approach is chosen, because this approach appears to provide in more response than free recall (Devenny et al. 2002). The research questions are: 1. How do PPTs revealed through CM alone versus CM with SR differ with respect to richness, coherence and content (i.e. adequacy)? 2. Does the combined strategy reveal PPTs better? Hence, the revealed PPTs will be examined with regard to the three, in the previous section defined, aspects of adequacy. The focus of the research project is methodological; on the ability of the two strategies to reveal PPTs. Explanations for differences in individual adequacy will be examined with regard to strategy used.
4 Methodology The project consists of two phases; a preparatory and a research phase. In the preparatory phase, a meeting is held with professionals from the participating students’ future occupational fields and content based teachers from their training courses (i.e., an expert meeting), since the researchers are no experts in these fields. This meeting is important in formulating collective professional theories of the occupational fields in the analysis. Based on qualification profiles (i.e. attainment targets) of the training courses, the experts describe the ideal professionals in the occupational fields. These descriptions provide in principles of being a good professional, which cover formal theories, work process knowledge, and notions about professional attitude. The research phase will be described in subsequent sections. 4.1 Participants Sixteen vocational education and training students participate in the research phase. They are from two different training courses, in order to reveal not only the general, but also the occupation specific nature of a PPT. Eight participants will become an ICT network administrator (‘administrator’ for short) and the other eight a multimedia designer (‘designer’ for short). These training courses were chosen because their occupational fields are relatively well defined in terms of their technical bases and professional norms (see Wilensky 1964).
123
Revealing PPTs
787
Table 1 Procedures for both strategies Stage
Content
Participant
Researcher
1
Introduction
No role
Personal introduction
2
Brainstorming in CM strategy
Responding to central theme and questions
Filming followed by SR interview in CM + SR strategy
Watching video Responding to central theme and questions
Explanation of intended procedure
3
Categorizing
Specifying category of term:
Asking questions to prompt response Listing response on paper Asking questions about thoughts and action on tape Listing response on paper Writing terms on cards:
Knowledge/traits
Knowledge = Blue Traits = Green
Specifying nature of term: Element/Example
Element = Hexagon Example = Rectangular
4
Structuring
Putting examples near element they exemplify Clustering elements belonging together Putting important elements closest to central theme
No role
Hence, it is assumed, their collective professional theories will be too, which will enable coding the PPTs more accurately. Because PPTs develop through training and experience, the researchers selected students who were in their final stages of training, so they will have had school based training and been in internships already. 4.2 Procedures Both groups of administrator and designer participants are further divided by the use of the two strategies: CM and CM in combination with SR. In this way, a two by two matrix with four participants in each cell is created. In both strategies, the researcher holds individual sessions with the participants, which result in 16 visual representations of the participants’ PPTs. The procedures of both strategies are summarized in Table 1. The CM strategy (referred to as CM) proceeds through four stages; introduction, brainstorming, categorizing, and structuring. The central theme in the brainstorming stage is (Based on Nickles 2003): “As a good administrator/designer professional…” Each item on the list generated during this stage will be referred to as a ‘term’. Brainstorming has finished when a participant can think of no more terms to be placed on the list. The division made in the categorizing stage serves to stimulate thinking about the multiple features of being a good professional. Specifying the nature of a term as either an element (i.e. fundamental part), or an example of an element stimulates an abstraction in thinking and simplifies the cluster forming in the subsequent stage, which is categorizing. The whole procedure (i.e., the four stages) takes about an hour.
123
788
P. M. Huijts et al.
In the procedure of the second strategy (referred to as CM + SR), the second stage is different: The participant is filmed for half an hour at his or her internship followed by an SR interview. The SR video is shown to the participant immediately after filming, because a minimal time delay between events and recall increases validity (Lyle 2003). This whole procedure lasts about 1 h and a half. 4.3 Analysis Collective professional theories are formulated for both training courses to make comparing the aspect of content possible. The 16 concept maps are coded by two raters independently. Only one of the raters was present at the sessions. Based on the coding, the maps can be given points on richness, coherence, and content, which together provide a grade on adequacy. The principles of good professionalism from the expert session are the main source for formulating the collective professional theories. The principles are described cautiously to enable coding the elements of a PPT. The principles cover formal theories, work process knowledge, and notions about professional attitude. However, in coding the elements, some principles of formal theories appear to be missing. The lack of principles in this domain is discussed with an administrator and designer professional, who formulate extra principles. The resulting collective professional theory principles are presented to professionals, who determine the most important principle or principles for each domain. In this way, the collective professional theories are based upon multiple sources, which is known as triangulation (Yin 1984) by data source (Miles and Huberman 1994). The collective professional theories are shown in Figs. 1 and 2. Descriptions of their principles can be found in Appendix Tables 7 and 8. Since research has shown that raters can score concept maps reliably even when complex judgments are required (Shavelson et al. 2005), both raters code the maps based on a codebook (see Appendix Table 9). The reliabilities obtained are displayed in Table 2. They are all considered good (.6–.75) to excellent (>.75; see Robson 2002, Chap. 14). All terms are coded with regard to category (trait/knowledge) and nature (element/example). For both categories, only elements are coded further, because examples are only meant to clarify elements. Elements are coded further with regard to cluster (no connections/number of elements in the cluster), number of examples accompanying the element (no examples/number of examples), importance (core element/outer circle element), and domain (formal theories/work process knowledge/notions about professional attitude). If elements suit the description of collective professional theory principles, they are coded with regard to principle as well. For instance; the element ‘Being creative’ of a designers’ concept map matches the principle ‘Creative’ of the designers’ collective professional theory (see Fig. 3; Table 3). The maps are given points on the three aspects of adequacy: richness, coherence, and content. All distribution rules are shown in Table 4. The points given provide a grade theoretically ranging from 0 to 10, whereby a six indicates sufficient adequacy. Since substance is thought to be more important than structure, the distribution of points is chosen such that a PPT can be graded a six on substance alone. Because distribution of points on content relates solely to substance, and distribution of points for the other two aspects relates to both substance and structure, it is decided richness and coherence obtain a maximum of three points each and content a maximum of four. With regard to richness, not just mentioning the possible domains (Buitink 2007), but also proportionate dispersal over the domains is taken into account, because this indicates higher adequacy. The number of examples given is also indicative of a PPT’s richness. With regard to
123
Revealing PPTs
789
Core
Importance
Formal Theories Work Process Knowledge Notions about Professional Attitude
Co e iv at ic un m m
al ci So us io ud St ng ni an Pl k or w e ol am lR Te na sio es of Pr up ng pi ee g K in m am gr
k or
e ar
e ar
w
w et
o Pr
N
d ar H
ftw
sic Ba
So
Fig. 1 Principles of the administrators’ collective professional theory
Core
Importance
Formal Theories Work Process Knowledge Notions about Professional Attitude
t en nd pe de In
us io ud St
e tiv ea Cr up ng pi ee K g tin us dj A g in ur ct ru St k or w am ol Te lR na sio es of Pr
sic Ba ia ed tim ul M
Fig. 2 Principles of the designers’ collective professional theory
123
790 Table 2 Interrater reliabilities for coded variables
P. M. Huijts et al. Variable
Cohen’s κ
Category
1.00
Nature
1.00
Cluster
.87
Examples
.92
Importance
.79
Domain
.73
Principle
.71
Fig. 3 Excerpt of a designer’s concept map, constructed in the CM + SR strategy
Being creative
* Creating * Offering alternatives
Table 3 Example of coding an element
Variable
Code
Term
Being creative
N
3
Strategy
CM + SR
Training
Designer
Category
Trait
Nature
Element
Cluster
No connections
Examples
2
Importance
Core element
Domain
Notions about professional attitude
Principle
Being creative
coherence, a division in line with Buitink is used. A map’s coherence is a ‘register type’ (Type 0) if elements are randomly connected to the central theme, a ‘semi-cluster type’ (Type 1) if some elements are in clusters and some are randomly connected to the central theme, and a ‘cluster type’ (Type 2) if clusters of elements are connected to the central theme. As cut-off rules between coherences, the researchers decided that if 0–10% elements were clustered it would be a Type 0, 10–50% would be a Type 1, and more than 50% would be a Type 2. The types are hierarchically organized, indicating lower or higher adequacy. Because clustering few elements might be easier than clustering many, the number of elements clustered is also taken into account. With regard to content, elements coded as matching the collective professional theory principles are counted. The number of matches is translated to a number of points. The matched principles are examined with regard to importance.
123
Revealing PPTs
791
Table 4 Rules for distributing points on the three aspects of adequacy Aspect
Points
Sub aspects
Distribution of points
Richness
1
Dispersal
1
Proportionate dispersal
1
Examples
2
Type
1/3 point for each domain mentioned The mean (m) is calculated (total elements/3) and divided by two (=x). A scale is set up ranging from (m − x) to (m + x). 1/3 point per domain if the number of elements is within these boundaries 1 point if the number of examples exceeds that of elements 0 point for a register type
Coherence
1 point for a semi-cluster type 2 points for a cluster type
Content
1
Number of elements
2
Agreement to collective professional theory
2
Agreement to importance of principles
1 point if the number of elements exceeds the minimum number of elements of the collective professional theory: Administrators >12 and Designers >10 Administrators: 1/6 point for each collective professional theory principle mentioned Designers: 1/5 point for each collective professional theory principle mentioned Agreement to importance of principles divided by total number of principles mentioned 0–24%:.5 point, 25–49%: 1.0 point 50–74%: 1.5 point, 75–100%: 2.0 points
5 Results Participants were well able to formulate and visualize their interpretations of being a good professional in the individual sessions. The maps made seemed to do justice to their interpretations. The researcher checked this at the end of the session, for instance by summing up the elements placed closest to the central theme: “Is it, according to you, most important for a good designer to know everything about design, be creative, update his or her knowledge and be independent?” Mean points and standard deviations on the three aspects richness, coherence, and content, and resulting mean grades and standard deviations on adequacy were calculated for both training courses and strategies (see Table 5). The maps which obtained the two extreme grades on adequacy are pictured in Fig. 4 (Map 1) and 5 (Map 2). Map 1 was constructed by a designer participant in the CM strategy, and obtained the highest grade (8.4) on adequacy. Map 2 was also constructed by a designer
123
792
P. M. Huijts et al.
Table 5 Mean points, grades and standard deviations per training course and strategy Participants
Richness m
SD
Coherence
Content
m
m
SD
m
SD 1.5
SD
Grade
Designers
2.1
.6
1.4
1.2
2.6
.5
6.1
Administrators
2.0
.9
2.0
.9
2.5
.4
6.6
1.3
CM
2.2
.7
2.0
1.1
2.7
.5
7.1
1.1
CM + SR
1.9
.7
1.4
1.1
2.4
.4
5.7
1.2
Element Example
* Strong and weak points Self reflection * Handel criticism * Being critical to oneself
* Actuality * Keeping up * No ICT knowledge only
Learning from Knowing the classmates market Working experience
*Improvising
Knowing oneself Offering alternatives Knowledge taught oneself
Diplomas/ certificats
* Self tuition * Courses * Researching
* Put one’s proud aside * The customer is the king * Researching the order * Advising and persuading the customer
Working with computers
* Planning of projects
* Customer is king Being communicative
Presenting oneself Sellling oneself Explaining
* Transferring information Putting proud Planning well * Stick to one’s * Advising aside tasks * Helping customers & * Planning & others persuade them Working making a risk * Being independently analysis amicable * Put problems * Breeding forward goodwill Being social Being motivated * Personal contact Being studious/ Look for * Working on a information project base * Prince 2 Being creative * Knowing what * Scrumming is allowed * More * Actuality knowledge Working in a Being inventive * Not only ICT means greater team related value * Knowing the * Researching Adjusting to the options * Self tuition Improvising company * Having courses * Adjusting to * Upgrade oneself target group Being flexible * Choosing Stress resistant * Working what medium to overtime * Handeling use * Assisting in pressure * Determining difficulties * Deadlines Traits layout
As a good design professional..
Advising customers Adjusting the product
Working structurally
Adjusting to Traits customer
Working systematically * Building a website
Knowledge
Fig. 4 Map 1, constructed in the CM strategy
student but in the CM + SR strategy and obtained the lowest grade (4.2). Coded variables for these maps are shown in Table 6 and will explain the distribution of points on the aspects of adequacy. With regard to richness, the strategies differed in number of examples in favor of the CM strategy (26.63 in CM vs. 19.25 in CM + SR per person). Map 1 also displays more examples than Map 2 (44 vs. 19). Maps 1 and 2 both mentioned elements in all three domains and hence obtained one point each for dispersal. But they did not obtain a high score on proportionality, which is shown in Figs. 6 and 7. In these figures, examples are left out, and the domains of the elements are indicated. In Map 1, 32 elements were given. In setting up a scale, the mean was calculated by dividing this number by three (10.7), and half of the mean added to and subtracted from the mean. The scale obtained for Map 1 ranged consequently from 5.4 to 16. However, there were three elements in the domain of formal theories, 10 elements
123
Revealing PPTs
793
Element Example Being curious Knowledge of matters *Offering alternatives
Selling oneself
Communi cation *Good contact with customers * Keeping customer informed * Making appointments *Making a schedule * Explain why something is not possible * Meeting with colleagues *Complete one another
Being customer friendly
* Good portfolio
*Experimenting
Educate oneself
As a good design professional..
Weak & strong points
Get inspiration Multimedia knowledge * Photoshop * Webdesign * Get lessons * Try oneself
Have fantasy
Wanting to learn further * Never done learning * Implicit & Explicit
Being honest * No plagiarism * Source indication
Being interested
Knowledge
Learning from others
* Unique designs
Traits
Fig. 5 Map 2, constructed in the CM + SR strategy
Table 6 Variables of Maps 1 and 2 explaining the distribution of points on the aspects of adequacy Aspect
Variable
Richness
Domain
Nature
Map 1
Map 2
Formal theories
3
1
Work process knowledge
10
2
Notions about professional attitude
19
11
Elements
32
14
Examples
44
19
Coherence
Cluster
Elements clustered
30
0
Content
Principle
Matches with collective professional theory
8
6
Importance
Match with collective professional theory
60%
40%
in the domain of work process knowledge and 19 elements in the domain of notions about professional attitude. Hence, only the 10 elements of work process knowledge fell within the scale’s range, which made Map 1 receive 1/3 point on proportionality. Map 2 obtained no points on proportionality at all. Figures 6 and 7 also visualize the coherence aspect for Maps 1 and 2 by drawing circles clustered elements. In Map 1, more than 50% of the elements were clustered. Therefore, Map 1 was considered a cluster type (Type 2). Besides being a cluster type, the number of elements clustered exceeded the minimum number of elements there should be in a PPT of a designer participant (30 clustered vs. 10 required). Therefore, one other point was assigned to Map 1, which consequently received the maximum of points on coherence (3).
123
794
P. M. Huijts et al.
Self reflection
Learning from classmates
Adjusting to customer
Market knowledge
Working experience
Knowing oneself
Knowledge taught oneself
Being communicative Offering alternatives
Diplomas/ certificats
Adjusting the product Planning of projects
Working structurally Working independently
Putting proud Explaining aside
Being social
Being motivated
Being creative
Working with computers
Being inventive
Being studious/ Look for information Improvising
Working systematically
Sellling oneself
Planning well
As a good design professional ..
Advising customers
Presenting oneself
Working in a team Adjusting to the company
Formal Theories Work Process Knowledge
Stress resistant
Notions about Professional Attitude
Knowledge
Cluster
Being flexible
Traits
Fig. 6 Map 1: without examples, but with dispersal and clustering of elements indicated
Being curious
Knowledge of matters
Being customer friendly Selling oneself Communi cation
Educate oneself Weak & strong points
As a good design professional..
Have fantasy Learning from others
Wanting to learn further
Get inspiration Multimedia knowledge
Formal Theories
Being honest Being interested
Work Process Knowledge Notions about Professional Attitude
Knowledge
Traits
Cluster
Fig. 7 Map 2: without examples, but with dispersal and clustering of elements indicated
123
Revealing PPTs
795
In Map 2, all elements were placed separately: Hence Map 2 was considered a register type (Type 0), and received no points on this aspect. In this regard, the difference between Maps 1 and 2 is illustrative: In the CM strategy, six concept maps were considered a cluster type, one a semi-cluster, and one a register type, while in the CM + SR strategy, only two concept maps were considered a cluster type, five a semi-cluster, and one a register type. In comparing content of the maps to the collective professional theories, the elements codes as collective professional theory principles were examined. In Map 1, eight of the 10 designers’ collective professional theory principles were mentioned. This was translated to a number of points (.2 per principle mentioned, hence 1.6 points). The matched principles were examined with regard to importance. The percentage of agreement to importance was calculated and translated into a number of points following the rules (see Table 4). In this way Map 1 received 3.1 points on content and Map 2 received 2.2 points on content. Between strategies, differences were consistent: All aspects received more points in the CM strategy, which resulted in a mean difference of 1.4 points in the final grade. Between training courses, differences were not consistent. The difference between strategies was largest for coherence. This was due to the fact that six out of eight maps displayed a structure type in the CM strategy, while only two did in the CM + SR strategy. The PPTs revealed in the CM strategy were consequently more coherent. In terms of substance and structure of a PPT, the structure aspect especially did not seem to benefit from the CM + SR strategy. Hence, the CM + SR strategy did not appear to reveal PPTs better, but t even seemed the other way round.
6 Conclusion In previous sections the researchers formulated two research questions: 1. How do PPTs revealed through CM alone versus CM with SR videos differ with respect to richness, coherence and content (i.e. adequacy)? 2. Does the combined strategy reveal PPTs better? It can be concluded that both strategies made revealing the participants’ PPTs possible. The central theme ‘As a good professional…’ (Based on Nickles 2003) appeared useful in revealing participants’ interpretations of formal theories, work process knowledge, and notions about professional attitude for their future occupations. The concept maps hence suited the definition of PPTs. They could be coded on several variables independently, as indicated by the interrater reliabilities. The three aspects of adequacy appeared useful to cover both content and structure of a PPT. The aspects made grading the concept maps possible. The grades served the methodological focus of this article: Comparing the strategies’ ability to reveal PPTs. With regard to the first research question, it appeared that the CM + SR strategy obtained fewer points on all three aspects of adequacy. Hence, the second research question had to be answered negatively: Contradictory to what was expected, based on literature, the CM + SR strategy did not appear to reveal PPTs better, but it even seemed the other way round. This could be due to the procedure used in the CM + SR strategy. It could be that instruction about filming, watching the video, and constructing a concept map exceeds the limits of one’s working memory. This explanation is in line with cognitive load theory, which addresses how instruction should be designed in order not to exceed the limits of the learner’s working
123
796
P. M. Huijts et al.
memory load when he or she is processing instruction (Kirschner 2002). Maybe that adding SR to the strategy would not be problematic if participants were already familiar with CM. Another explanation could be that the CM + SR strategy reveals more what Argyris and Schön (1974) have termed ‘a theory-in-use’. This theory is implied by one’s behavior or action, and is often not the theory one explicitly supports. The latter is termed the espoused theory, and is more in line with a PPT (cf. Argyris and Schön 1974). Therefore, a theory-inuse could come forward, instead of a PPT, when one is confronted with one’s behavior on tape. Or, possibly, that tape did not trigger a full articulation of a participant’s PPT. As PPTs can be articulated at different levels of detail, just as espoused theories (cf. Argyris et al. 1985), brainstorming could lead to a more detailed level of articulation, and SR to a less detailed one. The present study was only small scale. The number of participants does obviously not allow for any generalization of the results obtained. The CM + SR strategy has not been proven to be less appropriate for revealing PPTs. It can only be concluded that this procedure of the CM + SR strategy appeared less able to reveal these participants’ PPTs. However, despite its small scale, the study has put interesting results forward to reveal PPTs, which are worthwhile of further investigation.
Appendix See Appendix Tables 7, 8, 9
Table 7 Description of administrators’ collective professional theory principles Domain
Principle
Description
Formal theories
Softwarea
Theories about software. Familiarity with programs and applications Basic theories obtained at school, where participants also receive their diplomas. Often mentioned as an overarching term Theories about hardware and computer parts. One is able to build a computer or replace the hard disc Theories about the network and the system. One is able to construct a network and make computers communicate. Therefore one is acquainted with a server Theories about programming. One is able to program and code. One is familiar with code language and occupational jargon One knows how to keep up with the latest developments; otherwise one becomes old-fashioned and leaps miles behind within weeks. One has to be open to new techniques by using sources like the internet and magazines or by visiting fairs
Basic
Hardware
Network
Programming
Work process knowledge
123
Keeping upa
Revealing PPTs
797
Table 7 continued Domain
Principle
Description
Professional role
One knows one is often the only expert in the area. Through ones’ knowledge one is able to offer ideas and solutions, offer alternatives and advise the customer/company One knows how to work in a team, work together, to have meetings and hold sessions. One has to know how a team operates and what is meant by team management and team building One knows how to plan and prioritize. Emergent cases have priority. One is able to estimate the time needed for an order and switch fast between tasks One has to have a studious and curious attitude. Educating oneself is asked for, doing research and trying on one’s own. One learns from others and through experiences, at the workplace or at school. One has the will to learn One has to have a social attitude, because one is not isolated behind a screen One has to be communicative, not only to colleagues, but also to the ordering customer. Therefore, one has to be able to explain in understandable language and listen well
Teamwork
Planning
Notions about professional attitude
Studiousa
Social
Communicative
a Core principle
Table 8 Description of designers’ collective professional theory principles Domain
Principle
Description
Formal theories
Multimediaa
Theories about multimedia, which entails design, color use, audio and video Basic theories obtained at school, where participants also receive their diplomas. Often mentioned as an overarching term One knows one is often the only expert in the area. Through ones’ knowledge one is able to offer ideas and solutions, offer alternatives and advise the customer/company One knows how to work in a team, work together, to have meetings and hold sessions. One has to know how a team operates and what is meant by team management and team building
Basic
Work process knowledge
Professional rolea
Teamwork
123
798
P. M. Huijts et al.
Table 8 continued Domain
Principle
Description
Structuring
One knows how to work following certain procedures or steps. One is familiar with the general procedure One knows how to adjust to the wishes of the company or customer. Sometimes the product needs adjustment too. In adjusting, one has to deepen one’s knowledge of the customer/company One knows how to keep up with the latest developments, otherwise one becomes old-fashioned and leaps miles behind within weeks. One has to be open to new techniques by using sources like the internet and magazines or by visiting fairs One has to be creative, have imagination to think of create unique designs. One has to be innovative One has to have a studious and curious attitude. Educating oneself is asked for, doing research and trying on one’s own. One learns from others and through experiences, at the workplace or at school. One has the will to learn One has to stand up for oneself en show initiative. One has to be able to work independently. One has to be independent and convincing, applying some marketing techniques in selling oneself
Adjusting
Keeping up
Notions about professional attitude
Creativea
Studious
Independent
a Core principle
Table 9 Codebook concept maps Variable
Label
Description
Coding
1
Term
None
2
N
Terms on the cards of the concept maps Participant
3
Strategy
Strategy used to reveal the PPT
1–16 1 = CM 2 = CM + SR
4
Training
Training course of the participant
1 = Administrator 2 = Designer
5
Category
Category the term was placed in
6
Nature
Nature assigned to the term
1 = Knowledge 2 = Traits 1 = Element 2 = Example
Only elements are coded further 7
Cluster
Number of elements in the cluster the element is in
0 = None n = Number of elements in cluster
123
Revealing PPTs
799
Table 9 continued Variable
Label
Description
Coding
8
Examples
Number of examples accompanying the element
0 = None n = Number of examples given
9
Importance
Importance attached to the element, based on location on the map
1 = Core element 2 = Outer circle element
10
Domain
Domain the element belongs to
1 = Formal theories 2 = Work process knowledge 3 = Notions about professional attitude
12
Principle
Collective professional theory principle the element matches witha
a See description of principles in Appendix Tables 7 and 8
References Appleton, K.: Problem solving in science lessons: how students explore the problem space. Res. Sci. Educ. 25, 383–393 (1995) Argyris, C., Schön, D.A.: Theory in Practice: Increasing Professional Effectiveness. Jossey Bass Inc., San Francisco (1974) Argyris, C., Putnam, B., McLain Smith, D.: Action Science. Concepts, Methods, and Skills for Research and Intervention. Jossey-Bass Inc., San Francisco (1985) Baartman, L.K.J., Prins, F.J., Kirschner, P.A., van der Vleuten, C.P.M.: Determining the quality of competence assessment programs: a self-evaluation procedure. Stud. Educ. Eval. 33, 258–281 (2007) Barney, J.A., Fredericks, J., Fredericks, M., Robinson, P.: Internalization of professional attitudes of physicians: implications for health care. Education 117, 530–538 (1997) Boreham, N.: Work process knowledge, curriculum control and the work-based route to vocational qualifications. Br. J. Educ. Stud. 50, 225–238 (2002) Buitink, J.: Leren onderwijzen tijdens een duale lerarenopleiding [Learning to teach through cooperative education]. Pedagogische Studiën 84, 37–54 (2007) de Bruijn, E.: Changing pedagogic and didactic approaches in vocational education in the Netherlands. From institutional interests to ambitions of students. Eur. J. Vocat. Train. 31, 27–37 (2004) Devenny, D.A., Zimmerli, E.J., Kittler, P., Krinsky-McHale, S.J.: Cued recall in early-stage dementia in adults with Down’s syndrome. J. Intellect. Disabil. Res. 46, 474–483 (2002) Edwards, J., Marland, P.: What are students really thinking?. Educ. Leadersh. 42, 63–68 (1984) Eraut, M.: Developing Professional Knowledge and Competence. pp. 19–58. Falmer Press, London (1994) Eurydice/Centre Européen pour le Développement de la Formation Professionnelle: National actions to implement lifelong learning in Europe. Brussels: Author (2001) Kirschner, P.A.: Cognitive load theory: implications of cognitive load theory on the design of learning. Learn. Instr. 12, 1–10 (2002) Lyle, J.: Stimulated recall: a report on its use in naturalistic research. Br. Educ. Res. J. 29, 861–878 (2003) Meijer, P.C., Verloop, N., Beijaard, D.: Multi-method triangulation in a qualitative study on teachers’ practical knowledge: an attempt to increase internal validity. Qual. Quant. 36, 145–167 (2002) Miles, M.B., Huberman, A.M.: Qualitative Data Analysis: An Expanded Sourcebook. Sage, Thousand Oaks (1994) Nickles, D.A.: The impact of explicit instruction about the nature of personal learning style on first-year students’ perceptions of successful learning. J. Gen. Vocat. Train. Courses 52, 108–144 (2003) Nijhof, W., van Esch, W. (eds).: Unravelling Policy, Power, Process and Performance. The Formative Evaluation of the Dutch Adult and Vocational Education Act. CINOP Expertise Centre, ’s Hertogenbosch (2004) Reber, A.S.: Implicit Learning and Tacit Knowledge: An Essay on the Cognitive Unconscious. Oxford University Press, New York (1993) Robson, C.: Real World Research: A Resource for Social Scientists and Practitioner–Researchers, 2nd edn, chap. 14. Blackwell, Oxford (2002)
123
800
P. M. Huijts et al.
Schaap, H., de Bruijn, E., van der Schaaf, M.F., Kirschner, P.A.: Students’ personal professional theories in competence-based vocational education: the construction of personal knowledge through internalisation and socialisation. J. Vocat. Educ. Train. 61(4), 481–494 (2009) Schepens, A., Aelterman, A., Van Keer, H.: Studying learning processes of student teachers with stimulated recall interviews through changes in interactive cognitions. Teach. Teacher Educ. 23, 457–474 (2007) Shavelson, R.J., Ruiz-Primo, M.A., Wiley, E.W.: Windows into the mind. High. Educ. 49, 413–430 (2005) Stough, L.M., Palmer, D.J.: Special thinking in special settings: a qualitative study of expert special educators. J. Special Educ. 36, 206–223 (2003) Wilensky, H.L.: The professionalization of everyone?. Am. J. Sociol. 70, 137–158 (1964) Winitzky, N., Kauchak, D.: Learning to teach: knowledge development in classroom management. Teach. Teacher Educ. 11, 215–227 (1995) Yin, R.K.: Case Study Research. Chap. 4, Conducting Case Studies. pp. 78–97. Sage, Beverly Hills (1984) Zanting, A., Verloop, N., Vermunt, J.D.: Student teachers eliciting mentors’ practical knowledge and comparing it to their beliefs. Teach. Teacher Vocat. Train. Courses 12, 725–740 (2001) Zanting, A., Verloop, N., Vermunt, J.D.: Using interviews and concept maps to access mentor teachers’ practical knowledge. High. Educ. 46, 195–214 (2003)
123
