The relationship between continuing education and perceived ...

Original Research

The Relationship Between Continuing Education and Perceived Competence, Professional Support, and Professional Value Among Clinical Psychologists

STACY BRADLEY, MA; MARTIN DRAPEAU, PHD; JACK DESTEFANO, EDD Introduction: Continuing education is one of the means by which professionals maintain and increase their level of competence. However, the relationship between continuing education and the professional’s sense of personal competence and other practice-related variables remains unclear. This study examined practicing psychologists’ continuing education activities and how these relate to feelings of perceived competence, professional value, and professional support. Methods: Psychologists (n = 418) licensed to practice in Quebec were surveyed by pencil-and-paper mail-in survey concerning their continuing education activities, as well as their perceptions of their competence in practice, and their feelings of being professionally valued and professionally supported. Results: Results indicated that feelings of competence in practice were related to professional reading, taking courses/workshops, years being licensed, and attending psychology conferences/conventions. Feelings of professional value were related to age and participating in psychology networking groups, and feelings of professional support were related to participating in case discussion groups, supervision groups, and psychology networking groups. Discussion: The results showcase the complexity of professional development. Although relationships were found between continuing education activities and the 3 factors of interest, these relationships were moderate. Findings are discussed in the context of their value to individual psychologists, as well as to psychology licensing and regulatory boards, such as promoting participation in those activities related to feelings of competence and support. Key Words: continuing education, competence in psychology, professional value, professional support, perceived competence, professional practice, psychology, psychologist

Introduction The maintenance of competence is an essential principle in all codes of ethics governing professionals offering health and psychological services. The American Psychological Association Ethical Principles of Psychologists and Code of Conduct (2002) states that practicing psychologists should Disclosures: The authors report none. Ms. Bradley: Doctoral student, McGill University; Dr. Drapeau: Associate Professor, McGill University; Dr. DeStefano: Faculty Lecturer, McGill University. Correspondence: Martin Drapeau, ECP—McGill University, 3700 McTavish, Montreal, Quebec, Canada, H3A 1Y2; e-mail: martin.drapeau@ mcgill.ca. C 2012 The Alliance for Continuing Education in the Health Professions, the

Society for Academic Continuing Medical Education, and the Council on Continuing Medical Education, Association for Hospital Medical Education. r Published online in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/chp.21120

“provide services, teach, and conduct research with populations and in areas only within the boundaries of their competence, based on their education, training, supervised experience, consultation, study, or professional experience” and “undertake ongoing efforts to develop and maintain their competence.”1(pp3,4) Similar regulations exist in all states, provinces, and territories. For example, in Quebec, where the present study was conducted, the Code of Ethics of the Ordre des Psychologues du Québec (Order of Psychologists of Quebec) states that “psychologists must develop, perfect and maintain their knowledge and skills in the field in which they carry on their professional activities” (Article 39).2 When psychologists practice outside their domains of competence they put clients at risk of harm, which can result not only in a loss of trust in the clinician, but also in the profession as a whole.3 Therefore, establishing professional competence and maintaining it throughout a professional’s career continues to be a primary concern of training institutes and of regulatory and licensing boards,4,5 and of clinicians themselves.

JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS, 32(1):31–38, 2012

Bradley et al.

Competency, whether in psychology or elsewhere, is a complex concept that includes not only acquiring adequate knowledge, skills, and abilities, but maintaining these throughout one’s career. It has been estimated that half of a psychologist’s knowledge base will become outdated within the first 10–12 years of practice.6 With the constant evolution of client needs, and the advancement of clinical research and development, psychologists have an ethical responsibility to stay current. Advancements in the field put pressure on licensing boards and other professional groups to ensure that the continuing education (CE) initiatives undertaken by their members are consistent with the mandate of the profession and relevant to the needs of the professional. The field of psychology has been dedicated to the topic of CE for over three decades.3,7,8 Surprisingly, however, researchers continue to comment on the lack of data concerning current trends in CE utilization among psychologists,9 and more importantly on the lack of information concerning the effects of different CE modalities on practicing clinicians.8,10,11 Early research on CE focused on the need for and availability of different CE activities.3,4,7 Following this, researchers turned their attention toward assessing preferences for CE, or for those activities considered most valuable,12−18 although some would argue that research in this area remains scarce.11 Previous research on psychologists has indicated that professional reading and condensed courses/workshops (ie, 1 or 2 days) are highly preferred or valued CE activities.12−18 Additionally, it has been reported that professional reading is considered to improve competence and to help in developing new competencies14 and is viewed as having the greatest utility and ease of access, and most easily fits into the professionals’ lifestyle.17 Other CE activities reported to be valued include supervision or case discussion with other professionals,16,17 the latter of which has been reported as the most cost-effective, and in general the most rewarding.17 Practicing psychologists also report participating in CE activities when not required,18,19 believe that these activities make them more efficient clinicians,19 and feel that they can implement what was learned into practice.9,10,19 Two recent studies have shown high levels of satisfaction with CE activities, as the majority of participants in both studies rated their CE experiences quite favorably and reported moderate to high levels of learning.9,10 Although these are promising findings, what remains to be seen is the relationship between differing CE activities and the psychologists’ perceived sense of competence. Other important factors to be considered in addition to competence are perceived feelings of professional value and professional support. The literature on professional burnout indicates that the risk factors associated with professional burnout for health care providers include a lack of coworker or administrative 32

support, office politics, coworker stress, excessive hours, unrealistic expectations, lack of appreciation or recognition, and inadequate financial rewards.20−23 Most of these can be placed into 1 of 2 categories: a perceived lack of support from other professionals and feelings of being professionally undervalued. Professional burnout can result in anger, fatigue, boredom, anxiety, depression, social isolation, increased procrastination, decreased motivation, absenteeism, feelings of ineffectiveness, and, finally, impairment through unethical behaviors or incompetence.24−27 Due to the myriad occupational hazards that burnout can cause, it is essential to better understand the specific activities and factors that are related to feelings of being professionally supported and valued, as these are integral in the prevention of burnout. For example, participation in CE activities by family physicians has been found to be negatively associated with job stress and burnout, and positively associated with job satisfaction.28 Additionally, opportunities for professional development (ie, new training, workshops) among mental health workers have been considered valuable in protecting against negative work outcomes.29 However, there is a lack of information in the field of psychology on how specific facets of job satisfaction or professional development, such as feeling professionally competent, valued and supported, are affected by different CE activities Most previous studies focused on attitudes toward mandatory CE,9,19,30,31 and general information concerning CE participation, interest, and satisfaction.15,32−34 However, little is known about the effects different formats of CE delivery may have on the professional. The objective of the present study was to address this existing gap, to contribute much-needed knowledge in this field, and gain a greater understanding of the relationship between specific CE activities and diverse facets of professional development. More specifically, this study examined practicing psychologists’ continuing education activities and how these relate to feelings of perceived competence, professional value, and professional support.

Methods Data Source In 2006, the Ordre des Psychologues du Québec (OPQ), which is the licensing body for psychologists in Quebec, conducted an anonymous paper-and-pencil survey of its members, covering a number of different areas related to professional practice. The 145-item survey included a demographic section, as well as a section covering the psychologists’ participation in several types of CE activities (TABLE 1), a section on their interest for different themes and topics for future CE activities, and a section on their preferences for different types of CE providers (eg, hospitals, associations). The survey also included a section to assess the participants’

JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS—32(1), 2012 DOI: 10.1002/chp

Continuing Education in Psychology TABLE 1. Demographic and Continuing Education Items Used as Predictor Variables

Item #

Description

1

Age

2

Sex

3

Number of years licensed with the OPQ

6

Highest level of university education in psychology

7

What is your main work setting (public/private)

Respondents were required to indicate to what extent they use each of the following CE modality using a 6-point Likert scale ranging from 1—never to 6—often. 37

Professional reading

38

Attending psychology conferences and/or conventions

39a 40

a

University courses and workshops or seminars Courses and workshops and seminars in the private sector

41a

Courses/workshops/seminars in the workplace

42

Supervision meetings

43

Structured training programs

44

Structured peer activities (ie, case discussion groups)

45

Distance training

46

Mentoring activities

47

Personal research and publications

48

Other professional works (eg, writing manuals, designing courses)

49

Participating in psychology networking groups

The following CE modalities had an open rating system: 50

Hours spent on professional reading per year

52

Hours spent on CE excluding professional reading

a

Items 39, 40, and 41 were combined into 1 variable labeled “courses” and the average rating of the 3 combined items was computed.

sense of professional competence, professional value, and professional support received from their professional environment (items listed in TABLE 2). An invitation to participate in the survey and the survey were mailed by the OPQ to 2000 psychologists randomly selected from the pool of 8210 psychologists licensed to practice in Quebec. Those who agreed to participate were asked to return the completed questionnaire to the OPQ; 653 returned the completed survey, for a total response rate of 33%. Sample In accordance with the policy on support of research projects of the OPQ, a formal request was submitted by the authors to the OPQ for use of their survey data. This request described the goal of our study, which was to examine the relationship between clinical psychologists’ participation in CE (15 items) and their sense of professional competence

(5 items), professional value (6 items), and professional support received from their professional environment (5 items; see TABLES 1 and 2). Of the 653 who returned their survey to the OPQ, 418 self-identified as clinical psychologists and so met the inclusion criteria for this study. This sample of 418 was representative of the clinical psychologists licensed in Quebec in terms of age groups, with participants’ ages ranging from 30 to 40 (28%), 40 to 50 (25%), and 50 to 60 (32%); sex, with 74% female and 25% male; and education level, with 68% of the participating psychologists in this sample licensed with a master’s degree, and 18% with a doctoral degree (the doctorate became the required level of training in Quebec in 2006). Data Analysis First, the 16 self-perception items (5 items for self-perception of professional competence, 6 for professional value, and 5 for professional support) were subjected to a factor analysis


33

Bradley et al. TABLE 2. Factor Analysis of Dependent Variables

Item Description

Competence

Value

Support

0.78

0.27

0.02

I feel I have integrated the ethical guidelines.

0.74

−0.08

0.17

I have the sense of being competent in my field.

0.71

0.30

0.02

I feel competent to respond to the demands of my practice.

I feel competent to respond to the evolution of needs in my practice.

0.71

0.18

0.20

I feel well informed in regards to my profession in general.

0.69

0.15

0.12

I feel I am adequately paid as a psychologist.

0.03

0.67

−0.02

My workload is adequate.

0.19

0.67

0.05

I am socially valued for my profession.

0.16

0.63

0.08

I find my practice interesting and stimulating.

0.36

0.53a

0.23

I feel a sense of belonging to my profession.

0.26

0.46a

0.32

I feel I am in a constant learning process.

0.30

0.36a

0.17

I have good relationships with my colleagues from other disciplines.

0.08

−0.07

0.77

I have good relationships with my psychology colleagues.

0.08

0.18

0.73

I feel supported (emotionally) within my practice.

0.12

0.37

0.64

My work setting facilitates my practice as a psychologist.

0.14

0.43

0.55

It is easy for me to use new technologies.

0.22

−0.36

0.48a

a

Indicates the item was removed due to high side loadings.

to confirm that this section of the survey addressed the participants’ sense of professional competence, value, and support. Three stepwise multiple regression analyses were then performed independently for professional competence, value, and support. For each regression analysis the demographic variables and the CE activities were entered as predictors (18 predictors), and each of the 3 factors of perception of self in the context of practice (perception of professional competence, of professional value, and of professional support received) were entered as dependent variables. Demographic variables that were nominal variables (eg, gender, work setting) were dummy coded.

Results of the stepwise multiple regression analysis showed that 4 items significantly predicted feelings of professional competence (F(4, 295) = 16.42, p < 0.001), accounting for a total of 18% of the variance: professional reading, courses, years licensed, and conferences/conventions (see TABLE 3). Results of the second regression analysis indicated that two items significantly predicted professional value (F(2,297) = 10.19, p < 0.001), including age and psychology networking groups, which accounted for a total of 6% of the variance (see TABLE 3). Professional support was explained by three items (F(3, 288) = 23.9, p < 0.001), including case discussion groups, supervision groups and psychology networking groups, for a total of 20% of explained variance (see TABLE 3).

Results In accordance with the intentions of the surveyors to assess the respondents’ sense of professional competence, professional value, and professional support, results showed that a 3-factor solution was most parsimonious, with all 3 factors having Eigenvalues greater than 1.5, and each explaining at least 10% or more of the variance, for a total of 50% cumulative variance. However, for research purposes, we opted to drop 3 of the 6 items assessing sense of professional value and 1 item assessing sense of professional support because of high side loadings (see TABLE 2). 34

Discussion Three types of CE activities significantly predicted feelings of professional competence, including professional reading, courses, and attending and/or participating in psychology conferences/conventions. Professional reading was the strongest predictor of the participants’ sense of professional competence, accounting for over 60% of the variance explained by all predictors. This may help explain in part why reading materials, such as theoretical or practical books and articles, have been reported as ideal sources of information


Continuing Education in Psychology TABLE 3. Summary of Regression Analyses for Variables Predicting Competence, Professional Value, and Professional Support

Cumulative R Square

B

SEB

β

t

p

Professional Reading

0.11

0.66

0.12

0.30

5.54

0.00

Courses

0.14

0.14

0.05

0.14

2.66

0.01

Years Licensed

0.17

0.22

0.07

0.16

3.07

0.00

Conferences/Conventions

0.18

0.24

0.10

0.13

2.33

0.02

Age

0.04

0.39

0.13

0.17

3.02

0.00

Psychology Networking Groups

0.06

0.22

0.08

0.16

2.7

0.01

Case Discussion Groups

0.17

0.60

0.10

0.35

6.30

0.00

Supervision Meetings

0.19

0.20

0.08

0.13

2.37

0.02

Psychology Networking Groups

0.20

0.18

0.08

0.12

2.16

0.03

Variables

Competence

Professional Value

Professional Support

for practicing psychologists.15,17 Twenty years ago, it was estimated that psychologists were publishing an average of 100 articles per day.35 As of 2010, PsychINFO alone contains over 2.8 million records. The wide accessibility of literature and the explosion of online knowledge may be allowing psychologists to access information tailored to their practice needs more quickly and efficiently. The potential availability of information, as well as the greater control over the focus and content of the material as opposed to other CE activities, may in part be contributing to greater feelings of competence. These assertions, however, remain speculative and require further investigation. Results also indicated that taking courses/workshops is related to feelings of professional competence. Single courses, lectures, and workshops are predominant formats for CE activities,9,36 and although there is a positive relationship between courses/workshops and perceived competence, research criticizing lectures or workshops that are didactic and noninteractive must be taken into consideration. For example, Davis and colleagues37 indicate that lectures or workshops of a didactic nature do not serve to increase physician performance; therefore, courses/workshops that incorporate active participation such as role-playing and case discussion should be promoted over those of a didactic nature. It may be that the courses/workshops participated in by those in this sample are incorporating these aspects that have been shown to be related to increased competence. However, the present study investigated self-perceived competence, not an objective measure of competence. Research on mental health professionals in general suggests that these professionals tend to overestimate their competence in a number of settings.38,39 Furthermore,

research on self-assessment across professional domains indicates that much self-assessment is flawed, and that individuals tend to overrate their performance and capabilities.40,41 A meta-analysis of perceived competence versus actual competence within the medical field shows some evidence of a positive relationship between perceived and actual competence in practice; however, the overall consensus is the lack of ability to correctly assess one’s competence.42 Although it is of value to understand those CE activities related to perceived competence, given this information, investigating the link between CE and objective measure of competence is of great importance. Being licensed with the OPQ for a longer period of time was related to greater feelings of professional competence. This is in line with previous research concerning perceived therapeutic mastery.43 It is noteworthy, however, that years licensed in this sample was only, at best, a moderate predictor of the respondents’ sense of professional competence, and that professional reading and continued course work were in fact stronger predictors of perceived competence. This speaks to the complexity of professional development. Although it has been documented that experienced professionals feel comfortable and competent working with clients, and that much learning comes with direct exposure to clients,44 our results suggest that years licensed is not the strongest predictor of perceived feelings of competence. This may be due to the fact that in the present survey participants were asked about the number of years licensed with the OPQ, which may not necessarily mean more experience with clients. To obtain a more accurate measure of experience it would be useful to ask participants how many years they have


35

Bradley et al.

been in practice and their average client contact hours per week. The second factor, professional value, was positively related to age and participation in a psychology networking group. However, these variables accounted for no more than 6% of the variance in this factor, which suggests that CE activities are not strongly related to personal feelings of professional value. Older professionals may nonetheless have reached a point in their career where they feel adequately paid and are respected by their younger colleagues. On the contrary, younger professionals, often having undergone more years of education and perhaps greater debt due to increasing tuition costs for doctoral programs,45 may potentially have greater expectations both in their pay and work conditions than previous generations. Feelings of professional support were predicted by participation in case discussion groups, supervision groups, and psychology networking groups. It has been noted in the literature that support comes from positive interactions with colleagues in formal structured settings, as well as informal personal support such as that from family and friends.26,27,46,47 Peer relationships in the broader professional community have also been cited as essential sources of support for those in the helping field.48 Professional support is especially important in the helping professions, as these professionals are at a great risk of experiencing professional burnout—a condition that has significant negative consequences on professional effectiveness.24−26 The most important factor in the prevention of burnout is support. Skovolt27 recommends that practicing psychologists avoid isolation by seeking professional support in peer consultation groups, mentoring relationships, supervision, and informal support from colleagues. Receiving emotional comfort, assistance with difficult cases, and recognition for one’s work can aid in preventing isolation or withdrawal from others, and help to ensure confidence in one’s abilities as an effective professional. Promoting participation in professional groups as a mode of CE, and making these available to psychology professionals, could be beneficial. Additionally, it may be advantageous to design CE activities in such a way as to foster feelings of support (such as allowing time for discussion with other colleagues during courses/workshops), and awarding CE credits for activities such as peer consultation. The fact that there were types of CE activities that were unrelated to the 3 factors of interest is, in our view, an essential consideration. With the great amount of emphasis placed on the importance of supervision for development as a clincian,49−52 it is interesting to note that participating in supervision groups, as a supervisee receiving feedback on his or her clinical practice, was not related to feelings of professional competence. This finding can be interpreted in many ways. Those individuals undergoing supervision more 36

Lessons for Practice • Four continuing education activities signifi-

cantly predicted feelings of professional competence: professional reading, courses, years licensed, and attending and/or participating in psychology conferences/conventions. Professional reading was the strongest predictor of participants’ sense of professional competence. • When looking at the results of this model in gen-

eral, it becomes apparent that although these continuing education activities are related to feelings of competence, this relationship remains moderating. • Continuing education activities are not strongly

related to personal feelings of professional value. • Promoting participation in professional groups

as a mode of continuing education, and making these available to psychology professionals, could be beneficial to all parties involved in therapeutic practice, the professionals, the clients, and the licensing and regulatory boards. • Interestingly, with the great amount of impor-

tance placed on supervision, it is worthwhile to note that participating in supervision groups was not related to feelings of professional competence. • Those activities in which one has more control,

such as professional reading, seems to have the greatest impact on one’s perception of personal competence. • From the perspective of the respondents, it

would be advantageous for licensing and regulatory boards to award continuing education credits to individuals for reading materials most pertinent to their practice and to promote a certain level of professional reading. • It would be relevant for licensing and regula-

tory boards to promote participation in case discussion groups and networking groups, as the results of this study show the positive impact these activities have on professional support, which in turn may prevent emotional exhaustion.


Continuing Education in Psychology

frequently in the present sample may be those in the beginning stages of their career; therefore, their feelings of competence are lower than those in the later years of their career who may not undergo supervision as regularly. It may also be the case that informal supervision among colleagues that is not held in an organized context was not reported and therefore cannot be accounted for, or that the manner in which supervision is being conducted may not foster feelings of competence. In the present survey, doing research, publishing, and writing manuals were not predictive of any of the factors; however, this may be due to the fact that the majority of the sample stated that they do not conduct research or write such manuals. A sample more representative of psychologists who do these activities may indicate different results. Limitations When looking at the results, it becomes apparent that although these CE activities are related to feelings of competence, the relationship between these activities and perceived competence remains moderate. Other factors not assessed here that may lead to feelings of competence could include feedback from clients, peer recognition, or teaching novice professionals. As mentioned previously, an important consideration is that the survey focused on the participants’ perception of professional competence, value, and support. As such, none of these constructs was objectively and independently assessed, which is particularly important in the case of professional competence. Future research will need to determine the extent to which CE has an effect on actual competence and success in service delivery. Conclusion The results of the present study highlight the complexity of professional development. The 3 factors of interest— perceived competence, professional value, and professional support—are somewhat difficult to predict from one’s CE activities. What does become clear is that professional reading seems to have the greatest impact on one’s perception of personal competence. It therefore may be advantageous for licensing and regulatory boards to award CE credits to individuals for reading those materials most pertinent to their practice and to promote a certain level of professional reading. Furthermore, it may also be relevant for licensing and regulatory boards to promote participation in case discussion groups and networking groups; the results of this study show the positive impact these activities have on professional support, which in turn may prevent emotional exhaustion.26,27 However, continued research is needed to understand those activities that specifically lead to feelings of professional

value, as there was only a weak relationship between the CE activities and feelings of professional value.

Acknowledgment The authors thank the Ordre des Psychologes du Québec for their permission to use the data.

References 1. American Psychological Association. APA ethical principles of psychologists and code of conduct. Washington, DC: American Psychological Association; 2002. 2. Order of Psychologists of Quebec. Code of ethics. Available at: http://www.ordrepsy.qc.ca/en/protection/code ethics.html. Accessed April 9, 2010. 3. Jensen RE. Competent professional service in psychology: the real issue behind continuing education. Prof Psychol. 1979;10:381–389. 4. Jones NF. Continuing education: a new challenge for psychology. Am Psychol. 1975; 30:842–847. 5. Kaslow NJ. Competencies in professional psychology. Am Psychol. 2004;59(8):774–781. 6. Dubin SS. Obsolescence or lifelong education: a choice for the professional. Am Psychol. 1972;27:486–498. 7. Ross AO. Continuing professional development in psychology. Prof Psychol. 1974;5(2):122–128. 8. Webster, TG. National priorities for the continuing education of psychologists. Am Psychol. 1971;26:1016–1019. 9. Neimeyer GJ, Taylor JM, Philip D. Continuing education in psychology: patterns of participation and perceived outcomes among mandated and nonmandated psychologists. Prof Psychol: Res Pr. 2010;41(5):435– 441. 10. Neimeyer GJ, Taylor JM, Wear DM. Continuing education in psychology: outcomes, evaluations, and mandates. Prof Psychol: Res Pr. 2009;40:617–624. 11. VandeCreek L, Knapp S, Brace K. Mandatory continuing education for licensed psychologists: its rationale and current implementation. Prof Psychol: Res Pr. 1990; 21:135–140. 12. Schoenfeld LS, Kobos JC, Burstein AG. Continuing education: a professional dilemma. Prof Psychol. 1974;5(1):1–2. 13. Wolkon GH, Peterson CL, Gongla P. University-based continuing education and mental health system change. Am Psychol. 1982;37(8):966– 970. 14. McNamara JR. Patterns of continuing education for Ohio psychologists: a survey of interest and activities (1972–1974). Prof Psychol. 1977;8:368–376. 15. Donohoe MC, Atkinson MB, Scott WO, Lemoine RL. Continuing education among Louisiana psychologists. South Psychol. 1985;2(3):33– 40. 16. Morrow-Bradley C, Elliott R. Utilization of psychotherapy research by practicing psychotherapists. Am Psychol. 1986;41(2):188–197. 17. Allen GJ, Nelson WJ, Jr, Sheckley BG. Continuing education activities for Connecticut psychologists. Prof Psychol: Res Pr. 1987;18:78–80. 18. Brown RA, Leichtman SR, Blass T, Fleisher E. Mandated continuing education: impact on Maryland psychologists. Prof Psychol. 1982;13:404–411. 19. Sharkin BS, Plageman PM. What do psychologists think about mandatory continuing education? A survey of Pennsylvania practitioners. Prof Psychol: Res Pr. 2003;34:318–323.


37

Bradley et al. 20. Hafen BQ, Karren KJ, Frandsen KJ, Smith NL. Mind/body health: the effects of attitudes, emotions, and relationships. Boston, MA: Allyn & Bacon; 1996. 21. Maslach C, Leiter MP. The truth about burnout. San Francisco, CA: Jossey-Bass; 1997. 22. Suran B, Sheridan E. Management of burnout: training psychologists in professional life-span perspectives. Prof Psychol: Res Pr. 1985;16:741– 752. 23. Watkins C. Burnout in counselling practice: some potential professional and personal hazards of becoming a counsellor. Pers Guid J. 1983;61(5):304–308. 24. Freudenberger HJ. Staff burnout. J Soc Issues. 1974;30:159–165. 25. Freudenberger HJ. The staff burn-out syndrome in alternative institutions. Psychother: Theor Res. 1975;12(1):73–82. 26. Maslach C, Goldberg J. Prevention of burnout: new perspectives. Appl and Prev Psychol. 1998;7:63–74. 27. Skovholt TM. The resilient practitioner: burnout prevention and selfcare strategies for counselors, therapists, teachers, and health professionals. Boston, MA: Allyn & Bacon; 2001. 28. Kushnir T, Cohen AV, Kitai E. Continuing medical education and primary physicians’ job stress, burnout and dissatisfaction. Med Educ. 2000;34:430–436. 29. Acker GM. The effect of organizational conditions (role conflict, role ambiguity, opportunities for professional development, and social support) on job satisfaction and intention to leave among social workers in mental health care. Community Ment Health J. 2004;40:65–73. 30. deGroot G. Should CE be mandated for every psychologist? APA Monitor. 1994;25(6):41. 31. VandeCreek L, Brace K. Mandatory continuing education in the health professions. J Train Pract Prof Psychol. 1991;5:23–36. 32. Brown RA, Briley PL. Continuing education activities for Maryland psychologists: a survey. Psychother: Theor Res. 1979;10:285–292. 33. Katahn M. A survey of the interest in continuing education among mental health professionals in the southeastern states. Am Psychol. 1970;25:941–952. 34. Lewinsohn PM, Pearlman S. Continuing education for psychologists. Prof Psychol: Res Pr. 1972;3(1):48–52. 35. Thorngate W. The economy of attention and the development of psychology. Can Psychol. 1990:31:262–271. 36. Daniels AS, Walter DA. Current issues in continuing education for contemporary behavioral health practice. Adm Policy Ment Health. 2002;29(4/5):359–376. 37. Davis D, O’Brien MAT, Feemantle N, Wolf FM, Mazmanian P, Taylor-Vaisey A. Impact of formal continuing medical education: do

38

38.

39.

40.

41.

42.

43.

44.

45. 46. 47. 48.

49. 50.

51. 52.

conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA. 1999;282:867–874. Hannan C, Lambert MJ, Harmon C, et al. A lab test and algorithms for identifying clients at risk for treatment failure. J Clin Psychol: In Session. 2005;61(2):155–163. Lambert M. Presidential address: what we have learned from a decade of research aimed at improving psychotherapy outcome in routine care. Psychother Res. 2007;17(1):1–14. Dunning D, Heath C, Suls J. Flawed self-assessment: implications for health, education, and the workplace. Psychol Sci. 2004;5(3):69– 106. Eva KW, Regehr G. Self-assessment in the health professions: a re-formulation and research agenda. Acad Med. 2005;80(10): 46–54. Davis D, Mazmanian P, Fordis M, Van Harrison R, Thorpe K, Perrier L. Accuracy of physician self-assessment compared with observed measure of competence. JAMA. 2006;296(9):1094–1102. Orlinsky D, Ronnestad MH, Ambuhl H, et al. Psychotherapists’ assessments of their development at different career levels. Psychother Theor Res Pract Train. 1999;36(3):203–215. Ronnestad MH, Skovholt TM. The journey of the counselor and therapist: research findings and perspectives on development. J Career Dev. 2003;30(1):5–44. Norcross JC, Kohout JL, Wicherski M. Graduate study in psychology: 1971–2004. Am Psychol. 2005;60(9):959–975. Grosch WN, Olsen DC. When Helping Starts to Hurt: A New Look at Burnout Among Psychotherapists. New York, NY: W. W. Norton; 1995. Maslach C. Burnout: The Cost of Caring. New York, NY: Prentice Hall; 1982. Skovholt TM, Mullenbach MA. Emotional wellness and professional resiliency of master therapists. In: Skovholt TM, Jennings L, eds. Master Therapists: Exploring Expertise in Therapy and Counseling. Boston, MA: Allyn & Bacon; 2004:77–106. Holloway EL, Neufeldt SA. Supervision: its contributions to treatment efficacy. J Consult Clin Psychol. 1995;63:207–213. Lambert MJ, Ogles BM. The effectiveness of psychotherapy supervision. In: Watkins CE, Jr., ed. Handbook of Psychotherapy Supervision. New York, NY: Wiley; 1997:421–446. Milne DL, James I. A systematic review of effective cognitive– behavioural supervision. Br J Clin Psychol. 2000;39:111–129. Ray D, Altekruse M. Effectiveness of group supervision versus combined group and individual supervision. Couns Educ Superv. 2000;40:19–31.
