Curriculum evaluation - Nurse Education Today

3 downloads 0 Views 606KB Size Report
Chris Sconce and John Howard. This paper describes the development of a College-wide curriculum evaluation strategy. The strategy has been implemented ...
Curriculum evaluation -

a new approach

Chris Sconce and John Howard

This paper describes the development of a College-wide curriculum evaluation strategy. The strategy has been implemented on all courses offered by the Lancashire College of Nursing and Health Studies. An extensive review of the literature revealed that no evaluation models were suitable for use across all College courses. This was due to their over-emphasis on either the process, or the product of education; none of the published evaluation models adequetely address both of these components of professional learning. The criteria required, of both the model and the evaluation tools, included the need for a measurable component, inclusiveness, simplicity and practicality. In order to meet these a new model of evaluation and appropriate data collection tools/analytical systems were developed. This Professional Process and Product Model forms the basis for the described strategy. The model is easy to use due to its recognition of the professional nature of those involved in nurse education. this professional status, and the accountability which it implies, ensures that only relevant and significant information is included in the formal evaluation of courses. This ensures an ability to meet future contractual quality standards which will be a requirement for providers in the health care of the 1990s.

development,

INTRODUCTION Traditionally the nursing curriculum has been seen as a complex entity which is difficult to fully understand and thus evaluate. are still struggling suming procedures, 1990).

‘many nurse educators

with inappropriate,

time-con-

that yield little reward’ (Hogg

Such an approach

able. What is required, qualitative

information

is obviously unaccept-

is a system which provides for

use in curriculum

The

Chris Sconce BSc (How) RGN RHV RNT, Head of Quality Assurance 81 Public Relations, John Howard BSc (Hans) RGN RSCN RNT, Research and Information Manager, The Lancashire College of Nursing & Health Studies, Education Centre, Watling St Rd, Preston, Lanes, PR2 4DX, UK (Requests for offprints to CS) Manuscript accepted 17 November 1993

that

approach external

of the

which providers of

are now subject to.

introduction

requires

indicators

in order to meet the con-

tractual quality specifications education

of

the

internal

market

Colleges

adopt

a customer

to education.

Thus

the views of the

customers

(health care consortia,

led trusts,

students, health authorities, etc.) are of prime importance in assuring fitness for purpose. This is not to deny the role of educationalists motion

280

and measurable

quality of the product

recognise

of excellence

within practice,

the implications

in the promerely to

of the market

place

where customers are free to place contracts where they wish. Quality in nurse education number of ways including, ic excellence,

may be seen in a

attainment

practical competence

of academ-

or a combina-

NURSE EDUCATION

TODAY

1

Strategies

Fig. 1 Stufflebeam’s

CIPP model of curriculum

tion

of these.

sion

of a quality

demand

As contracting

evidence

evaluation.

the inclu-

purchasers

that educationalists

ing all criteria described only meaningful

requires

specification,

281

will

are achiev-

in that specification.

The

way that this can be achieved

is

through a process of evaluation.

including

any unintended

outcomes

of learning

and are thus inclusive rather than exclusive. Stufflebeam

(1969)

which concentrated

adopted

an

on the decision

approach

making pro-

cesses involved in the design and delivery of a curriculum. four

These processes

categories:

he allocated

to one of

Context-Input-Process-Product

(CIPP). Context

THE LITERATURE

evaluation

of curriculum Possibly the earliest attempt at curriculum tion

was provided

by Tyler

in the

evalua-

1930s.

approached

curriculum

design

development

of objectives

and their concomitant

evaluation,

the purpose

being

through

He

‘to see how far

these objectives are actually being realised’ 1949).

Such a behaviouristic

limit the evaluation level.

However,

the

measurability. The ‘Countenance

approach

of curricula beauty

the

(Tyler

tends to

to a mechanistic

is in its inherent

ment of curricular demands in nature

Model’

(Stake 1967) strove This framework

with the type

It involves the develop

objectives

of a changing

in the light of the

society. Whilst formative

it is also a component

evaluation.

Input

the resources

evaluation

required

of summative

is concerned

tion

about

the

various

Process evaluation

inter-relationship

final outcome

of informa-

strategies

proposed.

deals directly with the opera-

tion and implementation Finally product

of the curriculum of theory

evaluation

and

is concerned

of the curriculum;

with the

it assesses the

and out-

extent to which objectives have been achieved. model can be seen as cyclical, thus facilitating

the

ongoing

tions.

antecedents,

individual’s

inherent

Transactions

education

including

ment strategies. academic

refer

transactions

abilities

and

to

processes

the

motivaof

teaching methods and assess-

The outcomes

achievements

or

are measured practical

as

abilities,

By addressing process

modification

these processes of curriculum

(Fig. 1). Although

and

practice.

comes. Antecedents are those conditions which existed prior to the learning experience, such as

encompassed

with

to achieve the curriculum

objectives and involves the acquisition

the

to widen the basis for evaluation.

is concerned

offered.

sequentially, evaluation

the an and

elegant in its sim-

plicity and the ease with which it may be adapted to a variety of situations the inherent measuring and recording context

difficulty of and input

282

NURSE EDUCATION

evaluation

TODAY

may lead to a disregard

for these ele-

ments. (1972)

suggested the concept of ‘Goal-

Free’ evaluation,

Striven

this being based on the belief

that pre-specified

objectives

would

acknowledgement

of arising

issues and militate

against

the full utilisation

tion. Thus all learning

inhibit

of available

is addressed,

the

informa-

not simply

ment,

seem

These

ranged

unwieldly

(Crotty

& Bignell

from

for

widespread

103item

1988)

usage.

questionnaires

to personal

interviews

each lasting up to 90 minutes (Chambers1988). Thus, from the literature a number of key criteria emerge. These include: mearsurability inclusiveness,

of data,

simplicity and ease of use, object-

ivity, and the ability to prioritise issues raised.

that which was originally intended. During

the

1970s

product

of learning

advent

of

approaches became

person-centred

Rogers

(1969),

the

with the

curriculum

models

based on the work of theorists (1978),

involving

unpopular

Friere

such as Knowles (1973)

and Gange

(1977). Evaluation as illumination Parlett and Hamilton Evaluation’

was first proposed by

in 1972. Their ‘Illuminative

model uses description

tation rather than measurement

and interpre-

as its base. Using

THE PROFESSIONAL PROCESS AND PRODUCT MODEL OF EVALUATION Professional

evaluation

The professionalism

of those involved in the plan-

ning

of the

and

ensures

delivery

that context

observation of, and interviews with, those involved in the curriculum, as well as analysis of documen-

Stufflebeam

tation and a variety of other methods,

of a professional

evaluators,

to identify

enables the

and contextualise

issues

raised. Thus it is possible to address the issues in priority

order

to facilitate

curriculum’

develop-

ment. Whilst this is of particular importance strategy intended ety of courses,

for a

for everyday use on a wide vari-

the inability to easily measure

quality of the curriculum

renders

unsuitable for use in contractual

the

this approach

situations.

More recent attempts at curriculum

(1969),

and

United

drawings might indicate regarding

(Odro

1992),

of students’

their personal

thoughts

the course and their level of personal

achievement.

Whilst this may be of use to provoke

discussion and interaction tivity of interpretation

formatively, the subjec-

and, once again, the inabil-

Central

the light of the changing acknowledges selection

of nursing

courses.

The

which in

demands of society and

responsibility

for the professional’s

educational

programme.

When applied to nurse education

the areas of

and input are clearly integral to the proof those involved. Thus a three stage

model of evaluation emerges

(Fig. 2). This model

shows the linear progression

of the educational

experience

from the professional

body through

the process itself to the product of learning. experience the

professionals

(including

in that experience,

education

This

receives input from and is analysed by

those professionals

grids in the evaluation

to examine

and proper use of resources with regard

summative evaluation. the use of repertory

the

(UKCC’s) objectives

involved

et al describe

within

Council’s

1992)

ity to measure quality militates against its use in in McSweeney

by Baly (1984)

contained

Co& of Professional Conduct (UKCC

fessionalism

evaluation’

that interpretation

by

consid-

includes a responsibility

approach

to course

appropriately

code outlined

Kingdom

context

suggests

are

recommendations

include somd innovative ideas, such as ‘a pictorial which

curriculum

ered. This flows from the essential requirements

to the intended

evaluation

nursing

and input, as identified

the

students)

bringing

together

who are stake-holders

of nurses. These stake-holders

statutory bodies, educationalists,

in the include:

clinicians,

trusts,

use of repertory grids is an interesting yet highly complex and time consuming activity. The

regional

authors

agers, etc. Obviously many of these do not physi-

themselves

use of the technique aided analysis. Alternative

see this as a problem

in the

even when using computer

studies, whilst attempting

measure-

Health,

health

authorities,

community

the Department

health councils,

of

service man-

cally take part in curriculum development, but their views, beliefs and directives are included in the decision

making process through

the profes-

NURSE EDUCATION

TODAY

283

essential

that

I

Fig. 2 Professional process and product model of evaluation.

sional judgement and developing managers

of those

planning,

the course. Clinicians

will also help to identify desirable

comes in terms of the product. through the active involvement During the development egy, concern

out-

This is achieved of these profes-

sionals, at all stages of the curriculum

cators

delivering and service

process.

of the described customer

all

it was considered

concerned

with

should be accountable process.

the

educational

for their evaluation of that

In simple terms this requires

respondent’s

process

evaluation

that each

should be clearly identi-

fied. Whilst some previous workers seem to constrat-

has been raised by some nurse edu-

trying to reconcile

the professional,

orientation

tradict

this approach

anonymity Hungler

by stressing

to safeguard

the need

the respondent

for

(Polit &

1983), such a strategy appears incompat-

and the ethos of the market place with providing an educationally sound course, the fear being that

ible with a professional

approach

the UKCC’s guidance

clearly implies, anonymity

academic and professional

is incompatible

standards may be sacri-

ficed on the altar of commercialism. may be reasonable involved became environment

-

- particularly accustomed

until all those involvement

in all curriculum

activities and joint identification

of

development

of expected

out-

comes, both process and product should be acceptable both academically and clinically. Additionally the comprehensive demic

validation

processes

nature of the aca-

and professional

continues

to provide

with professionalism.

Whilst this

to the competitive

with appropriate

service personnel

to evaluation. As

accreditation

a

safeguard

of

Process evaluation In adult education

the student is a full-participant

in the learning

process. As Newman (1979)

when referring

to non-compulsory

Adult education

states

education:

is a cruel test of a tutor’s skills.

It is a sink or swim business. If the tutor does not have what it takes, people

stop coming.

His stu-

dents vote with their feet.

academic rigour and professional standards. A feature of any professional nurse is the need

Whilst such an extreme

form of evaluation may

be rare in nurse education

the point remains rele-

to be accountable

vant. Indeed

‘each registered

for their actions,

specifically

nurse, midwife or health visitor

must be able to justify any action or decision

not

to act taken in the course of her professional practice’ (UKCC 1989). As part of this development of

it is equally applicable

involved in the learning

process,

to all those

including

clini-

cal staff and the tutors. Thus, there is a requirement to collate the views of these stake-holders in a way that actively encourages their participation

284

NURSE EDUCATION

TODAY

The

data collected

and continuing involvement through an explicit valuing of their contribution. Evaluation must

quantitative.

therefore

element

occur in such a way that individuals are of

tools to enable the process evalua-

are both

rationale

the contractual

opinion

tors had a significant

nent of the module/course

Beattie’s Fourfold Model of Curriculum

(1987)

curriculum

of the College.

ditionally been approached: jects; as a schedule meaningful curriculum From

of

of the

‘what do we need

four elements

involved

in

curriculum

to

of nursing

The professional the

became

to address

minor

major ones enabled

issues

effect, or which required commented

ial information

to report on the factors which influ-

The request tive element course

provided.

required

of

further action would be

This the

facilitates

of the

continuous

educational

process

as

to assure the quality of the service protools are issued as fol-

for supporting

information

for each item: l

To

students

at the

end

of each

theoretical

module/course. (3 Item Questionnaire) l

To

tutorial

staff at the end

of a theoretical

module/course. (1 Item Questionnaire) l

To students at the end of each practical

place-

ment. (5 Item Questionnaire)

or triv-

is ‘filtered out’ of the formal pro-

provides a qualita-

the development

vided. The data collection

they

cess at the earliest stage and is dealt with at a local

. To placement

staff at the end of practical place-

ment.

level. Central

for comments

to influence

improvement

us to be confident

upon. In this way extraneous

good,

respondent

status of

that only those aspects which had a significant

compo-

was: excellent,

enced their opinion.

evaluation, before

stem, e.g. in my

poor, very poor; with space for the

tools

along with a system of weekly informal evaluation sessions

statement

the delivery of the theoretical

lows, with a request

these elements.

the design of the data collection

was a simple matter.

of

of impor-

issues. Thus the evaluation

know?’ and Beattie’s

those

as an agenda

must also encompass

this position

curricula,

as a map of key sub-

of basic skills; as a portfolio

experience;

tant cultural

has tra-

of data

satisfactory,

This model identifies

four ways in which the nursing curriculum

These

are in the form of a rating scale with 5 options

know?’ The answer is fairly obvious - ‘What fac-

had been selected as the basis for the Project 2000

of elements

quality specification.

with an appropriate

involved?’

and

to provide statisti-

of the achievement

tion was the question - ‘What do we really need to effect on the various people

qualitative

for the quantitative

being the requirement

cal evidence

keen to participate. Thus the starting point for the development data collection

The

(1 Item Questionnaire) to the success of this concept

detailed explanation commencement

of the system given at course

to all students, repeated

lar intervals throughout sions for all tutorial equal importance.

is the

the course.

at regu-

Briefing

staff and clinicians

As students,

ses-

are of

tutors and clini-

cians receive feedback reports and are able to see for themselves the usefulness of their comments, the detail

provided

on the questionnaires

increased

dramatically

providing

has

vital informa-

tion for course development. Information is sought from those professionals, who are closely involved with the educational process, i.e. nurse tutors, clinicians/service managers, and students, regarding and the integration of the two.

theory,

practice

The data collected

on these questionnaires

vide a formal evaluation

pro-

of the educational

pro-

cess. It is not, of course, the only method used. As previously described, courses simultaneous

throughout modules and informal evaluation of the

process

occurring.

is constantly

This

takes the

form of informal interviews, weekly forums, group discussions etc., which allow for a rapid response to changing

needs of the individual groups and

facilitates the flow of relevant information through formal mechanisms unobscured by trivia.

Product evaluation As previously discussed, lum requires

the quality of a curricu-

measurement

for a number

of rea-

NURSE EDUCATION

sons. Of particular ongoing

importance

curriculum

are the need for

development

and establish-

ing fitness for purpose of the College,

the curricu-

lum and the students who have undertaken learning.

Of prime importance

however, at least

in business terms, is the requirement ment as an indication

that

for measure-

of the achievement

of con-

tractual quality criteria. Product ponent

evaluation to

the

achievement The

strategy

defined

of

which

nurse

in a number

viewpoint

facilitates

the

education

may

of ways. From a marketing

the ‘product’

is the curriculum

A service manager

pack-

from a nurse who undertakes In

professional

could be defined

of

may view the ‘prod-

uct’ in terms of the skills or outcomes gramme.

be

they require

an education

terms

the

pro-

‘product’

as a competent autonomous practi-

ous knowledge would domain,

the individual may be measured against the stated

the potential

of

‘product’

differences

ology for, as previously

stated,

rates many things including In the framework imperative

effectiveness

of the course in allowing a student to

and

appqfniate

experiences

those

professionals

means

the

manager course,

product is indeed fit for purpose. dent

fail to demonstrate

appropriate riences,

knowledge,

Should the stu-

the aquisition

of the

skills, attitudes and expe-

the course will be seen to have failed in

providing fitness for purpose in that individual. If this situation arises, close attention of the

product

to the qualita-

evaluation

forms

the rationale for this failure, which may

be due to inappropriate an inherent

selection

of students, or

fault in course delivery or design.

CONCLUSION

are to be met

is, again, involved. after

undertaken In

and their

3-6 months

The approach

described

offers a comprehensive

by

courses, or component

practice

this

College

immediate

line

achieved

of Nursing

modules, offered within a

and Health

by an acknowledgement

Studies.

This is

of the profes-

completion

of the

sional nature of those involved, and thus a belief

on the

in their ability and desire

This delay is to allow a period

by both student and manager

which occurred

practioner’s

on the

and its relevance

work. The applicability

may only become

apparent

to the

and This

experiences product

identification effect

to participate

in the

of those factors which significantly

the quality of the process

or product

learning.

of the course

once the former

dent is actively using the skills, knowledge, course.

by

default. It is of vital importance that this assessment is undertaken in order to ensure that the

the exact date being dependent

of reflection

tudes

knowledge if only

yet easy to use framework for the evaluation of all

‘student’

course undertaken. learning

skills,

assessed,

it is

is achieved

‘voting with their feet’ (Newman

1979) avoided. Product evaluation

be

satisfaction. market

satisfaction

quality specifications

and purchasers

of termin-

attitudes, can

In this way the

quality incorpo-

customer

of the internal

that customer

if contractual

must allow for

in perception

by the

of the line manager.

identifies

that the evaluation

and, equally important,

expectations

opments

clear

to assess development in this

it is clear that the fitness for purpose of

tive element

It is therefore

particularly

and value systems.

of an individual prior to a course,

be unable

tioner with all of the cognitive and affective develthis term implies.

for purpose,

attitudes

Whilst it is true that a line manager, with no previ-

develop

age, as this is what is actually sold to purchasers education.

fitness

personal

course outcomes provides a summative com-

of all these requirements.

product

an ex-student’s regarding

285

TODAY

developed information

using a similar format of questionnaire

during

stuattithe

is collected to those in

use for the formal process evaluation. Much discussion took place during the develop ment of the data collection tools regarding the appropriateness of service managers evaluating

References Baly M 1984 Professional responsibility. J Willey. London, UK, l-17 Beattie A 1987 Making a curriculum work. In: Allen P. Jolley M (eds) The curriculum in nursing education. Groom Helm, London, UK 15-34 Chambers M 19X8 Curriculum evaluation: an approach towards appraising a post basic psychiatric nursing course. Journal of Advanced Nursmg 13: 330-340

of

286

NURSE EDUCATION

TODAY

Crony M, Bignell A R 1988 Using a quality assurance model to evaluate ENB course 998: ‘Teaching and assessing in clinical practice’. Nurse Education Today 8: 332-340 Friere P 1973 Pedagogy of the oppressed. Translated by M B Ramer, Penguin, Harmondsworth, UK Gagne R 1977 The conditions of learning, 3rd ed. Holt, Reinhart & Winston, New York, USA Hogg S A 1990 The problem-solving curriculum evaluation and development model. Nurse Education Today IO: 10~110 Knowles M S 1978 The adult learner-A neglected species, 2nd ed. Gulf Publishing, Houston, USA McSweeny P. Fisher B, Russell T 1991 Course evaluation. Senior Nurse ll(6): 4.347 Newman M 1979 The poor cousin. George Allen & Unwin, London, UK Odro A B 1992 A pictorial approach to course evaluation. Nurse Education Today 12: 57-60 Parlett M R, Hamilton D F 1972 Evaluation as illumination: a new approach to the study of innovatory programs. In: Parlett, Dearden (eds). Introduction to illuminative evaluation. Studies in higher education. Pacific Soundings Press 9-29

Polit D. Hungler B 1983 Nursing research principles and methods, Lippincott, Philadelphia, USA Rogers C 1969 Freedom to learn. C Merrill, Columbus, Ohio, USA Striven M 1972 The prose (sic) and cons about goal-free evaluation. Evaluation comment. The Journal of Educational Evaluation. 3: l-4. In: Hamilton et al (eds), 1977 Beyond the numbers game. Macmillan Education Stake R 1967 The countenance of educational evaluation. In: Hamilton D et al. 1977 Beyond the numbers game. Macmillan Education, 146-155 StufIIebeam D L 1969 Evaluating as enlightenment for decision making and improving educational assessments and an inventing of measures of effective behaviour. National Educational Association, Washington D.C. In: Steps in evaluating a programme of Nurse Education. Nurse Education Today 1986 6: 166-I 7 1 Tyler R W 1949 Basic principles of curriculum and instruction. The University of Chicago Press Ltd., London, UK (1st British Impression, 1971). 110 UKCC 1989 Exercising accountability. UKCC. London, UK UKCC 1992 Code of Professional Conduct, 3rd ed. UKCC, London, UK