Chris Sconce and John Howard. This paper describes the development of a College-wide curriculum evaluation strategy. The strategy has been implemented ...
Curriculum evaluation -
a new approach
Chris Sconce and John Howard
This paper describes the development of a College-wide curriculum evaluation strategy. The strategy has been implemented on all courses offered by the Lancashire College of Nursing and Health Studies. An extensive review of the literature revealed that no evaluation models were suitable for use across all College courses. This was due to their over-emphasis on either the process, or the product of education; none of the published evaluation models adequetely address both of these components of professional learning. The criteria required, of both the model and the evaluation tools, included the need for a measurable component, inclusiveness, simplicity and practicality. In order to meet these a new model of evaluation and appropriate data collection tools/analytical systems were developed. This Professional Process and Product Model forms the basis for the described strategy. The model is easy to use due to its recognition of the professional nature of those involved in nurse education. this professional status, and the accountability which it implies, ensures that only relevant and significant information is included in the formal evaluation of courses. This ensures an ability to meet future contractual quality standards which will be a requirement for providers in the health care of the 1990s.
development,
INTRODUCTION Traditionally the nursing curriculum has been seen as a complex entity which is difficult to fully understand and thus evaluate. are still struggling suming procedures, 1990).
‘many nurse educators
with inappropriate,
time-con-
that yield little reward’ (Hogg
Such an approach
able. What is required, qualitative
information
is obviously unaccept-
is a system which provides for
use in curriculum
The
Chris Sconce BSc (How) RGN RHV RNT, Head of Quality Assurance 81 Public Relations, John Howard BSc (Hans) RGN RSCN RNT, Research and Information Manager, The Lancashire College of Nursing & Health Studies, Education Centre, Watling St Rd, Preston, Lanes, PR2 4DX, UK (Requests for offprints to CS) Manuscript accepted 17 November 1993
that
approach external
of the
which providers of
are now subject to.
introduction
requires
indicators
in order to meet the con-
tractual quality specifications education
of
the
internal
market
Colleges
adopt
a customer
to education.
Thus
the views of the
customers
(health care consortia,
led trusts,
students, health authorities, etc.) are of prime importance in assuring fitness for purpose. This is not to deny the role of educationalists motion
280
and measurable
quality of the product
recognise
of excellence
within practice,
the implications
in the promerely to
of the market
place
where customers are free to place contracts where they wish. Quality in nurse education number of ways including, ic excellence,
may be seen in a
attainment
practical competence
of academ-
or a combina-
NURSE EDUCATION
TODAY
1
Strategies
Fig. 1 Stufflebeam’s
CIPP model of curriculum
tion
of these.
sion
of a quality
demand
As contracting
evidence
evaluation.
the inclu-
purchasers
that educationalists
ing all criteria described only meaningful
requires
specification,
281
will
are achiev-
in that specification.
The
way that this can be achieved
is
through a process of evaluation.
including
any unintended
outcomes
of learning
and are thus inclusive rather than exclusive. Stufflebeam
(1969)
which concentrated
adopted
an
on the decision
approach
making pro-
cesses involved in the design and delivery of a curriculum. four
These processes
categories:
he allocated
to one of
Context-Input-Process-Product
(CIPP). Context
THE LITERATURE
evaluation
of curriculum Possibly the earliest attempt at curriculum tion
was provided
by Tyler
in the
evalua-
1930s.
approached
curriculum
design
development
of objectives
and their concomitant
evaluation,
the purpose
being
through
He
‘to see how far
these objectives are actually being realised’ 1949).
Such a behaviouristic
limit the evaluation level.
However,
the
measurability. The ‘Countenance
approach
of curricula beauty
the
(Tyler
tends to
to a mechanistic
is in its inherent
ment of curricular demands in nature
Model’
(Stake 1967) strove This framework
with the type
It involves the develop
objectives
of a changing
in the light of the
society. Whilst formative
it is also a component
evaluation.
Input
the resources
evaluation
required
of summative
is concerned
tion
about
the
various
Process evaluation
inter-relationship
final outcome
of informa-
strategies
proposed.
deals directly with the opera-
tion and implementation Finally product
of the curriculum of theory
evaluation
and
is concerned
of the curriculum;
with the
it assesses the
and out-
extent to which objectives have been achieved. model can be seen as cyclical, thus facilitating
the
ongoing
tions.
antecedents,
individual’s
inherent
Transactions
education
including
ment strategies. academic
refer
transactions
abilities
and
to
processes
the
motivaof
teaching methods and assess-
The outcomes
achievements
or
are measured practical
as
abilities,
By addressing process
modification
these processes of curriculum
(Fig. 1). Although
and
practice.
comes. Antecedents are those conditions which existed prior to the learning experience, such as
encompassed
with
to achieve the curriculum
objectives and involves the acquisition
the
to widen the basis for evaluation.
is concerned
offered.
sequentially, evaluation
the an and
elegant in its sim-
plicity and the ease with which it may be adapted to a variety of situations the inherent measuring and recording context
difficulty of and input
282
NURSE EDUCATION
evaluation
TODAY
may lead to a disregard
for these ele-
ments. (1972)
suggested the concept of ‘Goal-
Free’ evaluation,
Striven
this being based on the belief
that pre-specified
objectives
would
acknowledgement
of arising
issues and militate
against
the full utilisation
tion. Thus all learning
inhibit
of available
is addressed,
the
informa-
not simply
ment,
seem
These
ranged
unwieldly
(Crotty
& Bignell
from
for
widespread
103item
1988)
usage.
questionnaires
to personal
interviews
each lasting up to 90 minutes (Chambers1988). Thus, from the literature a number of key criteria emerge. These include: mearsurability inclusiveness,
of data,
simplicity and ease of use, object-
ivity, and the ability to prioritise issues raised.
that which was originally intended. During
the
1970s
product
of learning
advent
of
approaches became
person-centred
Rogers
(1969),
the
with the
curriculum
models
based on the work of theorists (1978),
involving
unpopular
Friere
such as Knowles (1973)
and Gange
(1977). Evaluation as illumination Parlett and Hamilton Evaluation’
was first proposed by
in 1972. Their ‘Illuminative
model uses description
tation rather than measurement
and interpre-
as its base. Using
THE PROFESSIONAL PROCESS AND PRODUCT MODEL OF EVALUATION Professional
evaluation
The professionalism
of those involved in the plan-
ning
of the
and
ensures
delivery
that context
observation of, and interviews with, those involved in the curriculum, as well as analysis of documen-
Stufflebeam
tation and a variety of other methods,
of a professional
evaluators,
to identify
enables the
and contextualise
issues
raised. Thus it is possible to address the issues in priority
order
to facilitate
curriculum’
develop-
ment. Whilst this is of particular importance strategy intended ety of courses,
for a
for everyday use on a wide vari-
the inability to easily measure
quality of the curriculum
renders
unsuitable for use in contractual
the
this approach
situations.
More recent attempts at curriculum
(1969),
and
United
drawings might indicate regarding
(Odro
1992),
of students’
their personal
thoughts
the course and their level of personal
achievement.
Whilst this may be of use to provoke
discussion and interaction tivity of interpretation
formatively, the subjec-
and, once again, the inabil-
Central
the light of the changing acknowledges selection
of nursing
courses.
The
which in
demands of society and
responsibility
for the professional’s
educational
programme.
When applied to nurse education
the areas of
and input are clearly integral to the proof those involved. Thus a three stage
model of evaluation emerges
(Fig. 2). This model
shows the linear progression
of the educational
experience
from the professional
body through
the process itself to the product of learning. experience the
professionals
(including
in that experience,
education
This
receives input from and is analysed by
those professionals
grids in the evaluation
to examine
and proper use of resources with regard
summative evaluation. the use of repertory
the
(UKCC’s) objectives
involved
et al describe
within
Council’s
1992)
ity to measure quality militates against its use in in McSweeney
by Baly (1984)
contained
Co& of Professional Conduct (UKCC
fessionalism
evaluation’
that interpretation
by
consid-
includes a responsibility
approach
to course
appropriately
code outlined
Kingdom
context
suggests
are
recommendations
include somd innovative ideas, such as ‘a pictorial which
curriculum
ered. This flows from the essential requirements
to the intended
evaluation
nursing
and input, as identified
the
students)
bringing
together
who are stake-holders
of nurses. These stake-holders
statutory bodies, educationalists,
in the include:
clinicians,
trusts,
use of repertory grids is an interesting yet highly complex and time consuming activity. The
regional
authors
agers, etc. Obviously many of these do not physi-
themselves
use of the technique aided analysis. Alternative
see this as a problem
in the
even when using computer
studies, whilst attempting
measure-
Health,
health
authorities,
community
the Department
health councils,
of
service man-
cally take part in curriculum development, but their views, beliefs and directives are included in the decision
making process through
the profes-
NURSE EDUCATION
TODAY
283
essential
that
I
Fig. 2 Professional process and product model of evaluation.
sional judgement and developing managers
of those
planning,
the course. Clinicians
will also help to identify desirable
comes in terms of the product. through the active involvement During the development egy, concern
out-
This is achieved of these profes-
sionals, at all stages of the curriculum
cators
delivering and service
process.
of the described customer
all
it was considered
concerned
with
should be accountable process.
the
educational
for their evaluation of that
In simple terms this requires
respondent’s
process
evaluation
that each
should be clearly identi-
fied. Whilst some previous workers seem to constrat-
has been raised by some nurse edu-
trying to reconcile
the professional,
orientation
tradict
this approach
anonymity Hungler
by stressing
to safeguard
the need
the respondent
for
(Polit &
1983), such a strategy appears incompat-
and the ethos of the market place with providing an educationally sound course, the fear being that
ible with a professional
approach
the UKCC’s guidance
clearly implies, anonymity
academic and professional
is incompatible
standards may be sacri-
ficed on the altar of commercialism. may be reasonable involved became environment
-
- particularly accustomed
until all those involvement
in all curriculum
activities and joint identification
of
development
of expected
out-
comes, both process and product should be acceptable both academically and clinically. Additionally the comprehensive demic
validation
processes
nature of the aca-
and professional
continues
to provide
with professionalism.
Whilst this
to the competitive
with appropriate
service personnel
to evaluation. As
accreditation
a
safeguard
of
Process evaluation In adult education
the student is a full-participant
in the learning
process. As Newman (1979)
when referring
to non-compulsory
Adult education
states
education:
is a cruel test of a tutor’s skills.
It is a sink or swim business. If the tutor does not have what it takes, people
stop coming.
His stu-
dents vote with their feet.
academic rigour and professional standards. A feature of any professional nurse is the need
Whilst such an extreme
form of evaluation may
be rare in nurse education
the point remains rele-
to be accountable
vant. Indeed
‘each registered
for their actions,
specifically
nurse, midwife or health visitor
must be able to justify any action or decision
not
to act taken in the course of her professional practice’ (UKCC 1989). As part of this development of
it is equally applicable
involved in the learning
process,
to all those
including
clini-
cal staff and the tutors. Thus, there is a requirement to collate the views of these stake-holders in a way that actively encourages their participation
284
NURSE EDUCATION
TODAY
The
data collected
and continuing involvement through an explicit valuing of their contribution. Evaluation must
quantitative.
therefore
element
occur in such a way that individuals are of
tools to enable the process evalua-
are both
rationale
the contractual
opinion
tors had a significant
nent of the module/course
Beattie’s Fourfold Model of Curriculum
(1987)
curriculum
of the College.
ditionally been approached: jects; as a schedule meaningful curriculum From
of
of the
‘what do we need
four elements
involved
in
curriculum
to
of nursing
The professional the
became
to address
minor
major ones enabled
issues
effect, or which required commented
ial information
to report on the factors which influ-
The request tive element course
provided.
required
of
further action would be
This the
facilitates
of the
continuous
educational
process
as
to assure the quality of the service protools are issued as fol-
for supporting
information
for each item: l
To
students
at the
end
of each
theoretical
module/course. (3 Item Questionnaire) l
To
tutorial
staff at the end
of a theoretical
module/course. (1 Item Questionnaire) l
To students at the end of each practical
place-
ment. (5 Item Questionnaire)
or triv-
is ‘filtered out’ of the formal pro-
provides a qualita-
the development
vided. The data collection
they
cess at the earliest stage and is dealt with at a local
. To placement
staff at the end of practical place-
ment.
level. Central
for comments
to influence
improvement
us to be confident
upon. In this way extraneous
good,
respondent
status of
that only those aspects which had a significant
compo-
was: excellent,
enced their opinion.
evaluation, before
stem, e.g. in my
poor, very poor; with space for the
tools
along with a system of weekly informal evaluation sessions
statement
the delivery of the theoretical
lows, with a request
these elements.
the design of the data collection
was a simple matter.
of
of impor-
issues. Thus the evaluation
know?’ and Beattie’s
those
as an agenda
must also encompass
this position
curricula,
as a map of key sub-
of basic skills; as a portfolio
experience;
tant cultural
has tra-
of data
satisfactory,
This model identifies
four ways in which the nursing curriculum
These
are in the form of a rating scale with 5 options
know?’ The answer is fairly obvious - ‘What fac-
had been selected as the basis for the Project 2000
of elements
quality specification.
with an appropriate
involved?’
and
to provide statisti-
of the achievement
tion was the question - ‘What do we really need to effect on the various people
qualitative
for the quantitative
being the requirement
cal evidence
keen to participate. Thus the starting point for the development data collection
The
(1 Item Questionnaire) to the success of this concept
detailed explanation commencement
of the system given at course
to all students, repeated
lar intervals throughout sions for all tutorial equal importance.
is the
the course.
at regu-
Briefing
staff and clinicians
As students,
ses-
are of
tutors and clini-
cians receive feedback reports and are able to see for themselves the usefulness of their comments, the detail
provided
on the questionnaires
increased
dramatically
providing
has
vital informa-
tion for course development. Information is sought from those professionals, who are closely involved with the educational process, i.e. nurse tutors, clinicians/service managers, and students, regarding and the integration of the two.
theory,
practice
The data collected
on these questionnaires
vide a formal evaluation
pro-
of the educational
pro-
cess. It is not, of course, the only method used. As previously described, courses simultaneous
throughout modules and informal evaluation of the
process
occurring.
is constantly
This
takes the
form of informal interviews, weekly forums, group discussions etc., which allow for a rapid response to changing
needs of the individual groups and
facilitates the flow of relevant information through formal mechanisms unobscured by trivia.
Product evaluation As previously discussed, lum requires
the quality of a curricu-
measurement
for a number
of rea-
NURSE EDUCATION
sons. Of particular ongoing
importance
curriculum
are the need for
development
and establish-
ing fitness for purpose of the College,
the curricu-
lum and the students who have undertaken learning.
Of prime importance
however, at least
in business terms, is the requirement ment as an indication
that
for measure-
of the achievement
of con-
tractual quality criteria. Product ponent
evaluation to
the
achievement The
strategy
defined
of
which
nurse
in a number
viewpoint
facilitates
the
education
may
of ways. From a marketing
the ‘product’
is the curriculum
A service manager
pack-
from a nurse who undertakes In
professional
could be defined
of
may view the ‘prod-
uct’ in terms of the skills or outcomes gramme.
be
they require
an education
terms
the
pro-
‘product’
as a competent autonomous practi-
ous knowledge would domain,
the individual may be measured against the stated
the potential
of
‘product’
differences
ology for, as previously
stated,
rates many things including In the framework imperative
effectiveness
of the course in allowing a student to
and
appqfniate
experiences
those
professionals
means
the
manager course,
product is indeed fit for purpose. dent
fail to demonstrate
appropriate riences,
knowledge,
Should the stu-
the aquisition
of the
skills, attitudes and expe-
the course will be seen to have failed in
providing fitness for purpose in that individual. If this situation arises, close attention of the
product
to the qualita-
evaluation
forms
the rationale for this failure, which may
be due to inappropriate an inherent
selection
of students, or
fault in course delivery or design.
CONCLUSION
are to be met
is, again, involved. after
undertaken In
and their
3-6 months
The approach
described
offers a comprehensive
by
courses, or component
practice
this
College
immediate
line
achieved
of Nursing
modules, offered within a
and Health
by an acknowledgement
Studies.
This is
of the profes-
completion
of the
sional nature of those involved, and thus a belief
on the
in their ability and desire
This delay is to allow a period
by both student and manager
which occurred
practioner’s
on the
and its relevance
work. The applicability
may only become
apparent
to the
and This
experiences product
identification effect
to participate
in the
of those factors which significantly
the quality of the process
or product
learning.
of the course
once the former
dent is actively using the skills, knowledge, course.
by
default. It is of vital importance that this assessment is undertaken in order to ensure that the
the exact date being dependent
of reflection
tudes
knowledge if only
yet easy to use framework for the evaluation of all
‘student’
course undertaken. learning
skills,
assessed,
it is
is achieved
‘voting with their feet’ (Newman
1979) avoided. Product evaluation
be
satisfaction. market
satisfaction
quality specifications
and purchasers
of termin-
attitudes, can
In this way the
quality incorpo-
customer
of the internal
that customer
if contractual
must allow for
in perception
by the
of the line manager.
identifies
that the evaluation
and, equally important,
expectations
opments
clear
to assess development in this
it is clear that the fitness for purpose of
tive element
It is therefore
particularly
and value systems.
of an individual prior to a course,
be unable
tioner with all of the cognitive and affective develthis term implies.
for purpose,
attitudes
Whilst it is true that a line manager, with no previ-
develop
age, as this is what is actually sold to purchasers education.
fitness
personal
course outcomes provides a summative com-
of all these requirements.
product
an ex-student’s regarding
285
TODAY
developed information
using a similar format of questionnaire
during
stuattithe
is collected to those in
use for the formal process evaluation. Much discussion took place during the develop ment of the data collection tools regarding the appropriateness of service managers evaluating
References Baly M 1984 Professional responsibility. J Willey. London, UK, l-17 Beattie A 1987 Making a curriculum work. In: Allen P. Jolley M (eds) The curriculum in nursing education. Groom Helm, London, UK 15-34 Chambers M 19X8 Curriculum evaluation: an approach towards appraising a post basic psychiatric nursing course. Journal of Advanced Nursmg 13: 330-340
of
286
NURSE EDUCATION
TODAY
Crony M, Bignell A R 1988 Using a quality assurance model to evaluate ENB course 998: ‘Teaching and assessing in clinical practice’. Nurse Education Today 8: 332-340 Friere P 1973 Pedagogy of the oppressed. Translated by M B Ramer, Penguin, Harmondsworth, UK Gagne R 1977 The conditions of learning, 3rd ed. Holt, Reinhart & Winston, New York, USA Hogg S A 1990 The problem-solving curriculum evaluation and development model. Nurse Education Today IO: 10~110 Knowles M S 1978 The adult learner-A neglected species, 2nd ed. Gulf Publishing, Houston, USA McSweeny P. Fisher B, Russell T 1991 Course evaluation. Senior Nurse ll(6): 4.347 Newman M 1979 The poor cousin. George Allen & Unwin, London, UK Odro A B 1992 A pictorial approach to course evaluation. Nurse Education Today 12: 57-60 Parlett M R, Hamilton D F 1972 Evaluation as illumination: a new approach to the study of innovatory programs. In: Parlett, Dearden (eds). Introduction to illuminative evaluation. Studies in higher education. Pacific Soundings Press 9-29
Polit D. Hungler B 1983 Nursing research principles and methods, Lippincott, Philadelphia, USA Rogers C 1969 Freedom to learn. C Merrill, Columbus, Ohio, USA Striven M 1972 The prose (sic) and cons about goal-free evaluation. Evaluation comment. The Journal of Educational Evaluation. 3: l-4. In: Hamilton et al (eds), 1977 Beyond the numbers game. Macmillan Education Stake R 1967 The countenance of educational evaluation. In: Hamilton D et al. 1977 Beyond the numbers game. Macmillan Education, 146-155 StufIIebeam D L 1969 Evaluating as enlightenment for decision making and improving educational assessments and an inventing of measures of effective behaviour. National Educational Association, Washington D.C. In: Steps in evaluating a programme of Nurse Education. Nurse Education Today 1986 6: 166-I 7 1 Tyler R W 1949 Basic principles of curriculum and instruction. The University of Chicago Press Ltd., London, UK (1st British Impression, 1971). 110 UKCC 1989 Exercising accountability. UKCC. London, UK UKCC 1992 Code of Professional Conduct, 3rd ed. UKCC, London, UK
