Repositioning Quality Assurance and Accreditation in ... - CiteSeerX

CONTENTS

Department of Education, Training and Youth Affairs

Repositioning Quality Assurance and Accreditation in Australian Higher Education Grant Harman V Lynn Meek Centre for Higher Education Management and Policy University of New England 00/2 May 2000 Evaluations and Investigations Programme Higher Education Division

Instructions for pdf navigation • Use the arrows on the Acrobat menu bar to navigate forwards or backwards page by page •

Alternatively, use the arrow icons on your keyboard to navigate through the document.

• To enlarge the viewing screen either:

•

–

use the magnifying glass by clicking on the area you wish to enlarge or by forming a marquee over the area you wish to view (ie. hold the mouse button down and drag the magnifying glass over the area); or

–

use the view options menu bar at the bottom of the Acrobat screen.

To pan out from the page, hold down the option button on your keyboard to change the +ve symbol on the magnifying glass to a –ve symbol , then click the mouse.

• To search for a word or phrase use the binoculars icon on the menu bar. • The Contents pages are live, ie. if you click on a topic you will go to that page. •

You can return to the Contents page by clicking your mouse on ‘Contents’ on the top of each page.

CONTENTS

© Commonwealth of Australia 2000 ISBN 0 642 23992 4 ISBN 0 642 23993 2 (Online version) DETYA No. 6474.HERC 00A This work is copyright. Apart from any use as permitted under the Copyright Act 1968, no part may be reproduced by any process without permission from AusInfo. Requests and inquiries concerning reproduction and rights should be addressed to the Manager, Legislative Services, AusInfo, GPO Box 84, Canberra ACT 2601. This report is funded under the Evaluations and Investigations Programme of the Department of Employment, Education, Training and Youth Affairs. The views expressed in this report do not necessarily reflect the views of the Department of Education, Training and Youth Affairs.

Repositioning Quality Assurance and Accreditation in Australian Higher Education

Contents Executive summary

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .v

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1 2 Quality assurance and accreditation . . . . . . . . . . . . . . . . . . . . . . . . .7 Quality and higher education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8 The concept of quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9 Quality assurance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11 Related concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12 Accreditation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14 Main quality assurance approaches and methodologies . . . . . . . . . . . . . . . .16 Concluding comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .26

3 Australian higher education providers and current arrangements for accreditation and quality assurance . . . . . . . . . . . .29 International education enrolments . . . . . Current accreditation arrangements . . . . Special protection for international students Assessment . . . . . . . . . . . . . . . . . . . . .

... ... .. ...

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

.33 .35 .50 .51

4 The changing quality environment and the Modern Australian Model . .53 The changing quality environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .53 The Modern Australian Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .62 Other options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .63

5 Accreditation of courses and institutions . . . . . . . . . . . . . . . . . . . . . .73 Responsibility for accreditation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .74 Greater uniformity of legislation, criteria and processes . . . . . . . . . . . . . . . . .76 Reporting on accreditation of institutions and courses . . . . . . . . . . . . . . . . . .79 Other compliance functions for accrediting agencies . . . . . . . . . . . . . . . . . . .79 Staffing and resourcing of State and Territory accreditation agencies . . . . . . .80 Links between accreditation and quality assurance . . . . . . . . . . . . . . . . . . . .80

6 Quality assurance and improvement plans . . . . . . . . . . . . . . . . . . . .81 Comments on suggested model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .81 Who should require and publish plans and links with institutional audits? . . . .83

7 Quality audits and a new quality agency . . . . . . . . . . . . . . . . . . . . .85 Characteristics of and criteria for the new mechanism . . . . . . Legal basis and structure of new agency . . . . . . . . . . . . . . . Preferred model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Review teams will report to the Council. . . . . . . . . . . . . . . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . . .86 . . . . .91 . . . . .92 . . . . .94

Appendix A Project brief . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .95 Appendix B List of interviews . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .99 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .103 iii

TABLES AND ACRONYMS

CONTENTS Repositioning Quality Assurance and Accreditation in Australian Higher Education

Tables Table 2.1 Table 3.1 Table 3.2 Table 3.3 Table 3.4 Table 3.5 Table 3.6

Main approaches and methodologies at the national level Higher education accredited awards offered by private providers Total ‘off-shore’ student enrolments of major higher education providers, 1998 Largest off-shore enrolments in particular countries by institutions, 1998 Legislation providing for accreditation of courses and institutions Legislation relevant to the establishment and operation of Australian universities Summary of legislative protection offered nationally to Australian universities

Acronyms AAU

Academic Audit Unit

ANTA

Australian National Training Authority

AQF

Australian Qualifications Framework

AVCC CNAA

Australian Vice-Chancellors’ Committee Council for National Academic Awards

CNE

Comite National d’Evaluation

CRUI DETYA

Italian Standing Conference of Rectors Department of Education, Training and Youth Affairs

HEFCE

Higher Education Funding Council of England

NOOSR National Office of Overseas Skills Recognition MCEETY A Ministerial Council on Employment, Education, Training and Youth Affairs

iv

OECD

Organisation for Economic Cooperation and Development

RAE RTO

Research Assessment Exercise Registered Training Organisation

TAFE

Technical and Further Education

VET VSNU

Vocational Education and Training Association of Cooperating Universities of the Netherlands

17 31 34 34 35 40 41


Executive summary

Introduction 1.

This report has been prepared for the Commonwealth Department of Education, Training and Youth Affairs (DETYA) to assist in developing an improved national approach to both quality assurance and accreditation. The current arrangements are inadequate and Australia needs a more rigorous quality assurance and accreditation system: to protect the international reputation of our higher education; for public accountability purposes; to inform student choice; and to promote and improve quality processes and outcomes as well as disseminate good practice.

2.

Various models for a new approach to quality assurance and accreditation have been reviewed, including refinement of the current Australian model, the recently modified New Zealand model, the new United Kingdom model, the model used by the Vocational Education and Training (VET) system in Australia and a ‘Modern Australian Model’.

3.

The Modern Australian Model provides separate arrangements for both institutions which have been given power to accredit their own courses and for non-self accrediting providers. For institutions with power to accredit their own courses, the main requirements will be: (a) rigorous scrutiny of financial and quality aspects before founding legislation is passed or other authorisation is given; (b) annual publication of Quality Assurance and Improvement Plans for the forthcoming triennium; (c) a quality audit of each institution every five years following a detailed self-assessment; and (d) compliance with any additional measures which may be necessary to ensure the maintenance of acceptable high standards of degrees. For non selfaccrediting providers, the main features may include: (a) rigorous scrutiny of provider capacity before accreditation; and (b) review of provider performance and accredited courses every five years.

4.

The task for the project was to develop the Modern Australian Model as an alternative to the other four models; advise under whose authority it should be run and whether a legislative base is needed; assess whether it would be sensible and appropriate to make use of the AQF; elaborate the possible nature of the five yearly

v


self-assessments for self-accrediting institutions; comment on the desirability of focussing on outcomes and standards as well as processes; consider how to achieve rigour and independence while retaining the cooperation and confidence of universities; and advise on the role of professional associations within the model and the nature of the audit of the courses of non self-accrediting providers. We were requested to evaluate the Modern Australian Model against the following criteria: credibility; effectiveness; ability to provide legal clarity for students and providers; ability to promote and enhance improvement and good practice; how well the model could build on the key features of the current system and possibly exploit the role of professional associations in accrediting courses; minimum bureaucracy; and cost.

Quality assurance and accreditation

vi

5.

Quality assurance in higher education is defined as systematic management and assessment procedures adopted by a higher education institution or system to monitor performance and to ensure achievement of quality outputs or improved quality. Quality assurance aims to give stakeholders confidence about the management of quality and the outcomes achieved.

6.

Accreditation refers to a process of assessment and review which enables a higher education course or institution to be recognised or certified as meeting appropriate standards. In Australia, the term accreditation has developed three specialist meanings: a process of review or assessment conducted by a government agency to enable a Minister or approved authority to recognise and approve a higher education institution or course; a process of review carried out by a government registration body to enable graduates of particular courses to practise in the particular State or Territory; and a process of assessment and recognition carried out professional associations. In this report, we are primarily concerned with the first usage.

7.

Quality in the context of higher education can be defined as a judgement about the level of goal achievement and the value and worth of that achievement. It is also a judgement about the degree to which activities or outputs have desirable characteristics, according to some norm or against particular specified criteria or objectives.


8.

Over the past decade, extensive experimentation has taken place internationally with quality assurance and how it is managed. The literature reporting these developments points to tremendous variety in approaches and methods. The main approaches can be summarised under the following headings: the agency or unit with responsibility for the management of quality assurance; participation in reviews and other activities; the main methodologies employed; the focus of quality assurance activities; the purposes of such activities; and reporting and/or follow-up.

9.

The most common pattern at national level is for responsibility to lie with a specialised government agency, or with the central department responsible for higher education coordination. In a small number of countries, however, responsibility lies with an agency set up by higher education institutions themselves.

10.

An important variation between quality assurance systems is whether participation is voluntary or compulsory. Many countries began with institutional audits on a voluntary basis. Generally, however, with national reviews of disciplines, participation is compulsory and, even when participation is voluntary, strong moral and professional pressures usually produce in a high level of participation.

11.

Most quality assurance mechanisms depend on one or a combination of a limited number of methodologies, the most important of which are self-studies or self-evaluation; peer review by panels of experts; use of relevant statistical information and performance indicators; and surveys of key groups, such as students, graduates and employers. At the national level, the most common forms of assessment are ‘horizontal’ reviews of disciplines and ‘vertical’ evaluations of institutions.

12.

Quality assurance programs serve a variety of purposes but generally their primary purposes are a combination of public accountability, efforts to ensure credibility, improvement and renewal. In some cases, there is a gap between stated purposes and actual purposes, and frequently there is tension between accountability and improvement purposes.

13.

Reporting and follow-up activities are important, with a major challenge being to devise fair and effective methods likely to lead to improvements without damaging the institution being reviewed. Various approaches are used with regard to the distribution of reports. In some cases, reports are provided solely to the institution concerned but increasingly the practice is to make the results more widely available.

vii


Higher education providers and current arrangements

viii

14.

For the purposes of this study, Australian higher education providers can be categorized into five distinct groups: public universities and other public higher education institutions established under State, Territory and Commonwealth legislation; non-government institutions which operate under their own legislation and have self-accrediting powers; institutions not established by legislation but who have been given government approval to operate; private providers whose courses have been accredited by State or Territory accrediting agencies; and private providers whose courses have not yet been accredited.

15.

Accreditation of higher education institutions and courses is under the control of State and Territory Governments who view this responsibility as flowing from their responsibilities for education under the Commonwealth constitution. Generally, the relevant legislation makes provision for private providers to secure accreditation and approval to offer courses. In other cases, legislation provides for accreditation of both institutions and courses.

16.

The considerable differences between the provisions of State and Territory legislation and the criteria and processes constitute a major problem needing attention. In a number of cases, legislation provides control over the use of the terms ‘university’ and ‘degree’, and over degree titles. The most detailed legislative controls operate in Victoria, New South Wales, Queensland and Tasmania. Additional protection is afforded with regard to the establishment and recognition of universities by other Commonwealth, State and Territory legislation.

17.

Recently the States and Territories have agreed on procedures for considering applications and authorisation to offer higher education courses in two or more States and Territories, and operational guidelines to achieve this were endorsed by MCEETYA in April 1999.

18.

The current system of quality assurance operates at a number of levels and includes the activities of professional associations and associations and networks set up by groups of universities for benchmarking and other quality assurance purposes.

19.

Internal quality assurance processes in universities include: assessment of new courses and units of study; reviews of courses, units and departments; student evaluation of teaching; use of external examiners for higher degree research theses and sometimes bachelors honours theses; surveys of graduates and employers; use of performance


indicators; benchmarking and participation in benchmarking networks; and special projects for the improvement of teaching and special awards for teaching excellence. 20.

The current national quality assurance mechanisms include: reports by universities on quality assurance and improvement plans; encouragement of innovation and good teaching; and publication of Characteristics and Performance of Higher Education Institutions.

21.

Various professional bodies conduct accreditation of professional courses in fields such as medicine, law, engineering and architecture. Professional associations have formed a peak body, the Australian Council of Professions.

22.

Special Commonwealth mechanisms, consisting of legislation and a register of courses, provide protection for international students. For example, The Education Services for Overseas Students (Registration of Providers and Financial Regulation) Act 1991 helps ensure that only quality courses are offered to foreign students studying in Australia.

23.

While there are various desirable characteristics of the current quality assurance and accreditation arrangements, at the same time there are clear weaknesses that need attention. The major weaknesses in quality assurance is lack of a national agency that can publicly vouch for the quality of Australian higher education while with regard to accreditation there is need for uniform approaches and criteria across States and Territories and a better system of reporting and providing public access to information concerning which courses and institutions have been accredited.

Changing quality environment and the Modern Australian Model 24.

Important recent changes have taken place in the quality assurance environment. These can be categorised under the headings of: globalisation and changes in educational technology; international recognition of qualifications; recent changes in quality assurance in other industrialised countries; new quality assurance arrangements in ‘off-shore’ education countries; the needs of Australia’s education export industry; increased accountability pressures at home; incidents with private providers and increases in the number of private providers; and complaints from applicants seeking accreditation.

ix


x

25.

The Modern Australian Model of quality assurance and accreditation has many strengths. It makes important distinctions between the functions of accreditation and quality assurance, and between the possible treatment of self-accrediting institutions and non-self accrediting providers. DETYA documentation specifies that quality assurance and accreditation mechanisms should satisfy a number of criteria: the mechanisms relating to self-accrediting institutions should not be solely at their discretion; there needs to be some external review or audit of the claims made by institutions about quality and standards; the mechanisms should be credible with international and domestic interest groups and be able to protect the international reputation of Australian awards; the mechanisms should help satisfy Australian taxpayers of value for money; any audit mechanism should have rigour, but at the same time be cost effective, not unnecessarily intrusive and be able to retain the cooperation of public universities; and the mechanisms should provide legal clarity for students and providers and be able to promote good practice and facilitate improvement.

26.

Refinement of the current Australian model of accreditation and quality assurance would provide valuable improvements but Australia’s arrangements would still fall far behind international good practice and do little to provide additional safeguards for the education export industry, or to lend additional international credibility to Australian awards. The recently modified New Zealand Model provides for a national government agency but there is some uncertainty about how successful will be the plan for the new agency to approve various accrediting bodies. The new United Kingdom model is still developing its procedures but to date its proposals have been somewhat controversial and have yet to secure support from the well-established universities. The current VET model of accreditation and quality assurance is now well accepted in the VET sector and widely supported by industry but this model does not appear suitable for the higher education sector.

27.

Important models not canvassed in DETYA documentation are the Dutch and French models of quality assurance. The Dutch model is based on a well-organised program of disciplinary reviews, while the French model uses both disciplinary reviews and institutional audits. The Dutch program is operated by the VSNU, the association representing the heads of Dutch universities, while the French program is the responsibility of a special government agency.


Accreditation of courses and institutions 28.

Any process of accreditation would need to be concerned primarily with: approval for new universities to operate, to use the name university, and to offer degrees and other awards; approval and accreditation of courses of study leading to degrees and other awards by other higher education providers; and re-accreditation of institutions and awards.

29.

There is strong support for the proposition that accreditation is a matter for government and not the higher education sector, and that the States and Territories should continue to exercise their responsibilities in this area. Continuation and extension of the present work being undertaken by the MCEETYA Multilateral Joint Planning Committee seems likely to produce a professional national approach to accreditation.

30.

To date the Multilateral Committee has put most of its efforts into developing a common approach to the accreditation of universities. This has raised issues about the distinguishing characteristics of universities today in Australia and what criteria should be used in accrediting new and overseas universities. There differences in views over use of the titles of ‘university’ and ‘degree’. State and Territory officials see value in maintaining and strengthening current controls over these titles.

31.

Perhaps more important for the Multilateral Committee will be to develop uniform protocols for the recognition of new and overseas universities and agreement on the criteria to be applied. There appears to be agreement that criteria should include topics such as financial viability, the legal basis of the institution, and the processes of governance, internal quality assurance and accountability but less agreement about whether the criteria should include quantitative indicators with regard to staff, buildings and facilities, and library holdings and specialised laboratories. Other issues that need attention include: protocols and procedures for the accreditation of institutions other than universities; whether the recognition of new and overseas universities should automatically carry with it the rights of self-accrediting powers; whether all institutions need some form of accreditation before their courses can be accredited; requirements with regard to ‘out-state’ Australian institutions operating in other States and Territories; whether accredited institutions should be required to seek special approval to offer courses to international students at special international student campuses; whether

xi


universities and other self accrediting institutions need special approval to enter into franchise arrangements to offer higher education courses with non accredited institutions such as VET providers; and whether legislation in all States and Territories should provide for both the accreditation of institutions and courses.

Quality assurance and improvement plans 32.

Continuation and strengthening of the current requirements of the Commonwealth with regard to institutional quality assurance and improvement plans appears to be a well-conceived and sensible strategy. Good management practice requires that all institutions should have in place appropriate quality assurance and improvement plans, and submission of these to some outside body provides useful discipline for institutions to keep plans up to date. Such plans should cover all major aspects of operations, including ‘off-shore’ efforts and distance education offered internationally.

33.

While some universities have made major advances in benchmarking, this development has not been uniform across the sector and that it may be helpful to provide additional assistance. Institutions should be encouraged to document in detail their monitoring and quality assurance procedures for ‘off-shore’ efforts.

34.

With regard to quality assurance plans currently required by DETYA, it will be important that discussions take place with any new quality assurance agency to ensure that DETYA requirements do not in conflict with any documentation requirements for institutional audits.

Quality audits and a new quality agency

xii

35.

A non-intrusive and sensibly conceived quality assurance mechanism involving both the higher education sector and the State and Territories seems likely to attract considerable support. Certainly there is wide appreciation of some of the strong influences that require establishment of a new national mechanism.

36.

One important issue is the legal basis of any new agency, the governance structures to be employed and its accountability arrangements. In our discussions four main models were identified: a Ministerial Committee set up by the Commonwealth Minister;


an agency established by Commonwealth legislation; an agency established by joint Commonwealth and State legislation; and an agency set up as a company, possibly responsible to MCEETYA. 37.

Our preferred model for the new quality assurance mechanism and agency is as follows: • A new quality assurance mechanism should be established as a joint Commonwealth, State/Territory, and higher education initiative with the aim of strengthening public accountability, protecting academic standards and the reputation of Australian higher education providers and awards, and promoting good practice in quality assurance. We suggest that the new mechanism should be called the Higher Education Quality Assurance Council. • The central function of the Council will be conduct of program of institutional reviews or audits. Review teams will carry out site visits, following completion of self-assessments carried out by institutions, which will include reviews of the processes of managing quality including monitoring performance and benchmarking. Institutions will provide review teams with a report of their self-assessments, together with documentation on institutional mission and objectives, quality assurance and improvement plans, details on methods used to monitor and benchmark achievements and the results of monitoring and benchmarking. Participating institutions will be reviewed every five years. • The Council will be established as an independent agency, at ‘arms length’ from both government (Commonwealth and State) and from the higher education sector. It will be governed by a board consisting of an independent Chair, two Commonwealth nominees, two members representing the States and Territories, two representatives of the higher education sector and one representative drawn from those professional associations involved in accreditation within the higher education sector. The Executive Director will be an exofficio member and the board will have the power to coopt up to two additional members with special expertise in academic audits and assessment. Commonwealth representatives will be appointed by the Minister for Education, Training and Youth Affairs, while the two State and Territory representatives will be appointed by MCEETYA. Members will serve four year terms.

xiii


• Funding for the work of the Council will come from annual grants from the Commonwealth and from the States and Territories, and annual membership fees paid by individual higher education institutions who wish to participate in the program of reviews. • The terms of reference of the Council will be as follows: – to review within participating higher education institutions the mechanisms for quality assurance, monitoring performance and academic standards, and enhancing quality; – to publish the reports of reviews; – to report publicly from time to time on the effectiveness of quality assurance procedures in participating institutions, the extent to which procedures ensure academic standards and reflect good practice in maintaining and improving quality, and other relevant matters; – to identify and disseminate good practice in quality assurance in higher education; and – to undertake and sponsor studies related to effective quality assurance management practices and academic standards in higher education. • In carrying out reviews, review teams appointed by the Council will focus particularly on: – appropriateness of quality assurance and improvement plans in relation to institutional contexts and missions; – rigour of the mechanisms employed to review courses and academic organisational units, and monitor performance against institutional plans; – effectiveness in monitoring outcomes and in benchmarking, both nationally and internationally; and – success in communicating the results of the monitoring outcomes and academic standards to stakeholders. Review teams will report to the Council. • Each year the Council will draw up a program of reviews for the following year, after consultation with institutions likely to be reviewed. • Review panels, generally of no more than five members, will be appointed by the Council. Members of review teams will be drawn from the higher education sector, the Commonwealth and the States. Members may also be drawn from the professions and professional associations, and from business and industry.

xiv


Review panels will normally visit institutions for two consecutive days after the institution has completed a self-assessment and supplied other documentation as required. Institutions offering courses ‘off-shore’ for international students should document in detail the procedures followed for safeguarding and monitoring quality, and the results of any assessments. • Following the visit of the review team, the draft report will be forwarded to the institution for comment. Once the report is completed it will be considered by the Council and then published. Copies will be provided free to DETYA, State and Territory accrediting agencies, all participating higher education institutions, and relevant professional associations. For each review, a single report will be prepared and published. • Should a review reveal serious weaknesses, the institution concerned will be given up to 12 months to correct weaknesses prior to a supplementary review. Failure to rectify weaknesses would be a matter for DETYA to address (in the case of Commonwealth funded institutions) or for the relevant State or Territory accrediting agency. One possible action would be to remove the name of the institution from the AQF list of accredited institutions until such time that as minimum standards are achieved. • Every effort should be made to encourage private universities and non self-accrediting institutions to participate in the review program. • Prior to arrangements for the Council being finalised, the higher education sector should be consulted about the proposed terms of reference for the Council, the composition of the Council’s board and the method of conducting reviews.

xv


1

Introduction 1.1

This report has been prepared for the Commonwealth Department of Education, Training and Youth Affairs (DETYA) to assist in developing an improved national approach to both quality assurance and accreditation. The current arrangements are judged to be inadequate and DETYA considers that Australia needs a more rigorous quality assurance and accreditation system: • to protect our international reputation in respect of both the quality of our educational processes and our standards; • for public accountability purposes, particularly to satisfy the taxpayer about value for money and that government subsidies are supporting education activities of an appropriate standard; • to inform student choice, especially in the light of diversity of offerings; and • to promote and improve quality processes and outcomes at individual institutions as well as disseminate good practice, leading to overall system improvement (DETYA Project Brief 1999, p 3).

1.2

Various models for a new approach to quality assurance and accreditation have been reviewed, including refinement of the current Australian model, the recently modified New Zealand model, the new United Kingdom model, the model used by the Vocational Education and Training (VET) system in Australia and a ‘Modern Australian Model’.

1.3

Refinement of the current model could include strengthening the internal processes within universities through benchmarking and accreditation by external agencies as well as some modification to existing legislation relating to accreditation of new providers and new courses, and modifications of guidelines relating to Corporations Law.

1.4

The newly proposed modified arrangements for quality assurance in New Zealand will require providers to be quality assured through a recognised quality validation process in order to be eligible for government funding. The Quality Assurance Authority of New Zealand, which will have responsibility for the quality of publicly funded tertiary education, will grant recognition to those bodies that are able to provide credible and rigorous quality validation processes in the sector, rather than carrying out quality audits itself. Presumably the New Zealand universities may seek to have quality validation carried out by the Academic Audit Unit which was established in the early 1990s (Malcom 1993).


1.5

In the United Kingdom, quality assurance is the responsibility of the Quality Assurance Agency which was established in 1997, following submission of the Dearing Report (Higher Education for the Learning Society 1997) to the Blair Government. The Quality Assurance Agency has responsibility to assure • the standards of awards; • program outcomes against standards; and • the quality of learning opportunities. The new approach requires independent verification of programs delivered to ensure that: • they achieve their intended outcomes; and • student achievement meets the standards required of the institution for its awards by reference to subject benchmarks and the views of accrediting bodies. It is expected, for example, that Quality Assurance Agency reviewers will interact with subject/departmental review processes within universities, review overall academic management, and conduct periodic audits of overseas and other collaborative arrangements. While use will be made of members of panels from universities and the professions, a considerable amount of audit effort will be carried out by professional staff of the Agency.

1.6

2

The model of accreditation and quality assurance for the VET sector has been developed and further refined in recent years, following establishment of the Australian National Training Authority (ANTA). Under the Australian Recognition Framework that came into effect on 1 January 1998, initial registration of a training organisation is dependent on demonstrated capacity to meet National Core and relevant Product/Service Standards, and any other requirement of the relevant State or Territory. Continuing registration is dependent upon compliance measured through monitoring and audit. The Standards represent the core requirement, which the organisation needs to demonstrate to the recognition authority to become registered. Training Packages, which are developed and validated by industry, are endorsed nationally by the National Training Framework Committee of ANTA and by Education and Training Ministers. The Standards also specify national competency standards and assessment guidelines, and Australian Qualifications Framework (AQF) outcomes. Under the Australian Recognition Framework, there is also provision for the accreditation of courses where no relevant Training Packages exist.


1.7

The Modern Australian Model for quality assurance and accreditation that has been developed by DETYA seeks to build on current and recent practice. It provides separate arrangements for both institutions which have been given power to accredit their own courses and for non-self accrediting providers. For institutions with power to accredit their own courses, the main requirements will be: • rigorous scrutiny of financial and quality aspects before founding legislation is passed or other authorisation is given; • annual publication of Quality Assurance and Improvement Plans for the forthcoming triennium; • a quality audit of each institution every five years following a detailed self-assessment; and • compliance with any additional measures which may be necessary to ensure the maintenance of acceptable high standards of degrees.

1.8

For non self-accrediting providers, the main features of the approach may include: • rigorous scrutiny of provider capacity before course accreditation; and • review of provider performance and accredited courses every five years.

1.9

The task for our project was to develop Model 5 (a Modern Australian Model) as an alternative to the other four models; advise under whose authority it should be run; advise whether the framework would need a legislative base; assess whether it would be sensible and appropriate to make use of the AQF; elaborate the possible nature of the five yearly self-assessments for self-accrediting institutions; comment on the desirability of focussing more than in the past on outcomes and standards as well as processes; consider how to achieve rigour and independence for the process while retaining the cooperation and confidence of universities; and advise on the role of professional associations within the model and the nature of the audit of the courses of non self-accrediting providers. More specifically, we were asked to make a comprehensive assessment of the Modern Australian Model against the following criteria: • Credibility (how well the model would be credible with international and domestic interest groups and potential customers, and the marketability of the arrangements); • Effectiveness (ability to address learning outcome standards as well as quality assurance processes);

3


• Ability to provide legal clarity for students and providers; • Ability to promote and enhance improvement and good practice; • How well the model could build on the key features of the Australian higher education system, where universities are established under State/Territory/Commonwealth legislation as autonomous institutions with the power to accredit their own courses, and where higher education courses developed and delivered by other providers are accredited by State/Territory bodies; • How well the model could exploit the role of professional associations in accrediting courses; • Minimum prescription and bureaucracy; and • Cost (DETYA Project Brief 1999, pp 8–9).

4

1.10

The brief required consultation with key stakeholders (eg the Australian Vice-Chancellors’ Committee (AVCC), State accrediting bodies, the National Office of Overseas Skills Recognition (NOOSR) and professional associations) as appropriate. A copy of the project brief is reproduced in Appendix 1.

1.11

In carrying out the project, we have interviewed a range of key stakeholders, reviewed relevant literature and analysed key documents to which we gained access. A list of interviewees is provided in Appendix 2. Documents analysed included Commonwealth and State government reports and legislation; materials supplied by ANTA and the Australian Qualifications Framework Board Secretariat; materials from the Ministerial Council of Employment and Education, Training and Youth Affairs (MCEETYA); materials from the AVCC and professional associations; and material supplied by Universities and other higher education providers. As researchers, we acknowledge the generous assistance we received from many individuals and organisations.

1.12

For the purpose of this report, quality assurance in higher education is defined as systematic management and assessment procedures adopted by an higher education institution or system to monitor performance against objectives and to ensure achievement of quality outputs or improved quality. Quality assurance enables key stakeholders to have confidence about the management of quality and the outcomes achieved. Stakeholders are individuals and groups who have a major interest in the higher education institution or system and its achievements. This definition implies that missions, goals and objectives will be clearly specified and available to stakeholders; that


appropriate procedures will be in place to monitor and review performance; that the results of monitoring and review will be available to all stakeholders; and that mechanisms will be in place to ensure improvements in performance and the adoption of good practice. In any higher education system, it would be assumed that quality assurance mechanisms would address particular issues of concern, such as the quality of awards during a period of rapid expansion in student numbers, or the quality of courses offered to international students studying in Australia and for awards from an Australian provider through some ‘off-shore’ arrangement. 1.13

In its generic form, accreditation refers to a process of assessment and review which enables a higher education course or institution to be recognised or certified as meeting appropriate standards. In Australia, the term accreditation has developed three specialist meanings. The first is of process a review or assessment conducted by a government agency to enable a Minister or an approved authority, acting under the authority of appropriate legislation, to approve or recognise a higher education course and/or award as being of an appropriate standard and being delivered in an appropriate manner. In some cases, accreditation of a higher education institution means that from then on it is able to accredit or certify the quality its own courses, while it other cases accredited institutions must also seek accreditation for each course. A second specialist use of the term accreditation in Australia is in relation to processes carried out by a government registration body to enable graduates of particular courses to practise in the State or Territory. A third specialist use is in relation to assessment and recognition carried out professional associations in such areas as engineering, accounting, law and architecture. If a particular course is accredited, its graduates will be eligible for membership of the professional association. In this report we are primarily concerned with the first specialised Australian usage outlined above.

1.14

One important practical and theoretical question is the relationship between quality assurance and accreditation. Internationally, accreditation is often one of the main mechanisms of quality assurance but, as already noted, this report is concerned primarily with accreditation carried out by government agencies and relating in the first instance to new providers and new courses. Quality assurance, on the other hand, refers to processes of on-going review, assessment and monitoring that should apply to all recognised providers in order to ensure that courses and awards are of a high standard and that institutional monitoring of performance is effective. 5


While the standards and criteria used in the assessment of institutions and courses should inform quality assurance mechanisms, there is no clear reason why the same agency should be responsible for both accreditation and quality assurance. At the same time it is obviously desirable that Australia should have an overall consistent and robust approach to both quality assurance and accreditation. 1.15

6

In Chapter 2, a more detailed discussion is provided of the concepts of quality assurance and accreditation.


Chapter 2

Quality assurance and accreditation 2.1

This chapter explores in greater detail the concepts of quality assurance and accreditation. It also comments on the concept of quality and other concepts related to quality assurance and accreditation. A final section reviews in summary form the main approaches to the management of quality assurance at the national level that have been adopted internationally over the past decade.

2.2

Quality and quality assurance have become key issues for higher education internationally in the 1990s (Kells 1992; Kells and van Vught 1988; and Craft 1992 and 1994). In many countries, managers of higher education systems and institutions are concerned about quality and how to put in place appropriate quality assurance mechanisms. Governments are concerned about the costs of providing credible academic and professional awards and the need to ensure that standards are maintained at an appropriate level, while rapid increases in enrolments and often falling financial support per student unit raise doubts about whether quality is being maintained. Quality issues dominate the higher education debate in many countries, as ministers, bureaucrats, employers and business interests become increasingly concerned about the outputs of higher education institutions and the suitability of graduates to meet the needs of employers. Many people question whether their societies are getting real value for their massive investment in higher education and urge the adoption by governments of mechanisms to achieve more control over the work that higher education institutions do. Quality and accountability thus have become key elements in the efforts of many countries to become and remain internationally competitive in a world where interdependence in trade is rapidly growing. Apart from this, there is more emphasis on quality associated with increased mobility of professional and skilled labour, and the greater need for recognition of qualifications across national boundaries. As Craft (1994, p viii) points out: ‘globalisation’ and international migration mean that academic and professional qualifications need to be ‘portable’ across national borders, and so both institutions and nation states are keen to learn more about each other’s procedures for assuring the quality of tertiary education provision. 7


2.3

The main issues in the quality debate about higher education in many countries are the maintenance and improvement of levels of teaching, learning, research and scholarship; improvements in the quality and adaptability of graduates; how to define and measure quality; management approaches likely to improve outcomes from universities and colleges; the use of benchmarking and performance indicators; and how to convince stakeholders that institutions and systems are doing a competent job in ensuring quality outputs.

Quality and higher education

8

2.4

In one sense, the quality debate in higher education is not new, although in the past universities and government agencies used different terms such as academic standards, standards of degrees and diplomas, student assessment, and accountability (Harman 1994). In the past too, the main issues in the quality debate were largely about maintaining academic standards according to some national or international norm, the maintenance and improvement of levels of teaching and learning, and how to provide sufficient financial and other resources to achieve quality higher education. Many of these issues are still important today, but the new quality debate is largely about the achievement of quality outcomes; the establishment of appropriate management processes to monitor achievement and the extent to which specified goals and objectives are being met; assessing the suitability of graduates for the workforce and professions; and providing information to stakeholders in order to assure them of the quality and credibility of outputs. One of the big differences between the old quality debate and the new quality debate is that, while the old was concerned largely about inputs and national and international academic standards, the new is much more concerned about management processes and their effectiveness, the assessment of outputs and monitoring performance, and how well outputs meet employer and other needs.

2.5

Various studies and papers produced over the past decade or so have documented key aspects of the quality debate in OECD countries, and in major geographic regions such as Western Europe and North America (eg, Ball 1985; Sizer 1990; Frazer 1991; Neave 1991; Williams 1991; Anwyl 1992; Craft 1992; Lindsay 1992; van Vught and Westerheijden 1992; Lindsay 1994; Craft 1994; and Harman 1996a).


But quality is also becoming a major issue in the higher education systems of Asia and the Pacific, although within the region there are very considerable variations about how quality issues are perceived and in the priorities that different governments and higher education systems are giving to tackling quality problems.

The concept of quality 2.6

The recent international literature on quality and quality assurance in higher education reveals considerable difficulties and ambiguities in the definition of a number of key terms. This is not surprising as quality deals with a number of complex notions. For example, in the literature only in the widest sense is there broad agreement about what quality is, and how to define a number of key concepts used in the current debate about quality. Within many universities too, there are often quite surprising variations of views about the essential elements of quality, about what characteristics of institutional work are regarded as being of the greatest value and why, and what constitutes academic performance at the highest level and how such performance can be recognised. Sometimes too there is disagreement within many higher education institutions about what constitutes good teaching, and about which graduates have the most valued characteristics. Apart from this confusion and lack of agreement, scholars interested in researching quality issues differ significantly in their views about key terms. Many see quality as a relative concept, meaningful only from the perspective of particular judges at particular points of time, measured against some either explicit or implicit standard or purpose. But despite these differences in views, there is an increased degree of consensus emerging about key terms. For this report we draw on this growing consensus and set out a number of working definitions.

2.7

Apart from differences of views in the academic debate, managers and experts in educational measurement for many years have been wrestling with difficult technical questions about such matters as measuring academic performance of students, comparing academic standards over time and between different institutions, and devising means to ensure that teaching in academic departments or institutions is of consistently high quality.

2.8

By quality in the context of higher education, we mean a judgement about the level of goal achievement and the value and worth of that achievement. It is a judgement about the degree to which activities or outputs have desirable characteristics, according to some norm or

9


against particular specified criteria or objectives. Ball (1985) refers to quality as ‘fitness for purpose.’ According to this definition, a course of study in a higher education institution is of satisfactory quality when it conforms to the particular standards or levels of achievement for the purpose it was designed. In the absence of any overall agreed standards in a higher education system, it is necessary for institutions to specify their mission, goals and objectives and then be evaluated against these. 2.9

2.10

10

While the variety of different definitions of quality can be confusing and frustrating, different perspectives on quality have their positive aspects. The value of different perspectives can be well illustrated from the work of the American scholar, Robert Birnbaum, who in a study of American college presidents reports on three different views of quality in higher education that he found in practice. (Birnbaum 1994). He describes these as the meritocratic, social, and individualistic views of quality. With the meritocratic view, Birnbaum found that college presidents assess quality in relation to conformity to some institutional or universalistic professional or scholarly norm, and so generally use the academic profession as a reference group. Those presidents who take a social view of quality base their judgements on assessment about the extent to which the institution satisfies the needs of important constituencies and audiences -what people have now come to refer to as the stakeholders. Thus the reference point are the views of a particular community. Presidents who take an individualistic view emphasise the contributions that the institution makes to the personal growth of students, using the individual learner as the reference point. Lindsay (1992) has categorised key approaches differently, identifying two distinct approaches to discussions of quality in higher education. One approach, which he terms the ‘production-measurement’ view, treats quality as a synonym for performance, and so discussions of quality revolve around the definition and measurement of resources and outcomes. The other approach, which Lindsay terms the ‘stakeholder judgement’ view, is based on assessments by various key actors involved in higher education. These assessments may incorporate ‘imponderable elements of our conceptions of educational processes and outcomes, and their dependence on contested value positions’ (Lindsay 1992, pp 154-156).


2.11

Middlehurst (1992) usefully identifies four different ways that the term quality has been used in the recent higher education debate, primarily in the United Kingdom. These are quality as a defining characteristic or attribute; quality as a grade of achievement; quality as a particularly high level of performance or achievement which, by virtue of general consensus and reasonable stability over time, comes to be seen as a standard against which to judge others; and quality as fitness for purpose achieved through performance that meets specifications.

Quality assurance 2.12

The term quality assurance has come into the higher education vocabulary only over the past decade or so. While there are many definitions of quality assurance in the literature (eg, see Ball 1985; Birnbaum 1994; Lindsay 1992; van Vught and Westerheijden 1992), in essence, as already noted, quality assurance refers to systematic management and assessment procedures adopted to monitor performance and achievements and to ensure achievement of specified quality or improved quality. Some authors (eg Brennan 1997) prefer use of the term quality assessment instead of term quality assurance. However, while a great deal of effort in quality assurance relates to quality assessment, quality assurance in this report is thought of as a broader term which embraces not only assessment but also other activities including, for example, follow-up efforts aimed to achieve improvement.

2.13

While the concept of quality assurance is new, many of the ideas behind the concept are by no means new. What is new, however, apart from the new language, is a more systematic and far reaching approach to monitoring performance and ensuring that institutions and systems have in place appropriate and effective mechanisms for review and assessment, and for renewal and improvement. Compared with past approaches, the new mechanisms also put much more emphasis on external scrutiny, seeking the views of employers and graduates and, in various ways, making the results of assessments more widely available.

2.14

The quality assurance movement of the past decade has sprung from a variety of factors. As already noted, particularly important have been community and government concerns about academic standards and the levels of achievements of graduates in a time of major expansion in student numbers associated with decreasing government funding support per student unit. But the quality assurance movement has also

11


been driven by the impact of increased international competitiveness, the need for increased mobility of professional labour, demands for greater accountability by public institutions which flows from the emergence of the ‘evaluative state’ (Neave 1997), concerns related to the expansion of private higher education, and pressure from employers and the professions for university courses to become more relevant to work place needs. 2.15

Quality assurance has become a particularly important element in those higher education systems which have adopted a self-regulation approach to relationships between government and higher education. With this approach, governments set the policy framework and steer from some distance but put a major emphasis on monitoring performance (van Vught (1994b). Thus quality assurance becomes of vital importance. In addition, quality assurance becomes of great importance in countries where, as a result of increased government emphasis on competition, market forces and encouragement of private providers, there are community concerns about the possibility of quality being sacrificed in the search for profits.

2.16

Rajavaara (1998) has developed a typology of four different types of quality assurance: political-administrative; citizen-based; business-oriented; and professional. Under the political-administrative type, the main approaches used are legislation, service standards and quality indicators. Under the citizen-based type, the main approaches are action groups concerned about quality and the introduction of alternative social models. The business-oriented type is distinctively different depending on Total Quality Management, ISO 9000, quality awards and benchmarking, while the professional type depends mainly on professional training and professional ethics, professional audits, peer review and self evaluation.

Related concepts

12

2.17

There are a number of related concepts that are frequently used in discussion about quality. The most important of these are quality control, quality audit, quality assessment, quality management and self-study.

2.18

Quality control refers to the processes or mechanisms within an institution or system used to ensure compliance with quality standards or achieve improvements in performance. It is about evaluating and guaranteeing standards. The term appears to have been adapted from


industry where quality inspectors or quality controllers, somewhat independent of the main workforce, monitor the production process and particularly the quality of outputs. Experience in industry has shown that it is not sufficient to have an efficient quality monitoring group, but rather the workforce as a whole needs to be convinced of the importance of quality in order to achieve the highest levels of production performance. In higher education, quality control can focus on inputs, outputs, the mechanisms used to monitor quality, or some combination of these. 2.19

Quality audit refers to the processes of external scrutiny used to provide guarantees about the quality control mechanisms in place. Quality audit is based on the ideas of self-study and peer review, and on the notion of a detailed report which becomes available to the institution to assist in improving procedures and achieving enhanced outputs. The concept was developed and popularised following the establishment of an Academic Audit Unit in 1990 by the Committee of Vice-Chancellors and Principals in the United Kingdom (Frazer 1991).

2.20

Quality assessment has come to mean a review or systematic examination, usually conducted externally, to determine whether quality activities comply with planned arrangements and whether the ‘product’ (the educational process) is implemented effectively and is suitable for achieving the stated objectives. However, there are some differences of opinion in the literature as to whether the focus is, or should be, on the mechanisms to achieve quality, or the educational process and outcomes, or all of these.

2.21

Quality management has come to refer to the management of quality control and quality improvement, and to those aspects of the overall management functions that determines and implements the quality policy. (van Vught and Westerheijden 1992). It is also about the design and maintenance of quality assurance mechanisms.

2.22

The term self-study has come from the work of American accreditation agencies and refers to the internal preparation of detailed evaluation document to be presented to an outside review panel who will visit the institution and provide a written report. Many quality assurance approaches put a major emphasis on a self-study or self-evaluation.

13


Accreditation

14

2.23

The term accreditation in higher education originally came from the United States, but over the years many of the key ideas have been adopted by professional associations and government agencies internationally. This section will briefly review the development of voluntary systems of accreditation in the United States and more recently the development of accreditation by professional bodies and government backed accreditation systems in Britain and Commonwealth countries.

2.24

In the United States, for almost a century accreditation has been associated with quality assurance processes in higher education associated mainly with voluntary self-regulation carried out by professional accrediting organisations and regional accrediting associations, independent of government. Today there is a wide variety of professional accrediting organisations covering traditional professional areas such as medicine, engineering, law, dentistry and architecture and well as many newer occupational areas. Such associations accredit particular courses of study. Regional accrediting associations, on the other hand, accredit institutions (Dill 1997).

2.25

In the 1980s, in addition to the traditional processes of voluntary accreditation, a new quality assurance mechanism emerged in the United States under the rubric of assessment. By 1990, over two thirds of the states had passed legislation encouraging public institutions of higher education to implement various forms of student assessment, designed to place greater institutional attention on the improvement of student learning. Ultimately, all five regional accrediting associations also adopted an assessment criterion as one of their criteria for reviewing institutions of higher education.

2.26

In the 1990s the United States moved to establish a much more comprehensive national system of quality assurance beginning with passage of amendments to the Higher Education Act of 1992 which involved the Federal Government for the first time in quality assurance. This legislation required States to create State Postsecondary Review Entities with responsibility for reviewing the quality of all postsecondary institutions and their eligibility for federal student financial aid. In addition, under pressure from the Federal Department of Education the various accreditation agencies formed a National Policy Board on Academic Accreditation, which proposed more rigorous national standards for academic accreditation with particular emphasis on student learning. These changes were widely expected to lead to a more rigorous national system of quality assurance but, following the 1994 Congressional elections, Federal funding for the


proposed State Review Agencies was eliminate. The following year proposals from the National Policy Board on Academic Accreditation were rejected by public and private higher education institutions (Dill 1997, pp 15–16). 2.27

Prompted by the new pressures for strengthening quality assurance, over the past decade important changes have been made to the traditional American approach to accreditation in response to public criticism. This has led many of the regional accreditation associations to revise key elements of their approach, resulting in more emphasis on assessment of quality management mechanisms within institutions, reconsideration of the practice of making reports available only to the institution concerned, and making the results of accreditation processes more generally available to the public (Crow 1994). Accreditation agencies now have more detailed guidelines with clearly specified evaluative criteria. While there is considerable variation between accrediting agencies, detailed written guidelines generally focus on four main areas: • organisational and administrative matters; • resources currently available to the unit (including financial resources; personnel; space and equipment); • the curriculum; and • statistics showing the performance or other outcomes for graduates (El-Khawas 1993).

2.28

Over the past three decades, accreditation systems on the American model have been established in many countries of Asia and Latin America. In the Asian region, accreditation systems play a key role in the higher education systems of Japan, Korea, Taiwan and the Philippines. In the Philippines, a system of voluntary accreditation was developed among private colleges in the 1960s, but only in the last decade has this been extended to public universities and colleges (Arcelo 1992). Recent efforts have attempted to strengthen accrediting agencies and their procedures. The Federation of Accrediting Agencies of the Philippines, for example, has developed common accrediting standards, while a Congressional Commission recommended that the policies and practices of the accrediting agencies be reviewed periodically (Cooney and Paqueo-Arrezo 1993). Similarly, in Taiwan, well-established accreditation systems have been strengthened with academic associations being given an enhanced role (Su 1993). In Latin America, accreditation systems operate in Brazil, Colombia and Chile (Ayaraza 1994), although generally there is not a strong tradition of quality assurance, despite the extensive expansion of private higher education. 15


2.29

In Britain and many Commonwealth countries, somewhat similar mechanisms were developed for accreditation of courses by professional associations and government registration boards, and for accreditation or validation of courses in non-university sectors. Accreditation by professional associations goes back certainly to the pre Second World War period, although for many countries the history of these mechanisms is not well documented. Accreditation conducted by government agencies was first developed for non-university higher education and the particular approach of the Council for National Academic Awards (CNAA) in Britain soon became the model adopted by many Commonwealth countries. In 1990, for example, such a system was established in Hong Kong for non-university institutions with the tasks of advising on academic standards and validating degree courses offered by the two polytechnics, Lingnan College, the Academy for Performing Arts and the Open Learning Institute (Tsim 1993). In Australia, the current accrediting functions carried out by a number of State and Territory agencies grew out of the accrediting bodies for advanced education which operated in the 1970s and 1980s.

Main quality assurance approaches and methodologies

16

2.30

Over the past decade or so extensive experimentation has taken place with quality assurance and how it is managed. The literature reporting these developments points to tremendous variety in approaches and methods, and also to a significant degree of borrowing by national systems of higher education from one another. While such borrowing is to be commended, it is important that any procedures and approaches fit well within the culture of the particular system or institution. As Craft (1994 p ix) has warned, ‘procedures need to be adopted and adapted with care and sensitivity if the quality assurance/accreditation movement is not to be a new form of cultural imperialism’. Quality assurance policies in most countries currently are in a process of rapid evolution and change, although at the same time there is an increasing convergence internationally in terms of approaches.

2.31

The main approaches to quality assurance management are summarised in Table 2.1 which is drawn from a published paper by one of the authors (Harman 1998). This table sets out under separate categories information on the agency or unit with responsibility for the management of quality assurance; participation in reviews and other activities; the main methodologies employed; the focus of quality assurance activities; the purposes of such activities; and reporting and/or follow-up activities.


Table 2.1

A.

Main approaches and methodologies at national Level

Responsible agency/unit • Unit or section within a Government agency • Separate quality assurance agency established by Government • Separate agency established collectively by higher education institutions • Agency established jointly by Government and institutions

B.

Participation in reviews and other activities • Voluntary • Compulsory • Voluntary, with some measure of pressure/persuasion

C.

Methodologies of review and assessment • Self study or self evaluation • Peer review by panels of experts, usually with use of external panel members and site visits • Analysis of statistical information and/or use of performance indicators • Surveys of students, graduates, employers, professional bodies • Testing the knowledge, skills and competencies of students

D.

Focus • National reviews of disciplines – reviews of research only – reviews of teaching only – reviews of combination of research, teaching, and other activities • Institutional evaluations – reviews of teaching only – reviews of research only – reviews of quality assurance processes – comprehensive reviews usually including teaching, research, management, and quality assurance processes • Comprehensive national evaluations of higher education system

17


E

Purposes • Accountability • Improvement and renewal • Combination of purposes

F.

Reporting and follow-up activities • Report provided solely to the institution or unit concerned • Report provided to the institution or unit but also published or made more widely available • Formal reports provided to the Minister, Ministry, or coordinating board • Public reporting • Use of ranking and wide publication of the results of such ranking • Performance funding • Accreditation or validation • Improvement and renewal activities

Administrative responsibility

18

2.32

The most common pattern at national level is for responsibility to lie with a specialised unit or agency set up by Government, or with the central agency responsible for higher education coordination, whether it be a Ministry or a University Grants Commission. Thus in Denmark there is the Evaluation Centre set up by the Government with a mandate to initiate evaluation processes, develop appropriate methods for assessing academic programs, inspire and guide institutions in quality and evaluations, and compile information on national and international experiences (Thune 1994). Somewhat similar arrangements operate in France, Finland, Korea, and Thailand. On the other hand, in Sweden control of quality assurance lies with the Swedish National Higher Education Agency while in South Africa the Higher Education Quality Committee is a sub-agency of the Commission for Higher Education (Strydom 1997).

2.33

The two major issues concerning government quality assurance agencies are what degree of independence they should have both from Ministers and from major ministries and departments, and what links there are between quality assurance and funding. In summary, the main arguments for a high degree of independence are that such independence will lead to greater trust and confidence, and enhance professional judgments, whereas others argue for government control


in order to integrate quality assurance more closely with planning and coordination. An agency with considerable independence from the Minister and the agencies of executive government is the French Comite National d’Evaluation (CNE) which reports to the President and is independent of the Prime Minister, the Minister of Education and other executive agencies (van Vught 1994a). In other cases, attempts have been made to ensure that the management of quality assurance is captured neither by the Ministry nor by the higher education institutions; in Finland, for example, the Higher Education Evaluation Council is made up of 12 members appointed by the Ministry, together with representative of universities (4), polytechnics (4), student organisations (2) and business and industry (2) (Liuhanen 1997). 2.34

In a small number of countries, responsibility for aspects of quality assurance at the national level is under the control of an agency set up by higher education institutions themselves. Such examples are found in the Netherlands, Italy and in New Zealand (at least up to the present), where quality assurance programs are conducted by agencies set up by the peak association of universities. In the Netherlands, the current system of quality assurance for both the University and the non-University sectors sprang from a restructuring in the mid-1980s of the relationship between the Ministry of Education and Science and higher education institutions. An understanding was developed that, in exchange for a greater degree of financial and managerial autonomy, the institutions would demonstrate that they were offering quality education. Originally it was planned that this assessment of quality would be a responsibility of the Inspectorate for Higher Education but, in the end, after negotiations, the two voluntary bodies representing higher education institutions in both university and non-university sectors respectively agreed to take responsibility. However, follow-up activities are the responsibility of the Inspectorate of Higher Education, an independent body set up by the Government (Zijderveld 1997). In Italy since 1992 an important role in quality assurance has been performed by the Italian Standing Conference of Rectors (CRUI) which stimulates reflection and dialogue on issues related to the establishment within universities of periodic evaluation practices, and provides assistance to universities in setting up their internal evaluation systems. The CRUI has also created a common information system based on evaluations (Boffo and Moscati 1997). The New Zealand scheme for University audits has been under the control of universities rather than the Government (Malcom 1993).

19


2.35

An example of joint government-university control is in Korea where responsibility lies with both the Ministry of Education and the Korean Council for University Education. In 1992, the Ministry and the Council agreed to establish an independent accreditation body within the Council, called the Council of University Accreditation. The latter body is composed of 16 representatives from universities, industry and government. The Ministry of Education and the Korean Council for University Education jointly decide annually which university departments will be evaluated, while the Council for University Accreditation consults with relevant professional associations and organises accreditation committees made up of university staff. After each evaluation, staff of the Korean Council for University Education reviews reports and produce total scores for each department, leading to grading of departments as good, moderate, or poor. The list of good departments is announced and reports and documentation are considered by the Ministry of Education’s Advisory Council for Higher Education (Lee 1993).

Participation in the program

20

2.36

An important variation between quality assurance systems is whether participation is voluntary or compulsory. Many countries began with institutional audits on a voluntary basis. Thus, in Britain the institutional audits run by the AAU were voluntary (Williams 1991), while the Research Assessment Exercise run by the Higher Education Funding Council of England (HEFCE) for the funding councils continues to be based on the principle of voluntary participation. In Finland, when the Ministry of Education launched the program of university reviews in 1991 on an experimential basis, two universities—Oulu and Jyvaskyala—volunteered to be involved. When the Evaluation Centre was set up in Denmark one of its three guiding principles was that participation would be voluntary (Thune 1994).

2.37

Generally, however, with national reviews of disciplines participation is compulsory and even when participation is voluntary strong moral and professional pressures usually result in a high level of participation.


Methodologies 2.38

Most quality assurance mechanisms depend on one or a combination of a limited number of key methodologies, the most important of which are self studies or self evaluation; peer review by panels of experts; the use of relevant statistical information and performance indicators; and surveys of key groups, such as students, graduates and employers.

2.39

Self studies have proved both effective and cost efficient, achieving a high degree of ownership since key staff are heavily involved and such involvement increases the chances of improvements being achieved. Experience points to the value of combining self-studies with external peer review to ensure that evaluation is taken seriously and outside perspectives are included.

2.40

Self-studies first developed in the United States with institutional and course accreditation, but over the last decade or so it have become an important feature of many quality assurance systems. Self-studies have many positive features: they are cost effective, since the main work is done internally, often with little additional resources being necessary; they usually achieve a high degree of ownership since key staff are heavily involved and such involvement increases the chances of substantial improvement being achieved; and the process of review or assessment is made less threatening. On the other hand, experience points to the value of combining self study with some element of external peer review, especially to ensure that the self-study is taken seriously and to bring in outside perspectives. Combination of self-study with external peer review provides a strong incentive for staff to take the activity more seriously. One of the strongest pressures on any group of academics is the prospect of being judged by senior peers in the discipline.

2.41

Peer review is a well-established academic process and generally works well provided external members are included and panel members show respect for the values of those being evaluated and accept that often their main contribution will be in assisting with self-learning. At the same time, it must be recognised that peer review can easily introduce outside values and constructs. In its traditional format, peer review generally involves a visit by a group of well-regarded academics in the particular field but recent practice, especially for reviews of programs or disciplines, has been to add other experts to panels, such as persons from industry or business, practising professionals, or elected public officials.

21


22

2.42

External reporting often is thought necessary in order not only to ensure accountability requirements but that staff take a self-study seriously. Related questions are to whom should external reports go, and how widely and publicly should such reports be distributed.

2.43

Most evaluations combine self-study with the use of statistical information and/or performance indicators, and now frequently the results of surveys of students, graduates and employers. In some cases, a national program of reviews is accompanied by the use of national statistical collections and published performance indicators. In the case of France, for example, the CNE quality assessment disciplinary reviews begin with self-evaluation reports produced by the institutions being reviewed and statistical reports produced by the CNE, government offices and the institution (van Vught 1994a). In Finland, an extensive nationwide university data base (KOTA) was established in the late 1980s, containing key statistics about university performance by institutions and disciplines (Liuhanen 1997). This data base covers the following topics: applications for admission; home and foreign students; degrees including the duration of masters degrees; teaching and other staff; appropriations; premises; graduate placements; continuing education; open university instruction; researcher visits abroad; scientific publications; and the target number of degrees agreed in Ministry-University consultations.

2.44

In a number of counties, particularly the United States, various initiatives have been taken to develop tests to assess student knowledge, skills and competencies. The main efforts here have been at institutional level, although there are examples of statewide initiatives. By 1990, over two thirds of states had passed regulations encouraging public institutions of higher education to implement various forms of ‘student assessment programs’ designed to place more emphasis on improvement of student learning. Regional accrediting bodies also have adopted new assessment criteria for reviews of higher education institutions (Dill 1997).


Focus 2.45

At the national level, the most common forms of assessment are ‘horizontal’ reviews of disciplines and ‘vertical’ evaluations of institutions. Reviews of disciplines are usually carried out by panels of experts using site visits and analysis of documentary information, much of which is usually produced by the academic units being reviewed. Some disciplinary reviews concentrate on teaching only, some on research only, while others look at both teaching and research. Generally disciplinary reviews result in published reports, with often detailed comments being made on the work within each department or faculty.

2.46

Institutional reviews include academic audits of quality assurance processes and outcomes, and more extensive comprehensive reviews. The international practice of institutional academic audits has been considerably influenced by the methodology developed a decade ago by the AAU in the United Kingdom. Under this approach, the focus is a meta-evaluation of the mechanisms and approaches to quality assurance management, rather than an assessment of the quality achieved. Institutions are visited by small teams of academics, following a ‘negotiated invitation’, and these teams make an on-site audit using a checklist of good practice. A general report is prepared for the university as a whole, while confidential reports on sensitive issues are produced for the Vice-Chancellor. Practice varies on whether reports are published, and whether the results are reported officially to the government or funding agency.

2.47

In a number of countries, there is a long tradition of periodic national, comprehensive reviews of higher education, often conducted by special committees or panels. Thus in Britain, there was the Robbins Committee in the 1960s and more recently the Dearing Committee, while in Australia parallel reviews have been held under the leadership of Sir Leslie Martin and Mr Roderick West. In many cases, however, such comprehensive reviews are not considered as part of a national program of quality assurance, although in Sweden the brief of the new Swedish National Higher Education Agency includes investigating and evaluating the higher education system and its results in relation to the society’s overarching general goals for higher education (Asking and Bauer 1997).

23


Purposes 2.48

Quality assurance programs can often serve a variety of purposes but generally their primary purposes are a combination of public accountability, improvement and renewal. In some cases, there is a gap between stated purposes and actual purposes and frequently there is tension between accountability and improvement purposes.

2.49

Frequently the stated purpose of a national quality assurance program is linked to a particular philosophical approach to evaluation and to particular views about the role of government in the control of higher education. As already noted, the current system of quality assurance in the Netherlands was clearly linked to a new philosophy about the relationship between the state and higher education institutions and on a belief that the assessment process should be as non evasive as possible (van Vught 1994a).

2.50

Similarly, in the case of Sweden, the approach adopted by the Swedish National Higher Education Agency stems from the transition from ex-ante regulation to devolution of authority and ex-post control (Bauer and Franke-Wikberg 1993). While the Agency has a number of functions including both enhancing and controlling quality, to date it has given clear priority to supporting and enhancing activities. In its own guidelines, the Agency has underlined its supportive role and expressed its strong commitment to a ‘soft’ approach to its evaluative tasks (Askling and Bauer 1997).

Reporting and follow-up 2.51

24

Reporting and follow-up activities are vital components, but a major challenge is to devise fair and effective methods likely to lead to improvements without damaging the unit or units being reviewed. A variety of approaches are widely used with regard to the distribution of reports. In some cases, reports are provided solely to the institution or the unit concerned, but increasingly the practice is to make the results more widely available. At national level, reports for institutional evaluations or disciplinary reviews now are frequently provided to Ministers, Ministries and funding agencies. Precisely what happens to a report can be one of the most contentious issues in quality assurance programs. While participants in the institution or department being assessed often wish to limit circulation of a report, particularly one which includes critical comments, the demands of accountability usually require wider circulation.


2.52

Associated with this is a difficulty often experienced by review panels in preparing reports which will be considered by different stakeholders. What is desirable is that the issues of reporting and follow-up should be explicitly addressed in guidelines prior to commencement of any review. In this way both reviewers and the unit being reviewed will know from the outset who will see the final report. However, it is widely accepted that whatever the final distribution of a report, the institution or unit being assessed should have an opportunity to comment on the draft report.

2.53

Some agencies have adopted systems of rankings based on performance in relation to established criteria. In the United Kingdom, for example, participating university departments are ranked in separate reviews of performance in both teaching and research, and these results are publicly announced.

2.54

In a minority of cases, some element of performance funding is used as part of a quality assurance program. In the United Kingdom, performance in research as measured by the Research Assessment Exercise (McNay 1997) is used as the basis by the three funding councils and the Department of Education of Northern Ireland for allocating substantial research funding to institutions. In the United States, performance funding has been used for many years by the state of Tennessee to improve higher education by adopting a single set of outcomes and rewarding institutions for their performance (El-Khawas 1997). Currently the maximum reward for performance is an amount equal to 5.5 per cent of an institution’s overall budget. In the fiscal year 1995, $27 million was allocated.

2.55

A number of arguments are advanced in favour of performance funding: key ones are that such mechanisms provide strong incentives towards excellence and sends out clear messages from government agencies to institutions and academic staff. On the other hand, opponents argue that performance funding can distort the purposes of evaluation, damage the links between evaluation and improvement and, by denying funding to lesser performing departments or institutions, damage their reputations, their ability to recruit staff and students, and their capacity to improve.

2.56

In a limited number of cases, the final result is accreditation or validation of the program or institution. As already noted, accreditation has had a long history in the United States. In late 19th century America, growing diversity in institutional forms and lack of centrally defined standards led to a degree of chaos. In the end, institutions took the initiative and developed mechanisms of quality assurance which included accreditation of institutions and academic programs. 25


2.57

Accreditation usually involves a process of self-review by the organisation or unit seeking accreditation, resulting in the development of extensive documentation with detail on goals, resources, facilities and internal evaluation mechanisms, as well as an assessment of achievements in relation to goals. Production of the self-study is usually followed by a visit of a team of external assessors and a final decision, using pre-defined standards on whether or not the institution or the program meets the specified criteria.

Concluding comments

26

2.58

In reviewing recent international practice with regard to the management of quality assurance, especially for academic activities, this chapter has pointed to the tremendous variety in experimentation that has taken place in recent years, providing a rich source of models of evaluation and review, and of reporting and follow-up activities. In many countries, the main emphasis at the national level has been on academic audits and institutional evaluations, and reviews of disciplines and professional areas. The current experimentation seems likely to continue, especially with the effects of increased of increased trends towards globalisation and increased economic competition between nations. Many of the experiments appear to have produced positive benefits, including improvements in academic programs, closer links with employers and professions, and increased confidence among key stakeholders. On the other hand, questions are being asked about the financial and administrative costs of quality assurance mechanisms in relation to the benefits derived.

2.59

As a result of the experimentation of the past decade, there is now a growing body of experience and evidence available about how well different approaches are working in particular settings. Such information can be of considerable assistance to national higher education systems or institutions interesting in developing new quality assurance systems, or modifying existing systems. One lesson to be learnt is that great care should be taken is selecting mechanisms likely to enhance credibility both nationally and internationally and in estimating resource implications. A number of the methodologies in frequent use can prove expensive to implement in terms of both personnel time and financial resources, although with most methodologies there are usually ways of keeping administrative costs in check, at least to some extent.


2.60

International experience demonstrates well the value of placing a major emphasis on quality improvement within a quality assurance program. While most quality assurance programs quite understandably have accountability as a major driving requirement, it is highly desirable that all quality assurance programs be deliberately designed to achieve improvement and renewal.

2.61

In order to succeed and produce major benefits at either institutional or system levels, any quality assurance program needs the support of the higher education community. In particular, the benefits will be disappointingly limited unless academic and administrative staff can be persuaded to participate and provide support. Gaining this support may not be easy, since in a number of the recently introduced quality assurance programs academics have seen the particular initiative as constituting a threat to their professional independence and work.

2.62

In developing a new quality assurance program or evaluating an existing program, the following criteria are suggested as constituting highly desirable features: • The purposes of the program are explicitly stated, with the overall direction fitting well with the culture and values of the particular system or institution; • Approaches and methodologies are congruent with the stated purposes of the program, and likely to be cost effective and attract the support of staff; • The methodology incorporates elements of self-study, peer review, and external reporting; • Guidelines are clear and provide for a transparent process, where judgements will be based on analysis of evidence and the procedures will be fair to all parties involved; • Guidelines provide for checklists to assist review panels, and for the institution or unit being reviewed to have input into the choice of external panel members; • Administrative arrangements provide for an overseeing committee or group to have responsibility for the organisation of any review, including the appointment of panels, and review of the reports of panels; • Clear external reporting arrangements are specified in the guidelines, or the procedures provide for reporting arrangements to be agreed by the parties concerned prior to commencement of any review; and • The program places major emphasis on improvement, renewal and the application of ‘good practice’. 27


Chapter 3

Australian higher education providers and current arrangements for accreditation and quality assurance 3.1

This chapter describes and categorizes Australian higher education providers in the context of accreditation and quality assurance, and then outlines the current arrangements for accreditation and quality assurance. This is intended to provide a basis for assessing the strengths and weaknesses of the current system in relation to particular providers and proposed changes.

Australian higher education providers 3.2

For the purposes of this study, Australian higher education providers can be categorized into five distinct groups. The first group comprises public universities and other public higher education institutions established under State, Territory and Commonwealth legislation. This group includes the 37 public universities and also a small number of non-university higher education institutions, such as the Australian Maritime College in Launceston and Batchelor College in the Northern Territory. All these institutions have the power to approve or accredit their own courses and all receive operating grant funding from the Commonwealth under Higher Education Funding legislation. Significantly, the Australian Maritime College was established under Commonwealth rather than Tasmanian legislation.

3.3

A second group of institutions is made up of non-government institutions which operate under their own legislation and have selfaccrediting powers. The oldest of these is the Melbourne College of Divinity, established in 1910 by an act of the Victorian Parliament. The College was established because the University of Melbourne was unable to offer degrees in Divinity springing from traditions of separation of church and State (Smith 1998, p 3). It is operated by participating Christian denominations as a private institution, without the benefit of government funding. Costs are met through student fees and endowments. More recent institutions are Bond University on the Gold Coast and Notre Dame University at Fremantle, both of which enjoy their own acts of parliament, giving them similar powers to public universities. These two universities depend on

29


fee and endowment income, although the Commonwealth Government has agreed to fund Notre Dame courses for indigenous students at Broome.

30

3.4

The third group comprises institutions not established by legislation but who have been given government approval to operate. The best known example is Melbourne University Private, which is a joint venture between the University of Melbourne and private partners. It gained approval in 1998 to operate for a period of five years under the Victorian Tertiary Education Act 1993 (Smith 1998, p 11) but this was conditional on the University of Melbourne bering responsible for certification of its courses (but not for accreditation, which lies outside the statutory powers of the University of Melbourne) This category also includes the National Art School in Sydney, which was once a TAFE institution but now an independent higher education institution funded by the state. However, it is not self-accrediting and each of its courses must be considered separately for accreditation. In 1998, the School gained approval for a Bachelors Degree in Fine Arts under the approval process provided under the New South Wales Higher Education Act 1988.

3.5

The fourth group comprises private providers whose courses have been accredited by State or Territory accrediting agencies. In 1998 there were 68 authorised providers offering 225 accredited undergraduate degree and postgraduate award courses. Details of these are set out in Table 3.1


Table 3.1

Higher education accredited awards offered by private providers

Bachelor

Grad Cert

Grad Dip

Master

Doctoral

Vic

17

2

10

5

1

NSW

24

2

26

6

0

SA

26**

9

14

9

1

ACT

6**

25

15

–

–

Qld

9

3

6

3

–

WA

1

–

–

–

–

Tas

–

–

–

–

–

NT

–

–

–

–

–

Nat. Total

83

41

71

23

2

* appropriate numbers as advised by State and Territory Officials, August, 1998 ** includes TAFE degrees Source: Smith 1998, p 3.

3.6

Private providers have existed for a long period, generally operating in relatively small but often well-established market niches. The most durable of these providers are the theological colleges and churchrelated colleges, whose courses generally closely resemble university courses. The largest are Avondale College operated by the Seventh Day Adventist Church and the Australian College of Theology, which is a national federation of various denominational theological colleges. The Commonwealth funds teacher education courses at Avondale College. Another sub-group in this category are long-term ‘industry’ commercial providers who conduct courses at tertiary level to meet the needs of their particular market areas. Notable features of this subgroup are high motivation of students, a focus on particular discipline areas and user-pays principles.

3.7

Smith has further divided this sub-group of commercial providers into two separate sub-groups, with a third emerging (Smith 1998, p 3). First, there are the highly specific professional associations, such as the Securities Institute, the Institution of Engineers and the Royal Australian College of General Practitioners. For many years, these and similar bodies have conducted ‘in service’ professional education programs for their members, awarding credentials on completion.

31


Courses have been conducted at a level which has led to consistent success in obtaining accreditation for their courses.

32

3.8

The second set within the sub-group consists of established commercial providers such as general business colleges offering programs across the full spectrum of commercial courses. While there have been some spectacular individual failures in this area, these colleges have generally met an important market need and continue to do so.

3.9

A third set consists of loosely coordinated but rapidly growing institutions in the area of alternative health practice. Traditional Chinese Medicine and other alternative approaches to health care such as Naturopathy and Homoeopathy have become increasingly popular throughout Australia in recent years. While training courses in these areas have been available from private providers for many years, the considerable public interest at the present time and the need to ensure that such practices are safe to the public, is causing public health authorities to review the level and nature of training offered in alternative health practice.

3.10

Alternative health practitioners themselves are recognising that the growth of their profession will be dependent on having agreed minimum acceptable academic standards. Last year over thirty Traditional Chinese Medicine organisations across Australia have been meeting to reach agreement about minimum standards of training. These practitioners seek to determine standards appropriate for achieving professional status. Several universities, notably Southern Cross, Victoria University of Technology and RMIT have already responded to industry demands and are offering degree programs in acupuncture. Private provider programs in naturopathy have been approved in Queensland and Victoria.

3.11

A fifth category comprises private providers whose courses have not yet been accredited. Relatively little is known about this group, but it includes providers who offer sub-degree work and providers whose courses are approved for credit transfer for entry to degree programs at various public universities. Examples of the latter include providers established by universities (such as Insearch owned by the University of Technology, Sydney and the Sydney Institute of Business and Technology which has been established by Macquarie University) and private providers who have established special relationships with particular university faculties (such as the Faculty of Economics at the University of Sydney with the Universal Education Centre).


3.12

In 1998, public higher education institutions in category 1 enrolled about 671 853 students of whom some 72 183 were international students. Comparatively little is known about enrolments in the other five categories. However, one State officer has estimated that there may be 15 000 higher education students enrolled in courses accredited by State and Territory agencies while a Commonwealth official estimates that total enrolments in higher education courses outside Commonwealth funded institutions may total 40 000 to 50 000 students.

International education enrolments 3.13

Since pressures for strengthened accreditation and quality assurance arrangements are being driven substantially from concern to protect the higher education export industry, it is important to consider more closely details of international student enrolments. Of the total of 72 183 international students enrolled in 1998 in institutions funded by Commonwealth operating grants (plus the Australian Film and Television School, the National Institute of Dramatic Art and the Australian Defence Forces Academy), 37 719 were males and 34 464 were females. Total load was 59 463 EFTSU. The three most important sources of international higher education students by country, in rank order, are Malaysia, Singapore, and Hong Kong. In 1997, according to UNESCO figures, Australia ranked third after the United States and the United Kingdom in terms of the total number of enrolled overseas students. With the exception of Singapore (where Australia is the top provider) and Malaysia (where the United Kingdom is top provider), the United States is the most popular destination for international students from all of Australia’s top 10 source countries. Australia ranks second behind the United States in Hong Kong, Indonesia, India, Thailand and China; and third behind the United States and the United Kingdom in Taiwan, South Korea and Japan. Australia outranks Canada and New Zealand in all of Australia’s top 10 source countries.

3.14

Of the total number of international students enrolled in 1998, a total of 22 583 were enrolled in ‘off-shore’ programs. Of these, 65.9 per cent were enrolled in bachelors degrees while 21.6 per cent were enrolled in masters degrees by course work. Table 3.2 shows the total off-shore enrolments of Australia’s major ‘off-shore’ providers. Together, these seven providers account for almost 15 000 students. In terms of mode of attendance, 60 per cent of total Australian ‘off-shore’ higher education students are internal, 38.8 per cent are external and the remainder are classed as multi-modal. In terms of total numbers of

33


Australian off-shore students in different countries, the largest number are in Hong Kong (7 204) followed in rank order by Singapore (6 898), Malaysia (2 994), New Zealand (584) and Japan (560). The largest concentrations per country for single institutions are shown in Table 3.3.

Table 3.2

Total ‘off-shore’ student enrolments of major higher education providers, 1998

Institution

Enrolments

RMIT 3008 University of Southern Queensland

2323

Curtin University of Technology

2211

Monash University

2181

University of Ballarat

1836

University of South Australia

1711

Charles Sturt University

1549

Source: DETYA, Selected Higher Education Statistics 1998

Table 3.3

Largest off-shore enrolments in particular countries by institutions, 1998

Institution

Country

Enrolments

RMIT

Singapore

2202

VUT

Hong Kong

1255

Uni of South Australia

Hong Kong

1035

Curtin University

Hong Kong

975

Curtin University

Singapore

952

Uni of South Australia

Malaysia

928

Source: DETYA, Selected Higher Education Statistics 1998

34


Current accreditation arrangements 3.15

Accreditation of higher education institutions and courses in other than universities and other institutions established under their own legislation (State, Territory or Commonwealth) is under the control of State and Territory Governments who view this responsibility as flowing from their responsibilities for education under the Commonwealth constitution. The relevant legislation in each of these jurisdictions is shown in Table 3.4. Most acts listed make provision for private providers of higher education to secure accreditation and approval to offer courses. In other cases, legislation provides for accreditation of institutions and courses. The main aim of the various legislation is to protect the status and quality of awards, to ensure that private providers have met minimum criteria with regard to facilities and staff capacity, to ensure that the provision of higher education services by private providers is consistent with that offered by publicly-funded institutions and, in keeping with National Competition principles, ensures that private providers to offer courses which meet appropriate standards. Some of the legislation also aims to protect students from providers who cease to be financially solvent.

Table 3.4

Legislation providing for accreditation of courses and institutions

Victoria

Tertiary Education Act 1993

Queensland

Higher Education (General Provisions) Act 1993

South Australia

Vocational Education and Training Act 1994 and Business Names Act

Tasmania

Universities Registration Act 1995

New South Wales

Higher Education Act 1988

Australian Capital Territory

ACT Vocational Education and Training Act 1995

Northern Territory

Northern Territory Education Act 1995

Western Australia

Business Names Act 1962

Source: Smith 1998, p 6.

35


Accreditation of courses 3.16

While the details of the arrangements differ between the various states and territories, in each case providers make applications following a specified format. Applications are subject to a rigorous review process carried out by expert panels which include experienced personnel from public universities. Panels make recommendations to the Minister, or to a person or body with powers delegated from the Minister or provided for in legislation.

3.17

Each of the States and Territories has developed criteria and procedures that determine the way that applications for course accreditation are assessed. For example, the Victorian Tertiary Education Act 1993 , obliges the Minister to have regard to whether the course of study is equivalent in standard to a course leading to an award of that type or level in a University. In deciding whether or not to accredit a course, the Minister may take into account all or any of the following matters: (i)

student selection procedures;

(ii) the number of students; (iii) class sizes; (iv) student contact hours; (v)

curriculum;

(vi) premises, equipment, materials and resources; (vii) course nomenclature; (viii) qualifications and experience of staff (Tertiary Education Act 1993, p 10).

36

3.18

The Victorian Minister is advised by University academics and industry specialists through a Ministerial Standing Advisory Committee and various ad hoc expert panels (Smith 1998, p 8).

3.19

In New South Wales, under the Higher Education Act 1988, detailed guidelines have been developed dealing with the documentation requirements of applications for course accreditation, the composition and selection of new assessment committees, and the role of chairperson and secretary of assessment committees. For committees assessing courses leading to the award of a bachelors degree or higher level, the New South Wales Vice-Chancellors’ Conference nominates two members, one who is knowledgeable in the major academic field being addressed in the course under assessment and the second who is experienced in course planning and assessment. The 1988 Act was drafted with accreditation of advanced education courses in mind and,


possibly for that reason, no explicit specification is provided of the assessment criteria to be used. 3.20

In Queensland, accreditation of higher education courses is governed by the Higher Education (General Provisions) Act 1993. Under this legislation, the Minister is the accrediting authority for courses leading to higher education awards. The Minister may accredit a course for a higher education award or proposed award if he or she ‘is satisfied, following an assessment made in accordance with accrediting procedures and criteria approved by the Minister, that the course, and the way of delivering it, are appropriate to the type of award’ (Section 10 (2)). Detailed documentation is available to applicants from the Office of Higher Education in Education Queensland. This sets out procedures and criteria for the accreditation of higher education courses offered by non-university providers. These procedures and criteria are probably the most detailed of any State and Territory and cover topics including the nature and purpose of accreditation, the accreditation process, accreditation fees, criteria for accreditation, and required documentation. The extended section on criteria states that a course assessment panel ‘must satisfy itself that the application before it meets the criteria with respect to the standard and quality of the course and the capacity of the provider to deliver it’ (Education Queensland, Office of Higher Education 1997, p 19). The section on criteria then goes on to specify information required on such matters as statement of mission and purpose of the course, legal status of body providing the course and governance arrangements, educational oversight, composition of the course advisory committee, the financial standing of institution, qualifications and expertise of staff, and educational requirements. Under educational requirements, it is stated that, before recommending accreditation, a course assessment panel must be satisfied that: • the field of study in which the course is proposed does indeed constitute a coherent body of knowledge, supported by an appropriately developed theoretical framework, and a substantial body of scholarship and/or reflective professional practice; • the course proposed is comparable in standard and educational value to a course leading to a similar award in a university, in terms of the overall goals for the course, course entry requirements, the breadth and depth of course content, the structure of the course in terms of the mix of general and specialised skills and knowledge, the levels of skills and knowledge developed, the

37


duration and workload of the course, and the methods of delivery and assessment; • the course satisfies the guidelines for the proposed qualification laid down in the Australian Qualifications Framework (AQF); and • the general educational practices and standards of the provider, and the processes adopted for monitoring the operation of the educational program, are adequate to maintain the offering of the course (Education Queensland, Office of Higher Education 1997, p 23).

38

3.21

In contrast, procedures and criteria in use in the Australian Capital Territory are much briefer, reflecting the smaller staff capacity of the relevant section of the Department of Education and Community Services. Power to accredit higher education courses in the ACT comes from the Vocational Education and Training Act 1995 which gives the Accreditation and Registration Council power to ‘accredit courses in the higher education sector, including but not limited to vocational education and training courses, whether provided in the Territory or elsewhere’ (Section 13 (c)). While assessment criteria are not set out, these are specifically referred to and the Department must provide on request copies of assessment criteria. The Act gives the Minister power to examine, or cause to be examined, the operations of a non-university provider, including the standard of the course, the way of delivering it, and the ability (including financial ability) of a non-university provider to deliver the course. Such examinations may take place during the period for which a course is accredited.

3.22

Recently the States and Territories have agreed on procedures for considering applications and authorisation to offer higher education courses in two or more States and Territories, and operational guidelines to achieve this were endorsed by MCEETYA in April 1999. These guidelines cover the procedures for considering applications, format of applications, fee structure, determination of which state or Territory authority should consider an application, concurrent assessment panels, and accreditation outcomes. Joint or concurrent accreditation procedures are now in place and the states and territories are currently working to achieve greater consistency in criteria and procedures. Significantly, the document endorsed by MCEETYA made no mention of the criteria for assessment and accreditation.


Accreditation of new or overseas universities 3.23

The States and Territories have overall responsibility for the higher education institutions that operate within their jurisdictions and in a number of cases through legislation there is control over the use of the terms ‘university’ and ‘degree’, and over degree titles. However, there are significant differences in terms of legislation and the processes by which new or overseas universities can be recognised.

3.24

Table 3.5 summarises the provisions in the various States and Territories with regard to recognition of new universities, recognition of universities established in other States and Territories, and recognition of foreign universities, while Table 3.6 summarises legislative protection offered nationally to Australian universities. It will be noted that the most detailed legislative controls operate in Victoria, New South Wales, Queensland and Tasmania. From Table 3.6 it will be noted that The Trade Practices Act and related State/ Territories legislation protects against misleading advertising. Business names legislation restricts the use of the word ‘university’. The Corporations Law places restrictions on the use of the name ‘university’. Under relevant guidelines consent will normally be granted to use the word ‘university’ where a body has been given accreditation to operate as a university. This includes a foreign body accredited outside Australia. The difficulties arising from the breadth of the guidelines in The Corporations Law is currently the subject of discussion between the Treasury and DETYA. The guidelines established as part of this process may be useful to the States in relation to their business names legislation.

3.25

In Victoria, the Tertiary Education Act 1993 protects use of the terms ‘university’ and ‘degree’, regulates the titles of all higher education awards, and regulates the establishment of new universities in the State. There is no impediment to universities recognised elsewhere in Australia operating in Victoria, although they must have Ministerial approval if they wish to deliver courses to overseas students. Foreign universities require Ministerial approval to operate. To date, major reviews have been conducted under the 1993 Act to consider applications from Ballarat University College and Melbourne University Private. For the 1993 review of the application of Ballarat University College, the Panel placed considerable emphasis on 1989 criteria for the essential characteristics of Universities published by the AVCC. It recommended that a new University be established under its own act of Parliament, with a five year sponsorship by an existing major

39


university (Report of Review Panel 1993, p 23). The 1998 review of Melbourne University Private also used the 1989 AVCC criteria but noted that the AVCC had in 1997 published further criteria. The report of the panel commented as follows: The proposal from Melbourne University Private asserts that the AVCC criteria are not applicable. The panel does not agree with this view. However, the Victorian Government’s policy commit ment to the AVCC criteria relates to the earlier statement rather than the later one (Report of Panel 1998, p 16). Table 3.5

Legislation relevant to the establishment and operation of Australian universities

State/Territory

Legislation

Western Australia

• No generic legislation protecting title ‘university’, or degree or regulating establishment or operation of a university • There is no impediment to a university from another State operating in WA and no explicit protection against ‘bogus’ or overseas institutions.

Victoria

• The Tertiary Education Act 1993 protects the title ‘university’ and ‘degree’ and regulates the establishment of universities in Victoria • There is no impediment to universities recognised in other Australian States/Territories operating in Victoria. However, if they want to deliver courses to overseas students in Victoria, they must have the Minister’s endorsement. Foreign universities require Ministerial approval.

Australian Capital Territory

• There is no explicit legislation protecting the title ‘university’ or ‘degree’ or regulating the establishment and operation of a university, including a foreign university.

New South Wales

• The NSW Higher Education Act 1988 regulates accreditation of higher education courses & nomenclature and protects the use of the title ‘university’ • Varying levels of regulation apply to overseas universities seeking to operate in NSW, but not to other Australian universities • Universities are regulated by individual university enabling acts.

Northern Territory

40

• The NT Education Act places certain restrictions on the conferring of higher education awards (this includes overseas institutions, but not other State/ Territory universities). There is no explicit provision covering the establishment of a new university.


Queensland

• The Higher education (General provisions) Act 1993 explicitly protects the title ‘university’ and ‘degree’ and regulates the establishment of universities in the State. It allows other Australian universities to operate in QLD, but requires an overseas accredited university to seek Ministerial approval.

South Australia

• The Vocational Education, Employment and Training Act 1994 (VEET Act) provides for the accreditation of degree courses (except for a South Australian university, but including universities from other Australian States and overseas universities). • There is no explicit education legislation protecting the title ‘university’ or regulating the establishment or operation of universities in South Australia.

Tasmania

• Tasmania has explicit legislation (Universities Registration Act 1995 and Universities Registration Amendment Act 1997) protecting the title ‘university’ and ‘degree’ and regulating the establishment of universities in the State. It also controls foreign universities. • The University of Tasmania was established under its own legislation. • There is no impediment to universities from other States operating there.

Source: Papers for Meeting of Multilateral Joint Planning Committee, 30 June 1999

Table 3.6

Summary of legislative protection offered nationally to Australian ‘universities’

The Trade Practices Act and related State/Territory legislation

Protects against misleading advertising

Corporations Law

Protects term ‘university’, but requires tighter guidelines

State Business names Legislation

Protects titles ‘university’ & ‘degree’—but may need tighter guidelines

State Higher Education Legislation

• Protects the title ‘university’ in Victoria, NSW, QLD, Tasmania • Prevents overseas institutions operating except with permission in Victoria, NSW, NT, QLD, South Australia, Tasmania

Note: The legislation does not provide protection against an institution of questionable quality, which has been accredited in a State or Territory, from operating except in South Australia. Source: Papers for Meeting of Multilateral Joint Planning Committee, 30 June 1999

41


3.26

In applying the original AVCC criteria, the review panel for Melbourne University Private saw the need for some flexibility in view of the special nature of the proposed institution. At the same time, it placed considerable importance on the maintenance of standards. This led it to recommend that, as a condition of approval, awards be offered only if they were certified by the University of Melbourne. It also recommended that Melbourne University Private plan to develop its own independent research profile and have at least three per cent of its student load in graduate research programs (Report of Panel 1998, p 23).

3.27

In these two assessments and in assessments in other States and Territories, the original 1989 AVCC guidelines have played an important role, even though these were developed at the time of the foundation of the Unified National System as a device for controlling entry to membership of the AVCC. The 1989 criteria were detailed and highly restrictive. In order to achieve the status of a university, the AVCC guidelines specified that an institution must meet twelve criteria including a commitment by its staff ‘to the search for and preservation of knowledge by teaching and research’, ‘courses which meet national and international standards at a high level’, ‘a fundamental commitment to the training of researchers’, academic staff with ‘high qualifications and professional standing in the community and with their peers’, and ‘a high level of material and financial resources to support its educational activities on a continuing basis’ (AVCC 1989). In assessing whether an institution met the detailed criteria, the following minimum quantitative indicators were to be used: (i)

the institution should have a significant student load (of the order of 5 000 EFTSU) in each of at least three broad fields of study, such as humanities, science, engineering or education;

(ii) the institution will require a minimum proportion of its student load to be allocated to postgraduate research students (3 per cent of total student load); (iii) staff of the institution will be expected to have obtained a minimum number of competitive research grants (one per full-time equivalent staff of lecturer and above per annum); (iv) staff of the institution will be expected to have an average of 0.5 refereed publications per annum per full-time equivalent staff member; and

42


(v) at least 25 per cent of all academic staff (both full-time and parttime) of the institution will be expected to have both a relevant PhD and research experience. 3.28

For a well established university, a higher profile was expected including more than 5 000 EFTSU across four or five broad fields of study, a postgraduate load of more than 7 per cent, an average of three research grants to 20 full-time equivalent staff, and two to five refereed publications per annum per equivalent full-time staff. Various senior officers of the AVCC have commented that it would be interesting to apply all these indicators to all established public and private universities today, particularly those relating to research and research training activities, and the publication and research grants records of academic staff.

3.29

The revised 1997 AVCC criteria are expressed in much broader and less restrictive terms and do not include specific indicators of performance (AVCC 1997). No longer is there mention of courses meeting national and international standards and requirements having a commitment to research training, and there being an extensive library. Also there are no longer specific quantitative requirements about size, postgraduate load, staff qualifications and staff achievements in attracting research grants and in publications.

3.30

In New South Wales, the Higher Education Act 1988 protects the use of the title ‘university’, and there are varying levels of regulation applying to overseas universities wishing to operate in the state, but not to other Australian universities. In Queensland, the Higher Education (General Provisions Act) 1993 explicitly protects the titles ‘university’ and ‘degree’ and regulates the establishment of new universities in the State. It allows other Australian universities to operate in the State, but requires overseas accredited universities to seek Ministerial approval. In Tasmania, the titles university and degree are protected and there are powers under 1995 and 1997 legislation to regulate the establishment of new universities and the operation of foreign universities. In Western Australia, the Australian Capital Territory, and South Australia, there is no legislation protecting the titles university and degree, except in the case of the Australian Capital Territory and South Australia where there are some controls through accreditation powers.

43


Protection through other legislation 3.31

As Table 3.6 indicates, additional protection is afforded with regard to the establishment and recognition of universities by other Commonwealth, State and Territory legislation. All Australian providers offering education and training services to overseas students in Australia must be accredited to provide specific courses by the relevant State/Territory authority, be approved to provide those courses to overseas students by the relevant State/Territory authority, be registered on the Commonwealth Register of Institutions and Courses for Overseas Students (CRICOS). The Education Services for Overseas Students Act (ESOS) protects students’ fees and provides for certain other protection. Commonwealth Corporations Law and State and Territory business names legislation prevents a company from carrying on a business as a university unless it registers its names. The Trade Practices Act 1974 and related State and Territory legislation protect against misleading advertising while both Business Names legislation and Corporations law place restrictions on use of the word university. Action can be taken by the Australian Competition and Consumer Commission against an institution engaging in such conduct under The Trade Practices Act, or by State consumer affairs offices under relevant legislation.

3.32

In addition, entry to regulated professions is a matter for various registration authorities operating under State and Territory legislation. Regulated professions include most health-related professions, law and architecture. There are a number of other professions, notably accountancy and engineering, for which registration or licensing is confined to specific areas of professional activity. However, even when registration is not mandatory, membership of the appropriate professional body is often helpful for employment purposes and in these cases generally only the graduates of courses accredited by the appropriate professional association are eligible for full graduate membership. Later in the report the accreditation processes used by professional associations are discussed in more detail.

National recognition and guarantee 3.33

44

Currently the only national policy instrument guaranteeing with regard to accreditation and quality assurance is the AQF. Universities established by legislation and institutions otherwise accredited by State and Territory accrediting bodies are listed on the AQF as being empowered to accredit courses of study and to issue qualifications. It is believed that inclusion on the AQF register signals that governments


represented in MCEETYA vouch for the quality of the awards given by the institutions concerned.

Current system of quality assurance 3.34

The current system of quality assurance operates at a number of levels and includes the activities of professional associations and associations and networks set up by groups of universities for benchmarking and other quality assurance purposes. Some mechanisms such as peer review of research proposals and articles for refereed journals are international in character, while others are local. In summary, the main quality assurance mechanisms for universities currently are as follows:

Internal processes within universities 3.35

The internal quality assurance processes in Australian universities are similar to those in other OECD countries. However, these processes have been considerably strengthened since the early 1990s. One major factor leading to improvements was the 1993–1995 national quality assurance program and the publication of detailed reports including information on good practice. The main internal processes include the following: • Processes of assessment for new courses and units of study; • Regular review of courses and units; • Reviews of departments, faculties and research centres; • Student evaluation of teaching; • Use of external examiners for higher degree research theses and sometimes bachelors honours theses; • Surveys of graduates and employers to assess graduate satisfaction and information on course experience and suitability of graduates for employment; • Use of performance indicators for management purposes and for the allocation of funding; • Benchmarking and participation in networks which offer special opportunities for benchmarking and sharing of information; and • Special projects for the improvement of teaching and special awards for teaching excellence.

45


National mechanisms 3.36

The current mechanisms include the following: • Universities currently report on their quality assurance and improvement plans that set out their goals, the strategies adopted to achieve these, the indicators they use to assess their success in achieving these goals. The first set of institutional plans in relation to quality is about to be published by the Commonwealth. • Encouragement of innovation and good teaching through Committee on University Teaching and Staff Development and specific initiatives funded by under the Higher Education Innovation Program, including the development of an instrument to test graduate generic skills; • Publication of Characteristics and Performance of Higher Education Institutions, a report which provides indicators covering such topics as source of funds, distribution of expenses, research funding, gender and age distribution of students, basis of admissions, overseas students, mode of study, course breadth and staffing as well as graduate satisfaction with their courses and employment experience.

46

3.37

In addition, there are traditional peer review and assessment systems which are widely used in considering applications for competitive research grants, and in handling articles and book manuscripts submitted for publications to refereed journals and scholarly publishers including University presses.

3.38

As already noted, from 1993 to 1995, a major national quality assurance program operated. This was a three-year program introduced by Peter Baldwin as Minister for Higher Education. Although participation was voluntary, all universities participated. Like a number of other national quality assurance programs that were established in the late 1980s and early 1990s, this program was based on academic audits of participating universities. This involved self-assessment on a number of aspects following detailed guidelines, evaluation of institutional submissions and review teams, visits to campuses and public reporting by the Committee for Quality Assurance in Higher Education. But what was markedly different was that in the audits in the Australian program assessed not only quality assurance processes but also quality outcomes. Further, the scheme also included:


• ranking of institutions into bands, based on annual assessments of particular specified aspects of quality assurance, and publication of these rankings; • publication of detailed individual annual reports on each participating institution; and • performance funding, with funding coming from a special additional government allocation (Harman 1996b). 3.39

In retrospect, both critics and supporters agree that the positive effects of the program were substantial, leading particularly to a more serious approach to evaluation, increased attention to the assessment of outputs and increased integration of strategic planning with budgeting. Even now, it appears that many of these achievements remain, even though the 1993-1995 program was not followed by any substantial quality assurance mechanism. At the same time, there is doubt about whether the gains were worth the effort and costs involved and certainly there is a large measure of agreement that the program had serious effects on the reputations of lower performing universities and their subsequent ability to attract both students and staff.

3.40

In contrast, the quality assurance programs introduced in a number of European countries in the late 1980s and early 1990s did not publish overall rankings of institutions or detailed reports on individual institutional assessments, or used performance funding in an overt way. Most depended on either ‘horizontal’ national reviews of disciplines or ‘vertical’ reviews of quality assurance processes in individual universities, or a combination of these approaches. In the Netherlands, for example, the system of quality assurance developed in the late 1980s was based on horizontal reviews of academic disciplines. While published reports discussed study programs in each institution visited, there was no system of rankings of institutions and no use of performance funding (Zijderveld 1997). In France, the program included both institutional evaluations and disciplinary assessments, with reports going to both universities and the Minister responsible for the institutions visited. However, there was no system of rankings and the results of the assessments were not used directly in making annual allocations to institutions (van Vught 1994a).

Accreditation by professional bodies 3.41

For many years, various professional bodies and association have conducted accreditation of professional courses in fields such as medicine, law, engineering and architecture. More recently, accreditation systems have been developed for newer areas such 47


as computer science, software engineering and various health science areas. Professional associations have also formed a peak body, the Australian Council of Professions, whose secretariat is located in Canberra. 3.42

One of the oldest and most highly organised accreditation systems is that run by the Institution of Engineers, Australia. There is no compulsion on engineering schools to have their courses accredited except that only graduates for accredited courses are eligible for membership of the Institution. The accreditation process proceeds as follows. Engineering schools make a detailed application following a prescribed format. A panel of three members is appointed to conduct the review and in addition consultant panel members are appointed for each engineering speciality to be considered. After receiving advice from the consultant members, the three core members visit the institution for two days for discussions with staff and students, and to view facilities. However, consultant members may join the visit if they have particular concerns. Following the visit, a draft report is prepared by the Chair and the other core members of the panel and this is circulated to the university and consultant members for comment. Panels may recommend accreditation or provisional accreditation, or may recommend that accreditation is refused. Accreditation is for a period of five years after which each course must be re-accredited. In its accreditation system, the Institution does not rank universities or publish performance data. In its work in accreditation and quality assurance the Institution of Engineers works closely with the Council of Deans of Engineering Schools, especially in promoting best practice and methods of benchmarking.

3.43

Currently the Institution of Engineers is implementing a new approach to accreditation, following a review of engineering education in 19961997 conducted jointly with the Australian Council of Engineering Deans, and the Academy of Technologicial Sciences and Engineering (Changing the Culture 1996; and Beyond the Boundaries 1998). In its executive summary, the review explained that an initial finding was ‘the need for a culture change in engineering education, ultimately to extend throughout the profession’ (Changing the Culture 1996, p 7). The report went on: The present emphasis placed on engineering science resulting in graduates with high technical capacity has often acted to limit their appreciation of the broader role of engineering professionals. Graduates must understand the social, economic and environmental consequences of their

48


professional activities if the profession is fully to assume its expanding consequences (Changing the Culture 1996, p 7). 3.44

The review was conducted because of concern that engineering education tended to be somewhat introverted and in the modern world needed to produce graduates better able to interact with other professionals. This in turn prompted the review committee to think much more in terms of the desired outcomes for graduates.

3.45

This new approach focusses mainly on graduate attributes rather than inputs. The Institution is now well advanced in implementing the new approach. The framework was piloted in 1998 and since then two rounds of reviews have been completed. By the end of 1999, the new approach will have been employed in assessments in half the total number of engineering schools.

3.46

Documentation explains that, under the new approach to accreditation, engineering education at university level ‘provides the learning base upon which competence for a professional engineering career is built’ and that it is important ‘that the education provides the graduate with … generic attributes’ (Policy of Accreditation of Courses 1999, p 1). Generic attributes for a graduate are specified as follows: • ability to apply knowledge of basic science and engineering fundamentals; • ability to communicate effectively, not only with engineers but also with the community at large; • in depth technical competence in at least one engineering discipline; • ability to undertake problem identification, formulation and solution; • ability to utilise a systems approach to design and operational performance; • ability to function effectively as an individual and in multi-disciplinary and multi-cultural teams with the capacity to be a leader or manager as well as an effective team member; • understanding of the social, cultural, global, environmental and business responsibilities (including an understanding of entrepreneurship and the process of innovation) of the Professional Engineer, and the need for and principles of sustainable development; • understanding of an commitment to professional and ethical responsibilities; and • a capacity to undertake lifelong learning. 49


3.47

In implementing the new system of accreditation, the Institution has found that it is necessary to try to achieve a balance between an emphasis on outcomes and processes. Engineering schools also have found some difficulties in assessing particular graduate attributes, but the view of the Institution is that it is the responsibility of engineering schools to explain what they are doing and how they make educational judgments and on what basis. Overall, the Institution considers that engineering schools have been receptive to the changes.

3.48

To date, the Institution of Engineers has not been requested to accredit courses offered by private universities or other private providers. However, it is well aware of the likely problems with ‘off-shore’ teaching of international students and has adopted the policy that in accrediting an engineering program the engineering school must provide documentary detail on all pathways to graduation including courses offered at branch campuses, overseas campuses, by distance education, or through twinning of franchise arrangements. In the case of off-shore campuses, and twining and franchise arrangements the Institution requires information especially on teaching and assessment methods, staff qualifications and staff development for all academic personnel involved, whether they are employed by the University or a partner, and academic standards. Alternatively, accreditation of a twinning or other off-shore teaching operation may be treated as a separate accreditation.

3.49

The Institution of Engineers has close relationships with parallel bodies in other countries. It has been deeply involved in the development of the Washington Accord, an agreement between eight industrial countries about equivalence of engineering degrees, and regularly exchanges information and documentation with fraternal associations.

Special protection for international students 3.50

50

Special Commonwealth mechanisms, consisting of legislation and a register of courses, have been put in place to provide protection for international students. As already noted, The Education Services for Overseas Students (Registration of providers and Financial Regulation) Act 1991 helps ensure that only quality courses are offered to foreign students studying in Australia. All providers offering education and training services to overseas students must be accredited to provide specific course (and approved to provide these courses to overseas students) by relevant State and Territory authorities, and be registered


on the Commonwealth register of International Courses for Overseas Students. For Commonwealth registration, institutions must supply additional information including audited balance sheets, and have a designated trust account, a tuition assurance scheme, and a specific student fee refund policy. However, these mechanisms generally do not apply to international students studying ‘off-shore’ in campuses established by Australian providers, or under twinning of franchise arrangements. This legislation was amended in 1998 to provide for a three year extension to the original sunset clause, following a review which demonstrated universal agreement amongst stakeholders that continuation of the cooperative model as provided under the act was appropriate for the future regulation of the industry. 3.51

Some additional protection is provided by State legislation. In Victoria, for example, the Tertiary Education Act 1993 gives the Minister power to endorse or cancel the endorsement of any course offered by a post-secondary education provider as suitable for overseas students. In deciding whether or not to endorse a course, the Minister may take into account a number of matters including financial planning, marketing and promotional material, use of agents contracts with respect to students, student grievance procedures, welfare of students and student housing and accommodation as well as more academic matters including student selection procedures curriculum and course nomenclature (Section 6).

Assessment 3.52

While there are many strengths associated with the current quality assurance and accreditation arrangements for higher education, at the same time there are clear weaknesses that need attention. Perhaps the major weakness with respect to quality assurance is that there is no national agency, responsible for quality assurance that can publicly vouch for the quality of Australian higher education. This stands in contrast to the situation in the Netherlands, France, the United Kingdom and New Zealand. Associated with this is the problem that the rigour with which individual universities pursue quality assurance across all aspects is almost entirely at their discretion, even though there are some safeguards with the requirement to have quality assurance and improvement plans and with the use of publication of various performance indicators. The lack of a national agency is widely acknowledged as a drawback in the international marketing of Australian higher education. Amongst both government officials and

51


university management there is wide recognition of the need for a stronger mechanism related to off-shore education. 3.53

52

With regard to accreditation, there is clearly a need for uniform approaches and criteria across the States and Territories. Also needed are an increase degree of sharing of information and documentation on an ongoing basis between the various accrediting agencies and a better system of reporting and public access to information concerning which courses have been accredited and which providers have been given approval to operate as self-accrediting institutions. Some accrediting agencies have detailed criteria whereas in other cases the criteria are brief and possibly inadequate. Some accrediting agencies cover areas related to business plans and financial viability. Some provide for the accreditation of both institutions and courses while others deal with course accreditation only. Some have lists available of the courses that have been accredited whereas others do not. In most cases there is some difficulty with regard to what extent accreditation should be dependent on minimum standards of facilities including library holdings, and this is becoming more difficult with the use of distance and on-line teaching and use of on-line library and reference sources. Some State agencies are well staffed while others have been reduced to a bare minimum of personnel and thus unable to offer effective services.


4

The changing quality environment and the Modern Australian Model 4.1

In a number of important respects, the quality assurance environment for Australian higher education has changed to a marked extent in recent years. This chapter will first outline some of these changes and the pressures that are driving them, and then comment on key features of the proposed Modern Australian Model of quality assurance and accreditation.

The changing quality environment 4.2

The changes over the past decade in the environment with regard to quality assurance and accreditation have been substantial and somewhat unexpected. In the late 1980s and early 1990s, these changes led the Commonwealth Minister for Higher Education at the time, Peter Baldwin, to request the Higher Education Council to review quality assurance practices and outcomes in Commonwealth funded institutions and to recommend a process of external review. About the same time, a number of States introduced new legislation giving additional powers of accreditation and control over providers offering higher education courses to international students.

4.3

In the past four years, however, since the end of the 1993–1995 quality assurance program, there have been various further important developments, which make review and strengthening Australia’s accreditation and quality assurance arrangements urgent. These various changes can be summarised under the headings of globalisation and changes in educational technology, international recognition of qualifications, recent changes in quality assurance in other industrialised countries, new quality assurance arrangements in Australia’s ‘off-shore’ education destinations, the needs of Australia’s education export industry, increasing accountability pressures at home, incidents with private providers and the increased number of private providers, and complaints from applicants seeking accreditation.

53


Globalisation and changes in educational technology 4.4

Globalisation and rapid changes in educational delivery technologies are creating substantial changes internationally. Globalisation has meant that employers, government agencies, professionals and students are better acquainted than ever with developments in other countries. Bad news affecting international education now circulates more rapidly. Developments in electronic communications enable higher education providers to offer courses in new forms by distance education, both in Australia and overseas. Electronic communications are also providing students with access to new forms of educational resources. Now in many disciplines students may use resources available on the web as much as traditional library resources. In addition, the new electronic communications are enabling overseas higher education competitors to provide education services within Australia as well as targeting Australian overseas education markets, especially in Asia. All this in turn is creating pressures for concerted action by institutions and government agencies within and across countries to improve quality assurance and controls over new providers.

4.5

Another side of globalisation is the increased ability by both governments, students and potential students to compare the courses and awards of courses offered by providers in different countries. In many cases detail of courses is available on the web.

International recognition of qualifications

54

4.6

Globalisation, increased mobility of skilled personnel, international mobility of students and offering of higher education courses across national boundaries has led to increased mobility of labour and to increased pressures for reciprocal relations in the recognition of academic and professional qualifications. This in turn raises important questions about the standards of qualifications offered by Australian providers and the mechanisms used to guarantee quality and the academic and professional standards of awards.

4.7

Staff in NOOSR experience considerable frustration about the lack of a national quality assurance mechanism in Australia. They speak openly of the ‘charade of Australian quality assurance’. They explain that, in their experience, enquirers from government agencies and professional bodies in other countries are often puzzled by the lack of a government backed national quality assurance agency in Australia, while at other times spokespersons in other countries are openly critical of lack


of equivalent agencies to that found in the United Kingdom, France, the Netherlands and New Zealand. 4.8

In many overseas countries with which NOOSR has close cooperation, there is considerable concern about the activities of private higher education institutions, ‘fly-by-nighters’ and degree mills. NOOSR staff also see the need for a strong quality assurance framework to facilitate the international marketing of education and to assist in the recognition of Australian qualifications in other countries.

4.9

A particular need for a strong national quality assurance agency relates to international conventions and agreements signed by Australia with regard to the recognition of post-secondary education qualifications. The three key conventions where Australia is a signatory are the UNESCO convention on the recognition of qualifications for the European region, the UNESCO convention for the recognition of qualifications in Asia, and the recent Lisbon Convention on the recognition of qualifications concerning higher education in the European region. While the requirements of these conventions may not be regarded as particularly onerous, they do require signatory countries to provide pathways for the recognition of overseas qualifications and detailed information on local higher education qualifications and their standing. According to the Lisbon Convention, each signatory country is required to provide adequate information on any institution belonging to its higher education system and on any program operated by these institutions with a view ‘to enabling competent authorities of other Parties to ascertain the quality of the qualifications issued by these institutions’. According to the Convention, such information shall take the following form: (i) in the case of Parties having established a system of formal assessment of higher education institutions and programmes: information on the methods and results of this assessment, and of the standards of quality specific to each type of higher education institution granting, and or programmes leading to, higher education qualifications. (ii) in the case of Parties which have not established a system of formal assessment of higher education institutions and programmes: information on the recognition of the various qualifications obtained at any higher education institution, or within any higher education programme, belonging to their higher education systems (Council of Europe 1997, Article VIII.1). With many European countries establishing more rigorous national systems of quality assurance, it seems reasonable to predict that in the 55


future international conventions may well expect all signatories to have national systems of quality assurance in place. 4.10

In addition to conventions, Australia has signed various agreements or memoranda of understanding with other countries concerning the recognition of qualifications. Two recent agreements were the agreement with Italy of 1996 and the memorandum of understanding with Germany of 1998. While neither agreement explicitly mentions quality assurance, it is likely that in future negotiations national quality assurance mechanisms may become of increasingly importance. Further, it is clear that in the various bilateral negotiations to date there have been various points of dispute and in the future on such points it is possible that much more weight could be given to the views of national quality assurance agencies.

4.11

For the future an issue of particular importance is whether the Australian graduates from off-shore operations will be included in bilateral agreements and multilateral conventions. Associated with this is a generally growing concern about the proliferation of awards especially at the postgraduate level, and the considerable variations in the length of courses.

Recent changes in quality assurance made by other industrialised countries 4.12

56

Among both many senior officials in government and senior management in universities, there is a wide appreciation that a number of other countries have made recent changes to strengthen their accreditation and quality assurance arrangements. In particular, there has been considerable interest in the establishment and development of the new Quality Assurance Agency in Britain and the proposed Qualifications Authority in New Zealand. These developments reinforce the view that Australia’s quality assurance mechanisms do not stand up well internationally. At a recent international quality assurance conference, speakers from Britain drew attention to Australia’s lack of a national quality assurance agency. In addition, since countries in the Asia Pacific region often look to Australia not only as a source of university education but to benchmark for their own university standards, Australia needs an exemplary record in quality assurance and efforts to monitor academic standards.


New quality assurance arrangements in off-shore education countries 4.13

Another factor prompting review and strengthening of quality assurance mechanisms is that a number of counties in the Asia Pacific region have recently strengthened their own quality assurance mechanisms and are showing increasing concern about allowing foreign universities to operate within their borders. Malaysia recently has established an Accreditation Board that will cover the activities of foreign providers as well as local institutions, while in Hong Kong new regulations govern the activities of foreign universities operating in the Territory. The establishment of the Accreditation Board in Malaysia has been prompted by the Malaysian Government’s wish to encourage high quality foreign universities to establish campuses in Malaysia.

4.14

In India, a case went to the High Court of Madras in 1997 concerning the operations of an overseas university within India and to date this case has not been concluded. At issue here is whether institutions other than those specified under the 1956 University Grants Commission legislation can grant degrees in India. Essentially, this legislation specifies that only universities incorporated under Indian federal or state legislation can grant degrees. In March 1997, the Monopoly and Restrictive Trade Practices Commission restrained 16 institutions from offering courses on the grounds they were not approved institutions, while in July 1997 the High Court of Madras issued an order restraining foreign universities from granting degrees, directly or indirectly, in India. The catalyst for this was a series of advertisements inserted by a number of agents regarding admission to various international courses without making it clear that these courses were being offered in countries outside of India. One such advertisement was by two Australian agents for admission to an MBA course. The case came up for hearing on 27 August 1997 with the Government of India being named as one of the respondents. In view of various allegations made in Asian and Pacific countries about the operations of foreign universities, it seems likely that more effort will be made by countries within the region to regulate the activities of foreign institutions. One common allegation made in India is that some foreign universities are dumping low quality courses in Asian countries that these institutions would not be allowed to offer in their home countries.

57


Needs of Australia’s education export industry

58

4.15

One of the strongest arguments put by many of those interviewed for strengthening quality assurance and accreditation mechanisms relates to providing additional safeguards to protect Australia’s higher education and VET export business. Australia’s export education business continues to expand and is a major source of income both for providers and the country. To safeguard this industry, it is widely argued that there needs to be a national quality assurance agency as well as better mechanisms to accredit private higher education providers and courses. Many make the point that the VET system is much better placed than higher education to guarantee quality through government sponsored agencies.

4.16

In our discussions, a number of informants made the point that while the number of unfortunate incidents related to quality and accreditation with Australian providers were few to date, it is possible that even fairly minor but widely-publicized incidents could have particularly damaging effects. Such cases are often difficult to repair and their effects can continue for substantial periods. Disaffected staff can do considerable damage in making allegations about low academic standards and failure to follow specified procedures, whether or not these allegations might be true.

4.17

Various unfortunate events in the late 1980s and early 1990s, particularly the closure of private institutions, prompted the passage of both Commonwealth legislation and separate legislation in some States, and that in a number of cases the Commonwealth had to provide substantial funds to assist affected students. Closures of colleges in the late 1980s, resulting from the inability of a number of private providers to refund prepaid course fees to students who were refused student visas under tightened entry measures applied by the Department of Immigration and Multicultural Affairs in response to evidence of non-compliance with student visa conditions, predominantly from the People’s Republic of China, led to students in many cases being without a higher education place and the funds to pay fees. This in turn led to the 1991 Commonwealth Education Services for Overseas Students Act. As a result of these incidents, the Commonwealth expended over $66 million in refunds to students, with only $4.5 million being eventually recovered. In 1993, two liquidated colleges closed in Western Australia with no funds being held in special trust accounts and the Commonwealth had to provide $1.3 million in assistance to students.


4.18

Expansion of off-shore operations through the establishment of university campuses in foreign countries, twinning and franchising arrangements, and the delivery of courses to students in other countries by distance education, or a combination of distance education and face-to-face teaching, is creating new needs for more rigorous quality assurance mechanisms. As already noted, almost 23 000 of Australia’s international higher education student are enrolled through various ‘off-shore’ arrangements. ‘Off-shore’ delivery creates special management problems for providers and makes the need of well-developed, detailed administrative procedures and review mechanisms even more necessary.

4.19

Various well publicised incidents pointing to administrative and other failures of British higher education providers in ‘off-shore’ endeavours point to the kind of difficulties that similar incidents could have for Australian education exports. The most recent case was about the University of Derby and its off-shore operations in Israel (Times Higher Education Supplement, 23 July 1999). It is alleged that, for financial reasons, the University has been admitting unqualified students to degree courses and that it has been ‘dumbing down’ courses. These allegations have prompted an enquiry by the Israeli quality assurance agency.

4.20

A number of the major Australian higher education providers well recognise the dangers in off-shore delivery and in response have developed additional management and monitoring processes and have put in place their own quality assurance audits. Monash University, for example, has in place regular internal reviews of each of its ‘offshore’ operations, with each of these being chaired by an external member with special expertise in quality assurance and accreditation. In addition, Monash has contracted with an international quality assurance agency to conduct regular external audits of each off-shore operation. RMIT has similar internal reviews in place and in addition has used auditors from the Quality Assurance Services to gain ISO certification.

4.21

As already noted, another concern about ‘off-shore’ operations is whether all ‘off-shore’ courses in particular professional areas such as accounting are covered by accreditation conducted by Australian professional associations. Comparatively little information is available on this matter, but it could be a particular problem in cases where all teaching and examining is conducted by academic staff employed by partner institutions operating under franchise arrangements.

59


Increasing accountability pressures at home 4.22

Generally in the Australian community pressures continue to grow for increased accountability of public institutions and of government funding or subsidies. In the case of higher education, this is likely to increase further as public universities become more commercial in orientation. Further, community concern seems likely to demand stronger quality assurance mechanisms as the higher education system moves increasingly to further competition between institutions and possibly towards a system of student based funding.

Incidents with private providers and increased numbers of private providers 4.23

A small number of well-publicised cases concerning private providers has raised the level of concern generally about quality assurance and accreditation mechanisms. This concern has been felt particularly by Ministers, government officials and those universities with large commitments in the areas of international education and with links with private institutions.

4.24

In our discussions, various informants mentioned the University of Greenwich case, the case of the two institutions from South Australia which had secured approval for the use of the word university in company titles and various cases of private VET providers and language training schools closing because of insolvency. Apparently the University of Greenwich made enquiries from at least two government accrediting agencies in States and Territories before approaching the Norfolk Island administration.

4.25

The number of private providers offering accredited higher education courses clearly has increased substantially over the past five years and there are rumours of considerable interest by overseas universities about establishing campuses in Australia or delivering courses to Australian students by distance education. These overseas providers include major American universities as well as ‘no-frills’ providers.

Complaints from applicants seeking accreditation 4.26

60

Another reason why State and Territory officials are concerned about current accreditation and quality assurance arrangements is that over the past five years there have been an increasing number of complaints from private providers seeking accreditation.


4.27

Smith (1998) reports that there have been various complaints from private providers who sometimes complain that the accreditation process is biased against them. In particular, it is sometimes alleged by applicants that the ‘playing field’ is not level and that University academics on accreditation committees are not flexible enough to appreciate different paradigms and emerging disciplines. Private providers also suspect that the time taken in considering some course accreditation proposals is evidence that members of panels from public universities are trying to protect vested interests.

4.28

On the other hand, it should be noted that most of the relevant legislation is sufficiently general in wording and intent to cater adequately for diversity in course applications. Smith (1998, p 8) points out that in New South Wales accrediting panels: … are required to assess whether a proposed course displays academic objectivity and rigour and enables intellectual inquiry and discourse which are essential features in any higher education course. Panels do not concern themselves with the desirability of or need for a course, other than to be satisfied that there is sufficient demand to make a program academically viable.

4.29

Another common problem is that the role of review panels in accreditation is often not well understood. In each case, panels are appointed to evaluate submissions and offer advice to the Minister or the approving authority, and are not responsible for offering advice and assistance to applicants. At the same time in making assessments, panels often provide comment which applicants sometimes take aboard to modify and improve their proposals, but this function is not part of the formal responsibilities of panels.

4.30

Still another problem is that often the appeal process is not well understood. In Victoria, for example, unsuccessful applicants have the right of appeal to the Administrative Appeals Tribunal which has been in place since 1993. However, to date there have been few appeals and no appeal has proceeded to a formal hearing or judgment (Smith 1998, p 10).

61


The Modern Australian Model

62

4.31

The Modern Australian Model of quality assurance and accreditation proposed by DETYA has many strengths. In the first place, it makes important distinctions between the functions of accreditation and quality assurance and between the possible treatment of self-accrediting institutions and non-self accrediting providers. In Chapter 2, we defined both accreditation and quality assurance. Both terms are used internationally in a variety of ways, but in the Australian context they have developed particular meanings. In summary, quality assurance in Australian higher education has come to refer to a range of management and assessment procedures to monitor performance, to ensure the quality of outputs and to give stakeholders confidence, whereas in relation to government agencies accreditation refers to a process of assessment and review leading to recognition of a higher education provider or higher education course. As already noted, while closely related the two processes are somewhat different. For one thing accreditation is primarily concerned with new institutions and new courses, whereas quality assurance generally relates to the activities of established institutions. At the same time, it is seems highly desirable that there should be linkages between accreditation and quality assurance procedures and that information should be shared in the case that the two functions are carried out by separate agencies. This need for linkages is even more important if the functions are to be carried out by separate levels of government.

4.32

With regard to the distinction between self-accrediting institutions and others, it is important that adequate consideration should be given to the special characteristics of both sets of institutions, since they often differ significantly in size and administrative depth. However, ideally any new quality assurance mechanism should have the capacity to cover all higher education providers. In our interviews, a number of respondents made that point that, as far as possible, all higher education providers should be treated in a similar manner. It should be noted that for the 1993–1995 quality assurance program only public universities participated.

4.33

DETYA documentation specifies that quality assurance and accreditation mechanisms should satisfy a number of criteria. The mechanisms relating to self-accrediting institutions should not be solely at the discretion of the institutions themselves; there needs to be some external review or audit of the claims made by institutions about quality and standards; the mechanisms should be credible with international and domestic interest groups, and should and be able to


protect the international reputation of Australian awards; the mechanisms should help satisfy Australian taxpayers of value for money; any audit mechanism should have rigour, but at the same time be cost effective, not unnecessarily intrusive and be able to retain the cooperation of the public universities; and the mechanisms should provide legal clarity for students and providers and be able to promote good practice and facilitate improvement. We support these as desirable principles.

Other options 4.34

In its documentation, DETYA refers to four alternative options to the Modern Australian Model. These are refinements to the current Australian model, the New Zealand model, the British Quality Assurance Agency Model and the VET sector model.

4.35

Refinement of the current Australian model provides for enhanced accreditation processes which remain in the hands of the States and Territories, with institutions continuing to take major responsibility for their own quality assurance but with encouragement to strengthen these processes through benchmarking, and use of external audits such as having processes assessed according to ISO standards, and minor modifications to legislation. While the suggested improvements would provide for worthwhile improvements, the major difficulty is that Australia’s arrangements would fall far behind practice in a number of competitor countries and would do little to provide additional safeguards for the higher education export industry, or to lend additional international credibility to Australian awards. The biggest gap is the absence of some national agency that can certify the quality of Australian higher education.

4.36

The recently modified New Zealand model has a number of strengths. In particular, it provides for a strong national government agency with an appropriate legislative base, it includes an institutional audit process and it provides for a coordinated approach for the whole tertiary education sector. On the other hand, it is by no means certain how successful will be the approach of the Quality Assurance Authority in granting recognition to accreditation providers, especially in terms of achieving a reasonably uniform coverage in audits across different sectors and in gaining international recognition and local credibility. The appointment of only one accreditation agency that failed to deliver according to specifications could be sufficient to do major damage to the system. Presumably one reason for adopting this

63


approach is that since the early 1990s the university sector has had its own Academic Audit Unit and this system appears to be working well. Under the new arrangements, we assume that the New Zealand universities may choose to retain the services of the Academic Audit Unit. It should be noted, however, the Academic Audit Unit has not covered the work of the 25 polytechnics and four colleges of education. 4.37

The New Zealand Academic Unit was set up with the following terms of reference: • to consider and review the universities’s mechanisms for monitoring and enhancing the academic quality and standards which are necessary for achieving their stated aims and objectives; • to comment on the extent to which procedures in place in individual universities are applied effectively; • to comment on the extent to which procedures in place in individual universities reflect good practice in maintaining quality; and • to identify and commend to universities good practice with regard to maintenance of academic standards at national level (Woodhouse 1997, p 72). In fulfilling these terms of reference, the Academic Audit Unit focuses on mechanisms for quality assurance in the design, monitoring and evaluation of courses in teaching, learning and assessment, in relation to the appointment and performance of academic staff, and in research. The Unit is instructed to take account with respect to academic matters of the views of students, external examiners, professional bodies and employers. The Unit is funded jointly by the universities but is independent otherwise of the universities individually and the Vice-Chancellors’ Committee. Audit reports are public documents and there is considerable pressure on universities to take seriously any criticisms made. As in other institutional audit programs, universities carry out self audits before the visit of panels. Panels are made up of academics from New Zealand and overseas universities and members of the business community.

4.38

64

The new United Kingdom model based on the Quality Assurance Agency which was established in 1997 is still developing its procedures. However, to date its proposals have been somewhat controversial and it has still to secure strong support from the wellestablished group of older research universities. Further, in the judgement of many experts, the stated goals of the Agency appear to be somewhat unrealistic and to date many of the Agency’s proposals


have raised considerable controversy. Observers question the extent to which the Agency appears likely to intrude substantially into the work of universities and the extent of funding that a fully operational Agency will need. 4.39

It is important to recognise, however, that in the past decade there have been three other important British experiments in quality assurance, one of which is on-going. First, the Committee of ViceChancellors and Principals established the Academic Audit Unit, which in turn became the Higher Education Quality Council following abolition of the binary system. This development was based on the idea of institutional audits of quality assurance processes in the context of an institution’s stated aims and objectives. The aim was to ensure public accountability for the maintenance and improvement of academic quality by finding out how institutions discharge their obligations to provide high quality education and satisfy themselves about the academic standards they seek to uphold (Williams 1997, pp110–111). A report in 1996 at the end of the second round of audits noted that quality assurance procedures had become the norm in the higher education sector.

4.40

Second, in the early 1990s, the higher education funding councils established a system of quality assurance for teaching based on the assessment of disciplines. The purpose of these assessments was to ‘ensure that quality was satisfactory or better, to encourage improvement, to inform funding and to reward excellence’ (Williams 1997). Student learning experiences and achievements were assessed against the provider’s aims and objectives. The process involved a selfassessment by the department, a visit to the department by external assessors (mainly academics from other institutions trained by the funding council), and judgements made on the quality of education through observation of teaching and learning, scrutiny of students’ work and discussions with staff and students. Initially, the methodology developed by the Higher Education Funding Council of England (HEFCE) involved selective visits to subject providers on the basis of self assessments submitted, and this resulted in the award of the grades of excellent, satisfactory or unsatisfactory. Where no visits took place, subject providers were awarded a satisfactory grade. Criticisms of this methodology forced the HEFCE to revise its approach to one based around a graded profile of subject areas constructed against six aspects of provision (curriculum; teaching; learning and assessment; student progress and achievement; support and guidance provided; and quality assurance arrangements).

65


4.41

Third, since 1986 the higher education funding councils have conducted periodical assessments of research known as the Research Assessment Exercise (RAE). The purpose is to provide quality rankings for research carried out in each of the major subject areas in all government-funded higher education institutions. The last 1996 in RAE, which followed earlier exercises in 1986, 1989 and 1992, was conducted jointly by HEFCE, the Scottish Higher Education Funding Council, the Higher Education Funding Council for Wales, and the Department of Education for Northern Ireland. Each of these bodies distributes funds selectively to institutions on the basis of the quality judgements made by expert RAE panels and funds are intended to sustain a strong research infrastructure and a range of curiosity-driven basic and strategic research activities (Harman 1999).

4.42

RAE ratings are awarded by subject panels (60 panels in 1996 for 69 assessment areas) of about 10 members each, made up of distinguished researchers in the particular subject, appointed after consultation with interested bodies such as learned societies and professional associations (Research Assessment Exercise: Criteria for Assessment 1995). Higher education institutions in 1996 could submit for assessment any research carried out within the previous four years by those of their current staff they wished to present from the nominated subject areas. Universities were able to decide which departments to put forward for assessment and which staff in each department. Some universities presented only a small number of their total departments and staff for assessment, while high performing universities generally presented all departments and practically all academic staff.

4.43

Assessments are based entirely upon the written materials submitted. A standard electronic template is used across all institutions and subjects, and this includes: • details of those staff whose work is offered for assessment in each subject area and selected recent research outputs for each of them (up to four per staff member); • other information about research activity and the units of assessment (eg numbers of research students, and research grant and contract income from various sources); and • information about the institution’s support for research in each subject area (departmental structure, facilities and research plans) and other key information.

66


4.44

In 1996 panels awarded a rating for each individual subject submission on a seven point standard scale, ranging from 1 (research quality that equates to attainable levels of national excellence in none, or virtually none, of the sub-areas of the activity) to 5* (research quality that equates to attainable levels of international excellence in a majority of sub-areas of activity and attainable levels of national excellence in all others) (Research Assessment Exercise: Guidance for Submissions 1995, Annex B).

4.45

The RAE is a peer review exercise with ‘assessments being made by the panels in the light of their collective knowledge and experience of their field of academic research’ (Research Assessment Exercise: Criteria for Assessment 1995, p 1). In each case, panels are concerned with making judgements about quality based primarily on selective reading of listed works and other evidence of reputation and standing as set out in supplementary documentation. In making its assessment, each panel takes into account only the work of those staff listed as being ‘research active’.

4.46

Each of the funding bodies uses RAE results in somewhat different ways for allocating block grant funding to universities. For example, the HEFCE in 1998–1999 allocated 804 million pounds sterling on the basis of 1996 rankings. The total funds were allocated between 69 subject areas or ‘Units of Assessment’. Allocations to each department were based on the ranking given in the RAE, the cost weight of the subject area, and the number of research active academic staff. Only departments that are awarded a 3b or better receive funding.

4.47

The sum of 804 million pounds sterling allocated to all English higher education institutions by the HEFCE in the funding year 1998–1999 amounted to 20.7 per cent of total funds allocated for teaching and research. The largest total research allocations went in order to the University of Oxford, the University of Cambridge, University College London, and Imperial College. Many of the strongest research intensive universities received 50 per cent or more of their total HEFCE allocation on the basis for research, while in a number of ex-polytechnic universities the total research component amounted to less than half a million pounds and far less than 1 per cent of their total grant.

4.48

The RAE is a costly and labour-intensive form of assessment, both in terms of administrative costs centrally (2.25 million pounds in 1996) and in the time demands it makes on academics and academic and administrative departments. Academic departments play a major role 67


in planning strategies and in collecting data but university central administrations also play key roles in collecting data and providing overall coordination. The end result in each university is what one senior academic described as ‘a mountain of documentation’. A particularly heavy burden is borne by the assessment panels, which in 1996 considered the work of some 3 000 departments and 56 000 academics in 192 participating higher education institutions.

68

4.49

According to various reports, the RAE has had various unintended consequences, such as generating suspicion about the integrity of data submitted by other universities, helping to break down traditions of collegiality, encouraging academics to put additional efforts into seeking external research funding and adopting strategies to maximise the number of publications (such as dividing papers into two or more shorter papers and publishing books as journal articles prior to publication of the full work). Others have criticised the lack of a clearly articulated philosophy for the RAE, possible defects in a peer review approach and the fact that while RAE funding is on such a basis it is very difficult for less strong institutions to attract additional funding in to order to build on their strengths. Some newer universities have put considerable effort into building up research capacity of particular departments, but generally these efforts have attracted little additional funding and within such universities there is often considerable illfeeling about the fairness of the RAE and about the indirect adverse effects it has on teaching.

4.50

The current VET model of accreditation and quality assurance is now well accepted in the VET sector and widely supported by industry. It zalso has won admiration from various senior government officials, some who have suggested that it provides a suitable model for a strengthened quality assurance and accreditation mechanism for higher education.

4.51

The VET model has separate but related national mechanisms of accreditation and quality assurance. Accreditation of training providers certifies that the training meets industry needs and this is the responsibility of the States and Territories. It is based on the Australian Recognition Framework (ARF) which is a quality assured approach to the registration of training organisations seeking to deliver training, assess competency outcomes, and issue qualifications. In the past only private providers had to be registered but now all trainers need to be registered. National standards for registration comprise four sets of Standards and Evidence Requirements, which give effect to the National Principles. The registration cycle comprises four elements:


initial registration, self assessment and evaluation, compliance audit, and re-registration. Training organisations can by registered either for (a) the provisions of training delivery, assessment and issuing nationally recognised qualifications and statements of attainment, or (b) the provision of skill recognition services and issuing of nationally recognised qualifications and statements of attainment. Registered Training Organisations (RTOs) may be delegated power to self manage the scope of their registration and/or self manage accreditation functions. RTOs may include TAFE colleges and institutes, private commercial providers, community providers, schools, higher education institutions, enterprises and firms, industry bodies and any organisation that meets the requirements for registration. The Standards and Evidence Requirements contain a core which all organisations seeking registration must meet; product/service standards for organisations seeking to deliver training, assess qualifications and issue certificates and qualifications; product/service standards for organisations seeking to provide skill recognition only and to issue certificates and qualifications; and separate standards for Quality Endorsement which provide for organisations to self-accredit courses and/or self manage the scope of their registration, and which will operate in conjunction with the quality assurance systems of each State and Territory (Australian National Training Authority 1998 and 1999). 4.52

One of the main VET sector mechanisms of quality assurance is the national approval of Training Packages that define competencies for particular areas and the qualifications to be issued. These Packages are comprehensive, integrated products that provide national benchmarks and resources for the delivery, assessment and qualifications. They comprise endorsed components of national competency standards, assessment guidelines and qualifications, combined with non-endorsed components which may include learning strategies, assessment resources and professional development materials. Training Packages their main emphasis on outcomes and are meant to provide a more flexible approach than accredited courses. Approval of Training Packages is the responsibility of the National Training Framework Committee, which reports to the ANTA Board. To date 35 Packages have been approved.

4.53

In our discussions, we were surprised to find little support for the idea of integration of VET and higher education quality assurance systems. Rather, most respondents, whether in universities, government agencies or ANTA, stressed the major differences between the VET and higher education sectors. On the other hand, it must be admitted that there are some parallels between VET and higher 69


education institutional accreditation for private providers performed by the States and Territories.

70

4.54

Other important models not canvassed in the DETYA documentation are the Dutch and French models of quality assurance. The Dutch model is based on a well-organised program of disciplinary reviews, while the French model uses both disciplinary reviews and institutional audits. The Dutch program is operated by the VSNU, the association representing the heads of Dutch Universities, while the French program is the responsibility of a special government quality assurance agency. The Dutch scheme of reviews does not have any link to government funding of higher education institutions, while in France the results of both disciplinary reviews and institutional audits are used in developing funding allocations, although the precise links are not made clear. The Dutch model would seem to be the more useful to consider carefully, especially as irs system of disciplinary assessments is well developed with extensive documentation being available in English. On the other hand, it seems unlikely that any modification of Australian quality assurance mechanisms would be able to introduce both disciplinary assessments and institutional audits and, as will emerge in the following discussion, it is not clear how easily a Dutch style model designed for about 15 universities could be adapted to fit the needs of a public university system of 37 universities, plus private universities and other providers.

4.55

The current Dutch system of reviews of disciplines for both research and teaching had its origins in restructured relationships between higher education and the Ministry of Education and Science which were achieved in the 1980s. Following publication in 1985 of the policy paper, Higher Education: Autonomy and Quality, discussions were held between the higher education sector and the Ministry. In return for achieving increased financial and managerial autonomy, higher education institutions agreed to establish of an assessment mechanism that might demonstrate to society the delivery of quality education. Originally the Government intended that the assessment function would be carried out by the Inspectorate for Higher Education but, after negotiations, it was agreed that responsibility would lie with the Association of Cooperating Universities of the Netherlands (VSNU) for the universities, and the Council for nonuniversity institutions for the HBOs. The University assessment began with a pilot program in 1988 and commenced on a more formal basis the following year. Under this system, visiting committees review study programmes in all universities on a six year cycle. In preparation for the visiting committees, each participating study programme prepares


a self-evaluation. Visiting committees consist of about seven members and are appointed following consultations with the faculties to be reviewed. During visits committees hold discussions with Deans, senior management, academic staff and students (van Vught 1994a). 4.56

The reviews of research have been particularly successful and the detailed methodology used continues to attract considerable interest in other countries. Over the period 1993–1997, 28 disciplines or academic areas were reviewed, following a protocol that was agreed to in 1994. In the case of medical research, the VSNU and the Royal Netherlands Academy for Arts and Science shared responsibility for the assessment. A number of smaller institutes, some outside the university sector, were also assessed by special request. In 1997, the VSNU Committee on the Future of Quality Assurance evaluated the reviews of research and, as a result, it was agreed to have a further round under a slightly modified protocol which was agreed to in 1998. Under this new Protocol (VSNU 1998), there will be a much greater emphasis on the context specific aspects of research programs and faculties, and the review committees will be explicitly asked to answer questions relating to the missions and the state of the art in the academic area. The evaluation criteria continue to compromise the elements of academic quality, productivity, relevance and viability but the emphasis on context-specific aspects requires application of the criteria in the light of the faculty or institute’s mission. The Protocol states that, as ‘in the first round, the most important functions will be quality assurance (improvement of university research quality as a result of self-regulation within universities, faculties and research institutes); accountability; and collection of information that can be relevant to third parties (NSNU 1999, p ii).

4.57

No reports are yet available for reviews conducted under the new protocol, but the 1996 review of earth sciences illustrates well the approach used in the 1993–1997 round. The review of earth sciences considered the work of five faculties and their 25 research programmes. In addition, at the request of the Board of the Leiden Institute of Chemistry, the Committee also assessed Geo-biochemistry in that Institute although it was originally assigned to the Chemistry Review Committee. The assessment of earth sciences was conducted by a review committee of seven members, all of whom with the exception of the chair (who held the position of Professor of Astronomy at Utrecht University) were foreign experts from Australia, Belgium, France, the United Kingdom and the United States. Following the assessment, the review committee produced a detailed report of just under 100 pages, outlining its methodology, the key 71


characteristics of the faculties and research programmes being reviewed, and providing detailed comment on the state of earth sciences and on the work of each faculty and research programme. The committee assessed scientific quality, scientific productivity, scientific and societal relevance, and viability on a five-point scale of excellent, good, satisfactory, unsatisfactory and poor. While overall it reported favourably on the state of earth sciences, it identified a number of issues of concern, particularly issues about lack of critical mass in some institutions, workloads of senior staff, and the desirability of increased international mobility amongst students and postdoctoral fellows. Each research programme was given a descriptive grade under the four criteria (VSNU 1996). One of the strengths of the program of research assessments is that apart from constituting an important quality assurance and accountability device, review reports provide an overall detailed assessment of the various academic disciplines and the work in each of these in the various university faculties. They are also forward-looking reports that can be used to guide both universities and government agencies in their forward planning.

72


5

Accreditation of courses and institutions 5.1

This chapter considers the mechanisms for the accreditation of courses and institutions in a Modern Australian Model of quality assurance and accreditation. It will be recalled that the Model provides for different arrangements for institutions that have the power to accredit their own courses and for non self-accrediting institutions. For the first, the main mechanism would be rigorous scrutiny of financial and quality aspects before founding legislation is passed or other authorisation is given. For non self-accrediting institutions, the model is less well developed but it is suggested that the main elements could be as follows: • Rigorous scrutiny of provider capacity before course accreditation; and • Review of provider performance and accredited courses every five years.

5.2

The term accreditation with respect to this chapter is used to refer to a process of assessment and review, carried out by a government agency and with legislative backing, which enables a higher education course or institution to be recognised or certified as meeting appropriate standards. It is also a process leading to approval for higher education institutions to operate within a State or Territory or for particular courses leading to specified awards to be offered. As already noted, this process currently is carried out by the States and Territories.

5.3

With respect to the different categories of higher education institutions outlined in chapter 3, there is little problem with respect to institutions that currently have powers of self-accreditation, except that in highly unusual circumstances it is possible that a State or Territory Minister could institute an inquiry into a self-accrediting institution, or even dismiss the governing body. Further, as will be argued in a later chapter, it is highly desirable that, whatever national quality assurance mechanism is developed, it should cover private as well as public universities, and self-accrediting as well as non selfaccrediting institutions.

73


5.4

Acceptance of the points made in the previous paragraph means the main concern of any accreditation process would need to be with: • Approval for new universities to operate, to use the name university, and to offer degrees and other awards; • Approval and accreditation of courses of study leading to degrees and other awards by other higher education providers; and • Re-accreditation of institutions and awards.

5.5

There are also some related issues about whether there should be any restrictions on Australian universities operating in other states than the one in which they were established, about their ability to develop relationships with private providers in other states and to develop special courses or campuses for international students in other states, and about whether higher education providers who offer courses leading to particular awards of universities should be covered by accreditation.

5.6

Still another topic that needs consideration is what links there should be between accreditation and quality assurance in the proposed model. This will be taken up in later discussion.

Responsibility for accreditation 5.7

74

In our various discussions, we found strong support for the proposition that accreditation in the way defined in this chapter is clearly a matter for government and not the higher education sector, and that the States and the Territories should continue to exercise their responsibilities in this area. This means that States and Territories would continue to approve the establishment and operation of new and overseas universities and the approval of courses in non-self accrediting institutions. Many respondents considered it important that accreditation should have a legal basis, especially as in the future it is possible that there will be more challenges considering who should be able to offer courses leading to degrees and other higher education awards and what institutions should be able to use the titles of university and degree. State and Territory accrediting agencies clearly see accreditation as a function for their level of government and consider that, over the past decade, despite a number of weaknesses in criteria and processes, the various States and Territories have done a worthwhile job at a highly professional level. They point to existing legislation in place and emphasise the constitutional responsibilities of the States and Territories for particular aspects of education. Further, a number of those interviewed made the point that in recent years the


role of the States and Territories has been enhanced as the Commonwealth has moved from funding to subsidising universities. 5.8

We support the above arguments and note that a number of the States have well developed offices which have had considerable experience in accreditation, in some cases going back to the period of advanced education. Further, in a number of cases the expertise in accreditation is of a high order and some of the documentation that has been developed is particularly impressive.

5.9

We found no support at all for any other agency or body to perform the accreditation role. The AVCC considers that accreditation is a proper role for government rather than the sector and is concerned that accreditation should have a proper legislative basis. Professional bodies show no interest in the area and it should be noted that a large number of disciplinary and professional areas are not covered by the accreditation functions performed by bodies such as the Institution of Engineers.

5.10

Another possibility would be for a new national agency for quality assurance to take over the current work of the States and Territories in course and institutional accreditation. We found no support for this plan and we advise against it. It would raise difficult constitutional and intergovernmental issues and the accreditation functions could overburden any new agency charged with the difficult task of developing a new national quality assurance agency. Further, through continuation and extension of the present work being undertaken by the MCEETYA Multilateral Joint Planning Committee we consider that it should be possible to develop a professional national approach to accreditation, being operated by State and Territories working in close cooperation with one another. On the other hand, if the current work of the Multilateral Joint Planning Committee is unsuccessful, other options than may have to be considered.

5.11

In these circumstances we consider that accreditation in terms of the approval for the operation of new or overseas universities and the approval of higher education courses offered by non self-accrediting institutions should remain, at least for the present, a State and Territory responsibility. At the same time, it we see the need for each State and Territory to report annually to MCEETYA on any changes in accreditation legislation, guidelines and procedures, and provide details on those institutions and courses which have received accreditation over the past 12 months. We recommend adoption of such a policy. Appropriate linkages also will need to be developed between accreditation and any system of national institutional quality assurance audits. 75


Greater uniformity of legislation, criteria and processes 5.12

State and Territory accrediting bodies readily admit that there are problems in the current arrangements with respect to uniformity in legislation and other regulatory requirements, and in criteria and processes employed in accreditation. However quite rightly, they point to the substantial progress made in the past five years, particularly in strengthening legislation, in sharing documentation and information between offices, and in achieving reciprocal agreements that mean an institution operating nationally or in more than one State/Territory need only apply once for accreditation. They also point to recent progress made by the MCEETYA Multilateral Joint Planning Committee.

5.13

The MCEETYA meeting of 22–23 April 1999 dealt with a number of items related to the recognition of universities. That meeting agreed to refer the issue of a common approach to criteria and procedures employed in the accreditation of higher education institutions to the Multilateral Joint Planning Committee. It asked the Committee to report on the current criteria and procedures for the accreditation of higher education institutions in each State and Territory and make recommendations on the most appropriate instrument for a common approach.

5.14

The former Higher Education Task Force had commissioned a project in 1998 to be undertaken in Queensland to explore options to develop common principles and a cooperative approach to quality assurance of accreditation processes among relevant State and Territory jurisdictions with respect to the following areas: (i) accreditation of higher education courses offered by private (nonuniversity) providers, and registration of providers to offer courses; and (ii) accreditation recognition of overseas and/or private universities, including arrangements, if any, with respect to: • overseas universities that teach individual Australian distance education students in Australia, although these may have no presence in Australia, either through an agent or Australian branch office; • overseas universities that seek to operate in a State or Territory through an agent of by establishing a branch office or campus; and • local institutions seeking to use the title and operate as a university in a State or Territory.

76


5.15

A draft report on arrangements and procedures for the establishment and recognition of universities throughout Australia has been prepared under the Taskforce Consultancy. This is part of a larger report on accreditation of higher education institutions.

5.16

The 10th meeting of MCEETYA on 22–23 1999 April also formally endorsed the operational guidelines for use by State and Territory accreditation officers for concurrent accreditation and authorisation of private higher education providers to offer higher education courses in two or more States or Territories of Australia.

5.17

To date the Multilateral Committee has put most of its efforts into developing a common protocol for the accreditation of universities. In turn this is raising the issue of what are the distinguishing characteristics of a university in Australia at close to the turn of the millennium and that criteria should be used in accrediting new and overseas universities. Also it is planned to identify the gaps in legislative protection afforded to the Australian university system against domestic and overseas institutions operating in Australia without approval of the relevant State or Territory authority.

5.18

With regard to controls over use of the titles of ‘university’ and ‘degree’, there are some differences of opinion. State and Territory officials see value in maintaining and strengthening controls over these titles, especially as this is an important element of regulatory controls over new providers and overseas institutions, and in maintaining the status and international credibility of current universities. On the other hand, some key figures within public universities consider that Australia appears to be moving to an American type higher education system where the title of university carries less weight and where some prestigious institutions use the title institute or college and relatively low level institutions call themselves universities.

5.19

From a pragmatic point of view, it will be unfortunate if the Multilateral Committee is drawn into lengthy and time-consuming debates about the characteristics of modern Australian universities. Perhaps more important is the need to develop uniform protocols for the recognition of new and overseas universities and agreement on the criteria to be applied. There seems a high degree of agreement that criteria should include topics such as financial viability, the legal basis of the institution, and the processes of governance, internal quality assurance and accountability. But there appears to be less agreement about whether the criteria should include quantitative indicators with regard to staff, buildings and facilities, and library holdings and specialised laboratories. Other issues are whether all

77


universities should have an active involvement in research and research training and what might be minimum commitment to these activities for both new institutions and established overseas institutions. 5.20

Other issues that need attention include: • protocols and procedures for the accreditation of institutions other than universities; • whether the recognition of new and overseas universities should automatically carry with it the rights of self-accrediting powers, or whether some accredited universities might be treated in a similar fashion to the Melbourne Private University whose courses must be certified by the University of Melbourne and the accreditation limited to a period of five years; • whether or not all institutions need some form of accreditation before their courses can be accredited; • requirements with regard to ‘out-state’ Australian institutions operating in other States and Territories; • whether or not all accredited institutions need special approval to offer courses to international students at special international student campuses; • whether or not universities and other self accrediting institutions need special approval to enter into franchise arrangements to offer higher education courses with non accredited institutions such as VET providers, especially when all teaching and assessment is carried out by the staff of the franchisee; • whether or not legislation in all States and Territories should provide for both the accreditation of institutions and courses; • the linkages between accreditation of institutions and courses; • detailed protocols, criteria and procedures for the approval and accreditation of courses in institutions which do not have self-accrediting powers. Many of these issues are quite complex and raise difficult political issues. On the other hand, already there is available extensive documentation in the offices of accrediting agencies.

78


Reporting on accreditation of institutions and courses 5.21

One current weakness is a lack of national information available to members of the public, employers and higher education providers and potential providers about accreditation processes and which institutions and courses have been accredited and over what periods. Some accrediting agencies have information and consolidated listings of accredited institutions and courses readily available but in other cases this is not so. At a minimum, this information should be widely available nationally and information should provide legal clarity to students, providers, employers and professional associations. Further this information should be available in both print and on-line forms. It will be noted that we have already recommended that each State and Territory should report annually to MCEETYA on any changes in accreditation legislation, guidelines and procedures and what institutions and courses have been accredited over the past twelve months.

5.22

We had discussions with various bodies about how a national listing might be achieved. Some suggested a small office attached to MCEETYA or to the AQF Board Secretariat or that the developing and maintaining a listing might be the responsibility of a new national quality assurance agency. However, the AQF Board Secretariat suggested that possibly the most cost-efficient means might be for each accrediting agency to have available both print and on-line listings and for the AQF Web page to refer enquiries to the various State and Territory Web pages. In addition the proposed national quality assurance might do the same. While this solution would be a marked improvement over the current situation and would be costeffective, the main limitation would be that there would be no single national listing of all accredited institutions and courses. Further, this system would depend on the efficiency of each accrediting agency in keeping its listing up to date.

Other compliance functions for accrediting agencies 5.23

An important quality assurance function performed by State and Territory accrediting agencies is monitoring that all higher education institutions in their jurisdiction fulfil all statutory obligations with regard to accountability and providing annual reports to government agencies. The Victorian Office of Higher Education has a particularly well developed system, whereby all universities are reminded annually of their various accountability and reporting responsibilities. Each is 79


provided with a listing of all responsibilities and is required to sign off on each item as it has been met. This system could well provide a model for those agencies that do not have such a formal system.

Staffing and resourcing of State and Territory accreditation agencies 5.24

While we recommend that accreditation should remain a State and Territory function, it will be important that each State and Territory Government resource their higher education offices at an appropriate level. As already noted, currently the size and expertise of these offices vary to a considerable extent and in many cases administrative weaknesses are a direct result of lack of staffing and other resources. If State and Territory accrediting agencies are to have an enhanced role, there should be a clear understanding that adequate resources will be provided.

Links between accreditation and quality assurance 5.25

80

An important part of the current accreditation process is the reaccreditation of institutions at regular intervals. Generally the period of acccreditation is five years. Clearly with any accreditation system it is necessary to review and re-accredit institutions and courses on a regular basis. However, with a new national accreditation agency, it will be necessary to have clear policies about the relation between reaccreditation and quality assurance reviews. In particular, will a non self-accrediting institution that subjects itself to quality assurance reviews be subjected to the same re-accreditation requirements as one that does not? One solution could be that institutions which have had a quality assurance review within the past three years may be able to achieve re-accreditation via a less demanding process.


6

Quality assurance and improvement plans 6.1

This chapter considers the proposed strengthening possible under the Modern Australian Model of internal quality assurance mechanisms for self-accrediting institutions. As already noted the Modern Australian Model proposes putting considerable weight on the development and annual publication of Quality Assurance and Improvement Plans for the forthcoming triennium. These plans would outline the institution’s goals, strategies for achieving those goals and the indicators used to monitor progress in achieving the goals. The plans would also provide an analysis of the institution’s performance, commenting on such data as the outcomes of the graduate satisfaction survey (CEQ) and graduate employment outcomes (the GDS) over time and compared with appropriate benchmarks. The plans would also need to clearly outline the processes in place to assure quality of provision within its total ‘catchment area’.

6.2

This chapter comments on this proposal. It also discusses who should require institutions to submit the plans and who should publish these and the need for some integration with the proposed quality assurance audits. It also comments on whether or not the proposal for quality assurance and improvement plans might cover other than selfaccrediting institutions, and Commonwealth funded institutions as well as those institutions not funded by the Commonwealth.

Comments on suggested model 6.3

Continuation and strengthening of the current requirements of the Commonwealth with regard to institutional quality assurance and improvement plans appears to be a well-conceived and sensible strategy. Good management practice requires that all institutions should have in place appropriate quality assurance and improvement plans and submission of these to some outside body provides useful discipline for institutions to keep such plans up to date. With data available from the CEQ and GDS, it appears sensible that institutions should provide comments on these data for their own institutions, especially in relation to trends over time and make comparisons with appropriate benchmark data. The suggested requirement that institutional quality plans should cover all major aspects of operations 81


including ‘off-shore’ and distance education internationally, would constitute an important addition to current guidelines. However, ‘catchment area’ may not be the best term to use with regard to this matter as it already carries with it specific meanings with regard to the home address of students and those areas from which institutions generally draw their students.

82

6.4

While some universities have made major advances in benchmarking, particularly in relation to other members of established networks or individual institutions with similar characteristics, our impression is that this development has not been uniform across the sector and that it may be helpful to provide additional assistance. Such assistance could take the form of special projects funded by the Commonwealth that would lead to experience and good practice being shared. In addition there are various technical issues about aspects of benchmarking that require additional consideration.

6.5

With regard to ‘off-shore’ international education, whether it is in the form of separate campuses established by an Australian university, twinning or franchising arrangements, or offering distance education internationally, it is particularly important that institutions should document in some detail their monitoring and quality assurance procedures. In an early chapter it was noted that in 1998 almost 23 000 students were enrolled under ‘off-shore’ arrangements and that this form of enrolment seems highly likely to increase substantially. Further, ‘off-shore’ international education poses particularly difficult problems of management and monitoring, while unfortunate incidents related to such operations could have a particularly serious impact on Australia’s international education effort, especially across the countries of the Asia and Pacific region.

6.6

A number of major Australian international education providers already have in place special review and external monitoring arrangements for ‘off-shore’ arrangements. Such developments are highly desirable and it would be useful for the Commonwealth to fund a project which might address some of the special problems in quality assurance for ‘off-shore’ operations and help to circulate ideas of good practice.

6.7

Publication of Quality Assurance and Improvement Plans provides incentive for institutions to take the development of plans and monitoring of performance seriously. It also provides a useful mechanism for dissemination of good practice and innovation. Since the first publication of plans has not yet occurred it is difficult to know how publication will actually work in practice and what its


effects will be. However, it should be noted that publication of institutional reports from the second and third rounds of the 1993–1995 quality assurance program was found to be useful in many institutions. One important consideration will be the length and format of plans. While publication of plans may mean imposition of a tight word limit, this could work to reduce some of the value of institutions having detailed and comprehensive plans. Further, if the review of quality assurance and improvement plans was built into a new audit process this could well lessen the need for annual publication of plans.

Who should require and publish plans and links with institutional audits? 6.8

Currently DETYA requires institutions that it funds to prepare quality assurance and improvement plans as part of the annual profile exercise. Senior officials at State and Territory level and senior managers in the higher education sector expressed no problem with this arrangement, pointing out that any government agency that provides funding has every right to impose conditions on funding, particularly ones related to accountability. Further, it is widely acknowledged that such plans help promote good practice.

6.9

There are however two fairly minor problems with this arrangement that deserve some consideration. The first is that with the establishment of a new national quality assurance agency it would be desirable for there to be on-going discussions between DETYA and the quality assurance agency about what DETYA requires of the institutions it funds and what the quality assurance agency will require in terms of documentation and self-studies prior to the visit of a review or audit committee or team. Ideally what DETYA requires of institutions in terms of plans should be identical, or at least not in conflict with, required documentation for quality audits.

6.10

Second, since the requirements concerning preparation and submissions of plans applies only to those institutions funded by DETYA, there is the question of whether it would be desirable for some requirement of this kind to be placed on non-DETYA funded institutions, including the two private universities, other selfaccrediting institutions and private providers. This raises difficult issues about constitutional and legal responsibilities, but possibly the States and Territories might consider placing some requirements about annual quality assurance plans on other than Commonwealth funded institutions, or the new quality assurance agency could require all 83


institutions to submit plans as part of the documentation submitted as the first stage of institutional audits. 6.11

84

Associated with the need for on-going discussions between DETYA and the new quality assurance agency, it would also be desirable to have on-going discussions by DETYA and the new quality assurance agency concerning what requirements concerning the submissions of quality assurance and improvement plans and other documentation that professional associations put on institutions and faculties as part of accreditation and re-accreditation visits. While it may be difficult to achieve a fully integrated approach, at least it would be helpful if the key parties were able to exchange documentation on an on-going basis.


7

Quality audits and a new quality agency 7.1

The final chapter considers the proposal for a new national system of quality audits and the establishment of a new agency of some kind to take responsibility of these audits. It will be recalled the proposal for the Modern Australian Model is to have separate arrangements for self-accrediting and non self-accrediting institutions. For self-accrediting institutions, there will be a quality audit every five years and the actual audit will be proceeded by a self-study. However, non self-accrediting providers will not be subject to such audits but will be subject to reviews of performance every five years as part of a re-accreditation process.

7.2

The task for our project was to: develop the Modern Australian Model as an alternative to the other four models; advise under whose authority it should be run; advise whether the framework would need a legislative base; assess whether it would be sensible and appropriate to make use of the AQF; elaborate the possible nature of the five yearly self-assessments for self-accrediting institutions; comment on the desirability of focussing more than in the past on outcomes and standards as well as processes; consider how to achieve rigour and independence for the process while retaining the cooperation and confidence of universities; and advise on the role of professional associations within the model and the nature of the audit of the courses of non self-accrediting providers.

7.3

More specifically, we were asked to make a comprehensive assessment of the Modern Australian Model against the following criteria: • Credibility (how well the model would be credible with international and domestic interest groups and potential customers, and the marketability of the arrangements); • Effectiveness (ability to address learning outcome standards as well as quality assurance processes); • Ability to provide legal clarity for students and providers; • Ability to promote and enhance improvement and good practice; • How well the model could build on the key features of the Australian higher education system, where universities are established under State/Territory/Commonwealth legislation

85


as autonomous institutions with the power to accredit their own courses, and where higher education courses developed and delivered by other providers are accredited by State/Territory bodies; • How well the model could exploit the role of professional associations in accrediting courses; • Minimum prescription and bureaucracy; and • Cost.

Characteristics of and criteria for the new mechanism 7.4

DETYA documentation specifies the following criteria for a new national quality assurance mechanism for self-accrediting institutions: • The mechanism should not be solely at the discretion of the institutions themselves; • There needs to be some external review or audit of the claims made by institutions about quality and standards; • The mechanism should be credible with international and domestic interest groups, and should and be able to protect the international reputation of Australian awards; • The mechanism should help satisfy Australian taxpayers of value for money; • Any audit mechanism should have rigour, but at the same time be cost effective, not unnecessarily intrusive and be able to retain the cooperation of the public universities; and • The mechanism should provide legal clarity for students and providers and able to promote good practice and facilitate improvement.

7.5

86

As already indicated, we support these principles. To have both domestic and international credibility, the mechanism should not be under the direct control of higher education institutions. At the same time, we consider that to ensure success and acceptance the mechanism should be regarded as a cooperative enterprise between government and the higher education sector. We strongly support the idea of an external audit whose function will be to test the claims made by institutions about quality and standards. This should be combined with an institutional self-study which would take place prior to the visit of a review panel. An alternative would be to opt for the Dutch model of disciplinary reviews, but for a number of reasons the institutional audit would appear to be more suitable. The institutional audit is likely to be cheaper and more cost efficient. It is more


commonly used internationally than the disciplinary review mechanism and it is better able to cater for a diverse sector with self accrediting and non-self accrediting institutions and both public and private providers. Further, the Dutch model is particularly effective if the main focus is on research and the state of the disciplines and their future directions. Since the idea of institutional quality assurance audits or reviews is well understood both in Australia and internationally, we consider that, with appropriate structures and resources, and with the support of both government (including the States and Territories) and the sector, an audit mechanism should soon gain strong credibility with international and domestic interest groups, and should be able to protect the international reputation of Australian awards. The mechanism should also help satisfy Australian taxpayers of value for money. Generally institutional audit mechanisms prove to be cost effective and not necessarily intrusive and have been able to retain the cooperation of universities. In both Britain and New Zealand universities have been supportive of the mechanism. An audit mechanism should help to provide legal clarity for students and providers, although as we comment elsewhere we see important legal clarity and protection coming from the new accreditation arrangements. 7.6

Further still, while our interview schedule was limited because of time constraints, we formed the view that a non-intrusive and sensibly conceived quality assurance mechanism which involved both the higher education sector and the State and Territories would be likely to attract considerable support. Certainly both with the sector and with State and Territory accrediting agencies there is wide appreciation of some of the strong influences that require establishment of a new national mechanism.

7.7

Apart from the criteria mentioned above, we suggest that other broad principles should guide the establishment and operation of a new quality assurance mechanism. These include the following:

Cooperative Commonwealth/State/Territory and higher education effort 7.8

We consider it important that the proposed new quality assurance arrangements should be a cooperative Commonwealth/State/Territory and higher education sector effort, rather than a Commonwealth initiative. Not surprisingly, officials in State and Territory accrediting agencies spoke strongly that the new mechanism should not be simply a Commonwealth creation, arguing on the basis of the

87


constitutional responsibilities of the States in education, that overall accountability lay with the States and Territories, Commonwealth officials did not understand fully many of the key issues and that any new arrangements should accommodate the current legal and legislative responsibilities of State and Territory agencies. For somewhat different reasons, we feel confident that the idea of a cooperative effort would attract much greater support from the AVCC as opposed to the idea of the initiative led and controlled by the Commonwealth.

Small and cost efficient agency, with minimum bureaucracy 7.9

We found strong support for the idea that, should a new national quality assurance agency be established, it should be relatively small and cost efficient, it should be independent of DETYA, State and Territory Accrediting Agencies, and it should have a minimum of bureaucracy. The size and activities of a new agency should resemble that of the New Zealand Academic Audit Unit of the former British Academic Audit Unit and Higher Education Quality Council. We found no support for the idea that a new agency should be part of or have some link with the AQF.

Focus on processes rather than outputs 7.10

88

One of the most difficult questions facing the establishment of a new quality assurance mechanism is whether the main focus should be on processes rather than on outputs and standards. We recognise that there is wide community and international interest in the issue of academic standards generally and particularly in standard between degrees offered by different Australian universities. There is also considerable interest in how Australian degrees compare with those offered by universities in other industrialised countries, particularly those that compete with Australian providers in international education. There are also domestic accountability pressures working to direct more attention to standards and outputs. On the other hand, to put an emphasis on standards is fraught with danger and difficulty. Many universities will be far less comfortable with an emphasis on outputs and standards and will recall the controversy that attracted the ranking or ‘banding’ of institutions with the 1993–1995 quality assurance program. Many newer and smaller universities consider that the rankings and performance funding seriously damaged their reputations and are likely to be cynical about any audit program which would most likely give the oldest and research intensive


universities the strongest rankings. Any serious attempt at focussing on outputs and academic standards is likely to raise difficult methodological issues and be controversial. The recent experiences of the British Quality Assurance Agency point to how proposals to assess standards are likely to be highly controversial and to be regarded especially by major universities as being highly intrusive. We assume also that special technical staff would be required or the proposed agency would need to make use of the services of consultants with special expertise in educational measurement and judging the equivalence of academic standards. 7.11

Three further arguments need to be mentioned against an emphasis on outputs and standards. First, by placing the primary focus of audits on planning, management and monitoring processes within institutions, it is still possible for panels to collect considerable information on outputs and standards. Further, the issue of outputs and standards is probably best addressed in the context of reviewing institutional quality assurance and improvement plans in the light of institutional missions, how institutions monitor and make judgements about their performance, and what evidence they have to substantiate the judgements they make. Hence the focus is not on making judgements about institutional performance but how effectively and professionally institutions monitor their own performance and use the information gained for institutional planning and improvement.

7.12

Second, we suspect that assessments of outputs and standards are unnecessary in terms of credibility, both in Australia and overseas. Quite simply, most enquirers wish to know whether or not there is a national agency for quality assurance and what programs of reviews it has undertaken

7.13

Third, issues about outputs and standards can be addressed in simpler and more cost-effective ways. One possibility would for a new agency to have a small budget for investigations and evaluations and to commission studies that could address particular issues about standards. Possibly one or more of the professional associations involved in course accreditation might be interested in comparative studies of course requirements, desirable graduate attributes and assessment methods for particular disciplines.

89


Voluntary participation by higher education providers 7.14

We favour the principle of voluntary participation in any new quality assurance program. Such an approach is likely to be much more acceptable to the sector and likely to achieve a much higher degree of support. It will be recalled that participation in the 1993–1995 quality assurance program was voluntary yet, despite some threats of withdrawal, all universities participated in each of the three rounds. We favour an arrangement whereby all higher education institutions would be eligible for membership and with all members paying an annual subscription fee that could be based on student load. At the same time, review procedures would need to be sufficiently flexible to cater for institutions of differing size.

Name for new agency likely to attract support and credibility 7.15

The actual name of the new agency could be important in gaining support from the higher education sector and credibility both in Australia and internationally. We favour use of the words ‘quality assurance’ in the title, since this term has become well understood and has been used recently in both the United Kingdom and New Zealand. The words ‘authority’ or ‘agency’ seem suitable for a relatively small agency, but their use could lead to confusion with the British and New Zealand bodies. For this reason the words council or board might be more appropriate. Ideally the title of the organisation should be, as short as possible, but to assist with international education it could be thought useful to include the word ‘Australian’.

Mechanisms to commission studies about standards and good practice 7.16

90

If an new agency is to play an important role in dissemination of good practice and addressing questions about standards, this should be made clear in the brief and there should be some understanding at the outset about the means by which such aims could be achieved. As already mentioned, one cost efficient means would be the use of limited funds to commission studies, publish reports, and sponsor conferences and seminars.


Legal basis and structure of new agency 7.17

One important issue to address that will affect the success of any new quality assurance mechanism is the legal basis of the responsible agency, the governance structures employed and accountability arrangements. In our discussions a number of different models were suggested and these can be summarised as follows: • Ministerial Committee set up by the Commonwealth Minister; • Statutory body established by Commonwealth legislation; • Statutory body established by joint Commonwealth and State legislation; • Agency set up as a company. Each of these models has some attractions. A Commonwealth ministerial committee is relative simple to establish and was used for the 1993–1995 quality assurance, but it could carry the impression that it is a DETYA controlled agency. A similar problem would be associated with Commonwealth legislation. On the other hand, while having many attractions, complementary Commonwealth and State/Territory legislation could take time to achieve. The company model has many attractions. Possibly a company could be established with similar arrangements and links to MCEETYA as those for the Curriculum Development Centre.

7.18

Officers in one of the State accrediting agencies suggested a two level structure whereby most of the work in quality assurance would be done by the States and Territories, while at the national level a relatively small body would set basic principles and oversee the process. While this model would be likely to facilitate integration of accreditation and quality assurance functions, we favour a single national agency and consider that it is of utmost importance that the Commonwealth should be intimately involved.

7.19

We are not in a position to advise on the desirability of the proposed agency having a legislative base, but certainly one issue that needs consideration is whether, by having an appropriate legal basis, the agency would be more likely to be protected in the case of litigation.

7.20

Perhaps more important than the legal basis is the system of governance and accountability. We favour the new agency being managed by a board or council made up a Commonwealth, State/Territory, and higher education representation and having an independent Chair. In order to establish close links with the accrediting activities of professional associations, it could be useful to include a board or council

91


member drawn from one of the major professional associations or from the Australian Council of the Professions. Such representation should be of a person with special expertise and major involvement in accreditation. In addition, the governing body should have the power to add one or two additional members with special expertise in the areas of academic audits and assessment.

Preferred model 7.21

Our preferred model for the new quality assurance mechanism and agency is as follows: • A new quality assurance mechanism should be established as a joint Commonwealth, State/Territory, and higher education initiative with the aim of strengthening public accountability, protecting academic standards and the reputation of Australian higher education providers and awards, and promoting good practice in quality assurance. We suggest that the new mechanism should be called the Higher Education Quality Assurance Council. • The central function of the Council will be conduct of program of institutional reviews or audits. Review teams will carry out site visits, following completion of self-assessments carried out by institutions, which will include reviews of the processes of managing quality including monitoring performance and benchmarking. Institutions will provide review teams with a report of their self-assessments, together with documentation on institutional mission and objectives, quality assurance and improvement plans, details on methods used to monitor and benchmark achievements and the results of monitoring and benchmarking. Participating institutions will be reviewed every five years. • The Council will be established an independent agency, at ‘arms length’ from both government (Commonwealth and State) and from the higher education sector. It will be governed by a board consisting of an independent Chair, two Commonwealth nominees, two members representing the States and Territories, two representatives of the higher education sector and one representative drawn from those professional associations involved in accreditation within the higher education sector. The Executive Director will be an exofficio member and the board will have the power to coopt up to two additional members with special expertise in academic audits and assessment. Commonwealth representatives will be appointed

92


by the Minister for Education, Training and Youth Affairs, while the two State and Territory representatives will be appointed by MCEETYA. Members will serve four year terms. • Funding for the work of the Council will come from annual grants from the Commonwealth and from the States and Territories, and annual membership fees paid by individual higher education institutions who wish to participate in the program of reviews. • The terms of reference of the Council will be as follows: – to review within participating higher education institutions the mechanisms for quality assurance, monitoring performance and academic standards, and enhancing quality; – to publish the reports of reviews; – to report publicly from time to time on the effectiveness of quality assurance procedures in participating institutions, the extent to which procedures ensure academic standards and reflect good practice in maintaining and improving quality, and other relevant matters; – to identify and disseminate good practice in quality assurance in higher education; – to undertake and sponsor studies related to effective quality assurance management practices and academic standards in higher education. • In carrying out reviews, review teams appointed by the Council will focus particularly on: – appropriateness of quality assurance and improvement plans in relation to institutional contexts and missions; – rigour of the mechanisms employed to review courses and academic organisational units, and monitor performance against institutional plans; – effectiveness in monitoring outcomes and in benchmarking, both nationally and internationally; and – success in communicating the results of the monitoring outcomes and academic standards to stakeholders.

93


Review teams will report to the Council. • Each year the Council will draw up a program of reviews for the following year, after consultation with institutions likely to be reviewed. • Review panels, generally of no more than five members, will be appointed by the Council. Members of review teams will be drawn from the higher education sector, the Commonwealth and the States. Members may also be drawn from the professions and professional associations, and from business and industry. Review panels will normally visit institutions for two consecutive days after the institution has completed a self-assessment and supplied other documentation as required. Institutions offering courses ‘off-shore’ for international students should document in detail the procedures followed for safeguarding and monitoring quality, and the results of any assessments. • Following the visit of the review team, the draft report will be forwarded to the institution for comment. Once the report is completed it will be considered by the Council and then published. Copies will be provided free to DETYA, State and Territory accrediting agencies, all participating higher education institutions, and relevant professional associations. For each review, a single report will be prepared and published. • Should a review reveal serious weaknesses, the institution concerned will be given up to 12 months to correct weaknesses prior to a supplementary review. Failure to rectify weaknesses would be a matter for DETYA to address (in the case of Commonwealth funded institutions) or for the relevant State or Territory accrediting agency. One possible action would be to remove the name of the institution from the AQF list of accredited institutions until such time that as minimum standards are achieved. • Every effort should be made to encourage private universities and non self-accrediting institutions to participate in the review program. • Prior to arrangements for the Council being finalised, the higher education sector should be consulted about the proposed terms of reference for the Council, the composition of the Council’s board and the method of conducting reviews.

94


Appendix A

Project brief A Modern Australian Model This model, which builds on the current or recent practice, would embrace two related functions - accreditation and quality assurance. It would have two distinct branches: (a) For institutions which are given power to accredit their own courses The main requirements for these institutions would be: – Rigorous scrutiny of financial and quality aspects before founding legislation is passed or other authorisation is given. – The annual publication of Quality Assurance and Improvement Plans for the forthcoming triennium. These plans would outline the institution’s goals, strategies for achieving those goals and the indicators used to monitor progress in achieving those goals. An analysis of performance, including a consideration of the outcomes of graduate satisfaction surveys (the CEQ) and graduate employment outcomes (the GDS) over time (say, the previous five years) and compared with appropriate benchmarks, would form an integral part of these plans. The plans would need to outline very clearly the processes in place to assure quality of provision within its ‘catchment area’ —thus, if an institution operates offshore, whether physically or virtually, it would need to outline what mechanisms it has in place to assure quality in relation to that provision. – A detailed self-assessment, which would include benchmarking of standards, to be conducted every five years. This assessment would be audited on a whole-of-institution basis. The audit team could be made up of Government officials and/or members of independent bodies, such as the Council of the Learned Academies. Should the audit reveal serious areas of weakness, the institution would be given 12 months to address such matters. Failure to rectify serious deficiencies would result in the Government removal of the institution from the AQF list of accredited institutions.

95


– Compliance with any additional measures which may be necessary to ensure the maintenance of acceptable high standards of degrees, in the environment described in the Background of this brief. (b) For non self-accrediting providers Work is currently underway to develop a common approach to regulating the entry of private providers of higher education courses. The main features of such an approach might be: – Rigorous scrutiny of provider capability before course accreditation; and – Review of provider performance and accredited courses every five years.

Issues Numerous issues need to be addressed in a consideration of a possible model for Australia. These include: • Under whose authority the quality assurance and accreditation system would be run. There are two obvious options - the Commonwealth alone or the Ministerial Committee for Education, Employment, Training and Youth Affairs. The case for the Commonwealth alone assuring itself of provider quality rests on the availability of Commonwealth-funded subsidies. • Whether the framework would need a legislative base or whether it could be set up as an instrument of government policy (as is the case with the Australian Qualifications Framework (AQF)). • Whether it is sensible and appropriate to make use of the AQF. On the one hand, it has high public visibility; on the other, there may be resistance to building on something which has had a major vocational education and training focus. It may be prudent to use the AQF simply as a vehicle for listing accredited higher education institutions—giving such institutions an official imprimatur. • The nature of the five-yearly self-assessment for self-accrediting institutions. • The desirability of focussing (more than in the past) on outcomes and standards as well as processes. • How to achieve rigour and independence for the process while retaining the cooperation and confidence of the universities. • The role of professional associations within the model, and implications of accreditation by them.

96


• The nature of the audit of the courses of non self-accrediting providers. The extent to which audits would focus on courses rather than providers. The role and status of the audit teams need to be addressed.

The Task The task is to develop Model 5 as an alternative to models 1–4, mindful of the issues listed above, and make a comprehensive assessment of this model against the following criteria: • credibility. This involves assessing how well the model would be credible with international and domestic interest groups and potential customers, and the ‘marketability’ of the arrangements; • effectiveness, ability to address learning outcome standards as well as quality assurance processes; • ability to provide legal clarity for students and providers; • ability to promote and enhance improvement and good practice; • how well the model could build on the key features of the Australian higher education system, where universities are established under State/Territory/Commonwealth legislation as autonomous institutions with the power to accredit their own courses, and where higher education courses developed and delivered by other providers are accredited by State/Territory bodies; • how well the model could exploit the role of professional associations in accrediting courses; • minimum prescription and bureaucracy; and • cost.

Consultations Consultation with key stakeholders (e.g. the AVCC, State accrediting bodies, NOOSR, professional associations) should be undertaken as appropriate.

97


Appendix B

List of interviews Canberra Mr Michael Gallagher First Assistant Secretary, Higher Education Division Department of Education, Training and Youth Affairs Dr Tom Karmel Assistant Secretary, Higher Education Operations Branch Department of Education, Training and Youth Affairs Ms Rebecca Cross Chief Executive Officer, Australian Education International Department of Education, Training and Youth Affairs Mr Tom Calma Counsellor (Education & Training) Australian Education International Australian Embassy Hanoi, Vietnam Mr Giancarlo Savaris Assistant Secretary, NOOSR Department of Education, Training and Youth Affairs Ms Margaret Bell NOOSR Department of Education, Training and Youth Affairs Mr Stuart Hamilton Executive Director Australian Vice-Chancellors’ Committee Mr P Rodley Administrative Officer Australian Vice-Chancellors’ Committee

99


Ms Ann Ryle Associate Director, Education and Membership The Institution of Engineers, Australia Professor Peter Parr Consultant The Institution of Engineers, Australia Ms Minou Lamb Office of Training and Adult Education ACT Department of Education and Training

Sydney Ms Lyndsay Connors Director, Higher Education Office NSW Department of Education and Training Mr Graham Wood Principal Policy Officer, Higher Education Office of Higher Education NSW Department of Education and Training Mr John Williams Office of Higher Education NSW Department of Education and Training

Melbourne Dr Ian Allen Deputy Secretary Victorian Department of Education Mr Tim Smith Assistant Secretary Higher Education Branch Victorian Department of Education Ms Wendy Katz Manager, National Recognition Policy Australian National Training Authority

100


Dr Judy Forsyth Executive Officer Australian Qualifications Framework Advisory Board Secretariat Dr Grant McBurnie Director, Transnational Quality Assurance Programs Monash University Mr John McPartland Assistant General Manager Monash International Professor Allan Lindsay Deputy Vice-Chancellor Academic Monash University Ms Noreen Cruse Acting Director, Quality Assurance Planning and Quality Unit RMIT Associate Professor Craig McInnis Director Centre of the Study of Higher Education University of Melbourne

Brisbane Ms Leigh Tabrett Director, Higher Education Office Queensland Department of Education Ms Sian Lew Senior Policy Officer, Higher Education Queensland Department of Education Mr Steve McDonald Director, Industry and Training Framework Team Australian National Training Authority Armidale Professor Brian Stoddart Deputy Vice-Chancellor (Research and External) University of New England

101


References Australian National Training Authority (1998) Updated Guidelines for Training Package Developers, Brisbane. Australian National Training Authority (1998) Australian Recognition Framework Arrangements, Brisbane Anwyl, John (1992) Quality in Higher Education: Proceedings of the National Invitational Workshop held at the University of Melbourne, Centre for the Study of Higher Education, University of Melbourne, Parkville. Arcelo, Adriano (1992) ‘Governance and Management Issues in Philippine Higher Education’, paper presented at World Bank Seminar, Singapore. Asking, Berit and Bauer, Mairanne (1997) ‘The Role, Functions, and the Impact of a National Agency in the Evaluation of a Decentralised Higher Education System’, paper presented at CHER conference, Alicante. Australian Vice-Chancellors’ Committee (1989) ‘AVCC Guidelines on the Criteria for a University’, Canberra. Australian Vice-Chancellors’ Committee (1997) ‘Guidelines on the Criteria for a University Recognised by the AVCC’, Canberra. Ayarza, Herman (1994) ‘Latin America’ in Alma Craft (ed), International Developments in assuring Quality in Higher Education, Falmer, Brighton, pp 18–27. Ball, C (1985) Fitness for Purpose, SRHE and NFER-Nelson, Guilford. Ballarat University College Proposal for Recognition as a University: Report of Review Panel (1993), Melbourne. Bauer, Marianne and Franke-Wikberg, Sigbrit (1993), ‘Quality Assurance in Swedish Higher Education’, paper presented at the First Biennial Conference and General Conference of the International Network of Quality Assurance Agencies in Higher Education, Montreal. Beyond the Boundaries: Education and Training for the Engineering Workforce: A study into the Education of and Training of Engineering Technologists and Engineering Associates (1998), The Institution of Engineers, Australia, Canberra.

103


Birnbaum, Robert (1994) ‘The Quality Cube: How College Presidents Assess Excellence’, Journal of Higher Education Management, Vol 9, No 3, pp 71–82. Boffo, Stefano and Moscati, Roberto (1997) ‘Evaluation of the Italian Higher Education System: Many tribes, many territories, ... and many godfathers’, paper presented at CHER conference, Alicante. Brennan, John, de Vries, Peter, and Williams, Ruth (Eds.) (1997) Standards and Quality in Higher Education, Jessica Kingsley, London. Changing the Culture: Engineering Education into the Future, Review Report (1996) The Institution of Engineers, Australia, Canberra. Cooney, Robert P and Paqueo-Arrezo, Eliza (1993) ‘Higher Education Regulation in the Philippines: Issues of Control, Quality Assurance and Accreditation’, Higher Education Policy, Vol 6, No 2, pp 25–28. Council of Europe (1997) Convention on the recognition of qualifications concerning Higher Education in the European Region, Lisbon. Craft, A (Ed.) (1992) Quality Assurance in Higher Education, (London, Falmer). Craft, A (Ed.) (1994) International Developments in Assuring Quality in Higher Education, Falmer, London. Crow, Stephen (1994) ‘Changing Emphases in the USA’ in Alma Craft (Ed.) International Developments in Assuring Quality in Higher Education, Falmer, London, pp 118–123. Department of Education, Training and Youth Affairs (1999) ‘Project Brief: Quality Assurance and Accreditation in Higher Education’, Canberra. Dill, D (1997) ‘Accreditation, Assessment and Anarchy? The Evolution of Academic Quality Assurance Policies in the United States’ in John Brennan, Peter de Vries, and Ruth Williams, (Eds.), Standards and Quality in Higher Education, Jessica Kingsley, London. Education Queensland, Office of Higher Education (1997) ‘Procedures and Criteria for the Accreditation of Higher Education Courses offered by Non-University Providers’, Brisbane. El-Khawas, Elaine (1993) ‘Accreditation and Evaluation: Reciprocity and Exchange’, paper presented at EC Conference on the Frameworks for European Quality Assessment in Higher Education, Copenhagen.

104


El-Khawas, Elaine (1997) ‘Strong State Action but Limited Results: Perspectives on University Performance’, revised version of paper presented at CHER Conference, Alicante. Frazer, Malcolm (1991) ‘Quality Assurance in Higher Education’, Council for Academic Awards, London. Harman, Grant (1994) ‘Australian Higher Education Administration and the Quality Assurance Movement’, Journal of Higher Education Management, Volume 9, Number 2, pp 25–45. (Also published in Journal of Tertiary Education Administration, Volume 16, Number 1, pp. 25–43). Harman, Grant (1996a) Quality Assurance for Higher Education: Developing and Managing Quality Assurance for Higher Education Systems and Institutions in the Asia and the Pacific, Asia-Pacific Centre of Educational Innovation for Development, Unesco Principal Regional Office for Asia and the Pacific, Bangkok, 105+ix. Harman, Grant (1996b) ‘Quality Assessment with National Institutional Rankings and Performance Funding: The Australian Experiment, 1993–1995’, Higher Education Quarterly, Vol 50, No 4, pp 295–311. Harman, Grant (1998) ‘The Management of Quality Assurance: An International Review of International Practice’, Higher Education Quarterly, Vol 52, No 4, pp 345–364. Harman, Grant (1999) ‘Allocating Block Research Grants in Comprehensive University Systems: A Comparison of the British Research Assessment Exercise and the Australian Composite Index and Research Quantum’, Paper presented at the CHER Conference, Oslo, September. Higher Education Funding Council of England, Scottish Higher Education Funding Council, Higher Education Funding Council of Wales, and Department of Education of Northern Ireland (1995) ‘Research Assessment Exercise: Criteria for Assessment’. Bristol. Higher Education Funding Council of England, Scottish Higher Education Funding Council, Higher Education Funding Council of Wales, and Department of Education of Northern Ireland (1995), ‘Research Assessment Exercise: Guidance for Submissions’. Bristol. Kells, H R (1992) Self-Regulation in Higher Education: A Multinational Perspective on Collaborative Systems of Quality Assurance and Control, Jessica Kingsley, London.

105


Kells, H R and van Vught, F A (Eds.) (1988) Self-Regulation, Self-Study and Program Review in Higher Education, Lemma, Utrecht. Lee, Wha-kuk (1993) ‘Issues in the Development of University Accreditation in Korea’, paper presented at the First Biennial Conference and General Conference of the International Network of Quality Assurance Agencies in Higher Education, Montreal. Lindsay, A W (1992) ‘Concepts of Quality in Higher Education’, Journal of Tertiary Education Administration, Vol. 14, No 2, pp 153–163. Lindsay, A W (1994) ‘Quality and Management in Universities’, Journal of Higher Education Management, Vol 9, No 3, pp 57–70. Liuhanen, Anna-Maija (1997) ‘Internal Quality Assurance in Finnish Universities,’ paper presented at CHER conference, Alicante. Malcom, Wilf (1993) ‘The Development of an Academic Audit Unit in New Zealand’, paper presented to the First Biennial Conference and General Conference of the International Network of Quality Assurance Agencies in Higher Education, Montreal. McNay, Ian (1997) The Impact of the 1992 RAE on Institutional and Individual Behaviour in English Higher Education: the Evidence from a Research Report, Centre for Higher Education Management, Anglia Polytechnic University, Chelmsford. Middlehurst, Robin (1992) ‘Quality: An Organising Principle for Higher Education’, Higher Education Quarterly, Vol. 46, No 1, pp 20–38. Neave, G (1991) Models of Quality Assurance in Europe, CNAA, London. Neave, G (1997) ‘The Rise of the Evaluative State: The State of the Art’, paper presented at CHER Conference, Alicante. Policy on Accreditation of Professional Engineering Courses (1999) The Institution of Engineers, Australia, Canberra. Report of the Panel to Review the Submission by Melbourne University Private for Approval to Operate as a University (1998) Melbourne. Rajavaara (1998) Professionals and Quality Initiatives in Health and Social Services in Developing Quality in Personal Social Services, Ashgate Publishing, Vienna. Sizer, J (1990) ‘Funding Councils and Performance Indicators in Quality Assessment in the United Kingdom’ in L C J Goedegeburre, P A M Maasen and De F Westenheijden (Eds) Peer Review and Performance Indicators, Lemma, Utrecht. 106


Smith, Tim (1998) ‘Higher Education Private Providers: The New Ball Game’, paper presented at 22nd ATEM Conference, Darwin. Strydom, A H (1997) ‘A Cross-National and National Analysis of Policy Proposals for a Quality Assurance (Evalution) System for Higher Education in South Africa based on a “Cooperative Governance” Framework’, paper presented at CHER conference, Alicante, September. Su, Jin-Li (1993) ‘Specialized Accreditation in Taiwan: Issues and Perspectives’, paper presented at the First Biennial Conference and General Conference of the International Network of Quality Assurance Agencies in Higher Education, Montreal. Thune, Christian (1994) ‘Setting up the Danish Centre’ in Alma Craft (Ed.) International Developments in Assuring Quality in Higher Education Falmer, London, pp 67–72. Thune, Christian (1997) ‘’The Balance Between Accountability and Improvement: The Danish Experience’ in John Brennan, Peter de Vries, and Ruth Williams, (Eds), Standards and Quality in Higher Education, Jessica Kingsley, London. Tsim, T L (1993) ‘The Universities of Hong Kong’ in Commonwealth Universities Yearbook 1993, The Association of Commonwealth Universities, London, pp 957–960. van Vught, F A and Westerheijden, D F (1992) Quality Management and Quality Assurance in European Higher Education: Methods and Mechanisms, Center for Higher Education Policy Studies, University of Twente, Enschede. van Vught, Frans A (1994a), ‘Western Europe and North America’ in Alma Craft (ed.) International Developments in Assuring Quality in Higher Education, Falmer, London, pp 3–17. van Vught, Frans A (1994b) ‘Autonomy and Accountability in Government/University Relationships’ in Jamil Salmi and Adriaan Verspoor (eds), Revitalizing Higher Education, International Association of Universities and Elsevier Science, Oxford, pp 322–362. VSNU (1996) Quality Assessment of Research: Earth Sciences in the Netherlands, Utrecht. VSNU (1998) Assessment of Research Quality: Protocol 1998, Utrecht. Williams, P R (1991) The CVCP Academic Audit Unit, Birmingham.

107


Williams, Ruth (1997) ‘Quality and Diversity: The Case of England’ in John Brennan, Peter de Vries, and Ruth Williams, (Eds), Standards and Quality in Higher Education, Jessica Kingsley, London. Woodhouse, David (1997) ‘Qualifications and Quality in New Zealand’ in John Brennan, Peter de Vries, and Ruth Williams, (Eds), Standards and Quality in Higher Education, Jessica Kingsley London. Zijderveld, Diederick C (1997) ‘External Quality Assessment in Dutch Higher Education: Consultancy and Watchdog Roles’, Higher Education Management, Vol 9, No 1, pp 31–41.

108