COST-A6-WG2, European Commission Social - Emcdda

ISBN 92-828-2913-8

ISSN 1018-5593

European Commission

Social Sciences

&267$ Evaluation Research in Regard to Primary Prevention of Drug Abuse

Edited by

Alfred Springer & Alfred Uhl Ludwig-Boltzmann-Institute for Addiction Research Vienna - Austria

Directorate-General Science, Research and Development Brussels 1998

EUR 18153 EN I

Contents Preface of the Editors (Alfred Springer & Alfred Uhl)................................................................. 1 Evaluating Drug Prevention: An Introduction (Ambros Uchtenhagen & Katarzyna Okulicz-Kozaryn)........................... 5 Country Reports: An Overview, Including Some Remarks about Socio-Cultural Determinants of Primary Prevention and its Evaluation (Alfred Springer) ..................................................................................... 19 Building Expertise in Life Skills Programme Adaptation and Evaluation: The Experience of "Leefsleutels" (Annick Vandendriessche) ...................................................................... 65 Mia’s Diary: An Alcohol and Drug Primary Prevention Programme for the Nordic Countries (Line Nersnæs) ........................................................................................ 77 Evaluations of Substance Use Prevention Programmes Implications or Illicit Drugs (Mark Morgan) ........................................................................................ 91 Evaluation of Primary Prevention in the Field of Illicit Drugs: Definitions - Concepts - Problems (Alfred Uhl) ........................................................................................... 135 Index.......................................................................................................... 222

II

Evaluation Research in Regard to Primary Prevention of Drug Abuse

Preface of the Editors Alfred Springer, Chairman COST-A6-WG2 Alfred Uhl, Research Co-ordinator

___________________________________ We joined the COST-A6-action of the European Commission right from its beginning in December 1992. In the first workshop of that research initiative held in Zurich, we agreed to organise a work group on the evaluation of primary prevention (COST-A6-WG2). Our first step was to identify relevant experts from different countries in the European region who were then asked to produce country reports describing the situation of primary prevention and its evaluation in their countries. In June 1994 we organised a first meeting of international experts in Vienna. At this meeting we were happy to welcome several outstanding experts from the field of prevention evaluation as invited speakers. The aim of this initial meeting was to constitute an international working group on "Evaluation Research in Regard to Primary Prevention of Drug Abuse" within the COST-A6-Action of the European Union. The participating experts in this first COST-A6-WG2 meeting decided to split up the task into three distinct subprojects. • Project A was to give an overview over evaluated prevention projects throughout Europe focusing on illicit drugs based on Country Reports from participating countries (co-ordinator Alfred Springer). • Project B was to utilise the vast amount of experience regarding the evaluation of primary prevention programmes focusing on legal drugs for the field of illicit drugs. This subproject titled "Evaluations of Substance Use Prevention Programmes: Implications for Illicit Drugs" was carried out based on literature by Mark Morgan. • Project C was of theoretical nature. The goal of this subproject named "Evaluation of Primary Prevention in the Field of Illicit Drugs: Definitions - Concepts - Problems" was to reach a consensus among international experts over these theoretical matters (co-ordinator Alfred Uhl).

1

Preface of the Editors

The work on these subprojects proceeded in agreement with our working plan and was finished in 1997. While developing our own project we were confronted with the situation that two other projects in the fields of primary prevention and evaluation research were initiated by other international bodies and institutions in Europe: • The Pompidou Group ("Co-operation Group to Combat Drug Abuse and Illicit Trafficking in Drugs within the Council of Europe") started an initiative to develop a "Handbook Prevention" and • the EMCDDA ("European Monitoring Centre for Drugs and Drug Addiction") started another action to develop a manual for programme planners and evaluators ("Evaluation of Drug Prevention Intervention"). In charge of the Pompidou Group project were Jaap van der Stel and Deborah Voordewind in co-operation with Wim Buisman (Jellinek Consultancy, Amsterdam). A final report of this programme is due in 1998. In charge of the EMCDDA project are Christoph Kröger, Heike Winter and Rose Shaw (IFT, Munich). A final report of this programme is due in 1998 as well. During the running time of all the three projects a close collaboration - a true network development - took place. Knowledge and manpower from the COST-A6-WG2 action and from the Pompidou group initiative were fed into the EMCDDA action from the very start of the EMCDDA action. Members from COST-A6-WG2 and from the Pompidou group initiative met with EMCDDA representatives in Lisbon in 1995 for a constitutional meeting of the EMCDDA project. The major positive outcome of this early involvement was an agreement between the co-ordinators of all three projects to co-operate in a manner • that duplications and rivalries are avoided, • that synergy effects are utilised through exchanging information and supporting each other, • that a consistent use of terminology is guaranteed as far as possible and • that all three projects refer to each other, pointing out the specific emphasis of the other projects to the interested public.

2


The aims of all three projects were defined as clearly distinct from each other: • The goal of the COST-A6-WG2 - Project A was defined to give a comprehensive overview over evaluated primary prevention projects in Europe. • The COST-A6-WG2 - Project B was aiming at an utilisation of existing scientific evidence concerning evaluation of prevention in the field of licit drugs for the field of illicit drugs. • The COST-A6-WG2 - Project C focused on exact definitions and methodological problems. • The Pompidou Group Project "Handbook Drug Prevention" has a very global approach covering a variety of practical aspects from programme development to implementation and evaluation. • The EMCDDA Project focuses on practical instructions for programme planners and evaluators who are to evaluate their own prevention interventions before or after their implementation on a routine basis. There is hardly any overlap in major contents and the projects are far from duplicating each other - on the contrary, they can be seen as complementing each other. Very helpful in this regard was also that several experts contributed to more than one project. For the format of the presentation of our final report we chose to document our results in the chronological sequence of the progression of the project. The country reports we collected in the first phase of our project and the insights gained thereby concerning the rather poor state of the art in the evaluation of primary prevention initiated an effort to deal with the topic on a more fundamental level. The presentations of the adaptation process of the QUEST programme for the situation in Belgium and of the school based drug education programme "Mia's Diary", which is widely used in the Scandinavian countries, have been chosen as practical examples of prevention programmes characterised by very high standards and also for a specific approach to adapt programmes to different socio-cultural contexts. The subproject "Evaluations of Substance Use Prevention Programmes Implications for Illicit Drugs" carried out by Mr. Morgan as a comprehensive literature overview provided us with the necessary background for further theoretical and methodological steps.

3

Preface of the Editors

The summarised country reports and the Mark Morgan’s literature overview constituted the starting point for the Delphi-type consensus study on definitions and methodological concepts, based on very detailed inputs from 21 international experts. We have to thank all the excellent experts who donated a lot of time and knowledge to collaborate in the COST-A6-WG2 project without any financial reward, as invited speakers in one of the work group meetings, as authors of country reports, as active participants in the consensus study on definitions and methodological problems, as authors of chapters in this publication and/or as valuable partners in discussing relevant topics with us on an informal basis. We name them in alphabetical order: Stefan Brülhart (Suchtpräventionsstelle, Zurich), Gregor Burkhart (EMCDDA, Lisbon), Xavier Ferrer (A.B.S. and CEUDROG, Barcelona), Maria Xesús Froján Parga (Universidad Autonoma de Madrid), Willy F.M. De Haes (Rotterdam), Osmo Kontula (University of Helsinki) (Christoph Kröger, IFT, Munich), Han Kuipers (Trimbos Institute, Utrecht), Ralph Kutza (IFT, Munich), Mark Morgan (St. Patrick’s College, Dublin), Alice Mostriou (Eginition Hospital, Athens), Margareta Nilson (EMCDDA, Lisbon), Katarzyna Okulicz-Kozaryn (Inst. Psychiatry and Neurology, Warsaw), Flavia Pansieri (UNDCP, Vienna), Ulf Rydberg (Karolinska Hospital, Stockholm), Chafic Saliba (Cndt, Lyon), Reginald G. Smart (ARF, Toronto), Enrico Tempesta (Universita Cattolica del Sacro Cuore, Rome), Alberto Tinarelli (Sert, Ferrara), Ambros Uchtenhagen (ISF, Zurich), Annick Vandendriessche (Leefsleutels vzw Jongeren, Brussels), Brit Unni Wilhelmsen (University of Bergen, Bergen), Heike Winter (IFT, Munich). Their personal expertise - hopefully - will help to improve the future situation of prevention evaluation in Europe. Last but not least we have to acknowledge Klemens Widensky (LBISucht, Vienna) for reading and correcting the text repeatedly throughout the project.

4


Evaluating Drug Prevention: An Introduction Ambros Uchtenhagen, ISF, Zurich, Switzerland Katarzyna Okulicz-Kozaryn, Institute of Psychiatry and Neurology, Warsaw, Poland

___________________________________ 1 The need for developing evaluation research (Ambros Uchtenhagen)

1.1 From belief to science to practice Drug prevention as a professional task has emerged in pluralistic societies where traditional beliefs and lifestyles no longer guarantee generally accepted forms of substance use, limits of use and use patterns. Transgressors traditionally were not regarded as suffering from a specific condition; they were morally judged, outcast or punished. Prevention was a part of the educational mainstream how to live a life which is compatible with societal norms. As such, prevention and its effects did not need to be scientifically tested. Since norms and lifestyles have lost much of their educational value, and since contradicting norms and lifestyles coexist in pluralistic societies, everybody has to find his or her way between personal needs, given opportunities and their chances and risks. The opportunities for substance use are abundant, and apart from specific rituals in specific milieus, substance use is mainly geared by individual expectations. The prevailing theory nowadays, among a large number of addiction theories, is the self-medication theory: people use substances in order to better control and adapt their emotional and/or cognitive state and their behaviour. The prevailing concept, on the other hand, is the disease concept of substance related conditions (World Health Organisation 1991). In this context, drug prevention has the task to guide people how to avoid negative experience and substance related conditions, eventually how to make best use of available substances without undue risks.

5

Evaluating Drug Prevention: An Introduction

Such a prevention concept cannot be of value without empirical evidence of its own effects. Should drug prevention be helpful in order to reduce negative experience, there is a need for defined goals and strategies which have been tested. The implementation of empirically tested prevention strategies has again to consider carefully what is helpful for whom and under what circumstances. The practice of drug prevention, the step from scientific evidence to practice, calls for professionalism, not less than the step from belief to science.

1.2 The basic conflicts of drug prevention and the needs for a scientific basis for prevention Evaluation starts with clearly defined goals of preventive activities. Systematically speaking, we may differentiate between the prevention of • any use of a given substance • problematic or harmful use • compulsive use (dependence). From a Public Health point of view, the extent of use in a given population may be at stake, calling for strategies such as decreasing the number of users by reducing the availability of the substance, by postponing the age when use starts, by increasing the efforts to stop or decrease already started use e. a. An essential issue in Public Health is the reduction of harmful use and its negative implications for the health system and on economy. These goals often are perceived as being in conflict with each other. Especially, preventing harmful use is considered to be permissive, in contrast to a prohibitionist position. This basic conflict is even more accentuated when it comes to illegal substances, which per se are prohibited, but are more or less widely used, and where an intention to reduce harmful use can also be a legitimate goal. The conflict is also more problematic where ideological positions are involved, such as religious rules (e.g. against the use of alcoholic beverages) or, on the other hand, questions of professional or cultural identity involving the use of substances.

6


Finally, a basic conflict concerns the economic and/or political interests which may be hurt by preventing substance use. Living conditions, lack of adequate opportunities, excessive working hours, living far from one’s own family etc. eventually increase the risks for substance abuse; any structural changes reducing such risks are bound to hurt some interests. There is a saying that "No drug prevention can be effective unless it hurts somewhere!". The only way to overcome this kind of conflicts and to avoid a passive evasion from conflict or, even worse, engaging in a kind of pseudo-prevention which hurts nowhere - the only way is to carefully document which goal and which strategy is helpful under what circumstances. The conflict proneness of drug prevention is another argument for developing a scientific basis ensuring an optimum of positive effects and a minimum of harm produced by prevention itself.

1.3 Evaluation of drug prevention in Europe and the need for concerted action Some efforts have been made in order to gain an overview over the state of the art in European countries. We may mention two collections of selected drug prevention programmes: the one by Zaccagnini et al (1993) includes 75 projects from 14 countries, the other by Negreiros (1994), initiated on behalf of the Council of Europe, describes 47 projects from 18 countries. Both overviews demonstrate the wide spectrum of preventive approaches, with a majority of activities focusing on drug information and drug education. Both overviews also include information on evaluation efforts. In many of the described projects, some form of evaluation was foreseen. Negreiros mentions some form of monitoring or process evaluation in 32 projects, an impact assessment in 19 projects, and a combination of both in 14 projects; no evaluation was foreseen in 3 projects only. However, there is very little information on details and especially on evaluation results, revealing a general scarcity of prevention evaluation knowledge. A lack of evaluation theory, methodology and practice, and also a lack of networking among experts engaging in prevention evaluation, became clearly visible. It also became evident that national efforts alone cannot be sufficient in this domain. Concepts and instruments for comparative research on an international level are indispensable. World Health Organisation made a first attempt to produce guidelines for assessing alcohol and drug prevention programmes (WHO, 1991a). 7


This invited an initiative to be taken in the framework of the COST social science action A6 "Evaluation of action against drug abuse in Europe". This project started in December 1992, expiring by end of 1997. It gained participation from 15 countries and developed working plans for separate working groups which dealt with prevention evaluation, treatment evaluation, evaluation of drug policies and of drug related delinquency; another working group was designed for developing research instruments and protocols to be used in evaluation research. Each working group has established a network of experts, including observers from the Groupe Pompidou, from World Health Organisation, from the European Commission (Uchtenhagen 1994, 1996). The working group on the evaluation of prevention, chaired by A. Springer of the Viennese Ludwig-Boltzmann-Institute for Addiction Research, decided at an early stage to focus on primary prevention. It initiated a comprehensive Delphi study conducted by A. Uhl from the LudwigBoltzmann-Institute for Addiction Research, organised workshops on specific topics and stimulated national efforts to improve prevention evaluation projects. It also invited country reports reviewing the state of the art in the participating countries. This book presents selected results of the working group’s activities. It is my privilege to express my gratitude to all those who contributed to making these activities a worthwhile exercise, and especially to the members of the working group and their chairman.

1.4 References Negreiros, J.: Drug misuse prevention projects in Europe. University of Porto (Portugal), Co-operation group to combat drug abuse and illicit trafficking in drugs (Pompidou Group). Council of Europe (P-PG (94) 25), 1994 Springer A, Uhl A: Primärprävention des Drogenmißbrauchs und ihre Evaluation. Erfassung der österreichischen Situation und Diskussion der Rahmenbedingungen in Europa und in den USA. Ludwig-Boltzmann-Institut für Suchtforschung, Wien, 1995 (Manuskript) Uchtenhagen A: Evaluation of action against drug abuse in Europe: a COST social science project. Eur Addict Res 1, 1-2: 68-70, 1994

8


Uchtenhagen A: COST A-6 Evaluation of action against Drug Abuse in Europe. Progress report. Eur Addict Res 2, 1 : 6-65, 1996 World Health Organisation: Programme on substance abuse. Guidelines for assessing alcohol and drug prevention programmes, WHO/PSA/91.4, 1991a World Health Organisation: International Classification of Diseases. Clinical descriptions and diagnostic guidelines. Geneva, 1991b Zaccagnini, J. L. et al.: Catalogo de programas de prevencion de la drogadiccion. Promolibro Valencia, 1993

contact address: Ambros Uchtenhagen ISF, Konradstraße 32 CH-8005, Zurich, Switzerland Tel: +41 1 2734024 FAX: +41 1 2734064 e-mail: [email protected]

9


2 Some remarks on research based evaluation (Katarzyna Okulicz-Kozaryn)

2.1 Reasons to do evaluation Concerning evaluation of primary prevention we may think about a number of various sets of scientific activities and reports. The term "evaluation" has various denotations. One, very general description of "evaluation" says that it is the process of seeing if we accomplished what we set out to achieve (Hawkins, Nederhood, 1987). This general meaning allows us to apply the term "evaluation" to different activities, serving different purposes and different groups of people. There are different reasons for doing evaluation (Moberg, 1984). The most obvious, and the first one coming to one’s mind is the assessment of a prevention programme’s effects. Many evaluation studies are conducted because programme developers, sponsors and field workers want to know if their programme really works as expected. Evaluation studies of this type are concentrated on the outcomes of a particular programme. Evaluation goals may be much more global and reach far beyond the impact of specific prevention programmes. The purpose may be to assess the value of theories behind prevention programmes. This kind of evaluation is a very important and interesting issue, particularly for scientists: Are they useful? Is their application leading to a better understanding of the drug use phenomenon? etc. These questions go further than questions about the effectiveness of a single intervention. Answers can be generalised to other prevention programmes. To obtain generalisable data several fundamental methodological criteria concerning study design and measurement instruments have to be met. Somewhere in between the evaluation of specific evaluation programmes and evaluation theory is the evaluation of prevention concepts. The central question is if a class of related programmes characterised by a similar approach is promising. In evaluating prevention approaches the quality of design has high priority as well. This kind of research can give answers about prevention strategies (What to do?), programme activities (How to do it?) and target population (For whom?). The main goal of evaluation concerning particular preventive strategies is to decide if this type of intervention is worth being disseminated, to decide if it should it be

10


recommended for implementation at other sites or for other groups of people. Answers to these questions are crucial for decision makers. Evaluation may also satisfy the needs of programme developers and staff. Especially when the main reason for doing evaluation is to improve a programme. Evaluation may serve to identify problem areas and to choose better strategies to deal with arising problems. Evaluation may also permit better planning of future activities. Through asking present programme participants about their needs and expectations we can collect very useful information. We may e.g. find out, what are the gasps in our service, what are the trends in our target population (i.e. changes in types of drugs widely used), how many participants we may expect in the next edition of our programme. The accomplishment of any one of the above goals would be impossible without careful description of programme activities: What has been done? For how many participants? Are the programme elements understandable and feasible? What was the feedback from our clients? This kind of evaluation is often named "monitoring" or "process evaluation" and it constitutes a fundamental basis for all other evaluation studies. The most unpleasant and even frightening purpose of evaluation for practitioners is that of accountability. Funding agencies want to know how their funds were used. How many clients were served? How much time did the staff spend on the programme? Collecting these kinds of data may be very boring for prevention practitioners. From their perspective, it may look as a waste of time because this kind of evaluation generally provides information of marginal use for programme improvement or assessment of programme results. On the other hand, this kind of activity may be very important from an economic point of view.

2.2 Some problems related to outcome evaluation The purpose of a specific evaluation project determines what kind of data need to be collected. If evaluation is considered to assess the effectiveness of a programme, a policy or a strategy, the most important data are data concerning the outcome and not data concerning the process. To collect credible outcome data in the field of primary prevention, an evaluator has to face a lot of theoretical and methodological problems.

11


2.2.1 Problems related to the study design Designs determine the quality of data gathered and may determine conclusions about programme effectiveness (Fitz-Gibon, Morris, 1987; Hawkins, Nederhood, 1987). Among researchers there are hardly any serious controversies about the fact that a true experimental design is basically superior to any other design. It allows to compare the effects on programme participants with the effects on people not having participated in prevention activities. But there are many situations where only quasi-experimental designs are realistic under real-life conditions. Even if experimental designs are considered in the first place, the researches are frequently confronted with serious interfering factors causing deviations from the experimental design and resulting in less then optimal data quality. A fundamental problem is associated with random assignment. Under everyday conditions it is not always possible to assign subjects randomly to experimental or control conditions (Boyd, 1994; Holder et al., 1995) Sometimes it is virtually impossible not to offer any preventive intervention to randomly selected individuals or cohorts, without causing them to seek for alternative sources. Sometimes the target population is so small that there are not enough cases to assign randomly to experimental or control conditions. Limitations in the assignment of schools or local communities to different conditions are also well known.

2.2.2 Problems related to outcome variables Primary prevention is defined as interventions to prevent the onset of a substance use problem. Consequently the target population for primary prevention consists of persons - usually children and adolescents - who haven’t started to use drugs or who do not use them in a harmful way yet. Evaluation of primary prevention means that we are trying to assess changes in a dimension (drug use or problems related to drug use) that are not yet present and may or may not develop in the future. In dealing with the evaluation of a drug prevention programme for primary school students we should be aware that expected behavioural changes resulting from the programme may not be measurable for several years. Waiting for such a long time for the results of an evaluation is usually not acceptable to people engaged in prevention and/or evaluation. Funding agencies want to know if their money was invested well, the programme staff is interested in the results of their work, programme developers are waiting for a

12


confirmation of their background theories, and evaluators are interested in publishing their research findings. As a matter of fact evaluation conclusions are often based on short-term results, measured just after the end of a preventive intervention. By definition primary prevention focuses on behaviour of low prevalence in most target populations. Therefore it is unrealistic to expect significant changes in behaviour in a short time period. If we do not have time to wait for future behavioural changes, the only realistic alternative is to use "intermediate variables" known to relate to relevant behavioural changes in the future. In this context it is assumed that the presence or intensity of some features of individuals plays a significant role in amount and frequency of drug use. The concept of risk and protective factors is presently predominant in primary prevention. Modern prevention programmes are frequently based on studies concerning the relationship between drug use and individual factors (i.e. knowledge, attitudes, personality), family factors (i.e. family history of drug use, parental norms on drugs, parent-child relationship), or peer influence. But the relationship between various factors and drug use still remains uncertain. Results of studies concerning risk and protective factors, literature overviews and meta-analyses are not always convergent (Hansen, Rose, Dryfoos, 1993; Botvin, 1990; Howkins, Catalano & Morris, 1992; Morgan, 1998). Some approaches seem to be more promising though than others. As long as the prevalence of drug use is still increasing people trying to cope with alcohol and drug problems cannot afford the luxury to refrain from actions and passively wait until researchers formulate final statements about the most effective prevention strategies (Holder et al., 1995). Prevention programmes are often based on popular opinions and guesses without clearly formulated direct and indirect (mediating) goals. For some programme developers and decision makers (as well as for some evaluators!) explicitly stated programme goals do not have to be congruent with the dependent variables measured in the study and many conclusions about programme effectiveness are formulated on the basis of surrogate (intervening) variable measures (Dielman, 1995). E.g. a claim to prove effectiveness of a drug prevention programme may be derived of participants’ knowledge or attitudes towards drugs.

2.2.3 Other problems Another fundamental problem commonly found in evaluation is that evaluators are often engaged too late (Fitz-Gibon, Morris, 1987). 13


Programme developers usually concentrate on prevention activities while they conceptualise their programmes and any serious planning of evaluation is beyond the scope of their interest in this phase. They do not begin to think about evaluation before their programme has been finalised. Evaluation starts to be an issue after the newly developed programme has already been implemented or is going to be implemented soon. Whenever evaluators are engaged too late, they have to work under time pressure and they do not have enough sufficient time to plan their evaluation project adequately. Sometimes there is not even time to do any baseline measurements. This lack of time may impede the quality of measurement instruments to assess the overall programme goals as well as other programme objectives (related to the intermediate steps along a continuum of change described in the theoretical model of the programme. This could i.e. be "improvement of life skills", "knowledge", "attitudes towards drugs", etc.). Still another fundamental problem is related to financial limitations. In the field of drug prevention it is much easier to obtain funds for interventions than for research on intervention effectiveness. As a result, many interventions are being implemented without any evaluation at all or with insufficient evaluations produced by the fact that evaluators have to limit their research designs according to available sources (Schaps et al., 1981; Tobler, 1986; Ferrer et al., 1995; Springer, 1998). Financial limits often influence sample size and/or study design. If there are insufficient funds available it may e.g. not be possible to include a reference group. A scientific proof of effectiveness may be difficult for other reasons too. There are few interventions in the field of drug prevention that their authors describe as stable and completed (Fitz-Gibon, Morris, 1987). Each programme implementation may bring to light new problem areas and possibilities how to improve an intervention. Commonly founders believe that programmes that have already been implemented once are principally perfect and they expect summative statements about programme effectiveness. This may lead to situations, where programmes that still require further modifications are prematurely subjected to outcome evaluation, even though further formative evaluation should be planned to improve the programme before any summative evaluation should be considered (Uhl, 1998). A lack of standardisation in programme implementation may also influence evaluation results. Whenever people delivering the programme (programme multipliers) are not well prepared, or are not motivated to keep the quality of programme implementation, or if they are freely 14


changing programme content, it may deteriorate the programme’s impact (Hansen et al.; 1991).

2.3 The need for co-operation External evaluators of primary prevention programmes usually have been trained to understand various theoretical, methodological and practical research problems. Consequently they are primarily interested in research issues while sponsors, decision makers, programme developers and staff members are primarily interested in content; i.e. in delivering prevention services. Because of this conflict in background and orientation, it is very important for the scientific process, to consider who has control over whom in this process: the external evaluators or the others (Boyd, 1994; Holder et al., 1995) When prevention interventions are initiated by researchers in order to evaluate the intervention, the probability that methodological problems and theoretical problems can be avoided by applying an advanced design is greatly increased. Keeping certain methodological standards allows the attainment of ambitious evaluation goals. Through this kind of study it is possible to test prevention theory and the effectiveness of prevention approaches. Under these circumstances it is also feasible to collect data on long-term effects. Interventions controlled by programme developers or decision makers are generally much more challenging for evaluators. The evaluators have to deal with situations created by others and to compromise between methodological standards and real life conditions. In this situation it is much more difficult to obtain credible results since experimental or even quasi-experimental designs are often not applicable. On the other hand though, studies of a naturally occurring and programme-driven interventions may significantly improve our understanding of naturally occurring prevention process since the interventions are conducted under real life conditions and not artificial ones. These findings therefore may more easily be generalised to larger populations.

15


Presently the majority of drug prevention programmes are developed and implemented outside of scientific centres. Unfortunately there is a considerable lack of established and accepted standards for the evaluation of "naturally occurring" projects. It would be useful to have standards addressing the special characteristic of programme-driven evaluation.

2.4 References Botvin, G.: Substance Abuse Prevention Theory, Practice, and Effectiveness. In: Tonry, M.; Wilson, J. Q. (Eds.): Drugs and Crime: Annual review of research in crime justice. Vol. 13, 461-519, University of Chicago Press, Chicago, 1990 Boyd, G. M.: Methodology of Evaluative Studies. Paper presented at the Workshop on Evaluative Studies on Alcohol Prevention Programs, Warsaw, 1994 Dielman, T. E.: School-Based Research on the Prevention of Adolescent Alcohol Use and Misuse: Methodological Issues and Advances. In: Boyd, G. M.; Howard, J.; Zucker, R. A. (Eds.): Alcohol Problems Among Adolescents: Current Directions in Prevention Research. Lawrence Erlbaum Associates Publishers, Hillsdale, N.J. Hove, 1995 Ferrer, X.; Duran, A.; Larriba, J.; Spieldenner; J.: Catalogue of Educational Materials on Drugs. Selected from the European Union and Central and Eastern European Countries, Phare Project on Drug Demand Reduction, 1995. Fitz-Gibon, C. T.; Morris, L. L.: How to Design a Program Evaluation. Sage Publications, Inc., Newbury Park, 1987 Hansen, W. B.; Graham, J. W.; Wolkenstein, B. H.; Rohrbach, L. A.: Program Integrity as a Moderator of Prevention Program Effectiveness: Results for Fifth-Grade Students in the Adolescent Alcohol Prevention Trial, Journal of Studies on Alcohol, 52, 6, 1991 Hansen, W. B.; Rose, L. A.; Dryfoos, J. G.: Causal Factors, Interventions and Policy Considerations in School based Substance Abuse Prevention. Report submitted to the Office of Technology Assessment United States Congress, Washington DC, 1993 Hawkins, J. D.; Catalano, R. F.; Miller, J. Y.: Risk and Protective Factors for Alcohol and Other Drug Problems in Adolescence and Early Adulthood: Implications for Substance Abuse Prevention. Psychological Bulletin, 112, 1, 64-105, 1992 16


Holder, H.; Boyd, G.; Howard, J.; Fly, B.; Voas, R.; Grossman, M.: Alcohol-Problem Prevention Research Policy: The Need for a Phases Research Model. J. Pub. Health Pol. 16, 3, 324-346, 1995 Moberg, D. P.: Evaluation of Prevention Programs: A Basic Guide for Practitioners. Wisconsin Clearinghouse, Madison, 1984 Morgan, M.: Evaluations of Substance Use Prevention Programmes Implications for Illicit Drugs. In this publication, 1998 Schaps, E.; DiBartolo, R.; Moskowitz, J.; Palley, C. S., Churgin, S.: A Review of 127 Drug Abuse Prevention Program Evaluation. J. Drug Issues, 1981 Springer, A.: Country Reports: An Overview, Including Some Remarks about Socio-Cultural Determinants of Primary Prevention and its Evaluation. In this publication, 1998 Tobler, N. S.: Meta-Analysis of 143 Adolescent Drug Prevention Programs: Quantitative Outcome Results of Program Participants Compared to a Control or Comparison Group. Journal of Drug Issues, 16, 4, 537-567, 1986 Uhl, A.: Evaluation of Primary Prevention in the Field of Illicit Drugs Definitions - Concepts - Problems. In this publication, 1998 Wilson, J. Q. (Ed.): Drugs and Crime: Annual Review of Research in Crime and Justice, 13, 461-519, University of Chicago Press, Chicago, 1990

contact address: Katarzyna Okulicz-Kozaryn Institute of Psychiatry and Neurology 1/9 Sobieskiego, 02-957, Warsaw, Poland Tel: +48 22 422650 FAX: +48 22 6425375

17


18


Country Reports: An Overview, Including Some Remarks about Socio-Cultural Determinants of Primary Prevention and its Evaluation Alfred Springer LBISucht, Vienna, Austria

___________________________________

Results of an International Study within the COST-A6 Action of the European Union

19

Country Reports

Contents 1 Methodology............................................................................................ 21 2 The instrument ......................................................................................... 21 3 Results...................................................................................................... 26 3.1 The country reports........................................................................... 26 3.1.1 Austria ......................................................................................... 27 3.1.2 Belgium ....................................................................................... 28 3.1.3 Czech Republic ........................................................................... 28 3.1.4 Finland......................................................................................... 29 3.1.5 France .......................................................................................... 29 3.1.6 Germany ...................................................................................... 30 3.1.7 Greece.......................................................................................... 31 3.1.8 Ireland.......................................................................................... 32 3.1.9 Italy.............................................................................................. 32 3.1.10 The Netherlands ........................................................................ 32 3.1.11 Poland........................................................................................ 34 3.1.12 Spain.......................................................................................... 35 4 Summary .................................................................................................. 37 4.1 Typology of programmes.................................................................. 37 4.2 Aims of programmes ........................................................................ 37 4.3 The background of the problems of evaluation; socio-cultural reconsideration................................................................................ 38 4.3.1 The problem of outcome evaluation ........................................... 38 4.4 Excursus: Major trends in prevention in the USA and in Europe. A comparison .................................................................... 40 4.4.1 Prevention philosophy................................................................. 40 4.4.2 Primary prevention as a component of drug policies ................. 41 4.4.2.1 The situation in the USA ...................................................... 42 4.4.2.2 European attitudes................................................................. 44 4.4.3 Impact on evaluation issues ........................................................ 46 5 Appendix - three examples of original country reports as illustration ... 49 5.1 Report of Poland ............................................................................... 49 5.2 Report of Greece............................................................................... 53 5.3 Report of France ............................................................................... 60 6 References................................................................................................ 63

20


1 Methodology Our objective was to gain information about ongoing primary prevention and the state of its evaluation throughout Europe. To fulfil this aim country reports from as many European countries as possible were collected. Experts were recruited through official channels and through snowballing. On the one hand we contacted the officials concerned in the different countries and asked them to nominate experts known to be competent and prepared to undertake the venture. Additionally we used our own knowledge and expertise to identify eminent experts in the field. The experts were asked to identify those programmes in their countries that had been implemented more or less properly in a scientifically proper way and evaluated in some way. To arrive at comparable answers we developed a questionnaire, targeting important issues concerning the research field and also included questions concerning the experts’ estimation of the programmes and projects.

2 The instrument A

General data on projects

A1 Project title: A2 Key-persons / key-institutions A3 What state is the project currently in? • already finished? • ongoing? • in a planning stage? • I don’t know and couldn’t find out? A4 Have there been any publications, official presentations or grey literature yet regarding the project? • yes no ? • If yes, please give a full citation of all related papers or articles and if • possible, please include a photocopy of the article or articles. • If the material is not published in an international journal, please give the name and address of at least one author:

21

Country Reports

B

Data relating to the primary prevention programme

B1 Abstract goals concerning illicit drugs, such as • less consumption • later onset of consumption • safer use • harm reduction in terms of health problems • harm reduction in terms of social problems • harm reduction in terms of public disorder • harm reduction in terms of immoral conduct • no goals specified explicitly • etc. describe in detail: ...................................................................................

B2 Details on programme structure, such as • Is there a written concept • How many sessions were planned • Who is carrying out the programme • How is the staff trained to do the job • How much time is used for staff training • etc. describe in detail: ...................................................................................

B3 Type of programme in terms of • target population (e.g. school based, community based, mass media campaign, training of educators, teachers, physicians, policemen, parents etc., change of law or policy etc.) and • size of target population (e.g. nation wide media campaign, one school including 12 classes, etc.). describe in detail: ...................................................................................

22


B4 Techniques used, such as • general health education • changing lifestyles • changing attitudes • social skills training • deterrent information • factual information • offer of alternatives to drug taking • clubbing • modification of drug laws (e.g. decriminalisation of mere consumers, compulsory treatment of addicts, more severe legal sanctions for consumers etc.) • modification of drug policy (e.g. offer of syringe exchange, offering low level intervention, etc.) • change of police strategy (e.g. tolerate scene, focus on large scale dealers, etc.) • change of treatment policy (e.g. making methadone maintenance available) • etc. describe in detail: ...................................................................................

B5 Drugs mentioned in programme, such as • no drugs mentioned at all • only illicit drugs mentioned • only medical drugs mentioned • both legal and illicit drugs mentioned • only specific drugs mentioned (e.g. cannabis, heroin, cocaine) • etc. describe in detail: ...................................................................................

23

Country Reports

C

Data relating to evaluation

C1 What targets were/will be evaluated? • operational objectives (internal targets), like "Did the information reach the target population?"; "Was the programme concept carried out as planned?", etc. • intermediate targets (surrogate targets) associated to the final goals but not primary goals by themselves, like e.g. attitude change, increased self-esteem, higher information level, etc. • ultimate targets (primary targets) like e.g. less drug consumption, later onset of drug consumption, harm reduction, less problematic use, etc. describe in detail: ................................................................................... C2 How was the study sample defined? describe in detail: ................................................................................... C3 Sample size and drop-out rate? describe in detail: ................................................................................... C4 How often were data collected and at what time schedule? describe in detail: ................................................................................... C5 How were the data collected, resp. what data were used? • surveys / population surveys • telephone interview • face-to-face interview • questionnaire presented by investigator • mailed out questionnaire • public statistical data • police records • treatment records • other describe in detail: ................................................................................... C6 Was the programme successful and if yes, in which dimension(s)? describe in detail: ................................................................................... C7 Were there specific problems worth explicitly mentioning? describe in detail: ...................................................................................

24


D

Global part

D1 Personal summary on the different projects mentioned Please give a generalising overview out of your personal impressions over the state of evaluation on primary prevention research regarding illicit drugs in your country.

D2 Conclusions: What conclusions do you personally draw out of the present state of evaluation in your country and how do you feel about prevention and evaluation of prevention in this field generally • Which prevention concepts do you think are promising and which concepts are worthless? • How do you define the limitations of evaluation in this field? • etc.

D3 Is there anything like an official national prevention concept in your country? If yes, • • • •

what is the concept like? is it available as a written document? is there a gap to implementation? are there any relevant groups openly in opposition to the concept and who are they?

D4 Please indicate names and addresses of other researchers and institutions in your country that are active in the field of prevention evaluation?

25

Country Reports

3 Results 3.1 The country reports Finally experts from the following countries agreed to co-operate in the action: • Austria: Alfred Springer • Belgium: Annick Vandendriessche • Czech Republic: Official statement; no defined author • Finland: Osmo Kontula • France: Chafic Saliba • Germany: Christoph Kröger • Greece: Alice Mostriou • Ireland: Mark Morgan • Italy: Alberto Tinarelli • Poland: Katarzyna Okulicz-Kozaryn • Spain: Maria Xesús Froján Parga • The Netherlands: Han Kuipers The reports we received differed widely concerning content and punctuality. Some of the country representatives reported on each single project they were able to identify and proved worthy to be documented, others only described the projects globally and gave a summary on the overall situation of prevention and evaluation in their countries. Therefore the results of our initiative cannot in all cases be presented according to the outline we developed, but they are more or less a summary of the country reports we received. The summaries presented here are partially composed using original wording of the country’s experts. In some cases we have completed the reports using some information from official documents. We will nevertheless use the outline as a guideline for our interpretation.

26


3.1.1 Austria During the 90ies many preventive actions were started in different Austrian provinces. Prevention was officially declared to be of central importance. The leading background philosophy of primary prevention in Austria is to include drug abuse prevention in the more general field of health education and to develop a prevention concept that focuses on the prevention of "addiction" and not of "drug use". The concept addresses primarily "unspecific" social and psychological aspects. Drug specific issues are only touched marginally. The abstract goals concerning illicit drug use are basically abstinence oriented but at the same time nevertheless heavily influenced by concepts of harm reduction. The message given out is predominantly not a "no drugs"- message but a "safe use" message. In Austria different types of projects and programmes are in action. They are ranging from isolated information campaigns to more structured programmes on community level. Most of the activities are carried out in the context of school education and are seen as a central component of the health education concept. It is therefore typical that the first larger effort, done in a structured way, was the preparation of educational material for adolescents aged 13 - 18 years as a common project of the Ministry of Health, the Ministry of Education and experts from the education sciences. This project has been evaluated. A recent promising development is peer group oriented projects. They commonly include evaluation components. One of these projects will start a co-operation on the European level within the "Europeer" research project. All together we may conclude that the evaluation has not developed sufficiently until now. Very few preventive efforts have been covered by proper process evaluation and outcome evaluation is almost totally missing. In cases where some kind of evaluation had been included from the very beginning of a project -as in the case of the educational material mentioned above - some major difficulties have disqualified the evaluation process. Concerning outcome evaluation the very strict personality centred approach of "addiction prevention" represents a main obstacle.

27

Country Reports

3.1.2 Belgium According to the Belgian report, primary prevention in this country is oriented towards health promotion and health education. Concerning its theoretical reference frame mainly the social learning approach is used. Apart from this there is no national prevention concept, since there seems to be a certain degree of conflict prevailing between the levels of the political system (regional and federal) and within the regional departments concerning central issues like "harm reduction". Out of the prevention programmes which are implemented in the French and the Flemish parts of Belgium two projects are described in depth and discussed. Both belong to the school based drug education approach: • "Skills for Adolescence" and • "QUEST" The analysis of the material leads to the conclusion that in Belgium only few programmes are actually evaluated. This trend is changing though, since VIG, the umbrella organisation for health promotion clearly stated that proper evaluation is necessary in order to get projects funded. This organisation even offers training in e.g. how to deal with the problem of external evaluation.

3.1.3 Czech Republic In the Czech Republic primary prevention is seen as a major challenge nowadays. Priority is given to preventive efforts towards children and youngsters through school education, family, peers and role-makers among music, sports and other popular personalities. It is a field of interdisciplinary co-operation. The National Centre for Health Promotion, a national agency run by the Ministry of Health in co-operation with the Ministry of Education, focuses on methodology and development of model programmes. Co-ordination is being provided by the National Drug Commission. There is also a notable activity of NGOs. The NGO sector has been involved in prevention programmes for schools, in projects of "teaching the teachers", in "peer education programmes", in the production of printed materials / leaflets, in the organisation of exhibitions and in sponsoring concerts of popular music as well. In the field of primary prevention the Czech Republic has been involved in several international projects within PHARE, Group Pompidou, WHO. No information concerning evaluation of all these efforts has been provided in the report.

28


3.1.4 Finland The problem of illicit drug use in this country is comparatively small sized. Therefore only a small number of prevention programmes has started yet. The programmes carried out have been small-scale and almost regularly programme evaluation has been insufficient. Other typical features of Finnish drug prevention are a relatively large amount of teaching materials produced for prevention purposes and the implementation of a large sector of prevention by voluntary organisations and citizens. Totally missing is an evaluation of efficacy and impact. Most of the Finnish primary prevention programmes seem to aim at practical results in the subjects. Evaluation has typically played a minor part in any programmes and is not considered an objective in itself. In other words, programmes aiming at developing better future programmes are rare exceptions. Five projects that have been evaluated are described and discussed in the report. One of them was the empirical basis for a thesis on special education to get a university degree and this project used a true experimental design. Based on the report we can conclude that the quality of evaluation was not sufficient in all other programmes.

3.1.5 France Chafic Saliba, the French expert described and discussed many projects. He covered a wide range of preventive activities using a variety of different approaches. In his report a total of 170 activities / materials were reported. Out of this bulk of material Dr. Saliba selected and discussed 9 highly divergently evaluated projects, using very different designs. Among them are programmes for school-based drug education, mass media campaigns and video clips and their evaluation, the alternatives (sports against drugs), interactive theatre (role-acting) in the sense of DARE. The report mentioned that at a whole 170 prevention activities could be identified and that again 170 educational tools for the implementation of the programmes had been produced. Primary prevention is seen as a high profile venture and receives support from varied sources. Evaluation is estimated to be highly important too. The reality of prevention though presents a very different picture. The high intentions do not result in adequate actions.

29

Country Reports

According to Mr. Saliba two main causes are to blame for that situation. The first problem is the nowadays prevalent interpretation of a causal relationship concerning drug abuse, which could be named "fundamentalist". In France like in other western European countries the main interest in prevention background theories is directed at the latent causes of addiction which remain hidden in the individual’s psychic structure and are very difficult to detect and therefore also very difficult to influence. Dr. Saliba is very critical concerning this interpretation. He thinks that this frame of reference impedes the prevention programmes on all levels of implementation. These problems start already with the programmes’ conceptualisation and are also influencing the evaluation of the projects. The second problem is closely linked to the first one. It consists of lack of clarification of the aims of the intervention and of the outcome criteria. They are very vague and complex. This contributes to the difficulties concerning outcome evaluation and fosters an evaluation approach which relies nearly exclusively on the evaluation of the number of persons reached and of the implementation quality. The quality of implementation is usually evaluated using again very vague concepts and subjective categories like "satisfaction/dissatisfaction". The interpretation of programme results is very difficult because of serious methodological problems. It is nearly impossible to draw valid conclusions and generalisations out of them. This is also due to major changes in the programme during the time of its implementation.

3.1.6 Germany In this country many actions and programmes are carried out in the field of primary prevention of drug use. Few of them have been evaluated though. Evaluation is primarily done in projects sponsored by the German Federal Government. These projects are major mass-media campaigns and some "model type projects". The evaluation of these campaigns and projects is done professionally and financed by the Government and the Federal Centre for Health Education (Bundeszentrale für gesundheitliche Aufklärung). Usually, universities are neither involved in primary prevention nor in evaluation nor in related research activities. Several regional groups that do primary prevention evaluate some of their work, but in most cases this evaluation does not have high priority in the projects. Often the results are not even published because they do not meet minimum scientific research standards. There are some information systems available that distribute information on on-going and past prevention projects in the field of 30


primary prevention. Most projects do not have money or time for extensive evaluation. Most evaluation work concentrates on projects with the objective of training mediators in the prevention field. Outcome evaluation is interpreted as a very difficult task to implement. In the German report 12 prevention activities are described and discussed following our outline: • Bundesmodellprogramm Mobile Drogenprävention (DHS, Hamm) • Forschungsvorhaben auf dem Gebiet "Biologische und psychosoziale Faktoren von Drogenmißbrauch und Drogenabhängigkeit" (IFT, München) • Prevention of substance abuse in schools by means of life skills training (IFT, München) • Der Rausch des Lebens; Plakatserie (Aktion Jugendschutz, Bayern) • Sag doch was (mobile Drogenprävention Bayern, Fachhochschule Würzburg) • Seminar: Auszubildende und Drogenkonsum (Fachstelle für Prävention; Bottrop) • Kinder stark machen - eine Anzeigenkampagne (Bundeszentrale für gesundheitliche Aufklärung, Bonn) • Sunrise (Zentralstelle für Suchtvorbeugung, Kiel) • Das nordrhein-westfälische Programm zur Sucht- und Drogenprävention (Universität Bielefeld) • Train the Trainer-programme (Zentralstelle für Suchtvorbeugung, Kiel) • Echter Rausch kommt von innen (Zentralstelle für Suchtvorbeugung & Bund für drogenfreie Erziehung) • Spielzeugfreier Kindergarten (Aktion Jugendschutz, Landesarbeitsstelle Bayern e.V.)

3.1.7 Greece In Greece many rather unstructured preventive actions are going on. None of them seem to have been evaluated. In the academic field the leading philosophy is to include primary prevention of drug use and abuse into the overall health education in schools. A programme with the aim of "Education for Health to prevent dependence and addiction" has been done 31

Country Reports

in Athens. In that case process evaluation as well as outcome evaluation on selected variables have been carried out.

3.1.8 Ireland A substance abuse prevention programme to be used in schools was developed by the Departments of Health and Education in Ireland during 1990. This effort has been evaluated. This evaluation involved a quasi-experimental design in the sense of a comparison of pilot schools and control schools with a stringent statistical and quantitative strategy. The programme itself used several approaches to prevent substance abuse based on some conceptions of the influence that underpins initiation to various substances and did not rely on any single model of intervention. The central concept is that social skills and understanding may be critical in assisting young people to make independent decisions on these matters. The evaluation included process variables and outcome variables. It indicated that the programme was implemented in accordance with its aims and objectives in the classrooms and that the programme had positive effects on attitudes and beliefs regarding substance use, compared to a matched control group. Social behaviour and beliefs that were targeted in the programme were shown to be strongly related to reported substance use. Since effects on the organisational level were not examined the eventual impact of the implementation of the programme could not be defined.

3.1.9 Italy Many preventive efforts have been started, but no systematic evaluation has taken place. On the other hand as a result of some cultural changes and a shortage of money evaluation has become an increasing issue.

3.1.10 The Netherlands In the Netherlands primary as well secondary prevention-activities play an important role. Primary prevention is almost always practised in schools. Secondary prevention takes place at the location of drug (ab)use directly. The latter approach also includes (secondary) schools, but not to a large extent.

32


Since 1980 there has been a steady increase in primary prevention activities and even more increase in secondary prevention (special activities for risk groups). Actually the basic philosophy of the Dutch prevention policy follows the concepts of health education and of harm reduction. The following principals are typical of Dutch education programmes, based on research and practice: • Emphasis on individually responsible choice regarding risky substances and risky habits. • Focus on background and reasons for use and abuse (in opposition to an emphasis on information about substances) • Less emphasis on dangers of using substances (horror stories) because this might lead to more experimenting with drugs. • Educational messages on the use of hard drugs should promote "abstinence". • Educational messages on the use of cannabis should promote "caution"; it is not experimenting but excessive (ab)use that should be prevented. • Educational messages on drugs should be "low profile". The messages should function as "behavioural advice", as part of a comprehensive approach of alcohol and drug education programmes or general health education curricula in schools and training courses. This broader approach prevails in many current programmes. Concerning the theoretical background of the Dutch prevention system the most recent activities in the field of primary prevention of illicit drug use are based on a theoretical framework derived from social psychology backed up by empirical evidence from research on the effects of health education programmes. The most common strategy of discouraging people from using drugs is through information, education and health promotion. But also other approaches to improve the living conditions of people in poor social and economic positions are used. In that respect the Dutch have developed the concept of "social innovation", a policy to offer socially deprived groups new opportunities for (re)integration into society (education, housing, employment schemes).

33

Country Reports

An interesting aspect of Dutch prevention work is also that interventions are set on locations in meeting points used by members of youth culture group. Furthermore an important aspect of the Dutch policy is that drug prevention has developed from "spontaneous education" (aimed at the public in general) to "planned programmatic" prevention, based on research and analysis of specific target groups. In that sense preventive thinking has gradually achieved a more professional level. The implication of these changes is that prevention programmes are only warranted if they meet certain standards of analysis, pre-testing, planning and evaluation. Nevertheless research on the effects of prevention activities appears to be very scarce. Only a few examples of research on primary prevention activities fitting to the definition can be found. Some of them were conducted in the mid-seventies and early eighties; only one very recently. A typical overview of selected drug prevention programmes written for the occasion of the Drug Prevention Week 1994 listed 11 programmes. These activities differed in strategy and methodology. In eight out of the eleven programmes no formal evaluation had been done. Outcome evaluation had been conducted in one case only. For another study outcome evaluation has been planned.

3.1.11 Poland Modern trends in drug prevention in Poland can be identified starting from the mid-80ies on. Now there are more and more programmes, aiming at a wide range of reasons of drug use and designed to actively involve participants. With the growing market of evaluation programmes the need for evaluation research also has increased. This has led to a major change in the quality of primary prevention in Poland. While some years ago most intervention studies were based on the simplest design (post testing only), nowadays often quasi-experimental designs are implemented. One of the current problems in that context is the lack of ability to adapt evaluation questions more adequately to the contents or goals of the programme. There is also a shortage of scientific instruments. The most promising development in drug prevention in Poland seems to be the community approach. The evaluation of that promising approach is very difficult though. Nevertheless there are several community based projects going on and their results will be known in the near future. In the Polish report five projects, which are actually implemented in Poland, are described and discussed focusing on evaluation problems.

34


3.1.12 Spain During the last fifteen years there has been a boom in the idea of prevention within the field of drug dependency. The failure of repressive and welfare solutions to the drug problem and above all the evolution of its social perception have been major causes of this boom. In general the target of prevention programmes is focused on the preconditions for drug addiction, of which it would be useful to have both conceptual and working definitions (level of self-control, social skills, potential risks, educational achievement, etc.) and a clear idea of the aims of preventive intervention in these variables. In Spain only few of the prevention programmes have been evaluated - and there are even fewer that include a description, how the evaluation was carried out. That leads to the conclusion that in spite of many prevention efforts carried out in Spain one cannot conclude whether the results are positive or negative since in most cases nothing is known about the actual results. It seems that a certain chaos reigns in the field of drug prevention programmes, undoubtedly partially due to a lack of knowledge in evaluators regarding evaluation designs. Most of the preventive interventions take place in schools. The target populations are mainly school children and adolescents of school age and to a lesser extent school teachers, who are trained to act as preventive agents (mediators). A systematic review of all the prevention programmes that could be identified showed the following facts: • Less than half the programmes reviewed included data concerning evaluation. This evaluation was regularly limited to simple documentation of the number of participation and attendance of the distinct groups the programme was directed to. • Questionnaires are the most utilised instruments for programme evaluation. The majority of them has been developed specifically for the actual programme, although the efficacy of the programme is also evaluated by the number of attendants or participants and by the number of courses and seminars carried out. • The number of programmes which include evaluation components is minimal. Some of them inform that they are using or will use follow-up forms, questionnaires, surveys, but provide no specific information concerning type and scope of the planned evaluation.

35

Country Reports

• Most of the programmes included no evaluation of basic objectives and there are several examples where evaluation was done in a quite superficial and vague manner. That may be due to the fact that the majority of the programmes presented had very global, unspecific or confusing objectives. • As a general rule, there was a lack of operative expertise, which made it impossible to establish clear hypotheses and to define specific variables. Because of this it is commonly hard to judge, if the initial objectives had been achieved or not. • Most often structural objectives were evaluated, especially relating to the level of participation in courses and/or seminars but without specifying whether that participation went hand in hand with learning of their contents or if this had reached all the target populations. • Almost none of the programmes informed whether the introduction was carried out in accordance with the initial proposal or, in case not, what the differences were and what caused them. • None of the programmes included an evaluation of costs or of the cost/benefit relation. • Intermediate objectives were usually not evaluated. Only 8% out of the reviewed projects contained information concerning the evaluation of such variables and even they provided no information concerning the planned methodology. • Nor have final objectives been evaluated. • As for the results, many of the programmes do not provide any kind of information. It is relatively frequent for the people in charge of the programme to name the programme efficient simply on the basis of participation aspects without any reflections on changes or on impact variables.

36


4 Summary We received detailed information about 43 programmes. According to the descriptions we received the following overview over the programmes concerning structure of the programmes, type and aims is possible.

4.1 Typology of programmes Tab. 1: Type of Programmes; Target Groups kindergarten-programme

1

school based drug education

18

mass media campaigns; video clips

3

development of educational material

3

training of trainers

8

community action

2

youth culture and risk groups intervention

3

peer leader education

1

role-playing; interactive theatre

2

commercials; posters

2

4.2 Aims of programmes Tab. 2: Drug Specific Aims information

14

drug abuse resistance training

7

total abstinence message

--

reduction of incidence

2

less consumption

16

later onset of drug experimentation

4

harm reduction; safer use

7

37

Country Reports

Tab. 3: Non Drug Specific Aims skill training

15

attitude change

13

alternatives to drugs

5

health education

14

Tab. 4: Type of Drugs Targeted legal and illegal drugs

14

illegal drugs only

8

no specific drugs

1

no drugs at all

2

The evaluation most frequently done was restricted to operational objectives (assessment of the quality of the programme and its implementation). Outcome evaluation focusing on attitudinal changes also took place in some cases. The evaluation of programme efficacy concerning drug taking behaviour continues to be an unsolved problem. Such efforts took place in a very limited number of programmes. Impact evaluation seems to be missing totally. As it seems research is nowhere done on the relationship between the boom in prevention efforts and the development of drug use and the changing patterns of consumption in the communities during the 90ies.

4.3 The background of the problems of evaluation; socio-cultural reconsideration 4.3.1 The problem of outcome evaluation Ideally any prevention programme’s objective has to influence behaviour. Drug abuse prevention therefore has to aim at behavioural issues of drug 38


use. We have labelled them "primary aims" in our outline. Due to methodological reasons on different levels most of these primary aims/specific outcome variables are very difficult to assess, some of them cannot be assessed at all under regular research conditions. This complicated situation of the outcome oriented evaluation of primary prevention has led to a search for alternative methods and strategies of evaluation which might be more easily implemented. The mainly used alternatives can be found in two directions. • the definition and assessment of intermediate (behavioural) variables. • Many authors now propose the process oriented or management-focused approach as a proper way out. In the USA where the methodology of the process-focused way has been stressed, the effectiveness of prevention in itself is in a certain way always determined in regard to outcomes. In that context the power and the impact of the national drug policy in the 80ies for instance was documented in figures of consumption. The results of the "War On Drugs" strategy were interpreted as encouraging since in the early 80ies promising changes in drug taking behaviour were observable. That means that researchers and politicians in the United States of America did not do without outcome evaluation while promoting the processoriented approach, but that they used figures drawn from epidemiological data collections like national consumption - household-studies as frame of reference. It could be argued if this is a scientifically proper way. Of course, I think that some criticism of the approach described above suggests itself. The relationship might be virtual; and of course one cannot trust the consumption figures. In the political climate of the war on drugs a strong tendency to under-reporting had to be expected, while in the more liberal situation of the seventies perhaps over-reporting was the prevailing bias. The decrease in consumption as documented in household figures might therefore be a product of the policy changes, but not in the sense it has been interpreted but in the sense of a reduction of the trustworthiness of the results. But the question remains, how we should cope in Europe with the problems arising out of the avoidance of outcome evaluation as related to changes in drug taking behaviour and or addictive behaviour.

39

Country Reports

In that context it seems necessary to point out where the emphasis on process evaluation started and to reflect on differences between the prevention systems in the USA and in Europe to come to conclusions concerning the transferability of certain approaches from one cultural frame to the other. As a baseline for such considerations it is also of interest to outline common trends in European prevention approaches. One the one hand there is no such a system as a common European prevention philosophy or common strategy. Our study on the country reports indicates that there are many approaches in use not only on the international level but also in the different countries themselves. But at the same time some features are observable which allow to describe European prevention as distinct from the American way. It is clear enough that most concepts used in the European Union have been developed in the United States. But how they are used, under which conditions and with which objectives depends on the general drug policies in the different states. These policies also define the general philosophy of the aims of primary prevention. The differences can easily be illustrated by using two very distinct systems like the drug policies in the USA and the Netherlands. It is well known that in the US a highly repressive attitude was the prevailing one from the mid-80ies into the early nineties, usually called "War on Drugs". On the other side, the Netherlands have always been interpreted as an example of a highly individualised and liberal approach to drug consumption as well as concerning the strategies they use to keep drug related problems under control (Engelsman, 1990). These different philosophies have an impact on the prevention approaches. This impact can be illustrated by the following comparison. To develop that comparison official documents containing declarations about drug policies have been used.

4.4 Excursus: Major trends in prevention in the USA and in Europe. A comparison 4.4.1 Prevention philosophy Drug prevention US-style: "War on Drugs" background philosophies • Ultimate objective: to dissuade people from ever trying drugs 40


• Ultimate goal: to enable individuals to remain free from alcohol, tobacco and illicit drugs and also from steroids over his/her entire lifetime • Ultimate strategy: to co-ordinate prevention with treatment and law enforcement as part of a comprehensive strategy In the United States the basic philosophy of prevention is prohibition. This prohibition is directed against the use of all psychoactive substances, independent of their legal status. In this approach there is - ideologically no place for "harm reduction" or for the concept of "safe use". As the slogan states: "The only safe use is no use!" In contrast to it in the Netherlands most prominently, but in many other European countries as well, the approach of harm reduction has gained high profile. There is basically also the assumption of the possibility of safe use of psychoactive substances, since in Europe generally a sharp distinction is made between "legal" and "illegal" drugs. The concept of alcohol prohibition has never been very popular in Europe. Another major difference between the American and the European approaches runs in the same direction. While in the USA prevention philosophies ask for "denormalisation" of drug use and of drug users through different means including media representation, in Europe and especially in the Netherlands the opposite model of the "normalisation" of drug use and users has been developed. This "normalisation" is a declared goal of the Netherlands’ drug policy. A further major difference between the two drug control cultures lies in the fact that the Netherlands locate drug use and drug problems into the health sector, while in the USA substance abuse and its problems are primarily considered to be moral and societal evils and only secondarily as health problems. Therefore in the USA the overall moral judgement of drug use is much stronger and much more devastating than in most European regions. And there exists also a major difference concerning the involvement of the criminal law system into the control of substance abuse. While the official way in the Netherlands tries to minimise the impact of criminal law, the USA are relying very strongly on that tool.

4.4.2 Primary prevention as a component of drug policies These basic or background philosophies of drug policies curb and shape also the ideas concerning the objectives, aims and strategies to be applied to prevention issues. 41

Country Reports

4.4.2.1 The situation in the USA In the United States prevention efforts are defined as an element of the "War On Drugs" and have to follow the basic assumptions and aims of this particular approach. Therefore the ultimate objective of prevention is "to dissuade people from ever trying drugs", the ultimate goal is "to enable individuals to remain free from alcohol, tobacco and illicit drugs and also from steroids over their entire lifetime" and the ultimate strategy is "to co-ordinate prevention, treatment and law enforcement as part of a comprehensive strategy." This does not mean that intermediate aims are excluded. These intermediate aims are defined as partial effects in the sense that the receivers of the preventive message are not prevented from ever using drugs, but that the programmes used may very well contribute towards a person ultimately living a drug free life or at least decrease the likelihood of the user becoming addicted and a burden to society. According to the experts’ opinion in the USA these partial effects can be obtained through the implementation of diverse strategies: • by programmes that: • lower school dropout rates, • increase enrolment in post secondary education, • increase employment rates, • increase participation in drug free activities, • decrease time spent with drug-using friends, • develop various coping skills; • by programmes which can delay the onset of first use of a drug. These - according to the prevailing European attitude - seemingly "realistic" intermediate aims are nevertheless understood as a kind of compromising; the ultimate goal of prevention is lifelong abstinence as said before. In the US the following strategies are accepted as promising to reach the defined ultimate or primary goal of lifelong abstinence from psychoactive substances: • school based programmes, • media campaigns which propose "denormalisation" of use,

42


• monitoring of risk populations and of (mostly young) individuals at risk, • resistance training against social pressure to use drugs with strong directions against drug use, • drug free activities, • early childhood programmes, • strong community coalitions (between different social agents: church, industry, schools, parents, etc.) against drug use, • user accountability; This last item is of special interest. In this approach the drug user is to be blamed for all the evil effects his habit may bring to him, his family and to society. In this attribution of guilt even the impact of the criminal drug traffic is included. These are the strategies or methods which are accepted in the official USA literature as useful. Some others are labelled as "non promising" or even "obsolete". They include: • • • • • • •

the legalisation approach responsible use messages harm reduction approaches providing only information without strong directions against drug use scare technique self esteem exercises "magic bullets" (slogans, unstructured campaigns, unstructured anti-propaganda, etc.) • Peer group work also is criticised to have very limited effects.

43

Country Reports

4.4.2.2 European attitudes Let’s now have a look at some contrasting European views. For instance on Bühringer’s very precisely formulated goals of drug education (Bühringer, 1992).

GOALS OF DRUG EDUCATION

AREA 1

AREA 2

responsible use of psychoactive substances

high level of general competence

to raise/strengthen the awareness of social rules concerning consumption

to improve the ability to identify stress factors and to cope with them

to renounce the use of certain substances (illicit drugs, etc.)

to strengthen self-confidence self-esteem

to renounce legal drugs use under certain conditions (like gravidity, etc.)

to improve the ability for communication

to use the substances in a moderate way and in accepted situations

to improve the ability to be aware of the function of drug use: consumption only no use as substitute for human needs or to solve problems

44


Of interest for the actual level of our discourse are the drug specific goals mentioned by Bühringer. One can see that Bühringer’s main objective or goal is "responsible use" of psychoactive substances- exactly the message which is classified obsolete in the US. Furthermore Bühringer accepts the possibility of safe use and is strongly in favour of the harm reduction approach in the case of licit substance use, where it is the explicit goal to educate people to avoid the use of drugs in certain social situations and psychic states, where use could be dangerous or addictive. There is also a strong split between licit and illicit substances. One the one hand the attitude concerning illicit drugs is prohibitive, on the other hand concerning alcohol it is permissive with the message of responsible use. The goals as formulated in area two are strongly following the theories of the importance of the influence of risk factors and protective factors on drug use and are also compatible with the goals of health education resp. The "Health for All" concept of the WHO. The differences between the ideological fundaments of prevention in America and Europe are also responsible for differences in the single components of the prevention systems. Let’s look for instance at one of the major approaches in primary prevention, school based drug education. In the United States the ideal school based programme consists of three components. The first component is a comprehensive prevention curriculum on all grade levels which includes: • a clear message of No Use of illicit drugs, tobacco and alcohol • an encouragement of civic responsibility and respect of the law • the raising of the awareness of being healthy and drug free • accurate information about drug effects • teaching how to resist peer pressure to use drugs But a school based programme should also include as second component a strict drug free school policy and as third component the offering of drug free activities for the students. The aims of drug education as formulated by an excellent European expert like de Haes (1990) sound similar if one looks at them superficially. In reality they are very different. While the American programmes are a mixture of directive information about drugs, social influence and training in direct relation to the drug problem and political education the interpretation of de Haes is highly per45

Country Reports

sonality oriented and focuses only slightly on the drug issue. There is no message of "No Use", there are no strong directions against drug use, only the strengthening of already existing attitudes in relation to drugs. The conclusions he draws are very similar to the ones articulated by Bühringer. It is interesting that both, Bühringer and de Haes are giving their proposals as a result of the analysis of mostly American literature. It seems that this was possible, as both authors only focused on the programmes described and missed to analyse the socio-cultural background and context of these programmes and therefore also their strong relations to the "War on Drugs"- approach. It has to be pointed out that Bühringer’s and de Haes’s views are not isolated ones, but rather representative of the state of the art of prevention and of ideas about prevention which have been developed in the greater part of Europe. That conclusion is the outcome of the comparative research of many European country reports about prevention and evaluation of prevention. It therefore seems justified to draw a comparison between the American and European prevention efforts and to classify them as separate systems, which are corresponding in some aspects, but are highly different in many very important aspects. The most important conclusion may be that we cannot assume that personality strengthening methods in use in the USA under the special conditions of the War on Drugs policies will have the same or even a comparable effect on drug using behaviour when used in Europe under highly different conditions of social control in a cultural space where drug policies are more likely to accept a broader and very complex set of attitudes concerning drug use.

4.4.3 Impact on evaluation issues Drug education programmes in many European regions are aiming more at personality development and personality change than on a direct influence on drug using behaviour. These aims cannot easily be assessed. It is for instance not possible to use a weak method like the use of consumption surveys as outcome references. De Haes (1990) correctly stated that generally the assessment of the effect of personality strengthening approaches is even more difficult than the impact of drug specific strategies because of the aims stated. Measurement instruments have to be developed that relate to the personality dimensions one wishes to influence. Existing psychometric scales tend to be not 46


appropriately specific. Very often instruments related to the dimensions one wants to influence will not be available. Constructing such a psychometric scale takes a number of years if one wants to ascertain the reliability and validity of the measurements made. This effort is most often not made, thus the measured effects remain doubtful. Are any pseudo effects shown, or are the existing effects not revealed? That means that we have to rely on theories or even assumptions concerning the causes of drug taking and addiction and their proper use and implementation in prevention efforts. But are the theories proven enough and can we trust the theory-based designs of such efforts concerning their outcome efficacy, so that we are able to do without outcome evaluation at all? In the realm of behaviour change in general and drug education or primary prevention of drug abuse things seem really difficult. We have to be aware that we neither have really confirmed and sound information in respect to the nature of the problem of drug taking and addiction at our disposal nor do we have sufficient evidence concerning effective drug education strategies. At the given state of the art a programme which eventually proves highly adequate if we use the approach of process evaluation can easily be without any impact on the behaviour of the treatment group. Some authors certainly think that we are in a miserable position. Take for instance the critical view of Nicholas Dorn (1990) that could be labelled "British scepticism": He stated: "Drug education internationally has been more concerned with presenting or implying particular images of drug users, than with realistic appraisals of practical possibilities. Therefore: 1. The ’Just Say No - Approach’ just doesn’t work. It is based on the unrealistic assumption that people experiment with drugs because they are argued into doing so and lack the skill to act independently. 2. The ’Person-Focused Approach’ doesn’t work either. It is based on the equally false assumption that people experiment with drugs because they lack self-esteem or decision making abilities."

47

Country Reports

Dorn might be right. We have to take this position into account and should be very cautious indeed to embrace the now dominant interpretations of the psycho-social background of drug use and of addiction channelled into the construction of particular images of drug users. We should be equally cautious of the message that if we accept these constructions we can do without outcome evaluation, as some people embrace the promises given and the hopes awakened by a new drug. At the time given and keeping the eye on the development of some addiction prevention ideologies it is definitely necessary to come to conclusions concerning the relevance of theories about drug consumption problems for prevention programmes and their evaluation and concerning minimal standards of such programmes. If we don’t do so, we leave the field to a blind pragmatism and to wild speculations becoming reality in a vicious circle of self fulfilling prophecies.

48


5 Appendix - three examples of original country reports as illustration 5.1 Report of Poland A

General data on projects

A1 Project title: Sunrise

A2 Key-persons / key-institutions: Regina Kostrzewa, Zentralstelle für Suchtvorbeugung, Kiel

A3 What state is the project currently in? • already finished? …….............................................. • ongoing? ................................................................... x • in a planning stage? .................................................. • I don’t know and couldn’t find out?......................... A4 Have there been any publications, official presentations or grey literature yet regarding the project? • yes no : ? ready in two month

• If yes, please give a full citation of all related papers or articles and if • possible, please include a photocopy of the article or articles. • If the material is not published in an international journal, please give the name and address of at least one author: B


B1 Abstract goals concerning illicit drugs, such as • less consumption ...................................................... x • later onset of consumption ...................................... x • safer use ................................................................... • harm reduction in terms of health problems ............ • harm reduction in terms of social problems ............ • harm reduction in terms of public disorder .............. • harm reduction in terms of immoral conduct ........... • no goals specified explicitly..................................... • etc.............................................................................. describe in detail: .................................................................................

49

Country Reports

B2 Details on programme structure, such as • Is there a written concept - yes • How many sessions were planned - unknown • Who is carrying out the programme Zentralstelle für Suchtvorbeugung

• How is the staff trained to do the job Train the trainer programme and preliminary discussion • How much time is used for staff training - depends on the staff

• etc. describe in detail: ........................................................................................... B3 Type of programme in terms of • target population (e.g. school based, community based, mass media campaign, training of educators, teachers, physicians, policemen, parents etc., change of law or policy etc.) and • size of target population (e.g. nation wide media campaign, one school including 12 classes, etc.). describe in detail: in schools for pupils between 16-17 years

B4 Techniques used, such as • general health education ........................................... • changing lifestyles .................................................... x • changing attitudes..................................................... x • social skills training.................................................. x • deterrent information ................................................ • factual information ................................................... • offer of alternatives to drug taking........................... x • clubbing .................................................................... • modification of drug laws (e.g. decriminalisation of mere consumers, compulsory treatment of addicts, more severe legal sanctions for consumers etc.) ................................... • modification of drug policy (e.g. offer of syringe exchange, offering low level intervention, etc.)........................ • change of police strategy (e.g. tolerate scene, focus on large scale dealers, etc.) .............................................................. • change of treatment policy (e.g. making methadone maintenance available)................................................................... • etc.............................................................................. describe in detail: ................................................................................... 50


B5 Drugs mentioned in programme, such as • no drugs mentioned at all ............................................................. x • only illicit drugs mentioned.......................................................... • only medical drugs mentioned...................................................... • both legal and illicit drugs mentioned.......................................... • only specific drugs mentioned (e.g. cannabis, heroin, cocaine) .. • etc.................................................................................................. describe in detail: ................................................................................... C

Data relating to evaluation

C1 What targets were/will be evaluated? • operational objectives (internal targets), like "Did the information reach the target population?"; "Was the programme concept carried out as planned?", etc. • intermediate targets (surrogate targets) associated to the final goals but not primary goals by themselves, like e.g. attitude change, increased self-esteem, higher information level, etc. • ultimate targets (primary targets) like e.g. less drug consumption, later onset of drug consumption, harm reduction, less problematic use, etc. describe in detail: ................................................................................... C2 How was the study sample defined? describe in detail: after every project C3 Sample size and drop-out rate? describe in detail: ................................................................................... C4 How often were data collected and at what time schedule? describe in detail: ................................................................................... C5 How were the data collected, resp. what data were used? • surveys /population surveys ..................................... • telephone interview .................................................. • face-to-face interview ............................................... • questionnaire presented by investigator ................... x • mailed out questionnaire........................................... • public statistical data ................................................ • police records............................................................ • treatment records ...................................................... • other .......................................................................... describe in detail: ...................................................................................

51

Country Reports

C6 Was the programme successful and if yes, in which dimension(s)? describe in detail: yes, but not enough participation

C7 Were there specific problems worth explicitly mentioning? describe in detail: problems in organisation

D

Global part

D1 Personal summary on the different projects mentioned Please give a generalising overview out of your personal impressions over the state of evaluation on primary prevention research regarding illicit drugs in your country. D2 Conclusions: What conclusions do you personally draw out of the present state of evaluation in your country and how do you feel about prevention and evaluation of prevention in this field generally • Which prevention concepts do you think are promising and which concepts are worthless? • How do you define the limitations of evaluation in this field? • etc. D3 Is there anything like an official national prevention concept in your country? If yes, • • • •

what is the concept like? is it available as a written document? is there a gap to implementation? are there any relevant groups openly in opposition to the concept and who are they?

D4 Please indicate names and addresses of other researchers and institutions in your country that are active in the field of prevention evaluation?

52


5.2 Report of Greece A

General data on project

A1 Project title: Education for Health preventing dependence and addiction: Experience of a H.E. pilot project aiming at preventing drug abuse in a community oriented approach applied in a suburb of Athens (Greece) A2 Key-persons / key-institutions A. Kokkevi, A. Mostriou, E. Lentaki, C. Stefanis Department of Psychiatry Athens University Medical School A3 State of the project Already finished A4 Publications See Annex 1 B


B1 Abstract goals concerning illicit drugs: The aim of the so called "Education for health preventing dependence and addiction" programme is to enhance in the long term, both the individual’s own understanding and problem-solving capacities in relation to his health, and the social and cultural environment of the community in which the individual lives. Drug abuse is not considered as a specific and isolated subject: by focusing upon the whole person, the programme is directed towards those problems associated to drug use. B5 Drugs mentioned in programme: Both legal and illegal

53

Country Reports

C

Data relating to evaluation:

The evaluation procedure performed two main functions: • Formative evaluation aimed at testing the feasibility of the project through the assessment of its development and acceptance by the school and the community. • Summative evaluation, aimed at collecting information for assessment of the outcome of the H.E. programme. Pupils, parents, teachers and the authorities were involved in the assessment of the project. C1 How was the study sample defined: Pupils, who followed the programme for 3 years, parents, teachers involved in the programme as well as pupils of a control school for the outcome evaluation

Sample size: Pupils of target school: Pupils of control school: No of teachers involved: Parents interviewed Parents answering questionnaire

No

Drop out rate

151 / 142 126 / 96 31 / 25 379 372

9% 8% 8% 14% 10%

C2 How often were data collected and at what time schedule? Before, during (every year) and after implementation of the programme C3 Methods and techniques of the evaluation: In order to attain the aims mentioned above, monitoring techniques (close documentation of the events and experiences of the project) and social research techniques (surveys on the needs of pupils, teachers and the community) were used leading to critical reflection upon the results by the project team. The tools used for the process and outcome evaluation were: 54


• Questionnaires filled in by pupils of the target school as well as by pupils of a control school investigating health needs, behaviour linked with health risks, attitudes, values and beliefs towards health. • Interviews given by parents and questionnaires filled in by them providing us with social and health indicators on the families involved in the project as well as on their beliefs, attitudes and life style as regards to health. • Questionnaires filled in by teachers concerning their health habits and knowledge, their educational methods, their attitude towards H.E. and its implementation in the school curriculum. All questionnaires were filled in before and after the implementation of the programme. • Records kept by teachers through out the 3 year period on the experience they had with H.E. in the classroom. • Records kept by the school co-ordinator and community co-ordinator on activities related to the project. C4 Results of the programme: After the three year application of the programme, outcome evaluation results were based on the following data: a) Participation -Teachers’ involvement: In Hymettus, 50% of the teaching staff implemented the programme on a voluntary basis during the first year. The number of teachers involved in the programme decreased every year: persuading teachers to collaborate in an innovating programme is a difficult task, but keeping them motivated while carrying out the whole project is a much more demanding situation. - Pupils’ involvement: The interest shown by the pupils during the implementation of the programme was almost unanimous as quoted by the teachers. Half of the pupils were also involved in the extra curricular activities although most of them took place during their leisure time. Pupils acceptance and active involvement in the programme was a major incentive for teachers to apply it. - Community involvement: Although parents did not have a participation in community activities through the years of the project implementation, however, most parents 55

Country Reports

were aware of the subjects presented in school during the H.E. sessions and had discussed about them with their children. b) Reactions to the programme: − At the end of three year application of the pilot project, most teachers accepted that the programme had a positive impact on themselves, their pupils and their own family. They agreed that the programme should be inserted officially in the general curriculum of the school. − Pupils' response was very positive and was considered as a major incentive for the teachers’ to continue applying the programme in spite of the difficulties encountered. − Parents’ commitment: Answers given by parents to an anonymous questionnaire during the final stage of the programme show that they seemed more optimistic than during the initial stages of the programme regarding the influence they can have on their children in order to avoid the use of drugs. Some of them even declared having tried to reduce smoking and drinking and unanimously, they asked for the programme to carry on. c) Knowledge, attitude and behavioural changes: Knowledge, attitude and behavioural changes were measured through answers given by the pupils themselves, before and after the implementation of the programme in the target school as compared to answers given by pupils in a control school. The outcome evaluation showed the following:

56


Knowledge and attitude changes: − More boys of the target school as compared to the control school admit the dangers of smoking, alcohol abuse and use of illegal drugs, while more girls in the target school admit the dangers of alcohol abuse and use of legal drugs (medicines). − Boys and girls of the target school show more reservation when talking about the probability for them to smoke, use alcohol or illegal drugs in the future. Behavioural changes: - Smoking and alcohol use by boys: - Smoking and alcohol use by boys increased during the 3 years of the implementation of the programme in both the target and the control school. This finding was to be expected since pupils were three years older when the outcome evaluation took place. We note however that more boys in the target school agree to try and quit smoking as compared to the control school. - Abuse of alcohol increased in the control school while it decreased in the target school - Four times as many pupils in the control school as compared to the target school report having got drunk at least once in a lifetime - Smoking and alcohol use by girls: - Increase in smoking is observed only in the control school - Occasional alcohol use increased more in the control than in the target school - Five times as many girls report having got drunk in the control school as compared to the target school at least once in a lifetime - Use of legal and illegal drugs.

57

Country Reports

- Use of legal and illegal drugs: The actual number of pupils using legal drugs without prescription, for other reasons than medical ones, as well as pupils having tried or used more than once illegal drugs, was very small in both schools. However the increase of illegal drug use by boys as well as the increase of both legal and illegal drugs by girls is bigger in the control than in the target school. Process and outcome evaluation of the pilot project in Hymettus lead us to the following general conclusions: Managing the development of the project in Hymettus, involving teachers, parents authorities and political key persons, gaining the approval and active involvement of the pupils and assessing the impact of the programme on them, made it possible to propose the expansion of the programme to a larger number of schools. C5 Specific problems in the project Constraints encountered during the implementation of the programme were the following: 1. Curriculum development of drug misuse prevention: - Finding teachers who would implement the programme on a voluntary basis - Assisting teachers in their health education role - Keeping teachers motivated - Providing appropriate educational material for teachers and pupils 2. Linking school and community H.E. was a challenge to face. There were definite difficulties in: - raising awareness of the community that Health prevention relies upon assuming responsibility for one’s own health - strengthening co-operation and contacts between school and community - actively involving pupils and teachers in the health promotion activities taking place in the community - getting parents and the local community involved

58


3. Administrative and legislative provisions: In order to deal with bureaucratic difficulties encountered while planning implementing the programme, administrative and legislative provisions had to be taken. These were possible only through systematic contacts with key persons in order to: - prove to politicians, authorities, the media and through them to broader community that PRIMARY PREVENTION is at least as important as THERAPY in resolving the addiction problem of young people and certainly that is worth investing in. - convince key persons that primary prevention is based on education. It is TEACHERS in the schools themselves who deal with health education (H.E.) must be developed within the context of the school curriculum - increase the awareness of administrators and organisations of the need for long term commitment to innovations that are planned for several years in order to ensure programme continuity - adapt legislation to support H.E. and more specifically drug misuse prevention D

Global part

In Greece, it is only recently that professionals involved in Public Health acknowledged the fact that health education (H.E.) should be seen in the broad sense of health promotion - its main objective being to help individuals acquire the appropriate knowledge and the motivation to adopt and maintain healthy patterns of behaviour in their lifestyles. Up to this day however, the approach of most prevention programmes is mainly informative. In order to implement health promotion programmes in schools initiatives have been taken by the Department of Psychiatry of Athens Medical School. These are mainly epidemiological research picturing the drug situation in Greece, school surveys and pilot projects for the implementation of health promotion programmes in schools. An increasing number of communities in Greece started to implement drug prevention programmes - including school health education - supported by their respective Municipalities. Funds are however usually limited and programme continuity is often not ensured.

59

Country Reports

5.3 Report of France A

PROJET: CREDIT1

B

DONNEES GENERALES CONCERNANT LE PROJET

B1 TITRE: Programme d’Education pour la Santé et Prévention des Toxicomanies et du SIDA dans les Alpes-Maritimes: 1989-1990 B2 PERSONNES/INSTITUTIONS DE REFERENCE P. J. SIMON Comité Départemental d’Education pour La Santé, Alpes-Maritimes (CODES) Centre de Recherche de Documentation et d’Information en Toxicomanie (CREDIT) 10, avenue Malausséna - 06000 Nice B3 ETAT ACTUEL DU PROJET: terminé :

en cours

en préparation

B4 PUBLICATIONS: oui :

non

Programme d’Education pour la Santé et Prévention des Toxicomanies et du SIDA dans les Alpes-Maritimes: 1989-1990. Rapport d’évaluation, juin 1991 C

DONNEES RELATIVES AU PROGRAMME DE PREVENTION PRIMAIRE

C1 RESUME DES OBJECTIFS CONCERNANT LES DROGUES ILLICITES Action globale d’éducation pour la santé, donc par définition non spécifique "toxicomanies". Un des objectifs était de permettre aux jeunes d’évoluer quant à leurs REPRESENTATIONS et CONNAISSANCES (santé, toxicomanie, SIDA). Ceci en facilitant le dialogue et en développant la participation des jeunes et des personnels des établissements scolaires. C2 STRUCTURE DU PROGRAMME 60


Le concept général annoncé: Education pour la santé, approche GLOBALE, PARTICIPATIVE et INTEGREE. Action expérimentée dans 6 établissements scolaires. Dans chaque établissement des personnes ont été "choisies" et formées (6 mois). Programme de formation non décrit. Une session de 2 jours centrée sur la communication est signalée. L’action faisait partie du projet d’établissement et s’est déroulée sur une année scolaire: septembre 89 juin 90, avec différentes formes d’interventions et de productions (vidéo, peinture, dessins, textes, chansons, journaux, ...) C3 TYPE DE PROGRAMME EN TERMES DE * population cible (qualité) Six établissements scolaire avec une classe par établissement. La répartition devait être aléatoire. En fait choix plutôt délibéré et désignation de classes jugées comme plutôt difficiles. * population cible (taille) 126 élèves de 13 à 16 ans. B4 TECHNIQUES UTILISEES: EDUCATION POUR LA SANTE: Le principe général de la méthode pédagogique adoptée est décrit comme suit: "Eviter le discours moral, mieux comprendre les croyances et les représentations des jeunes, mieux communiquer". Techniques utilisées et formes d’intervention ont varié selon les établissements, comme indiqué plus haut. B5 DROGUES MENTIONNEES DANS LE PROGRAMME Aucune d’une façon spécifique. L’évaluation aborde la perception des "DROGUES DURES", du CANNABIS, TABAC et ALCOOL. C

DONNEES RELATIVES A L’EVALUATION

C1 INDICATEURS (CIBLES) EVALUES OU A EVALUER: Objectifs opérationnels: déroulement du programme. Objectifs intermédiaires: Perception de la santé et des risques. Evolution des attitudes, opinions et comportements. Système relationnel (groupe d’amis, famille). C2 DEFINITION ET CHOIX DE L’ECHANTILLON:

61

Country Reports

Groupe expérimental: les élèves des 6 classes concernées. Groupe témoin: élèves n’ayant pas participé dans des établissements scolaires différents. Le choix devait être aléatoire, or il a été possible de constater une différence entre les 2 groupes au départ. Les classes du groupe expérimental ont en fait été désignées car jugées "en difficulté". C3 TAILLE DE L’ECHANTILLON ET TAUX DE CAS PERDUS 219 élèves répartis en 2 groupes, expérimental (n=126) et témoin (n=93). Aucune indication sur la mortalité expérimentale, qui semble toutefois assez réduite. C4 PLAN D’OBSERVATION Observation (questionnaire) avant (début de l’année scolaires) puis après (fin de l’année) avec deux groupes expérimental et témoin. C5 MODALITES DE RECUEIL DE DONNEES ET DONNEES UTILISEES Questionnaire abordant l’ensemble des paramètres évoquées. Analyse qualitative et quantitative des réponses. C6 RESULTATS POSITIFS ET DIMENSIONS CONCERNEES Le processus s’est déroulé conformément aux objectifs, notamment en matière du travail interpartenarial et participatif souhaité. Au niveau de l’impact du programme: amélioration du climat relationnel. Meilleure maturation des projets professionnels des élèves. Impact positif sur certains indicateurs du mal-être telle que l’anxiété. Amélioration des connaissances et aussi correction de certaines idées fausses, notamment à propos du SIDA. C7 PROBLEMES SPECIFIQUES QUI MERITENT D’ETRE RELEVES: Biais lié à la sélection du groupe expérimental.

62


6 References Buisman, W. R.: Drug Prevention in the Netherlands. Hilversum, 1994 Bühringer. G.: Drogenabhängig. Wie wir Mißbrauch verhindern und den Abhängigen helfen können. Freiburg: Herder, 1992 Centre National de Documentation sur les Toxicomanies (Ed.): Jalons pour des actions de prévention. Guide critique des outils de prévention des toxicomanies. Lyon, 1994 Dorn, N.: British Policy on Prevention. In: Ghodse et al. (Ed.), 1990; op. cit. Engelsman, E. L.: Dutch policy; on the management of drug-related problems. In: Ghodse et al. (Ed.), 1990; op. cit. Ghodse, H. A.; Kaplan, C. D.; Mann, R. D. (Ed.): Drug Misuse and Dependence. Casterton Hall, Carnforth: Pantheon, 1990 de Haes, W. F. M.: Problems of Evaluation. In: Ghodse et al. (Ed.), 1990; op. cit. Kunz, G.: Primärprävention an Schulen als Beitrag zur Suchtprophylaxe. Diplomarbeit. Wien, Juni 1995 Künzel-Böhmer, J.; Bühringer, G.; Janik-Konecny, T.: Expertise zur Primärprävention des Substanzmißbrauchs. IFT-Bericht Bd. 60. München, 1991 PEDDRO: International Annotated Bibliography on the Prevention of Drug Abuse Through Education. UNESCO, 1994 Smart, R. G.; Fejer, D.: Drug Education: Current Issues, Future Directions. ARF Books No.3; Toronto, 1974 The Drug Policy in the Netherlands. December 1992 version. Ministry of Welfare, Health and Cultural Affairs and Ministry of Justice. The Netherlands, 1992 Understanding Drug Prevention. An Office of National Drug Control Policy White Paper, Washington DC, May 1992 Zaccagnini, J. L.; Colom, R.; Santacreu, J. (Ed.): Catalog de programas de prevencion de la drogadiccion. Valencia: Promolibro, 1993

63

Country Reports

contact address: Alfred Springer LBISucht, Vienna, Austria Tel: +43-1-8882533-112 FAX: +43-1-8882533-77 e-mail: [email protected]

64


Building Expertise in Life Skills Programme Adaptation and Evaluation: The Experience of "Leefsleutels" Annick Vandendriessche Programme Director of "Leefsleutels", Flanders, Belgium

___________________________________

Paper Presented at the COST-A6-WG2 Workshop "Socio-Cultural Aspects of Primary Prevention of Drug Abuse and its Evaluation" December 13th and 14th 1996 in Vienna

65

The Experience of "Leefsleutels"

Contents 1 History...................................................................................................... 67 2 From adaptation to development ............................................................. 67 3 Goals and instruments.............................................................................. 68 4 Some changes in "Life Skills for Youngster" ......................................... 68 5 Levels of difference ................................................................................. 69 6 Evaluation ................................................................................................ 70 6.1 Limitations to the effect study.......................................................... 71 6.2 Results............................................................................................... 71 6.2.1 Formative study........................................................................... 71 6.2.2 Process study ............................................................................... 72 6.2.3 Effect study.................................................................................. 72 7 Actions for "Life Skills for Youngster" .................................................. 73 8 Influence on the programme development of "Life Skills in Action" .... 74 8.1 Training............................................................................................. 74 8.2 Books ................................................................................................ 74 9 Practitioners towards evaluators.............................................................. 74 10 Development process............................................................................. 75 10.1 Basis................................................................................................ 75 10.2 Working group/pilot group............................................................. 75 10.3 Sources of feedback........................................................................ 75 11 Conclusion ............................................................................................. 76

66


1 History 1

In 1990 "Leefsleutels" ("Life Skills ") in Flanders started with the programme "Life Skills for Youngster" ("Leefsleutels voor Jongeren") an adaptation of the American drug prevention programme "Skills for Adolescence" developed by Quest International. This programme is used with 12-14-year-olds in Flanders while the age range in the US is wider (10-14-year-olds). In 1990 the programme was already in use in several European countries. We started our adaptation from the Swiss version. In 1994, after four years of experience with "Life Skills for Youngster" ("Leefsleutels voor Jongeren"), we started to adapt another programme the "Skills for Action" programme - suited for 15-18-year-olds. This adaptation "Life Skills in Action" ("Leefsleutels in Actie") deviated even more from the original programme than "Life Skills for Youngster". Since then, we have developed from scratch and still are developing various new initiatives, oriented to specific risk groups or certain specific issues: e.g. a programme for youth in institutions, a programme for youth workers, a programme for youngsters in special education, a special module on alcohol, drugs and traffic, implementation workshop, etc.

2 From adaptation to development To sum it up, we moved from programme adaptation to real programme development. One of the resources allowing us to develop new programmes was an external evaluation study carried out between 1991 and 1994. Another resource was the network we developed including approximately 10000 trained persons in Belgium, of whom a large group has worked with the "Leefsleutels" programmes from one through six years by now. Around 40% of the first graders in secondary school are confronted with "Life Skills for Youngster" at school. Among their teachers we find a large group of skilled and motivated people to supply us with feedback. The last resource is our trainers who acquired a huge experience in working with the target groups.

1

The word "Leefsleutels" translates into "Life Skills". The literal translation for "Leefsleutels" is "Life Keys".

67


3 Goals and instruments Our programmes have a number of common goals. "Leefsleutels" does drug prevention by • training youngsters "life skills", like "listening", "expressing feelings", "decision making", "saying yes", "saying no", "risk taking", "goal-setting", etc. • promoting positive relations at home, in school and in the community • reinforcing of personal and social responsibility This is done based on three approaches: • The first approach is books that we edit and distribute; thus creating resources for pupils, parents and teachers. • The second approach is training of teachers and educators. Three or four days of workshops, where the participants experience a group process, learn from each other, sit on pupils chairs and prepare and try out some try-out lessons for their colleagues. • The third approach is networking. We try to keep teachers together and reinforce their skills using a magazine, follow-up days, regional meetings, thematic meetings and regional co-ordinators ready to assist the schools. The programme for the 12-14-year olds contains 70 lessons, to be spread over 2 years. The programme for the 15-18-year olds contains 50 lessons to be spread over 2 to 4 years. Goals in "Life Skills for Youngster" are "group building", "self-confidence", "listening skills", "dealing with feelings and emotions", "friends", "peer pressure and conflicts", "home", "critical decision making", "drugs" and "planning ahead".

4 Some changes in "Life Skills for Youngster" When in 1990 a group of 28 pioneers (teachers, drug prevention workers) followed the first Leefsleutels workshop, they agreed that there was a good base here for doing drug prevention in schools, but that this material was too American and needed to be adapted. About ten of them participated actively in the adaptation process.

68


Some of the changes we made are: • We dropped the if-then notion. In the American version statements, like ‘If you do this game, teacher, you will have such a process in the class”, "If you follow these steps, pupil, you will get those results”, etc. are used too often. • We added several lessons about negative feelings and issues. E.g. about a friend who leaves you, about a friend hurting you, about you hurting a friend, about frustration, about things you fail in, etc. How to cope with the fact that you won’t reach certain goals, you have set for yourself, e.g. because of socio-economic factors. • We changed the lessons on drugs. "Just Say No”, doesn’t work in Europe. We want youngsters to learn to make their own choices, knowing all factors. • Another difference is the issue of complexity. We actually recognised this as a value in our versions. For some lessons it is an objective to make the pupils doubt. Sometimes we must authentically communicate to the pupils that what he thought to be true might not be true, without imperatively giving another answer. • Finally, we don’t only train teachers to "go ahead with the material”. We give them follow-up support. So that, back in the day-to-day reality after a workshop, they still have access to support. This is more complicated, but important to guarantee a long-term implementation.

5 Levels of difference If you look at these examples, you see that the changes go far beyond simple translation. If you take a foreign programme, you can work on different levels. Translation. You can simply translate and reformulate it. Cultural adaptation. You can use other examples and situations, other stories and illustrations. You can talk about football instead of rugby etc. This is quite simple. Deeper adaptation. • The examples given in the paragraph Changes in "Life Skills for Youngster", have more to do with a deeper level of adaptation. Here we recognise the different starting situation of Flemish compared with American teachers. Flemish teachers received a less interaction-oriented

69


training. Furthermore, in many schools there still is a strong accent on acquiring knowledge as such. • The concept of a friend, a comrade, a girlfriend is different. • On the occasion when American trainers came over to run a pilot workshop, we could see and they recognised that they were not so used to dealing with a critical audience. Programme development. The last level is programme development, which we started in the context of "Life Skills in Action", the programme for 15-18-year-old. From there on, we moved further to programme development from scratch (e.g. programme for youth in institutions). Recent literature concerning the subject, other framework, other activities, taking into account the typical situation of teachers, schools, pupils and health educators were taken into consideration. In this programme, only a small percentage of the original material is kept. We discussed with the original programme makers to understand why they chose their way of working. • E.g. they decided for a less interactive programme, because they followed their teachers. And teachers in high school are less trained to interactive teaching than teachers for 10-14-year-old. We consciously chose to keep a very interactive approach, since our teachers for the younger age group don’t get much training in interactive teaching either and the interactive approach did work with them. In this regard we didn’t want to follow the original version. Furthermore, interaction is very important when prevention is concerned.

6 Evaluation It was in the adaptation process of "Life Skills for Youngster" that the question for evaluation arose: • To further develop and improve the programme. • To know how the programme was implemented (if we programme makers survey the results ourselves, that could falsify the results, especially because at that time the programme was offered free of charge). • We needed credibility and were seeking authority as a social profit, private organisation, in a sector dominated by public funded organisations.

70


The evaluation was done by the University of Gent for Flanders (Laboratorium voor Interactionele Psychologie) and by the University of Liège in the French speaking Community of Belgium (Centre d’Enseignement et de Recherche en Education pour la Santé). It became a three-step study: • A formative phase. What kind of programme is this? Which theory does it base itself on? • In a second step we evaluated the process. How are the teachers, the schools applying this programme? • Parallel to this an effect study was carried out. How does the programme affect the pupils?

6.1 Limitations to the effect study Teachers didn’t have much experience with the programme. Our first workshops were in September 1990. The effect study occurred between November 1993 and June 1994. Thus, most of the teachers involved had only had one year of experience with the programme. The pre-test was done quite late. The school year starts in September. The pupils already had worked with the programme for two months by the time the pre-test was administered. The post-test was too early, considering that 75% of the users had applied the programme spread over two school years. The questions were sometimes formulated in a language too difficult for the students. Important groups were excluded from the study sample (pupils from special schools). Finally, it is difficult to find good measurement instruments for small changes in quite small populations.

6.2 Results 6.2.1 Formative study The programme was reported to be in line with recent literature on drug prevention. The programme is operationalising a lot of wide-spread, theoretical concepts. The adaptation process was described as thorough. We received some concrete suggestions on how to reformulate goals and objectives, so that they could actually be achieved. E.g. "pupils will learn to listen better" is too vague.

71


6.2.2 Process study The process study gave us very interesting information about the application of the programme. • application: 67% of the teachers in the study used it one hour a week. During the year of the study, they went through two to four chapters. Only 35% of the teachers really followed the order of the lessons. There is a clear tail-effect in the application: the last lessons of each chapter are used less, and the last chapters are also used less. E.g. "Life Skills for Youngster" contains seven chapters. From the 33 classes observed, 26 worked with chapter one. Only five worked with chapter six. • Teachers feel insufficiently informed about certain areas, like developmental psychology, personal and social skills, the advantages and disadvantages of drug use. They feel confident with active methods like talking in a circle, group work, but not so confident with more demanding active methods like role-play, brainstorming and energisers. • Internal support is expected and needed. It is expected mostly from pupils and the director, and in a lesser extent from social workers and colleagues. The director supports the programme less after the start of the implementation. Very positive methods are work with parents, exchanges with other schools and additional groups of trained persons in their own school. • External support occurs through follow-up-days. But only 50% of the trained teachers actually used this opportunity. An other way to get support is through following other forms of trainings. 73% of the trained teachers followed other trainings, linked to the subject of "Leefsleutels".

6.2.3 Effect study The effect study gave the following results: • an increase in self-confidence of pupils • a positive effect on class climate • improvements in aspects like feeling lonely, difficulty of making friends were lower in classes with "Leefsleutels" • an improvement of skills to express themselves (especially talking about problems, with parents and friends) • an increase of well-being of pupils in the class 72


The effect study doesn’t show a change in drug consumption, but it is clearly in line with the goals of the first chapters of the programme "Life Skills for Youngster". It is only these chapters that have been taught to many classes in quite a complete way in the time frame of the study.

7 Actions for "Life Skills for Youngster" Some actions were undertaken as a consequence of the evaluation for "Life Skills for Youngster" • Dissemination of results: we organised study days and articles were written in different educational magazines concerning the issue. • Attention to the theoretical background of the programme was reinforced; e.g. on training, the trainers explicitly make the link between the theoretical model and the activities the workshop participants have had. In the second adaptation of "Life Skills for Youngster", a lot of attention went to this aspect and how we can stimulate it to be read more often. • A specific part on the development of adolescent was written • In the methodology and in our magazine, a more detailed explanation was given about the short versions you can apply if you have less time, with a clear explanation about goal and time frame • a follow-up day on implementation of the programme was developed, to help teachers and school directors with their implementation problems • a day for directors was developed. Even though we encourage directors to come to our workshop, they can’t always find the time. On such a day, they get an overview of the programmes, and the trainer goes into depth with them on their role in the programme. • We have other workshops under development that deal with what the teachers expressed as a lack. E.g. other active methods, teamwork at school. • We have appointed a co-ordinator per province to assist schools in the implementation process.

73


8 Influence on the programme development of "Life Skills in Action" 8.1 Training The American programme puts forward a three day workshop. The effect hereof is a one-shot motivation of teachers, but this doesn’t secure the implementation. Coming back to real classroom situations, a teacher might feel: "it’s not like at the workshop, so I can’t do much with this." In the European versions of the adolescence programme we already set up structured follow up days and a networking system. We do this again for "Life Skills in Action". And give particular attention to a number of aspects, since we have teachers without much teachers training More attention to demonstration lessons and animation techniques is given, as well as clear processing moments and using different processing methods. Furthermore, if we look at the evaluation study, teachers obviously have problems to plan their lessons realistically and adequately for their pupils groups. That’s why this is explicitly practised in the follow-up days.

8.2 Books Tools for personal planning and evaluation of the teachers’ work are included. These help along with the attention to this element at the workshop. Each lesson also has a slightly different structure than in the other programme, with a section specifically providing animation tips for each lesson. After each exercise, possible processing questions are put.

9 Practitioners towards evaluators From our experience as practitioners, we would like to share some ideas with you. • Be aware of which evaluation you demand when. Try to have a very open discussion with the evaluator about what is possible at the stage of the programme you are in. E.g. for an effect evaluation, the group was quite limited, the time frame short and the experience of teachers with the programme was small. Is it then most adequate to have such a study? • Be aware of the different goals the different parties have. Negotiate a clear contract. As a practitioner, finding elements to improve your programme is very important. A university has most interest in 74


producing articles it can publish, and the sponsor wants to know if it’s worth investing in this project. Thus, a common evaluation project has to be drawn, where certain expectations will be fulfilled and others maybe not. If this is not clearly negotiated, it creates problems later. • Objectivity doesn’t mean no interaction during the study. Participants to the study found questions too difficult. This could have been avoided if we, as practitioners, had been through the questions beforehand.

10 Development process Finally, I am providing you with some key steps in an adaptation or development process.

10.1 Basis We departed from an available basic concept and from literature. This is a written basis to start from, be it original or not. If possible, discuss this written concept with its authors, to get more background information on the educational environment and find out if that is similar to your background.

10.2 Working group/pilot group We utilised a working group of trained teachers. Those went through a pilot workshop with pilot books. They all had a mandate from their principals. The necessary time to work at this had to be guaranteed. There were feedback sessions with the whole group, and days with a subgroup of teachers, who participated more actively. We regarded the books really as pilot versions, with the openness to change and revise important parts of them. The workshop was also completely changed after the pilot workshop. An adaptation of the workshop model is very important as well.

10.3 Sources of feedback We used different sources of feedback: • Feedback from the pilot group through feedback sessions • Feedback from the pilot group through questionnaires • Feedback from experts doing site visits

75


Finally, the concept has to be reviewed, making use of all this information. In this programme, we worked together with Flanders, the French Community of Belgium and Holland, which didn’t make it easier to develop the programme, but which was perceived as an extremely fruitful experience by everyone involved.

11 Conclusion Adaptation and evaluation clearly interact with each other. They stimulate quality in the work of practitioners. It’s a way to use evaluation not only to "prove" the programme, but also to "improve" it. We are happy to notice an increasing attention for evaluation by Flemish authorities and field workers. On the Flemish level, we discuss how we can make evaluation more practical (effective, but not too heavy for the schools involved and not too costly for the financier). On European level, we notice this attention as well. Thanks to these experiences, we monitor our activities more and more through the years. Next to that, we are considering an external evaluation again - if we can find funds to finance it.

12 References Orban M., Marichal E., Colin P., Verjus N., Piette S., Czapla S & Wuidar H., Evaluation du programme Clefs pour l'Adolescence en Communauté française de Belgique, Centre d'enseignement et de recherche en éducation pour la santé de l'Université de Liège, Belgique, 1995 Van Oost P., De Backer G., De Potter B.& Maes L., Onderzoeksrapport: proces- en effectevaluatie van het programma 'Leefsleutels voor Jongeren', unpublished report, universiteit Gent, Belgium, 1995 contact address: Annick Vandendriessche Leefsleutels vzw Jongeren Leopold II laan 63 bus 3 B-1080 Brussels, Belgium Tel: +32 2 4216720 FAX: +32 2 4216729 e-mail: [email protected] 76


Mia’s Diary: An Alcohol and Drug Primary Prevention Programme for the Nordic Countries Line Nersnæs Department of Research and Health Promotion Norwegian Ministry of Health and Social Affairs, Oslo, Norway

___________________________________ Paper Presented at the COST-A6-WG2 Workshop "Socio-Cultural Aspects of Primary Prevention of Drug Abuse and its Evaluation" December 13th and 14th 1996 in Vienna

77

Mia’s Diary

Contents 1 Short introduction to the educational programme and the underlying assumptions behind the implementation ................................................. 79 2 The theories behind the programme ........................................................ 80 3 The evaluation approach.......................................................................... 83 4 A brief presentation of the main results .................................................. 85 5 Concluding remarks................................................................................. 87 6 References................................................................................................ 89

78


1 Short introduction to the educational programme and the underlying assumptions behind the implementation Mia’s Diary is an alcohol and drug primary prevention programme, developed as a result of a Nordic co-operation. It is part of the general effort to delay onset and minimise involvement of substance use among adolescents. The Nordic Committee on Narcotic Drugs has been responsible for the compilation of the material. The Nordic Committee on Narcotic Drugs comes under the auspices of the Nordic Council, which is a forum for joint discussions between the parliaments and governments of the Nordic countries. Currently in the Nordic countries there is a large number of educational programmes aimed at preventing alcohol and drug use among adolescents. The programmes offer different preventive strategies and all claim to represent the best solution to how this education should be implemented most successfully. Mostly these programmes are based on modern pedagogical theories and have been carefully worked out. Anyhow many of the programmes in use in the Nordic countries have been developed in the United States and are deeply rooted in the American culture, which makes the transmission to the Nordic reality a bit difficult. An example can be the commonly used Quest programme "Skills for growing", developed at the Quest Institute. The background of initiating a joint Nordic educational programme must be seen in the light of the request to develop a programme mirroring Nordic conditions and Nordic culture, to be used in schools in the Nordic countries. It was thought that having similar social and cultural backgrounds, the Nordic countries could well share the same educational package. Mia’s Diary was implemented in all the Nordic countries during 1994 except in Iceland. Mia’s Diary was written by Jørgen Svedbom, an assistant professor at Jønkøping University College in co-operation with students from a lower secondary school in Sweden. The educational programme is built around a diary, written by the fifteen year old student Mia, who writes down her experiences from the year she attended 8th grade at school. The target group for this educational programme is students in the 7th or 8th grade of lower secondary school. In other words young people of the age 13 through 15. The programme consists of ten lectures, of about 45-60 minutes. Each lecture starts with reading today’s text which consists of Mia’s experiences and is followed by assignments where the students are supposed to deal with the questions raised in the lecture. In contrast to 79

Mia’s Diary

many other educational programmes in drug education Mia’s Diary does not require special education of either teachers or students in advance of implementation. The programme package which is sent to every class consists of an instructive guide for the teachers, discussing the methods and the models used in the programme.

2 The theories behind the programme After this short introduction about the background of initiating this programme, I am going to give you a brief presentation of the theories behind "Mia’s Diary”. Intended as a preventive educational package, Mia’s Diary is based on the methods of active and experiential pedagogic. Gilbert Botvin describes the distinctive characteristics of this approach to teaching as follows (Botvin, 1986): • Firstly the aim is to get children to recognise the factors that influence their substance-related behaviour. • Secondly the aim is to provide practice in social skills and to get the children to work actively against adverse effects. This will help the children to make the right choices in situations involving drugs and alcohol. The aim ultimately is, to support the process of identityformation and to encourage young children to assess the choices they have made. • And thirdly this method of education is characterised by the aim of increasing the awareness of schoolchildren about their own and others’ values. The purpose is to set right any misconceptions about the behaviour and norms of others and in this way to prevent these from being adopted by the schoolchildren as the basis of their own behaviour. When describing the theoretical model more specifically, I will quote the main author of the material, Jørgen Svedbom and Anders Stymne, executive secretary in the Nordic Committee on Narcotic Drugs. As mentioned earlier Mia's diary is intended for students 13 through 15 years of age, which corresponds to the seventh through eighth year in school in Scandinavian countries; in Great Britain to the fifth or sixth year of secondary school. At this age young people usually make their debut in the world of alcohol, in the sense that they start drinking alcohol outside their homes and often with their friends. As young people will make their own experience with alcohol during this period, which coincides with their high school years, school education on 80


alcohol should be aimed at the students’ own behaviour in relation to alcohol. A model that often has been used for this type of education is the so called "KAB-model": Knowledge → Attitudes → Behaviour This model is based on the assumption that information on the dangerous effects of alcohol will provide the students with knowledge that will create a negative attitude towards alcohol, an attitude that, in its turn, affects their behaviour so that they will drink less or abstain entirely from drinking alcohol. Research does however not give strong support for the function of this model. Gaining knowledge within an area such as drugs, does not automatically affect attitudes and behaviour (Bagnall, 1991; Thorsen, 1988). It seems that the experience, attitudes and values that students already have, will act as a kind of "filter" letting information through that support and strengthen their own views, while other information is filtered out. If the limited time available in schools for teaching about drugs is going to be able to contribute to change attitudes, values and behaviour of students, the information about alcohol and drugs must be rounded out by active work on attitude and values through behavioural training. Mia’s Diary is therefore partly built upon the suggestions given by Abrams, Garfield, and Swisher in "Accountability in drug education" from 1979 - where the authors suggest that the aim of drug education, in addition to giving information about alcohol and drugs, should be complemented with the following four goals: • to increase the ability to make decisions • to increase the ability to clarify one's own values • to increase self confidence • to increase participation in leisure activities which provide an alternative to using drugs

81

Mia’s Diary

One of the requirements placed on this teaching material is that it should be action oriented and provide competence to take action. This competence can be placed at various levels: • Partly that the lessons contain or lead to actions that are goal oriented and affect the circumstances that are studied. • Partly to develop readiness to act, so that the students can act consciously, when they later in life get into situations such as those described in the teaching materials. • Partly that teaching in school will increase students’ self confidence to the extent that they will have the will and ability to influence both their own lifestyles and the sociological development in contemporary society. A prerequisite for conscious and rational action is an awareness of the conditions one wants to affect through one's own actions. Awareness naturally requires knowledge, but it also requires that the knowledge is related to one's perspective of life. This comprises components such as the perception of the future and of social circumstances, experiences of how things interact, an overview and possibility of being in control, a sense of responsibility, complexity and conflict tolerance, etc. The methods used and the manner of working on the assignments given in connection with Mia’s Diary; like role play, value clarification, analyses, forum play and group discussions have all been chosen to coincide with the pedagogical theories presented above, but also considered recommendations that have emerged from research on young people. Regardless of the manner of working and methods used in school, education aiming at influencing attitudes, values and behaviour, should involve four steps or parts, namely theory, experience, value and action. Mia’s Diary has been compiled including all four parts and the objectives of this educational programme can be summed up in the following main points: • The first main objective is to increase the students’ knowledge of alcohol and the effects of alcohol use This objective is taken care of in the so called "theory part" of the programme. This part consists of facts about alcohol including effects as well as the alcohol policy in various countries. Particularly consequences of consumption and related damage to the body are stressed. 82


• The second main objective is to take advantage of and use the students own experiences This objective is taken care of in the so called "experience part" of the diary. Through practical studies, educational visits, group discussions and drama/role plays, this part tries to focus on the students’ own experiences concerning use of alcohol and drugs. • The third main objective is to increase the students’ self knowledge as well as to increase the awareness of their own and other people's attitudes and values and thereby reveal the majority fallacy This objective is taken care of in the so called "personality part" of the diary and in the assignments through discussions, group practice and studies. The aim is to increase the students’ insight into the standards one lives by, how one acts and what consequences one's actions could have. The concept of "majority fallacy" can be defined as an incomplete or false interpretation of opinions held by the "other group members". The Norwegian psychologist Ragnar Hauge holds that false impression of other group members easily arise in groups with loose structure and superficial relationships between members. The others - of whom in fact little is known - appear to be more lawless and non-conformist than they actually are. Based on this fallacy, a fictious norm pressure develops and the most poorly integrated will try to live up to the norms and values they believe are held by "the others". This theory was investigated by Hauge in an empirical study of alcohol consumption in adolescent boys and he reported that the boys exaggerated the others’ drinking habits and he concluded that this false notion most probably leads to a perceived pressure to drink alcohol (Hauge, 1970). • The fourth main objective is to develop the students’ readiness for action in situations where drinking of alcohol occurs This objective is taken care of in the so called "practice part" of the diary. Through various types of individual or group work the students are stimulated to imagine situations where alcohol is involved and to develop strategies to handle this situations in a satisfactory way.

3 The evaluation approach After this brief introduction to the educational programme and the theoretical perspective I'm going to say something about the part of the project which I have been engaged in, namely the evaluation of Mia’s Diary. 83

Mia’s Diary

When the Nordic Committee on Narcotic Drugs took the initiative to develop an educational programme, they certainly wanted evaluation to sum up the experiences with the project in this rather unique situation where a similar programme was introduced simultaneously in four countries. The Nordic Council for Alcohol and Drug Research (NAD) was put in charge of co-ordinating this evaluation project. NAD decided that each country should conduct separate evaluations, but that similar questionnaires should be used to enable the preparation of a joint report. In order to assess the impact of the programme on the target group the evaluation was designed as an outcome evaluation. The students where confronted with a questionnaire at two occasions: one as a pre-test three weeks before they had the first lesson in Mia’s Diary and one as a post-test three weeks after the programme had been finished. The two questionnaires were almost identical except for the last seven questions included in the post-test scheme, where the students were asked to give their opinion on the Diary and the assignments. We also worked out a questionnaire to the teachers responsible for the implementation of the programme in the classes. This questionnaire was to be filled in after the programme had been finished. Norway was the only country that used a control group in this study. In addition to the outcome evaluation carried out in all the participating countries the Norwegian evaluation report also includes a field study. Originally this field study had been planned to be quite extensive but this became quite a problem to accomplish. Many of the schools asked to participate hesitated and gave the impression that they certainly would not like to be "controlled by the authorities", as one of the headmasters put it. Anyhow four headmasters gave us a permission to visit their schools and to observe the implementation of the programme in the classrooms. In spite of these difficulties the possibility to observe in the field gave us a valuable supplement to the information gathered from the questionnaires. The subjects for this study were students in the 7th and 8th grades in Norway, Denmark, Finland and Sweden, who went through the educational programme "Mia’s Diary" during the spring and autumn term of 1994. A sample was drawn from the participating schools of each country. The final sample consisted of 70 classes including 1563 students. (553 students from Norway, 210 in Finland, 472 from Denmark and 327 students from Sweden) The total study population consisted of approximately equal numbers of males and females.

84


In Norway, Denmark and Finland the data were gathered during the spring term of 1994. In Sweden the gathering took place in the autumn term of 1994.

4 A brief presentation of the main results Based on the main objectives of the educational programme the Nordic evaluation addressed nine questions : • To what extent have the students acquired factual knowledge about alcohol and drugs through their work with Mia’s Diary? The results indicate a fairly strong increase in factual knowledge among the students during the project period. This is evident for each and all questions concerning facts about alcohol and drugs. This result is not unexpected since other studies also have shown that most programmes concerning alcohol and drugs are successful in increasing knowledge among the students (Baklien, 1993; Botvin 1986; Hansen, 1992). The increase in the factual knowledge level seems to be a bit higher in Norway and Denmark than in Sweden and Finland. • To what extent did the students’ attitude to alcohol and to the effects of alcohol change during the project period? We did not observe any significant attitude change among the students who went trough "Mia’s Diary". The majority of the students - around 60 percent - gave the impression of being favourable to alcohol in the pre and the post-test. In line with other studies, therefore not unexpected, we found that those students who had been drinking alcohol expressed more favourable attitudes to alcohol than students without such experiences (Baklien 1995; Hansen, 1992). • To what extent has the educational programme improved the competencies of the school children in situations involving alcohol ? There was no indication in the material of any noticeable improvement in the schoolchildren’s competence to deal with situations involving alcohol. In the contrary, the results in this respect were very similar at both point of measurement • To what extent did the educational programme have any effect on the amount of majority fallacies among the participating students? There was a tendency towards a decline of the amount of fallacies linked to the students’ perception of how their classmates would act in the situations sketched in the questionnaire. To some extent it seems likely that the 85

Mia’s Diary

students have acquired better knowledge of each other during the project period. In other words: what we did observe among the students was a decline in the amount of false interpretations of the opinions held by their classmates. • To what extent was it possible for the students to identify with the main character and the events described in the diary? Our data show that the objective to develop an educational programme where the students could identify with persons and events described, could be achieved. A great majority of nearly 90% of the students claimed that the situations described in the diary certainly could also happen to someone at their own school. Girls showed more positive attitudes than boys. The identification with the main character seemed to be somewhat easier in Norway, Sweden and Denmark than in Finland. • To what extent did the educational programme lead to discussions concerning alcohol and drugs outside of the classroom? By asking the students if they had been discussing alcohol and drug related themes more than usual during the project period, we wanted to develop a measure of the informal spread of the message in the educational programme. Nearly 30% of the students in our sample reported that they had been talking more with their friends about these topics and 20% stated that they had been talking more with their parents during the project period. The girls seems to have been discussing the message to a somewhat greater extent than the boys, a finding not unexpected since other studies have shown that educational programmes concerning alcohol and drugs have greater appeal to girls than to boys. Summing up we may conclude that the informal spread has been limited. One of the reasons might be - as a student from a participating class told us - that "when we talk about these topics we are usually not talking about accidents, harm to the body and pressure to drink but we talk about the last party, who in the class was drinking, how drunk we were, an so on." • To what extent did the educational programme create commitment among the students? Based on the feedback we got from the teachers both in the questionnaires and through informal conversation we concluded that the educational programme did succeed in making the students take an active part in the accomplishment of the programme. The classroom observations made in Norway support this conclusion. In addition we found that 65% of the students thought that teaching and assignments had been interesting. Again the girls seems to be more positive than the boys. Most positive were the 86


Norwegian girls: where nearly 90% claimed that they did find the educational programme either very interesting or fairly interesting. • How did the educational programme work in pedagogical terms? The teachers gave the programme a very good testimonial, but there are three conditions that seem to have caused problems in the practical work with the Diary and the assignments. In the first place many of the teachers seem to have had problems to accomplish each lecture within the proposed 45 - 60 minutes. Many teachers claimed that interesting discussions had to be finished too early. This lack of time also prevented the teachers from summing up lectures and discussions, which was mentioned by many teachers as a problem. Secondly some of the teachers reported problems with discussions in the classrooms concerning personal issues. This problem seems to have been present in situations where teachers did not possess sufficient background knowledge about their students prior to starting the programme. The teachers who reported this problem all claimed that they had not known the students well enough to be able to initiate such discussions. The third problem mentioned concerns the continuity in the course of the programme. In some of the classes as many as six teachers had been involved in executing the programme which turned out quite problematic for teachers and for students. The best results were reported from classes where only one teacher had been involved in the programme and in these classes none of the teachers reported problems in discussing personal issues. • Did the educational programme work differently in the Nordic countries? Our data show that the educational programme may have been a bit more adapted to the Norwegian students than to the students from Denmark, Sweden and Finland. In Norway 80% of the students, reported that they did find the programme very interesting or fairly interesting, compared with 62% of the Danish students and 59% of the Finnish and Swedish students. All in all, however, this joint Nordic project can certainly be regarded as a success.

5 Concluding remarks Comparing the evaluation results with the main objectives of the educational programme "Mia’s Diary", there is little evidence to support 87

Mia’s Diary

that the programme had any measurable effects on the students’ attitudes and behavioural patterns concerning alcohol. What we did find was a significant increase in the students’ knowledge of facts about alcohol and clear evidence to support the assumption that the educational programme did offer the students an opportunity to identify with the main character "Mia" across the frontiers in the Nordic countries. Thus we are dealing with an educational programme that reaches the target group very well, that is easy to execute and in addition seems to be appreciated by students and teachers - but that doesn't "work" in the sense that measurable effects on attitudes and behaviour can be observed. Failing to obtain measurable changes in attitudes or behaviour concerning alcohol can be explained through several reasons. One possible reason lies in the education material itself, which raises questions about the validity of its theoretical foundation. This, however, can hardly be an exhaustive explanation. Numerous studies have shown that other similar packages of teaching methods applying the same approach have produced good results (Hansen, 1992, Klepp et al. 1993). In one respect however Mia’s Diary is different from these other courses. This difference has to do with the level of pupil participation. The best results have been achieved with a two-stage education model in which so-called peer leaders are first introduced to a certain subject area and then the peer leaders go on to teach what they have learned to their classmates (Baklien, 1995; Wilhelmsen, 1994). It is perhaps because of the teaching method relying heavily on the teacher that "Mia’s Diary” failed to produce visible changes in pupils’ behaviour. Another reason why the evaluation did not demonstrate any significant changes in attitude or behaviour concerning alcohol may lie in the method of evaluation applied, i.e. in the assessment approach. It is possible that relevant changes did happen, but that the questionnaire just didn’t reveal them. The time span within which the questionnaires were administered also may have been of importance to explain why we failed to demonstrate a relevant impact of the material. Changes in behaviour and attitudes are slow processes and the identification of such changes requires repeated measurements. Within the confines of this project there were definite limits to assess this kind of effect. Other studies have shown that educational programmes not yielding promising immediate results in follow-up studies show positive results several months after the intervention (Hansen, 1992). Thus we should be careful to judge the programme’s effectiveness merely on the background of short term effects.

88


A final factor that may limit obtainable results in a prevention programme like Mia’s Diary, deals with the fact that school based prevention projects alone - no matter how well designed - may not be sufficient. As Partanen and Montonen put it, it is evident that individual changes in attitudes and behaviour are heavily influenced by social relationships and the cultural environment, which implies that the real target of prevention efforts should be broadened and include cultural patterns of a community or even the whole society (Partanen & Montonen, 1988). This, however, does not mean that school based prevention is wasted, but it emphasises the need of well co-ordinated efforts involving educators, parents, community leaders and the mass media. The difficulties of prevention are neatly captured in one of our informants’ questionnaire responses: "It was fun working with Mia’s Diary, but I think it is difficult to influence peoples attitudes. If people have made up their minds about something you just don’t go on and change it."

6 References Abrams, Garfield, Swisher : Accountability in Drug Education, The Abuse Council, New York, 1979 Botvin, G.: Substance Abuse Prevention Research: Recent Developments and Future Directions. Journal of School Health, 56, 9, 369-374, 1986 Bagnall, G.: Educating Young Drinkers. Routledge, London, 1991 Baklien, B.: Two Step Drug Education in Norway. Journal of Drug Education, 23, 2, 171 -182, 1993 Baklien, B.: Ungdommelig entusiasme møter skolehverdagen - Totrinns rusundervisning - en evalueringsrapport. Rapport nr. 1. Institutt for sosiologi, Universitetet i Oslo, 1995 Hansen, W. B.: School-Based Substance Abuse Prevention: A Review of State of the Art in Curriculum, 1980-1990. Health Education Research, 7, 3, 403 - 430, 1992 Hauge, R.: Kriminalitet dom ungdomsfenomen. Universitetsforlaget, Oslo, 1970 Klepp, K. I.: Alkoholundervisning blant ungdomsskoleelever. Tidsskrift for den norske lægeforening, nr. 2, 113, 202-205, 1993 Partanen, J.; Marjatta M.: Alcohol and the Mass Media. Euro Reports and Studies, 108, WHO, Copenhagen, 1988

89

Mia’s Diary

Thorsen, T.: Opplysning som forebyggelse inden rusmiddelområdet. Alkohol og Narkotikarådet Skriftserie nr. 12, 1988 Wilhelmsen, B. U.: Effekter av forebyggende undervisning om alkohol for sjuendeklassinger. Stoffmisbruk 4/92, 25-27, 1994

contact address: Line Nersnæs Norwegian Ministry of Health and Social Affairs Moellergt. 24, P.O. Box 8011 N-0030, Oslo, Norway Tel: +47 22 248655 FAX: +47 22 248656 e-mail: [email protected]

90


Evaluations of Substance Use Prevention Programmes: Implications for Illicit Drugs Mark Morgan St. Patrick’s College, Dublin, Ireland

___________________________________

paper written for COST-A6-WG 2 June 1995

Thanks are due to Maria Xesús Froján Parga, Alfred Uhl, Cathal Higgins and Joel W. Grube for comments on earlier drafts

91

Evaluations of Substance Use Prevention Programmes

Contents 1 Introduction and abstract ......................................................................... 93 2 Conceptual foundations ........................................................................... 93 2.1 The knowledge/attitudes model........................................................ 94 2.2 The decision making model.............................................................. 94 2.3 Social skills/refusal and assertiveness training ................................ 95 2.4 Normative education approaches ..................................................... 96 2.5 The alternatives approach................................................................. 97 3 Programmes and their effects .................................................................. 98 3.1 Cigarettes .......................................................................................... 98 3.2 Substance abuse prevention phase 1: facts and scare tactics ......... 102 3.3 Social influences, meta-analyses and faulty evaluations ............... 103 3.4 Recent stronger evaluation designs ................................................ 105 3.5 Recent studies of training and generalisation of social skills ........ 109 4 Risk-focused interventions .................................................................... 111 4.1 Identification of high-risk factors................................................... 111 4.2 Reduction of risk factors ................................................................ 114 4.3 Difficulties with risk-focused approach ......................................... 115 5 What has been learned from community-based programmes? ............. 118 5.1 Evaluation issues in relation community programmes .................. 119 5.2 Recent examples of community-based programmes...................... 119 6 General issues relating to evaluation of substance-use prevention programmes ........................................................................................... 121 6.1 What kind of effects are reasonable to expect of substance use prevention programmes? .............................................................. 121 6.2 Do adolescents’ feelings of invulnerability reduce the impact of prevention programmes? .............................................................. 122 6.3 Is it necessary to modify programmes to suit particular groups? .. 122 6.4 Are there any indications that there may be harmful effects of various programmes?.................................................................... 123 6.5 Issues of evaluation methodology .................................................. 123 6.6 Threats to internal validity.............................................................. 125 6.7 What are the aims of programmes? ................................................ 125 6.8 Measurement of outcomes in evaluations ...................................... 126 6.9 Practical issues of implementation and the effects on evaluation . 127 7 General conclusions............................................................................... 128 8 References.............................................................................................. 130

92


1 Introduction and abstract Approaches to primary prevention of substance misuse vary greatly between countries. A review of the work in 29 countries shows that while little systematic research was being conducted, a variety of educational and information programmes were in places in many countries (Smart et al., 1988). This review examines the extant evaluations of substance use with particular regard to the implications of the findings for prevention of illicit drug use, concentrating especially but not exclusively on school programmes to prevent substance use. The rationale for current approaches is examined together with evidence on the effectiveness of each approach. It is shown that approaches relying exclusively on information have not been successful while those approaches based on social skills seem somewhat more promising. However, many of the studies supporting these claims are poorly designed and even the reviews of the literature are sometime difficult to interpret. On balance it would seem that programmes targeting cigarette smoking have enjoyed relatively more success than others. While there is great enthusiasm for community-based programmes, their outcomes are especially difficult to evaluate rigorously. Several general issues need attention like the nature of the aims of substance prevention programmes and the extent to which they should be based on generic skills or be specific to particular circumstances and substances.

2 Conceptual foundations Educational approaches to substance abuse prevention frequently fail to specify the nature of the behaviour change that is supposed to take place as a result of the intervention. In many instances, the model of behaviour change is implicit or represents a gross over-simplification of the state of knowledge regarding the factors that impinge on behaviour change. However, it is fair to say that many current approaches are based on one of four conceptualisations of behaviour change, viz., • the knowledge/attitudes model, • values/decision making model, • self-efficacy/social competency model and • the normative education approach. The relatively novel risk-focused approach is in not a model of change but rather an approach that can encompass any model of behaviour change and is therefore treated separately. Below we consider these four models of behaviour change with particular emphasis on the rationale for each.

93


2.1 The knowledge/attitudes model This model suggests that if knowledge about the negative consequences of substance use is assimilated, then less favourable attitudes towards use of substances should ensue. In turn, these negative attitudes should result in a decreased likelihood of substance use. While this model of behaviour change had a particularly strong influence on research in the fifties and sixties and while it has an intuitive plausibility, the recent social-psychological literature has shown that this view provides at best an incomplete picture of the events determining behaviour. The greatest difficulty for the model is that attitudes and behaviour are less than perfectly related. Two factors seem to be especially important in this regard. First, attitudes are only one of the influences on behaviour. Thus, actual substance use is influenced by an array of other variables (normative pressures, etc.) in addition to the attitude to that substance. The attitude substance use relationship was examined in a study by Grube et al. (1986) who found that while attitudes were moderately related to intentions to smoke and actual smoking, a variety of social influences gave a more comprehensive understanding of the adolescents’ decision to smoke. The second point is that attitudes can be expected to change behaviour only in those cases where there is a correspondence between the measured attitude and the specific behaviour. In other words, changes in overall attitude to substance use may not change the specific intention that a person may have to drink on a particular occasion. It may be easier to bring about negative attitudes to substances than it is to change attitudes in relation to specific personal use of such substances.

2.2 The decision making model This approach focuses on the individual and attempts to increase self-awareness of a range of values and the way in which substances can serve in promoting or preventing the fulfilment of those values. The central idea is to prevent substance use through a self-examination of values. Essentially, young people are required to ask themselves whether this behaviour is consistent with a variety of beliefs and values, which they themselves regard as important. This approach has been used in a variety of other contexts, including health education, moral education, and interpersonal problem-solving.

94


While some evaluations of the decision making model have been quite dismissive (e.g., Moskowitz, 1989), there may be considerable value in exploring the potential of this approach with some students. However, it may be relatively more effective with an older age group or with people who are relatively bright. Certainly there are important educational philosophical grounds in having such a component in a programme in the sense that it is essentially non-directive and contrasts with the didactic style of older approaches. Of the various decision making models, values-clarification is still one of the most popular approaches. While there are considerable variations of this model, the different versions are designed to help students to discover values that will provide clear guidelines for their behaviour. However, it has to be conceded that the review of the available studies by Lockwood (1978) concluded that ... "it is not warranted to claim that values clarification positively affects the interpersonal relationships of students or that it contributes to reduced drug usage" (p.359).

2.3 Social skills/refusal and assertiveness training One popular models assumes that individuals develop problems with substances because they lack particular social skills. This model exists in various forms. One of the most common involves teaching of skills to resist peer pressure/media pressure. In this approach young people are taught how to identify the various kinds of influences that are brought to bear on them in interpersonal situations and in the media and to acquire a repertoire of skills to withstand these influences. Many of the social skills programmes have been heavily influenced by assertiveness training. While the components of assertiveness training vary considerably between programmes, they contain elements of • identifying rights, • recognising manipulative strategies, • identifying a strategy to cope with such pressures and • making use of this strategy without resorting to aggression. In general, social skills techniques frequently involve active involvement by the participants. Thus, rather than learning about such strategies, students actually enact these in role playing and modelling situations. In addition, many approaches involve peer leadership as a critical component of presentation of the programmes. 95


Because of the dominance of social skills training, evaluation of the effectiveness of this approach is central to an understanding of the effectiveness of prevention programmes in general. Frequently, while social skills approach is central to a programme, other features are also included. Thus, it is not uncommon to find a programmes that mainly based on the acquisition and performance of social skills but which also involves knowledge and attitude change as well as decision making skills. In fact, eclectic approaches like this are perhaps the dominant in recent times.

2.4 Normative education approaches These approaches take as their point of departure the consistent relationship found between normative support and substance use. In several studies it has been demonstrated that young people who drink, smoke and use illicit substances perceived a high level of social support for these behaviours including widespread use among same-aged peers. Thus, normative education curricula are designed to make salient to young people that the norms regarding substance use are conservative. The components often include the provision of evidence that substance use is not as widespread among peers as they may think, encouragement for young people to make public commitments not to drink, the depiction of alcohol use as socially unacceptable and the use of peer leaders to teach the curriculum. An example of normative education is the work of Hansen and his colleagues (Hansen & Graham, in press). This work has shown that a programme designed to correct the erroneous perceptions among students about the prevalence and acceptability of alcohol, actually deterred the onset of use of drinking. Specifically, it was shown that normative education reduced the incidence of drunkenness and the prevalence of alcohol problems among students in Junior High Schools in California. Furthermore, Hansen and Graham have demonstrated that normative education was more effective than resistance skill training in reducing the onset of drinking behaviour. While these initial tests of the effects of normative education are promising, some considerations about the nature of peer influence are worth considering. First, the available evidence would suggest that information about same-age peers should have relatively little influence compared to the closer peer group (friends and the "best friend" see Morgan & Grube, 1991). A second consideration is the existence of a pervasive bias that tends to cause people to see their opinions and 96


behaviours as more typical than they actually are. The very large literature on this "False-consensus effect", has shown that such beliefs are not easily modified and may have a deeper significance for the individual who holds them. Thus, it may well be that the "establishment of conservative norms" may indeed be an effective means of reducing substance use, the real difficulty may well be in how such norms can be established. On the other hand, the normative education approach has one important implication for teachers, parents, etc. In the writer’s experience, many efforts to combat the onset of drinking often begin with the information that the problem in question (i.e., underage drinking) is widely prevalent. This may unwittingly undermine any subsequent benefit that the advice/attempt to persuade may otherwise have had.

2.5 The alternatives approach This model is based on the idea that if young people have other goals, activities and pursuits they are less likely to be involved in substance use. Swisher & Hu (1983) are often cited as the main proponents of the alternatives approach. There is indeed considerable evidence that adherence to certain goals, especially conventional goals is associated with lesser use of various substances (Grube & Morgan, 1986). A number of studies have found that a commitment to school and associated activities is likely to have a restraining influence. On the other hand, a recent study carried out in Brazil by Carlini-Cotrom & Aparecida de Carvaiho (1993) points to a different conclusion. They examined the association between systematic participation in extracurricular activities and consumption of alcohol and other substances among over 16,000 high school students. No association emerged between attendance in artistic, community and sports related activities and the use of such substances. However, a weak relationship was found between substance use and attendance at religious services. The provision of alternative activities may take several forms, e.g. opening a youth centre that provides alternative activities for young people in the community, outdoor activities, games, athletics etc. Other alliterative activities include participation in spiritual activities, yoga, transcendental meditation, and sensitivity groups. While a greater many of the school-aged programmes are school based, other programmes have been developed at sites including public housing developments, church groups and recreational centres.

97


While the alternatives approach is plausible, it seems not to have been used systematically in prevention research. If often guides the implicit assumptions of schools which emphasise games and sporting activities, i.e. if adolescents are involved in games they will have less time for experimentation with various substances. As a result, formal evaluations of this approach are hard to come by. Nevertheless, it deserves more serious attention in the literature than it usually receives.

3 Programmes and their effects Below we first consider the evaluations of programmes to prevent cigarette smoking and subsequently look at the effectiveness of those programmes that target other substances. This division is convenient for two reasons. Firstly, some programmes have specifically targeted cigarette smoking only while many others are aimed at preventing the use of a range of substances especially alcohol and marijuana. Secondly, while the programme focusing on smoking have had at least modest success, those aimed at other substances have more questionable outcomes.

3.1 Cigarettes Approaches to Prevention of Smoking. In contrast to the apparent limited effectiveness of interventions to prevent the onset of alcohol and illegal drug use, there are claims that many of the interventions focusing on cigarette smoking are much more effective. As in the case of other substances, the prevention programmes can be classified according to the strategy they take in attempting to influence young people. Thus, some programmes attempt to prevent smoking by presenting young people with facts about tobacco, usually about the physiological and behavioural effects. Persuasion attempts differ from knowledge approaches in that actively attempt change expectancies and normative beliefs regarding cigarette smoking by using principles of attitude change. Inoculation and social skills strategies take as point of departure that many of not most adolescents have unfavourable attitudes towards smoking that they have acquired from their parents and relevant others. However, because these attitudes have been accepted unquestioningly, they may not be supported by a structure of well reasoned underlying beliefs. The use of commitment is another method of stabilising intentions not to smoke. The assumption of this approach is that individuals are less likely to change their behavioural intentions if they have made a public commitment to them. Finally, the alternatives approach attempts to involve young people in activities that increase personal 98


growth and develop interests that are incompatible with smoking. Such alternatives might include sports or service activities such as clubs or community projects. Flay (1985) surveyed 17 evaluations of programmes delivered to students in grades 5-9. Most of these programmes involved several components, particularly information about cigarette smoking and especially about the negative social consequences and the short-term physiological effects. In addition, in most programmes children were taught to identify and resist the kind of influences (interpersonal and mass media) that were described above. Some programmes incorporated other features like communication skills, coping strategies and enhancement of self-esteem. Most of the programmes that were reviewed by Flay were conducted by a specially trained person during 4-20 class sessions. In a number of studies the programme was delivered by either same-aged peers or by a student who was somewhat older than the class students. Finally in a small number of cases the class teacher was in charge of the delivery. While most of these studies have yielded some positive effects there are important gaps in our knowledge. In some cases the positive effects were confined to students who had never tried smoking while in others the effects were found only for existing smokers. The other problem is in working out which feature of the programmes has been responsible for the positive outcome The review of smoking prevention programmes by Best et al. (1989) is comprehensive, conceptually rich while being somewhat unduly lenient in relation to methodology. The review covers 25 published study of which about two-thirds indicated positive results. However, it is worth noting that the nature of the positive results vary greatly from study to study. Many of the study simply indicate that there were statistically significant differences between the experimental (programme) group and the comparison group. Others present results indicating that the prevalence of current smoking has is lower by a specific percentage than was the case for the control group. Still others have shown that the programme in question may have delayed the onset for a particular length of time.

99


Nevertheless while there are not many consistent effects when looking at particular studies, the general point emerging from the 25 studies taken together is that the programmes have had considerable success. This is especially the case if one considers that the important behavioural effects have been found in many studies and that in some cases the effects were demonstrated to have lasted two years or more after the intervention. However, while this success has to be acknowledged, there are important shortcomings in our insights into why this success has been achieved. It is difficult pin down this success to • type of curriculum, • dimensions of the programmes or • target population. A great many of the programmes that have been reviewed by Best et al. focus on the various social influences that affect smoking and are designed to develop skills to resist pressures from peers and media. However, in addition many programmes provide information on the consequences and thus contain elements of persuasion. Furthermore, many convey information about prevalence of substance use and thus may involve elements of normative influences. Some of the programmes focus exclusively on smoking while others targeted other substances particularly alcohol and marijuana. Furthermore, while the majority of programmes are aimed at preventing young people from beginning to smoke, some are also aimed at stopping young people who have already started and thus involve smoking cessation as well as smoking prevention. The programmes vary greatly in terms of their duration and intensity. In some cases the delivery took place within one week while in others the programme lasted over a number of years. Sometimes, the programme was tightly packaged and was designed to be "teacher-proof", that is, it was to be delivered in specific ways with no allowance for circumstances whereas in other cases flexibility was built into the provision. There is also great variation in teaching methods. While the very important emphasis has been on active learning, some elements of programmes are based on didactic methods. The association between programme effectiveness and the individual characteristics of programme participants is of interest. Obviously it would be worth knowing whether the effects are greater for young people who have already experimented with cigarettes as opposed to those who had not. There are other aspects that might also be of importance e.g. the 100


effects of gender, social background, and a variety of relevant attributes like independence. The difficulty here is that there seems to be uniformly positive but modest beneficial effects across all these differences relating to conditions, dimensions of programmes and individual differences. The fact that no particular feature is especially likely to bring about relatively greater (or lesser benefits) makes inferences about the effective component somewhat problematic. Thus, the suggestion that the programmes against smoking are effective only because societal norms are also changing is something that is worth considering. It is especially unfortunate that almost all of the rigorous evaluations of smoking prevention programmes have come from countries in which the normative climate of opinion was beginning to become hostile to smoking cigarettes. Recent Evaluations of Programmes. Given this limitation in the work reviewed by Best at al., a recent meta-analysis of the relative effectiveness of various programmes is of particular interest (Bruvold, 1993). This meta-analysis was performed on the results of studies published in the 1970s and 1980s evaluating 94 anti-smoking interventions. Studies with weak research methods were screened and omitted. The resulting 70 programmes were classified according to the basic prevention methodology. Behavioural effects were greatest for interventions with a social skills orientation, moderate for interventions with a developmental or social norms orientation and lowest for interventions with a "knowledge" orientation. Attitudes effect sizes followed the same pattern but knowledge effects were similar across all our kinds of approach. The recently reported study by Sussman et al. (1993) gives the findings of the Project towards No Tobacco Use. This is a 5 year school-based tobacco and use prevention and cessation project. The programme taught refusal skills, awareness of social misperceptions about tobacco use, and misconceptions about physical consequences. Nearly 7000 7 th grade students participated in the project. Results suggested that physical consequences information could be used to compose a curriculum that is as successful as a social influence programme. A normative social influence programme was not as efficacious for tobacco use prevention as are other types of programmes, except of prevention of tobacco onset. The same tobacco use prevention programming could be successful when targeted to use of both cigarettes and smokeless tobacco in the school-based context. One important factor seems to be have been overlooked in the consideration of various programmes to combat smoking viz., the scope 101


and intensity of the programme. This may be illustrated by reference to two recently published one of which produced negative results (Nutbeam et al. 1992) and the other which produced positive findings (Morgan et al. 1994). The study by Nutbeam et al., lasted for a relatively short time and seemed to have no additional support features. On the other hand the programme described by Morgan et al. had several support features involving cross curricular components, parental support, awards for the commitment not to smoke as well as social skills training and normative support. Thus, it is hardly surprising that the results are in such strong contrast with each other.

3.2 Substance abuse prevention phase 1: facts and scare tactics In a review of the evaluations of prevention programmes to combat substance use, a convenient starting point is a consideration of the programmes of the 60s and early 70s which placed a heavy reliance on teaching "the facts" about various drugs. For example, in some countries there was a legal obligation on schools to teach about the "...nature of tobacco, alcohol and narcotics and their effect on the human system" (Michigan State School Code, 1955). While instances of miseducation (that is excessive exaggeration of the harmful consequences of particular substances) are difficult to document, there is little doubt that teaching the "facts" may have been a contributory factor in such occurrences. The effects of such exaggeration was to undermine not only the claims made in relation to the substance involved but also the credibility of the teachers and school in relation to other worthwhile efforts to influence young people. Leaving aside the scare tactics, real questions arose as to what precisely were the effects of teaching facts about the various substances. A study by Stuart (1974) is instructive in this regard. He examined the effects of a 10 session fact-oriented drug programme which was offered in two formats (student led to teacher led) by means of self-report questionnaire measuring drug information, attitudes relevant to drug use as well as reported use. Results indicated that relative to controls subject receiving drug information did indeed increase their knowledge about drugs. However, their anxiety about drugs also decreased and more significantly their use of alcohol and marijuana and LSD was greater than for controls. The results of Stuart’s study also showed that while knowledge about drugs and decreased anxiety tended to predispose young people towards use, other factors not measured in the study seemed to be much more 102


important. In other words, cognitive factors are only a part of the influences and presumably only moderately influential in relation to prevention.

3.3 Social influences, meta-analyses and faulty evaluations The next wave of studies tended to have broader theoretical bases (as in those described above), used a wide of strategies and seemed to be showing some success. A number of reviews which examined these various evaluations used meta-analysis to give an overview of the results. Meta-analysis is a quantitative way of aggregating results from several studies and involves calculating an effect size for each study. Frequently effects sizes are calculated in terms of differences between experimental and control groups. calculated in terms of standard deviations. The meta-analysis by Tobler (1986) was an attempt to include evaluations of all available prevention programmes that were available in the mid-eighties. The inclusion criteria were as follows: • Availability of quantitative measures, • Use of a comparison or control group, • Programme participants being in secondary school (from Grades 6 to 12), • Primary prevention of substance use to be the goal. This meta-analysis categorised the outcome measures as follows: • knowledge relating to various substances, • attitudes and values relevant to various substances, • self-reports of use of various substances, • skills relevant to withstanding pressures to use drugs, • outcome measures like arrests and hospitalisations. Furthermore the programmes were categorised into various types as follows: • Those importing facts only, • programmes aimed at increasing personal growth, • programmes attempting to equip adolescents with the skills to withstand peer pressure, • alternatives approach, • combinations of these approaches. The main features of the results can be summarised briefly. Firstly, the biggest effects were on knowledge, while other measures showed average effect sizes which were substantially smaller than was the case for knowledge. Secondly it emerged that programmes designed to equip 103


adolescents to resist peer pressure were much superior to the others. Whether considered together or singly the other approaches were less satisfactory than the social skills approach on all measures but especially in relation to use. Thirdly, the duration of programmes did not show any relationship with magnitude of effect. Fourthly, the results for the various substances seemed to be quite similar. Finally, the alternatives programmes seemed to be at least moderately successful with adolescent who were at high risk. While the general thrust of the Tobler’s review is plausible, there are a number of points that are worth noting. Firstly, the study utilises the information from the results of all evaluations regardless of the rigour with which these were carried out. Thus, if there were no control group, extrapolations were made on the basis of other data sources. This procedure is extremely dubious. Secondly, it is noteworthy that of the 143 programmes, the majority do not indicate the extent to which the implementation of the programme was successful. This is a serious limitation of the studies as a whole and taken with the finding that duration of programme was not related to effect, is something that leaves a certain feeling of unease. Thirdly, great efforts are made in this review to make sense of the pattern of results. Thus, we find that certain programmes give particular results with certain kinds of populations and only when measurements are taken in certain ways. While these outcomes may be plausible, the number of studies involved inevitably diminish to a point where the results are reflecting the outcomes of a very small number of programmes. The real crunch is the extent to which such outcomes give predictions that can be replicated. Another relatively recent approach to evaluation of various approaches uses a correlational design and attempt to find out whether those schools that use certain approaches are more likely to have more different mean levels of substance use than do other schools. In other words, instead of manipulating programmes, the strategy involves examining the effects of naturally occurring variations between schools. This is the approach taken by Eiser et al. (1988), who report a ten school survey of smoking among young people in England. The students were asked to describe the context of the smoking education that they received, that is whether it was science-based or situation within personal and social education. The results indicated that those students who recalled that smoking education took place in the context of social and personal education, were in fact less likely to smoke than the others, thus leading the authors to proclaim that "social education is good for health".

104


A similar strategy has been used by Regis et al. (1994) who found that who collected drinking information from over 4000 adolescents in 30 schools. Their results initially confirmed the thesis put forward to by Eiser et al., viz., that an approach through science education is less good than through social and personal education. However, they also found evidence that other effects may be at work having to do with school "ethos"; effects that may be more significant than the specific content of programmes. While acknowledging that the substantive point about the limitations of scientific and factual approaches is probably accurate, this correlational methodology is suspect on several grounds. Recollections of what kind of programme that adolescents experienced may itself be influenced by whether or not one is a substance user. For example, for several years it is known that smokers are more likely to perceived a much greater level of support for smoking than actually exists. Another possible confounding factor is that particular programmes may be a response to a problem rather than being a cause. For example, remedial programmes for reading are most frequently found in schools where children have reading problems, that is where they are needed. Yet it would hardly be appropriate to attribute relatively low reading achievement to such programmes. This may account for the finding by McAteer (1991) in a study in Northern Ireland which found that a higher percentage of current drinkers claimed to have received alcohol education than did the non-drinkers

3.4 Recent stronger evaluation designs A number of studies carried out over the last decade have been characterised by evaluations designs that were much stronger than in the earlier studies. Many of these studies have targeted several substances and have frequently have included a number of grade levels. For example, Moskowitz et al. (1984) examined the effects of a programme based mainly the values/decision making model and found an immediate positive effect on use of alcohol and marijuana use among 7th grade girls. However, it was also found that the effect did not persist one year later among this group. Furthermore, there were no significant effects among 7th grade male students nor among either male or female 8th graders. Botvin (1987) has described and evaluation of a comprehensive programme of alcohol and drug education that involved information, social skills and decision making. A 20 session Life Skills training programme was delivered to 7th grade students either alone or followed by booster sessions in 8th grade and in 9th grade, by teachers or peer leaders. Ten schools were randomly assigned to one of these four experimental 105


conditions or a no-treatment control condition. The only systematic positive effects were reported for the peer lead programme delivered with boosters. In contrast there seemed to be systematic negative effects for the teacher-led programme with boosters. There were other anomalous effects in this study which are difficult to understand. Part of the problem may be the units of treatment, i.e. whole schools. Note that there are only 10 schools and when these are allocated randomly, there is likely to be considerable initial differences between the conditions. What this kind of study requires is baseline data are needed which would indicate whether or not there was an equivalence across conditions in relation to the key variables in the study. This matter frequently arises in studies of the evaluation of programmes for the prevention of substance use. Botvin et al. (1990) report a one-year follow-up data from this study. While the authors proclaim that the data support the view that the programme was highly effective, a close examination of the results show that significant behavioural effects were found only with a combination of peer led conditions together with additional booster sessions. While the effect in this particular combination is apparently substantial, it is not clear why only this condition was so effective and why there were no effects in other conditions. Furthermore, the flaws of the original design would have to be taken into account in considering the outcomes. Moskowitz (1989) describes an evaluation of two substance use education programmes based on the SMART project. One of these was based on the social skills approach and the other on a values-decision making model. Ten schools were randomly assigned to one of the two experimental conditions or to a control group. The results revealed a positive effect for the social influences programme but only on cigarette smoking and not on alcohol and illegal drugs. Furthermore this effect did not persist beyond the first year. The decision making programme showed now significant effects on any substance nor in any year.

106


Cherry et al. (1989) report the results of a four-country pilot study of alcohol education on young people. The countries involved were Australia, Chile, Norway and Swaziland. In each country 25 schools representing middle and lower class populations were randomly assigned either to peer led education, teacher led education or a control condition. While there were many components to the programme, the central feature involved interpersonal and social skills (including assertiveness), in order to equip students with skills to withstand pressures. Overall the students in the peer led groups reported significantly less use of alcohol than did those in the teacher led group. In addition, they gained more knowledge, had better attitudes and had fewer friends who drink at the post-test. However, it is interesting that the differences in relation to social skills did not correspond with the trend in the other outcomes making it difficult to interpret the findings. Ellickson et al. (1993) report an examination of the impact of Project ALERT on the intervening (cognitive) variables hypothesised to affect actual drug use. These variables included adolescent beliefs in their ability to resist, perceived consequences of use, normative perceptions about peer use and tolerance of drugs as well as expectations of further use. A survey of more than 4000 7th and 8th graders revealed effects of the programme for perceptions assumed to be linked to each target substance (alcohol, cigarettes and marijuana) across all subjects and for those at different levels of risk for future use. It seemed that the curriculum successfully reduced cognitive risk factors from each of the above categories for cigarettes and marijuana, indicating that social influence programmes can impact a broad range of beliefs associated with the propensity to use drugs. Leaving aside the fact that many studies have shown only modest relationships between these beliefs and substance use, the critical outcome of this type of study is of course, substance sue. It has been known for some time that it is relatively easy to modify cognitions associated with use but less easy to change actual behaviour. Morgan et al. (1996) have carried out an evaluation of year long programme to combat substance use in eight schools in Ireland. The programme was broadly based and featured information on several aspects of substance use, the enhancement of self-esteem, decision making as well as assertiveness skills. In addition, the teachers who implemented the programme underwent an extensive period of training in the particular style of instruction considered crucial to the success of the programme. There were also efforts to have whole school involvement particularly with regard to the Principals and other staff members. 107


The evaluation of the programme involved a comparison with eight other schools which were matched on the basis of location and socio-economic measures. The following major conclusion emerged from the evaluation: • the programme was implemented as the planners has intended in 6 of the 8 schools, • teachers involved in the programme were very favourably disposed towards it in terms of its educational potential, • the children in the pilot schools compared to those in control school made significant gains in terms of the social behaviours that were central to the programme, that is assertiveness and self-esteem, and • there were highly significant differences between pilot and control students with regard to beliefs about the positive and negative consequences of alcohol use (in the "desired" direction), • the overtly expressed attitudes of young people in the pilot schools towards alcohol was less positive than for controls, • there were no significant differences in relation to actual substance use behaviour between control and pilot schools. There are two weaknesses in this evaluation by Morgan et al.. The first is that while the control schools were matched, the assignment to pilot and control conditions was not random. The other difficulty is that the information was collected entirely by questionnaires. While this is an appropriate way to collect data on substance use, it is hardly the ideal way to gauge how well children fared as a result of social skills training. Nevertheless, this evaluation is indicative of some of the more favourable outcomes that have emerged in this area. The DARE programme (Drug abuse resistance education) teaches students skills for recognising and resisting social pressures to use drugs. It also focuses on development of self-esteem, coping, assertiveness, communication skills, decision making and identification of positive alternatives to drug use. This programme is taught by a uniformed police officers and involves 17 weekly lessons, each of 45-50 minutes duration. The programme was developed from a second generation curriculum (project SMART (Self-management and resistance training). There have been several evaluations of this programme and the more rigorous the evaluation, the less the effects that have been found. For example in a study Becker et al. (1992), in California showed that compared to a control group students who participated in the DARE programme, at best maintained their level of substance use after the 108


programme. More significantly in a study by Harmon (1993) differences emerged between control and experimental groups on only a few dependent outcomes, that is less association with drug-taking peers as well as attitudes to substance use. But no differences emerged for a range of behaviours including cigarette, alcohol and marijuana use, frequency of any drug in the past month, attitudes about police, coping strategies, attachment and commitment to school, rebellious behaviour and self-esteem. A study by St. Piere et al. (1992) tested the effectiveness of the "Stay SMART" programme in a community setting. Three groups were studied over a 27 months period. One group consisted of five clubs who were offered the Stay SMART programme, another were offered the same programme with a booster while a third group were controls. The results indicated that there were effects for behaviour related to substance use. These were especially strong as regards attitudes and beliefs. However, the behavioural effects were much weaker. Furthermore, there was a problem about the equivalence of the control group.

3.5 Recent studies of training and generalisation of social skills One extremely important point about the social influence strategies (i.e., the social skills/competency approach) is whether the skills learned generalise to other domains. There are two versions of this issue. On the one hand it would be important to know whether the skills learned in the context of combating substance use transfer to other contexts that do not involve substance use. For example, the social skill of assertiveness might be expected to generalise to other domains, e.g., withstanding bullying, coping with manipulation, etc. In this regard, it is instructive to look at the vast literature on the effectiveness of Social Skills training, that is not necessarily related to the prevention of substance use. There are a variety of reasons why social skills training is of interest in educational contexts including the value of such skills per se and also because of the evidence that poor social skills contribute to academic underachievement (Hughes & Sullivan, 1988). Social skills training if often used to assist children who are having problems in getting along with other children. Frequently, the outcome measures include peer acceptance, social interaction and problem-solving. Several studies have examined the generalisation of training effects. Specifically some have studied the extent to which the behaviours learned generalise over time (maintenance), over various settings (from the setting 109


in which skills were learned to other settings) and across different persons (the range of people who are interacting with the individual). It has be to be admitted that the generalisation effects that are found are weak as the reviews by Hughes & Sullivan (1988) and those by Ogilvy (1994), and by DuPaul & Eckert (1994) have shown. Of 38 studies reviewed by Hughes and Sullivan only 7 were able to demonstrate an improvement in "real life" performance of the targeted skills. With regard to maintenance of the skills over time, only a small number have actually tried to maintain the assess this beyond six months. The review by Ogilvy (1994) of the effectiveness of social skills training concludes that it may be a necessary component for bringing about changes in adolescents’ lives. However, she concludes that it is not sufficient without modification of the social context within which social functioning take place. The evaluation of 7 commercially available social skills programmes by DuPaul and Eckert found that even explicit attempts to foster generalisation met with only mixed success. They concluded that alteration of the natural environment led to the most significant maintenance effects. The lessons for the social skills approach to the prevention of substance use, are significant. We cannot expect that there will be an automatic generalisation of what is learned in classroom environments to the actual situation. Additional steps need to be taken to modify the climate within which the learned skills might be elicited if the training is to be effective. Some attention has also been given to the second question, viz., the extent to which programmes focusing on one substance generalise to another, without any specific material on the second substance. Elickson and Bell (1990) sought to extend the social influence model of smoking prevention to alcohol and marijuana. Overall, the results showed a modest success. In addition to the reductions in smoking, modest reductions in drinking for students at three risk levels were observed immediately after the peer led version of the programme, but disappeared at a one-year follow-up. On the other hand, the curriculum was associated with significant reductions in both initiation and later use of marijuana. The researchers speculated that the apparent effectiveness of the social influence approaches for tobacco and marijuana may reflect generalised norms against those two substances, while for alcohol, social influence in training is less effective because society has not developed a consensus against its use. On the other hand, Biglan et al. (1987) found no generalisation of an anti-smoking skills programme to alcohol or marijuana. Doi & DiLorenzo 110


(1993) examined the question of whether refusal skills training that focused on smoking would extend to smokeless tobacco. Their programme which emphasised increased understanding of the pressures to use tobacco as well as the associated coping skills was pilot tested with 7th grade students. The indications were (both in immediate evaluations and in a one-year follow-up) that the skills learned to prevent smoking onset did indeed generalise to smokeless tobacco. It must be stressed that the numbers involved in this study were relatively small (around 160 students in total). Thus, the question of whether such social skills programmes transfer to other substances, is unresolved at the moment. Given the importance of this question, it deserves greater attention in future evaluation studies.

4 Risk-focused interventions An apparently promising approaches to the prevention of adolescent substance use drug problems is through a risk focused approach (Hawkins, Catalano & Miller, 1992). The risk focused approach has been advocated largely on the grounds of the success of campaigns to reduce risk factors for coronary heart disease. In addition, proponents of this approach have pointed to the failure of existing campaigns that have ignored risk factors. A risk-focused approach requires a number of steps: • identification of high-risk factors for substance use, • identification of the strategies that are effective in reducing such risk factors, and • application of such methods to high-risk and general population studies.

4.1 Identification of high-risk factors Below we consider the identification of risk factors with a particular focus on • parental risk factors, • peer risk factors and • problem behaviour. Parental factors. There is considerable evidence that low parental disapproval tends to be associated with high level of substance use. The Grube & Morgan (1986) study found that in general, perceived disapproval of smoking and drinking by parents tended to be related to lower levels of 111


smoking and drinking by offspring. However, the situation is not straightforward. There was a suggestion in the study by Akers et al. (1979) that a curvilinear relationship existed between parental attitude and adolescent drinking, with higher levels of drinking being associated with both indifference and with extreme disapproval. In other words, the best outcome (in terms of low consumption of alcohol) was brought about where parents tended to have moderate, rather than extreme, attitudes towards children’s drinking. In the context of the effects of parental disapproval on adolescent alcohol use, a study by Atkin & Atkin (1986) is of particular interest. This study found that teenagers tend to underestimate the extent of parental disapproval of their use of alcohol. In a survey of 1,700 Michigan high school students and their parents, it was found that 85 per cent of parents strongly disapproved of their teenager getting drunk, 81 per cent strongly disapproved of party-going and 68 per cent strongly disapproved of their teenager having a few drinks with friends. In contrast, 49 per cent, 39 per cent and 29 per cent of teenagers perceived their strong parental disapproval of these activities. Similarly, this study showed that parents consistently underestimated the frequency of drinking and driving with a drinking driver by their teenager. Furthermore, parents reported that they had a high frequency of communication about alcohol-related matters with their teenager and that they closely monitored the activities of children at weekends. In contrast, the majority of teenagers reported a low frequency of communication about their drinking and perceived little or no chance that their parents could detect their drinking. Thus, parental expectations, disapproval and overall attitudes may frequently not be communicated to their offspring. Peer influences. Of all the factors that have been thought to be related to substance use among youth, perhaps more attention has been given to peer influences than to any other. Certainly, there seems to be a strong association between friends’ drinking and reported drinking. In the Grube & Morgan (1986) study, of those students who reported that none of their friends were drinkers, 82 per cent were themselves non-drinkers. In contrast, if their good friends were drinkers, only 22 per cent were non-drinkers. Other studies like that of Bank et al. (1985) found that peer drinking was a strong predictor of reported drinking in the United States, France, Norway and Australia. While these studies have established a strong association between peer behaviour (drinking) and reported drinking, there are a number of problems of interpretation of such findings. For one thing, there may be an 112


element of misperception in the reports of the young drinkers resulting in their seeing greater support for their own behaviour among their friends than actually exists. Another possibility is that some of the apparent peer influence is due to selective friendships. In other words, young people may become friends with each other on the basis of their common behaviour. In other words, the friendships may result from drinking as opposed to causing it. The study by Morgan & Grube (1991) attempted to disentangle these influences. The results suggest that part of the apparent influence of friends may be due to selective friendship. However, peer example is still a factor in initiation to drinking. Another interesting point to emerge from this latter study is that peer disapproval is not a major influence, relative to the other parent and peer factors discussed here. Some other recent studies have also addressed the question of how peer influence is actually mediated. The work of Sellers & Winfree (1990) was designed to test the extent to which the acquisition of favourable or unfavourable definitions underlie peer influences. They argue that an individual learns, in close intimate interactions, evaluations of behaviour as either appropriate or inappropriate, good or bad. Drinking is more likely to occur when people develop a greater balance of favourable to unfavourable definitions of that behaviour. The results of the Sellers and Winfree study among American high school students were largely supportive of the view that an exposure to an excess of definitions that favour drinking are likely to increase the chances of alcohol use. Problem Behaviour and Alcohol. There is considerable evidence that there is a strong relationship between and various kinds of problem behaviour. It has often been suggested that adolescent problem behaviours (smoking, drinking, stealing, etc.) form a single dimension that reflect a general underlying tendency to nonconformity or deviance. In support of this general deviance hypothesis, it has been shown that there is a positive correlation between a wide range of problem behaviours and that such behaviours appear to be influenced in a similar fashion by the same variables (Donovan & Jessor, 1985). These latter researchers have demonstrated by means of factor analysis that there is a single common factor underlying problem behaviours, including being drunk, illegal drug use, shoplifting and vandalism. In support of this view, several studies have shown that young people who drink tend to be involved in other deviant kinds of behaviour like illicit drug-use and truancy (Jessor & Jessor, 1977). In addition, in those studies that have examined attitudes towards deviance, it has been shown that

113


acceptance of deviant behaviour tends to relate strongly to substance use (Brook et al., 1983). The recent study by McAteer (1991) examined the association between alcohol and "joy-riding" (stealing a car and driving through the streets for enjoyment). She found that joy-riding was more than twice as likely to occur among regular drinkers than among the non-drinkers. Furthermore, a comparison of occasions when drinking vs. occasions when not drinking indicated that joy-riding was about twice as likely to occur on those occasions when the young person had been drinking.

4.2 Reduction of risk factors Once risk factors have been identified then the next critical stage in this approach involves interventions to lessen the risk factor and hopefully the subsequent level of substance use. Rather than attempt a review of a range of studies it may be more enlightening to consider efforts in relation to one risk factor viz., problem behaviour. As shown above, problem behaviour is associated with substance use. In turn, it has been suggested that such problem behaviours comes about because children lack the competencies and social skills that are crucial in interacting with other. Thus, if children were taught such skills, an improvement in social interaction would occur coupled with a reduction in problem behaviour as well as a lesser likelihood of substance use. Social competence promotion approaches have used a number of methods. For example, socially rejected youths have been taught social interaction skills to increase the frequency of their social interactions (Ladd & Asher, 1985). However, while such programmes have been tested in relation to their effects on short-term outcomes such as adjustment at school and relationship with peers only a small number of studies have examined effects on later substance use. However, a few studies which have measured alcohol-related outcomes have yielded promising results. Lochman (1988) examined the effects of an anger management programme during school hours for boys identified as aggressive by their teachers. The programme included role-playing, goal-setting, social-problem solving skills as well as modelling of alternative ways of coping with anger-arousing situations. Three years later, the boys in the programme were found to have significantly lower rates of alcohol and marijuana use compared to a matched (but not random assigned) group of adolescent boys. This is probably the single strongest 114


evidence for the effectiveness of this kind of approach. It will however be noted that the treatment and control conditions were not randomly assigned. However, it must be admitted that most of the studies in this area have only gone through some of the stages required to demonstrate the effectiveness of such interventions for primary prevention use. Many of the studies have examined short-term effects only. Most studies have sought evidence of change the risk-factor only but without indications of effects on substance use behaviour.

4.3 Difficulties with risk-focused approach Some of the difficulties with the risk-focused approach include: • the extent to which risk factors are susceptible to modification, • the extent to which risk-factors are causally related to substance use, and • the range of scope of risk factors and • possible interactions between identified risk factors. Can risk factors be modified? The real difficulties with the approach lie in the fact that the factors being addressed are extremely difficulty to control since they involve matters like parental behaviours, enhancement of school-achievement and learning to use alternatives to aggression. For example it has indeed been demonstrated that school failure is associated with substance use. Thus, one approach is to use techniques that are likely to enhance academic achievement, as with early childhood education, alterations in classroom teachers’ instructional practices and tutoring of low achievers. For example, Hawkins et al. (1987) trained classroom teachers in interactive teaching, proactive classroom management and co-operative learning. It emerged that the children taught by these teachers showed greater achievement gains in mathematics and greater commitment to school as well as significantly lower levels of suspensions and expulsions than did children in control classrooms. The problem with the attempted widespread use of such a strategy is that it involves efforts to change variables that have proven to be difficult to modify. School failure involves an interaction between home and school factors each of which is difficult to bring under effective control. A recent review of home factors (Kellaghan et al., 1993) showed that at least the following factors are important in addressing the issue of school failure:

115


• parental expectations and aspirations, • academic guidance and support, • opportunities for exploration of ideas and events, • priority given to schoolwork over other activities and • style of language in interaction with children (Kellaghan et al.; Wang et al., 1993). In the case of addressing factors associated with school failure, the problems are even greater. While some studies have demonstrated the possibility of changing aspects of teaching style, there are real difficulties in demonstrating related changes in student achievement scores. We do not deny the significance of studies like that of Hawkins et al,. Rather, it is to draw attention to the vast literature on school and teacher effectiveness as well as home/school interventions which have been concerned with these factors in their own right and which have not been especially successful in changing what are here identified as "risk" factors. Are "risk factors" causally related to substance use? An important issue is whether the factors associated with substance use are indeed risk factors in the traditional sense. In other words, to what extent can it be demonstrated that these are causally related to substance use. If they are not causally related then modifications of the "risk factors" will not results in changes in substance-use behaviour. This problems can be illustrated by referring to one domain of influence viz., the extent to which peer group pressure is a risk factor. There is considerable evidence that association with drug-using peers is often a consequence of the use of various substances, rather than a cause and can hardly be described as a risk factor in the classical sense. The study by Morgan & Grube (1991) showed that peer influence are much more subtle than the peer group pressure concept suggests. For one thing, the influences of peers who were described as "friends" is much stronger than were the corresponding influences of same-aged peers, while the person described as the "best friend" was especially influential. Furthermore the pattern of peer influence for maintenance of substance use was different from that for initiation. For example the several good friends were important (both means of their behaviour and approval) in the initiation to drug use while the best friend had a critically important role in the maintenance of drug use. Thus, the pinpointing of the precise risk factors associated with peer group influence is itself quite problematic.

116


Range and scope of risk factors. The risk-focused approach has been criticised by Brown & Horowitz (1993) in relation to the lack of precision about the term. Students are often identified as being at risk on the basis of criteria such as test scores, retention in a grade, or on the basis of social variables like being from lone parent family. In many schools, this process results in the majority of students being identified as being at risk. The California DATE (Drug, alcohol, tobacco Education) identifies at risk youth on the basis of a broad range of factors. Even the sheer number of factors in the DATE programme is worthy of note. Included are • family risk factors including lack of clear expectations of behaviour, poor monitoring, inconsistent or severe discipline, lack of caring, viz. parental substance use, tolerance towards use by offspring, • school risk factors including lack of clear policy regarding drugs, school transitions, academic failure, low commitment to school, • community risk factors including economic and social deprivation, community disorganisation, norms favourable to drug use and availability of drugs, • individual/peer risk factors including early anti-social behaviour, alienation and rebelliousness, greater influence by peer rather than parents, friends who use drugs, tobacco or alcohol, approval of substance use by friends, early first use. It is easy to see the problems with such a listing of risk factors. There is hardly any young person who would be regarded as being at risk, given the number listed. How do risk factors interact with each other? This problem is related to the scope and number of such factors. However, it has to do specifically with the finding that some risk factors interact with each other in ways that the presence to two factors is different from what would be expected on the basis of either factor on its own. Some of the ways in which risk factors may interact with each other is demonstrated in the work of Grube & Morgan (1990). Specifically this study examined the interaction between two influences that are normally considered to be important risk factors for substance use viz., support of peers for usage and attitudes to use. As noted above several studies have shown that these factors operating singly are inclined to increase the likelihood of initiation and maintenance of use. What was especially interesting was that substance use was more frequent among adolescents when attitudes were favourable and when perceived social support was 117


favourable. Otherwise substance use was relatively infrequent. Thus, without knowing what other influences are interacting with a particular risk factor, it would be impossible to say what increment will be added to risk. Finally, one risk factor (poverty and economic deprivation) which has been the target of so many other interventions does not seem to be associated with substance use. For example Murray et al. (1987) found that mother’s occupational prestige was positively related with monthly alcohol use as well as marijuana use. Similarly the US 1988 national household survey on drug use found higher usage among those with some college education as compared to those with only a high school education (Adams et al. 1990).

5 What has been learned from community-based programmes? Community based programmes have frequently targeted Coronary Heart Disease (CHD). The critical features of community programmes is that they serve a defined population, with a programme being directed at the entire community rather than a specific group or number of individuals within that community. Interventions take place in the context of the normal environment in which people live and work and is often incorporated into the existing health services. A heavy emphasis is usually placed on participation and involvement. Community interventions have utilised a range of strategies. In varying degrees, the mass media, health and educational interests, and voluntary, state and commercial interests have played some part. Some community interventions are based on the diffusion of innovations model that suggests that changes come about through natural social networks and that opinion leaders within the system are needed to achieve the changes in health behaviour (Rogers, 1983). Most community interventions have used some version of the Persuasion/mass communication model initiated by Hovland and colleagues at Yale during the second world war and afterwards by others in relation to health behaviour (e.g. McGuire, 1984). In this model the effectiveness of a communication and the resultant attitude change is said to be related to variables relating to • the source of the communication (e.g. credibility) • the message (e.g. whether threatening or not) • the medium (e.g. whether newsprint or television), 118


• the receiver (e.g. position of the issue being targeted) and • behaviour change being targeted (e.g. adoption of new behaviour or cessation of established behaviour).

5.1 Evaluation issues in relation community programmes Among the major difficulties are the small number of community units which are studied as well as the lack of a random choice of the intervention and reference area. Other difficulties include differences in risk-factor patterns between intervention and reference communities at baseline. The most effective evaluation design is a cross-community multiple time series. However even with the best design, several factors influence the power of the analysis including the number of communities, community size and composition, sample sizes in the surveys, whether there are cohorts or cross-sectional surveys, the number of surveys conducted in each community and the assumptions about the latencies of the effects (Salonen et al., 1986).

5.2 Recent examples of community-based programmes One example of an apparently successful community based prevention project is the US Midwestern prevention project that begun initially in the Kansas city area. This project includes mass media programming, school-based programme for youths, parent education and organisation, community organisation and a health policy, which were introduced sequentially into communities during a 6-year period. Pentz et al. (1989) report on the first phase (two years) of the project and present result for prevalence rates of cigarettes, alcohol and marijuana. Analyses for 42 schools suggest that the rates of use are lower in the intervention condition than in a delayed intervention condition, with or without controlling for grade, socio-economic status. While the pilot and controls were essentially equivalent at baseline, the current (4-week) prevalences for cigarettes use were 22% and 15% for control and intervention schools, while the corresponding prevalence rates for alcohol were 12% and 9% and for marijuana 7% and 4%. Two possible threats to invalidity should be considered in relation to this study. The first has to do with the possible non-equivalence of the control group. While there was little evidence for this. it is worth noting that the assignment to conditions was not random. The authors of the study also raise the possibility that the changes may be in reporting of substance use 119


rather than in actual use. While this is a possibility there is little evidence in other studies of this kind of effect. This must therefore be considered one of the most convincing studies of this kind. As part of the Minnesota Heart Health Promotion Programme a seven year cohort study of adolescents in two different communities was conducted (Prokhorov et al, 1993). A school-based intervention was implemented in one of the communities which addressed aspects of cardiovascular health promotion and risk-factor prevention. Based on social learning theory, the focus was on group norms, providing alternative health role models, teaching students social skills to enable them to resist pressure to engage in risky behaviours, and the generation of health enhancing alternatives. Prevention measures within the framework were targeted at reducing smoking and alcohol use, increasing regular physical activity, and promoting healthful dietary habits. The results showed that there was a substantial decline in the participation by all students in both control and experimental communities. Just over half of the treatment community group were still in the study at the end, while in the case of the reference (control) community only around 30 per cent were involved. A feature of the study was the change in lifestyle pattern over the years including physical appearance, school performance, family, amount of exercise, number of friends, kind of food, amount of money, and amount of TV. The study showed that physical appearance was the most valued characteristic and the only value which grew in importance over time. Students who participated in the community intervention tended to retain their positive value about physical exercise while the reference community demonstrated gradual reductions. Furthermore girls in the intervention group tended to value the kind of food they eat more than those in the reference group. A recent community based prevention programme in Wexford, Ireland was especially ambitious in that it involved efforts to reduce both the supply and demand for alcohol (Wexford Community Action Programme, 1992). The town of Wexford has a population of about 10,000 and in common with many towns in Ireland is concerned with young adolescents’ drinking. Among the features were a school programme, parents worships, server-programme as well as an Identification Card scheme to prevent underage adolescents being served. The results showed that in comparison to a reference town of the same size, the beliefs about the negative consequences of alcohol were greater among those students who participated in the intervention. However, this 120


study also demonstrated the difficulty of mobilising all the interests involved in order to provide a truly community based initiative.

6 General issues relating to evaluation of substance-use prevention programmes Below we consider a number of general issues concerning the design and evaluation of substance use programmes. These are general considerations that relate to all kinds of programmes and have particularly important implications for planning for the future especially with regard to illegal substances. Among the questions considered are the following: • How big are the effects than can reasonably be expected as a result of these kinds of interventions? • Do adolescents’ feelings of invulnerability reduce the impact of prevention programmes? • Is it necessary to modify programmes to suit particular groups? • Are there any indications that there may be sometimes be harmful effects of programmes?

6.1 What kind of effects are reasonable to expect of substance use prevention programmes? It may be worth noting that many of the variables that are targeted in substance abuse programmes are rather weakly related to actual use and that the correlation of such variables with actual substance use declines dramatically one control are applied for other factors. This point has been addressed by Mauss et al. (1988). They examined the behaviour and skills that are frequently the target of substance use programmes including knowledge of substances, self-concept, strategies for coping with peer pressure, attitudes and beliefs regarding substance use, and taking responsibility with regard to alcohol use. Inevitably these variables are influenced by factors that are outside the domain of what the curriculum can reasonably be expected to influence e.g. home, peer and religious influences. The results showed that while the bivariate analysis of potential curricular influences with substance use behaviour was moderately high (accounting for about 20 per cent of the variance), this relationship diminished greatly once controls were applied for the noncurricular variables which are logically and chronologically prior to these influences. In other words it could be said that while contemporary substance abuse programmes 121


address variables that when considered alone appear to be related to substance use, the indications are that the same variables make such a small independent contribution to behaviour that it is unlikely that even a highly successful classroom intervention directed at these variables would do much to prevent substance use. If this view is accepted it is appropriate to target variables that are not normally considered to be within the ambit of prevention programmes. Otherwise the expected outcome is bound to be rather limited.

6.2 Do adolescents’ feelings of invulnerability reduce the impact of prevention programmes? It has often been suggested that one reason why adolescents’ behaviour is hard to modify is that they feel "invulnerable" to dangers. Aristotle's view that the "young are full of passion, which excludes fear" may explain some of the difficulty. There is indeed convincing evidence that people generally underestimate their own chances of dangers relative to others. However, the extent to which this is particularly true of adolescents, has not been established. For example, Quadrel et al. (1993) found that the perception of relative vulnerability was no greater among adolescents than among others.

6.3 Is it necessary to modify programmes to suit particular groups? There is evidence that there are gender differences in responses to smoking prevention programmes (Gilchrist et al., 1989). There is a suggestion that current smoking prevention programmes may not optimally benefit young women. It may be that girls require less emphasis on social skills training and more emphasis on avenues of self-definition and expression. Other evidence regarding motivation and affect suggest the potential for gender-specific approach to smoking prevention. On the other hand a study Daly et al. (1993) of the factors associated with late smoking initiation among women, found that many of the factors influencing initiation to smoking were similar to those of adolescents. These included peer smoking, perceived social acceptability of smoking, poor grades in high school and low church attendance. In fact, there is no contradiction between the two viewpoints since one refers to factors associated with initiation and the other to gender differences in intervention effects. Questions also arise about the extent to which there is a need to modify existing programmes for student in special education programmes. Kress & 122


Alias (1993) have reviewed the evidence on the effectiveness of the various approaches to substance use prevention in such populations. They note the high risk of these young people and the failure of research to address appropriately the most valuable avenue of prevention. They suggest that there may be potential in the social problem-solving programmes which seem to have had some positive outcomes with students with behavioural disorders and learning disabilities. However, they also draw attention to the failure to systematically evaluate existing efforts. Any information on evaluation that exists relates to training and implementation. For example, it would seem that teachers trained in at least one programme (Project Oz), report a high level of confidence in their ability to deliver the programme in a satisfactory way.

6.4 Are there any indications that there may be harmful effects of various programmes? Above we discussed the findings that earlier programmes emphasising "facts" and "scare tactics" may be counterproductive. Is there any further evidence that this may be a widespread outcome of prevention programmes? There are at least some grounds for believing that such effects could occur. In providing such programmes, the assumption might be conveyed that such behaviours are relatively common and this in turn might trigger normative influences. Goodstadt (1980) found 15 studies where drug education had a negative effect on either attitudes or behaviours towards drug use (i.e. students were more likely to use the substances following the programme). However, even in these same studies there were positive effects on some groups and in relation to some behaviours and attitudes. Goodstadt took the view that negative effects may have to be tolerated in order to achieve effects that are beneficial to the vast majority of students. This issue has received little attention in recent times and is certainly worthy of further consideration.

6.5 Issues of evaluation methodology Several important questions arise from the use of the quasi-experimental designs that is most commonly used. The term quasi-experimental derives from the fact that designers of evaluations attempt to model their work on true experiments yet almost always the design is not truly experimental in the sense that it involves random allocation of units to experimental/control conditions.

123


One of the key problems arises from the nature of the unit of randomisation and the related question of the unit analysis. Researchers can choose to randomise entire school districts, schools, entire grades within schools, selected classes of the same grade in a school, or individual students. The view put forward by Killeen & Robinson (1989) is that the entire school represents the most appropriate unit of randomisation They suggest that the potential for reduction of treatment effects through diffusion of between treatment and control groups within a single school is thus avoided. A related difficulty is ensuring that the units involved are equivalent. This is especially difficult when there are a relatively small number of schools. Two strategies can be used to try to make this happen. The first is to make

124


sure that schools are similar on a variety of relevant demographic variables. Of particular concern in substance-related research are matters like percent dropping out at the end of compulsory schooling, level of absences from school as well as features of socio-economic status (although this latter factor is less important in substance use than is the case with outcomes like school achievement. The second important strategy is to ensure that schools are similar on pre-test variables that may be relevant to whatever outcome variables that are selected. This might include a measure of substance use behaviour as a pre-test or possible a measure of related beliefs and attitudes.

6.6 Threats to internal validity One frequent source of threat to internal validity is differential attrition. Some tests of selective attrition are applicable in longitudinal prevention research. These are: • differences between "drop-outs" and completers on baseline values of the main outcome variable (or related variables), • differences in change scores for drop outs and completers. Another sources of invalidity is differential history effects. This may be especially important with a relatively small number of schools. We have very little control over or even an ability to measure accurately many school experiences that may have a major impact on important outcomes. At the very least researchers need to be aware of potentially important events that occur in schools. Equally important is awareness of control schools of the occurrence of the treatment manipulation in other schools and their reaction to that. It is important to know whether control schools are affected by resentful demoralisation or whether they attempt to compensate for the absence of the programme by some other means which may in itself be affect the outcomes.

6.7 What are the aims of programmes? It has sometimes been suggested that anti-smoking programmes have been relatively more successful because of the relatively simple objectives, that is no smoking whatsoever. In contrast the goals for alcohol programmes

125


are much more complex. This raises the issue of the appropriateness of certain kinds of goals. What is the ideal? Implicit in many prevention programmes is that the lesser use of drugs the better, particularly in those that emphasise the importance of the "gate-way" role of drugs... This suggests therefore that total abstinence is an ideal that should be sought. However, a recent study by Shedler & Block (1990) raises serious questions about this "ideal" state. It is a particularly important feature of this study that it was prospective in nature, that is, the measures of personality and psychological health were obtained from age five onwards while the measures of drug use were obtained at age 18 years. What was of special interest in the Shedler & Block study was that they did not simply assume a continuum of nonuser to abusers. Rather they identified and contrasted discrete groups of nonusers, experimenters and drug abusers. The main measures of personality characteristics were the California Q-sort which consists of 100 personality descriptive statements from which a profile is obtained. These measures were obtained at the same time but independently of the measures of substance use. In addition, psychological descriptions were available of these subjects from early childhood onwards. The results indicated that adolescents who had engaged in some drug experimentation (primarily with marijuana) were the "best" adjusted of the sample. In contrast adolescents who used drugs frequently were maladjusted showing a distinct personality syndrome marked by alienation, poor impulse control and manifest emotional distress. On the other hand adolescents who by age 18 had never experimented with any drug were relatively anxious, emotionally constricted and lacking in social skills.

6.8 Measurement of outcomes in evaluations One of the problems with current evaluations of substance abuse prevention programmes is the lack of agreement on the appropriate kinds of outcomes that should be measured. Unlike other areas there are no established schedules or inventories, that have been shown to measure reliably the outcomes that are of concern.

126


An examination of current approaches to measuring the outcomes of evaluations suggests that among the very common measures used are the following: • attitudes • beliefs, • actual use and • "offer" measures. The problems with these measures is that there are problems with each one. For example, the association between attitude and substance use depends greatly on what measure of attitude is used, particular the extent to which the measure refers to personal use or simply refers to general use. In other words, what I believe about "drugs" may not translate into my personal behaviour with regard to such substances. There are at least four different aspects of beliefs, depending on whether positive or negative outcomes are considered and whether or not it is shortor long-term consequences that are the focus. With regard to cigarettes, it matters greatly whether long-term consequences are considered (like damage to health or getting lung cancer) or short-term consequences (like having smelly clothes). With regard to "actual use" some of the most popular and valuable measures are life-time prevalence and previous month’s or previous year’s prevalence. Less valuable are those measures that involve asking people how much they "usually" use. The problem seems to be that "usually" implies occasions that are very hard to pinpoint and seems to result in systematic under- or over-estimation. Quite frequently surveys have inquired about the frequency of being "offered" drugs. Beyond giving an indication of subjective availability this measure seems especially imprecise.

6.9 Practical issues of implementation and the effects on evaluation The practical administrative questions of actually implementing a project so that it can be rigorously evaluated, is a major difficulty. Some of these practical issues are dealt with by Piper et al. (1993) who have subtitled their paper "Lessons our textbook did not teach us". The important point is that the workers in the area of substance use prevention are not those who have all "flunked" their methodology courses in Graduate school. Rather 127


the failure to have precise designs is forced on them by various considerations. Piper et al., list some of the problems including the tensions between the needs of independent school districts and the research goals of the project, (b) expectations of schools and teachers regarding appropriate education for prevention of substance use, (c) difficulties of recruitment of schools and teachers, (d) the variation in school district approval processes and decision making styles and their effect on implementation and (e) variation in responses of school administrators and faculty to programme messages and teaching techniques. Another major problem related to implementation is the diffusion including teacher adoption and implementation and the effectiveness of teacher training as well as school principal involvement in increasing implementation. In a study by Rohrbach et al. (1993), the subjects were 60 teachers and 25 school principals and 1,147 5th grade students from 4 Los Angeles area school districts. During the first year 78 percent of the teacher implemented one or more programme lessons. However, during the second year only 25 percent maintained implementation of the programme. Furthermore, there were systematic differences between those who implemented the programme and those who did not. Those who implemented the programme had fewer years of experience, strong self-efficacy, enthusiasm as well as principal encouragement. Furthermore, it was found that integrity of programme delivery was positively associated with immediate programme outcomes. The results suggested that widespread teacher use of innovative programmes cannot be taken for granted. Such findings are especially relevant for the results of evaluations that fail to find significant outcomes.

7 General conclusions A. Approaches to prevention of substance use based on mere information are likely to be at best neutral in their effects on drug use and may be more likely to have counterproductive effects especially if they have elements of miseducation. B. Prevention programmes that have targeted cigarette smoking have been relatively successful compared to campaigns against other substances. What is less clear however, is whether this success is due to the nature of the programmes, the climate of increasing disapproval of smoking in the general population or the fact that anti-smoking programmes have specific targets that are easier to communicate (total abstinence as opposed to "moderation").

128


C. Taken together the studies that have evaluated substance use prevention programmes have found that some programmes have had at least moderate success. This seems to be especially the case for studies that have used social skills approaches. However, the meta-analyses that support these conclusions often include several evaluation with very weak designs so that the overall conclusions about the positive effects are hard to sustain. A related point is that the bias by journal editors favouring "significant" results may mean that the published results are relatively more favourable towards programmes than the total body of evaluation work (This point was suggest by Alfred Uhl in his comments on an earlier draft). D. While social skills approaches have generally been regarded as the most promising avenue of prevention, rigorously designed studies do not provide grounds for unrealistic optimism. It is worth noting that evaluations of social skills programmes (not just those targeting substance use) have found little evidence of generalisation from training to real-life situations. E. While community-based programmes are plausible in terms of their rationale, especially in the present political climate, the positive outcomes that emerged from some evaluations of such programmes are hard to interpret. Almost all of the studies have weak designs and the use of "reference" communities as controls is especially problematic given the difficulties of establishing equivalence. This often results in enormous underestimation of the error since adjustments are seldom made for "nesting" effects. F. Despite the appeal of "risk-focused" approaches to prevention, some difficulties are inherent in this approach including: • the extent to which risk factors are susceptible to modification, • the extent to which risk-factors are causally related to substance use, and • the range of scope of risk factors and • possible interactions between identified risk factors. G. There is a lack of clarity about the "ideal" situation in many programmes to combat substance use. This is especially the case given the evidence that young people who never experiment with illegal substances are not necessarily the healthiest in psychological terms.

129


H. It is not clear how much programmes need to be modified in order to take into account the particular situations in which high-risk groups are found. For example, little is known about the extent to which participants in special education programmes might need programmes that are different from other classrooms. I. Overall, the area could benefit from increased rigour in the design of evaluations as well as greater effort to provide a plausible rationale for what changes are supposed to occur and how.

8 References Adams, E. H.; Blanken, A. J.; Ferguson, L. D.; Kopstein, A.: Overview of selected drug use trends. Rockville, MD: National Institute on Drug Abuse, 1990 Best, J. A.; Thomson, S. J.; Santi, S. M.; Smith, E. A.; Brown, K. S.: Preventing cigarette smoking among school children. American Review of Public Health, 9, 161-201, 1989 Botvin, G. J.: Factors inhibiting drug use: Teacher and peer effects. Report to the National Institute of Drug Abuse, New York, Cornell University Medical College, 1987 Botvin, G. B.; Baker, E.; Filazzola, A. D.; Botvin, E. M.: A cognitive behavioural approach to substance abuse prevention: One year follow-up. Addictive Behaviors, 15, 47-63, 1990 Brown, J. H.; Horowitz, J. E.: Deviance and deviants: Why adolescent substance abuse programs do not work. Evaluation Review, 17, 529-555, 1993 Bruvold, W. H.: A meta-analysis of adolescent smoking prevention programmes. American Journal of Public Health, 83, 872-880, 1993 California Department of Education: Application for comprehensive, drug alcohol and tobacco education programme funding. Sacramento: California Department of Education, 1992 Carlini-Cotrim, B.; Aparecida de Carvaiho, V.: Extracurricular activities: Are they an effective strategy against drug consumption. Journal of Drug Education, 23, 97-104, 1993 Daly, K. A.; Lund, E. M.; Harty, K. C.; Ersted, S. A.: Factors associated with late smoking initiation in Minnesota women. American Journal of Public Health, 83, 1333-1335, 1993

130


Doi, S. C.; DiLorenzo, T. M.: An evaluation of a tobacco use education prevention program: A pilot study. Journal of Substance Abuse, 5, 73-78, 1993 DuPaul, G. J.; Eckert, T. L.: The effect of social skills curricula; Now you see them, now you don’t. School Psychology Quarterly, 9, 113-132, 1994 Eisenman, R.: Who receives drug education in our schools? A paradox. Journal of Drug Education, 23, 133-136, 1993 Eiser, J. R.; Morgan, M.; Gammage, P.: Social education is good for your health. Educational Research, 30, 20-25, 1988 Ellickson, P. L.; Bell, R. M.; Harrison, E. R.: Changing adolescent propensities to use drugs: Results from project Alert. Health Education Quarterly, 20, 227-242, 1993 Gilchrist, L. D.; Schinke, S. P.; Nurius, P.: Reducing the onset of habitual smoking among women. Preventive Medicine, 18, 235-248, 1989 Goodstadt, M. S.: Drug education - a turn on or a turn off? Journal of Drug Education, 10, 89-99, 1980 Grube, J. W.; Morgan, M.; Seff, M.: Drinking beliefs and behaviors among Irish adolescents. International Journal of the Addictions, 24, 101-112, 1989 Grube, J. W.; Morgan, M.: The Development and maintenance of smoking drinking and other drug use among Dublin post-primary pupils, ESRI General Publications, 1990 Grube, J. W.; Morgan, M.: The structure of problem behaviours among Irish adolescents. British Journal of Addictions, 85, 667-675, 1990 Harmon, M. A.: Reducing the risk of drug involvement among early adolescents: An evaluation of drug abuse resistance education (DARE). Evaluation Review, 17, 221-239, 1993 Hawkins, J. D.; Doueck, H. J.; Lishner, D. M.: Changing teaching practices in mainstream classrooms to improve bonding and behaviour of low achievers. American Educational Research Journal, 25, 31-5, 1988 Hughes, J. N.; Sullivan, K. A.: Outcome assessment in social skills training with children. Journal of School Psychology, 26, 177-183, 1988

131


Kress, J. S.; Elias, M. J.: Substance abuse prevention programmes in special education populations: Review and recommendations. Journal of Special Education, 27, 35-51, 1993 Lockwood, A. L.: The effects of values clarification and moral development curricula on school-age subjects: A critical review of recent research. Review of Educational Research, 48, 325-364, 1978 McGuire, W. J.: Public communication as a strategy for inducing health-promoting behavioural change. Preventive Medicine, 13, 299-319, 1984 Mauss, A. L.; Hopkins, R. H.; Weisheit, R. A.; Kearney, K. A.: The problematic aspects for prevention in the classroom: Should alcohol education programmes be expected to reduce drinking by youth. Journal of Studies on Alcohol, 49, 51-61. 4, 73-83, 1988 Morgan, M.; Grube, J. W.: Adolescent cigarette smoking: A developmental analysis of influences. British Journal of Developmental Psychology 7, 179-189, 1989 Morgan, M.; Grube, J. W.: Closeness and peer group influence. British Journal of Social Psychology, 30, 159-169, 1991 Morgan, M.; Doorley, P.; Hynes, M.; Joy, S.: An evaluation of a smoking prevention programme with children from disadvantaged communities. Irish Medical Journal, 87, 56-58, 1994 Morgan, M.; Morrow, R.; Sheehan, A. M.; Lillis, M.; 1996 Prevention of substance misuse: Rationale and effectiveness of the programme "On My Own Two Feet". OIDEAS: Journal of the Department of Education, 44, 5-26. Moskowitz, J. M.; Schaps, E.; Malvin, J. H.;; Schaeffer, G. A.: The effects of drug education at follow up. Journal of Alcohol and Drug Education, 30, 45-49, 1984 Murray, D. M.; Richards, P. S.; Luepker, R. V.; Johnson, C. A.: The prevention of cigarette smoking in children: Two and three year follow-up comparisons of four prevention strategies. Journal of Behavioural Medicine, 10, 595-611, 1987 Nutbeam, D.; Macaskill, P.; Smith, C.; Simpson, J. M.; Catford, J.: Evaluation of two smoking education programmes under normal classroom conditions. British Medical Journal, 79, 1371-1376, 1993

132


Ogilvy, C.: Social skills training with children and adolescents: A review of the evidence on effectiveness. Educational Psychology,1994 Pentz, M. A.; Dwyer, J. H.; MacKinnon, D. P.; Flay, B. R.; Hansen, W. B.; Wang, E. Y.; Johnson, C. A.: Journal of the American Medical Association, 261, 3259-3266, 1989 Piper, D. L.; King, M. J.; Moberg, D. P.: Implementing a middle school health promotion research project: Lessons our textbook didn’t tell us. Evaluation and Program Planning, 16, 171-180, 1993 Prokhorov, A. V.; Perry, C. L.; Kelder, S. H.; Klepp, K.: Lifestyle values of adolescents: Results from the Minnesota Heart Health Youth program. Adolescence, 28, 637-647, 1993 Qaudrel, M. J.; Fischhoff, B.; Davis, W.: Adolescent (In)vulnerability. American Psychologist, 48, 102-111, 1993 Regis, D.; Bish, D.; Balding, J.: The place of alcohol education: reflection and research after Eiser at al. (1988). Educational Research, 36, 149-156, 1994 Rogers, E. M.: Diffusion of innovations. New York: Free Press, 1983 Rohrbach, L. A.; Graham, J. W.; Hansen, W. B.: Diffusion of a school-based substance abuse prevention program: Predictors of program implementation. Preventive Medicine: An International Journal Devoted to Practice and Theory, 22, 237-260, 1993 Salonen, J. T.; Kottke, T. E.; Jacobs, D. R.; Hannon, P. J.: Analysis of community-based cardiovascular disease prevention studies: Evaluation issues in the North Karelia project and the Minnesota heart health program. International Journal of Epidemiology, 15, 176-182, 1986 Shedler, J.; Block, J.: Adolescent drug use and psychological health: A longitudinal inquiry. American Psychologist, 45, 612-630, 1990 Shelley, E.: Can community programmes promote heart healthy lifestyles? Irish Journal of Psychology, 15, 164-178, 1994 Smart, R. G.; Murray, G. F.; Arif, A. A.: Drug abuse and prevention programmes in 29 countries. The International Journal of the Addictions, 23, 1-17, 1988 Swisher, J. D.; Hu, T. W.: Alternatives to drug abuse: Some are and some are not. In: T. J. Glynn et al. (eds.): Preventing adolescent drug abuse. Washington DC: Government Printing Office, 1983

133


St. Piere, T. L.; Kaltreider, D.; Lynne, M.; Aikin, K. J.: Drug prevention in a community setting: A longitudinal study of the relative effectiveness of a three-year primary prevention program in Boys and Girls clubs across the nation. American Journal of Community Psychology, 20, 673-706, 1992 Sussman, S. Dent, C. W.; Stacy, A. W.: Project towards no tobacco use: 1-year behaviour outcomes. American Journal of Public Health, 83, 1254-1250, 1993 Stuart, R. B.: Teaching facts about drugs: Pushing or preventing. Journal of Educational Psychology, 66, 189-201, 1974 Tobler, N. S.: Meta-analysis of 143 adolescent drug prevention programs: Quantitative outcome results of program participants compared to a control or comparison group. The Journal of Drug Issues: 16, 537-567, 1986 Wexford Community Action Programme: Report of Wexford Community Action Programme on Alcohol Abuse. Wexford: Author, 1992

contact address: Mark Morgan St. Patrick’s College Drumcondra, Dublin 9, Ireland Tel: +35 31 8376191 FAX: +35 31 8376197 e-mail: [email protected]

134

Definitions - Concepts - Problems

Evaluation of Primary Prevention in the Field of Illicit Drugs Definitions - Concepts - Problems Alfred Uhl LBISucht, Vienna, Austria

___________________________________ Results of an International Consensus Study within the COST-A6 Action of the European Union Participants of the Consensus Study: Stefan Brülhart, Suchtpräventionsstelle, Zurich Xavier Ferrer, A.B.S. and CEUDROG, Barcelona Maria Xesús Froján Parga, Universidad Autonoma de Madrid Osmo Kontula, University of Helsinki Christoph Kröger, IFT, Munich Han Kuipers, Trimbos Institute, Utrecht Ralph Kutza, IFT, Munich Mark Morgan, St. Patrick’s College, Dublin Alice Mostriou, Eginition Hospital, Athens Margareta Nilson, EMCDDA, Lisbon Katarzyna Okulicz-Kozaryn, Inst. Psychiatry / Neurol., Warsaw Flavia Pansieri, UNDCP, Vienna Ulf Rydberg, Karolinska Hospital, Stockholm Chafic Saliba, Cndt, Lyon Reginald G. Smart, ARF, Toronto Alfred Springer, LBISucht, Vienna Enrico Tempesta, Universita Cattolica del Sacro Cuore, Rome Alberto Tinarelli, Sert, Ferrara Ambros Uchtenhagen, ISF, Zurich Alfred Uhl, LBISucht, Vienna Annick Vandendriessche, Leefsleutels vzw Jongeren, Brussels

135


Contents 1 Introduction............................................................................................ 139 2 Introductory remarks concerning the terminology................................ 140 2.1 The use of cursive type faces and reference................................... 140 2.2 Drug (ab)use vs. substance (ab)use ................................................ 140 2.3 The term "abuse" ............................................................................ 140 2.4 "Effectiveness" vs. "efficacy" vs. "efficiency"............................... 141 2.5 The "Delphi method" ...................................................................... 142 3 "Evaluation of prevention" - everyday vs. scientific understanding .... 143 3.1 The term "prevention" - everyday vs. scientific understanding..... 143 3.2 The term "evaluation"..................................................................... 143 3.3 How should we deal with the confusing conflict between the everyday meaning of "evaluation" and the peculiar way "evaluation" is understood in a scientific context........................ 145 4 Classification of prevention programmes.............................................. 147 4.1 Classification based on types of target persons ............................. 147 4.1.1 Classification based on the size of the target population ......... 147 4.1.2 Classification based on specific characteristics of the target population ................................................................................. 147 4.1.3 Classification based on the type of social environment ........... 148 4.1.4 Classification based on the way how target persons are reached (directly or indirectly) ................................................. 149 4.2 Classification based on types of goals............................................ 150 4.2.1 Classification based on problem levels..................................... 151 4.2.1.1 Position A............................................................................ 152 4.2.1.2 Position B............................................................................ 153 4.2.1.3 Practical considerations ...................................................... 153 4.2.1.4 Conclusions......................................................................... 154 4.2.2 Classification based on the central strategy.............................. 154 4.2.3 Classification based on substance specificity........................... 155 4.2.4 Classification based on number of substances involved .......... 156 4.2.5 Classification based on legal status of substances involved .... 156 4.2.6 Classification based on consumption oriented goals vs. problem oriented goals ............................................................. 157 4.2.7 Classification of programmes based on abstinence orientation vs. responsible use orientation............................... 159 4.2.8 Classification based on specific goals ...................................... 160 4.3 Classification based on theoretical aspects .................................... 161 4.3.1 Classification based on the role theory plays ........................... 161 4.3.2 Classification based on the kind of: theory involved ............... 161 136


4.4 Classification by types of actions / strategies ................................ 163 4.4.1 Classification based on direct vs. indirect approaches ............. 163 4.4.2 Classification based on structural vs. communicative approaches ................................................................................ 163 4.4.2.1 Classification within structural approaches........................ 165 4.4.2.2 Classification within communicative approaches .............. 166 4.4.3 Classification based on programme duration ........................... 171 5 Six-phase-model of the process leading to the implementation prevention programmes ......................................................................... 172 5.1 Phase 1: Basic research................................................................... 172 5.2 Phase 2: Prevention research .......................................................... 173 5.3 Phase 3: Concept phase .................................................................. 173 5.4 Phase 4: Development phase .......................................................... 173 5.5 Phase 5: Testing phase.................................................................... 174 5.6 Phase 6: Routine phase ................................................................... 175 6 Classification of Evaluation .................................................................. 176 6.1 Classification based on the kind of: data used (data dimension)... 176 6.1.1 Process evaluation vs. outcome evaluation vs. impact evaluation.................................................................................. 176 6.1.2 Structural data vs. process data vs. outcome data..................... 176 6.1.3 Classification system integrating two related concepts............ 177 6.2 Classification based on the state of the programme to be evaluated (state-of-programme dimension) ...................................... I 6.3 Classification based on the epistemological significance of the findings (methodological dimension) .......................................... 179 6.4 Classification based on persons in charge of evaluation (internal vs. external evaluation = evaluator dimension)............. 180 6.5 A four-dimensional classification concept based on established scientific terminology................................................................... 183 6.6 A contents oriented classification approach................................... 184 6.6.1 Ethical evaluation...................................................................... 185 6.6.2 Historic evaluation .................................................................... 186 6.6.3 Methodological evaluation........................................................ 187 6.6.4 Formative evaluation................................................................. 188 6.6.5 Feasibility evaluation ................................................................ 189 6.6.6 Monitoring of unexpected adverse side effects ........................ 190 6.6.7 Efficacy evaluation.................................................................... 190 6.6.7.1 Global proof of effectiveness.............................................. 191 6.6.7.2 Partial proof of effectiveness .............................................. 191 6.6.7.3 Historic deduction of effectiveness .................................... 192 6.6.8 Quality assurance (QA)............................................................. 192 137


6.6.9 Structural evaluation ................................................................. 194 6.6.10 Context evaluation .................................................................. 194 6.6.11 Impact evaluation .................................................................... 195 6.6.12 Efficiency evaluation = cost-benefit (CBA) and cost-effectiveness analysis (CEA)............................................ 195 7 Methodological aspects relevant to evaluating prevention programmes ........................................................................................... 198 7.1 The importance of a written study protocol ................................... 198 7.2 Types of outcome variables............................................................ 200 7.2.1 Primary efficacy variables vs. secondary variables .................. 201 7.2.2 Efficacy variables vs. surrogate variables (vs. intermediate: variables) .................................................................................. 203 7.2.3 Short-term, medium-term vs. long-term effects........................ 204 7.2.4 Outcome variables suited as "primary efficacy variables" in efficacy studies ......................................................................... 205 7.2.4.1 The role of knowledge, attitudes, personality characteristics, life skills, etc. as outcome variables ......... 206 7.2.4.2 The role of "any substance use" as outcome variable: A paradoxical relationship to "problematic substance use".. 206 7.2.5 Protective factors and risk factors - the problem of causality once more.................................................................................. 208 7.3 The problem of heterogeneity......................................................... 208 7.4 The influence of simultaneous interventions ................................. 210 7.5 Dependency on context .................................................................. 210 7.6 Generativity .................................................................................... 211 7.7 Effect-size and incidence of primary efficacy variable.................. 212 7.8 Power considerations - sample size................................................ 212 7.8.1 The basic principle of power calculation.................................. 212 7.8.2 A practical example for power considerations in prevention research ..................................................................................... 213 7.8.3 Practical implications in the evaluation of substance abuse prevention programmes regarding the sample size.................. 215 7.8.4 Problems with underpowered design: the publication bias ...... 215 7.8.5 Approaches to solve the problem of underpowered designs.... 216 7.9 Measurement problems with self-reported consumption............... 217 7.10 Dependency of observational units .............................................. 218 8 References.............................................................................................. 218

138


1 Introduction Most people concerned about the drug problem invest much hope in primary prevention. This is understandable: There cannot be any doubt that preventive interventions before a problem becomes manifest are far superior to any curative and confining actions after problems already cause harm. - But what are the best approaches to primary prevention? World-wide there is an abundance of primary prevention concepts. These concepts are extremely heterogeneous and most of them highly controversial. They differ greatly in content and theoretical concept. Approaches implemented with much enthusiasm by some experts are considered worthless or even counterproductive by others and vice versa. The public and decision makers - confused by contradicting prevention concepts and expert opinions - expect that properly conducted evaluation of existing programmes will yield reliable results, helping them to chose the most promising approaches and to reject ineffective and counterproductive strategies. Due to this expectation they demand more evaluation in the field of primary prevention and they mean a "proof of effectiveness". It is easy to claim that all prevention programmes should be evaluated properly, but what does "proper evaluation" mean? Very often the term "evaluation" is associated with effectiveness and interpreted as "prove of effectiveness", but is this the only sensible interpretation of "evaluation"? Does it make sense to demand that the effectiveness of any preventive intervention should be demonstrated in "decision studies" based on experimental or quasi-experimental designs and high methodological standards? What are the methodological problems and limitations we encounter in evaluating prevention programmes and how do we proceed if proof of effectiveness seems beyond reach concerning a specific approach? Before we can start to deal rationally with the above questions we should as a first step - try to clarify the terminology used. We have to identify vague and ambiguous terms and try to arrive at more complete and unequivocal concepts. The second step must be to identify and analyse the most central methodological problems and limitations we encounter in this area of research and to discuss the practical and theoretical implications of these problems.

139


The present project, involving 21 experts from 15 different countries in the frame of the "COST A6 Action - Work Group 2" focused at both of these goals. The scientific technique used to utilise the knowledge and experience of the participating experts and to arrive at a consensus at the end of the project was the so called "Delphi method" (→2.5).

2 Introductory remarks concerning the terminology 2.1 The use of cursive type faces and reference References to terms dealt with in more detail in other chapters are marked with the sign "→" followed by the number of the relevant chapter. The terminology considered central for the following article is printed in cursive type faces and included in the glossary.

2.2 Drug (ab)use vs. substance (ab)use The use of the term "drug" is very ambiguous. Even though it is generally accepted in the scientific community that the term "drug" includes prescription drugs, licit drugs and illicit drugs, it is still common practice to use the term "drug" as abbreviation for "illicit drug" as well. In order to avoid this equivocalness we decided to use the term "substance (ab)use" for "drug (ab)use" and "substance dependence" for "drug dependence".

2.3 The term "abuse" It makes sense to divide the term "substance use" into at least four categories: • no use • experimental use and recreational non-harmful use • non-addicted harmful use and • addicted use The category "harmful use" is commonly translated into "abuse" or "misuse" and persons using substances in a harmful way are commonly referred to as "abusers" or "misusers". We decided to go along with this 2 convention for simplicity reasons , at the same time being fully aware that

2

"Substance abuser" or "substance misuser" is much shorter than "person using substances in a harmful way".

140


there is an engaged controversy going on, particularly concerning the term 3 "abuse" .

2.4 "Effectiveness" vs. "efficacy" vs. "efficiency" The terms "effectiveness" and "efficacy" relate to the question if an intervention causes the desired effects and the term "efficiency" relates to the economic question, if an intervention causes the desired effects with a minimum of costs and efforts. The terms "effectiveness" and "efficacy" are commonly used synonymously in the everyday context but there is a certain distinction of usage in a scientific context. The term "efficacy" is primarily used when interventions are tested under controlled conditions, while the term "effectiveness" is primarily used in relationship to effects caused under everyday conditions 4 in naturalistic settings (Flay , 1986). We speak e.g. of "primary efficacy variables" (→7.2.1) and of "cost-effectiveness analysis" (→6.6.12). If we go along with this differentiation the relationship between "effectiveness" and "efficacy" gets rather complex. We are usually interested in the functioning of interventions under everyday conditions (effectiveness), but in order to measure effects reliably we have to control background conditions experimentally or at least quasi-experimentally . That way we end up assessing efficacy as a surrogate for effectiveness. In other words, assessing efficacy is the scientific way of indirectly assessing effectiveness. The question whether generalising from efficacy to effectiveness is justified in scientific research is commonly referred to as a matter of "external validity" (Campbell & Stanley, 1963).

3

E.g. the editors of the "International Journal of Addiction" reject articles using the term "drug abuse". In the chapter "Information for Authors Submitting Manuscripts", they state: "Living organisms are or can be abused; objects are either used or misused."

4

Efficacy trials provide tests of whether a technology, treatment, procedure or programme does more good than harm under optimum conditions. Effectiveness trials provide tests of whether a technology, treatment, procedure or programme does more good than harm when delivered under real-world conditions (Flay, 1986).

141


2.5 The "Delphi method" The name "Project Delphi" was given to an US Air Force sponsored Rand Corporation study, starting in the early 1950’s (Dalkey and Helmer, 1963). The term "Delphi" relates to the oracle in the ancient Greek city of Delphi, an institution that was consulted by contemporary decision makers if problematic decisions had to be made. The Delphi method, as a "self-correcting" approach to opinion polling, was popularised on a larger scale by Lindstone & Turoff (1975). The original goal of the Delphi approach was to arrive at better prognoses in complex situations via a consensus of opinions among experts, thereby circumventing obstructive and destructive elements of group discussions. Soon the approach was extended to facilitate the solution of other complex problems. Since then an innumerable number of very different Delphi projects have been published all over the world. Usually the first steps for a study co-ordinator to start off a Delphi project are to formulate a draft and guidelines defining intent and scope of the project, to contact knowledgeable experts in the field and to confront the participating experts with draft and guidelines. The panel of participating experts is then instructed to work on the draft by themselves, to comment openly and to answer the questions posed in the draft. The panellists can focus their efforts on any aspects they feel particular competent in, they may spend as much time as they want on single items without hindering the work of others participants. They are free - under the cloak of anonymity - to adjust their opinions about any questions put to them at earlier stages based on new information from other participants. The self-correcting Delphi approach involves a multi-stage iterative process of at least three rounds until a consensus is reached or until it turns evident that a consensus is not possible. The responses of the participants in every round are collected, synthesised by the co-ordinators and presented to the group again to start a new round of the process. The majority position as well as any minority positions are fed back to the group after every round, but the source of the positions is never disclosed to the panel.

142


3 "Evaluation of prevention" - everyday vs. scientific understanding 3.1 The term "prevention" - everyday vs. scientific understanding In the everyday language the term "prevention" is reserved for actions to prevent problem onset before a problem has started. In a scientific context the term "prevention" is usually understood in a much broader sense. The common classification of "prevention" into "primary prevention", "secondary prevention" and "tertiary prevention" (→4.2.1) lays down that "prevention" in a scientific context includes • interventions before first signs of a problem are present, • early intervention (when a problem exists but is not yet fully manifested - in order to prevent problem manifestation), • treatment and assistance to self-help (when a problem is manifest - to prevent further harm) and • relapse prevention (after a substance problem has been treated successfully - to prevent reoccurrence). In other words: "prevention", according to the everyday understanding is a subcategory of "prevention" in the scientific sense.

3.2 The term "evaluation" Evaluation according to the everyday sense is the process of determining whether a technique or a strategy is of any value. The central questions are: • "Does it work?" and • "Is it ethically justifiable?" Less important are questions like "How does it work?" or "Why does it work?"

143


Quite contrary is the scientific understanding of the term "evaluation". The latter term covers any strategies from • documentation and description via • hypotheses generating exploratory research (exploratory data analysis; EDA) through • hypotheses testing confirmatory research (confirmatory data analysis; CDA) (→6.3), it includes research efforts • while a first programme concept is being developed on a purely reflective base, using already existing information and logical principles (preformative evaluation phase = concept phase), • while a programme, based on this concept, is being formed (formative evaluation = development phase) • after a programme has been finalised, to judge its overall performance (first summative evaluation phase = testing phase) and • after a programme is applied on a routine basis (second summative evaluation phase = routine phase) (→6.1.2) and it includes research efforts • based on structural data, describing extent and setting of programme execution (structural evaluation), • based on process data systematically gathered while a programme is being executed (process evaluation), • based on outcome data gathered after programme execution concerning explicitly expected effects (outcome evaluation and • based on impact data gathered after programme execution concerning not explicitly expected effects going beyond the intended target groups (impact evaluation (→6.1).

144


The central differences between everyday understanding and scientific understanding of prevention can be summed up as follows: • "Evaluation" in the everyday sense • includes judgements based on personal values (ethical evaluation →6.6.1) • excludes mere description • excludes exploratory research • includes confirmatory research • "Evaluation" in the scientific sense • excludes judgements based on personal values • includes mere description • includes exploratory research • includes confirmatory research

3.3 How should we deal with the confusing conflict between the everyday meaning of "evaluation" and the peculiar way "evaluation" is understood in a scientific context Of great importance is the distinction between the everyday interpretation and the scientific interpretation of the term "evaluation" if statements like "All drug prevention programmes should be evaluated!" are formulated. • Everyday context According to the everyday understanding the statement "All drug prevention programmes should be evaluated!", translates into "If programmes are implemented to prevent drug problems before they start, specific research should be considered to check whether these measures bring about the desired success and whether they are ethically justifiable!". This demand is very specific but often unrealistic in the area we deal with. We have to realise that a sound scientific proof of programme effectiveness (→6.6.7) is commonly impossible on epistemological and economic grounds. This point will be dealt with later in detail (→7). • Scientific context In a scientific context the statement "All drug prevention programmes should be evaluated", translates into "If any measures in relationship to drug problems are taken, at least some aspects concerning programme execution or programme outcome should be recorded". This demand is rather vague but easy to fulfil under almost any circumstances.

145


If the terms "prevention" and "evaluation" are used without explicitly stating whether the scientific meaning or the everyday meaning is referred to, confusion is bound to happen. We should always to bear in mind that the public and policy-makers commonly refer to the everyday meaning, while programme developers, programme staff, professional evaluators and other scientists usually refer to the scientific meaning - and this conflict is hardly ever made an explicit topic when persons with a different background talk about prevention. If the public and policy-makers demand evaluation they usually expect a convincing answer to the question "Is the programme any good?" In line with the everyday interpretation of "evaluation" they aim at "programme effectiveness" (→6.6.7). Programme developers, programme staff, professional evaluators and other scientists, confronted with the demand to evaluate, quite often encounter insurmountable epistemological and economic limitations (→7). If this is the case, they have no means to assess "effectiveness" adequately - and here the scientific terminology comes in handy. Several other approaches within in the scientific concept of "evaluation" can always be considered without any mayor problems. The above described scientific understanding of the terms "prevention" and "evaluation" is solidly established in the scientific community by now. We may question if the way scientists interpret these terms contrary to the everyday understanding is sensible. We may stress that this ambiguity leads to avoidable confusion - but we have to be realistic and accept that we probably will have to live with these conflicting terminologies. In order to minimise further confusions, it makes sense though, to rigorously demand that any scientist referring to evaluation of prevention should always make explicitly clear to the persons he communicates with • that the terms "prevention" and "evaluation" in the scientific sense have little in common with the everyday meaning of these terms, • that the term "prevention" covers anything from prevention before a problem starts to treatment and relapse prevention, • that the term "evaluation" is commonly not associated with anything like a proof of effectiveness (→6.6.6) and • that both terms should always be specified in detail.

146


4 Classification of prevention programmes 4.1 Classification based on types of target persons There are three ways to classify prevention programmes based on target persons: • by the size of the target population (→4.1.1) • by specific characteristics of the target population (→4.1.2) • by the social environment the target persons belong to (→4.1.3) • by the way how target persons are reached: direct approach vs. indirect approach (→4.1.4)

4.1.1 Classification based on the size of the target population Some prevention programmes are of a very limited scope and others include many nations. According to the size of the target population we can divide into four categories: • international programmes • nation-wide programmes • regional programmes • local programmes This classification is neither complete nor clear-cut, but it addresses a very essential aspect in spite of these shortcomings. In terms of programme evaluation it makes e.g. a lot of difference whether the target group consists of all students in a country, of all students of a region or of the students in a single school.

4.1.2 Classification based on specific characteristics of the target population Sloboda & David (1997) divided prevention programmes into three categories. The classification is somehow related to the classification primary prevention, secondary prevention vs. tertiary prevention (→4.2.1), refers to a relevant dimension to categorise programmes but is not very precise in the above formulation. It doesn’t not allow to draw clear borderlines between the categories. 147


The three categories are: • Universal programmes reach the general population - such as all students in a school. • Selective programmes target groups at risk or subsets of the general population - such as children from drug users or poor school achievers. • Indicated programmes are designed for people who are already experimenting with drugs or who exhibit other risk-related behaviours. According to the above definition the relationship between target groups and programme types cab be characterised as follows: • The general population is target population for universal programmes (primary prevention). • Subsets of the general population not considered to be specific groups at risk are target populations for universal programmes or for selective programmes (primary prevention). • Groups at risk not using drugs (yet) are target groups for selective programmes or indicated programmes (secondary prevention). • Drug experimenters, drug users and drug addicts are target groups for indicated programmes (secondary prevention or tertiary prevention).

4.1.3 Classification based on the type of social environment Prevention programmes reach target persons in different social environments. A possible categorisation of programmes according to the social environments divides into five categories: • school • work • family • other institutions • free / spare time The five categories ("school", "work", "family", "other institutions", "free / spare time") mentioned are only examples and not a complete enumeration. Related to "school" there are similar categories ranging from "kindergarten" through "college" and "university" to "postgraduate courses". The unspecific term "other institutions" includes hospitals, prisons, etc. The term "free / spare time" stands for "youth centres", "clubs", "cultural events" etc. 148


4.1.4 Classification based on the way how target persons are reached (directly or indirectly) According to the way how target persons are reached prevention programmes can be divided into three categories: • direct approaches (aiming at primary target persons) • indirect approaches (aiming at intermediate: target persons) • mixed approaches (aiming at primary target persons and intermediate: target persons at the same time) The direct approach addresses primary target persons (e.g. school children) directly. The indirect approach addresses intermediate: target persons (mediating persons and institutions), who are expected to influence the primary target persons into the desired direction. Mixed approaches are addressed to primary target persons and intermediate: target persons simultaneously. Indirect approaches are e.g. • training of teachers • training of social workers / medical staff • training of youth workers • training of peers (peer group approach) • training of parents • training of policemen • training of journalists • and many others Since resources in prevention are usually limited, the indirect approach is a very economic way to increase the impact of a preventive strategies. After a programme has been developed, commonly it is not possible to finance a specialised programme staff to execute the programme on a large scale. The only way in these situations is to turn the programme over to intermediate: target persons (mediators, like teachers, counsellors, youth workers, peers, etc.), who are expected to act on the primary target persons according to the programme concept. This can only be effective if much time is invested in the training of these mediators.

149


To expect that e.g. an average teacher, counsellor, youth worker, peer etc. can relate a complicated message and execute a complicated programme without intensive training is naive. Written guidelines, even if they are very detailed and comprehensive, are not sufficient to implement programmes on a larger scale. Training before a programme starts is necessary, but mere training of intermediate: target persons in this context is not enough in order to retain a programme. Support groups, continuing training, supervision and/or intervision are essential as long as a programme goes on. Also situational factors have to be looked at. Is there a good team-work? In case the mediators are employed by an institution, does the director back up the programme and is there a habit of starting new things? etc. If structural problems are identified, the programme developers and/or programme staff have to go beyond working with the multipliers. E.g. one way to get more support through school administrations and teachers not directly involved in the programme could be offering support in drug policy development or assisting with acute drug problems, etc. If surrounding factors counteract the efforts of multipliers involved in a prevention programme, it is likely that the effects are drastically reduced and that the persons will soon lose their motivation to continue the programme adequately. Since direct approaches and indirect approaches are different strategies aiming at different target groups, this classification will also be mentioned under (Classification by types of actions / strategies →4.4).

4.2 Classification based on types of goals The are eight ways to classify prevention programmes based on types of goals based on • problem levels (→4.2.1) • the central strategy (→4.2.2) • substance specificity (→4.2.3) • number of substances involved (→4.2.4) • legal status of substances involved (→4.2.5) • consumption oriented goals vs. problem oriented goals (→4.2.6) • abstinence orientation vs. responsible use orientation (→4.2.7) • specific goals (→4.2.8)

150


4.2.1 Classification based on problem levels A very popular classification of prevention programmes based on types of goals according to problem levels divides into three categories: • primary prevention • secondary prevention • tertiary prevention Within the scientific understanding of "prevention" we can differentiate four distinct areas of preventive action. • Primary prevention is to prevent the onset of a substance related problem. • Secondary prevention is to intervene if a problem is likely to occur (prevention in high-risk-groups) and/or if a problem exists but is not yet fully manifested. In both cases the aim is prevention of problem manifestation. • Tertiary prevention - type A is to deal with a problem once it is fully manifested (prevention of further harm in addicts). • Tertiary prevention - type B is to prevent a problem from reoccurring again once it has been successfully treated (relapse prevention). Obviously the four distinct areas of preventive actions defined above cannot be unambiguously described with three categories. Therefore it has been suggested by members of the working group to introduce the term "quaternary prevention" as unique new entity for relapse prevention. This suggestion certainly makes sense, since the concept of relapse prevention is of increasing importance to the field. On the other hand, to propose a modification within an internationally already established classification system may cause a lot of confusion and therefore two thirds of the working group were against introducing a new category "quaternary prevention". A compromise within the classical classification system is to split tertiary prevention into tertiary prevention - type A and tertiary prevention - type B.

151


The above classification is still vague and needs some further specification to draw a more precise borderline • between primary prevention and secondary prevention on the one side and • between secondary prevention and tertiary prevention on the other side. Presently there exist two very different positions among experts, concerning the question where to draw the line. Both positions - we call them position A and position B - have been expressed in our working 5 group and since there was no clear preference for any of the two positions both positions will be presented here as equal.

4.2.1.1 Position A The central principle for position A is, to define the target problem in substance abuse prevention as "any substance use". Consequently • "experimental use and recreational use" translate into "existing but not fully manifested substance problem" and • "non-addicted harmful use" as well as "addicted use" translate into "full manifestation of the problem". This results in the following classification: • Primary prevention is to prevent non-users from starting to use substances. • Secondary prevention is to influence experimenters and recreational users to stop using drugs, to prevent them from using substances in a harmful or problematic way and/or to prevent them from getting addicted. • Tertiary prevention - type A is to prevent persons using substances in a harmful way from getting addicted and to help addicts to overcome addiction. • Tertiary prevention - type B or quaternary prevention is to prevent former addicts from relapsing.

5

Roughly a fourth of the working group was in favour of position A, another fourth was in favour of position B, still another fourth thought that both positions were acceptable, as long as there was an explicit referral to position A or B, and the last fourth meant that we should discourage using the classification into the four areas of prevention, because of existing ambiguities.

152


4.2.1.2 Position B The central principle for position B is, to define the target problem in substance abuse prevention as "problematic substance use". Consequently • "Experimental use and recreational use" are prior to problem onset. • "Harmful non-addicted use" translates into "existing but not fully manifested substance problem". • "Addicted use" translates into "full manifestation of the problem". This results in the following classification: • Primary prevention is to prevent non users, experimenters and recreational users from substances in a harmful way or getting addicted, • Secondary prevention is to prevent non-addicted persons using substances in a harmful way from getting addicted, • Tertiary prevention - type A is to help addicts to overcome addiction and • Tertiary prevention - type B resp. quaternary prevention is to prevent former addicts from relapsing.

4.2.1.3 Practical considerations Commonly programmes have an impact on all four levels of prevention at the same time, e.g.: • Giving information to an unselected group of people on negative consequences of substance use (primary prevention), may have an impact on users to use the substances in a less harmful way, on persons using substances in a harmful way not to get addicted, to motivate addicts to get rid of their habit (tertiary prevention - type A) and even to support former addicts not to relapse (tertiary prevention - type B). • Any successful treatment programme (tertiary prevention) reducing the number of addicts who finance their daily drug demand through pushing and dealing drugs, may reduce the availability of drugs and therefore may have an indirect primary preventive spin-off as well.

153


4.2.1.4 Conclusions For the above mentioned reasons we draw two conclusions. • As long as different positions where to draw the line between the four levels of prevention exist the concept only makes sense with an explicit referral to position A or position B. • The distinction between four levels of prevention is only a rough tool to organise one’s thinking in terms of target persons and goals, but it is not really suitable to systematically classify individual prevention programmes. Usually the effects of a prevention programme fall into more than one category!

4.2.2 Classification based on the central strategy A very simple and popular approach to classify programmes by types of goals according to the central strategy is the dichotomy: supply reduction vs. demand reduction. This classification is not complete though. Many programmes aiming at harm reduction do not necessarily result immediately in supply reduction or demand reduction. Because of this fact it makes sense to include the category "harm reduction" as an additional entity and arrive at the following trichotomy. • programmes aiming at supply reduction • programmes aiming at demand reduction • programmes aiming at harm reduction To a large degree approaches aiming at supply reduction are structural approaches and approaches aiming at demand reduction are communicative approaches, but demand reduction can be structural and supply reduction can be communicative as well. The harm reduction approach includes both structural and communicative elements to a similar extent (→4.4.2). It is a well established economic fact that supply and demand are not independent of each other. If we reduce / increase demand we automatically reduce / increase supply in the long run as well and vice versa. Similarly harm reduction is causally related to demand reduction and to supply reduction. Because of this fact the classification should be seen as a rough tool to organise one’s thinking in terms of target persons and goals, but it is not really suitable to systematically classify single prevention programmes. 154


4.2.3 Classification based on substance specificity Programmes based on types of goals according to substance specificity can be divided into • programmes aiming at substance related goals only; i.e. any programmes primarily designed and implemented to influence problems related to substance use • programmes aiming at substance related goals among others. This could e.g. be a health promotion programme, where substance related goals are only a fraction of the claimed effects. The differentiation between "programmes aiming at substance related goals among others" and "programmes aiming at substance related goals only" has some important implications concerning the definition of primary efficacy variables (→7.2.1) if a proof of effectiveness (→6.6.7) is intended. If too many primary efficacy variables are considered simultaneously in one confirmatory oriented study (→6.3) the adequate 6 statistical procedure "adjustment of the nominal significance level" renders it virtually impossible to reach any statistically significant results. Interventions of a broader socio-political scope very likely have a strong impact on substance use. Along these lines there are very many important projects going on that might have a tremendous influence on substance (ab)use but were never designed specifically to prevent substance (ab)use, e.g. if schools try to offer more interesting activities to their students, if communities create new attractive youth centres, if governments try to reduce unemployment, etc. Very often effects in terms of less substance (ab)use are not even considered at the beginning. These programmes without any explicit substance related goals at all are not subject to our considerations regarding the evaluation of actions to prevent substance (ab)use and therefore not included in this classification - at least until substance specific effects are claimed and demonstrated. These programmes are not subject to our considerations regarding the evaluation of actions to prevent substance (ab)use and these programmes should not be labelled "substance abuse prevention".

6

The most common method to accomplish this task is the so called "Bonferroni Method" to adjust the nominal significance level by dividing through the number of independent significance tests; in our case by the number of primary effectiveness variables.

155

Evaluations of Substance Use Prevention Programmes 7

The term substance related goals should not be mixed up with substance 8 related techniques . Programmes without any substance related techniques are definitely subject to our considerations - at least as long as substance related goals are considered explicitly.

4.2.4 Classification based on number of substances involved An important aspect to classify prevention programmes based on types of goals according to the number of substances involved is whether the programmes are more or less substance specific. • Some programmes target specific substances specifically, e.g. "ecstasy only", "nicotine only", "alcohol only" etc. • Other programmes target a class of substances; e.g. "stimulants", "inhalants", "all licit drugs", "all illicit drugs", "all prescription drugs", etc. • Still other programmes target all substances, regardless if they are licit, illicit or prescription drugs.

4.2.5 Classification based on legal status of substances involved The classification based on types of goals according to the legal status of substances involved discriminates between • licit drugs • illicit drugs • prescription drugs • one of the four possible combinations of these three categories The classification into licit, illicit and prescription drugs is rather arbitrary and reflects cultural and historic developments rather than substance specific elements. Because of these circumstances it makes sense to treat licit, illicit and prescription drugs together as one topic in many settings. On the other hand, if specific problems are associated with specific substances it may make sense to focus attention on this specific substance.

7

E.g. "changing drug taking behaviour", "changing problems related to drug consumption", etc.

8

Substance related techniques are "specific drug refusal training", "information on harmful drug effects", etc. Substance unrelated techniques are "assertiveness training without any drug specific elements", "offering drug-free alternatives, like leisure time activities", etc.

156


One example of an ecstasy-specific prevention approach is handing out ecstasy-information brochures and/or offering chemical testing of ecstasy quality at locations where many people use ecstasy.

4.2.6 Classification based on consumption oriented goals vs. problem oriented goals Goals of substance abuse prevention programmes can be divided into • consumption oriented goals and • problem oriented goals Consumption oriented goals are e.g. "life-time prevalence of substance use (including experimental use and recreational use)", "time of consumption onset", etc. and problem oriented goals are "any problems directly or indirectly caused by substance use". Even though "consumption" and "problems caused through consumption" are usually highly correlated - if a certain substance is not consumed at all, it cannot possibly cause any problems and if it is consumed by many persons, it is more likely that substance related problems evolve - it is very essential to decide which class of goals is of primary interest. In order to illustrate the importance of this differentiation let us consider three prevention programmes: • "Programme A" is very successful in reducing the amount of problems resulting out of substance (ab)use in high-risk-groups but at the same 9 time increases experimental use in low-risk-groups . "Programme A" is a success in terms of problem reduction (less harmful use, less misuse in spite of increased experimental use of substances) but a failure in terms of consumption oriented goals (more life-time prevalence of substance use) • "Programme B" is very successful in reducing experimental use of substances in low-risk-groups but increases drug misuse in high-risk10 groups . "Programme B" is a failure in terms of problem reduction (more harmful use, more misuse) but a success in terms of consumption oriented goals (less life-time prevalence).

9

E.g. regular information on bad quality of certain substances may reduce problems and complications caused in regular recreational users and/or abusers but it may simultaneously have an advertising effect for nonusers, resulting in increased experimentation with these substances.

10

A very emotional "Don't use drugs approach!" might lead to a polarisation among target persons. This polarisation may result in less substance experimentation in the low risk group and in more heavy and harmful use in the high risk group.

157


• "Programme C" is very successful in reducing experimental use of substances in low-risk-groups and decreases drug misuse in high-risk11 groups as well. "Programme C" is a success in terms of consumption oriented goals (less life-time prevalence) and a success in terms of problem reduction (less harmful use, less misuse). If the effects of interventions in terms of consumption and associated problems go into the same direction, like in Programme C, the above differentiation in consumption oriented goals and problem oriented goals is of minor importance. If the effects contradict each other though - like in Programme A and Programme B - it is quite evident that the only adequate approach is problem orientation. This position is e.g. in line with the official "Standards for Documentation and Treatment Evaluation for the German speaking Countries", developed by an expert committee of German, Dutch, Swiss and Austrian experts (German Society for Addiction Research and Addiction Therapy, 1992). The authors proposed a multidimensional problem oriented approach to assess treatment effectiveness in the field of substance (ab)use including the following five dimensions: "substance (ab)use", "health", "social integration", "integration into the work market", "delinquency". To come back to the above mentioned three Programmes: Only when "Programme C" is evaluated, consumption parameters and problem parameters can serve as substitutes ("surrogate variables" →7.2.2) for each other. As will be discussed later, mere correlation is not sufficient to justify using any variable as a surrogate for another. The common assumption that correlating variables can serve as substitutes for each other is very convenient - e.g. it is much easier to assess "life-time prevalence of 12 substance use" than "problems caused by substance (ab)use" - but the assumption is not automatically true, as the above examples "Programme A" and "Programme B" illustrate.

11

A very emotional "Don't use drugs approach!" might lead to a polarisation among target persons. This polarisation may result in less substance experimentation in the low risk group and in more heavy and harmful use in the high risk group.

12

Sometimes the argumentation is based on the well known Ledermann Model suggesting that any increase or decrease of overall alcohol consumption results in a non-linear but concordant change in the amount of heavy use. On the other hand there are numerous empirical findings challenging this position. The relationship between average consumption and heavy alcohol consumption varies quite a bit from region to region and similar things are observed regarding illicit drugs as well. E.g. in the Netherlands ethnic minorities are characterised through less prevalence of drug consumption and at the same time through a much higher rate of heavy use (Korf, 1995).

158


4.2.7 Classification of programmes based on abstinence orientation vs. responsible use orientation Some persons are strictly abstinence oriented in relationship to all licit and illicit substances, while others believe that responsible use is acceptable at least in some areas. The two positions could be labelled • abstinence orientation and • responsible use orientation The dichotomy "abstinence orientation" vs. "responsible use orientation" should not be mixed up with the dichotomy "consumption orientation" vs. "problem orientation" (→4.2.6). A person who is convinced that "no use of a certain substance" is the central goal to pursue may at the same time aim at "problem oriented goals" and not at "consumption oriented goals" for practical reasons. In other words, he may conclude that "Programme A" (reducing the overall amount of problems caused through substance use at the cost of increasing experimental use and/or recreational use;) is superior to "Programme B" (reducing experimental use and/or recreational use at the cost of increasing the overall amount of problems caused through use of this substance), even though he is basically in favour of abolishing any substance use completely if this goal could be reached. Very often the abstinence vs. responsible use controversy is focusing around legal drugs, like alcohol, nicotine, caffeinated beverages, etc., but prescription drugs, not considered to be absolutely necessary from a medical point of view (like headache pills, sleeping pills, etc.) and illicit drugs considered to be less harmful (like cannabis) play a certain role in this controversy as well. The abstinence vs. responsible use controversy is commonly focusing on a few substances, based on medical, legal and cultural arguments and highly connected to fundamental personal values (ethical evaluation; →6.6.1). Usually the argumentation is not directly related to the empirical question, whether strict abstinence orientation or responsible use orientation (→4.2.7) as background philosophy is more effective in terms of problem reduction.

159


4.2.8 Classification based on specific goals There are lots of different variables used in evaluation studies to assess the efficacy of prevention programmes. The following list - far from being complete - states a list of possible variables in this context without discussing their usefulness for that purpose. The question which of the named variables are useful criteria to assess efficacy in drug prevention programmes will be dealt with in chapter 7.2.4. • changing substance use • reduction of any substance use (including experimental use) • later onset • less life-time prevalence • reduction of problematic substance use, e.g. • less frequent use • less heavy use • less daily use • less harmful use = less adverse drug effects • less use of contaminated street drugs • less injecting and/or needle sharing • changing problematic behaviour related to substance use • improvement of health • better social integration • better integration into the work market • less delinquency • increasing knowledge • changing attitudes • modifying personality characteristics • development of certain life skills, e.g. • better stress management • better self-esteem • higher assertiveness • resistance to social pressure • better social skills • better decision making capacity • better problem solving capacity • better self-development • better alternative activities to substance use • better self-control • improved autonomy and ability to choose 160


4.3 Classification based on theoretical aspects The are two ways to classify prevention programmes based on theoretical aspects: • based on the role theory plays (→4.3.1) • based on the kind of: theory involved (→4.3.2)

4.3.1 Classification based on the role theory plays Some prevention concepts are based on a theoretical concept, where the theory behind the approaches is explicitly stated and backed up with literature. Commonly the background theories are traditional behaviourist, cognitive or non-directive "Rogerian" oriented, but there are also other theoretical backgrounds relevant, like a psychoanalytic approach to reveal the unconscious underlying motives for substance (ab)use, etc. Other concepts combine different elements from different theoretical backgrounds referring to the different backgrounds. This approach could be called "multi-theoretical" or "eclectic approach". Some concepts use a variety of elements that have been used in different programmes without making reference to a theoretical background. Anything that has been useful in other programmes and is somehow plausible to the authors is put together to a concept. This approach could be called "non-theoretical". This results in a classification trichotomy: • Theory oriented approaches = one theoretical framework • Eclectic approach or multi-theoretical approaches = more than one theoretical framework • Non-theoretical approaches = no specific theoretical framework

4.3.2 Classification based on the kind of theory involved The theories and models prevention approaches referred to are countless and because of this we can only give a rather unsystematic sample of theories and models used to explain why certain preventive interventions have certain effects. In classifying theories and models we have to realise that we are confronted with different degrees of specificity.

161


The highest level constitutes • heterogeneous large classes of theories (e.g. a classification of theories based on behavioural concepts), the next level refers to • large scope theories associated with an innumerable number of important contributors (e.g. "traditional behaviourism" or "psychoanalytic oriented theories") and the lowest level refers to • models and limited scope theories primarily associated with the original authors (e.g. McGuire’s "Inoculation theory" or Ajzen & Fishbein’s "Theory of reasoned action"). An example for heterogeneous large classes of theories based on behaviour concepts is the following system • rationalistic theories: People act according to their knowledge • utilitarian theories: People act in order to maximise gain • learning theories: People act according to previous experience • voluntaristic theories: People act according to drives and needs • deterministic theories: People act according to pre-established patterns of behaviour (genetically or socially determined) A sample of relevant large scope theories, far from being complete is • traditional behaviourism • cognitive behaviourism • non-directive "Rogerian" theory • psychoanalytic oriented theory • communication theory • and many more A sample of relevant models and limited scope theories, again far from being complete is: • cognitive (cognitive-affective) models, like Ajzen and Fishbein’s theory of reasoned action • social learning models (e.g. Bandura; Aker) • development models, like Hawkins & Weis’s social development model • communication / persuasion models, like McGuire’s Inoculation theory • social norms approaches, like Elliot’s Social control theory • problem behaviour theory (Jessor & Jessor) • life skills approach (Botvin) • healthy lifestyles approach (WHO) • functional equivalents to drug use (Silbereisen)

162

Definitions - Concepts - Problems 13

• and many more

4.4 Classification by types of actions / strategies The are two ways to classify prevention programmes based on theoretical aspects: • based on direct vs. indirect approaches (→4.4.1) • based on structural vs. communicative approaches (→4.4.2) • based on programme duration (→4.4.3)

4.4.1 Classification based on direct vs. indirect approaches A useful classification of prevention programmes based on types of actions / strategies is the following: • direct approaches, addressing the target group directly and • indirect approaches, focusing on persons who interact with the target group and • mixed approaches, using both strategies in one programme simultaneously Since direct approaches and indirect approaches are different strategies aiming at different target groups, this classification has already been referred to under (Classification based on types of target persons →4.1.4).

4.4.2 Classification based on structural vs. communicative approaches A common classification divides into • structural approaches = environment oriented approaches and • communicative approaches = person oriented approaches Structural approaches aim at changing the environment - including the social environment - in a way that individuals are more likely to behave in a desired way. Communicative approaches are approaches that try to influence individuals or their social environment directly to behave in a desired way. Usually structural approaches try to reduce the availability of substances and communicative approaches try to reduce the demand for substances, but there are also structural approaches trying to reduce demand (e.g. more and better youth centres, better families) and 13

For more detailed information on this topic see e.g. Bell & Battjes, 1984 or Künzel-Böhmer, 1994

163


communicative approaches to reduce supply (e.g. information about high penalties for dealing with drugs).

164


4.4.2.1 Classification within structural approaches Samples of structural approaches are e.g. • Projects to reduce social problems that are related to substance (ab)use, like welfare and income support, employment and housing programmes, free clinics to treat people with psychic problems, counselling for people with partner problems, financial problems, legal problems etc. • Projects to offer drug-free alternatives, like youth centres, sporting facilities, low priced cultural events • Projects to organise support for at-risk individuals by providing social support through family, peers and counsellors • Police efforts to reduce the availability of drugs through arresting users, importers, dealers and seizing drugs, • Methadone maintenance programmes to get addicts off the street and reduce dealing and pushing drugs • Strict prescription procedures for certain drugs • Laws to oblige restaurants to sell non alcoholic beverages, • Higher taxes on cigarettes or alcohol, banning cigarette machines • Changing legal purchase ages • Having a clear policy on licit and illicit drugs in school • Routine urine tests with a consequent threat of being expelled from school or work place or losing passport • A threat to be fired from a job if working under substance influence or not to be hired to begin with • Suspension of drivers’ licenses if a motor vehicle is driven under substance influence • etc.

165


Structural approaches can be divided into • non-repressive strategies (e.g. projects to reduce social or psychiatric problems), • repressive strategies (e.g. sentences for users and dealers of illicit drugs) and • strategies that cannot be classified into either category (e.g. strict prescription procedures for certain drugs are not repressive towards the consumers but they are quite repressive against medical doctors and pharmacists, who do not behave accordingly.)

4.4.2.2 Classification within communicative approaches Communicative approaches are e.g. • Deterrence = Scare tactics • Selective information giving • Factual information giving • Life skills training • Normative approaches • Persuasion to take a commitment against drugs Deterrence = scare tactics is showing selectively some extreme possible negative outcomes of substance use and giving a biased extract of research evidence. This approach was traditionally a popular strategy concerning illicit drugs. More and more research evidence has demonstrated though that this approach is ineffective or counterproductive. If persons treated with scare tactics are confronted with reality for the first time and realise that they have been systematically misinformed they very often • begin to distrust all previous information including important factual elements that might prevent hazardous use and other severe problems, • begin to rely on "street information" or information from peer drug users that might be biased into the opposite direction and • totally lose trust in those people who misinformed them in the first place that way cutting the adults off from the position to intervene positively in case substance problems arise.

166


Selective information giving is an approach that is not usually addressed as a separate approach, but often practised in the everyday life of prevention. You don't give wrong information but you keep back information that make substance use look less harmful or more popular among relevant peer groups. The latter is especially important in context with normative approaches. Factual information giving is an ambiguous topic in the present substance abuse prevention discussion. There is one commonly mentioned argument against factual information giving, namely the fact that some empirical research evidence suggests • that providing factual information doesn’t have a measurable immediate impact on substance (ab)use of target persons - even though it does increase the information level. On the other hand there are lots of arguments for a certain amount of factual information giving. • Since factual information is so ubiquitous, we don't know what would be the effect if people did not know that nicotine smoking or heroin consumption etc. are dangerous. Before people knew that smoking is harmful more people smoked. • There is a "health belief model" predicting that most people will do what is best for them if they are well informed. • Factual information giving undoubtedly is an important factor to make repressive strategies work. Threat of repression to individuals can only be effective if the target persons are aware of possible adverse consequences and that there is a high chance that these 14 consequences will occur . • Many administrators, parents and teachers, who want to act against substance (ab)use, are not satisfied with approaches that do not inform about substances at all. If no substance specific materials are supplied to them, they will look for substance specific information by themselves and develop their own information block - information that may be much worse than anything prevention experts would come up with.

14

E.g. in Austria many students experience that being caught by the police with small amounts of drugs does not result in any immediate consequences for them (the case is dropped without any legal consequences) and they do not realise that their names are registered in police files for ever - which later may have some severe implications if they want to enter some types of schools (like schools for nursing) or if they want to join the public service.

167


• Children and students are permanently (mis)informed about substances through many different sources (mass media, books and electronic media, peers, parents etc.) As a result many drug -inexperienced children develop myth-ridden ideas about illicit drugs which work in two directions: They exaggerate and misinterpret the positive aspects ("All illicit drugs cause an incredible, beautiful, hallucinogenic feeling!") as well as the risks ("Trying any illicit drugs just once causes usually inevitable addiction, insanity and death within no more than a few weeks!"). The attractiveness of illicit drugs is to some degree associated with these myths. It therefore makes sense to believe that the attractiveness of illicit drugs will be reduced if these unrealistic expectations are substituted by something like a balanced view of the phenomenon. • School in western societies is an institution, where passing on correct information and correcting misconceptions is the central issue. The idea is to educate students systematically in order to get knowledgeable adults who can make sensible decisions about their own lives, even if their decisions don’t fit with our expectations or wishes. This concept is incompatible with a notion to refrain from correcting wrong conceptions about drugs, just because the information approach has no short acting effect on the reduction of substance use. • As a matter of fact most existing school based prevention programmes include at least some factual information anyway. 15

Life skills training = promotion of psychosocial competence The central concept of life skills training is to teach students techniques and skills to deal more successfully with everyday situations of life. Based on a conception that substance (ab)use is to a large extent related to deficits in some areas of social and personal competence it is believed that those cognitive behavioural oriented approaches indirectly influence the rate of substance (ab)use towards a desired direction.

15

A good introduction is given in the brochure "Life Skills Education in Schools" (WHO, 1994)

168


Life skills can be divided by contents into • decision making = teaching strategies to make rational decisions • problem solving = assistance to deal constructively with problems in our lives • creative thinking = contributes to decision making and problem solving by enabling persons to explore available alternatives and consequences of actions and non-actions • critical thinking = ability to analyse information and experiences in an objective manner • effective communication = ability to express ourselves verbally and non-verbally in an appropriate manner • social skills = interpersonal relationship skills = ability to relate in positive ways with other people • self-awareness = recognition of ourselves, our strengths and weaknesses, our desires and dislikes • empathy = the ability to imagine what life is like for other persons • coping with emotions = ability to recognise our emotions, see how they affect us and to respond to them appropriately • coping with stress = stress management = techniques to cope with stress • self-development = training to foster self-development • attainment of autonomy = help to reach autonomy, in particular the ability to choose • improving self-esteem = teaching to accept and play down failures as well as appreciate one’s strengths • resistance to social pressure = assertiveness training = training to identify and resist social pressures • improving self-control = teaching to resist behaviours which provide a high level of instant gratification without taking negative medium and long-term effects into account • utilisation of alternative activities to substance use = provide experience in activities that are believed to be incompatible with drug use • value clarification is a strategy to help persons to identify and eliminate contradictions within their value system and contradictions between their behaviour and their values. According to this definition the approach is classified correctly as a life skills approach. On the other hand, since the outspoken idea is to convince target persons that drug taking does not correspond with the personal value system, there may be a highly persuasive element involved and if this is true the technique is closer to normative approaches. 169


Another way to subdivide life skills by substance specificity is: • substance specific techniques, like resistance to drug taking offers, vs. • substance unspecific techniques, like listening skills, expressing feelings, etc. The different techniques mentioned to increase life skills are neither complete nor independent from each other. E.g. in order to increase assertiveness you have to work on self-esteem and so on. Normative approaches try to point out to target persons that the social norms concerning substance use are "conservative" in nature. In other words that a vast majority of the population is against using illicit drugs and against misusing licit drugs. Some of the techniques involved are called "norm setting", "pledge", "goal-setting" and some forms of "value clarification" • Norm setting is an attempt to establish a conservative norm regarding drug use through pointing out that drug use represents a minority behaviour and that the vast majority is against using drugs. This approach makes use of statistics based on opinion polls and discussing the practical implications of these statistics. • Pledge is a method to encourage individuals to adopt an open commitment against substance (ab)use. This strategy uses stickers, engagement in specific groups taking a public stand against substance (ab)use, etc. • Goal-setting is a technique to encourage the adoption of achievement orientation and to develop goals believed to be incompatible with substance use • As already mentioned value clarification can be seen as an element of life skills training and as an element of a normative approach, depending on the way it is used. Presently normative approaches are widely spread in the USA and they are a central part of the official US-American prevention doctrine, while the European doctrines of prevention primarily stress non-normative elements. Persuasion to take a commitment against drugs Whenever someone tries to change attitudes or behaviour of target persons into a specific direction, a certain amount of persuasion is almost inevitable. In particular normative approaches are commonly quite persuasive in nature and they transport social values that could be subject to an intensive and controversial discussion (ethical evaluation →6.6.1).

170


4.4.3 Classification based on programme duration Since everybody is permanently exposed to an abundance of information concerning substance use from a variety of different sources and since personal skills, attitudes etc., assumed causally related to harmful substance use are influenced by many different sources as well, it is naive to expect that under normal circumstances any small programme (short duration and/or low-exposure programmes) can cause massive long-term effects on the target behaviour. This of course doesn't rule out that small interventions can have a large impact in individual cases or that specific consequences of a small intervention - e.g. intensive media coverage - can cause a strong unexpected impact on a large group of people ("generativity" →7.6). Programmes can be classified according to programme duration into • Short-term programmes • Medium-term programmes • Long term programmes Programmes can also be classified according to the average number of exposures per target person into • Low-exposure programmes • Medium-exposure programmes • High-exposure programmes An exposure could be seeing a TV-Spot once, one lesson of a school based programme, etc. Since single exposures can be of different intensity and duration, not all programmes can be adequately described based on the simple number of exposures though. Depending on the programme characteristics different units may be more appropriate. This could e.g. be the total time of exposure, etc. It is not easy to define cut-off-points between "short / low", "medium" and "high" in the above classifications and the work-group didn’t arrive at a consensus in this issue. In the following table the median values of the suggested cut-off-points as well as the range of different suggestions

171


within the working group are depicted: • upper threshold short-term programmes: median value: 3 months (range: 1 - 6 months) • upper threshold medium-term programmes: median value: 12 months (range: 6 - 24 months) • upper threshold low-exposure programmes: median value: 10 units (range: 4 - 12 units) • upper threshold medium-exposure programmes: median value: 30 units (range: 10 - 50 units)

5 Six-phase-model of the process leading to the implementation prevention programmes Theoretically, under ideal conditions, the process leading from elementary research activities to routine application of prevention programmes can be divided into six stages. This six-phase-model leading from first basic research activities to the routine application of a prevention programme is an ideal conception. Commonly some steps are omitted or undertaken in a reduced manner. • Phase 1: Basic research (→5.1) • Phase 2: Prevention research (→5.2) • Phase 3: Concept phase (→5.3) • Phase 4: Development phase (→5.4) • Phase 5: Testing phase (→5.5) • Phase 6: Routine phase (→5.6)

5.1 Phase 1: Basic research Basic research relevant to our considerations covers all research areas useful for the development of prevention programmes that are not directly aiming at prevention issues. This covers e.g. areas like • the development of valid and reliable tools to assess (ab)use and other relevant variables, • epidemiological research to identify problem areas and trends concerning substance (ab)use, • research to explain the development of substance (ab)use and addiction, etc. 172


5.2 Phase 2: Prevention research Prevention research is research directly aiming at prevention issues; that is research undertaken with the explicit goal to develop causal models and theories to influence the emergence, perpetuation, reduction and termination of substance (ab)use in specific target groups. This includes the identification of risk factors and protective factor.

5.3 Phase 3: Concept phase The process of developing a prevention concept in the concept phase, before the process of improving and forming the programme based on a first preliminary draft starts, is commonly called "preformative phase" (→6.1.2).The concept phase starts with a plan to develop a new prevention programme, is characterised by the development of a concept and ends with the first preliminary draft of this programme. In this purely reflective phase ethical considerations (ethical evaluation →6.6.1), an analysis of already existing scientific evidence (historic evaluation →6.6.2) and the inclusion of methodological and statistical principles to judge whether the existing research justifies the conclusions the authors drew (methodological evaluation →6.6.3) are of central importance. The basis for historic evaluation in this context is research evidence gathered in phase 1 (basic research →5.1), research evidence gathered in phase 2 (prevention research →5.2) and evaluation results about already existing prevention programmes.

5.4 Phase 4: Development phase The goal in the development phase is to work on concept and first preliminary draft developed in the concept phase (→5.3) and to arrive at a final programme version. The central idea in this phase is to identify obvious shortcomings and to repeatedly improve the draft based on these findings before the concept is finalised. Very often small trials and pilot studies are sufficient to evoke major improvements to the draft version under scrutiny. The development phase - the phase when the programme is formed - it is commonly referred to as "formative phase" (→6.1.2). The unsystematic, exploratory approach in the development phase naturally focuses on feasibility, efficacy and adverse reactions, but since these aspects are not analysed systematically and in any sense finally, we

173


should speak globally of formative evaluation (→6.6.4) and reserve the terms "feasibility evaluation" (→6.6.5), "efficacy evaluation" (→6.6.6) and "monitoring of averse side effects" (→6.6.6) for the more systematic approaches in phase 5 (testing phase →5.5). It is very important to remain in the development phase sufficiently long. This phase allows flexible, rapid and relatively cheap improvements of intermediate: draft versions. Only if one can conclude that obvious shortcomings have been identified and eliminated, it make sense to finalise the programme and enter the testing phase (→5.5). The research procedures necessary to confirm (→6.3) the value of a final programme version are much more costly and time consuming. All these efforts are wasted if it finally turns out that the programme didn’t meet the original expectations and has to enter the development phase once more.

5.5 Phase 5: Testing phase After a programme has been finalised in the development phase (phase 4 →5.4), the next step is to confirm (→6.3) its usefulness. Under ideal conditions this is done via implementation on a larger scale under controlled conditions (experimental or quasi-experimental design ) to systematically test • whether the programme is feasible under real life conditions ("feasibility evaluation" →6.6.5), • whether expected effects can be demonstrated under real life conditions ("efficacy evaluation" →6.6.6) and • whether any undesired averse side effects emerge ("monitoring of averse side effects" →6.6.6). In case economic or methodological limitations (→7) render it impossible to design a methodologically appropriate experimental or quasi-experimental study to assess "global programme effectiveness" (→6.6.7.1), it is usually at least possible to test parts of the causal model behind the prevention approach; an approach that could be called "partial proof of effectiveness" (→6.6.7.2) or to base the proof on existing empirical evidence ("historic deduction of effectiveness" (→6.6.7.3).

174


5.6 Phase 6: Routine phase The routine phase is characterised by final programme implementation and routine application. Once the effectiveness of a finalised programme has been demonstrated, it is legitimate to implement a programme on a larger scale for practical purposes. In this last phase - since effectiveness has already been established - any further proof of effectiveness is not central any more. In this phase it is important • to ensure that the programme is carried out adequately and that the level of programme execution doesn't wear off in the long run (quality assurance →6.6.8) • to judge where the programme is being used, by whom, how often, etc. (structural evaluation →6.6.9) • to judge if generalisation to the intended target population is appropriate (this aspect is associated with the terms external validity (Campbell & Stanley, 1963), generalizability (Cronbach, 1972), historic cultural comparability) and • whether central preconditions have changed in the meanwhile and whether relevant situational background conditions in the area of programme application are still similar to those in the experimental condition (context evaluation →6.6.10). • to continuously observe if originally not explicitly expected - desired and undesired - effects seem to emerge (impact evaluation →6.6.11) and • whether the programme effects are massive enough to justify the use of this programme (efficiency evaluation →6.6.12) A fundamental programme reconception (phase 4: development phase →5.4) followed by an other testing phase (phase 5 →5.5) is indicated, whenever strong indications emerge that the programme execution wears off dramatically that the programme is hardly applied any more that the context changed in central dimensions and that relevant averse long-term effects have emerged.

175


6 Classification of Evaluation 6.1 Classification based on the kind of: data used (data dimension) 6.1.1 Process evaluation vs. outcome evaluation vs. impact evaluation A very popular way to classify evaluation is the trichotomy process evaluation vs. outcome evaluation vs. impact evaluation (e.g. Clayton and Cattarello; 1991). Process evaluation is concerned with the systematic assessment of the process from the beginning of a programme to the end of the follow-up period. This covers explicitly expected as well as not explicitly expected effects and includes • all interventions by the programme staff (input), • all reactions of the target population (output) and • of all relevant conditions that might have an influence on the relationship between input and output (context). Outcome evaluation is concerned with the question: "Could the objectives of the programme be attained?". This approach focuses on explicitly expected effects in the target group? This includes at least one follow-up measurement, commonly a baseline measurement as reference value and never any recording of the process while the programme is going on. Impact evaluation addresses the question: "Did any positive or negative effects occur that were not explicitly planned?" In other words, "impact evaluation" is synonym to "assessment of not explicitly expected effects". Since most not explicitly expected effects happen in populations, not originally intended as target persons, one of the major implications in the above defined sense is the assessment of effects in persons not belonging to the original target group (E.g. to assess if a school based programme targeting students has an impact on their parents as well).

6.1.2 Structural data vs. process data vs. outcome data Closely related to the classification "process evaluation vs. outcome evaluation vs. impact evaluation" is the classification "structural quality 176


vs. process quality vs. outcome quality" (Donabedian, 1980). The latter classification has been developed in relationship to quality assurance in hospitals and it is based on the kind of: data used, just like the former classification. • structural data describe the structural context, e.g. the place of intervention, the qualification of the persons executing the programme, characteristics of target persons, etc. • process data describe parameters of programme execution and • outcome data describe intervention effects The classification by Donabedian differs from the former classification in three aspects. Donabedian includes the concept of "structural evaluation", he doesn’t mention "impact evaluation" and "process evaluation". It is defined in a narrower sense, primarily focusing on programme input (→6.1.1).

6.1.3 Classification system integrating two related concepts Since the two above mentioned classification concepts address the same dimension (kind of data used), it makes sense to integrate both into an extended system and to add the important category "context data" →6.6.10, not included in either system. Context variables assess relevant background conditions such as drug-specific attitudes, knowledge, experiences, fashions, etc. This results in the classification • structural evaluation based on structural data • process evaluation based on process data • outcome evaluation (=evaluation of explicitly expected effects) based on outcome data and • impact evaluation (=evaluation of not explicitly expected effects) based on impact data • context evaluation based on context data

177


6.2 Classification based on the state of the programme to be evaluated (state-of-programme dimension) Another very common classification is the dichotomy formative evaluation vs. summative evaluation (Scriven, 1967, 1991). • Formative evaluation is evaluation in the formative phase; i.e. while a programme is still being developed and not yet final. The purpose in this phase is to form (develop and improve) the programme as well as its background theories. • Summative evaluation stands for evaluation in the summative phase; i.e. after a programme is not worked on any more. The aim of summative evaluation is to sum up expected or unexpected effects caused by a final programme. Scriven (1981) explained the distinction between formative and summative 16 very illustratively by citing a statement by Bob Stake : "When the cook tastes the soup that’s formative; when the guest tastes the soup that is summative." Scriven pointed out that some authors split the "formative phase", into a • "preformative phase", a purely reflective phase, while a concept and a first preliminary draft are being developed without any practical, empirically oriented steps • "formative phase" in the closer sense of the word; an empirical phase, where a preliminary draft is subjected to repeated small trials and small pilot studies followed by modifications until all obvious major short-comings have been removed and the programme is declared final. "Summative evaluation", the way Scriven defined it, covers two very different phases: • The programme testing phase that could be referred to more precisely as "first summative phase" and • the routine phase that could be named "second summative phase".

16

Scriven doesn’t offer an exact citation

178


If we relate concept of Scriven to the Six-phase-model (→5) we can identify a close correspondence to the last four stages: • "Phase 3 - concept phase" is equivalent to "preformative phase" • "Phase 4 - development phase" is equivalent to "formative phase" • "Phase 5 - testing phase" is equivalent to "first summative phase" • "Phase 6 - routine phase" is equivalent to "second summative phase"

6.3 Classification based on the epistemological significance of the findings (methodological dimension) A third particularly essential way to classify evaluative research is the trichotomy descriptive evaluation vs. exploratory evaluation vs. confirmatory evaluation. This classification is central for any quantitative empirical research and based on the kind of conclusions a researcher can legitimately draw out of his data on epistemological grounds. The dichotomy exploratory data analysis (EDA) vs. confirmatory data analysis (CDA) is primarily associated with Tukey (1977). Tukey was the first well-known statistician who explicitly stressed the importance of exploratory techniques in the field of empirical research. Even though the basic concepts behind this approach have been dealt with in every elementary book on statistics as well as research methodology by now, the implications are still widely neglected in many areas of applied research. Descriptive evaluation is a synonym for collecting and recording data, to document phenomena, to categorise them and to summarise the findings, without directly aiming at the formulation of new hypotheses and theories. Description constitutes the lowest level of scientific research. Exploratory evaluation aims at beyond mere description. Exploratory research goes from collecting basic information in rather unexplored scientific areas to the hypothesis driven development of new models and theories. Exploratory research constitutes the second level of scientific conduct. This form of data analysis • aims at discovery of new phenomena, • creates impulses to develop new hypotheses and theories, • is principally divergent oriented, • is not subject to stringent methodological rules and • all results have a preliminary character. 179


There are no strict rules concerning procedures in exploratory studies. Basically anything that has a chance to give a better insight into relevant phenomena is possible and acceptable, as long as it is made explicitly clear that the results of the exploratory phase are not final in any sense. Confirmatory evaluation is not concerned with the discovery of new phenomena and the formulation of new hypotheses but only with the scientific proof of existing hypothesis and the effectiveness of techniques and programmes. Confirmatory evaluation uses the principles of probability theory and inductive statistics to discriminate substantial effects from irrelevant chance effects. Confirmatory research constitutes the highest level of scientific activities. This form of data analysis • aims at confirmation of hypotheses and theories • is principally convergent oriented, • is subject to stringent methodological rules and 17

• all results may be considered scientifically proven in a certain sense . If it is feasible on epistemological and economic grounds, all exploratory results should eventually be tested in confirmatory studies. There is no fundamental objection to include descriptive, exploratory and confirmatory sections in one evaluation study, as long as the descriptive part, the confirmatory part and the exploratory part are explicitly separated right from the beginning of a project.

6.4 Classification based on persons in charge of evaluation (internal vs. external evaluation = evaluator dimension) In the course of programme development and application it naturally makes a lot of difference who organises and directs programme application and programme evaluation.

17

The term "certain sense" refers to the epistemological fact that research designs and statistical data analyses can never provide anything like a proof of hypotheses and theories in the "strict sense" of word. This is evident on two grounds: • Statistical inference is always based on several implicit and explicit background assumptions, which may be correct, possibly are very likely to be correct, but cannot be proven. We are confronted with background assumptions and not with background truths. • Even if it is very realistic to assume that central background assumptions are true and if the statistical results suggest that it is sufficiently unlikely that the observed results occurred merely by chance, we have to realise that "unlikely" is not a synonym for "impossible". There is nothing like an inductive proof of hypotheses (Popper, 1975).

180


Basically this can be done by • programme developers • programme staff • decision makers • external experts There are different factors jeopardising the objectivity of evaluation depending on the parties involved: • Programme developers usually want to sell their product and/or gain scientific credits for developing successful programmes. They are naturally interested in positive evaluation outcome and if they plan an evaluation it is considered to be an "internal evaluation". • The situation of the programme staff - considered to be internal experts as well - is much more diverse. • Sometimes the staff is hired specifically for a prevention programme and economically dependent on the programme. Under these circumstances the staff naturally is highly interested in positive outcomes. • Some programmes rely on volunteers and committed professionals (like teacher, counsellors etc. doing prevention within their regular working time). These persons are not economically dependent on the programme but they naturally prefer to demonstrate that their efforts have made sense (effort justification). • Sometimes persons are assigned to execute a programme within their regular work duties. Confronted with extra work without extra pay, some of them wouldn’t be unhappy if this programme could be proven to be ineffective. • Decision makers, who are in the position to decide if and which prevention programme will be implemented, have to justify the expenditures towards their superiors and towards the public. To start out with they need simple facts to help them decide which programme is superior and after programme implementation they prefer positive evaluation results to justify their initial decision. On the other hand, if they want to stop financing ongoing programmes they are in favour of negative evaluation results. Commonly they neither have the time nor the scientific know-how, to plan an adequate evaluation study without professional help, but nevertheless some evaluation projects are co-ordinated by decision makers themselves. • Many people demand that evaluation projects should be directed by external experts. They state that programme developers, the programme staff and decision makers have personal interest in specific evaluation 181


outcomes and claim that only external experts can be objective. At first sight this line of argumentation seems to make sense, since external experts are paid to do the job objectively and they receive their fees independent of evaluation results. At the second sight though it becomes evident that external experts are personally involved as well. They have to meet the expectations of their clients, in order to get future contracts. A too sceptical outlook concerning the feasibility of an evaluation project, a complicated design that is much more expensive than the design of their competitors, a long follow-up period and/or results that are contrary to the expectations of the clients may cause serious problems to acquire future projects. In addition to these vital economic interests external experts commonly have specific scientific interests as well (publication of the results, acceptance by fellow scientists, etc.) and this is another source of a possible bias. Considering the specific interests of all parties involved in an evaluation project, it makes sense to involve all parties to some extent in the course of the events. In phase 4 (development phase = formative phase →0) the project management naturally should be in the hands of programme developers (internal evaluation). They usually have an adequate methodological background to design appropriate procedures and they are most familiar with programme structure as well as background theories. If they need methodological assistance they should consult external experts but this is not necessary if they are competent enough in relevant disciplines. Since the results of the formative phase are not final in any sense, the possible lack of objectivity is acceptable. After a programme has been finalised (phase 5: testing phase = first summative phase →0), the project management (responsibility for study design, control of data quality, data analysis and data interpretation) of any confirmatory evaluation (→6.3) project should preferably be in the hands of external evaluators. The latter should closely co-operate with all parties involved but at the same time be as independent in their decisions as possible from programme developers and decision makers. In phase 6 (routine phase = second summative phase →0) the programme staff and external experts, play an equal role. Systematic data collection and documentation should be part of any programme staff’s routine work. This documentation should be used by the staff to assess and improve the quality of programme application (quality management →6.6.8) and it

182


could serve as data basis for external evaluation (quality control →6.6.8) as well.

6.5 A four-dimensional classification concept based on established scientific terminology The scientifically established classification systems introduced in chapter 6.1 through 6.4, refer to four very distinctly logical categories that shouldn’t be mixed up. The common notion to interpret formative evaluation to be synonymous with process evaluation or exploratory research and to interpret summative evaluation as synonymous with outcome evaluation or confirmatory research is clearly in contradiction to the definitions mentioned and has to be rejected. Even though process evaluation and exploratory techniques usually play a superior role in the formative phase, there is no methodologically sound reason to definitely rule out the use of outcome data or confirmatory techniques in a formative phase and the same holds true for the relationship between outcome evaluation, confirmatory data analysis and the summative phase. The integration of these 4 dimensions to one four-dimensional classification system results in the following structure: • data dimension (→6.1) • structural data • process data • outcome data (=explicitly expected effects) • impact data (=not explicitly expected effects) • context data • state-of-programme dimension (→0) • development phase = preformative phase • development phase = formative phase • testing phase = first summative phase • routine phase = second summative phase • methodological dimension (→6.3) • descriptive approach • exploratory approach • confirmatory approach • evaluator dimension (→6.4) • internal evaluation 183


• external evaluation

6.6 A contents oriented classification approach The four-dimensional classification concept (→6.5), even though it integrates the most common scientifically established classification concepts is not complete though. All four dimensions together are only applicable for prospective oriented empirical evaluation studies. This doesn’t include any value oriented approaches (ethical evaluation →6.6.1), any evaluation on existing data (historic evaluation →6.6.2) and any evaluation on logical basis (methodological evaluation →6.6.3). For the latter areas only the state-of-programme dimension (→0) and the evaluator dimension (→6.4) are appropriate as well. Undoubtedly the four-dimensional classification concept can be a useful tool to communicate more precisely about evaluation, but since the concept is not complete, it makes sense to round up the scientific terminology with a complementary contents oriented classification approach. The following list - being conceptualised along these lines - is neither complete nor clear cut and it is partly overlapping with dimensions of the four-dimensional classification concept. The list has to be understood as an attempt to arrive at a set of categories by contents that are not overlapping and to define them properly. Possibly this attempt will evoke further suggestions from within the scientific community to improve on the list. • ethical evaluation (→6.6.1) • historic evaluation (→6.6.2) • methodological evaluation (→6.6.3) • formative evaluation (→6.6.4) • feasibility evaluation (→6.6.5) • monitoring of unexpected adverse side effects (→6.6.6) • efficacy evaluation (→6.6.7) • quality assurance (QA) (→6.6.8) • structural evaluation (→6.6.9) • context evaluation (→6.6.10) • impact evaluation (→6.6.11) • efficiency evaluation = cost-benefit (CBA) and cost-effectiveness analysis (CEA) (→6.6.12)

184


6.6.1 Ethical evaluation There are many different beliefs and ideologies concerning substance use and misuse in society, concerning substance abuse prevention and concerning evaluation. If experts are asked to evaluate programmes and policies scientifically, usually their first step is to judge whether the techniques and strategies suggested are ethically acceptable; i.e. if these suggestions correspond with their personal values. Several preventive approaches that have been applied in the course of history - like "death penalty for mere drug consumption", "highly manipulative misinformation", etc. - are nowadays strongly objected by most experts and decision makers. They oppose these approaches on ethical grounds regardless of the question, whether these approaches might be effective or not. Other strategies are rigorously rejected by some persons and exuberantly welcomed by others. Relevant values relate to many different aspects, from goals of prevention through techniques of prevention to the ethics of research: • Some people feel that the primary goal of public substance abuse prevention should be to eliminate any substance use - including alcohol - to the furthest possible degree, while others feel that responsible use of some of these substances is fully acceptable. • Some people strongly oppose any techniques they think to be manipulative, authoritarian, dishonest. etc., while others interpret the same techniques in a different light and still others express that the end justifies the means. • Some people strongly oppose some scientific methods, like following up individual patients based on treatment records for scientific reasons, coercive drug tests in schools to have reliable consumption data or combining of different personal databases to estimate prevalence, asking very private questions in population surveys, etc. while others do not even understand these scruples. These personal convictions of evaluators naturally have a strong impact on process and results of any evaluation. Value judgements about the goals and strategies always play an important role in the public dispute with prevention experts and among them, but if scientific evaluation of programmes and programme implementation is considered, this form of value oriented ethical evaluation, being in conflict with the objectivity demand in science, is commonly excluded from consideration, underemphasised or disguised as logical or factual rather than 185


value-oriented. Since value based judgements naturally influence programme design and evaluation procedures to some degree it is far better to make ethical evaluation an explicit topic, than to ignore the influence of personal values in the process - claiming scientific objectivity. Values relevant in this context can e.g. be classified into • moral values (ethical standards, human rights) • political values (effectiveness and credibility of authorities, stability of governments) • social values (functioning of family, corporation, social network, community) • individual values (personality development, maturation, integration) • monetary values (costs and gains in relation to resources) • symbolic values (coded messages in symbols) • emotional values (satisfaction of emotional needs) • spiritual values (religious / spiritual enhancement) Ethical evaluation is particularly important in the concept phase (→5.3), but naturally plays an important role in all later phases as well.

6.6.2 Historic evaluation A central step whenever an expert is confronted with the task to develop a new prevention programme and to judge if a considered strategy will work (concept phase →5.3) is to base the expertise on already existing evidence. Already existing data can be labelled "historic data", a terminology established in clinical trials methodology, it makes sense to call this very approach "historic evaluation". The use of historic data for evaluative purposes is naturally not restricted to the concept phase (→5.3). This strategy plays an important role in any of the following three phases as well. Whenever anything doesn’t work out as expected in the development phase (→5.4) or in the testing phase (→5.5) it also makes sense to utilise existing evidence to explain the failure and/or to identify promising alternatives. Commonly persons asked to judge routinely applied prevention programmes (routine phase →5.6) will not only study specific evaluation protocols but utilise independent historic evidence as well.

186


Historic evaluation can be categorised into • experience based historic evaluation (= experience based expertises) and • research based historic evaluation (= research based expertises) The central question for an experience based expertise is, whether experiences of a single person or a group of persons suggests that a proposed programme could work. The central question for a research based expertise is, whether scientific research suggest that a proposed programme could work. There are several approaches available to utilise research findings: • The most common and most simple form are literature overviews to identify consistencies of findings across different studies regarding the proposed programme. 18

• A more sophisticated approach is called meta-analyses . Scientists doing meta-analyses aggregate single research results to one global result using specific quantitative methods. Mostly historic evaluation is based on both sources: research findings and personal experience at the same time. • In case such a historic evaluation is not to be based on one single expert only, the most common approach is to constitute work groups to arrive at a consensus. • A more sophisticated way to utilise the personal experience of several experts using a Delphi technique (→2.5).

6.6.3 Methodological evaluation Methodological evaluation is concerned with the question if previous and future empirical research is done methodologically / statistically appropriate. Methodological evaluation related to previous research is closely related to historic evaluation (→6.6.2), since conclusions based on empirical research projects in this context should only be accepted if design and

18

Sometimes the documentation quality concerning experimental procedures and resulting data is not sufficient for a proper metaanalysis and many papers have severe methodological shortcomings. If a large portion of relevant papers has to be excluded from the metaanalysis on those grounds, a simple literature overview very often remains the only sensible thing to do.

187


method of data analysis justify the conclusions of the authors (internal validity) and if generalisation to the intended target population is appropriate (external validity). This form of methodological evaluation is particularly important in the concept phase (→5.3). Methodological evaluation in relationship to future research is relevant in all phases where evaluation strategies are planned. Many specific problems likely to evolve in the evaluation of prevention programmes can easily be anticipated on logical and statistical grounds - and it doesn’t make sense to spend much time and money on projects that have not the slightest chance of yielding any interpretable results.

6.6.4 Formative evaluation The term "formative evaluation" relates to a specific phase in the course of programme development (development phase = formative phase →5.4) and to the specific strategy appropriate for this phase. Usually formative evaluation is organised by the programme developers (internal evaluation →6.4). The task in the development phase is to check whether preliminary programme drafts and consecutive drafts work as 19 expected. If elements of the draft version under scrutiny do not work , the reasons for the problems have to be identified. Based on these findings adjustments of the programme must be considered to circumvent the problems jeopardising the programme. Under ideal conditions a rapid sequence of checking and adapting, based on a series of small trials and pilot studies, is kept up until the programme is feasible and likely to be effective. This continuous process of investigation and modification should continue until all obvious major shortcomings have been removed successfully. Evaluation in this phase is concerned with the complex relationship between programme activities, intermediate: variables (7.2.2) and programme objectives (outcome variables →7.2). This requires an open minded observation of the process (process evaluation →6.1) from the first intervention until the programme is finished, including continuous assessment of intermediate: variables that could possibly explain the mechanism relating the programme steps to the outcome variables. This

19

E.g. Booklets handed out are rejected and/or never read. Teachers signed up to take over certain actions do not read the instructions. A technique to get students involved to participate in a programme is opposed by their parents, etc.

188


also includes considering structural constraints (structural evaluation →6.6.9) influencing opportunities for behaviour change. Many problems in preliminary programme drafts turn obvious immediately in most practical situations: Because of this fact confirmatory research strategies (→6.3) involving specific hypotheses, complex designs, large sample sizes and strict methodological rules are of little importance in this phase. The investigator should look divergently into all directions (exploratory orientation →6.3) to identify unexpected problem areas. Exploratory approaches can help to formulate hypotheses, why and how specific interventions cause specific results but never answer the question whether a programme is actually effective in terms of programme objectives. A scientifically sound efficacy claim can never be based on exploratory research alone. In an exploratory context any results regarding effects are of preliminary nature and should be formulated as hypotheses rather than as empirical facts - at least until further evidence is gathered in confirmatory studies. In this chapter we argued that the main emphasis of formative evaluation is on process data that the main orientation is exploratory and that it usually is not necessary to involve external evaluators (→6.4), but this does not at all rule out observing outcomes and impact (→6.1) before a programme is finalised, this naturally also doesn’t rule out confirmatory strategies if they are appropriate and this naturally doesn’t rule out using external experts if the team of programme developers decides that this may be of value for the project.

6.6.5 Feasibility evaluation If it turns out that the proposed programme staff is incapable of using certain techniques, if target persons consequently reject central elements of an approach, if important messages are commonly misinterpreted, etc., we have to consider fundamental modifications to make the programme feasible. As long as the programme cannot be carried out under real life conditions, it makes little sense to spend much energy on effectiveness considerations (→6.6.7). The hypothetical question "Would the programme work if it could be executed?" is not very useful as long as we know that the programme is not practicable at all. Since it doesn’t make any sense to design costly efficacy studies for programmes that are not feasible, feasibility evaluation should always be considered prior to any form of efficacy oriented approach.

189


Feasibility considerations are of interest in all evaluation phases, particularly in the development phase (→5.4) and in the testing phase (→5.5). The evaluation strategies are very different in these two phases though. • The assessment of programme feasibility in the development phase via small trials and pilot studies, an unsystematic, exploratory approach (→6.3), is part of formative evaluation (→6.6.4) and therefore should not be labelled "feasibility evaluation" to avoid misunderstandings. • The term "feasibility evaluation" should be reserved for the systematic assessment of programme feasibility using larger samples in the testing phase.

6.6.6 Monitoring of unexpected adverse side effects The number of potential problem areas is almost unlimited and new problem areas may arise if contextual conditions change. Therefore monitoring of unexpected adverse side effects is a central aspect in every phase of evaluation. As long as adverse side effects have not yet been identified, the strategy naturally has to be divergent, exploratory oriented (→6.3) focusing on process data (→6.1). Once specific problems are apparent, convergent, confirmatory strategies (→6.3) focusing on outcome data (→6.1) are appropriate as well, especially in the testing phase (→5.5).

6.6.7 Efficacy evaluation The central question most people associate with programme evaluation is whether the programme under scrutiny works. In other words if it is effective in achieving the predefined goals (primary efficacy variables →7.2.1). In case economic or methodological limitations (→7) render it impossible to design methodologically appropriate experimental or quasi-experimental studies to assess programme effectiveness in a global fashion (global proof of effectiveness →6.6.7.1), it usually is at least possible to test parts of the causal model behind the prevention approach (partial proof of effectiveness →6.6.7.2). In some cases the proof of effectiveness can even be derived from an empirically based theory (historic deduction of effectiveness →6.6.7.3). The question of programme effectiveness naturally plays an important role in all phases of programme development and application, but the question is most central in the testing phase (→5.5)

190


6.6.7.1 Global proof of effectiveness Global proof of effectiveness is the experimental or quasi-experimental demonstration that the programme is capable to evoke the desired effects in the target group. This approach comes as close to a proof of effectiveness as possible but naturally it is not a "proof" in the literal sense of the 20 word . The term "desired effects" refers to the primary efficacy variables (→7.2.1). Studies set up to assess effectiveness in the above defined manner are commonly called "efficacy studies" or "decision studies". (The term "decision studies" refers to the fact that important decisions - if any possible - should be based on this type of study.) Translated into the established evaluation terminology the terms "efficacy study" or "decision study" translate into "confirmatory outcome evaluation in the testing phase". This form of evaluation - like any confirmatory studies (→6.3) has to correspond with a high level of research methodology and constitutes the primary source to claim programme effectiveness. Whenever possible we should consider a global proof of effectiveness (efficacy study) after a prevention programme has been finalised, but since epistemological and economic limitations (→7) commonly render this approach unrealistic, it does not make much sense to define this inductive (statistical) approach as an indispensable standard in the field of drug prevention evaluation.

6.6.7.2 Partial proof of effectiveness If a global proof of effectiveness (→6.6.7.1) is unrealistic for any given programme, evaluators may aim at the experimental or quasi-experimental proof that parts of the underlying causal model are correct. E.g. if a programme is based on the assumption that higher social competence immunises against drug problems and if it is unrealistic to demonstrate that the programme is effective in reducing the future rate of drug problems, it may at least be possible to investigate if the programme is capable of increasing social competence. The better the causal relationship between the intermediate variable (→7.2.2) "social competence" and the primary efficacy variables (→7.2.1) "less drug problems" is scientifically established, the higher is the importance of the partial effectiveness claim

20

A "high probability" is not identical with "certainty" and therefore this demonstration cannot be considered to be a proof in the literal sense of the word (compare e.g. Popper, 1975).

191


based on this variable. Since the central assumption that increasing social competence reduces the risk to develop drug problems is not challenged in this type of research, these studies should not be classified as "efficacy studies" or "decision studies". Whenever a stringent causal relationship between those intermediate: variables a design is based on and primary efficacy variables has been established scientifically beyond doubt, these intermediate: variables may be classified as surrogate variables (→7.2.2). Using surrogate variables in a scientific design is almost equivalent to using primary efficacy variables. To give an example: If we design a programme to reduce the AIDS risk in a target group, and if we cannot wait until we are in the position to diagnose manifest AIDS in the target persons, it is perfectly acceptable to assess the HIV-Infection rate in the follow-up period. Cross-validated positive HIV-Tests can be regarded as sufficient indicators (surrogate variables) for a future manifestation of AIDS.

6.6.7.3 Historic deduction of effectiveness Historic deduction of effectiveness, i.e. proper historic evaluation (→6.6.2) and methodological evaluation (→6.6.3) may be sufficient in given situations, if the fact that the programme will work, can be derived solely by already well established scientific evidence. If we know that a certain strategy works, we do not have to subject it to further tests over and over again - at least as long as the approach is not severely challenged by new scientific facts. To give an example: Since there is enough sound evidence available that treating open wounds with antiseptic substances reduces the infection rate dramatically, it is perfectly acceptable to formulate a standard for any institutions that open wounds should be treated accordingly. The vast body of scientific evidence in this case is sufficient to base such a decision on historic evidence.

6.6.8 Quality assurance (QA) Internationally the concept of quality assurance (QA) plays an increasing 21 role . Some authors understand "QA" in a narrower sense as "evaluation of implementation quality under routine conditions" others in a very wide sense as one more synonym for "evaluation" and some understand the term

21

WHO puts much emphasis on adequate QA-Measures in the field of Health Facilities (e.g. Bertolote, 1994) and many national administrations parliaments in member states adopted laws to set up quality assurance structures within their area of authority.

192


somewhere in between. There are many related terms that are sometimes understood as synonyms for QA (quality control, quality management, quality assessment, etc.) and sometimes defined in a very different manner. Almost every author defines the above terms differently and there seems to be no convergence towards a majority position up till now. In this chaotic situation it makes sense to suggest a solution based on utility considerations - and really useful is only the narrower definition as "evaluation of implementation quality under routine conditions". We positively don’t need any more synonyms for the vague and ambiguous term "evaluation", even though the literal meaning of the words "value" and "quality" would justify such a wide conception. QA utilises primarily process data (→6.1) and QA can be organised internally (→6.4) by the programme staff and externally (→6.4) by independent evaluators. Since the control aspect is predominant in the case of external evaluators and the management aspect if the programme staff themselves try to improve the quality of their work, it makes sense to clearly differentiate between the two approaches through using the following two terms: We propose • quality control for external strategies • quality management for internal strategies. According to the suggested definition, QA is restricted to the period after a programme has been implemented on a routine basis (routine phase →5.4). In this phase we have to expect that programme effectiveness (→6.6.7) has already been demonstrated that the programme is feasible (→6.6.5) that the interventions are ethically justified (→6.6.1), etc. Now continuous QA measures are necessary to guarantee that the original programme structure is not modified essentially and that the level of programme execution doesn't wear off in the long run. The situation in routine phase is quite different from the situation in the previous phases. People involved in the development (development phase →5.4) and testing of a new programme (testing phase →5.5) are usually trained well by the programme developers, they are selected by the programme developers, they are highly motivated and they have access to supervision by the programme authors whenever this is necessary. After a programme has been implemented routinely on a larger scale (routine phase) though, the original programme developers are commonly not available to the programme users any more. Often the only source of information is a written programme manual, leaving many details open. Since the persons carrying out prevention programmes on a routine basis 193


are on the average much less motivated than the original programme staff, programmes implemented on a routine basis may wear off quite rapidly. The purpose of QA is to identify and stop this deterioration process. To give a illustrative example for the above definition: If QA is considered in a hospital unit treating diabetics, the purpose of QA is not to investigate if Insulin is effective in Diabetics. It is only concerned with questions like: Does the staff apply the substance in the correct dosage? Are basic hygiene standards met in application? Do the patients accept the therapy? Are they informed well enough to be able to continue therapy by themselves after being dismissed from hospital? etc.

6.6.9 Structural evaluation Structural evaluation, like quality assurance (→6.6.8), takes place primarily in the routine phase (→5.6) and is related to the term structural quality. This primarily descriptive oriented approach aims at structural components of programme execution (structural data →6.1.2). E.g. in relationship to a school based prevention programme: How many schools were involved? How many classes within each school were involved? How many teachers and students were involved? Was there specific training for teachers? If yes, how many teachers went through this training? etc. or in relationship to a media campaign: How many people saw the spots? Logically speaking structural conditions are necessary but not sufficient conditions for programme success. To give an example: If target persons do not see a preventive TV-spot, they cannot possibly be influenced directly by the spot, but having seen the spot does not necessarily imply that the spot caused the expected preventive effects.

6.6.10 Context evaluation Prevention programmes are always developed for certain target groups under specific historic and cultural conditions (context). Relevant background variables (context variables) such as drug-specific attitudes, knowledge, experiences and fashions are different in different cultures and subject to rapid change. Programmes that prove to be effective under certain contextual conditions may turn out ineffective or even counterproductive under different conditions.

194


Evaluation results should never be generalised to other situations characterised by a different context without further research. Because of inevitable changes of context as time goes by even very effective programmes may end up outdated and only close monitoring of context can guarantee that these changes are recognised in time and that the necessary adaptations of the programme are done. Even though very relevant contextual changes can evolve in all phases of programme development, this type of evaluation is most central in the routine phase (→5.6).

6.6.11 Impact evaluation The term "impact evaluation" relates to a specific data quality (not explicitly expected effects) and has been dealt with in chapter 6.1.1. Impact evaluation is particularly important in the routine phase (→5.6) and at the beginning primarily exploratory oriented (→6.3). After relevant effects are identified, it is possible to include those effects in confirmatory research approaches (→6.3).

6.6.12 Efficiency evaluation = cost-benefit (CBA) and cost-effectiveness analysis (CEA) Efficiency evaluation includes two very different approaches: Cost-effectiveness analyses (CEA) and cost-benefit analyses (CBA). Both are techniques for comparing positive and negative consequences of alternative strategies - commonly to justify previous expenditures or to allocate further resources. CEA and CBA are related in concept and purpose but not identical. CBA compares the value of the benefits obtained from a programme with its costs. In this case the alternative is to use the programme or not to use the programme. There are two questions related to this approach: • Is a programme worthwhile at all? (Do the benefits outweigh the costs?) and • Given a certain budget, which projects should be realised to maximise the gains expressed as difference between total benefits and costs? In this framework very different projects can be compared. Even the question, Is it better to invest more into treatment facilities or is it better to improve the economic infra-structure? is possible within this framework.

195


CEA compares the efficiency of a given amount of resources to achieve some objective. The two central question are: • Which one of a set of programmes is most economical in terms of costs to reach a similar result? and the other way round • Which set of equally expensive programmes is more effective in terms of outcome? The central problem involved with CEA / CBA is that there are usually lots of different costs and different benefits to be considered in a single analysis and • this requires a common denominator to enumerate all relevant costs and benefits on one scale or • a method to keep all those costs and benefits constant that cannot be expressed by one chosen common denominator. In CBA all benefits as well as costs are usually evaluated in monetary 22 terms . Effects that cannot be expressed in common terms are ignored or somehow related to a common term. E.g. effects like "better quality of life", "years of life gained", etc. may result in increased productivity, an aspect that can be expressed as financial gain or loss for society. Since CEA is not interested in the overall value of a programme or in comparing programmes with different goals, but only in the question which programme is more effective to reach a certain goal, it is much easier to use non-monetary criteria as well. The trick is to reduce the number of incomparable dimensions through keeping some of them constant. E.g. if the monetary costs of a set of programmes are identical, there is no problem involved with only comparing non-monetary measures like "quality of life" or "years of life gained" without trying to convert them into monetary measures. The idea to do CBA / CEA is very appealing, since it makes sense to use sparse public resources in a way to reach a maximum impact. Unfortunately there are severe problems involved that make sensible approaches quite unrealistic in many areas.

22

Some authors (e.g. Jolicoeur et al, 1992) claim that the common denominator is always monetary, others stress that it only has to be the same unit (Yates, 1994).

196


At first sight there are many basic dimensions that seem promising as criteria for a CBA / CEA, but at second sight many of them turn out to yield quite absurd results. To give some examples • If the criterion "overall costs for the public" is used to evaluate successful programmes to prevent or quit smoking, it may turn out that the productivity of the target persons is not increased very much (benefits) but that the overall public costs (pensions, medical treatment, etc.) increase enormously because the persons live longer. Taking this criterion, effective prevention programmes would have to be classified as counterproductive and counterproductive prevention programmes would turn out to be optimal. • If the gross national product is taken as a criterion, any therapy, even if 23 it is absolutely ineffective, adds positively to the balance . The more a therapist charges for his therapy the better in terms of this criterion. • Very often even the CEA approach to reduce the number of relevant dimensions to one by keeping all others variables constant renders impossible on logical grounds. If more than one measure has to be considered at the same time (e.g. "quality of life" vs. "years of life gained"), and if a given treatment influences one variable into a positive direction and the other variable into a negative direction, subjective value judgements are necessary to arrive at a common unit for comparison - and this makes any results highly arbitrary. The CEA and CBA approaches require exact and quantitative results regarding programme outcomes. In areas where many measurement problems concerning central dimensions are unsolved and where it is hard to prove that any effects can be attributed to a programme under scrutiny, it is too early to engage in serious CEA or CBA.

23

The reason for this absurdity is that any work of individuals that is officially paid for, adds positively to the gross national product while any work not officially paid for, doesn't add to the gross national product. The quantity of personal productivity is only assessed through the money the persons officially get for their work (assessed by the internal revenue service) and not by the estimated real value of their production. An absurd consequence of this concept is that people building their own houses, volunteers in community projects, house-wives etc. do not add anything to the gross national product, while people doing absolutely ineffective work for a high salary add a lot to the gross national product.

197


7 Methodological aspects relevant to evaluating prevention programmes Evaluation of primary prevention in the field of illicit drugs is an area characterised by severe methodological problems. Some of these problems can be solved easily with appropriate strategies and moderate efforts, other problems require large efforts - that may not be justified in many specific situations - and still other problems constitute insurmountable research limitations of economic and/or epistemological nature. The following chapter 7 addresses some of the most striking methodological problems relevant in this area of research and stresses the importance of some fundamental research principles to avoid substantial errors.

7.1 The importance of a written study protocol Final statements on the value of any prevention programme, like any final statements based on empirical research, should rely on confirmatory research (→6.3). The ideal approach to assess programme effectiveness is undoubtedly a confirmatory outcome study (→6.3) aiming at a global proof of effectiveness (→6.6.7.1). If only a partial proof of effectiveness (→6.6.7.2) or a historic deduction of effectiveness (→6.6.7.3) is sensible and/or possible, all relevant research these approaches depend on, should be of confirmatory nature. Results primarily based on exploratory research (→6.3) are of limited trustworthiness and their hypothetical character has to be stressed. One central criterion for confirmatory research projects is the existence of a detailed written study protocol, stating the hypotheses, all details on the research design and the planned methods to analyse the data prior to beginning with the study. The written study protocol serves several purposes: • If a researcher is forced to write down all planned steps in detail, the chance that he recognises implicit contradictions, tautologies and practical limitations before starting the project is increased to a large extent. • If all procedural steps, including data analyses, are written down in advance, the likelihood of scientific opportunism is greatly reduced.

198


This is very important, since scientific opportunism renders the logic of 24 testing statistical hypotheses worthless . • Since many relevant details of research projects are of no interest to most readers and since journals usually restrict the size of research articles, reading the protocol is very often the only way to judge the appropriateness and stringency of implications. There should be a basic scientific standard stating that confirmatory research projects should always be based on pre-existing written study protocols. These protocols should be made available to anybody, who wants to read them before and after the project is finished. A second standard should specify that relevant deviations from the protocol have to be fully documented in any publications based on the study. If very severe deviations occur, mere documentation of these deviations is not sufficient though. In this case another newly designed confirmatory study has to be considered. The deviations from the study protocol can be classified into • Aspects concerning the quality of sample (target population) This includes aspects like: • To what degree was the target population reached by the programme actions? • To what degree did the target population participate in the programme? • Was there a systematic selection bias in the sense that specific subgroups (e.g. persons with an increased risk) did not enter the programme or dropped out early?

24

Scientific opportunism includes e.g. systematically not documenting insignificant or unexpected significant results, testing several effectiveness variables without α-adjustment (="fishing for significances"), increasing the sample size based on unplanned interim analyses after the study already started, experimenting with different ways to aggregate data, trying different statistical models consecutively on the same data, etc. until publishable significant results can be produced in the end.

199


• Aspects concerning the quality of the programme (input, resources, activities undertaken): This includes aspects like: • To what extent were the programme activities implemented? • Were the persons responsible for the execution of the programme capable of doing this job? Whenever the successful execution of a specific programme depends on specific experience by the programme staff, the evaluation has to allow the staff to get experienced, before it makes sense to assess programme effectiveness. A too early evaluation that fails because the staff was still in a training phase naturally is no argument against the programme but an argument against the evaluator. • Did they stick to the instructions or did they modify the programme systematically? • How did the participants react to the programme? • Aspects concerning the quality of target variable assessment. This includes aspects like: • The reliability of the measurements • The validity of the measurements • The objectivity of the measurement process and of the data analysis

7.2 Types of outcome variables In chapter 6.5 five central categories of variables (data) relevant to the field of evaluation have been defined • structural variables • process variables • outcome variables • impact variables • context variables In the following chapters some particularly relevant subcategories of outcome variables will be defined and discussed.

200


7.2.1 Primary efficacy variables vs. secondary variables The dichotomy primary efficacy variables (= ultimate target variables) vs. secondary variables is a central concept to further specify outcome variables within confirmatory research projects (→6.3). The terminology has been developed in clinical trials methodology and it makes much sense to transfer this useful terminology to the area of evaluation as well. Outcome variables are often referred to as "target variables" or more technically speaking as "study endpoints" and therefore the term "variable" in the above expressions may be substituted with "target", "target 25 variable" or "target variables" . In an exploratory study (→6.3) • the focus is to explore a field, to accumulate new insights or to formulate new hypotheses. • To accomplish this task a divergent strategy is necessary and justified. • There is hardly any restriction on the scientific methods used and • no limitation on the number of variables. • All variables under scrutiny are of similar importance; there is no distinction between primary and secondary variables. • All results have preliminary character only. In contrast to this confirmatory studies (→6.3) • aim at proving existing hypotheses to base decisions on the results. • Naturally the strategy has to be convergent oriented. • There are specific methodological standards regulating the scientific methods to be used. • The number of central outcome variables (primary efficacy variables) has to be strictly limited and • all other variables play no role in a confirmatory context and are to be classified "secondary variables". • Results are final in a certain sense.

25

Primary Effectiveness Variables = Primary Targets = Primary Target Variables = Primary Endpoints Secondary Variables = Secondary Targets = Secondary Target Variables = Secondary Endpoints

201


There is no objection against planning a heterogeneous study including a confirmatory section and an exploratory section • as long as both substudies are clearly separated from each other before the study begins (explicitly defined in the written study protocol →7.1) and • as long as the confirmatory part doesn’t depend on results from the exploratory part. The ideal situation in any confirmatory research project is to have one primary efficacy variable only. This primary efficacy variable can be • a specific variable or • an index generalising over a more or less heterogeneous set of variables. Whenever it seems impossible to reach this ideal of one primary efficacy 26 variable the nominal significance level has to be adjusted to the number of primary efficacy variables used. Usually their total number should not exceed two or three, so that enough sufficient statistical power remains to detect existing effects. If it is not possible to reduce the number of primary efficacy variables through selection or aggregation without any further empirical research, it is too early for a confirmatory study. In this case further exploratory research should be considered to gain necessary insights for adequate variable selection or a reasonable aggregation strategy - and then to plan a confirmatory study based on these findings. For methodological reasons it is not acceptable to utilise exploratory techniques to aggregate data into indices and to base a confirmatory analysis on these indices within just one study. It is perfectly acceptable to have a large number of secondary variables in addition to a limited number of primary efficacy variables, as long as they are explicitly labelled to be secondary variables; in other words as long as no decisions for or against specific prevention concepts are based on them. Since secondary variables serve only a descriptive or exploratory purpose, they do not have to be included in the process of adjusting the nominal significance level!

26

The most common method to accomplish this task is the so called "Bonferroni Method" to adjust the nominal significance level by dividing through the number of independent significance tests; in our case by the number of primary effectiveness variables.

202


7.2.2 Efficacy variables vs. surrogate variables (vs. intermediate: variables) Another important way to classify outcome variables is the dichotomy efficacy variables vs. surrogate variables - a terminology developed in clinical trials methodology too. Here again it is possible to substitute the 27 term "variable" for "target", "target variable" or "endpoint" . The dichotomy efficacy variables vs. surrogate variables is based on the concept of indirect measurement via a causally linked dimension - a very common procedure in science. Surrogate variables are used to assess phenomena in cases where direct assessment of efficacy variables is hard to do or impossible. Popular synonyms for surrogate variables are indicator variables or proxy measures. Whenever the causal relationship between an intervention and the ultimate target variable (= primary efficacy variable →7.2.1) is not direct but happens via in between dimensions (intermediate: variables), the idea to use these intermediate: variables as surrogate variables is near at hand. To give a simple and plausible example: If an information campaign on HIV-Transmission via needle sharing is effective in decreasing risky behaviour in intravenous drug users and if this behavioural change causes less manifest AIDS in the long run, the intervention (information campaign), the intermediate: variable (risky behaviour in intravenous drug users and the efficacy variable (AIDS manifestation in the long run) are causally related. In case we do not have enough time to observe how AIDS prevalence develops in the target group over a long period of time, it may be sufficient to use the intermediate: variable "risky behaviour in intravenous drug users" as a surrogate variable for the primary efficacy variable. The use of surrogate variables to assess programme effectiveness indirectly is perfectly justified given that causal relationships between intermediate: variables and efficacy variables are well established through experimental or quasi-experimental empirical research - as in the above example. The same procedure is highly questionable though if the causality assumption is based on correlation only. Causality implies correlation but unfortunately this is not true the other way round (for more details see e.g. Collins, 1994).

27

True Effectiveness Variables = True Targets = True Target Variables = True Endpoints Surrogate Variables = Surrogate Targets = Surrogate Target Variables = Surrogate Endpoints

203


We have to be very aware of the fact that even highly relevant intermediate: variables are not necessarily sensible starting points for interventions or surrogate variables to indirectly assess efficacy. In other words: Manipulating intermediate: variables does not necessarily result in corresponding changes in the ultimate target dimension and observed changes in intermediate: variables often do not correspond with changes in primary efficacy variables. To give an illustrative example: If persons infected with a certain virus react with fever (intermediate: variable) and later develop pneumonia (ultimate target variable), we naturally may not conclude that influencing the intermediate: variable will have a relevant influence on the ultimate target variable. Reducing the temperature with antipyretic drugs will not hinder pneumonia from developing in infected persons, just as raising the body temperature - e.g. in a sauna or in a hot bath - will not cause pneumonia in healthy persons. Similarly a reduction of fever (it could be caused by antipyretic drugs) is not a reliable indicator that the disease is vanishing and fever (it could be an indicator for different types of infections) is not a reliable indicator that a person will develop pneumonia.

7.2.3 Short-term, medium-term vs. long-term effects • Programme outcomes can be divided into • short-term effects, • medium-term effects and • long-term effects. If we design programmes to prevent problems, we are primarily interested in lasting effects (long-term effects). Short-term effects and medium-term effects may play an important role as secondary variables (→7.2.2) - to explain the mechanisms leading from specific actions to the desired effects -, but the existence of these effects is not sufficient to call programmes successful. Short-term effects and medium-term effects may also serve as surrogate variables (→7.2.2) to indirectly assess the ultimate problem dimension, but they are definitely no substitution for essential long-term effects. Prevention programmes that produce impressive short-term and medium-term effects only and have no lasting impact on the ultimate problem dimension, are not really worth-while.

204


7.2.4 Outcome variables suited as "primary efficacy variables" in efficacy studies In evaluating the effectiveness of primary prevention in the field of illicit drugs we are frequently confronted with circumstances, where the variables representing the ultimate programme goals (→7.2.1) are not feasible because of economic or epistemological limitations (→7). If this is the case we should openly admit that we are not in the position to deal with programme effectiveness directly (global proof of effectiveness →6.6.7.1) and either consider a partial proof of effectiveness (→6.6.7.2) or a historic deduction of effectiveness (→6.6.7.3). If these approaches are not possible either, we should admit that further research is needed before we can start to consider assessing programme effectiveness adequately. Primary prevention in the field of illicit drugs is concerned with a reduction of drug related problems. If we want to evaluate the effectiveness of such programmes, the primary efficacy variables (→7.2.2) have to be related to • problematic use (less frequent use, less heavy use, less daily use, less safer use, etc.) as well as to • problem areas related to substance use (health, social integration, integration into the work market, delinquency, etc.). Other outcome variables may serve • as secondary variables (→7.2.2) to explain the mechanisms leading from specific actions to the desired effects or • as surrogate variables (→7.2.2) to assess the primary efficacy variables indirectly, but they must not to be classified as primary efficacy variables in this context. To make this very clear, there are many problem areas where public prevention efforts are indicated ("aggressive behaviour", "bad scholastic achievement", "political radicalism", "lack of social competence", etc.) and these dimensions may be primary target dimensions in programmes focusing on these very problems, but these variables are no primary target variables in substance (ab)use prevention programmes. The other way round, if a programme influences social competence successfully but has no impact on the level of substance related problems, the programme is to be called "programme to increase social competence" but not "substance (ab)use prevention programme". 205


7.2.4.1 The role of knowledge, attitudes, personality characteristics, life skills, etc. as outcome variables Commonly variables like knowledge, attitudes, personality characteristics, life skills, etc. play a prominent role in the evaluation of prevention outcome and many researchers treat these variables as if they were "primary target variables" to assess effectiveness in substance (ab)use prevention programmes - but this is not acceptable. These variables are definitely no primary target variables in this context. Empirical research suggests strongly that these variables do not even qualify as surrogate variables. It has commonly been observed that any changes in these variables do not result in relevant behaviour change or in a significant problem reduction - and if this holds true, they are disqualified as surrogate variables.

7.2.4.2 The role of "any substance use" as outcome variable: A paradoxical relationship to "problematic substance use" Another very popular outcome variable in the evaluation of substance (ab)use prevention programmes is "any substance use" (including experimental use and recreational use) instead of "problematic use" or "problem areas related to substance use". This dimension is commonly referred to as "life-time prevalence of substance use (LTP)". Before we can answer the question, whether this strategy is justified or not, we have to turn to a phenomenon that has been observed and documented in scientific literature for a long time. The u-shaped relationship between a variety of personal problems (severe physical, neurological, psychological and social problems) and substance use. To give some examples: Shaper et al.(1988) showed that alcohol abstainers and heavy users are characterised by a significantly reduced life-expectancy. Hurst (1973) showed that alcohol abstainers and heavy users have a dramatically increased risk to get involved in traffic accidents. Uhl and Springer (1996) showed that alcohol abstainers and heavy users have a dramatically increased level of psychological, social, psychiatric and neurological impairment. Shedler & Block (1990) demonstrated a similar relationship in concerning social maladjustment and cannabis use. And there are many more similar findings to be cited.

206


This u-shaped relationship can be interpreted in a sense that persons with severe personal problems tend to either use substances in a problematic manner (problematic substance use) or to avoid them (total abstainers). In a way this is a paradoxical situation: The same risk factors (→7.2.5) responsible for an elevated probability to end up with problematic substance use in some persons are responsible for total abstaining in others. This phenomenon makes much sense though. It is very plausible that some vulnerable persons who perceived in time that substance use could constitute a major risk for them decided to stay away from substances totally - as a kind of: self protection mechanism - while those who do not stay away from substances end up with severe problems. The above model to explain the u-shaped relationship between personal problems and substance use has three central implications. • Based on the u-shaped relationship, any significant reduction of personal problems (reduction of risk factors) can be expected to result in less abstaining and in less heavy use at the same time. More technically speaking, a significant reduction of risk factors caused by successful prevention programmes - should result in more life-time prevalence and in less problematic use simultaneously. Consequently life-time prevalence of substance use cannot be seen as a sensible variable to assess programme effectiveness in substance abuse prevention. • Total abstaining from substances in young adults is an indicator that the affected persons have severe personal problems and that they may develop a variety of consecutive problems in the future. Obviously manipulating this risk factor (e.g. by suggesting to drink some alcohol once in a while, to try cannabis occasionally, etc.) would not solve any of the problems or prevent any of the future problems. On the contrary, knowing that these vulnerable persons stayed away from substances, because they knew that they probably could not handle the substances properly, this approach would very likely add an additional problem - a substance abuse problem - to the already existing amount of problems. • If it is possible to influence a variable highly correlated with the ultimate target dimension without causing concordant changes in this target dimension, this variable is not suited to be used as surrogate variable to assess the ultimate target dimension.

207


7.2.5 Protective factors and risk factors - the problem of causality once more It is common practice in prevention research to look for variables that correlate with favourable and unfavourable outcomes and to call them "protective factors" and "risk factors". Some of these factors are context variables (→6.1.3) that cannot be changed (e.g. gender, ethnic background, etc.) and others are intermediate: variables (→7.2.2) like attitudes, level of education, skills etc. that can be influenced to a varying degree through appropriate interventions. A great problem with this terminology is that the expressions "protective factor" and "risk factor" imply causality - an implication that is not justified if the classification is based on mere correlation, as usual. Commonly protective factors and risk factors are seen in three functions: • as indicators to identify high and low-risk-groups, e.g. to identify target groups for specific secondary preventive measures, • as starting points for preventive intervention strategies and • as surrogate variables (→7.2.2) to indirectly assess primary efficacy variables (→7.2.1) that are impossible or hard to assess directly. The first one of these three functions (indicator) is relatively unproblematic, but the latter two functions (starting points for interventions, surrogate variables) are highly problematic if the definition as protective factors and risk factors is based on mere correlation only (compare chapter 7.2.4.2).

7.3 The problem of heterogeneity A central implicit assumption behind many statistical procedures is homogeneity of effects. In other words the assumption that all subjects react more or less homogeneously on the interventions under scrutiny. Whenever the homogeneity assumption is grossly violated, statistical procedures based on this assumption may yield highly misleading results. To give an illustrative example: Let us assume we could split students into a subgroup with a relevant risk to acquire drug problems (high-risk-group) and into a subgroup without any risk to develop drug problems (no-risk-group). Lets furthermore assume that inducing certain changes of attitude in persons at risk (intermediate: variable →7.2.2) could reduce the chance that these persons develop a substance abuse problem. Lets further assume that our intervention is received well by the no-risk-group - causing favourable changes in the intermediate: variable - and at the same time 208


rejected by the high-risk-group - causing adverse changes in the intermediate: variable. If we analysed the effects for both subgroups independently, we would instantly realise that the programme was a complete fail28 ure, since positive changes in the no-risk-group are irrelevant and unfa29 vourable changes in the high-risk-group result in a boomerang effect . The no-risk-group and the high-risk-group have no simple tags and therefore the statistical analysis will have to treat them as one homogenous group. The no-risk-group being larger, we can expect an average attitude change into the desired direction. We can furthermore assume that the authors will conclude that the intervention was a success in terms of a scientific proof of effectiveness. At the same time we know that the programme was highly counterproductive. A very illustrative example how uncontrolled heterogeneity can lead to erroneous conclusions in research is the famous Grand Rapid Study (Borkenstein et al., 1964, 1974). This study was a milestone in traffic research relating the effects of alcohol consumption to traffic safety, serving as basis for defining legal Blood Alcohol Concentration (BAC) thresholds for drivers in the USA and in many European Countries. One result of this large and methodologically excellently designed study was that drivers with 0.03% BAC had the lowest risk to cause traffic accidents. The unexpected u-shaped relationship between BAC and impairment was referred to as Grand-Rapid-Dip and commonly explained in a manner that minimal levels of alcohol activate and therefore are capable of reducing the likelihood of road accidents. Hurst (1973) reanalysed the data and showed that the published effect was purely artificial. He divided the sample by the average drinking frequency into sub-samples and the analysis showed that increasing BAC levels were always associated with higher impairment in any subgroup. The cause producing the Grand-Rapid-Dip was that abstainers - who naturally always have 0.00% BAC - have a fourfold greater risk to get involved in accidents than sober (0.00% BAC) non-abstainers, who drink once a week or more. These non-abstainers reach a similar degree of impairment with a BAC level around 0.08%. Confounding these sub-samples to one heterogeneous total sample in the statistical analysis led to the erroneous conclusion that small amounts of alcohol could enhance traffic safety.

28

The likelihood to develop drug problems cannot be reduced any further in the no-risk-group

29

The term "boomerang effect" refers to effects that are contrary to the declared intervention goals.

209


7.4 The influence of simultaneous interventions An ideal condition for any empirical research is if causal relationships are rather stable over time. Unfortunately this ideal condition is not the case in the field of substance (ab)use prevention. Almost all media touch the subject substance (ab)use in regular intervals, students are continuously confronted with the topic drugs in many different courses and drugs are a relevant topic among youngsters as well as between parents and their children. This multitude of influences causes very rapid culture and subculture specific changes and fashions in the kinds of drugs used and in the way the drugs are used. Whatever preventive intervention we plan to prevent substance (ab)use, we have to realise that we compete with an abundance of other concurring influences. Because of this situation, the only proper way in efficacy studies to separate programme induced effects from effects caused through other concurrent influences is to use experimental designs with at least one control condition and at least two points of measurements. Whenever true random assignment is not possible a quasi-experimental design may be considered. If a quasi-experimental design is chosen, adequate strategies of data collection and data analysis should be taken to guarantee that possible biases are reduced to a minimum.

7.5 Dependency on context Another ideal condition for empirical research is if causal relationships exist independently of situational factors (context). This ideal condition is not at all the case in the field of substance (ab)use. The kinds of substances used, the circumstances of use and the susceptibility to certain preventive approaches vary greatly from culture to culture, from subculture to subculture and from cohort to cohort - and all these factors furthermore are subjected to rapidly changing fashions and trends. This has two central implications: • One implication is that prevention strategies developed in specific environments should not be transferred to other environments without evaluating the strategy again. Strategies that proved to be very successful in certain environments may turn out unworkable in another context. • The other implication is that we have to keep a close eye on systematic changes over time (Monitoring of context, →6.6.10). Concepts that worked out well two years ago may be absolutely obsolete nowadays. 210


7.6 Generativity "Generativity" is a possible term for the well known fact that small interventions may generate unpredictable effects that evoke further unpredictable effects and so on, until finally an abundance of unpredictable and non-reproducible consequences have arisen. According to Chaos Theory (e.g. Steward, 1989) this kind of: unpredictable cause-effect relationship is a common phenomenon in nature. One of the central contentions of Chaos theory is that the wing stroke of a butterfly in South America may cause a typhoon in Indonesia ("butterfly effect"). Those who consider the butterfly effect in the original version to be rather implausible, will probably have no problem to accept that the same butterfly can cause an even greater catastrophe if he diverts the attention of a nuclear power plant technician in a crucial moment. The phenomenon of generativity is responsible for some severe problems in evaluating prevention efforts. Generativity in the above defined sense constitutes an extremely skewedly distributed nuisance component of variance in evaluation designs. In some situations the magnitude of generativity may outweigh the systematic prevention effects by far. To give a specific example: It is possible that a short presentation by a drug-expert in a school - usually having limited positive impact only - is misinterpreted by the son of an important politician, stimulating his indignant father to react massively on different levels. That way a small cause may generate a tremendous effects on the direct political level, via mass media, etc. The generated effects may be positive in terms of the primary goals but they could be negative as well (boomerang effect). Generativity should not be mixed up with initially unanticipated but predictable systematic effects that may be considered as expected effects in future evaluation (e.g. purposely stimulating public discussion in a certain way, changing public opinion, generating structural changes, etc.). The assessment of originally not explicitly expected but never-the-less basically predictable and systematic effects is called impact evaluation (→6.6.11).

211


7.7 Effect-size and incidence of primary efficacy variable Variation caused by uncontrolled simultaneous interventions (→7.4), by contextual variation (→7.5), by generativity (→7.6), etc. constitutes nuisance variance in research designs. We only have a realistic chance to prove programme effects in spite of large uncontrolled variance if the effects are massive and/or if the sample sizes are large. Within these considerations we are not interested in the relative effects but in the absolute effects - and this moves the incidence of our problem into the focus of consideration. If a certain problem, e.g. the manifestation of problematic use of illicit drugs in a given population can be estimated to be 0.1% per year and if our preventive intervention reduces problem manifestation by 20% (relative effect) we can expect that the incidence: of the problem will be reduced from 0.1% to 0.08% within this year. This is equivalent to an absolute effect of 0.02%. In other words: The intervention will only prevent 1 out of 5000 persons per year from developing this very problem. These considerations illustrate the fact quite well that particularly in areas with low problem incidence, only very massive effects can be established by statistical means. Naturally small programmes with limited effects may be worthwhile as well, but we have no scientific means to prove their effectiveness.

7.8 Power considerations - sample size 7.8.1 The basic principle of power calculation The question whether a scientific project has a relevant chance (power) to prove existing effects, can be estimated by basic statistical methods. The power of a research design depends on the amount of uncontrolled variance, on the effect-size and on the sample size. If the uncontrolled variance is small and if the effect-size is large, even very small sample sizes are sufficient. If on the other hand uncontrolled variance is large and/or the effect-size small, the necessary sample size may reach dimensions that render the research project impossible on economic and/or epistemological grounds. In sample size calculations the a-error is usually laid down at 1% or 5% and the b-error is laid down at 10% or 20%. In other words we decide to accept a 1% or 5% risk to erroneously conclude that an ineffective 212


programme is effective and a 10% or 20% risk to erroneously conclude that an effective programme is ineffective.

7.8.2 A practical example for power considerations in prevention research In chapter 7.7 we based a fictive example on the assumption that • the problem of interest is "severe problems caused through the consumption of illicit drugs", • that a prevention programme is capable of reducing the rate of problem manifestation by 20% and • that the annual incidence of the problem is 0.1% in the age group between the 14th and the 23rd year of age. Now we will base a sample size calculation on these assumptions to demonstrate the magnitude of possible sample sizes. To start out, we can conclude that the above assumptions seem to be realistic: • "Severe problems caused by the consumption of illicit drugs" is a useful outcome variable (→7.2.4) • A prevention programme resulting in a 20% reduction of problem manifestation can be considered to be an extremely successful programme. • The assumption of a 0.1% problem incidence per year is appropriate for Austria if we consider • that problems with illicit drugs usually start in the age range between the 14th and the 23rd year of age, • that the incidence is roughly constant over this period of age and • that this adds up to a 1% prevalence of severe drug problems in the age cohort of the 24 year old Austrians (Uhl, 1992). In order to be able to calculate the sample sizes, we have to decide on some further conventions and assumptions: • If we accept an • a-error of 5% and • a b-error of 20%, • if we assume that there are no measurement and/or coding errors, • if we assume that there is no interaction within the school classes (allowing us to treat data concerning all students as independent observations),

213


• if we manage to execute a large scale randomised experiment • with experimental schools exposed to the programme and • with control schools not exposed to the programme and • if we consider a follow-up period of 12 month, it turns out that we need 181 000 students in the experimental group and 181 000 students in the control group, adding up to a total sample size of 30 362 000 students . For persons who do not trust abstract sample size calculations it is possible to illustrate the situation in a less abstract and formal manner: In a control group of 181000 students we would expect - according to an incidence of 0.1% annually - that 181 students who had no severe drug problem at baseline level will have a problem at the end of the follow-up period. In the experimental group we would expect 80% of this rate, which is equivalent to 145 students. The expected difference is 36 persons out of a total sample of 362 000 students. Realistically speaking we have to be aware that the calculated total sample size of 362 000 students is not even really sufficient, since two of the above assumptions were highly unrealistic. • It is practically impossible to assess severe drug problems in large samples without any measurement and/or coding errors and • we have to expect relevant interaction within the school classes (observational unit not individuals but groups of individuals →7.10). If we e.g. expect a measurement and/or coding errors of 0.2% only - a rate 31 that is still very low - the sample size required doubles to 752 914 subjects. If we expect relevant interaction within the school classes, or if we are interested in subgroup analyses (e.g. which subgroup responds better to the programme), even further increases of sample size are necessary.

30

This sample size calculation is based on Fleiss (1981). The specific calculations can easily be done with the shareware computer programme "PC-SIZE" (Dallal G. E., 1985) or with a more recent shareware programme "POWER" (Dupont, W. D. & Plummer W. D, 1990). Both programmes are distributed by the authors free of charge.

31

Commercial institutes usually estimate errors due to coding and data entry near 1% and this does not even include deliberately and/or erroneously wrong answers by the subjects under investigation.

214


7.8.3 Practical implications in the evaluation of substance abuse prevention programmes regarding the sample size In order to reach realistic sample sizes • the expected changes in the control group (uncontrolled nuisance variation) should be small, • the effects of the programme should be large and • the measurement and/or coding errors should be as small as possible There are some several strategies to increase the power of research designs in order to allow smaller sample sizes. • One way is to use programmes or a package of programmes with a high impact • Another method is to increase the follow-up period. The common argument against this option that prevention programmes usually do not have any long lasting effects is not acceptable. If programmes do not have any relevant impact on the ultimate target behaviour in the long run they are useless. Another argument against longer follow-up periods is very relevant though. We should know if our prevention programmes work out well before they are outdated because of contextual changes. • Still an other possibility is to concentrate specifically on high-risk-groups (secondary prevention), but this doesn’t help us very much in relationship to primary preventive efforts. • A very common but never-the-less highly problematic approach is to concentrate on easily assessable intermediate: variables where high effects can be expected (compare chapter 7.2.4 and 7.2.5).

7.8.4 Problems with underpowered design: the publication bias Depending on whether designs are adequately powered or underpowered the outcome in terms of significant results vs. insignificant results have different implications. • In adequately powered large designs significant outcomes as well as insignificant outcomes are interpretable and therefore circulated in the scientific community as well as published in scientific journals. • If underpowered designs reach significant results, the results are interpretable as well and most scientists do not consider the projects to 215


be failures. They usually circulate the results and scientific journals do not reject these papers. • On the other hand, if underpowered designs do not yield significant results though, the results are not interpretable and most scientists consider the projects to be failures. They usually do not circulate these results. In case they consider a publication anyway most serious scientific journals reject these papers. • If papers are not considered as individual papers, but if the published results regarding a certain topic are seen on a global basis, these mechanisms lead to a dramatic publication bias in underpowered designs, caused by random significances and the systematic absence of contradicting evidence. • This situation is very unsatisfactory, since any prevention approaches regardless if they are effective or not - eventually may look promising based on the published literature, if many small underpowered studies are executed.

7.8.5 Approaches to solve the problem of underpowered designs There are two possible ways to handle this problem: • A central protocol registration approach

32

or a

• systematic rejection of underpowered designs approach

33

The idea of a protocol registration approach is to guarantee that scientific results are systematically made available to the scientific community regardless of outcome through a rigid registration mechanism. According to this strategy all protocols of confirmatory studies would have to be handed out to a central registration body before they start and at least a short version of the results should be added after the projects are finished or prematurely terminated. All protocols and results collected in this registry should be made available to the scientific community. Using this

32

One field, where a "protocol registration approach" has been systematically implemented is the procedure to register new pharmacological drugs in the USA. The central body where the protocols have to be handed in before any confirmatory research starts is the American Food and Drug Administration (FDA).

33

The FDA is much stricter than the here proposed "either or" -standard. The FDA insists on both approaches to avoid scientific opportunism simultaneously. This agency insists on protocols handed in before any confirmatory projects start and turns down any protocols that do not meet minimum scientific standards including adequate power.

216


source of information anybody could get an unbiased view on the various outcomes of confirmatory projects. After such a strategy is implemented, results from scientists who do not hand out their protocols ahead should definitely not be accepted by the scientific community or by any scientific journals. Until a protocol registration approach is implemented, the only sensible alternative is rejection of underpowered designs. According to the latter approach any results from underpowered confirmatory designs, regardless if they yield significant or insignificant results, should not be quoted informally by scientists or accepted by journals for publication.

7.9 Measurement problems with self-reported consumption The primary assessment strategy for alcohol and other drug use behaviour is the individual self-report (Forman & Linney; 1991) Prevention programmes rely commonly on self-reported consumption only - and it is a well known fact that self-reported consumption is usually not a very reliable source of information. The rate of admitting illegal behaviour like "using illicit drugs" is highly dependent on the context. It is much easier to influence verbal behaviour with small interventions or through manipulation of situational variables (e.g. to convince persons admitted having taken drugs at t1 that it is safer not to admit these experiences at t2) than to influence the actual drug consumption behaviour. Impressing results to illustrate the lack of validity and reliability in self reported drug consumption can e.g. be found in Uhl & Springer (1996), who were confronted with the fact that almost two thirds of several age cohorts who had admitted the use of illicit drugs 10 years before decided not to admit any previous consumption 10 years later. Another quite extreme result in this context was found by an Austrian commercial survey institute (IFES, 1995a, 1995b). The institute conducted two representative surveys on illicit drug consumption in 1995, one covering the whole Viennese adult population and the other covering 14 through 26 year old Viennese only. Comparing the self-reported life-time prevalence rates of illicit drug consumption in the age group between 14 through 26 years of age, it turned out that the rate from one survey was twice as high as the rate based on the other survey.

217


7.10 Dependency of observational units It is a common situation in prevention that the observational units are sampled in clusters or groups and not as simple random samples. Commonly the sampling clusters are schools or classrooms, since many interventions focus on groups and not on individuals. If we have three school classes with 30 students each, we are not allowed to treat them as 90 independent observations, but we have to treat them as three independent clusters of dependent observations. Dielman (1994) dealt in detail with this problem and suggested techniques to handle the problem in a statistically correct manner. In practical research the above problem is usually solved by simply ignoring it. It is common practice to use statistical standard tests even in cases, where the implicit assumption of independent observations is obviously grossly violated. That way a-errors and b-errors are systematically underestimated, resulting in an increased rate of significances by mere chance and in an underestimation of the required sample size(→7.8).

8 References Bertolote, J. M.(ed.): Quality assurance in Mental Health Care Check-Lists & Glossaries. Division of Mental Health, WHO, Geneva, 1994 Borkenstein, R. F.; Crowther, R. F.; Shumate, R. P.; Ziel, W. P.; Zylman, R.: The Role of the Drinking Driver in Traffic Accidents. Indiana Deptartment of Police Administration, Indiana University, Bloomington 1964 Borkenstein, R. F.; Crowther, R. F.; Shumate, R. P.; Ziel, W. P.; Zylman, R.: The Role of the Drinking Driver in Traffic Accidents. Blutalkohol, 11, 1-131, 1974 Campbell, D. & Stanley, J.: Experimental and Quasi-experimental designs for Research. Chicago, Rand-McNally, 1963 Clayton, R. R., Cattarello, A.: Prevention Intervention Research: Challenges and Opportunities. In: Leukefeld, C. G., Bukovsky, W. J. (ed.): Drug Abuse Prevention Intervention Research: Methodological Issues. NIDA Research Monograph 107, Rockville, 1991

218


Collins, L. M.: Some Design, Measurement and Analysis Pitfalls in Drug Abuse Prevention research and How To Avoid Them: Let Your Model Be Your Guide. In: Cázares, A. & Beatty, L. A. (ed.): Scientific Methods for Prevention Intervention Research. NIDA Research Monograph 139, Rockville, 1994 Cronbach, L. J.; Gleser, G. C.; Nanda, H.; Rajaratnam, N.: The Dependability of Behavioural Measurements: The Theory of Generalizability for Scores and Profiles. John Wiley & Sons, USA, 1972 Dallal G. E.: PC-SIZE - A Shareware Computer Program. USDA Human Nutrition Research Centre on Ageing at Tufts University, Boston, 1985 Dielman, T. E.: Correction for the Design Effect in School-Based Substance use and Abuse Prevention research: Sample size Requirements and Analysis Considerations. In: Cázares, A. & Beatty, L. A. (ed.): Scientific Methods for Prevention Intervention Research. NIDA Research Monograph 139, Rockville, 1994 Donabedian, A.: Explorations in Quality assessment and Monitoring. Vol.1. The Definition of Quality and Approaches to its Assessment. Health Administration Press. Ann Arbor, 1980 Dupont, W. D. & Plummer W. D.: POWER - Program Description and Shareware Computer Program. Controlled Clinical Trials, 11, 116-128, 1990 Fitz-Gibbon, C. T., Morris, L. L.: How to Design a Program evaluation. SAGE Publications, 1987 Fleiss, J. L.: Statistical Methods for Rates and Proportions. 2nd ed., John Wiley & Sons, New York, 1981 Flay, B. R.: What We Know About the Social Influences Approach to Smoking Prevention : Review and Recommendations. In: Bell, C. S. & Battjes, R. (ed.): Prevention research: Deterring Drug Abuse Among Children and Adolescents. NIDA Research Monograph 63, Rockville, 1985 Forman, S. G. & Linney, J. A.: Increasing the Validity of Self-Report Data in Effectiveness Trials. In: Leukefeld, C. G., Bukovsky, W. J. (Ed.): Drug Abuse Prevention Intervention Research: Methodological Issues. NIDA Research Monograph 107, Rockville, 1991

219


German Society for Addiction Research and Addiction Therapy (ed.), Documentation Standards for the Treatment of Substance Addicts, 2nd edition, Germany 1992 Hurst, P. M.: Epidemiological Aspects of Alcohol in Driver Crashes and Citations. Journal of Safety Research, 5, 3, 130-147, 1973 IFES:

Suchtmittelstudie 1995 - Bevölkerungsbefragung Wien, Tabellenband. Institut für empirische Sozialforschung, Wien, 1995a

IFES: Suchtmittelstudie 1995 - Wiener Jugendliche, Tabellenband. Institut für empirische Sozialforschung, Wien, 1995b Jolicoeur. L. M.; Jones-Grizzle, A. J.; Boyer, J. G.: Guidelines for Performing a Pharmacoeconomic Analysis. Am-J-Hosp-Pharm. 49 / 7, 1741-1747, 1992 Korf, D. J.: Dutch Treat - Formal Control and Illicit drug Use in the Netherlands. Thesis Publishers, Amsterdam, 1995 Künzel-Böhmer, J., Bühringer, G., Janik-Konecny, T.: Expert Report on Primary prevention of Substance Abuse. IFT Research Report Series, Vol. 60e, 1994 Lindstone, H. A. and Turroff, M.: The Delphi method: Techniques and Applications. Addison-Wesley, Massachusetts, 1975 Martinez, B.: Understanding Drug Prevention - An Office of National Drug Control Policy White Paper, Washington, 1992 Popper, K. R.: The Logic Of Scientific Discovery, 8th rev. edition. Hutchinson, London, 1975 Scriven, M.: Evaluation Thesaurus, 4th Edition. Sage, Newbury Park, 1991 Scriven, M.: The Methodology of Evaluation. In. Tyler, R. W., Gagne, R. M., Scriven, M. (ed.): Perspectives of Curriculum Evaluation. Rand-McNally, Chicago, 1967 Sloboda, Z.; David, S. L.: Preventing Drug Use Among Children and Adolescents. A Research-Based Guide. NIDA, Rockville, 1997 Shaper, A. G.; Wannamethee, G.; Walker, M.: Alcohol and Mortality in British Men: Explaining the U-shaped Curve. Lancet,1267-1273, 1988 Shedler, J.; Block, J.: Adolescent drug use and psychological health: A longitudinal inquiry: American Psychologist, 42, 612-630, 1990 Steward, I.: Does God Play Dice? The Mathematics of Chaos. Penguin, London, 1989 220


Stufflebeam, D. L.: The Personal Evaluation Standards. How to Assess Systems for Evaluating Educators. Corwin Press, Newbury Park, sixth printing, 1995 Tukey, J. W.: Exploratory Data Analysis. Addison-Wesley, Reading, 1977 Uhl, A.: Ein Modell zur Schätzung der Drogenerfahrungsprävalenz nach Altersgruppen, Geschlecht, Bildung und Region auf der Basis der Konsumrepräsentativerhebung 1984. Wien.Z.Suchtforsch., 15, 4, 11-17, 1992 Uhl, A.; Springer, A.: Studie über den Konsum von Alkohol und psychoaktiven Stoffen in Österreich unter Berücksichtigung problematischer Gebrauchsmuster - Repräsentativerhebung 1993 / 94. Originalarbeiten, Studien, Forschungsberichte des Bundesministeriums für Gesundheit und Konsumentenschutz, Wien, 1996 WHO, Division of Mental Health: Life Skills Education in Schools, Geneva, 1994 Yates, B. T.: Toward the Incorporation of Costs, Cost-effectiveness Analysis, and Cost-benefit Analysis Into Clinical Research. J-Consult-Clin-Psychol., 62,4,729-736, 1994

contact address: Dr. Alfred Uhl LBISucht Vienna, Austria Tel: +43-1-8882533-158 FAX: +43-1-8882533-138 e-mail: [email protected]

221


Index ___________________________________ assistance to self-help 143 attainment of autonomy 168 attitude change 24, 38, 51, 57, 85, 96, 98, 118, 208 attitudes towards drugs 13, 14 Austria 26, 27, 158, 166, 212, 216 autonomy and ability to choose 160 availability 6, 117, 127, 153, 163, 164

-Aabolishing any substance use 159 absolute effect 211 abstinence orientation 150, 159 total 81, 205, 206, 208 vs. responsible use controversy 159 abuse definition 140 academic guidance 115 accountability 11, 43 addicted use 140, 152, 153 adverse consequences 166 drug effect 160 effects 80 reaction 172 side effect 189 all substances 156 alternative activities 97, 160, 168 activities to substance use 160 alternatives approach 97, 98, 103 to aggression 115 to drug taking 23, 50 to drugs 38 ambiguity 139, 140, 151, 166, 192 any substance use 6, 152, 159, 160, 184, 205 ARF 4, 63, 135 assertiveness 160, 169 training 95, 156, 168 assessment of not explicitly expected effects 175

-Bbaseline measurements 14 basic research 171, 172 behaviour change 47, 93, 94, 118, 187, 205 behavioural concept 162 Belgium 3, 26, 28 boomerang effect 208, 210 butterfly effect 210 -Ccausality 202, 207 assumption 202 problem of causality 207 CBA 183, 194, 195, 196 CDA 144, 178 CEA 194, 195, 196 central strategy 150 CEUDROG 4, 135 changing attitudes 23, 50, 160 lifestyles 23, 50 substance use 160 cigarettes 98 class of substances 156 classification

222

Index

a contents oriented classification approach 183 based on consumption oriented goals vs. problem oriented goals 157 based on direct vs. indirect approaches 163 based on legal status of substances involved 156 based on number of substances involved 156 based on persons in charge of evaluation 179 based on problem levels 151 based on programme duration 170 based on specific characteristics of the target population 147 based on specific goals 160 based on structural vs. communicative approaches 163 based on substance specificity 155 based on the central strategy 154 based on the epistemological significance of the findings 178 based on the kind of data used (data dimension) 175 based on the kind of theory involved 161 based on the role theory plays 161 based on the size of the target population 147 based on the state of the programme to be evaluated (state-of-programme dimension) 177

based on the type of social environment 148 based on the way how target persons are reached (directly or indirectly) 149 based on types of goals 150 based on types of target persons 147, 163 by types of actions / strategies 150, 163 four-dimensional classification systems 182 of evaluation 175 of prevention programmes 147 of programmes based on abstinence orientation vs. responsible use orientation 159 within communicative approaches 165 within structural approaches 164 Classification based on theoretical aspects 161 classroom intervention 121 clubbing 23, 50 Cndt 4, 135 coding error 212, 213, 214 cognitive behavioural oriented approach 167 behaviourism 162 commitment not to smoke 102 communication / persuasion model 162 communication theory 162 communicative approach 154, 163, 165 community action 37 approach 118 223


intervention 118, 120 interventions 118 programme 93, 118, 119, 129, 133 Community Action Programme 120, 134 compulsive use 6 compulsory treatment 23, 50 concept phase 144, 171, 172, 185, 187 Concept phase 178 confirmation 173 confirmation of hypotheses and theories 179 confirmatory analysis 201 context 200 data analysis 144, 178, 182 evaluation 178, 179, 181, 182 oriented study 155 outcome evaluation 190 project 216 research 144, 145, 173, 179, 182, 188, 194, 197, 198, 201 research project 200 section 201 strategies 188, 189 study 179, 188, 190, 197, 198, 200, 201, 215 techniques 182 conservative norms 97 consumption data 184 orientation 159 oriented goal 150, 157, 158, 159 parameter 158 contaminated street drugs 160 contents oriented classification approach 183 context 174, 175, 193, 194, 200, 209, 216

data 176, 182 evaluation 174, 176, 183, 193 variable 176, 193, 199, 207 contextual condition 189, 193 variation 211 control aspect 192 condition 12, 108, 114, 123 group 32, 84, 99, 103, 104, 106, 108, 109, 119, 124, 213, 214 controlled conditions 141 convergent 179, 189, 200 coping with emotions 168 with stress 168 COST-A6 1, 2, 3, 4, 8, 19, 65, 91, 135 cost-benefit analysis 183, 194 cost-effectiveness analysis 141, 183, 194 Council of Europe 2, 7, 8 country reports 1, 3, 4, 8, 19, 21, 26, 40, 46, 49 creative thinking 168 credibility 70, 102, 118, 185 criminal law 41 critical thinking 168 cultural identity 6 Czech Republic 26, 28 -Ddaily use 160, 204 DARE 29, 108, 131 data dimension 182 DATE 117 decision maker 15, 139, 142, 180, 181, 184 making 160, 168 making model 93, 94, 95 224

Index

study 139, 190, 191 decriminalisation 23, 50 definitions - concepts - problems 1, 17, 135 delinquency 158, 160, 204 Delphi method 4, 8, 140, 142, 186, 219 demand reduction 154, 163 dependency of observational units 217 dependent observations 217 description of programme activities 11 descriptive evaluation 144, 178, 179, 182, 193, 201 research 145, 178 desired effect 141, 190, 203, 204 deterministic theories 162 deterrence 23, 165 development model 162 phase 144, 171, 172, 173, 174, 181, 182, 185, 187, 189, 192 Development phase 178 deviant behaviour 113 different strategies 150, 163 target groups 150, 163 direct approach 147, 149, 150, 163 disapproval 111, 112, 113, 128 of drinking 111 of smoking 111 disease concept 5 distrust all previous information 165 divergent 178, 189, 200 documentation 144, 181, 198 Documentation 158 drug

(ab)use 140 dependence 140 laws 23, 50 policy 23, 39, 41, 50, 150 problem 139, 145, 150, 190, 204, 207, 212, 213 specific attitudes 176, 193 -Eearly childhood education 115 intervention 143 eclectic approach 161 economic deprivation 118 interests 7 limitations 145, 146, 173, 179, 189, 190, 204 EDA 144, 178 educational achievement 35 material 27, 37, 58 programme 93 effect size 101, 103, 211 effective communication 168 effectiveness 10, 11, 12, 13, 14, 15, 39, 88, 93, 96, 98, 100, 101, 109, 110, 114, 115, 116, 118, 122, 128, 132, 134, 139, 141, 145, 146, 155, 158, 173, 174, 179, 183, 185, 188, 189, 190, 191, 192, 194, 197, 199, 201, 202, 204, 205, 206, 208, 211, 220 efficacy 141, 155, 160, 172, 173, 188, 189, 190, 200, 201, 202, 203, 204, 207 evaluation 173, 183, 189 oriented evaluation 188 study 188, 190, 191, 204, 209 variable 141, 155, 189, 190, 191, 200, 201, 202, 203, 204, 207 225


efficiency 141, 195 evaluation 174, 183, 194 Eginition Hospital 4, 135 EMCDDA 2, 3, 4, 135 empathy 168 endpoint 202 environment oriented approach 163 epistemological limitations 145, 146, 179, 190, 204 equivocalness 139, 140 erroneous conclusion 208 ethical evaluation 143, 145, 159, 169, 172, 183, 184, 192 Ethical evaluation 185 European Commission 1, 8 prevention doctrine 41, 44, 169 Union 1, 16, 19, 40, 135 evaluation definition 10, 143 dimension 182, 183 faulty 103 methodology 123 of implementation quality 191 of prevention 146 of prevention approaches 10, 15, 40, 161, 215 of theory 7, 10 research 2, 5, 8, 34 results 7, 14, 87, 172, 180, 181 evaluator 2, 3, 13, 14, 15, 35, 74, 146, 181, 184, 188, 190, 192 everyday conditions 141 context 141, 143, 145 interpretation 145, 146 meaning 146 sense 143, 145 understanding 143, 145, 146 existing scientific evidence 172

expected change 214 experience 140, 186, 199 based expertise 186 based historic evaluation 186 experimental condition 12, 105, 106, 174 design 12, 15, 29, 32, 34, 123, 139, 173, 209, 217 use 140, 152, 153, 157, 158, 159, 160, 205 experimenters 152, 153 expertise 185, 186 explicitly expected 174, 175, 210 expected effect 144, 175, 176, 182, 194 exploratory approach 172, 188, 189 context 188 data analysis 144, 178 evaluation 178, 179, 182, 188, 201 orientation 188 oriented 189, 194 research 144, 145, 178, 182, 188, 197, 201 result 179 section 201 study 200 techniques 178, 182, 201 external evaluation 15, 181, 182, 188, 192 expert 180, 181, 188 strategies 192 validity 141, 174, 187 -Fface-to-face interview 24, 51 fact-oriented 102 factual information giving 23, 50, 123, 165, 166, 167 false-consensus effect 96 226

Index

family 148, 164, 185 FDA 215 feasibility evaluation 11, 172, 173, 179, 181, 183, 187, 188, 189, 192, 204 final programme implementation 174 financial limitations 14 Finland 26, 29, 84, 85, 86, 87 first summative phase 144, 177, 178, 181, 182 focus on large scale dealers 23, 50 formative evaluation 14, 71, 144, 172, 173, 177, 182, 183, 187, 188, 189 phase 172, 177, 178, 181, 182, 187 four-dimensional classification concept 182, 183 France 26, 29, 60, 112 free / spare time 148 frequent use 160, 204 functional equivalents to drug use 162 fundamental research principles 197

harm reduction 6, 22, 24, 27, 28, 32, 37, 41, 43, 45, 49, 51, 154 harmful use 6, 140, 152, 153, 157, 158, 160 health 155, 158, 160, 204 behaviour 118 belief model 166 education 23, 27, 28, 31, 32, 33, 38, 45, 50, 58, 59, 94 healthy lifestyles approach 162 heavy use 160, 204 heterogeneity 207, 208 heterogeneous large classes of theories 162 high-exposure programme 170 high-risk-group 151, 157, 158, 207, 214 historic data 185 deduction of effectiveness 173, 189, 191, 197, 204 evaluation 172, 183, 185, 186, 191 homogeneity 207 how target persons are reached 147 -Iideological position 6 IFT 2, 4, 31, 63, 135, 219 illicit drugs 156 immoral conduct 22, 49 impact 204 data 144, 176, 182 evaluation 144, 174, 175, 176, 183, 188, 194, 210 variable 199 implementation 3, 6, 11, 14, 16, 21, 25, 28, 29, 30, 32, 34, 38, 39, 42, 47, 52, 54, 55, 56, 57, 58, 59, 67, 69, 70, 72, 73, 74, 79, 84, 104, 107, 108, 120, 123, 127, 128, 133, 139, 145, 155,

-Ggenerativity 170, 210, 211 Germany 26, 30, 218 global programme effectiveness 173 global proof of effectiveness 189, 190, 197, 204 Global proof of effectiveness 190 goal-setting 114, 169 Goal-setting 169 Greece 26, 31, 53, 59 -HHandbook Drug Prevention 3 227


173, 174, 180, 184, 191, 192, 193, 199, 216 improving self-control 168 self-esteem 168 incidence 211, 212, 213 of primary efficacy variable 211 of problem 211, 212 independent clusters 217 observation 212, 217 indicated programmes 148 indicator variable 202, 203, 206, 207 indirect approach 147, 149, 150, 163 information approach 7, 96, 99, 100, 101, 102, 105, 107, 128, 167 programme 93 initiation 32, 110, 113, 116, 117, 122, 130 injecting 160 inoculation 98 theory 162 Institute of Psychiatry and Neurology, Warsaw 4, 5 integration into the work market 158, 160, 204 intermediate target person 149, 150 target variable 24, 51 variable 187, 190, 191, 202, 203, 207, 214 internal evaluation 15, 180, 181, 182, 187 expert 180 strategies 192 validity 125, 186 international programme 147

interpersonal relationship skills 168 intervening variable 13 invulnerability 121, 122 Ireland 26, 32, 107, 120, 134 ISF 4, 5, 9, 135 Italy 26, 32 -JJellinek Consultancy 2 joy-riding 114 Just Say No 69 approach 47 -KKAB-model 81 Karolinska Hospital 4, 135 kind of data used 176 theory involved 161 knowledge 2, 13, 14, 56, 57, 81, 140, 160, 167, 176, 193, 205 about drugs 102 knowledge/attitudes model 93, 94 -Llarge scope theories 162 later onset 6, 22, 24, 37, 49, 51, 110, 160 law enforcement 41, 42 LBISucht 8, 19, 64, 135, 220 learning theories 162 Leefsleutels 67, 68, 72, 135 in Actie 67 voor Jongeren 4, 67, 76 legal sanctions 23, 50 status of substances 150, 156 legalisation 43 less consumption 37, 49 drug consumption 24, 51 problematic use 24, 51, 206 less consumption 22 228

Index

mediators 149, 150 medium-exposure programme 170, 171 medium-term effect 168, 203 programme 170 programmes 171 meta-analysis 101, 103, 129, 130, 186 methadone maintenance 23, 50 methodological dimension 182 evaluation 172, 183, 186, 187, 191 limitations 139, 173, 189 problems 3, 4, 11, 15, 30, 139, 197 rules 178, 179, 188 standards 15, 139, 200 Mia's Diary 3, 77, 79, 80, 81, 82, 83, 84, 85, 87, 88, 89 misuse 140 mixed approach 149, 163 models and limited scope theories 162 monetary terms 195 monitoring 7, 11, 43, 54, 117, 173, 183, 189, 194 of averse side effects 173 of context 194, 209 of unexpected adverse side effects 183, 189 moral development 132 education 94 judgement 5, 41, 185 multi-theoretical approach 161

licit drugs 156 life skills 14, 31, 65, 67, 68, 69, 70, 72, 73, 74, 105, 138, 160, 162, 167, 168, 169, 205, 220 approach 168 training 165, 167 Life Skills for Youngster 67, 68, 69, 70, 72, 73 Life Skills in Action 67, 70, 74 lifestyle 5, 23, 50, 59, 82, 133, 162 pattern 120 life-time prevalence 157, 158, 160, 205, 206, 216 listening skills 68, 169 literature overview 1, 3, 4, 13, 21, 43, 46, 49, 70, 71, 75, 93, 94, 96, 98, 109, 116, 161, 186, 205, 215 living conditions 7 local programme 147 long-term effect 168, 170, 174, 203 programmes 170 low level intervention 23, 50 low-exposure programme 170, 171 low-risk-group 157, 158 LTP 205 -Mmagic bullets 43 maintenance of drug use 116 of use 117 management aspect 192 mass communication 118 mass media 22, 29, 37, 50, 89, 99, 118, 119, 167, 210 measurement error 212, 213, 214 problems 216

-Nnation-wide programme 147 naturally occurring interventions 15, 16, 104 needle sharing 160, 202 229


no use 41, 44, 45, 46, 140, 159 nominal significance level 155, 201 non addicted persons 153 harmful use 140 normative elements 169 theoretical approach 161 users 153 no-risk-group 207 norm setting 169 Norm setting 169 normalisation 41, 43 normative approach 165, 166, 168, 169 approaches 96 education approach 93, 97 influence 100, 123 social influence 101 support 96, 102 norms favourable to drug use 117 not explicitly expected effect 144, 175, 176, 182, 194 nuisance variance 211 number of exposures 170 substances involved 150, 156

variable 12, 32, 39, 125, 138, 187, 199, 200, 202, 204, 205, 212 overall amount of problems 159 -Pparental expectations 112, 115 norms 13 support 102 partial effectiveness claim 190 proof of effectiveness 173, 189, 190, 197, 204 peer behaviour 112 education programme 28 group approach 149 group oriented project 27 group work 43 influence 13, 96, 112, 113, 116 leader 88, 95, 96, 105 leader education 37 pressure 45, 68, 95, 103, 121 perception of relative vulnerability 122 permissive attitude 6, 45 person oriented approach 163 personal competence 167 experience 186 growth 98, 103 problems 205, 206 values 145, 159, 184, 185 personality change 46 characteristics 160, 205 development 46, 185 orientation 46 person-focused approach 47 persuasion 98, 118, 162, 165, 168, 169

-Oobservational unit 213, 217 opinion leaders 118 other institutions 148 outcome data 11, 144, 176, 182, 189 evaluation 10, 11, 14, 27, 30, 31, 34, 39, 40, 47, 48, 54, 55, 56, 57, 58, 84, 126, 144, 175, 176, 182, 188, 190 measure 103, 109 quality 176 study 197

230

Index

to take a commitment against drugs 165 persuasion to take a commitment against drugs 169 phase 1 171, 172 phase 2 171, 172 phase 3 171, 172 Phase 3 178 phase 4 171, 172, 173, 174, 181 Phase 4 178 phase 5 171, 173, 174, 181 Phase 5 178 phase 6 171, 174, 181 Phase 6 178 pilot study 172, 177, 187, 189 planning of evaluation 13 pledge 169 Pledge 169 Poland 26, 34, 49 police records 24, 51 strategy 23 political education 45 interests 7 Pompidou Group 2, 3, 8, 28 poverty 118 power 201, 214 adequately powered design 214 calculation 211, 218 considerations 211, 212 underpowered design 214, 215, 216 underpowered study 215 preformative evaluation phase 144 phase 172, 177, 178, 182 prescription drugs 156 prevention approach 173, 189 concept 139, 201

definition 143 effort 204 efforts 210 expert 167, 184 philosophy 41 programmes 172 research 171, 172, 207 strategies 209 primary efficacy variable 141, 155, 189, 190, 191, 200, 201, 202, 203, 204, 207 prevention 1, 2, 3, 8, 10, 11, 12, 13, 15, 21, 22, 25, 27, 28, 29, 30, 31, 32, 33, 34, 39, 40, 45, 47, 49, 52, 53, 59, 79, 93, 115, 134, 139, 143, 147, 148, 151, 152, 153, 197, 204, 214 target dimension 204 target person 149 target variable 204, 205 problem areas related to substance use 204, 205 behaviour 111, 113, 114, 131, 162 behaviour theory 162 levels 150, 151 orientation 158, 159 oriented goal 150, 157, 158, 159 parameter 158 reduction 157, 158, 159, 205 solving 114, 160, 168 problematic substance use 153, 160, 205, 206 use 6, 152, 204, 206, 211 problems caused by substance (ab)use 158 process

231


psychoanalytic oriented theory 162 public health 6, 59, 130, 134 publication bias 214, 215

data 144, 176, 182, 188, 189, 192 evaluation 7, 11, 27, 31, 40, 47, 72, 144, 175, 176, 182, 187 quality 176 variable 199 programme developer 10, 11, 12, 13, 15, 146, 150, 180, 181, 187, 188, 192 duration 163, 170 effectiveness 145, 146, 173, 189, 190, 192, 197, 199, 202, 204, 206 effects 214 evaluation 189 evaluators 2, 3 feasibility 189 objectives 187, 188 planners 2, 3 staff 11, 15, 146, 149, 150, 175, 180, 181, 188, 192, 199 testing phase 177 prohibition 41 prohibitionist position 6 promotion of psychosocial competence 167 proof of effectiveness 14, 139, 145, 146, 155, 173, 174, 189, 190, 197, 204, 208 protective factor 172, 207 protocol 198 deviations from the protocol 198 registration approach 215, 216 written study protocol 197, 198, 201 prove of effectiveness 139 proxy measure 202 pseudo-prevention 7

-QQA 183, 191, 192, 193 quality assessment 192 assurance 174, 176, 183, 191, 193 control 181, 192 management 181, 192 of sample 198 of target variable assessment 199 of the programme 199 quantitative measure 103 quasi-experimental design 12, 139, 141, 173, 189, 190, 202, 209 quaternary prevention 151, 152, 153 QUEST 3, 28, 67, 79 -Rrandom assignment 12, 105, 106, 107, 108, 114, 119, 123, 124, 209, 213, 215, 217 sample 217 rationalistic theories 162 receiver 42, 118 recreational use 140, 152, 153, 157, 159, 205 refusal training 95 regional programme 147 rejection of underpowered designs approach 215 relapse prevention 143, 146, 151, 152, 153 relative effect 211 rely on street information 165 232

Index

reported drinking 112 repressive strategies 165, 166 research based evaluation 10 based expertise 186 based historic evaluation 186 findings 186 limitations 197 resistance to social pressure 160, 168 resistance training 37, 43, 108 responsible use 43, 44, 45, 150, 159, 184 responsible use orientation 150, 159, 184 risk factor 45, 107, 111, 114, 115, 116, 117, 129, 172, 206, 207 focused approach 93, 111, 115, 116 focused intervention 111 group 214 groups intervention 37 role theory plays 161 role-playing 37, 114 routine phase 144, 171, 174, 177, 181, 182, 185, 192, 193, 194 Routine phase 178

context 141, 143, 144, 145 interpretation 145 meaning 146 opportunism 197 proof of existing hypothesis 179 sense 143, 145, 146 understanding 145, 146, 151 second summative evaluation phase 144 phase 177, 178, 181, 182 secondary prevention 143, 147, 148, 151, 152, 153, 214 variable 200, 201, 203, 204 selective information giving 165, 166 programmes 148 self awareness 168 confidence 44, 68, 72 control 35, 160, 168 development 160, 168 efficacy/social competency model 93 esteem 24, 44, 47, 51, 99, 107, 108, 109, 160, 168, 169 medication theory 5 report 103 self-reported drug consumption 216 Sert 4, 135 short duration 170 time period 13 short-term effect 203 programmes 170, 171 results 13 situational variable 216 six-phase-model 171, 178 size of the target population 147

-Ssafe use 27, 41, 45 safer use 22, 37, 49, 204 sample size 14, 119, 188, 197, 211, 212, 213, 214, 217, 218 scare tactics 102, 123, 165 technique 43 school 148 achievement 115 based intervention 120 failure 115, 116 risk factors 117 scientific 233


skill training 38, 96 Skills for Adolescence 28, 67 small trial 172, 177, 187, 189 smoking prevention 99, 100, 101, 110, 122, 130, 132 social acceptability 122 competence 167, 190, 204 control theory 162 deprivation 117 development model 162 environment 147, 163 environment/b 148 influence 45, 94, 100, 101, 103, 106, 107, 109, 110 integration 158, 160, 204 interaction 109, 114 learning model 162 norms 162, 169 skills 23, 32, 35, 50, 72, 80, 93, 95, 96, 98, 101, 102, 104, 105, 106, 107, 108, 109, 110, 111, 114, 120, 122, 126, 129, 131, 132, 160, 168 skills training 23, 50, 96, 102, 108, 109, 110, 122, 131 societal norms 5, 101 socio-cultural background 46 contexts 3 Spain 26, 34 specific characteristics of the target population 147 goals 150 phase 187 strategy 187 substance 156 sponsor 10, 15 St. Patrick’s College 4, 91, 134, 135 starting point 207

state-of-programme dimension 182, 183 statistical significance 155 stress management 160, 168 structural approach 154, 163, 164, 165 change 7, 210 components 193 conditions 193 data 144, 176, 182, 193 evaluation 144, 174, 176, 183, 187 quality 176, 193 variable 199 vs. communicative approaches 163 Structural evaluation 193 study design 10, 12, 14, 181 endpoint 200 subgroup analyses 213 substance (ab)use 140, 155, 158, 164, 167, 169, 204, 209 (ab)useprevention 209 abuse prevention 32, 93, 126, 130, 133, 152, 153, 155, 157, 166, 184, 206, 214 dependence 140 related goal 155, 156 related goals only 155 related problem 157, 204 related technique 156 specific effect 155 specific information 166 specific technique 169 specificity 150, 155, 169 unspecific technique 169 use behaviour 108, 115, 121, 125

234

Index

Suchtpräventionsstelle, Zurich 4, 135 summative evaluation 14, 144, 177, 182 phase 177, 178, 181, 182 supply reduction 154, 163 surrogate target variable 24, 51 variable 13, 158, 191, 202, 203, 204, 205, 206, 207 surveys 24, 35, 46, 51, 54, 59, 119, 127, 184, 216

of reasoned action 162 oriented approach 161 time of consumption onset 157 too late engagement 13, 14 total abstinence 37, 126, 128 traditional behaviourism 162 beliefs 5 training of trainers 37 treatment 143 effectiveness 158 policy 23, 50 programme 153 records 24, 51, 184 Trimbos Institute 4, 135 types of goals 150, 151, 154, 155, 156

-Ttarget behaviour 170, 214 dimension 203, 204, 206 group 144, 147, 150, 163, 172, 175, 190, 191, 193, 202, 207 person 147, 148, 149, 150, 154, 163, 166, 168, 169, 170, 175, 176, 188, 191, 193, 196 population 10, 11, 12, 13, 22, 24, 35, 36, 50, 51, 100, 147, 148, 174, 175, 186, 198 problem 152, 153 variable 199, 200, 202, 203, 204, 205 teaching style 116 telephone interview 24, 51 terminology 2, 139, 140, 146, 183, 185, 190, 200, 202, 207 tertiary prevention 143, 147, 148, 151, 152, 153 type A 151 type B 151 testing phase 144, 171, 173, 174, 177, 181, 182, 185, 189, 190, 192 Testing phase 178 the Netherlands 26, 32, 63 theoretical problems 11, 15 theory

-Uultimate problem dimension 203 programme goals 204 target dimension 203, 206 target variable 24, 51, 200, 202, 203 uncontrolled variance 211 UNDCP 4, 135 universal programmes 148 Universidad Autonoma de Madrid 4, 135 Universita Cattolica del Sacro Cuore 4, 135 University of Bergen 4 University of Helsinki 4, 135 unpredictable effect 210 USA 39, 40, 41, 42, 43, 46, 169, 208, 215, 218 US-American prevention doctrine 41, 42, 169 u-shaped curve 219 relationship 205, 206, 208

235


utilisation of alternative activities to substance use 168 utilitarian theories 162

spiritual 185 symbolic 185 voluntaristic theories 162

-Vvagueness 139, 145, 152, 192 value clarification 168, 169 values emotional 185 individual 185 monetary 185 moral 185 political 185 social 169, 185

-WWar on Drugs 40, 41, 46 War On Drugs 39, 42 WHO 5, 7, 8, 9, 28, 45, 89, 162, 167, 191, 217, 220 work 148 market 158, 160, 204 place 164 written concept 22

236

Index

237