Experiment Spot-Checks: A Method for Assessing the ... - CiteSeerX

poI1: EthJa1l. GuJdelJnes.for

the Prot.«Uon

qfHtlroon~ 'MIshIngtm. D.C.: D.HE.W. PllbilAottrwt78-0012, 1978. 4. GJass.ES: Rr8Uuctur1ngIJ1bnm ansent: Lqpl thmIpy fir the doctor-paUmt reIatIcnshIp. YaleLaw .kxunal197O; 79: 1533-76.

5.=~~~::-

Ycxk:AmeItcan ElsevIerPublIshIng Co.. 1974. 6. Mebd. A. and KaI:JnIck.1D: Jnbmed CXI18eDt 10medical beatment: An analysis drecent ~ttQr\. [kWersIy ifPtJJsburgh Law Revfew 1980; 41:407-45. 7. Presldent'sa...llUdt.o!IrIM ir the Study d EthIcal Problems In MedIdne and BkmedIcal and BehaYIcraIResesrch:

~ HwnanSuljects.~ D.C.:U.s. GoYemment PrtntlngOfllce. 1982. 8. Katz. J: The SIlent \\btd f/ DodDrand PotJent.New York: The FnJePre8Is.1984. 9. Faden.RR.andBmuchamp.1L:AH& tory and Theory t8lr!formed O:.nwnt. New Ycxk: Oxird Un1YersItyPress. 1986. 10. LevIne.RJ: Ethk:s and R.tigubtbt if CfInf. cat Research.2nd eeLNew ~ Yale Uniw::rSttyPress. 1988. 11. \atch. RM: ThePI:ItIentas Partner.A Theory t8 Hwnan bperfmenIDtIon Etha. BkxxnIngtoo: indiana UnIYmIty Press. 1987.

Experiment Spot-Checks: A Method for Assessing the Educational Value of Undergraduate Participation in Research by R. Eric Landrum and Garvin Chastain For years. psychologydepartments around the nation have Justified the use of subject pools as having educational value for students.l.2.3That is, they claim that students can learn firsthand about the research process through participation. and this out-ofclass activity fosters an understanding and appreciation for psychological research.Ideally. there is a dual benefit (to the researcher and the student participant) with mtntmal risk (typically the time necessary to participate).In the last fifteenyears.however. somehavebegun to questionwhether participation In a subject pool is a valuable educational experience.4.S A related issue is that of coercion or perceived coercion for students to participate In research. While an Interesting issue. it is not the focus of this paper. We attempt to answer the following: (1) Do students feel that participating in research is a learning experience? and (2) Can we have

someconfidencethat the Instrument Leak7foundthat studentsviewedreused to measure student perceptions has somevalidity? These questions have been addressed In the past with mixed resuits. Britton4 developeda questionnatre to assessthe ethical and educational aspects of subject pool participatton. and found that an experimenter's politeness.student comfort. and the explanation given for performing

search participation positively. In exploring student attitudes toward research. Nimmer and Handdsman8 performed a quasi-experiment with groups of students working In different research situations. They found that students felt that research participation doeshave someeducationat value. Further. they recommend adequate debriefings and giving a

theexperimentratedhighly.whilethe one-pagequestionnaireto students

educational value of the experience rated somewhat lower. Britton urged superviSOrsof subject pools to gather Information about the subjects'expertence. and suggested improving the debriefing processas a method of enhanctng that experience.Debrieflngs tend to be seen as a critical component. Coulter6suspectedthat tnsufficient debrieflngswere responsiblefor students rating research experiences as boring, irrelevant, and a waste of time. Of course, student opinion of reR.EricLandrum.PbD,isassociJlte professor search participation is nUxed. AlandGarvinCIwtain, PhD,isproftssor, Dethough Coulter6 found students held partmrnt ofPsychology, Boise StateUniversity, negative attitudes toward research. Boise. Idaho.

4

for their feedback. In their survey of graduate departments In psychology. Sieber and Saks5 noted that some departments have developedan evaluation form to assess educational value. The present study is an extension of that approach. InvoMng the development of an assessment Instrument directed to student-participants. and its use by a group of researchersfor one academicyear. Method Partfc#pant8. Each student 111 the General Psychologycourse at Boise

~

July-August 1995 State Un1VerS1tyis required to complete some sort of outside-of-class act1v1ty exposing him or her to psychologtcal research. Most students choose to be research participants. Duringthe 1993-1994 academic year. all researchers in the psychology department cooperated by having 10 percent oftheirexpertmental subjects fill out an experiment spot-check form. Questions on the form are presented in Table 1. Two hundred subJects completed these forms during the 1993-1994 academic year. Materials. Based on a review of the literature and the general concerns

Table

1.

Means

(M)

and

Spot-Check

about the value of research participation. we formulated six items to which students replied using5-po1nt Likertscale (strongly disagree to strongly agree) responses. We also tracked the particular experiment for which the form was completed. the date. the number of other projects in which the students had participated (our students need two research experiences). the semester of participation. and whether or not the student signed the spot-check form (the student's signature was optional). Design. and Procedure. Spotcheck forms were distributed to all

StaDdarcI

DeviatioDa

(SD)

I was

treated

research

fairly

and

II

with

respect

during

my

aD

4.87

.35

session.

~~

that

I

6.

The

think

this

research

experience

was

a

waste

partictpatlng

understand

research

purpose

to

Into

this

project

helped

me

was

adequately

Tested

Number

of

Student

Signature

2=sprtng) ParUcipal1ons

blyes)

Responses

to

Items

1-6

on

a

5-pomt

L1kert

sca1e

3.60

.89

4.41

.60

22.65

simultaneously.

factor

response

rotations

a

mum

eigenvalue

from

1

-

strongly

2. Factor

Item 2. Learned 5. Understand 3. Variety 4. WasteofTtme Experiment No. Days into Semester 6. Explained SemesterTested 1. Respect Signature

(..oadfn~

Factor 1 Educational Value

Based on Rotated Factor Matrix Factor 2 Ttme

Factor 3 Factor .Professlonal1smDIsclosure

.818 .810 .651 -.605 .812 .710

among

variables.

a

of

In

the

resulting

degree

1.0.

a

m1n1-

solution

(An cutoff

of

clustered

a

the

iterations. to

ma-

study.

and

of

7

often mathe-

current

rotation

similar

a

the

the

vartmax

In is

variety

of

values.

using

of

multiple

are

using

of

of

patterns

interrelationships

1.46.83

value

results

ident11Y

among

maximized

tr1x

The

analysIs

1.30.46

disagree to 5 = strongly agree. ti - 200.

Table

stu-

abies

.60.44

were

the

.82

cating Note:

or not

form.

1.94

converged (O=no,

whether the

.75

matical

Research

experi-

4.19

70.1

(l-fall.

PrIor

the

A factor analysis was performed on the student response data. Factor analysis is a multivariate statistical tool that allows for the examination of multiple relationships between vari-

me.

Semester

Semester

and

signed

These

experiment

when

.94

better.

ofthts

explained

Days

In

elapsed

research.

a that

had

ment was conducted. the semester of participation. the number of times the student had already participated in

3.52

. 5.

Table 1 presents means and standard deviations for the six questionnaire items. as well as information on the number of days into the semester

dent

2. I learned about psychology by participating In a research project. 3. The research experience Is a good way to add variety to Introductory psychology. . 4.

Results

that

Items and Other Data

Item 1.

for

researchers at the begtnning of each semester. They were asked to administer this form to 10 percent of their research subjects. Researchers varled in how they selected their sample: for example. those running stngtesubject sessions administered the form to every tenth subject. wh1le those running group sessions occasionallyasked an entire group of students to complete the form.

value

eigenindi-

communality variables;

itera-

tions refers to the number of rotations necessary to find the optimal statistical solution.) Table 2 shows the factor loadings. scores Indicating the degree of expression on anyone particular factor (all above 0.5). Factors 1 (educational value) and 3 (profesSionalism) are the most important. Factor 2. time. seems to be an Index of how far Into the semester the experiment was completed. and Factor 4. with only one question loading on it. seems to identify disclosure (signature).

Di8cussion .744 -.574 .566 .898

Note: A factor loading representsthe degreeof relationship (or weight) between a parUcu1arItem and a factor. 0nJy factor loadings above0.5 are displayed here. Actually. each Item has a loading on all four factors. but It Is desired that the Item load highly (above0.5 on one factor. and low on the other factors). Fortunately. thai Is the case In the present study.

5

Do subjeCtpools have educational value? Our students' answer is yes. Students agreedwith statements indicating that participating helped them to learn about psychology and to understand research better. and students strongly agreed that they were treated fairly and with respect. They further indicated that participating in the experiment added varI-

IRB ety to the course. and that the purposeof the experimentwas adequateIy explained. Contrary to the findings of Coulter. our students disagreed that the experience was a waste of Ume. These findings. on the whole.

that make goodsenseconsidering the ortgtnalttems and the purpose of the spot-check. Finding such underlying patterns and factors Is part of the processor establishing validity or the spot-check questions.

questions developedhere. others can have confidence In measuring the educational value and professionalIsm of research partlctpatlon.

provide us with some confidence that students value the educational experlence provided by their participation

Having curren t Information about the performance and outcomes of the department subject pool is valuable.

1. Orne.M: On the!OdEll~ d the ~c:xpertmenL.AmencxlnPslt choIogCst 962: 11:776-83.

References

In research. and that our researchers It provides a snapshot of current per2. s.dJer..IE: PIarn11~ ~.... and student researchassistants do a formance. a method or """&aAO RPYnI.u~the raeardl:Agut:iefr.studentandll'*mal """'6 AeIJEuI Boards. NeM:Iury Park. CaJI(: good

job

of

treating

students

fairly

educational

value

of research

part1ci-

SBge. 1992.

and respect. and provide an adequatewith debriefing.

patlon (accountability). a vehicle for Identifying and solvingandsubJect-re-

3 . pd. Unck:rwaxL KqJG: \bbeI BJ ~asSchwenn. ~ E and10 paint

Additionally. the spot-check form revealed that most of our students completed research In the second half of the course (70.1 days Into the se-

lated problems should they arise. Use of such a spot-check form has other benefits as well. Distributing the form to researchers at the beginnIng of the

dttme m the8Chcdtam. Jounw1I rfVerbalL&lrT*lgand\t:JtalBehavtr 1964: 3: 222-25. 4. BrIttm. BK:EthicalandechraaDnaIas-

mesterl.that more studentspartlcl- semestersensitizesthem to theseIspated In research In the fall semester sues. since they know that their re-

=~;S~~ddogy

~

6:~95-98.

(69.5% of the total). that the ava-age number of experiments previously completed when ftl1tn2 out the form

search project and personnel wtll be evaluated. Britton suggests that an experimenter'sbehaviormaybelnfiu-

was 1.45. and that 78.5 percent of the students signed the form. The results of the factor analysis merely indicate that trends or pat-

eneed by the use of a quesUOnruUre. 6. CouIIrr.X:AcadaDlcY8luedreseard1 1b1s procedure also emphasizes the .-~by~Amerf. Importance of debrtefing in expertmnPsycholcglst 1986:41:317. mental studies. and the spot-check 7. Leak.OK:Studmt~cq:)tiondcccrdcn

and that thereare setsof Itemsthat

in that area.

tend to evoke stmilar responses. InterpreUng the factors Is subjective.

Other researchers concerned with the educatlonalvalue of research par-

8: 147-49. 8!111rm1e1: JG andHaa.dd8... MM:Ef. I'edsd~pooIpoIicy;"U~udentat-

bu t we beUevethat the most meaningful factors to emerge are educational value and professionalism. factors

tlclpatlon are encouraged to pursue those issues empirically. Based on the psychometric qualities of the

UtudesUMwd~ and ~ 1ogIcaIreaean:b. T~cfPsydk1lDgy 1992:19. 141-44.

terns exist in the students' responses.

form clearly captures student opinion

5. s.dJer.JE.~S&ks.M1:Aomsusd ~ poolch8-.tb1tA... md paIde8. Amermn~ 1989:44:1(1)3..61.~

and

~

~~=

Clinical Trials Committees: How Long Is the Protocol Review and Approval Process in Spain? A Prospective Study by Rafael Ortega and Rafael Dal-Re Since 1982 regulation of all clInJcal tr1al protocols (phases I to M In Spain. Irrespectiveof sponsorship. Invalves mandatory review and approval by (1) the clInJcal tr1als commlttee (CTC) at each participating center. and (2) the Ministry of Health. 1.2 The regulation also Ineludes guidelines for the composition of the CTCs(whJchmust be approved by the Minisby of Health). and states that they must evaluate ethical and sclentlflc aspects of the protocol. as do ethics review commJteesIn other western countries. The content of the dossier submitted Is quite standard1zed:protocol (accordJngto a 23-

(alsoomdal format). casereport fonn. addition. analysis of factors that and updated Investigator'sbrochure. could potentially lnfiuence--e1ther In recent years. a patient Information posttlvely or negatively-the time sheet has also been requested.In ad- consumedby the reviewand approval dJtlon. health Insurance coveragefor process Is also of Interest. since the potentlal damages for subjects who results may bring possible sourcesof participate In trials Is also required by Improvement. law.3 All documents should be In Spanish. but In practice this Is limited Material and Methods to the protocol and forms related to patient's consent. The fIrst 10 drug protocols submltStudying the protocol review and ted by our companyto the CTCssince approval process at the ere level Is 1 July 1992 were evaluated. A datarelevant becauseof the Impact It may base of study and CTC-related feahaveon the tlming of cl1n1calresearch tures was designed In advance for projects. hence In their proper plan- this prospectivestudy. The following rung. Ddays in InItlatlngresearch due were recorded:type (local or multlna-

It"""' .~

to rPVIf"W - thl!ll. __n_n n~' r

fn

",.t\ , nAtl~t

t'nn~t ---

fnrm

IWaeI Ortep, MD, it Qi1dca1 Rnti81ch Mmago, Medial Department. andRafaelOat-Re, MD, rhO, it MedicalDirtdor. SmithJ(Jirw 8«chRmPlwnrlllaUtiaIls.Mm1riJ,Spain.

have been a~ ---~

source of concern.4-9 and they are likely to become Increasmgty so In the future, when shorter times for the International cllnJcal development of druQ$ are to be soUjl,ht actively, 10 In

6

tkmaJl and nbase of --. ---- nrotocol. n_no

essenu-

Ual features of design (comparattve. use of placebo. double-blind. multlcenter. etc.), time (days) from submission (by the Investigator) to approval and from approval to reception