Procedural Text Structure and Reader Perceptions ...

The Journal of General Psychology, 2002, 129(1), 18–35

Procedural Text Structure and Reader Perceptions and Performance VIRGINIA A. DIEHL Department of Psychology Western Illinois University CAROL BERGFELD MILLS Department of Psychology Goucher College

ABSTRACT. This research tested the relationship between text structural variables (on vs. off the causal chain, active vs. static information, and hierarchical structure) and reader perceptions, reading time, and true–false performance for procedural texts. Twenty-four college-age participants each read 3 procedural texts. As predicted, sentences that were (a) on the causal chain, (b) active, and (c) associated with task steps were read more slowly and judged to be more important (p < .05). The results were similar to those previously found with narratives, except for the hierarchical structure variable. Also as predicted, text differences were found such that as the variation in the type of information contained in the text increased, relationships with the structural variables increased. Key words: procedural text, reader perceptions of text, reading time, structural text variables

EVERY DAY we are confronted with written instructions. For most of us, our ability to use the forms and devices in our world depends on how well we understand the instructions. In this study, we examined factors that influence how the sentences from instructional texts (also called procedural texts) are perceived and processed. Most of the research in this area uses narratives (stories) to investigate the relationship between structural variables, such as the text’s hierarchical organization, and performance variables, such as reading speed and memory. In the present study, we looked at structural variables in procedural texts to see if their This research was supported by a Western Illinois University Research Council Grant. The authors thank Jennifer Miller and Christine Willingmyre, who helped with data collection and scoring. ——Address correspondence to Virginia A. Diehl, Psychology Department, 1 University Circle, Western Illinois University, Macomb, IL 61455; [email protected] (e-mail). 18

Diehl & Bergfeld Mills

19

relationships with readers’ perceptions and performance were similar to those in narrative texts. According to the text representation model (van Dijk & Kintsch, 1983), when a reader reads a text, two types of mental representations are formed. One, called the textbase model, is based on the information in the text. The other, called the situation model, goes beyond the information stated in the text to represent the situation described by the text. For example, a procedural text about making a wind-up toy might include the sentence “Loop the rubber band around the pencil several times.” The situation model we construct from reading that sentence would draw on our knowledge of rubber bands (that they stretch) and pencils (that they do not stretch). Studies have demonstrated that a reader’s textbase and situation models can develop to different extents as he or she reads. This depends on factors in the environment that lead the reader to focus on certain aspects of the text, causing differential performance on measures that tap the two representations (Mannes & Kintsch, 1987; Perrig & Kintsch, 1985). These studies, and most studies of text comprehension, have used narrative or expository texts. Procedural texts, which provide information about how to do a task, allow us to think about the situation representation in a slightly different way. The situation model for procedural texts involves information about how to do a task. If a reader is concerned about performing the task well, he or she should pay more attention to the parts of the text that are important to getting the task done (i.e., building a situational representation). Several studies with procedural text have provided evidence for this notion. Mills, Diehl, Birkmire, and Mou (1995) carried out a study in which participants were instructed to read a procedural text with the goal of either doing the task or recalling the text. The reader’s purpose for reading interacted with the importance of the information to task performance for both reading rate and recall. For less important information, task-oriented readers (those who read to do the task) read faster and recalled less than those who were more text oriented (those who read to recall the text). Diehl and Mills (1995) looked at the effect of what the participant did while reading on his or her ability to do the task and remember the text. Task-oriented readers (those who did the task or watched the experimenter do it) performed the task faster but recalled less than those who only read the text. Diehl (2001) had half of the participants perform the task steps while reading (Read and Do) and had half just read the text (Read Only). In general, given only a procedural text (i.e., reading and not doing), participants tended to focus on the textbase. They nit-picked about word usage and punctuation. On the other hand, when readers were asked to do the task while reading, they focused on getting the task done, and they judged the sentences in that light. These studies support the text representation theory of van Dijk and Kintsch (1983) with procedural text (a type of text that has as its focus getting a task done), rather than with narrative. A task orientation, whether manipulated by the reader’s goal or by what the reader does while reading, appears to lead the reader to

20

The Journal of General Psychology

focus on aspects of the text that are important for getting the task done. In the present study, we tried to cultivate a “task orientation” by telling participants that they might be asked to do the task at the end of the study. This led us to predict that, with a task orientation, the levels of the structural variables that are closely related to task performance will be read more carefully and judged more harshly. We did not manipulate task orientation in this study because we wanted to focus on the relationships of the structural variables to our outcome measures. We measured participants’ reading rates for their first and second readings and their memory for and comprehension of the text. We also asked them to make judgments about each of the sentences in the texts. These judgments were on how difficult the sentence was to understand, what (if anything) was wrong with it, and how important the sentence was for understanding the task. One of the structural variables in this experiment that has been shown to be important with narratives is whether the sentence falls on the causal chain (Trabasso & van den Broek, 1985). The causal chain links the setting of a text to its closing in a chain of events that have a cause-and-effect relation to one another. Sentences that are not on the chain tend to be elaborations and background information. The chain plays an important role in tying the text together in a series of cause-and-effect ideas (and is therefore critical to task performance). We therefore expected that for procedural texts, sentences on the causal chain would be read more slowly, rated as more important and more difficult, and remembered better than those not on the chain. In addition, we predicted that people would identify more problems with the sentences on the chain because these sentences tend to be important for understanding the task. A second variable that we investigated and that has been studied with narrative and expository texts is the type of information contained in a sentence. For example, Graesser, Higgenbotham, Robertson, and Smith (1978) found that, when reading naturally, people remember more active information (related to how or why an event occurs) than they do static information (telling who, what, when, where, or how much). Mills et al. (1995) found that participants who read with the goal of performing the task recalled more of the ideas from the active sentences than they did from the static sentences. Because active sentences in procedural texts are those that describe a task step, we expected that they would be read more slowly, rated as more important, and remembered better than would static sentences. We also predicted that readers would find more problems with the active sentences for the same reason. A third variable that has been shown to be important for memory and reading time with narratives is the level of information in the text’s hierarchical structure (Cirilo & Foss, 1980). With narratives, high-level sentences are remembered better, read more slowly, and judged to be more important than are lower level sentences. We expected a different pattern to emerge with procedural text, based on where the task steps occur in the hierarchy. If one of the levels contains the majority of the task steps (Level 2, with our procedural texts), we would expect

Diehl & Bergfeld Mills

21

sentences on that level to be remembered better, read more slowly, and judged to be more important than those on the other levels. We also predicted that more problems would be identified with the sentences on that level. The texts that were used in this study seem quite different from each other. “How to Make and Use a Spool Vehicle” (“Spool”) is storylike. It contains task steps as well as historical information and explanations. The language is quite simple. “Wiring and Using a Shunt-Type Ohmmeter” (“Ohmmeter”) seems to be the most technical text. It is almost purely procedural, and the subject matter is less familiar to most people. “The Torn and Restored Paper Ribbon Trick” (“Trick”) is in-between because its language is simple, but the task is quite complex. We expected that Spool would be the easiest text to read and that Ohmmeter would be the most difficult. Because Spool’s sentences have more variation, we expected that the structural variables would be most strongly related to the outcome measures with the Spool text. Because the sentences in the Ohmmeter task are very similar to each other, we expected that the structural variables would account for little of the variability in the outcome measures with this text. We also examined the relationship between text structural variables and reading rate, an objective measure of online reading difficulty, and ratings of sentence difficulty. The participants also rated the importance of each sentence for understanding the task. Finally, a true–false measure was given at the end of this experiment to provide a measure of comprehension and inferencing. In our previous studies, comprehension measures have not been enlightening (Diehl & Mills, 1995; Mills et al., 1995). In this study, we made a systematic attempt to vary the types of questions included in the true–false measure in an effort to better understand the comprehension process. Method Participants Twenty-five introductory psychology students participated in this study for extra course credit. One participant’s data were dropped because she did not follow instructions. Participants were randomly assigned to each of the 12 combinations of the six text orders and the two true–false versions, with the constraint that a male and a female participant received each combination. Materials Three procedural texts were used in addition to a practice text. All texts contained sentences that were on and off the causal chain, had active and static information, and were on three hierarchical levels. Spool (Herbert, 1980), about a children’s toy, contained 17 sentences and had 297 words. Ohmmeter, taken from

22

The Journal of General Psychology

“Science Fair 160 in One Electronic Project Kit” (Radio Shack, Catalog number 28-258), was about an electronic device. It had 24 sentences and 397 words. Trick (Hay, 1975), about a magic trick, had 28 sentences and 451 words. A text that described how to set a digital clock was used as a practice text. The three experimental texts are presented in the Appendix. The Spool text described the construction of a child’s wind-up toy. Making the toy involves putting a rubber band through a hole in the spool, tacking one end of the rubber band to the side of the spool, and putting the other end through a washer and attaching it to a pencil. The pencil is turned to make the spool move, and this winds up the rubber band. The Ohmmeter text described the wiring of an electronic device called an ohmmeter. This is done using spring-clip connectors on an electronic project board. The Magic Trick text explains how to perform a trick that involves tearing a strip of paper into small pieces in front of an audience. The torn pieces are secretly placed into a prop called a thumb tip, after a whole (untorn) strip is surreptitiously removed. The whole strip is then presented to the audience. The chain variable (on or off the chain) was determined for these texts in a previous study (see Mills, Diehl, Birkmire, & Mou, 1993). Causal analyses were performed to determine whether a text unit was on the causal chain that led from the opening to the closing of the text. Judges considered whether each idea was causally related to every other idea in a text. For a unit to be causally related to another, it had to pass several tests (e.g., a counterfactual reasoning test of the form “If not A, then not B”). If the two units were causally related and were part of a continuous chain of causal connections, they were judged to be on the chain. Across the three texts there were 47 sentences on the chain and 22 off (see Appendix). Type of sentence (active or static) was determined with the criteria of Graesser et al. (1978), with the additional constraint that an active sentence had to be necessary for task completion. There were 36 active sentences and 33 static sentences (see Appendix). Level was determined by outlining the texts and numbering the “height” a sentence had in the hierarchy. There were 3 levels: 16 sentences were rated high (Level 1); 32 sentences, medium (Level 2); and 21 sentences, low (Level 3). At least two judges conducted each of these analyses. Discrepancies, which were few, were resolved through discussion. As would be expected, these structural variables were not completely independent in these naturally occurring texts. Correlations of active and chain, active and level, and chain and level (each with 67 degrees of freedom) were .34, −.30, and −.49, respectively (ps < .05). Active sentences tended to be on the chain and higher in the hierarchy, and sentences that were on the chain tended to be higher in the hierarchy. As noted earlier, Level 2 contained most of the active sentences and was composed mostly of active sentences: Of the 32 sentences at this level, 25 (of the 36) active sentences were on Level 2 (compared with 8 at Level 1 and 3 at Level 3). As noted earlier, we expected that sentences

Diehl & Bergfeld Mills

23

on the second level (which contains many active sentences) would be remembered better, read more slowly, and judged to be more important than those at the other levels. A true–false test provided a measure of comprehension of the sentences in the texts. The true–false test contained two types of items. One type—direct items— tested memory for the surface structure of the text. An example of a direct item from the Spool text is “One end of the rubber band is placed through the hole of a washer.” The other item type—inference items—tested the understanding of the task— for example, the spatial layout of the part. An example of an inference item from Spool is “When the spool vehicle is completed and set down, one tip of the drag will touch the table.” Inference items were not directly stated in the text. The reader had to understand how the task was done or how the device worked, to be able to get these items correct. The items on the test queried the reader on all types of sentences from the texts: on and off the chain, active or static, and different hierarchical levels. There were two versions of the test, each with different items (to increase generalizability). Each test contained 12 items from the Ohmmeter text, 11 from the Trick text, and 9 from the Spool text, including 14 items off the chain and 18 on; 18 active items and 14 static; 7 items at Level 1, 10 at Level 2, and 15 at Level 3; and 16 direct statements and 16 inferences. The device or materials associated with each text were also used in the study. For the Spool text, these parts were the spool, a rubber band, a washer, tack, and a matchstick. For the Ohmmeter text, the Science Fair 160 in One Electronic Project Kit (Radio Shack, Catalog number 28-258) was used. For the Trick text, a thumb tip and several slips of paper were provided. For the practice text, a Radio Shack digital timer clock was used. PsyScope (Cohen, MacWhinney, Flatt, & Provost, 1993) was used in writing an experiment program for a Macintosh computer to present the materials (texts, rating scales, and tests) on-screen to the participants, to record their responses, and to time their reading. Procedure Participants were told that the computer would take them through the experiment step-by-step. The opening screen welcomed them to the study and informed them that they would read several texts that would tell how to do a task (such as setting a digital clock), sentence-by-sentence, twice. Then, as the texts were presented a third time, they would make some judgments about how hard each sentence was to understand, what made it hard, and how important it was. Finally, their memory for the information in the texts would be measured. For the practice text, they were asked to read the text carefully as though they were going to have to do it later, and they were told they might be asked to perform the tasks described by the procedural texts at the end of the study. They were instructed to not let their attention wander during the first two readings so that their reading

24

The Journal of General Psychology

times would be accurate. They were asked to press h as soon as they had finished reading the sentence, and the next sentence would be presented. They were shown the digital clock and then it was put away. They read the practice text, sentenceby-sentence, twice. On the third reading, they made the three judgments for each sentence. Each sentence was presented on the screen, along with one of the three judgment questions. The first was “How difficult is the sentence to understand?” Participants were instructed to choose a number between 1 and 5 that described the difficulty of the sentence. If the sentence gave them any trouble at all, they were told to choose a number other than 1. The second judgment was “What’s wrong with the sentence in terms of how hard it is to understand?” If they had given a difficulty rating of 1 (very easy), they were told to choose the rating of w. Otherwise, they selected one of the other choices. They were instructed to choose x if there was something wrong with the structure of the sentence—for example, length, punctuation, or awkward wording; to choose y if the sentence was in the wrong place, in the sense that it should have come sooner or later, or was not relevant; and to choose z if the sentence needed something more, for example, a definition, more explanation, or a diagram. The third judgment was “How important is this sentence for being able to understand the task?” The rating scale ranged from 1 to 5, with 5 indicating that the sentence was essential for understanding the task. The participants were informed that understanding the task involved more than just being able to do the task. They were asked to consider how important the sentence was for understanding what the task was or how it worked. Once they had practiced reading and rating the sentences, they were told that they would also perform a true–false test about the information from the experimental texts at the end of the study. The participants were invited to take a short break before reading each experimental text if they so desired. They were also reminded that they should concentrate on their reading and that they might be asked to do the task at the end of the experiment, so they should read the text carefully. They were given an opportunity to ask questions and were then shown the task materials for that text (e.g., the spool, the rubber band, the tack, and the pencil for the Spool text). The device was then taken away. The participants read the text through twice, then made the three judgments for each of the sentences when they were displayed the third time. This was repeated for each of the three experimental texts, which were presented to participants in counterbalanced order. The participants performed the true–false test after they had read through and rated all three experimental texts. They were told that they would see a series of statements about each of the texts they had read, and that the statements for each text would be presented together, in the order in which they read the texts. They were instructed to answer true (press t) if the statement was true according to what they read in the text and to answer false (press f) is the statement was wrong on the basis of what they read in the text. There was no time limit.

Diehl & Bergfeld Mills

25

Results Outcome Measures Six measures were collected: (a) difficulty rating, (b) importance rating, (c) reading time per sentence for the first reading, (d) reading time per sentence for the second reading, (e) number correct on the true–false test, and (f) problem type (i.e., what was wrong with the sentence). All reading times were converted to words read per minute, then transformed to base 10 logarithms for the analyses because they were positively skewed. All data (with the exception of problem type) were averaged across participants and were analyzed by treating each sentence (or item, in the case of true–false) as the unit of analysis. The reason for this was that the text structural variables and the dependent measures were based on sentences (e.g., was the sentence on or off the chain and how quickly did the participant read that sentence?). In this section, for each of the three structural variables (chain, active, and level), the main effects (if any) are presented. The results are presented separately for each outcome measure. Next, the results concerning text differences are presented for each outcome measure. Finally, statistics from the problem type variable are presented. These are presented separately because problem type was a nominal variable. Table 1 shows the means and standard deviations of each of the quantitative outcome measures, broken down by the structural variables. For all of the following analyses, alpha was set to .05. Structural Variable Effects Difficulty rating. As predicted, comprehension difficulty ratings showed a main effect of chain, F(1, 63) = 6.10, with sentences on the chain rated as more difficult than sentences off the chain (see Table 1). There were no main effects of active or level. Importance rating. The importance of the sentence for understanding the task also showed an effect of chain, F(1, 63) = 9.67. As predicted, sentences on the chain were rated as more important than those off the chain (see Table 1). Also as predicted, there was a main effect of active, F(1, 63) = 37.83, with active sentences judged to be more important than nonactive sentences. Because Level 2 contained the majority of the task steps, we expected sentences on that level to be judged as most important. This was confirmed with a main effect of level, F(2, 60) = 17.02; planned comparisons showed that Level 2 was judged to be more important than Levels 1 and 3. Reading rate. Table 1 shows the means and standard deviations for the average reading time (in words per minute) for the two trials separately. Larger numbers

26

The Journal of General Psychology

TABLE 1 Means and Standard Deviations as a Function of the Structural Variables Structural variables Active Level On Off Yes No 1 2 3 (n = 47) (n = 22) (n = 36) (n = 33) (n = 16) (n = 32) (n = 21) Chain

Dependent measures

Difficulty ratinga M 1.66 SD .54 Importance ratingb M 3.91 SD .60 Reading ratede First reading M 181.01 SD 42.33 Second reading M 253.76 SD 68.61 True–falsej M .85 SD .17

1.38* .33

1.67 .59

1.45 .34

1.57 .53

1.63 .58

1.47 .31

3.52* .82

4.16 .41

3.38* .72

3.59 .90

4.11 .47

3.45c .61

200.79* 179.23 38.86 46.87

196.14f* 165.79 34.54 42.06

188.40 44.34

202.07g 32.02

304.12* 242.99 66.02 69.41

299.08h* 257.80 59.84 84.37

258.48 70.28

296.24i 53.11

.88 .14

.77 .20

.81 .18

.82 .19

.84 .17

.87 .14

a Difficulty rating where 5 = very difficult to understand. bImportance for understanding the task where 5 = essential for understanding. cLevel 2 differs from Levels 1 and 3. dReading rate is in words per minute. eThere were main effects of chain and active for reading rate when collapsed across the two reading trials. fSimple effect of Active × Trial (see text). gSimple effect of Level × Trial: Level 1 differs from Level 3. hSimple effect of Active × Trial (see text). iSimple effect of Level × Trial: Level 2 differs from Level 3. jThe true–false data were collapsed across participants, so that the outcome measure was the proportion of people that was correct on that item. *p < .05.

indicate faster reading. In addition to looking at the effects of the structural variables (as in the previous analyses), we also included a within-participants variable of trial (first or second reading). For the chain variable, there was a main effect of chain, F(1, 63) = 8.17. As predicted, sentences on the chain (M = 217.38) were read more slowly than those off the chain (M = 252.46). There was an effect of practice, as indicated by the main effect of trial, F(1, 63) = 254.14, with the first trial (M = 187.32) being read more slowly than the second (M = 269.82). The Chain × Trial interaction was not significant. As predicted, there was a main effect of active, F(1, 63) = 9.78, with active sentences (M = 211.11) being read more slowly than were nonactive ones (M = 247.61). There was an interaction of active and trial, F(1, 63) = 15.69; although the active effect was significant for both readings, it was stronger on the second reading. There was no main effect of level, however there was a Level × Trial inter-

Diehl & Bergfeld Mills

27

action, F(2, 60) = 5.82. We expected that the sentences on Level 2 would be read more slowly than those on Levels 1 and 3 for both reading trials. Newman-Keuls tests showed that, for the first reading, Level 1 was read significantly more slowly than Level 3, and for the second reading, Level 2 was read significantly more slowly than Level 3. The difference between Levels 1 and 3 was not significant for the second reading, although the means are suggestive. Table 1 shows the means for these comparisons. True–false. For the true–false measure, the true–false item was the unit of analysis, and the proportion of participants who were correct on that item was the outcome measure (see Table 1). We expected that items that were active, on the causal chain, and on Level 2 would be remembered better than items that were static, off the chain, or on Levels 1 and 3. We were also interested in the effects of the type of item: Whether it was directly stated in the text or whether it was an inference based on what was stated. There was no effect of test version, and test version did not interact with any of the structural variables or text; therefore, version was ignored in the following analyses. The item type variable (inference or direct statement) was included in these analyses. There was no effect of item type for any of the analyses. Chain, active, and level showed no main effects. It may be that our true–false measure was not sensitive enough to detect differences. Correlational analyses were performed to see if difficulty and importance ratings of the sentences were related to accuracy on the true–false test. The analyses were performed by sentence—in other words, the importance and difficulty ratings related to the sentence, which was tested by a true–false item (averaged across participants), were correlated with the proportion of participants who got that item correct. We found no significant correlations, which suggested that importance and difficulty ratings (at least when averaged across participants) are not good predictors of memory accuracy on this true–false test. Text Differences We conducted text analyses to explore predictions about the effects of observed text differences on the strength of the relationships between the structural and outcome measures variables. Because of differences in the language, level of complexity, and types of information included in the texts, we predicted that the Spool text would be judged as less difficult and less important than the Ohmmeter text (with the Trick text in the middle), and that the structural variables would be most strongly related to the outcomes measures in the Spool text, and most weakly related in the Ohmmeter text. The following analyses looked at the effect of text in interaction with each of the structural variables. Difficulty rating. As predicted, there was a main effect of text, F(2, 66) = 4.56, with the Ohmmeter text (M = 1.80) judged as more difficult to understand than

28

The Journal of General Psychology

the Trick (M = 1.49) and the Spool (M = 1.38) texts. None of the structural variables interacted with text. Importance rating. There was no main effect of text. There was a Chain × Text interaction, F(2, 63) = 4.84. As predicted, chain had a greater effect on Spool and Trick than it did on Ohmmeter. Sentences on the chain for the Spool (M = 3.86) and Trick (M = 3.91) texts were judged to be more important than those off the chain (Ms = 2.74 and 3.35, respectively). However, there was no effect of chain for the Ohmmeter text (M = 3.96 for On, and M = 4.11 for Off). The Active × Text interaction was also significant, F(2, 63) = 3.86. Active sentences were judged to be more important for all the texts (Ms = 4.35, 4.03, 4.22 for Spool, Trick, and Ohmmeter, respectively) than for nonactive sentences (Ms = 2.96, 3.37, 3.77, respectively). As expected, this effect was largest with the Spool text. The level variable also had its largest effect on importance ratings on the Spool text. The Level × Text interaction, F(4, 60) = 7.39, showed that, for the Spool text, all levels differed from all other levels on importance ratings (Ms = 2.25, 4.35, 3.27 for Levels 1, 2, and 3, respectively). Level 2 was judged to be the most important, Level 3 was next, and Level 1 was the least important. For the Trick text, Levels 1 and 2 were rated as more important than Level 3 (Ms = 3.92, 3.94, 3.06 for Levels 1, 2, and 3, respectively). For Ohmmeter, the levels did not differ (Ms = 3.87, 4.10, 4.02 for Levels 1, 2, and 3, respectively). Reading rate. There was a main effect of text, F(2 , 63) = 3.31. As expected, the Trick (M = 212.48) and Ohmmeter (M = 227.56) texts were read more slowly than was the Spool (M = 256.49) text. The interactions of chain and text, and active and text were not significant. There was a Level × Text interaction, F(2, 60) = 3.07, with no differences between the levels for Spool and Ohmmeter, but with Level 3 being read faster than Levels 1 and 2 for Trick (Level 1, M = 186.09; Level 2, M = 206.10; Level 3, M = 262.56). True–False. The only text difference found for true−false was an Active × Text interaction, F(2, 58) = 5.13. Newman-Keuls tests showed that active items were answered correctly more often than were nonactive items for the Spool text (Ms = .97 and .76, respectively). For the Trick text, active items were answered correctly less often than were nonactive items (Ms = .79 and .91, respectively). Problem Type When there was a problem with a sentence (i.e., when readers rated it as something other than easy to understand), they used the x category (standing for “it is something about the way the sentence is written”—a grammatical problem)

Diehl & Bergfeld Mills

29

most frequently (65.81%), followed by the z category (i.e., “I don’t have the background knowledge needed to understand this sentence,” 23.87%). The y category (standing for “the sentence doesn’t seem to fit in with or follow from the other sentences”—an order problem) was used least frequently (10.32%). We predicted that there would be more problems with the sentences that were on the chain, that were active, and that were on Level 2. The proportion of problems for sentences on the chain (M = .11, SD = .06) was significantly higher than for those off the chain (M = .07, SD = .05), t(67) = 2.42. Although the means were in the predicted direction, there was no significant difference between active (M = .10, SD = .07) and static (M = .08, SD = .05) sentences. There was no difference between the proportion of problems across the levels (Level 1, M = .09, SD = .07; Level 2, M = .10, SD = .07; and Level 3, M = .08, SD = .04). Discussion The present study investigated three structural variables that readers might use to construct their mental representation of the text. The experiment showed that, as predicted, for most measures, two of these variables (chain and active) influenced the reading and rating of procedural texts in ways similar to narratives, and one variable (level) in a dissimilar way. In previous research (Trabasso & van den Broek, 1985), sentences on the causal chain (those that form a continuous flow of cause and effect in the text) were shown to increase text coherence. In the present research, whether information was on or off the causal chain was related to perceptions and performance: Sentences on the chain were read more slowly than were those off the chain, and they were judged to be more difficult, to cause more problems, and to be more important. Active information (information required for task completion) has been shown to increase memory for narratives (Graesser et al., 1978) and, when the readers have a task orientation, to improve memory for procedural texts (Mills et al., 1995). In the present study, sentences that were active were judged to be more important and were read more slowly than nonactive sentences. However, no differences were found on true−false performance. The participants saw the device before reading, and they were told that they might be asked to do the task at the end of the study. These steps were taken so that they would read carefully and develop their situational representation. Perhaps this method of instilling a task orientation is not enough to show an effect on memory performance. However, participants in the study by Mills et al. also had a fairly weak task orientation. In that study, participants were simply told to read for the purpose of doing the task, yet they were able to demonstrate a relationship between active sentences and memory performance. Perhaps if readers are given a purpose for reading it instills a task orientation more strongly than does the suggestion that they might have to do the task later because the instruction is more direct and clear.

30

The Journal of General Psychology

Level in the hierarchy had a result unlike that found for narratives. Instead of the highest level leading to the highest importance ratings, sentences at the second level were judged to be the most important (as predicted). This may be because most of the active sentences (which by definition were required for task performance) were at that level. The reading-time data were less clear-cut. With narratives, sentences at the highest level were read most slowly, whereas in the present study the highest level was slower than the lowest level for the first reading, but the middle level was slower than the lowest for the second reading. It may be that the level variable does not affect reading rate until the second reading. However, this is not an adequate explanation because there was an (unexpected) effect on the first reading and the other structural variables showed main effects (not interactions with trial) on reading rate. Another relationship that has been shown with level is that high-level sentences tend to be remembered better (Cirilo & Foss, 1980). In the present study, there was no significant difference in memory across the levels; however, the difference between Levels 1 and 2 compared with Level 3 was over 10% and had a probability close to alpha. This pattern is consistent with the reading-time data in the sense that the lowest level was read fastest and remembered most poorly. We conclude that the relationship between level and performance on a comprehension test warrants further investigation. It is important to note that the three structural variables that were investigated in this study, though by no means the same, are not as distinct as we thought. Future studies could systematically manipulate these variables to render them at least less dependent than they are in naturally occurring texts. Another interesting finding was that the magnitude of the effects depended on the particular text. The three texts used in this study differed a great deal in the kind of information that they contained (e.g., Ohmmeter contained a great deal of technical information, whereas Spool had low-level background information and explanations). These different texts showed differences in the participants’ reading speeds and judgments. In general, the chain, active, and level effects showed decreasing strength in their effects on the different outcome variables moving from Spool (strongest) to Ohmmeter (weakest or nonexistent). These results make sense in the light of other findings. One is that Ohmmeter was judged to be more difficult overall than were Spool and Trick. The second is that Spool was read more quickly than Ohmmeter. The predictor variables in the present study are hypothesized to be the ways the reader “analyzes” or “makes sense of” what he or she is reading and to guide the reader’s construction of a mental representation. If a text is more difficult, the reader will read it more slowly, but may also “represent” it incorrectly if it cannot be understood. Therefore the structural variables examined in the present study may predict less well when texts are hard to understand. Additionally, when a reader has a task orientation (as in the present study), the reader may pay more attention to the parts of the text that have to do with getting the task done. When the text is almost exclusively about get-

Diehl & Bergfeld Mills

31

ting the task done (as was the case with the Ohmmeter text), the reader may not be able to use the structural variables as a means of determining the cognitive requirements of the various sentences in the text. We expected to find the predicted relationships between the structural and outcome variables on all measures. In the present study, readers made their ratings after reading through the texts twice. After two readings, the readers’ mental model of the text was better developed than it would have been if they had made the ratings as they were reading the text for the first time. This, and the fact that the texts were presented one sentence at a time, may have lessened the relationship between some of the structural variables and the ratings and, perhaps, the accuracy on the true−false test. The lack of predicted relationships on the true−false test may be related to a speed-versus-accuracy tradeoff. Because readers spent more time on sentences that were perceived to be more difficult (e.g., those on the chain or the active ones), they may have compensated for their difficulty and therefore performed just as well as on easier items on which they spent less time. Thus, a true−false test may not be a sensitive measure of text or task memory. The average percent correct on this comprehension test was 83. This finding suggests that the test may have been too easy (perhaps because the texts were read more than once). A better approach might have been to increase the retention interval or to have a multiple choice test with four or five options rather than a true− false test. One implication of this research is that readers do perceive structural variables in procedural texts, whether consciously or unconsciously. People who write instructions (such as teachers and technical writers) should be aware that readers with a task orientation (which occurs when readers read with the intent to do the task) pay more attention to information that is directly related to task completion. They slow down their reading rates and judge these sentences to be more important. As a result, procedural text writers need to use other means, such as bold print or indentation, to ensure that their readers pay attention to those parts of the text that are important but that are not task steps. A second implication for writing procedural text has to do with writing difficult or complex procedures. On the basis of the present research, it appears that, when the text is difficult to understand, readers cannot or do not use the structural variables in the text as a guide to figuring out what is important. The relationships between the structural variables and the outcome variables depended on the difficulty of the text, even though the texts used in the present study did not differ in the proportion of sentences that had each level of the structural variables (i.e., no chi-squared values were significant). The most difficult text showed no relationship between these variables, whereas the two easier texts did. This finding suggests that procedural text writers would do well to make their texts as easy to understand as possible, so that the reader can use the structural variables to the fullest extent.

32

The Journal of General Psychology

REFERENCES Cirilo, R. & Foss, D. (1980). Text structure and reading time for sentences. Journal of Verbal Learning and Verbal Behavior, 19, 96−109. Cohen, J. D., MacWhinney, B., Flatt, M., & Provost, J. (1993). PsyScope: A new graphic interactive environment for designing psychology experiments. Behavioral Research Methods, Instruments, and Computers, 25(2), 257−271. Diehl, V. (2001). Effects of a task orientation on mental representations of procedural texts. Manuscript in preparation. Diehl, V., & Mills, C. (1995). The effects of interaction with the device described by procedural text on recall, true/false, and task performance. Memory & Cognition, 23, 675− 688. Graesser, A., Higgenbotham, M., Robertson, S., & Smith, W. (1978). A natural inquiry into the National Inquirer: Self-induced vs. task-induced reading comprehension. Discourse Processes, 1, 355−372. Hay, H. (1975). Learn magic: The magician’s basic tricks explained step-by-step in 20 simple lessons (2nd ed.). New York: Dover. Herbert, D. (1980). Mr. Wizard’s supermarket science. New York: Random House. Mannes, S. M., & Kintsch, W. (1987). Knowledge organization and text organization. Cognition and Instruction, 4, 91−115. Mills, C., Diehl, V., Birkmire, D., & Mou, L. C. (1993). Procedural text: Predictions of importance ratings and recall by models of reading comprehension. Discourse Processes, 16, 279−316. Mills, C. B., Diehl, V., Birkmire, D., & Mou, L. C. (1995). Reading procedural text: Effects of purpose and predictions of reading comprehension models. Discourse Processes, 20, 79−107. Perrig, W., & Kintsch, W. (1985). Propositional and situational representations of text. Journal of Memory and Language, 24, 503−518. Trabasso, T., & van den Broek, P. W. (1985). Causal thinking and the representation of narrative events. Journal of Memory and Language, 24, 612−630. van Dijk, T., & Kintsch, W. (1983). Strategies of discourse comprehension. New York: Academic Press. APPENDIX Experimental Texts and Structural Variable Designations for Each Sentence Spool text On causal chain, static, Level 1

On causal chain, static, Level 1 On causal chain, active, Level 2 On causal chain, active, Level 2 On causal chain, active, Level 2 Not on chain, static, Level 3

A very long time ago someone figured out how to wind up a rubber band inside a spool and then make the rubber band unwind slowly to make the spool crawl across the floor. A modern version uses a spool of any size. Slip a rubber band, which is about the same length as the spool, through the opening, so that it passes from one end of the spool to the other. Anchor one end of the rubber band with a tack to the end of the spool. Slip the other end of the rubber band through the hole in a metal washer. The washer creates enough friction to keep the rubber band from unwinding quickly and yet is slip-

Diehl & Bergfeld Mills

Not on chain, static, Level 3 Not on chain, static, Level 3 On causal chain, active, Level 2 Not on chain, static, Level 3 On causal chain, active, Level 2 On causal chain, active, Level 2 On causal chain, active, Level 2 On causal chain, static, Level 3 On causal chain, static, Level 3 On causal chain, static, Level 3

Not on chain, static, Level 1 Ohmmeter text On causal chain, static, Level 1 On causal chain, static, Level 2 On causal chain, active, Level 2 Not on chain, static, Level 3 Not on chain, static, Level 3 On causal chain, active, Level 2 On causal chain, active, Level 2 On causal chain, active, Level 1 On causal chain, static, Level 2

Not on chain, static, Level 3 Not on chain, static, Level 2

33

pery enough to allow the wheel to turn slowly. That’s why the washer is called a slipper. If you don’t have a washer, you can make a slipper from the plastic of a refrigerator container or coffee-can top. Into the end of the rubber band that is through the slipper, put the last part—the drag. It can be a match stick, knitting needle, pencil, and so on. If the rubber band is not taut, wind it several times around the drag to take up the slack. To energize your spool vehicle, turn the drag to wind up the rubber band inside the spool. Set the spool down on the table. The drag keeps its end of the rubber band from turning. The twisting action of the rubber band is transferred via the anchor to the spool. With the right amount of friction from the slipper, the spool continues across the table until most of the energy you wound into the rubber band is released. You can have races and battles if your friends also make spool vehicles. Your Electronic Project Kit requires two 1.5-volt AA penlight cells and a 9-volt battery. You can use any standard penlight cells for the 1.5volt cells and a transistor radio battery for the 9volt battery. Install the penlight cells in the holder next to terminals 122 and 123. Be sure to install the cells according to the markings inside the holder. The end of the cell with the small metal cap is plus (+) and the flat metal end of the cell is minus (–). Install the 9-volt battery by first snapping the battery clip to its terminals. Then press the battery into its plastic clip. To connect a wire to a spring terminal, just bend the spring over to one side and insert the wire into the opening. Sometimes 2 or 3 hookup wires are connected to a single spring terminal so make sure the first wire doesn’t come loose when the second and third wires are installed. The easiest way to do this is to push the spring on the side opposite where the first wire has already been inserted. Be sure that only the exposed, shiny part of the hookup wire is inserted into the spring terminal.

34

The Journal of General Psychology

Not on chain, static, Level 3 On causal chain, active, Level 2

On causal chain, active, Level 2 On causal chain, static, Level 3 On causal chain, static, Level 1 Not on chain, active, Level 2 Not on chain, active, Level 2 On causal chain, active, Level 2 Not on chain, active, Level 3 Not on chain, active, Level 3 On causal chain, active, Level 2

On causal chain, static, Level 1 On causal chain, active, Level 1 Trick Text On causal chain, static, Level 1 On causal chain, static, Level 1 On causal chain, active, Level 2 On causal chain, active, Level 2 On causal chain, active, Level 2 On causal chain, active, Level 2 Not on chain, static, Level 3 On causal chain, active, Level 2 On causal chain, active, Level 2 On causal chain, active, Level 2 On causal chain, active, Level 2 On causal chain, active, Level 2

If the plastic insulation part of the wire is inserted into the spring, electrical contact will not be made. Connect a wire from Level 17 to Level 38; connect a wire from Level 16 to Level 10; connect a wire from 111 to 120; connect a wire from 121 to 8; and connect a wire from 37 to 9. Connect another wire to 8 and a different wire to 9. These latter wires will serve as probes. Ohmmeter operation is as follows: 1. Turn ohmmeter ON with the slide switch. 2. Adjust 50K control for a scale reading of Level 10 on the top (blue) scale. 3. Place the probes across the resistance to be measured. Make sure the circuit under measurement has NO voltage present or this meter may be damaged. Also make sure no parallel resistances are present (including your hands). 4. Determine the resistance from the formula R = 650 I over Level 10 - I ohms, where R is the resistance being measured, and I is the top (blue) meter calibration with R in the circuit. The main thing to always remember is to NEVER place the ohmmeter probes across voltage (including charged capacitors). Measure such things as the relay winding, transformer windings and low-value resistors. The torn and restored paper ribbon trick is done in various forms, with everything from a cigarette paper to a newspaper. The thumb tip is just about big enough to take the paper ribbon, so we shall use the thumb tip. Roll one piece of ribbon up quite tight and jam it into the thumb tip. Put the tip in its usual spot-—your lower right vest pocket. To start the trick, get the tip on your thumb. Hold up the ribbon at shoulder height, with a hand at each end, thumbs behind, fingers going over the top. You notice that the thumb tip is kept strictly out of sight all the time. Swing once from side to side so that everyone can see. Then shift both hands to the middle of the ribbon, still keeping the same grip. Tear the ribbon in half. Hold your hands up and apart, with the two pieces of ribbon hanging down. Lay one piece over the other, and tear both in half.

Diehl & Bergfeld Mills

Not on chain, active, Level 2 Not on chain, static, Level 3 On causal chain, active, Level 2 On causal chain, static, Level 3 Not on chain, static, Level 3 Not on chain, static, Level 3 On causal chain, static, Level 3 On causal chain, static, Level 3 On causal chain, active, Level 2 On causal chain, static, Level 2 On causal chain, active, Level 2 On causal chain, active, Level 2 On causal chain, active, Level 2

Not on chain, static, Level 3 Not on chain, static, Level 3 Not on chain, active, Level 2

35

Separate your hands again. You don’t want people to say afterward, “Oh, he never tore it at all.” Keep on tearing the halves, and laying them together, until the pieces are nearly square. You must be neat, but not fussy about it. In fact, the whole trick depends on neatness rather than on anything you could call dexterity. If you are messy about handling the scraps, they may not fit into the thumb tip. So long as you are neat, you won’t need any special skill. Finally, you have a small square stack of paper scraps, held in both hands, thumbs at the back, first and second fingers in front. Catch the thumb tip between your left thumb and fingers, and withdraw your right thumb from it. Now you can fish out the second strip with your right forefinger. Do that, and then use your right thumb to poke the torn pieces into the thumb tip. Let go of the thumb tip with your left hand, putting it back in place on your right thumb. You are now ready to start unrolling the restored strip, which you should do in as nearly as one movement as you can, so that people won’t think it is being unrolled, but just pulled out. For that reason, you may prefer one of two other ways to prepare the second strip. You can roll it up from both ends toward the middle; or you can fold it zigzag, in accordion pleats. Experiment until you find the best size of strip and the best way of folding, so that you can finish by coming up with a single sweep of the hands and a big smile.

Note. The structural variables are explained in the text. Causal chain variable: On causal chain or Not on chain. Type of sentence: active or static. Level in the hierarchy: Level 1, Level 2, or Level 3.

Manuscript received October 26, 2000 Revision accepted for publication April 12, 2001