Primary school students' learning experiences of, and

Learning and Individual Differences 28 (2013) 54–65

Contents lists available at ScienceDirect

Learning and Individual Differences journal homepage: www.elsevier.com/locate/lindif

Primary school students' learning experiences of, and self-beliefs about competence, effort, and difficulty: Random effects models Lars-Erik Malmberg a,⁎, Theodore A. Walls b, Andrew J. Martin c, Todd D. Little d, Wee H.T. Lim a a

University of Oxford, UK University of Rhode Island, USA University of Sydney, Australia d Texas Tech University, USA b c

a r t i c l e

i n f o

Article history: Received 26 April 2011 Received in revised form 24 June 2013 Accepted 5 September 2013 Keywords: Ability Effort Difficulty Intensive longitudinal data Multilevel structural equation model

a b s t r a c t Expanding research on individual differences in students' self-beliefs about ability, effort and difficulty, we investigated the variability and interrelatedness of situation-specific learning experiences of competence evaluation, effort exertion and task difficulty during one week at school. In total, 292 students in years 5 and 6 (Mage 10.5 years) filled in electronic questionnaires during 15.3 learning episodes on average during one week (SD = 4.3; Range = 2–34, Total nexperiences = 4,566). Students' learning experiences varied substantively across situations (rICC from .21 to .28), and were differentially interrelated between students (rSD from .28 to .40; random slope SDs .14 to .20). Using multilevel structural equation models (MSEM), we found that students who on average, across situations, evaluated their competence higher exerted less effort in situations and evaluated their competence higher at difficult tasks. Higher performers exerted more effort at difficult tasks, girls exerted more effort than boys for the same level of competence evaluation, and students who in general found school difficult evaluated their competence higher at easier tasks. The investigation of situation-specific learning experiences provides insights into student belief systems in educational contexts which complement our knowledge of individual difference in such beliefs. © 2013 Elsevier Inc. All rights reserved.

1. Introduction Research on students' self-related beliefs about their competence (i.e., ability) and effort (e.g., effort exertion, effort regulation) in relation to their school performance is central in several fields of educational psychology (e.g., Covington & Omelich, 1979a, 1979b; Little, 1998; Skinner, Chapman, & Baltes, 1988). In many theoretical conceptualisations, ability and effort beliefs have been related to perceived difficulty or demand levels (Heider, 1958; Malmberg & Little, 2007; Malmberg, Wanner, & Little, 2008; Nicholls, 1984). Relations between these beliefs have generally converged around the following patterns: A higher level of ability allows the person to exert less effort to be successful given a certain difficulty level of a task; More effort needs to be exerted to compensate for a lower level of ability, particularly when attempting to solve a difficult task; Effort exertion and time spent on a task provide feedback to the individual on their ability, which in the longer term forms a base for attempting or withdrawing from subsequent tasks and challenges (Nicholls, 1984; Nicholls & Miller, 1984). Students' self-related beliefs about ability, effort and difficulty vary by school subject (Malmberg,

⁎ Corresponding author at: Department of Education, University of Oxford, 15, Norham Gardens, OX2 6PY Oxford, UK. E-mail address: [email protected] (L.-E. Malmberg). 1041-6080/$ – see front matter © 2013 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.lindif.2013.09.007

Hall, & Martin, 2013), school-type (Malmberg et al., 2008), country (Stetsenko, Little, Gordeeva, Grasshof, & Oettingen, 2000), and change over time (Little, Stetsenko, & Maier, 1999). These beliefs are susceptible to performance feedback in the classroom (Hattie & Timperley, 2007), and in experimental conditions (Mueller & Dweck, 1998). There are only few studies of situational experiences of such beliefs. These are the diary studies of Schmitz and Skinner (1993), Musher-Eizenman, Nesselroade, and Schmitz (2002), Schmitz and Wiese (2006), and Tsai, Kunter, Lüdtke, and Trautwein (2008), and the electronic diary study of Tolvanen et al. (2011). We contribute to the literature on children's self-beliefs in ability, effort and difficulty by focusing on the situationspecific learning experiences of competence evaluation (i.e., task success and understanding), effort exertion, and task difficulty. In the present study, we go beyond previous cross-sectional, longer-term longitudinal and diary studies in three ways. First, we collected situation-specific learning experiences over repeated learning episodes during one week at school using Personal Digital Assistants, PDAs (Malmberg, Woolgar, & Martin, in press). Second, we investigated the variability of, and interrelations between learning experiences. Third, we investigated whether person characteristics (age, gender, and school performance) and selfbeliefs (agency beliefs in ability, effort, and perceived difficulty) predicted and moderated learning experiences. We specified state-of-the art random slope models in Multilevel Structural Equation Models (MSEM).

L.-E. Malmberg et al. / Learning and Individual Differences 28 (2013) 54–65

The time-perspective within which we investigated situation-specific learning experiences is posited between diary studies (e.g., one report at each math or language class for up to 30 lessons; Schmitz & Skinner, 1993) and micro-analytic studies (e.g., trace data from computer environments; Azevedo, Moos, Johnson, & Chauncey, 2010). The time-frame of all learning experiences during one week at school was chosen as students were expected to experience a reasonable number of different school subjects, a range of tasks and projects within each subject, and interact with a broad range of peers and teachers. On this basis we expected to yield samples of experiences of each student, with plausible variability (“ups and downs”), stationarity (“stable flow”) and differential interrelatedness (“different reactions to various challenges”) of their learning experiences. We hope that insights into the microcosm of learning experiences can contribute to our understanding of learning as a process (Hattie, 2008; Schmitz, 2006), allowing for a unique window into student belief systems and intrapersonal dynamics (e.g., Molenaar, Huizenga, & Nesselroade, 2003; Nesselroade, 2001). 1.1. Ability, effort and difficulty In models of action-control (Kuhl, 1985; Kuhl & Goeshke, 1985; Little, 1998) and perceived control (Skinner, 1996; Skinner, ZimmerGembeck, & Connell, 1998), actions are considered volitional, goaloriented and self-regulatory. Such a perspective of human agency is rooted in action-theoretical approaches to human development (Brandtstädter, 1998; Heider, 1958) and motivation (Heckhausen & Heckhausen, 2008). While means–ends beliefs and strategy beliefs refer to the perceived determinants of performance (e.g., ability, effort, luck), agency-, capacity-, and self-efficacy beliefs are defined as an individual's perception of possessing the skills, resources and abilities required for realizing a certain goal (Little, 1998; Skinner et al., 1998). Students' agency beliefs in ability and effort are consistently the strongest predictors of academic success (e.g., Little, Oettingen, Stetsenko, & Baltes, 1995), and differ from causal attributions to effort and ability (Schmitz & Skinner, 1993). A personal sense of agency is formed in early childhood through mastery experiences and gradual realization that outcomes are contingent on one's own actions (Skinner, 1986). In the school year sequences of mastery experiences and past performance form a base for concurrent self-evaluations (Bong & Skaalvik, 2003). This enables a gradual increase in volition and self-regulation of behavior and cognition, particularly through the volitional exertion of effort (Kochanska, Murray, & Harlan, 2000; Kuhl, 1985). Viewing effort as a limited resource of energy within the individual, effort exertion renders the self (i.e., ego) to be depleted after energizing volition (Baumeister, Bratslavsky, Muraven, & Tice, 1998), depending on the level of challenge the individual is up against (Dermitzaki & Efklides, 2001; Heider, 1958; Malmberg & Little, 2007; Malmberg et al., 2008; Schmitz & Skinner, 1993). Children learn to discern their level of ability as a function of how much effort or time they exert, in relation to how difficult a task is (Nicholls, 1984). Effort can be gauged by time spent on a task and level of exhaustion upon completion. As effort exertion is considered to be the behavior most controllable by the self (Kuhl, 1985; Pintrich, 2000; Schmitz & Skinner, 1993), it can be regarded as a “double-edged” sword, as students who fail after exerting effort have been shown to attribute their failure to inability (e.g., Heider, 1958), while those who fail after exerting less effort are less likely to do so (Covington & Omelich, 1979a, 1979b). Children's attributions have also been found to be susceptible to whether they receive feedback that attributes success to their effort or their ability. For example Mueller and Dweck (1998) found that children who received initelligence praise instead of effort praise after failure displayed less task persistence, less enjoyment, and made more low-ability attributions. Towards the end of primary school children learn to differentiate between how able and effortful they are, and how difficult or challenging they find school (Malmberg & Little, 2007). Such differences are also

55

evident among youth in different secondary school tracks (Malmberg et al., 2008). Students' use absolute (e.g., performance referenced), normative (e.g., teachers, parents, peers, or society expectations), or relative comparisons (i.e., comparison of outcomes with others; Nicholls, 1984) for arriving at conclusions about the level of difficulty of a task. From these comparison processes they gradually become aware that not everyone can be “the best” (Nicholls & Miller, 1984). The difficulty of a task is a crucial indicator of how much effort might be needed for successful completion. Students' ability to regulate their overt behavior, that is to exert more effort when they are confronted with an optimally challenging or difficult task, is a key feature of a mastery approach (Pintrich, 2000). Likewise, withdrawal of effort in a challenging task constitutes a helpless pattern (Kuhl, 1985; Pintrich, 2000). 1.2. Situation-specific learning experiences Although much knowledge about students' personal beliefs about their ability, effort, and perceived difficulty has been accumulated, there are to date only few studies of students' situation-specific competence, effort, and difficulty perceptions, namely the diary study of Schmitz and Skinner (1993) and Musher-Eizenman et al. (2002). In their diary study (Schmitz & Skinner, 1993) students reported on average two assignments or tasks in each subject per week, on more than 25 occasions. Students reported on interpersonal measures of control, situational measures of subjective time use effort exertion, subjective performance evaluation and subjective estimations of task difficulty (prior to assignment was graded) attributions in the case of both success and failure. They found larger intrapersonal variation (i.e., pooled differences within individuals) in competence-related beliefs (i.e., daily reports of maths and language related effort, performance, attributions and expected control) than interpersonal variation (i.e., differences between individuals), showing that students varied more within themselves than between each other. Using correlational, lagged, and multivariate time-series analyses of intrapersonal and interpersonal beliefs and perceptions (i.e., aggregated reports during the diary days), the authors concluded that intrapersonal perceptions and beliefs have functional relationships different from interpersonal perceptions and beliefs. Using Dynamic Factor Analysis in a pooled sample analysis (subsamples of the Schmitz & Skinner, 1993 study), Musher-Eizenman et al. (2002) found stronger concurrent associations between constructs and a greater stability in perceived control and task demands from one day to the next for higher achieving students. Neither of these two studies allowed for individual differences in direction and magnitude in associations between variables (e.g., associations between two variables can vary from positive to negative and weak to strong). Such individual differences in associations can be investigated using random slope models. 1.3. Self-beliefs and learning experiences in the Learning Every Lesson (LEL) study In their overview of definitions and measurements of self-concept and self-efficacy, Bong and Skaalvik (2003) suggested that self-beliefs such as self-concepts and self-efficacy are relatively stable over time, while situation-specific cognitions and perceptions are more malleable and susceptible to contextual circumstances. Models of action-control (e.g., Kuhl & Goeshke, 1985) and self-regulation (e.g., Boekaerts & Corno, 2005; Pintrich, 2000) pose that students with high levels of control, agency and self-beliefs can in situations draw on their skills, resources and abilities when implementing goal-directed actions in real time. At the situation-specific learning experience we conceptualized competence evaluation to be a situation-specific “equivalent” of action control beliefs of ability, effort exertion a situation-specific “equivalent” of agency belief in effort, and task difficulty a situation-specific equivalent of perceived difficulty. Empirical findings support this model. For example, Schmitz and Skinner (1993) found students with higher

56


control (self-) beliefs exerted themselves more while working on math assignments and needed less time to complete assignments (p. 1017). Students' ability to regulate their overt behavior, that is to exert more effort when they are confronted with an optimally challenging or difficult task, is a key feature of a mastery approach (Pintrich, 2000). Likewise, withdrawal of effort in a challenging task constitutes a helpless pattern (Kuhl, 1985; Pintrich, 2000). Following our theoretical model of self-beliefs regulating situationspecific experiences, the Learning Every Lesson (LEL) study was designed to collect both self-beliefs (cross-sectional questionnaire) and situationspecific learning experiences. In order to capture the real-world realtime aspects of these situation-specific learning experiences, we used Personal Digital Assistants (PDAs) for registering experiences across all “naturally” occurring school subjects and learning episodes. Such momentary assessments accessed respondents' feelings and experiences at a fixed time-point shortly thereafter, which reduces recall bias (Beal & Weiss, 2003; Mehl & Conner, 2012; Stone & Shiffman, 2002). The rise of affordable technology such as handheld computers (Beal & Weiss, 2003; Mehl & Conner, 2012) and accessible methodology for intensive longitudinal data (Walls & Schafer, 2006), enables us to investigate school children's situation-specific learning experiences (beliefs, perceptions, and behaviors), as they unfold in naturalistic settings in real-time. The situation-specific and self-belief data enable us to analyze our data using three types of constructs: self-beliefs based on the crosssectional questionnaire, situation specific learning experiences based on the PDA questionnaire reports, and a third construct, the aggregated learning experiences of each student during the week (i.e., the average situation-specific belief across situations). 2. Research questions and hypotheses Our research questions are: (1) To what extent are situation-specific learning experiences, situation-average and self-beliefs of ability, effort and difficulty related? Hypothesis 1. We expected agency belief in ability to be strongly positively related with agency belief in effort, and strongly negatively related with perceived difficulty. Agency belief in effort was expected to be weakly related with personal difficulty (Little, Oettingen, Stetsenko, & Baltes, 1995; Malmberg & Little, 2007; Malmberg et al., 2008). As indicators of convergence between self-beliefs and situation-specific experiences, we observed relations between agency belief in ability and situation-average competence evaluation; agency belief in effort and situation-average effort exertion, and personal difficulty and situationaverage task difficulty. (2) Are there individual differences in the relationships between students' situation-specific learning experiences? Hypothesis 2. In line with Schmitz and Skinner (1993), we expected large individual differences in relationships between situation-specific learning experiences. As indicators of individual differences we observed descriptives of within-person variability and random effects. (3) Does task difficulty differentially predict effort exertion? Hypothesis 3. As students have been found to either increase or decrease effort when solving difficult tasks (Kuhl & Goeshke, 1985; Pintrich, 2000) we expected individual differences in the relation between effort exertion and task difficulty. (4) Do personal characteristics (age, gender and academic performance) and self-beliefs (agency belief in ability, effort and perceived difficulty) predict effort exertion, and moderate the relationship between task difficulty and effort exertion?

Hypothesis 4. We expected positive individual characteristics and selfbeliefs (e.g., high performer, being smart, effortful, and finding school easy) would exert more effort (fixed effect), and also exert more effort the more difficult the task is (moderation effect; Kuhl & Goeshke, 1985; Pintrich, 2000). (5) Do effort exertion and task difficulty differentially predict competence evaluation? Hypothesis 5. We expected higher situation-specific effort exertion to predict higher, and task difficulty to predict a lower evaluation of competence. We also expected a positive interaction effect, i.e., the more effort exerted at a difficult task the higher the evaluation of competence (Schmitz & Skinner, 1993). (6) Do personal characteristics and beliefs predict competence evaluation and moderate the relationship between task success and task difficulty, and between task success and effort exertion? Hypothesis 6. Students with high self-beliefs were expected to evaluate their competence higher the more effort they exerted at more difficult tasks (Boekaerts & Corno, 2005). 3. Method 3.1. Sample and procedure In total 353 students in 16 classrooms in 11 schools received signed parental or guardian consent for participating in the Learning Every Lesson (LEL) study (roughly 90% participation rate). The sample was taken from two local education authorities in southwest England. Both were quite diverse regarding the distribution of poverty and ethnicity. During an initial classroom visit students completed a paper-andpencil questionnaire and were taught how to use a handheld computer (Personal Digital Assistant, PDA) to complete an electronic questionnaire after completing each learning task (i.e., an event triggered response), or at least once every lesson (i.e., a fixed response) for one week's time. Students were instructed to identify learning episodes. They were given examples of one lesson split into two learning experiences (e.g., individual guitar tutorial at the beginning of a lesson and then returning to do mathematics), and two lessons being treated as one learning episode (e.g., when students had a double lesson in outdoor sports). Researchers monitored compliance of the participants and performance of the technology during the data collection period. After collecting the PDAs, data was uploaded and error-checked. The study started on Mondays with pick-ups on Friday afternoon, or on Tuesdays with pick-ups the following Monday afternoon. In some cases (e.g., a full day trips or other event that only offered one learning experience a day) students completed the PDA questionnaires an additional day and the corresponding pick-up was delayed by a day. Of the 5025 lesson responses, 45 (0.1%) responded to b6 of the 19 items and were removed. Due to a data collection mishap, key variables were not collected in two classrooms, leaving 4566 lesson responses (Mlessons = 15.3; SD = 4.3; Range = 2 to 34) from 298 students. Of these 104 (34.9%) were in Year 5 and 194 (65.1%) in Year 6. There were 135 (45.3%) boys and 163 (54.7%) girls. Their mean age was 10.5 years (SD = 0.63). 3.2. Measures 3.2.1. Situation-specific learning experiences In the electronic questionnaire students were asked which subject and what task they had been working on (Walls, 2006). Thereafter, they responded to four items about their beliefs and perceptions about working on the task: “how well were you doing at this task” on


a five-point scale (1 = poorly, 5 = very well), “how much did you understand” on a four-point scale (1 = all of it, 4 = none of it; reverse-coded), “how difficult was the task” on a four-point scale (1 = very easy, 4 = very difficult), and “how much effort did you put in” (1 = none at all, 4 = a lot). We refer to these items as success evaluation, understanding, task difficulty and effort exertion. We tested whether a 2- or 3-factor solution fits better. In the 2factor model we specified task success, understanding, and task difficulty as indicators of the competence evaluation construct, and effort exertion as a one-item construct. In the 3-factor model we specified success evaluation and understanding as indicators of the competence evaluation construct, and kept effort exertion and task difficulty as one-item constructs. We first tested a model in which the factor loadings were allowed to vary across the two levels. Then we tested a model in which the factor loadings were equated across levels (Mehta & Neale, 2005). We finally tested the fit of partially saturated models proposed by Ryu and West (2009).1 Good model fit would be indicated by a value below .08 on the Root Mean Square Error of Approximation (RMSEA; and the partially saturated RMSEAPS_B for between, and RMSEAPS_W for within parts respectively), and Standardised Root Mean Square Residual (SRMRB for between and SRMRW for within parts respectively), and above .90 on the Comparative Fit Index (CFI, and the partially saturated CFIPS_B for between, and CFIPS_W for within parts respectively), and the Tucker–Lewis Index (TLI) (Browne & Cudeck, 1993). The fit indices for the 2-factor model in which factor loadings were freely estimated across the two levels were: (χ2[4] = 169.42; p b .001; CFI = .932; TLI = .797; RMSEA = .097; SRMRB = .040; SRMRW = .049), and for the model in which the factor loadings were equated across the levels: (χ2[6] = 161.37; p b .001; CFI = .936; TLI = .873; RMSEA = .075; SRMRB = .056; SRMRW = .039). The RMSEA fit of the partially saturated models for between-level model fit was CFIPS_B = .965 and RMSEAPS_B = .131, and within-level model fit was CFIPS_W = .960 and RMSEAPS_W = .127. The fit indices of the 3-factor model were: freely estimated factor loadings across levels: χ2[2] = 10.10; p b .01; CFI = .997; TLI = .980; RMSEA = .030, SRMRB = .010; SRMRW = .017; factor loadings equated across the within and between levels also fit data well (χ2[2] = 10.39; p b .05; CFI = .997; TLI = .988; RMSEA = .023, SRMRB = .011; SRMRW = .022). Between-level model fits were CFIPS_B = .997 and RMSEAPS_B = .057, and within-level model fit were CFIPS_W = .998 and RMSEAPS_W = .042. Overall the 3-factor solution fits better than a 2-factor MCFA. Consequently we proceeded with modeling three constructs: competence evaluation, task difficulty and effort exertion. Intra-class correlations (i.e., the proportion of variance found between students) of the items were rICC = .19 to .53, warranting multilevel analyses in two levels.

3.2.2. Self-beliefs We included three self-beliefs: agency in ability, agency in effort, and general perceived difficulty at school, modified from previous studies (Little, Oettingen, & Baltes, 1995; Malmberg et al., 2008). Students indicated to what extent they agreed with each statement (1 = strongly disagree, 2 = disagree, 3 = agree, 4 = strongly agree). Agency in ability was measured by three items (‘I'm quite clever’, ‘I'm bright’, ‘I'm pretty 1 Ryu and West (2009) suggest alternative ways to assess model fit of the within and the between-levels respectively, where h i h i

CFIPSXB ¼ 1−

MAX

χ 2hypB;satW −d f hypB;satW ;0

h

MAX

i,

χ 2indB;satW −d f indB;satW ;0

CFIPSXW ¼ 1−

χ 2PSXB −d f PSXB d f PSXB ð J Þ

MAX

χ 2satB;hypW −d f satB;hypW ;0

h

i

,

χ 2satB;indW −d f satB;indW ;0

rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi RMSEAPSXB ¼

MAX

RMSEAPSXW ¼

χ 2PSXW −d f PSXW

d f PSXW ðNÞ , and , where “PS” denotes partially saturated, “B” the between-level model and “W” the within-level model, “hyp” the hypothesized factor structure, “sat” a saturated model (i.e., all manifest indicators covary), and “ind” is the independence model (i.e., in which indicator covariances are fixed to zero).

57

smart’; α = .86), agency in effort by three items (‘In class I really pay attention’, ‘I really work hard in class’, ‘I put an effort in class’; α = .73), and perceived difficulty by three items (‘Learning something new is difficult for me’, ‘I think understanding new things in school is hard’, ‘I have trouble figuring out new lessons in school’; α = .70). A multilevel confirmatory factor analysis (MCFA) in which the indicators of three self-belief constructs were specified to load on their a priori constructs, fit data excellently (χ2[24] = 32.98; p = .104; CFI = .986; TLI = .977; RMSEA = .009; SRMRB = .040). All factor loadings were significant and all reliabilities (R2) were substantive. We also included three personal characteristics as covariates in the study, school performance, gender (0 = boy, 1 = girl) and age, and as situation-specific covariates we controlled for the school subject taught. 3.2.3. School performance We used students' teacher-reported curriculum levels in mathematics, English, and science. We coded the curriculum levels into integer values ranging from 1 to 5 (1a = 1.67, 2c = 2, 2b = 2.33, 2a = 2.67, 3c = 3, 3b = 3.33, 3a = 3.67, 4c = 4, 4b = 4.33, 4a = 4.67, 5c = 5). In England, these are absolute measures of performance in relation to curriculum standards, and thus provide an index of the extent to which students have successfully learned the course contents covered. Consequently, the 6th grade students had higher curriculum levels in mathematics (t[268] = −6.79; p b .001), English (t[278] = −6.52; p b .001), and science (t[268] = −8.67; p b .001). Internal consistency of the raw-score curriculum levels was α = .92. We used within-classroom standardized values (M = 0, SD = 1) of each curriculum level. The use of teacherreported curriculum levels seems reasonable as teacher awarded curriculum levels have been strongly correlated with national (UK) exam results in the 6th grade (r = .77 in English and r = .82 in math, K. Toth, personal communication, 18/06/2013), as is the case in the 9th grade (Sammons et al., 2011). An MCFA in which the indicators of self-belief constructs and academic performance were specified to load on their a priori constructs, including age and gender as covariates, fits data excellently (χ2[64] = 90.66; p = .016; CFI = .978; TLI = .969; RMSEA = .010; SRMRB = .040). All factor loadings were significant and all reliabilities (R2) were substantive. 3.3. Analytic strategies 3.3.1. Multilevel structural equation models We specified multilevel structural equation models (MSEM; Dyer, Hanges, & Hall, 2005; Muthén, 1991; Reise, Ventura, Nuechterlein, & Kim, 2005) in Mplus 7.0 (Muthén & Muthén, 2009), so that items load on their substantive constructs at the within-level (i.e., each learning experience (i)) and at the between-level (i.e., students (j)). Thus, all personal beliefs were specified at the between-level, while the situation-specific beliefs had both within- and between components. At the within-level the situation-specific beliefs represent learning episodes clustered in students, and at the between-level students' average competence evaluation, effort exertion, and task difficulty. We controlled for measurement error of the level-2 individual characteristics and self-belief constructs as these were specified as latent constructs (Marsh et al., 2009). To test the first research question, we inspected latent correlations between situation-specific learning experiences (Table 1). To answer the second research question, we examined intra-class correlations σ 2B ICC ¼ σ 2 þσ and descriptive statistics for person-level aggregate 2 B

W

correlations of situation-specific learning experiences (Table 2). To answer the subsequent research questions we then set up a series of multilevel structural equation models (MSEM), starting with variance component models as a baseline (effij = b0 + u0j + e0ij and compij = b0 + u0j + e0ij). We first specified effort exertion as the dependent variable (‘eff’) and task difficulty (‘diff’) as the predictor, as shown in Eq. (1). Task difficulty was centered within clusters (i.e., xij −x j ; Enders & Tofighi, 2007), in order to interpret the effect of deviations

58


Table 1 Latent correlations between self-beliefs, individual characteristics, situation-average and situation-specific learning experiences (Multilevel Structural Equation Model). 1.

2.

Personal beliefs 1. Agency: ability 2. Agency: effort 3. Perceived difficulty

3.

4.

5.

0.48*** −0.52***

−0.24**

Individual characteristics 4. School performance 5. Age 6. Gender

0.49*** 0.14* −0.07

0.21** −0.03 0.32***

−0.57*** −0.27*** 0.00

0.15* 0.02

Situation-average 7. Competence evaluation 8. Effort exertion 9. Task difficulty

0.49*** 0.16* −0.30***

0.40*** 0.50*** −0.27**

−0.45*** −0.03 0.21*

0.46*** 0.15* −0.20**

6.

7.

8.

−0.01 0.26*** 0.04

0.34*** −0.77***

−0.09

0.19*** −0.77***

0.00

0.12* −0.08 0.02 0.17**

Situation-specific 7. Competence evaluation 8. Effort exertion 9. Task difficulty Note: Estimates are from Mplus 7.0 (Muthén & Muthén, 2012). * = p ≤ .05, ** = p ≤ .01, *** = p ≤ .001.

from the mean of each individual, on deviations in effort exertion. Subscripts indicate level j being level 2 (student) and ij level 1 (learning experience), and ‘CWC’ to denote variables centered within cluster. eff ij ¼ b0 j þ b1 diff CWCij þ e0ij

ð1Þ

(‘ACAPERF’), and self-beliefs: agency-belief in ability (‘ABL’), agency-belief in effort (‘EFF’) and perceived difficulty (‘DIFF’) are indicated in upper case denoting that they are latent constructs, minimizing measurement error at the between level of the model.

with the level-2 equations

eff ij ¼ b0 þ b1 diff CWCij þ b2 AGEGMC j þ b3 GENDERGMC j þ b4 ACAPERFGMC j þ u0 þ u1 j diff CWCij þ e0ij

b0 j ¼ b0 þ u0 j

eff ij ¼ b0 þ b1 diff CWCij þ b2 AGEGMC j þ b3 GENDERGMC j þ b4 ACAPERFGMC j þb5 ABLGMC j þ b6 EFFGMC j þ b7 DIFFGMC j þ u0 j þ u1 j diff CWCij þ e0ij :

b1 j ¼ b1 þ u1 j and level-2 variance-covariance matrix

u0 j u1 j

"

¼

#

2

σ u0 σ u10

2

σ u1

:

We then estimated the effects of personal characteristics (Eq. (2)), and personal characteristics and personal beliefs (Eq. (3)). These level-2 variables were grand mean centered (i.e., x j −x) using ‘GMC’ for short. This enables interpretations of effects of individuals being above or below the grand average on the outcome. Personal characteristics: age (‘AGE’), gender (‘GENDER’), and academic school performance Table 2 Situation-specific learning experiences: Intraclass correlations and individual differences in within-student correlations.

rICC

Effort exertion

Task difficulty

Competence evaluation

0.28

0.21

0.22

ð3Þ

We then estimated whether the difficulty slope (u1jdiffCWCij) varied as a function of personal characteristics and personal beliefs (a ‘slopes as outcome’ model; Bryk & Raudenbush, 1992). We entered each predictor of the slopes one by one as to not overload the model. In the next step, we regressed competence (‘comp’) on effort exertion and task difficulty on (Eq. (4)), allowing the effects of effort exertion and task difficulty to vary across students. compij ¼ b0 þ b1 eff CWCij þ b2 diff CWCij þ u0 j þ u1 j eff CWCij þ u2 j diff CWCij þ e0ij

ð4Þ 3 2 2 u0 j σ u0 4 u1 j 5 ¼ 6 4 σ u01 u2 j σ

3

2

u02

2 σ u1

σ u12

7 5: 2 σ u2

Next we estimated fixed effects of personal characteristics and self-beliefs on competence evaluation (Eq. (5)). Random effects were specified as in Eq. (4). compij ¼ b0 þ b1 eff CWCij þ b2 diff CWCij þ b2 AGEGMC j þ b3 GENDERGMC j þ b4 ACAPERFGMC j

Task difficulty rj 0.00 (0.00) r jSD 0.39 (0.47) rjMin–Max −0.90 to 0.91 (−1.45 to 1.54) Competence evaluation rj 0.15 (0.19) r jSD 0.39 (0.49) rjMin–Max −0.86 to 0.95 (−1.30 to 1.88)

ð2Þ

þb5 ABLGMC j þ b6 EFFGMC j þ b7 DIFFGMC j þ u0 j þ u1 j eff CWCij þ u2 j diff CWCij þ e0ij :

ð5Þ

−0.59 (−0.80) 0.28 (0.47) −0.99 to 0.55 (−2.72 to 0.62)

Note: For descriptive purposes we, for each student, calculated intrapersonal correlations between the three variables. Based on the pooled aggregates we then calculated the average correlation across students r j , as well as individual differences in intrapersonal correlations (r jSD , rjMin–Max). We in each cell present the raw as well as the (in brackets) Fisher z-transformed correlations. Three extreme observations of r = −1.00 or 1.00 were excluded from calculations.

We then, using slopes as outcome models, estimated whether the effort (u1jeffCWCij) and difficulty slopes (u1jdiffCWCij) varied as a function of personal characteristics and personal beliefs. We entered each predictor of the slopes one by one as to not overload the model. All models were specified in Mplus 7.0 (Muthén & Muthén, 2012). 4. Results Our first research question was to investigate the extent to which situation-specific, situation-average and personal beliefs are related.


To do so, we observed latent correlations in a MCFA, in which we specified learning experiences at both the within and between levels, the within-level constructs representing the pooled withinstudent associations across learning experiences. At the between level, the situation-average learning experiences represent the average situation-specific learning experiences pooled across the students. The situation-average learning experiences were associated with the six latent constructs representing three self-beliefs (agency: ability, agency: effort, and perceived difficulty) and three individual characteristics (school performance, gender, and age). The model fit was excellent (χ2[111] = 208.39; p b .001; CFI = .981; TLI = .967; RMSEA = 0.014; SRMRW = 0.004; SRMRB = 0.028).2 All estimated factor loadings were significant and all reliabilities (R2) were substantive. At the within-level (Table 1, situation-specific part), we inspected the relations between situation-specific learning experiences. The positive moderate association between competence evaluation and effort exertion (ρ = .19) indicates that on average, across students, learning episodes in which more effort was exerted were associated with a higher competence evaluation (i.e., students felt more successful and understood more). The strong negative association between competence evaluation and task difficulty (ρ = −.77) indicates that on average across students, learning episodes in which a task was appraised as more difficult were associated with a lower competence evaluation (i.e., students feel less successful and understood less). The lack of association between situation-specific effort exertion and task difficulty (ρ = −.00) indicates that on average across students, learning episodes in which a task was appraised as more difficult were unrelated to the level of effort exertion. At the between-level (see Table 1, person-specific part) situationaverage learning experiences were as follows. On average, students who had a higher situation-average competence evaluation (i.e., a higher average across the week) exerted more effort (ρ = .34) and found tasks less difficult (ρ = −.77). A higher level of situation-average effort exertion was related to lower situation-average task difficulty (ρ = −.09). Confirming Hypothesis 1 about the relations between self-beliefs at the between-level of the MCFA, we found that agency belief in ability was positively related with agency beliefs in effort (ρ = .48), agency belief in ability was negatively related with perceived difficulty (ρ = −.52), and agency belief in effort was negatively related with perceived difficulty (ρ = −.24). Some instances of convergence between self-beliefs and their situation-average counterparts were found. As shown in Table 1 (see person-specific part): agency belief in ability was strongly correlated with situation-average competence evaluation (ρ = .49), agency belief in effort with situation-average effort exertion (ρ = .50), but perceived difficulty was moderately related with situation-average task difficulty (ρ = .23). The latter correlation was weaker than that between perceived difficulty and situation-average competence evaluation (ρ = −.47). Higher school performance was associated with higher agency belief in ability (ρ = .49), situation-average competence evaluation (ρ = .46), agency belief in effort (ρ = .21) situation-average effort exertion (ρ = .15), lower perceived difficulty (ρ = −.57) and lower situation-average difficulty (ρ = −.20). Personal difficulty was related to a lower agency belief in effort (ρ = −.24), but unrelated to situation-average effort exertion (ρ = −.03). Situation-average effort exertion was also unrelated to situation-average difficulty (ρ = −.09).

2

In a series of alternative models we included a set of dummy coded variables to control for the particular school subject that was taught each lesson (history, geography, art, English, science, physical education and other (e.g., guitar class, health care project)). Including this set of covariates in the MCFA shown in Table 1 these explained 4.0%, 1.5% and 4.2% of the variance of situation-specific competence evaluation, effort exertion and task difficulty respectively. No parameter estimates changed substantively by including this set of covariates into our other models (Models 1–3), whereby this set of covariates was dropped.

59

4.1. Individual differences in relationships between situation-specific learning experiences We confirmed Hypothesis 2, suggesting variability of, and individual differences in, the associations between students' situation-specific learning experiences. As shown in Table 2, slightly more than a fifth of the variance was between students for these three constructs (rICC from .21 to .28) warranting inspection of individual differences using aggregated within-person correlations (e.g., the correlation between effort exertion and task difficulty, for each participant). To facilitate interpretation we present both raw and Fisher z-standardized correlation coefficients. Excluding three cases with extreme correlations, the average within-person correlations ranged between r j = −0.99 and 0.95 (zr j = −2.72 and 1.88). In line with Hypothesis 2, the individual differences in within-person correlations between the three constructs were wide, from r jSD = .28 to .39 ( zr jSD = .47 to .49), warranting the use of random slope models. To illustrate the individual differences in within-person correlations, a student for whom effort exertion and task difficulty were negatively associated would indicate that he or she puts in less effort at difficult tasks. A student for whom effort exertion and task difficulty are positively associated would indicate that he or she puts in more effort at difficult tasks. 4.2. Effort exertion differentially predicted by difficulty In order to answer the third research question, whether task difficulty differentially predicted effort exertion, we regressed effort exertion on task difficulty, and estimated both fixed and random effects. As shown in Model 1 (see Table 3) we found a non-significant fixed effect of task difficulty (B = .016; p = n.s.; ES = .0513), indicating that the average student did not exert higher or lower effort when confronted with tasks of varying difficulty levels (i.e., above or below their personal average level of task difficulty). We found that students varied significantly with regard to their average level of effort exertion (σ2u0j=. 245; p b .001). Confirming Hypothesis 3, we found significant individual differences with regards to the random effort exertion on task difficulty slope (σ2u1j=. 068; p b .001) showing individual differences between students, individual slopes between B = −.506 to B = .536, explaining 9.9% of the level-1 variance4: some students exerted less and others exerted more effort when confronted with tasks above their personal average level of difficulty. The effort exertion by task difficulty covariance was negative (σu10=−. 034; p b .05) indicating that a higher average effort exertion was related to a less steep slope (r = −0.36). This means that students, who were more effortful across the situations, exerted relatively less effort when confronted with a more difficult task. Alternatively, students who were less effortful across situations exerted relatively more effort when confronted with a more difficult task. In the next step (Model 2, Table 3) girls were more effortful than boys (B = .257; ES = .114) and higher achievers were more effortful than lower achievers (B = .115; ES = .303), explaining 8.6% of the variance at level-2. In the next step (Model 3, Table 3) personal characteristics and self-beliefs explained 28.2% of the level-2 variance: agency belief in effort predicted effort exertion (B = .465; ES = 1.004). We then, one by one, included individual differences and beliefs as predictors of the slopes. Hypothesis 4 (personal characteristics and self-beliefs as moderators of the effort exertion on task difficulty prediction) was partially 3 Effect sizes were calculated using recommendations by Marsh et al. (2009): ES = (2 × B × SDpredictor) / ψ, where B is the unstandardized regression coefficient in the MLM, SDpredictor is the standard deviation of the predictor variable at Level 2, and ψ is the total variance of the dependent variable. 4 We calculated the explained variance the proportion of total level-specific error vari-

σ 2ejb −σ 2ejm

ance (Hox, 2002), for example: R2level2 ¼ ð variance for the baseline model and R2level2 ¼ 0:124−0:066 ¼ :468 . 0:124

σ

, where σ2e|b is the lowest level residual j Þ that for the comparison model, giving

e b2

σ2e|m

60


Table 3 Situation-specific effort exertion regressed on task difficulty, personal characteristics and self-beliefs. Model 1 Fixed effects Intercept (b0) Situation-specific: Task difficulty Age Gender (0 = boy, 1 = girl) School performance Person-specific Agency: ability Agency: effort Perceived difficulty

B 3.266 0.016

Model 2

Model 3

SE

p

B

SE

p

B

SE

p

0.031 0.021

***

3.265 0.016 −0.043 0.257 0.115

0.031 0.021 0.047 0.059 0.046

***

3.266 0.017 0.017 0.067 0.110 −0.061 0.465 0.093

0.031 0.021 0.046 0.067 0.061 0.079 0.091 0.076

***

*** *

***

Random effects

σ2

SE

pa

σ2

SE

p

σ2

SE

p

Effort exertion intercept (u0j) Task difficulty slope (u1j) Effort exertion × task difficulty (u10) Residual (e0ij) R2 between R2 within

0.245 0.068 −0.034 0.539

0.028 0.011 0.014 0.025

*** *** * ***

0.224 0.069 −0.040 0.539 0.086 0.099

0.026 0.011 0.014 0.025

*** *** ** ***

0.176 0.070 −0.046 0.539 0.282 0.099

0.027 0.011 0.014 0.025

*** *** *** ***

0.099

Note: We used Wald-tests of significance for fixed effects, and for random effects we applied the correction factor by Berkhof and Snijders (2001): 2(LR1 − LR0) for extracting the χ2 value, using p/2 as significance level. We calculated the explained variance the proportion of total level-specific error variance (Hox, 2002; see Footnote 4). * = p ≤ .05, ** = p ≤ .01, *** = p ≤ .001.

confirmed, as school performance predicted the effort on difficulty slope (B = .062; p b .05). For higher achievers a relatively more difficult task predicted a higher level of effort exertion, while for lower achievers a relatively more difficult task predicted low effort exertion (see Fig. 1). 4.3. Competence evaluation differentially predicted by effort exertion and task difficulty Our fifth research question was to investigate how situation-specific effort exertion and task difficulty differentially predicted competence evaluation. As shown in Model 1 (see Table 4), situation-specific competence evaluation was predicted by a higher level of situation-specific effort exertion (Β = .168; p b .001; ES = .43), and when a task was perceived as less difficult (B = −.476; p b .001; ES = −1.51). Students varied significantly with regard to their average competence evaluation (σ2u0j=. 214; p b .001). Confirming Hypothesis 5 (i.e., differential effects of task difficulty and effort exertion on competence evaluation), the effect of the situation-specific effort exertion slope on competence evaluation varied significantly across the students (σ2u1j=. 035; p b .001), giving slopes from B = −.21 to .54. Likewise, the effect of the situationspecific task difficulty on competence evaluation varied significantly across the students (σ2u2j=. 027; p b .001), giving slopes between B = −.80 and −.15. The covariance of competence evaluation and effort exertion was negative (σu10 =−. 029; p b .001; r = −0.46), indicating that a higher competence evaluation intercept was related to a slightly negative or close to zero slope (i.e., closer to the average slope), or that a lower competence evaluation intercept was related to a more positive slope (i.e., more effort predicts a higher level of competence evaluation). For low competence evaluation students more effort is needed for being as successful as a high competence-evaluation student. The covariance of competence evaluation and task difficulty was positive (σu20=. 020; p b .01; r = .39) indicating that a higher competence evaluation intercept was related to a less negative slope. This means that competent students were less affected by difficult tasks, while less competent students felt mired by difficult tasks. 4.4. Personal characteristics and self-beliefs predict competence evaluation When we entered personal characteristics (Model 2) higher performing students evaluated their competence higher (B = .236; p b .001; ES = .65). When both individual characteristics and beliefs (Model 4) were entered together an age effect emerged, older students evaluating their competence lower (B = − 0.103; p b .05;

ES = − .222), and higher performers a higher competence evaluation (B = .126; p b .05; ES = .349).5 Partially confirming Hypothesis 6 (personal characteristics and selfbeliefs as moderators), we found two predictors of the random slopes, gender and personal difficulty (Fig. 2). Gender predicted a more positive competence evaluation on the effort exertion slope for girls than for boys (B = .09; p b .05; Fig. 2). This means girls exerted more effort than boys for being as successful as boys felt. A higher level of perceived difficulty (i.e., students who perceive school as difficult) predicted a more negative competence evaluation on situation-specific task difficulty slope (B = .09; p b .05; Fig. 3). This means all students evaluated their competence equally low at the most difficult tasks. Students who found school difficult felt more competent at easier tasks than students who found school less difficult.

5. Discussion The aims of this study were to investigate students' self-beliefs in ability, effort and difficulty, and their situation-specific learning experiences of competence evaluation, effort exertion and task difficulty. We investigated variability of, and individual differences in the interrelations between learning experiences, and whether personal characteristics (age, gender school performance) and self-beliefs (agency beliefs in ability and effort, and perceived difficulty), were related with, predicted and moderated the constellations of learning experiences. Our timeframe was defined as real-time learning experiences during one week of school, positing our time-perspective between diary studies and micro-analytic studies of educational processes. We found instances of convergence and divergence between self-beliefs and learning experiences, substantial variability within students, and individual differences in interrelations between learning experiences, as discussed below. We specifically found students who differentially exerted effort depending on how difficult they perceived a particular learning task, differentially evaluated their competence depending on how much effort they exerted at tasks of different challenge level. Person characteristics (academic performance and gender) and self-beliefs (perceived difficulty) moderated relations between learning experiences. We discuss 5 In an alternative model we entered self-beliefs on their own. In this model higher perceived difficulty predicted a lower evaluation of competence (B = −.163; p b .05; ES = −.35). A higher level of agency belief in ability predicted a higher evaluation of competence (B = .134; p b .05; ES = .33). These self-belief effects were diminished in Model 3, suggesting mediation.


61

Fig. 1. Situation-average task difficulty × school performance as predictor of effort exertion.

Several findings in line with Hypothesis 2 demonstrated withinindividual variability (i.e., intraclass correlations rICC from .21 to .28), and individual differences in associations between situation-specific beliefs (rSD from .28 to .40), warranting a random slope modeling approach.

implications for educational practice, and measurement and modeling of intrapersonal constructs in educational contexts.

5.1. Variability and interrelatedness of beliefs across self and situations At the personal level of the belief system we replicated known relationships (Hypothesis 1) between agency beliefs in ability and effort, and perceived difficulty (Malmberg & Little, 2007; Malmberg et al., 2008). We found instances of convergence (e.g., agency belief in effort was related to situation-average effort exertion), but interestingly also some divergence (e.g., perceived difficulty was moderately related situation-average task difficulty) between self-beliefs and situationaverage learning experiences. Of particular interest are self-beliefs in personal difficulty, which were related to a lower agency belief in effort but unrelated to situation-average effort exertion, as was situationaverage difficulty with situation specific task difficulty. This indicates that perceptions of difficulty at school and levels of difficulty of different tasks can take different meaning (Nesselroade, 2001; Schmitz & Skinner, 1993). In our study the generally strong performance-to-ability belief association was as strong as in the Schmitz and Skinner (1993) study, in which capacity (agency) beliefs in both effort and ability were strongly associated with mean daily test results in both math and mother tongue.

5.2. Differential effort exertion at difficult tasks Consistent with Hypothesis 3, we found a wide variability in random slopes (ranging from B = −.51 to .54), demonstrating that effort varied by challenge level (i.e., task difficulty) across individual students. We found substantial individual differences in effort exertion when tasks were above or below the students' average perceived level of task difficulty. Some students increased their effort indicating a mastery approach, while others lowered their effort indicating a helpless response (Pintrich, 2000). Such a pattern of helplessness appears when individuals do not perceive outcomes contingent on their actions (Seligman, 1975). Prolonged exposure to such non-contingency, particularly in the school context can lead to a number of negative consequences including passivity, apathy, negative and depressed affect, inhibited action implementation, and disengagement (Skinner et al., 1998). We found partial support for Hypothesis 4 suggesting that a high level of self-beliefs would predict the relation between situation-

Table 4 Situation-specific competence evaluation regressed on effort exertion and task difficulty, personal characteristics and self-beliefs. Model 1

Model 2

Model 3

Fixed effects

B

SE

p

B

SE

p

B

SE

p

Intercept Situation-specific: Effort exertion Task difficulty Effort exertion × task difficulty Age Gender (0 = boy, 1 = girl) School performance Agency: ability Agency: effort Perceived difficulty

3.247 0.168 −0.476 0.025

0.028 0.018 0.014 0.019

*** *** ***

3.246 0.167 −0.476

0.028 0.018 0.014

*** *** ***

3.245 0.169 −0.479

0.042 0.018 0.014

*** *** ***

−0.081 0.035 0.236

0.044 0.053 0.044

0.042 0.051 0.049 0.061 0.082 0.073

*

***

−0.103 0.002 0.126 0.100 0.129 −0.123

Random effects

σ2

SE

pa

σ2

SE

p

σ2

SE

p

Competence evaluation intercept (u0j) Effort exertion slope (u1j) Task difficulty slope (u2j) Competence evaluation × effort exertion (u10) Competence evaluation × task difficulty (u20) Effort exertion × task difficulty (u21) Residual (e0ij) R2 between R2 within

0.214 0.035 0.027 −0.029 0.020 0.005 0.288 n.a. 0.508

0.031 0.006 0.005 0.007 0.008 0.004 0.015

*** *** *** *** *

0.176 0.036 0.027 −0.023 0.022 0.004 0.288 0.178 0.508

0.025 0.006 0.005 0.007 0.007 0.004 0.015

*** *** *** ** **

0.149 0.036 0.027 −0.017 0.017 0.004 0.288 0.304 0.508

0.024 0.006 0.005 0.007 0.007 0.004 0.015

*** *** *** * *

***

***

*

***

Note: We used Wald-tests of significance for fixed effects, and for random effects we applied the correction factor by Berkhof and Snijders (2001): 2(LR1 − LR0) for extracting the χ2 value, using p/2 as significance level. We calculated the explained variance the proportion of total level-specific error variance (Hox, 2002; see Footnote 4). * = p ≤ .05, ** = p ≤ .01, *** = p ≤ .001.

62


Competence Evaluation

0.30 0.20 0.10 Male

0.00

Female

-0.10 -0.20 -0.30 -1 SD

0 SD

1 SD

Situation-specific Effort Exertion Fig. 2. Situation-average effort exertion × gender as predictor of competence evaluation.

specific task difficulty and effort exertion (Kuhl & Goeshke, 1985; Pintrich, 2000). Students who in general thought they were effortful at school (i.e., a higher level of agency belief in effort) exerted more effort in situations on average, characteristic of high achievers and girls (girls had higher levels of agency belief in effort). In addition, students who on average were more effortful across situations exerted relatively less effort when confronted with a more difficult task, indicating that these students would be likely to exhaust less effort when attempting a task more difficult or challenging than usual. Schmitz and Skinner (1993) suggested that children's concurrent effort is influenced by generalized perceived control, that is their “running total” (e.g., grand average or moving average) of personal control in regulating subsequent effort rather than over-weighting a single proximal experience, which would run the risk of allowing one failure to derail subsequent engagement. Our results and suggestions by Schmitz and Skinner (1993) indicate that students with high agency (i.e., control) beliefs in effort have a large resource of effort available to them, which they are able to draw upon when situations demand. A smaller increment in effort exertion gives less fatigue than for those who need to exert more. In contrast to effortful students, students who on average exerted less effort exerted relatively more effort when confronted with a more difficult task. These students, who believe they possess a limited resource amount of effort, might feel that they need to exhaust the resource they have when the situation demands (Baumeister et al., 1998). Our moderation analysis showed (see Fig. 1) that higher achievers exerted relatively more (additional) effort at more difficult tasks while lower achievers exerted relatively less effort. This indicates that higher achievers, who also thought they were more able, effortful and perceived school as less difficult, had resources to exert when tasks so demanded. In contrast low achievers, who thought they were less able, less effortful

and perceived school as more difficult, did not have resources to exert when tasks so demanded. Instead they withdrew from exerting effort. 5.3. Differential effects of effort regulation at difficult tasks on competence evaluation Consistent with Hypothesis 5, we found a positive fixed effect of effort exertion and a negative fixed effect of task difficulty on competence evaluation. These fixed effects are similar to Schmitz and Skinner (1993), who found subjective evaluation of performance associated with success attribution to effort and ability, attributions of homework success to task ease (lack of difficulty), and test errors to task difficulty. We found large individual differences in average competence evaluation, competence evaluation on effort exertion slopes and competence evaluation on task difficult slopes. Regarding the relation between competence evaluation and effort exertion, the individual differences in slopes echo Schmitz and Skinner (1993) who found the association between subjective (i.e., situation-specific) effort exertion and subjective performance evaluation to vary across children, for some positive, for some absent and for some (particularly for anxious children) even negative. In our study a higher average competence evaluation was related to a reduced predictive power of task difficulty, i.e. students who felt that they were more competent on average were less likely to rate their competence lower when faced with difficult tasks. In contrast, a lower average competence evaluation intercept was related to an increased predictive power of task difficulty, i.e., students who felt less competent exerted more effort for doing as well as their peers who felt more competent.

Competence Evaluation

0.80 0.60 -1 SD Perceived Difficulty

0.40 0.20

0 SD Perceived Difficulty

0.00 -0.20 -0.40

+1 SD Perceived Difficulty

-0.60 -0.80

-1 SD

0 SD

1 SD

Situation-specific task difficulty Fig. 3. Person-specific perceived difficulty × situation-specific task difficulty as predictor of competence evaluation.


Consistent with Hypothesis 6, we found gender to moderate the effort exertion on competence evaluation slope (Fig. 2), and perceived difficulty to moderate the task difficulty on competence evaluation slope (Fig. 3). We found that girls felt more effortful in general and also exerted more effort than boys did, and girls exerted more effort than boys for evaluating their competence at par with boys (see gender × effort exertion effect, Fig. 2). This finding is partly in line with findings of differences in boys' and girls' agency beliefs about school performance (Stetsenko et al., 2000), in which girls were found to rate their effort higher than boys in countries where they outperformed boys academically. In our study, based in England, there was no difference in academic performance between boys and girls, in contrast to the cross-national study (Stetsenko et al., 2000), which found girls' and boys' effort beliefs to be equal when academic performance was equal. Other studies have found that boys feel more confident and have higher self-concepts than girls in mathematics (ds from .10 to .33) across countries (Else-Quest, Hyde, & Linn, 2010). In their review, Meece, Glienke, and Burg (2005) found girls to attribute their successes to effort and hard work, which may undermine their expectations for success as mathematics increases in difficulty. Furthermore, Stipek and Gralinski (1991) found that girls rated their ability lower, expected to do less well, and were more likely to attribute failure to low ability than boys did. Girls also reported less pride in their success and a stronger desire to hide their paper after failure and were less likely to believe that success could be achieved through effort. We found that older (i.e., 6th grade) students evaluated their competence lower than younger (i.e., 5th grade) students. This might reflect the closeness in time to the Key-Stage 2 (KS2) exam, a standardized test which all 6th grade students in England sit. Thus, a lower competence evaluation can reflect an increase in both curriculum and teacher expectations (Nicholls, 1984) as well as realistic perceptions of demands in the school system (Dermitzaki & Efklides, 2001; Heider, 1958). A higher level of perceived difficulty predicted a more negative competence evaluation on situation-specific task difficulty slope. While all students evaluated their competence equally low at the most difficult tasks, students who found school difficult evaluated their competence higher at easier tasks than students who found school less difficult (Fig. 3). Whether the students experiencing difficulty select easier tasks (e.g., so they do not need to attribute failure to incompetence) or are given easier tasks to work on by their teacher (e.g., protecting students from such failure attributions) would be an important aspect to investigate in future studies. 5.4. Implications for practice Previous intervention studies have often queried at which level of the educational system interventions might be most effective, at the student belief level (O'Mara, Marsh, Craven, & Debus, 2006), the student-level (Wehmeyer, Palmer, Agran, Mithaug, & Martin, 2000), the teacher level (Reeve, Jang, Carrell, Jeon, & Barch, 2004), or the school level (Martin, 2008). The wide intrapersonal variability we found in our study could indicate that there would be opportunities for teachers and teaching assistants to capitalize on moments of learning opportunities (Pianta, La Paro, & Hamre, 2005), in efforts to adapt teaching to the individual needs of students (Corno, 1995; Corno & Snow, 1986; Pianta, Belsky, Vandergrift, Hours, & Morrison, 2008). In order to design comprehensive intrapersonal interventions it would be important to further unpack the sources of intrapersonal variability. Intervening with situation-specific aspects of the learning experience as shown in the current study, would be aspects of the task (e.g., available choice and selection of task, difficulty of the task, duration of task); aspects of feedback on the task (e.g., assistance in task-selection, provision of help, detection of concealment of need of help; Hattie & Timperley, 2007); and aspects of the situation (e.g., time of day, school subject, and seating). Notwithstanding this, intrapersonal variability also presents something of a potential

63

challenge as the differentiating at the task level is likely to present more of an ongoing complexity than differentiating only at the student level. For a start, the sum of individual tasks across all individual students vastly outnumbers the sum of students thus, attending to each task for each student is a greater load on the teacher than simply attending to students. How this is done among the realities of complex classroom life is an area for further thought and research. Future use of PDAs and other modern equipment for data collection, could be effectively implemented in self-monitoring interventions (e.g., Rock, 2005), in which students could learn to keep a record of their activities, perceived task difficulty, engagement, on-task behaviors, and perceived and teacher-reported performance outcomes. Selfreflection based on self-monitoring could also be enhanced through electronic feedback, for example students accessing time-series graphs of their repeated self-reports. 5.5. Measures and methods Researchers need to take care when operationalizing situationspecificity and domain-specificity of items at different levels of measurement. A three-factor solution fitted data well, demonstrating that it was possible to discriminate between competence evaluation (success evaluation, understanding), effort exertion and task difficulty. A previous study (Boekaerts, 1999) defined a competence-construct based on three situation specific perceptions: outcome expectations, perceived level of difficulty, and self-efficacy. Our situation-specific beliefs were clearly focused on each target construct (see Measures) and avoided confounding and overlapping wording between the concepts of ability, effort and difficulty. Such distinctive wording is necessary as children develop concepts of ability by information and cues about their own and other's effort and experienced difficulty levels (Nicholls, 1984; Nicholls & Miller, 1984). This finding demonstrates that researchers who wish to measure ability, effort and difficulty beliefs, need to take care in the operationalization of each construct in order to not confound one belief with the other. For example, the following prototypical “ability” items confound effort and difficulty, e.g., “I can do even difficult tasks pretty fast”, and ability and difficulty: e.g., “I'm pretty good at solving hard tasks”. The following “effort” item confounds effort and difficulty: e.g., “I don't give up easily when working on a difficult task”, and all three “If I try hard to solve difficult problems I am always successful”. 5.6. Limitations and future research Some limitations of the present study are obvious. First, inspecting individual differences in intrapersonal correlations and estimating predictive relationships using random slope models ignore the timedimensions in the data. Each individual is treated as his or her “own control”, meaning that we investigated how individuals vary across tasks of different levels of difficulty, even though these tasks are not considered in chronological order. In order to capture the timedimension, future studies using Dynamic Factor Analysis (Browne & Zhang, 2007; Musher-Eizenman et al., 2002) would serve as an important expansion of the present study. Second, we presented descriptive measures of interpersonal differences in intrapersonal correlations but did not represent intrapersonal variability as a construct in its own right (Eid & Diener, 1999; Jahng, Wood, & Trull, 2008). Thus modeling intrapersonal variability rather than interpersonal differences in intrapersonal correlations, would also add further depth to the present study. Third, this study was based on students' self-reported learning experiences. An important direction for future intensive longitudinal electronic questionnaires would be to collect both student- and teacherreports of the same lesson. Such a study could ultimately show whether teachers, through adaptive practices, could alleviate student disengagement and maladaptive beliefs, perceptions and behaviors, in line with Liew, Chen, and Hughes (2010) who found that teachers buffered

64


negative learning experiences of low achieving students (who were disadvantaged or at developmental risk). Finally, although we in the present study modeled individual characteristics and self-beliefs as predictors of situation-specific beliefs and perceptions, that are traits predicting states, we need to keep in mind that the directionality in the regression model is statistical, not causal. In future studies it would be important to model effects of belief dynamics during particular time-periods and the effects of these on subsequent selfbeliefs. 6. Conclusions We expanded previous cross-sectional and longer-term longitudinal studies of action-control beliefs by investigating both situation-specific learning experiences and self-beliefs of competence, effort and difficulty. As a complement to the existing diary studies of one or two school subjects (science, Tsai et al., 2008; language and maths, Schmitz & Skinner, 1993), we investigated student reports across all school subjects during all days of one calendar week. Students' learning experiences varied substantively across situations, and were differentially interrelated between students. We first modeled situation-specific effort exertion given demand levels (i.e., task difficulty), and second, competence evaluation given the effort exerted and demand level. Using multilevel structural equation models (MSEM), we confirmed hypotheses derived from previous studies that students who on average across situations evaluated their competence higher exerted less effort at demanding tasks, and felt more successful at difficult tasks. Higher performers exerted more effort at difficult tasks, girls exerted more effort than boys for the same level of competence evaluation, and students who in general found school difficult evaluated their competence higher at easier tasks. The investigation of situation-specific learning experiences provides insights into student belief systems in educational contexts that complement our knowledge of individual difference in such beliefs. Acknowledgments We are thankful to all students, teachers and teaching assistants who took part in this study, and for the support of the head teachers and representatives of two Local Education Authorities in England, and particularly to Cassandra Woolgar (Halliburton), Anthony Riches and Dan Archer. The study was supported by the Research Development Fund, at the University of Oxford, during the first author's Research Counsels UK (RCUK) fellowship. Dr. Theodore A. Walls received partial support from the American Cancer Society during this work. References Azevedo, R., Moos, D. C., Johnson, A.M., & Chauncey, A.D. (2010). Measuring cognitive and metacognitive regulatory processes during hypermedia learning: Issues and challenges. Educational Psychologist, 45, 210–223. Baumeister, R. E., Bratslavsky, E., Muraven, M., & Tice, D.M. (1998). Ego depletion: Is the active self a limited resource? Journal of Personality and Social Psychology, 74, 1252–1265. Beal, D. J., & Weiss, K. M. (2003). Methods of ecological momentary assessment in organizational research. Organizational Research Methods, 6, 440–464. Berkhof, J., & Snijders, T. A. B. (2001). Variance component testing in multilevel models. Journal of Educational and Behavioral Statistics, 26(2), 133–152. Boekaerts, M. (1999). Motivated learning: studying student * situation transactional units. European Journal of Psychology of Education, 14, 41–55. Boekaerts, M., & Corno, L. (2005). Self-regulation in the classroom: A perspective on assessment and intervention. Applied Psychology: An International Review, 54, 199–231. Bong, M., & Skaalvik, E. M. (2003). Academic self-concept and self-efficacy: How different are they really? Educational Psychology Review, 15, 1–40. Brandtstädter, J. (1998). Action perspectives on human development. In R. M. Lerner (Ed.), Theoretical models of human development (pp. 807–863) (5th ed.). New York: Wiley. Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen, & J. S. Long (Eds.), Testing structural equation models (pp. 136–162). Beverly Hills, CA: Sage. Browne, M. W., & Zhang, G. (2007). Developments in the factor analysis of individual time series. In R. Cudeck, & R. C. MacCallum (Eds.), Factor analysis at 100. Historical developments and future directions (pp. 265–291). Mahwah, NJ: Lawrence Erlbaum.

Bryk, A. S., & Raudenbush, S. W. (1992). Hierarchical linear models in social and behavioral research: Applications and data analysis methods. Newbury Park, CA: Sage Publications. Corno, L. (1995). The principles of adaptive teaching. In A.C. Ornstein (Ed.), Teaching: Theory into practice (pp. 98–115). Boston: Allyn and Bacon. Corno, L., & Snow, R. E. (1986). Adapting teaching to individual differences among learners. In M. C. Wittrock (Ed.), Handbook of research on teaching (pp. 605–629) (3rd ed.). New York: American Educational Research Association. Covington, M. V., & Omelich, C. L. (1979a). Effort: The double-edged sword in school achievement. Journal of Educational Psychology, 71, 169–182. Covington, M. V., & Omelich, C. L. (1979b). It's best to be able and virtuous too: Student and teacher evaluative responses to successful effort. Journal of Educational Psychology, 71, 688–700. Dermitzaki, I., & Efklides, A. (2001). Age and gender differences on students' evaluations regarding the self and task-related experiences in mathematics. In S. Volet, & S. Järvelä (Eds.), Motivation in learning contexts: Theoretical advances and methodological implications (pp. 271–293). Amsterdam: Elsevier. Dyer, N. G., Hanges, P. J., & Hall, R. J. (2005). Applying multilevel confirmatory factor analysis techniques to the study of leadership. The Leadership Quarterly, 16, 149–167. Eid, M., & Diener, E. (1999). Intraindividual variability in affect: Reliability, validity, and personality correlates. Journal of Personality and Social Psychology, 76, 662–676. Else-Quest, N. M., Hyde, J. S., & Linn, M. C. (2010). Cross-national patterns of gender differences in mathematics: A meta-analysis. Psychological Bulletin, 136, 103–127. Enders, C. K., & Tofighi, D. (2007). Centering predictor variables in cross-sectional multilevel models: A new look at an old issue. Psychological Methods, 12, 121–138. Hattie, J. (2008). Processes of integrating, developing, and processing self-information. Self-processes, learning, and enabling human potential: Dynamic new approaches. In H. W. Marsh, R. G. Craven, & D.M. McInerney (Eds.), International advances in self research series, Vol. 3. (pp. 51–86). Information Age Press. Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77, 81–112. Heckhausen, J., & Heckhausen, H. (2008). Motivation and action. New York: Cambridge University Press. Heider, F. (1958). The psychology of interpersonal relationships. New York: Wiley. Hox, J. (2002). Multilevel analysis: Techniques and applications. Mahwah, NJ: Lawrence Erlbaum. Jahng, S., Wood, P. K., & Trull, T. J. (2008). Analysis of affective instability in ecological momentary assessment: indices using successive difference and group comparison via multilevel modeling. Psychological Methods, 13, 354–375. Kochanska, G., Murray, K. T., & Harlan, E. T. (2000). Effortful control in early childhood: Continuity and change, antecedents, and implications for social development. Child Developmental Psychology, 36, 220–252. Kuhl, J. (1985). Volitional mediators of cognition-behavior consistency: Self regulatory processes and action versus state orientation. In J. Kuhl, & J. Beckman (Eds.), Action control: From cognition to behavior (pp. 101–128). Berlin: Springer-Verlag. Kuhl, J., & Goeshke, T. (1985). A theory of action control: Mental subsystems, modes of control, and volitional conflict-resolution strategies. In J. Kuhl, & J. Beckmann (Eds.), Volition and personality. Action versus state orientation (pp. 93–124). Seattle: Hogrefe & Huber. Liew, J., Chen, Q., & Hughes, J. N. (2010). Child effortful control, teacher–student relationships, and achievement in academically at-risk children: Additive and interactive effects. Early Childhood Research Quarterly, 25, 51–64. Little, T. D. (1998). Sociocultural influences on the development of children's action-control beliefs. In J. Heckhausen, & C. Dweck (Eds.), Motivation and self-regulation across the life span (pp. 281–315). New York: Cambridge University Press. Little, T. D., Oettingen, G., & Baltes, P. B. (1995). The revised control, agency, and means–ends interview (CAMI). Materialen aus der Bildungsforschung No. 49. Berlin: Max-PlanckInstitut für Bildungsforschung. Little, T. D., Oettingen, G., Stetsenko, A., & Baltes, P. (1995). Children's action-control beliefs about school performance: How do American children compare with German and Russian children? Journal of Personality and Social Psychology, 69, 686–700. Little, T. D., Stetsenko, A., & Maier, H. (1999). Action-control beliefs and school performance: A longitudinal study of Moscow children and adolescents. International Journal of Behavioral Development, 23, 799–823. Malmberg, L. -E., Hall, J., & Martin, A. J. (2013). Academic buoyancy in secondary school: Exploring patterns of convergence in Mathematics, Science, English and Physical Education. Learning and Individual Differences, 23, 262–266. Malmberg, L. -E., & Little, T. D. (2007). Profiles of ability, effort, and difficulty: Relationships with worldviews, motivation and adjustment. Learning and Instruction, 17, 739–754. Malmberg, L. -E., Wanner, B., & Little, T. D. (2008). Age and school-type differences in children's beliefs about school performance. International Journal of Behavioral Development, 32, 547–557. Malmberg, L. -E., Woolgar, C., & Martin, A. (in press). Quality of the Learning Experience Questionnaire (LEQ) for Personal Digital Assistants (PDA). International Journal of Quantitative Research in Education, 00, 000-000 (in press). Marsh, H. W., Lüdtke, O., Robitzsch, A., Trautwein, U., Asparouhov, T., & Nagengast, B. (2009). Doubly-latent models of school contextual effects: Integrating multilevel and structural equation approaches to control measurement and sampling error. Multivariate Behavioral Research, 44, 764–802. Martin, A. J. (2008). Enhancing student motivation and engagement: The effects of a multidimensional intervention. Contemporary Educational Psychology, 33, 239–269. Meece, J. L., Glienke, B. B., & Burg, S. (2005). Gender and motivation. Journal of School Psychology, 44, 351–373. Mehl, R. M., & Conner, T. S. (Eds.). (2012). Handbook of research methods for studying daily life. New York, NY: Guilford Press. Mehta, P. D., & Neale, M. C. (2005). People are variables too: Multilevel structural equations modeling. Psychological Methods, 10, 259–284.

L.-E. Malmberg et al. / Learning and Individual Differences 28 (2013) 54–65 Molenaar, P. C. M., Huizenga, H. M., & Nesselroade, J. R. (2003). The relationship between the structure of interindividual and intraindividual variability: A theoretical and empirical vindication of Developmental Systems Theory. In U. M. Staudinger, & U. Lindenberger (Eds.), Understanding human development: Dialogues with lifespan psychology (pp. 339–360). Dordrecht: Kluwer Academic Publishers. Mueller, C. M., & Dweck, C. S. (1998). Praise for intelligence can undermine children's motivation and performance. Journal of Personality and Social Psychology, 75, 33–52. Musher-Eizenman, D. R., Nesselroade, J. R., & Schmitz, B. (2002). Perceived control and academic performance: A comparison of high- and low-performing children on within-person change patterns. International Journal of Behavioral Development, 26, 540–547. Muthén, B. O. (1991). Multilevel factor analysis of class and student achievement components. Journal of Educational Measurement, 28, 338–354. Muthén, L. K., & Muthén, B. O. (2009). Mplus statistical analysis with latent variables: User's guide. Los Angeles: Muthén & Muthén. Muthén, L. K., & Muthén, B. O. (2012). Mplus User’s Guide (Seventh Edition )Los Angeles, CA: Muthén & Muthén. Nesselroade, J. R. (2001). Intraindividual variability in development within and between individuals. European Psychologist, 6, 187–193. Nicholls, J. G. (1984). Achievement motivation: Conceptions of ability, subjective experience, task choice, and performance. Psychological Review, 91, 328–346. Nicholls, J. G., & Miller, A. T. (1984). Development and its discontents: The differentiation of the concept of ability. Advances in Motivation and Achievement, 3, 185–218. O'Mara, A. J., Marsh, H. W., Craven, R. G., & Debus, R. (2006). Do self-concept interventions make a difference? A synergistic blend of construct validation and meta-analysis. Educational Psychologist, 41, 181–206. Pianta, R. C., Belsky, J., Vandergrift, N., Hours, R., & Morrison, F. (2008). Classroom effects on children's achievement trajectories in elementary school. American Educational Research Journal, 45, 365–397. Pianta, R. C., La Paro, K., & Hamre, B. (2005). The Classroom Assessment Scoring System Pre-K Manual. Unpublished manuscript, University of Virginia. Pintrich, P. R. (2000). The role of goal orientation in self-regulated learning. In M. Boekaerts, P. P. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 451–502). San Diego: Academic Press. Reeve, J., Jang, H., Carrell, D., Jeon, S., & Barch, J. (2004). Enhancing students' engagement by increasing teachers' autonomy support. Motivation and Emotion, 28, 147–169. Reise, S. P., Ventura, J., Nuechterlein, K. H., & Kim, H. K. (2005). An illustration of multilevel factor analysis. Journal of Personality Assessment, 84, 126–136. Rock, M. L. (2005). Use of strategic self-monitoring to enhance academic engagement, productivity, and accuracy of students with and without exceptionalities. Journal of Positive Behavior Interventions, 2005(7), 3–17. Ryu, E., & West, S. G. (2009). Level-specific evaluation of model fit in multilevel structural equation modeling. Structural Equation Modeling, 16, 583–601. Sammons, P., Sylva, K., Melhuish, E., Siraj-Blatchford, I., Taggart, B., Toth, K., et al. (2011). Effective Pre-School, Primary and Secondary Education Project (EPPSE 3–14) — Influences on

65

students' attainment and progress in Key Stage 3: Academic outcomes in English, maths and science in year 9. London: Institute of Education, University of London/Department for Education. Schmitz, B. (2006). Advantages of studying processes in educational research. Learning and Instruction, 16, 433–449. Schmitz, B., & Skinner, E. (1993). Perceived control, effort, and academic performance: Interindividual, intraindividual, and multivariate time-series analyses. Journal of Personality and Social Psychology, 64, 1010–1028. Schmitz, B., & Wiese, B.S. (2006). New perspectives for the evaluation of training sessions in self-regulated learning: Time-series analyses of diary data. Contemporary Educational Psychology, 31, 64–96. Seligman, M. E. P. (1975). Helplessness: On depression, development, and death. San Francisco: Freeman. Skinner, E. A. (1986). The origin of young children's perceived control: Mother contingent and sensitive behavior. International Journal of Behavioral Development, 9, 359–382. Skinner, E. A. (1996). A guide to constructs of control. Journal of Personality and Social Psychology, 71, 549–570. Skinner, E. A., Chapman, M., & Baltes, P. B. (1988). Children's beliefs about control, means–ends, and agency: Developmental differences during middle childhood. International Journal of Behavioral Development, 11, 369–388. Skinner, E. A., Zimmer-Gembeck, M. J., & Connell, J. P. (1998). Individual differences and the development of perceived control. Monographs of the Society for Research in Child Development, 63(2–3, Serial No 254). Stetsenko, A., Little, T. D., Gordeeva, T., Grasshof, M., & Oettingen, G. (2000). Gender effects in children's beliefs about school performance: A cross-cultural study. Child Development, 71, 517–527. Stipek, D. J., & Gralinski, J. H. (1991). Gender differences in children's achievement-related beliefs and emotional responses to success and failure in mathematics. Journal of Educational Psychology, 83, 361–371. Stone, A., & Shiffman, S. (2002). Capturing momentary, self-report data: A proposal for reporting guidelines. Annals of Behavioral Medicine, 24, 236–243. Tolvanen, A., Kiuru, N., Leskinen, E., Hakkarainen, K., Inkinen, M., Lonka, K., et al. (2011). A new approach for estimating a nonlinear growth component in multilevel modeling. International Journal of Behavioral Development, 35, 370–379. Tsai, Y. -M., Kunter, M., Lüdtke, O., & Trautwein, U. (2008). Day-to-day variation in competence beliefs: How autonomy support predicts young adolescents' felt competence. Self-processes, learning, and enabling human potential: Dynamic new approaches. In H. W. Marsh, R. G. Craven, & D.M. McInerney (Eds.), International advances in self research series, Vol. 3. (pp. 119–144). Information Age Press. Walls, T. A. (2006). Unpublished measure. Walls, T. A., & Schafer, J. S. (2006). Models for intensive longitudinal data. New York: Oxford University Press. Wehmeyer, M. L., Palmer, S. B., Agran, M., Mithaug, D. E., & Martin, J. (2000). Teaching students to become causal agents in their lives: The self determining learning model of instruction. Exceptional Children, 66, 439–453.