READING ACHIEVEMENT AND PARENTAL INVOLVEMENT 1 ...

RUNNING HEAD: READING ACHIEVEMENT AND PARENTAL INVOLVEMENT

1

1

Parental involvement and pupil reading achievement in Ireland: Findings

2

from PIRLS 2011

3

Introduction

4

Research has shown that parental involvement is positively associated with a range of

5

student outcomes such as achievement, level of educational attainment and behaviour

6

(Byrne & Smyth, 2010; Desforges & Abouchaar, 2003; Fan & Chen, 2001; Klemenčič,

7

Mirazchiyski & Sandoval-Hernández, 2014; McNeal, 1999; OECD, 2012; Postlethwaite

8

& Ross, 1992). Furthermore, parental involvement has been shown to mitigate the

9

negative effects of low socioeconomic status (Dearing, Kreider, Simpkins & Weiss,

10

2006; DENI, 2006). Not surprisingly therefore, education systems focus on involving

11

parents in their child’s education (DES, 2011; DfE, 2010; U.S. Department of

12

Education, 2002). One way in which parental involvement is encouraged is by ensuring

13

that parents are provided with information about their child’s school. In many countries,

14

schools are legally required to provide this information.

15

In the USA, the No Child Left Behind Act (NCLB; U.S. DoE 2002) requires

16

parents to be provided with information on the performance of students, schools, and

17

school districts. Reforms to the NCLB (U.S. DoE, 2008) further emphasised

18

communication between schools and parents, requiring schools to give timely and clear

19

notification to parents of school choice and supplementary educational services. Since

20

2006, parents in Northern Ireland (2006) are legally entitled to receive an annual report

21

about their child’s progress at school. Although the role of parents in education has long

22

been officially recognised in the Republic of Ireland, the 2011 Strategy for Literacy and

23

Numeracy (DES, 2011) highlights the need for improved information for parents about

24

their children’s learning. In England, the 2010 Schools White Paper (DfE, 2010)

25

requires schools to be accountable to parents and to provide sufficient information on

READING ACHIEVEMENT AND PARENTAL INVOLVEMENT

2

26

student and school performance. It has been shown however, that it is not always easy

27

for parents to interpret information about schools, in particular performance data, so

28

increasing quantities of information available to parents does not necessarily support

29

greater levels of involvement in children’s education (De Wolf & Janssens, 2007).

30

Parental involvement is also promoted by ensuring that parents have reasonable

31

access to school staff and sufficient opportunities to volunteer (U.S. DoE, 2002). Again,

32

these may not necessarily lead to the type of parental engagement desired by policy

33

makers. Having access to school staff does not necessarily support a partnership

34

between teachers and parents if the access is provided though formal parent-teacher

35

meetings. The parent may be a passive recipient of information and meetings can serve

36

to reinforce existing power relations (Henderson & Mapp, 2002; Power & Clark, 2000).

37

Involvement through volunteering has a negligible association with achievement,

38

although it may have other benefits such as fostering community spirit (Sui-Chu &

39

Willms, 1996; Okpala, Okpala & Smith, 2001). It is likely that in order to influence

40

student achievement, deeper engagement in learning is required on the part of parents. It

41

is therefore useful to distinguish between parental involvement in school-based

42

activities, which may include activities such as volunteering, and parental engagement

43

with their child’s learning (Harris & Goodall, 2007).

44

This paper examines two questions. Firstly, is parental involvement across a

45

broad spectrum of activities related to pupil reading achievement in Ireland? Secondly,

46

what aspects of parental involvement are most important in explaining variation in

47

reading achievement?

48

Dimensions of parental involvement and association with pupil outcomes

49

Bourdieu’s (1977) concepts of the different types of capital – economic, cultural, and


3

50

social – can be used to provide a theoretical underpinning for the importance of parental

51

involvement in education. Parental involvement is inextricably linked to the concept of

52

cultural capital in particular. Kellaghan (2001) operationalises cultural capital in an

53

educational context by outlining the structures and activities of homes that are

54

associated with it. The six components he describes that are associated with cultural

55

capital are: (i) modelling (in the use of complex language and in planning and

56

organisation); (ii) stimulation to explore and discuss ideas and events; (iii) providing

57

motivation for, guidance in, and reinforcement of school-related activities and

58

independence in decision-making; (iv) holding and communicating high academic

59

aspirations and expectations; (v) providing guidance on school matters and direct

60

instruction, and monitoring/helping with homework; and (vi) ensuring that activities

61

engaged in are developmentally appropriate.

62

Parental involvement also relates to social capital. McNeal (1999) suggests that

63

parental involvement in school organisations is an example of social capital as it gives

64

parents access to shared information through their extended network. He argues that

65

such involvement is less likely to be associated with academic achievement and more

66

likely to be protective against behaviours such as truancy, as parents in the network take

67

greater collective responsibility for each other’s children. Parental involvement is less

68

likely to represent a form of economic capital, although Lareau (1989) posits that

69

parental involvement is greater amongst those high in economic capital.

70

Various typologies of parental involvement have been proposed, including

71

influential work by Epstein (1992) and Hoover-Dempsey and Sandler (1995, 1997).

72

Epstein considers the overlapping spheres of influence of the family, the school and the

73

community while Hoover-Dempsey and Sandler examine the psychological

74

underpinnings of parents’ decisions to become involved in their children’s education. In


4

75

reviewing empirical studies of parental involvement, Sui-Chu and Willms (1996)

76

distinguish between two types of home involvement and two types of school

77

involvement. The first type of home involvement relates to discussing school activities;

78

the second relates to monitoring the child’s out-of-school activities. The first component

79

of school involvement refers to contacts between parents and school personnel; the

80

second relates to involvement in activities such as volunteering in the school and

81

attending parent-teacher meetings.

82

McNeal’s (1999) dimensions of parental involvement can also be separated into

83

those that relate to parenting activities that take place in the home and those that involve

84

parents more directly in school activities. The four dimensions he identifies are parent-

85

child discussion; parent involvement in school organisation; monitoring; and direct

86

involvement in educational processes. McNeal posits the importance of discussion as a

87

theoretical mechanism which allows the importance of education to be conveyed to the

88

child. In addition, discussion allows parents to identify disengagement at an early stage.

89

McNeal suggests that monitoring activities, such as setting limits on time spent

90

watching television, reflect a more structured home environment. However, as greater

91

monitoring may be adopted as a consequence of children underperforming, McNeal

92

hypothesizes that associations between achievement and monitoring practices would be

93

inconsistent or negative. Empirical research has indeed reported contradictory findings

94

on the association between monitoring practices and student outcomes. In particular,

95

studies have found negative associations between achievement and parental help with

96

homework (e.g., Eivers et al., 2010; Madigan, 1994; Muller, 1993). In Ireland, findings

97

from the National Assessments of Reading and Mathematics show that pupils who

98

receive more help from parents with homework had lower mean scores in reading and

99

mathematics (Eivers et al., 2010). Furthermore, a moderate negative correlation is


5

100

reported between time spent on homework and achievement in reading and mathematics

101

(but cf Keith, Reimers, Fehrman, Pottebaum, & Aubey, 1986). Eivers et al. conclude

102

that weaker pupils may need more help and more time for homework (see also Epstein,

103

1988). A positive association was reported in an earlier Irish national assessment

104

between achievement and parental rules about time spent watching TV or playing

105

computer games, and between achievement and parental rules about the content of

106

permissible TV programmes and computer games (Eivers, Shiel, Perkins & Cosgrove,

107

2005).

108

Turning to direct involvement in educational processes, this refers to activities

109

such as attending parent-teacher meetings or speaking to the child’s teacher. It is likely

110

that these forms of parental involvement may increase when a child is experiencing

111

difficulties. In Ireland, 2004 National Assessment findings showed that an increased

112

number of home-school contacts was associated with a significant decline in pupil

113

achievement, although direction of causality cannot be determined as it was a cross-

114

sectional study (Eivers et al, 2005).

115

Previous research at secondary-level in Ireland distinguished between formal

116

and informal parental involvement (Byrne & Smyth, 2010). Formal involvement

117

includes activities such as attending meetings, talking to teachers, involvement with the

118

parents’ association and being a member of the board of management. Activities such as

119

helping with homework and study, discussing progress in school and giving advice

120

about programme and study options are classified as informal parental involvement.

121

Both formal and informal parental involvement measures were associated with student

122

outcomes. Parent-child discussion and attendance at parent-teacher meetings were

123

positively associated with achievement, while checking homework was negatively

124

associated with achievement (Byrne & Smyth, 2010).

READING ACHIEVEMENT AND PARENTAL INVOLVEMENT 125

6

Despite the general recognition of the key role played by parents in their

126

children’s education, research findings in the area are often inconsistent (e.g. Epstein,

127

1988; Madigan, 1994; McNeal, 1999; Muller 1993). One possible explanation is that, as

128

previously noted, parents may become involved in school activities only when students

129

are struggling (OECD, 2012). While the converse has also been shown, i.e., that parents

130

are more involved when children have higher levels of attainment (Harris & Goodall,

131

2007), the possible confounding effect of parental socioeconomic status cannot be

132

overlooked (Lareau, 1989).

133

Other reasons for inconsistencies in the associations between achievement and

134

parental involvement relate to differences in the achievement measures used and

135

differences in the interpretation of what constitutes parental involvement. As noted

136

earlier, it is relevant to distinguish between parental involvement in school activities and

137

deeper engagement in student learning on the part of parents. A review by Desforges

138

and Abouchaar (2003) found that the aspects of parenting that relate to activities in the

139

home are more strongly associated with achievement and adjustment than other aspects

140

of parental involvement, such as those in the school. A meta-analysis by Fan and Chen

141

(2001) found that parental expectations and aspirations have the strongest association

142

with academic achievement, while supervision at home has a much weaker association.

143

A limitation of the analysis by Fan and Chen was the lack of control for parental

144

socioeconomic status. Sui-Chu and Willms (1996) also show that high expectations for

145

children’s performance and general monitoring of performance are positively associated

146

with achievement. Conversely, helping with homework and attending parent-teacher

147

meetings do not have strong effects and may be negatively associated with achievement.

148

A final reason that may account for inconsistencies in research findings relates to a


7

149

failure to take into account variations in the national context in which parental

150

involvement is taking place.

151

Parental involvement in the Progress in International Reading Literacy Survey

152

The Progress in International Reading Literacy Survey (PIRLS; Mullis, Martin, Foy &

153

Drucker, 2012) offers an opportunity to examine parental involvement in education and

154

its association with reading achievement at Fourth grade. An advantage of the PIRLS

155

dataset is that it allows detailed consideration to be given to the national context when

156

considering the associations between parental involvement and student outcomes.

157

Taking, for example, parental support for homework as an indicator of parental

158

involvement, clearly PIRLS data can be used to gain an insight into the variation across

159

participating countries. PIRLS 2011 revealed a strong reliance on homework for home-

160

school communication in Ireland (Eivers & Creaven, 2013). Almost all parents in

161

Ireland (95%) reported setting aside time for homework on a daily or almost daily basis.

162

This is a higher percentage than in most other high performing PIRLS countries such as

163

Hong Kong SAR (68%), Russian Federation (87%), Finland (77%), and Singapore

164

(72%). Similar to parents in the Russian Federation (71%), almost 70% of parents in

165

Ireland indicated that they help with homework daily or almost daily. This compares to

166

just 26% of parents in Finland, 50% in Singapore and 56% in Hong Kong SAR. While

167

homework activities are supported by parents in Ireland, the prevalence of private

168

tutoring in some other countries suggests that elsewhere, help with homework may be

169

supported by tutors (Bray, 2007).

170

Compared to the average levels across PIRLS countries, Irish schools were far

171

less likely to give parents regular updates on the behaviour and well-being of their child,

172

and less likely to discuss parents’ concerns or wishes about their child’s learning (Eivers

173

& Creaven, 2013). In spite of this, Irish parents were very positive about their child’s


school and almost all respondents indicated that the school both provided a safe

175

environment and cared about their child’s education.

176

8

Irish parents were less likely than those in many other countries to receive

177

information about the school in general, such as updates about general academic

178

achievement or about the school’s educational goals. One quarter of parents in Ireland

179

never received updates about the overall academic achievement of the school. Among

180

the highest performing countries in PIRLS 2011, i.e., those with significantly higher

181

mean reading scores than Ireland, only Finland (32%) was similar in this regard (Eivers

182

& Creaven, 2013).

183

PIRLS also revealed similarities across countries regarding parental

184

involvement. In particular, parental support for academic achievement was a universally

185

positive correlate of reading achievement (Eivers & Creaven, 2013). Also, a majority of

186

participating countries had policies for the involvement of parents in their children’s

187

education (Mullis, Martin, Minnich, Drucker & Ragan, 2012).

188

Previous research in the area of parental involvement using the PIRLS 2011

189

dataset has used a single measure of parental involvement derived from school principal

190

responses (Klemenčič et al., 2014). It also used a limited number of explanatory

191

variables across a large number of countries. In the current paper, data from the School,

192

Teacher, Parent and Pupil questionnaires are used to give a more detailed insight into

193

parental involvement in children’s educational activities. Also, this paper focuses on the

194

Irish context as this allows a more detailed look at parental involvement in one

195

education system. Much previous work on formal parental involvement in Ireland was

196

conducted in secondarylevel schools (Byrne & Smyth 2010; Cosgrove, Gilleece &

197

Shiel, 2011). As parental involvement declines and reduces in effectiveness as children


9

198

get older (Crosnoe, 2001; Domina 2005), it is pertinent to use the PIRLS dataset which

199

provides data on primary school children (aged about 10).

200

Data and Method

201

Data for this study are drawn from the combined 2011 PIRLS/TIMSS (Trends in

202

International Mathematics and Science Study) database (Mullis, Martin, Foy & Arora,

203

2012; Mullis, Martin, Foy & Drucker, 2012; for Irish results, see Eivers & Clerkin,

204

2012), focussing on reading achievement as the outcome measure. Reading was selected

205

as the dependent variable as measures of home background have been shown to be

206

somewhat more effective at explaining variance in reading achievement than

207

achievement in other domains (Cosgrove & Creaven, 2013; Sui-Chu & Willms, 1996).

208

Pupils in Ireland achieved a mean reading score in PIRLS 2011 that was significantly

209

above the PIRLS scale centrepoint (i.e., a reference point determined in the first cycle of

210

PIRLS in 2001) and were ranked tenth of participating countries.

211

In 2011, the administration of PIRLS coincided with the administration of

212

TIMSS and Ireland, as well as many other participating countries, participated in both

213

assessments. In addition to the achievement tests, the studies gathered extensive

214

demographic and background information from pupils, parents, teachers and school

215

principals. For the current study, measures of parental involvement in their children’s

216

education have been derived from questions on the pupil, parent and school

217

questionnaires. Relevant questions on the pupil questionnaire asked about the frequency

218

with which parents ask the child what he/she is learning in school, the frequency of

219

talking about schoolwork with parents, the frequency of parents ensuring that time is set

220

aside for homework and the frequency of parents checking homework. The parent

221

questionnaire asked parents of participating pupils for comparable information. Each

222

country participating in PIRLS added up to six country-specific wealth indicators in the


10

223

pupil questionnaire. In Ireland, pupils were asked if they had their own iPhone and

224

whether or not they had a television in their bedroom. Thus, quite detailed information

225

is available at the pupil level about pupils’ home background and the informal

226

involvement of parents in educational activities in the home. While direct instruction on

227

the part of parents at home is likely to be associated with achievement, parent questions

228

on the frequency of activities such as reading books with the child and telling stories

229

refer to the period before the child began primary school. Given the potential for

230

changed circumstances and the length of time elapsed since the child started school,

231

these variables were not included in this analysis.

232

The school questionnaire asked for information about more formal involvement

233

of parents in activities such as volunteering for school trips or serving on school

234

committees. Thus, measures of formal or school involvement are available at school

235

level only.

236

It is particularly useful to examine the PIRLS 2011 data for Ireland as PIRLS

237

took place prior to the implementation of the National Strategy for Literacy and

238

Numeracy (DES, 2011). Therefore, it allows an examination of the role of parental

239

involvement prior to any increase that may be expected to take place in response to the

240

strategy.

241

Sample

242

TIMSS and PIRLS employ a two-stage random sampling process, first selecting a

243

stratified sample of schools and then one or more classes within each school (Martin &

244

Mullis, 2012). In Ireland, 4524 pupils from 221 classes in 151 schools completed the

245

PIRLS test, representing a 93.8% pupil response rate (Eivers & Clerkin, 2012). Parent

246

questionnaires were returned for all participating pupils. The response rates for the

247

Teacher and Principal questionnaires were 99.5% and 96.0%, respectively. After


11

248

deletion of cases missing data on multiple variables, the current analysis is based on

249

4231 pupils from 209 classes, representing 94% of the dataset. The mean reading

250

achievement of the full Irish sample and the sample used in the current analysis were

251

551.6 (SE = 2.26) and 552.4 (SE = 2.35), respectively.

252

Variables

253

The dependent variable in the current analysis is reading achievement in PIRLS 2011.

254

Reading achievement is represented by five plausible values. This methodology uses

255

background data and pupils’ responses to test items to impute multiple achievement

256

scores. This leads to more accurate estimation of population parameters (Beaton, 1987;

257

Martin & Mullis, 2012; Mislevy, 1991). It is necessary to employ this approach in

258

PIRLS as the rotated booklet design means that not all pupils have answered all

259

questions; therefore, it is necessary to infer missing values to get overall estimates of

260

achievement.

261

Independent variables were divided into those that could be considered to

262

represent background or control variables, informal parental involvement, formal

263

parental involvement and school-home communication variables. One reason for

264

categorising variables in this way is that the formal/informal distinction was used in

265

previous research of parental involvement in second-level education in Ireland (Byrne

266

& Smyth, 2010). Also, informal involvement could be considered an overarching term

267

for two of McNeal’s dimensions (monitoring and discussion), while formal involvement

268

covers involvement in parent-teacher organisations and educational support strategies.

269

The school-home communication group of variables is included as they represent

270

activities initiated by the school rather than the parent and appear to represent the

271

school’s attempts to encourage parental involvement rather than acting as measures of

272

parental involvement per se.


12

Variables are classified as either Level 2 (school- or class-level) or Level 1

274

(pupil-level). The rationale for using multilevel modelling and dividing variables into

275

two levels is explained in the next section. The specific variables examined in this

276

analysis are outlined in Table 1 (percentages shown are the percentages of the 4231

277

cases used in the current analysis). All variables (apart from total jobs in the household)

278

were categorical variables, entered in the models using dummy coding (see Table 1).

279

The total jobs variable was not centered in order to facilitate the interpretation of the

280

parameter estimate associated with this variable; the parameter estimate for total jobs

281

corresponds to the expected change in reading achievement associated with one

282

additional job in a pupil’s household.

283

The variables representing pupil ownership of an iPhone or bedroom television

284

are taken as indicators of the degree to which parents monitor their child’s activities. It

285

is expected that having either or both of these gives the child largely unsupervised

286

access to various media. The frequency of the teacher talking to parents about the

287

child’s learning progress is included as a measure of informal parental involvement as

288

this measure allows for phone calls or short chats to be counted. Also, the frequency of

289

teachers sending home progress reports is included as an informal measure as progress

290

reports could encompass an informal note written by the teacher in a pupil’s homework

291

notebook.


292

13

Table 1: Variables used in modelling: Coding and descriptive information Variable (Level)a and description Outcome: Reading achievement (5 Plausible values)

Coding (weighted % of pupils per category)b Mean = 552, SE= 2.4, SD = 73.9

Background variables DEIS statusc1 (2): Indicator of socioeconomic disadvantage

Band 1 (7%), Band 2 (7%), Rural (4%) & Non-DEIS (82%)

Enrolment size (2): Stratifying variable indicating likely number of fourth grade pupils at test time

Small (≤ 20 pupils, 31%), Med (21 – 34 pupils, 27%) & Large (35+, 42%)

Language of instructionc2 (2)

English (94%) or Irish (6%)

School gender mix (2)

Boys (12%); Girls (9%); Mixed (73%); Infant mixed/girls only (5%)

Pupil gender (1)

Girl (49.9%), Boy (50.1%)

Parental education (1)

≤ Lower sec (11%), Upper sec (27%), Short third-level (ISCED 5b, 23%), ≥ Degree (31%), Missing (8%).

Parental employmentd (1): Total jobs in the household (Range 0 – 2)

Mean=1.2; sd=0.59

Number of books at home (1)

≤10 books (10%); 11 – 25 (22%); 26 – 100 (33%); >100 (32%), Missing (4%).

Frequency of speaking English at homeb (1)

Always (81%), Sometimes (13%); Never (2%), Missing (4%).

Informal parental involvement Teacher report of parental support for pupil achievement (2): if missing, data from principal teacher are substituted

Low (6%), Medium (31%), High (46%), Very high (17%)

Freq – teacher meeting or talking to parents individually to discuss child’s learning progress (2):

≤ 3 times per yr (80%), More often (13%), Missing (7%).

Freq – teacher sending home a progress report on the pupil’s learning (2)

≤ 3 times per yr (83%), More often (11%), Missing (7%).

Pupil has iPhone (1)

Yes (12%), No (84%), Missing (4%)

Pupil has TV in bedroom (1)

Yes (51%), No (45%), Missing (3%)

Parent report of expectations for pupil educational attainmentc3 (1)

≤ PLC (10%), Diploma (19%), ≥Degree (64%), Missing (8%).


14

Freq – pupil talks about schoolwork to parentse (1)

Never or almost never (12%), Once or twice a month (8%), Once or twice a week (29%), Daily (52%).

Freq - parent asking pupil about learning in school (1)

Never or almost never (6%), At least monthly (31%), Daily (63%).

Freq – parents ensuring pupil sets aside time for homeworke (1)

≤ 2 a month (10%), Once or twice a week (7%), Daily (83%).

Freq – parents checking if pupil does homework e (1)

Daily (86%), 1 or 2 per week (9%), Monthly or less (6%), Missing (1%)

Formal parental involvement (principal reports) Freq – school asks parents to volunteer for school projects (2)

Never (9%), Once per yr (19%), 2-3 times (41%), > 3 per yr (31%).

Freq –parents asked to serve on sch committees (2)

Once a year or less (48%), 2-3 times (24%), > 3 per yr (28%)

Freq – workshops or seminars for parents on learning or pedagogical issues (2)

Never (43%), 1-3 times per yr (49%), > 3 (8%).

Level of parental involvement in school activities (2)

Low or very low (17%), Medium (37%), High or very high (46%).

Classroom has adult volunteer to work with pupils who have reading difficulties (2)

Always (2%), Sometimes (13%), Never (77%), Missing (7%).

School-home communication (based on principal reports) Freq – inform parents – child’s learning progress (2)

Once a year or less (17%), 2-3 times (71%), > 3 times per yr (13%)

Freq – informing parents about behaviour and wellbeing of child at school (2)

Once a year or less (11%), 2-3 times (69%), > 3 per yr (21%)

Freq – discussing parents’ concerns or wishes about child’s learning (2)

Once a year (17%), 2-3 times (50%), > 3 per yr (33%).

Freq – informing parents – overall academic achievement (2)

Never (26%), Once a year (55%), 23 times per yr or more (20%)

Freq – informing parents – sch accomplishments (2)

Once a year or less (8%), 2-3 times (30%), > 3 per yr (63%).

Freq – informing parents – educational goals & pedagogic principles of school (2)

Never (19%), At least yearly (78%), Missing (3%)

Freq – inform parents – school rules (2)

Once a year or less (64%), 2-3 times (21%), > 3 per yr (14%), Missing (0.7%)

Freq – discuss parents’ concerns or wishes about school’s organisation (2)

≤Yearly (46%), 2-3 times (34%), >3 per yr (20%), Missing (0.5%)

READING ACHIEVEMENT AND PARENTAL INVOLVEMENT 293 294 295

a

15

Level 2: school or class, Level 1: pupil; bBold: reference category cNFootnote N has further detail.dVariable has missing indicator to preserve cases in the dataset; eIf pupil data are missing, parent data are substituted where possible.


16

296

Method

297

Given the hierarchical nature of the data (pupils within classes within schools),

298

multilevel modelling using MLwiN (Version 2.30, Rasbash, Browne, Healy, Cameron,

299

& Charlton, 2014) was employed. This approach allows the contribution of various

300

explanatory variables to be examined simultaneously. For each independent variable in

301

the model, the parameter estimate describes the linear relation between this variable and

302

the outcome (reading achievement), having taken into account the effects of all other

303

variables in the model.

304

Earlier analyses of the Irish PIRLS data examined the use of two-level models

305

(pupils within schools) compared to three-level models (pupils, classes and schools) and

306

concluded that a two-level model using class as the grouping variable is most

307

appropriate (Cosgrove & Creaven, 2013). This work showed that even in schools with

308

two classes, there was little variation in reading achievement between the classes (4%),

309

but using class as the Level 2 grouping variable facilitated the inclusion of teacher

310

variables. Therefore, for the current study, the approach advocated by Cosgrove and

311

Craven is adopted, using a two-level structure with class as the grouping variable.

312

Between-class variance is computed on the basis of the null model using the first

313

plausible value as the outcome variable. As many schools had only a single class

314

participating in PIRLS, and given the low between-class variance in schools with two

315

participating classes, between-class variance can be considered as an approximation of

316

between-school variance in this paper.

317

The analytical approach involved testing each explanatory variable compared to

318

the null model for statistical significance (p < .05). The first plausible value only was

319

used for this step. Variables that were statistically significant individually (p < .05) were

320

retained in their block (i.e. background, informal, formal or communication) and blocks


17

321

of variables were then added one at a time to the null model. This was done five times,

322

once for each plausible value. When blocks were added to the null model, variables that

323

were not statistically significant in their blocks were dropped. Once each block was

324

finalised, all blocks were added to the null model together to give the final model. The

325

final model was run five times to get parameter estimates, standard errors, and statistical

326

tests (either chi-square or t) for each plausible value. Variance explained was computed

327

on the basis of the fitted model compared to the null model.

328

The plausible value methodology employed in PIRLS requires certain additional

329

steps to be taken when conducting multilevel analyses. Parameter estimates and tests of

330

statistical significance such as chi-square were first computed for each plausible value

331

separately; i.e., the model was run five times using each plausible value as the outcome

332

variable. The final parameter estimate or test statistic is equal to the average of the five

333

individual estimates (OECD, 2009). Averages should not be computed at the student

334

level as this will lead to a biased estimate (for details, see OECD 2009). Standard errors

335

for the final estimates were computed using the standard errors associated with the five

336

individual estimates as well as the sampling variance, the imputation variance and the

337

number of plausible values.

338

A further complexity associated with large scale survey data relates to the non-

339

straightforward issue of sampling weights (see e.g., Aitkin & Aitkin, 2011;

340

Rabe_Hesketh & Skrondal, 2006; Rutkowski, Gonzalez, Joncas & von Davier, 2010).

341

Relevant literature has examined calculation methods (e.g., Rutkowski et al., 2010),

342

their application in different software packages (Centre for Multilevel Modelling,

343

2011), and alternative approaches (e.g., Aitkin, Francis & Hinde, 2005). Using the

344

weighting documentation of the Irish PIRLS national study centre and following the

345

methods outlined in Rutkowski et al. (2010), class weights and pupil within class


18

346

weights were computed. Given the very high response rates on PIRLS in Ireland,

347

adjustments for non-response were minimal. Using the first plausible value as the

348

outcome measure, an examination of models computed with and without weights found

349

little difference in the parameter estimates and standard errors between the two.

350

Therefore, following Aitkin et al. (2005), stratifying variables were examined as part of

351

the modelling process and weights were not included. The remainder of this paper refers

352

to unweighted models.

353 354

Effect sizes for the model were calculated as follows (Strand, 2004, p.51): 

The effect size of a dichotomous variable was calculated as the regression

355

coefficient divided by the outcome standard deviation (SD), i.e., the student

356

level SD in reading achievement associated with the first plausible value;

357



the effect size of a continuous variable was calculated as the regression

358

coefficient multiplied by twice the variable’s SD, divided by the outcome SD.

359

The effect size gives the difference (in terms of standard deviation units)

360

between the predicted reading scores of pupils who are one SD above the mean

361

and those who are one SD below the mean on the predictor variable.

362

Effect sizes are interpreted as small, if less than 0.20; small but noteworthy if between

363

0.20 and 0.50; medium if between 0.51 and 0.80; and strong if above 0.80 (Cohen,

364

1977). The What Works Clearing House (WWC), an initiative of the Institute for

365

Education Sciences at the U.S. Department of Education, define effect sizes of 0.25

366

standard deviations or larger to be “substantively important”. Even if not statistically

367

significant, effect sizes of this magnitude “are interpreted as a qualified positive (or

368

negative) effect” (WWC, 2014, p.23).


19

369

Results

370

When background variables were added one-by-one to the null model, all except school

371

enrolment size were significantly associated with reading achievement. When the

372

significant variables were examined together, school gender composition was not

373

statistically significant. Therefore, subsequent analyses excluded school enrolment size

374

and school gender composition.

375

When measures of informal parental involvement were added one-by-one to the

376

null model, only one of the three class/school variables was statistically significant. The

377

only significant variable was the teacher-reported measure of parental support. No

378

significant effects were found for the frequency with which teachers talked to parents

379

individually to discuss their child’s learning progress nor the frequency with which

380

teachers sent home a progress report on the pupil’s learning. At pupil-level, all but one

381

of the variables were statistically significant. The only variable that was not statistically

382

significant at the pupil-level in the informal block was the frequency of having

383

homework checked by parents.

384

When measures of formal involvement were added one-by-one to the null

385

model, two out of five were statistically significant. The two significant variables were:

386

frequency of workshops or seminars for parents, and, principals’ reports of parental

387

involvement in school activities. There were no pupil-level variables in this block.

388

Of the communications variables examined, only one was statistically

389

significant when added one-by-one to the null model. This was the frequency of the

390

school discussing parents’ concerns or wishes about their child’s learning. This variable

391

was not statistically significant in subsequent analyses that included the background

392

variables. No significant associations were found for the frequency with which schools

393

informed parents about: their child’s learning progress; their child’s behaviour and


20

394

wellbeing at school; the school’s overall academic achievement; school

395

accomplishments; the school’s educational goals and pedagogic principles; the rules of

396

the school; or, the school’s organisation (e.g. rules and regulations, timetables, or safety

397

measures). Given the non-significance of the communications variables, no further

398

reference is made to them.

399

Table 2 presents three models: background variables only; background variables

400

and measures of informal parental participation; and, background variables and formal

401

parental participation. (Effect sizes will be presented later for the final model only.) In

402

interpreting these models, it should be noted that the parameter estimate for a variable

403

represents the association between this variable and outcome, having controlled for all

404

other variables in the model. Looking at Level 2 variables in Model 1 (i.e., the model

405

with background variables only), it can be seen that having controlled for the other

406

variables in the model, pupils in schools where the language of instruction is Irish have

407

significantly higher reading scores than those in schools where the language of

408

instruction is English. Also, all else being equal, pupils in DEIS schools have

409

significantly lower achievement than those in Non-DEIS schools. There is a large

410

standard error associated with the parameter estimate for Rural DEIS, thus the statistical

411

significance of the DEIS variable is largely attributable to the differences between

412

pupils in non-DEIS schools and those in Band 1 and Band 2 schools.

413

Looking at pupil-level variables, Model 1 shows that girls significantly

414

outperform boys, all else being equal (Table 2). Parental education is also statistically

415

significant, with lower achievement among pupils whose parents have completed lower

416

secondary education or below compared to those whose parents have completed upper

417

secondary. Higher levels of achievement are associated with higher levels of parental

418

education such as short third-level courses (e.g. Diploma or non-degree level courses)


21

419

or university qualifications. Having controlled for the various other variables in the

420

model, the numbers of books in a pupil’s home is also relevant: children with 10 or

421

fewer books have significantly lower achievement than those with 26 to 100 books

422

(reference category). Children with 11 to 25 books have significantly lower

423

achievement than those in the reference category, i.e., 26 to 100 books, while children

424

with more than 100 books have significantly higher levels of achievement than those

425

with 26 to 100 books. Children who reported ‘sometimes’ or ‘never’ speaking English

426

at home had significantly lower scores than those who reported ‘always’ speaking

427

English at home. Model 1 explains 73% of between-class variance, 18% of within-class

428

variance and 27% of the total variance in reading achievement.

429

Model 2 presents the results of adding the background variables and the

430

informal participation variables to the null model (Table 2). Looking firstly at the

431

background variables, the parameter estimates for DEIS are seen change substantially

432

between Models 1 and 2, suggesting that informal parental involvement may account

433

for some of the differences found between pupils in DEIS and non-DEIS schools. The

434

effects of parental education are also reduced between Model 1 and Model 2, suggesting

435

that differing levels of informal parental involvement may be associated with parental

436

education. In particular, the parameter estimates associated with short third level

437

courses and university degree are reduced in Model 2 compared to Model 1 so it

438

appears that informal parental involvement may be one mechanism by which parental

439

education plays a role in pupil achievement.

440

All informal measures of parental involvement added to Model 2 are statistically

441

significant after controlling for background characteristics. At school-level, parental

442

support for pupil achievement is statistically significant. Pupils whose teachers report

443

low levels of parental support for pupil achievement have significantly lower reading


22

444

achievement than those whose teachers report medium levels. High and very high levels

445

of parental support are associated with significantly higher reading achievement than

446

medium levels of support (Table 2).

447

All pupil-level measures of informal parental involvement are also statistically

448

significant (Table 2). Having controlled for the other variables in the model, pupils who

449

have an iPhone have significantly lower levels of reading achievement than pupils who

450

reported not having an iPhone. There is quite a large difference of 26 points (roughly

451

one third of a standard deviation) between pupils with and without an iPhone. This is

452

the the net effect of having an iPhone, given the inclusion of all other variables in the

453

model. Also, pupils who have a TV in their bedroom have significantly lower

454

achievement than those without.

455

Parental expectations of academic attainment are significantly associated with

456

reading achievement; i.e. ceteris paribus, pupils whose parents expect them to complete

457

a third-level certificate or diploma have higher reading scores than those pupils whose

458

parents expect them to finish formal education at or below the level of a Post-Leaving

459

Certificate (PLC) course or apprenticeship. Pupils whose parents expect them to study

460

to degree level or beyond have higher average achievement than those expected to study

461

to PLC level or below.

462

Parental frequency of asking about what pupils are learning in school is

463

negatively associated with achievement; i.e., all else being equal, pupils whose parents

464

ask on a daily basis about what they are learning have significantly lower achievement

465

than those whose parents ask on a monthly basis. This suggests that parents may

466

question lower achieving pupils on a more frequent basis about what they are learning

467

in school; i.e., that asking about learning is a reaction to low achievement.


23

Talking about schoolwork is found to be positively associated with achievement,

469

although the main advantage is found for pupils who talk to their parents about

470

schoolwork on a monthly basis compared to those who never do. The parameter

471

estimate associated with daily talking about school work is smaller than its associated

472

standard error so there is no significant difference in reading achievement between

473

pupils who never talk about school work (reference category) and those who talk about

474

it on a daily basis. This non-significant difference could be a consequence of the daily

475

discussion group including a mix of pupils, some of whom have regular discussion in

476

response to low achievement and others for whom daily discussion of educational issues

477

is encouraged, regardless of achievement.

478

Having time set aside for homework on a regular basis is associated with higher

479

average achievement, all else being equal. Thus pupils who report that their parents

480

ensure that they make time for homework on a daily basis have significantly higher

481

reading achievement than those who indicated that this took place on a monthly basis or

482

less often. Model 2 explains 81% of between-class variance, 24% of within-class

483

variance and 33% of the total variance in reading achievement.


24

Table 2. Models of reading achievement – background variables only, background and informal parental involvement, background and formal parental involvement. Model 1: Background PE

SE

538.28

7.337

13.99

6.745

Band 1

-20.31

8.434

Band 2

-25.58

8.655

5.46

12.621

p

Model 2: Bkgrd & informal PE

SE

511.66

7.105

11.44

6.073

-1.09

6.075

-12.03

5.699

13.80

8.611

Low

-11.88

6.335

High

8.13

3.347

14.90

4.472

Intercept School level Language of instruction is Irish

*

p

^

Model 3: Bkgrd & formal PE

SE

533.43

4.834

9.03

6.445

-14.09

6.365

-16.96

7.138

9.08

8.992

2.63

4.743

11.65

3.244

p

NS

Participation in DEIS (Ref: Not in DEIS)

Rural

***

^

*

Parental support (Ref: medium)

Very high

***

Parental involvement in school activities (Ref: medium) Low or very low High or very high

***

Frequency of workshops for parents (Ref: 1-3 per yr) Never More than 3 Pupil level Gender (Ref: Boy)

7.43

2.207

-14.65

4.094

Short third lev

11.04

3.147

Degree or above

23.61

3.339

10.73

1.922

Parental

Educationa

4.85

2.119

-9.72

3.925

2.86

3.225

10.58

3.236

8.66

1.851

*

3.258 6.421

7.52

2.185

-14.32

4.091

10.89

3.140

23.93

2.981

10.89

1.910

*

***

(Ref: Upper sec)

Lower sec or below

Parental

***

-5.84 -11.86

employmenta

(Total jobs in household)

*** ***

*** ***

*** ***


25

Books at homea (Ref: 26-100) 10 or fewer

-45.00

3.929

-38.98

3.868

11 to 25

-17.31

2.944

-13.69

2.848

20.78

2.827

18.08

2.735

Sometimes

-13.08

3.079

-12.57

3.009

Never

-43.84

7.501

-40.56

7.308

-26.42

2.977

***

-11.03

2.364

***

Diploma

14.06

4.139

University degree or above

32.99

3.843

Never or almost never

-2.25

4.701

Daily

-7.59

2.341

Once or twice a month

10.93

5.080

Once or twice a week

6.25

4.339

Daily

2.23

3.863

Once or twice a week

-6.50

5.102

Daily

13.60

3.559

More than 100 Pupil speaks English at

homea

Pupil has TV in

***

3.938

-17.21

2.935

21.02

2.817

-12.92

3.085

-43.75

7.505

***

(Ref: Always)

Pupil has iPhonea (Ref: No) bedrooma

***

-44.25

(Ref: No)

***

***

Expectations for pupil's educationa (Ref: PLC or below) ***

Parent asks pupil what he is learning in school (Ref: monthly) **

Parent and pupil talk about schoolwork (Ref: Never) *

Parents ensure time for homework (Ref: monthly or less)

Note: aVariable has missing indicator, *** p