Chicago Public Schools and Student Achievement - SAGE Journals

URBAN/ STUDENT Sander EDUCATION ACHIEVEMENT / JANUARY 2001 IN CHICAGO SCHOOLS

CHICAGO PUBLIC SCHOOLS AND STUDENT ACHIEVEMENT WILLIAM SANDER DePaul University

This study examines whether students in the Chicago Public School System are at a disadvantage relative to students in suburban school systems and other school systems in Illinois after many background factors are taken into account. It is shown that grade school students in Chicago do as well as their counterparts elsewhere. However, high school students in Chicago have a much larger drop-out rate and lower test scores in reading after adjusting for background variables.

Former United States Secretary of Education once called Chicago’s Public School System “the worst in America.” To some extent, his perception of the quality of education in Chicago public schools was based on educational outcomes such as test scores and high school graduation rates. A few years ago, about one out of three public high schools in Chicago had average American College Testing (ACT) scores that placed them in the bottom 10% nationally. Furthermore, about half of the high school population dropped out before graduation (Sander, 1993). At least since the Coleman Report (Coleman et al., 1966), it has been well known that educational outcomes are strongly linked to family background. Students from disadvantaged backgrounds do not perform as well as students from more advantaged family backgrounds for a variety of reasons. Because the majority of the students in the Chicago Public School System are products of lowincome families, it is not surprising that academic achievement in AUTHOR’S NOTE: The author would like to thank two anonymous reviewers for their comments. He would also like to thank DePaul University for research support. A preliminary version of this article was presented at the American Educational Research Association’s annual meeting in San Diego in 1998. URBAN EDUCATION, Vol. 36 No. 1, January 2001 27-38 © 2001 Corwin Press, Inc.

27

28

URBAN EDUCATION / JANUARY 2001

Chicago is lower than achievement in, for example, the more affluent suburbs of Chicago. However, it is not necessarily the case that Chicago schools are less productive than suburban schools and other schools in Illinois if differences in student background, school resources, and other factors are taken into account. Regarding the effect of resources, in a review of many studies on the relationship between public expenditures on primary and secondary schooling and academic achievement, Hanushek (1986) concludes that resources are not a very important determinant of educational outcomes. He argues that lowering class size or paying teachers more has not resulted in appreciable improvements in education. Another widely cited review of some of the same studies comes to the conclusion that money matters (Hedges, Laine, & Greenwald, 1994). Expenditures per pupil in the Chicago Public School System are slightly higher on the average than average expenditures in the suburbs of Chicago and other school districts in Illinois. However, some suburbs of Chicago spend substantially more. The discrepancy between expenditures in Chicago schools and expenditures in affluent suburban schools has been documented elsewhere (e.g., in a legal suit regarding equalization, Committee for Educational Right v. Thompson, n.d.; Kozol, 1991). The objective of this article is to test whether schools in the Chicago Public School System have lower test scores and, in the case of high schools, higher drop-out rates than other public schools in Illinois after many background factors are taken into account. It is shown that Chicago grade schools are as effective as their nonChicago counterparts. This is not the case for Chicago high schools. Public high schools in Chicago are found to have higher drop-out rates, even after many background factors are taken into account. THE MODELS AND DATA

Third-, 6th-, and 10th-grade Illinois Goal Assessment Program (IGAP) test scores in mathematics and reading, high school drop-out rates, and percentage of high school seniors who take the ACT (as proxy for the percentage college bound) are estimated. All

Sander / STUDENT ACHIEVEMENT IN CHICAGO SCHOOLS

29

of the dependent variables are measured at the school level in 1996. The IGAP achievement test scores are the result of a 1985 state mandate that schools test students periodically and report the results to the public. The base year test scores have a mean value of 250, with a possible range of 0 to 500. One of the positive features of the IGAP test is that almost all of the students (more than 90%) take it. Special education students and students with limited Englishspeaking ability are excluded. It is important to note that more recently, the IGAP test has been replaced by a new achievement test. One of the concerns with the IGAP test was that, at least in the case of achievement in reading, it was not reliable. Thus, my results for reading test scores should be interpreted with caution. Although the IGAP test score in reading may be a precise measure of reading ability, it does not necessarily follow that this biases my results, because I am comparing reading scores in Chicago with other areas. That is, a bias would occur if reading test scores in Chicago were biased relative to other schools. In a related study, Hess (1999) reviews some of the issues in measuring achievement in Chicago. Ordinary least squares multiple regression analysis is used to estimate the dependent variables. Ordinary least squares enables one to measure the independent effects of numerous variables on the measures of academic achievement. It is important to take into account many factors that affect academic achievement so that academic achievement in Chicago is not confounded with other factors such as family background. The same model is estimated for each dependent variable. The model takes into account school-related variables, family-related variables, and community-related variables that might affect academic achievement. Estimates are undertaken for all schools in Illinois. A few schools are excluded because of either missing data or inaccurate data. In addition to adjusting for a Chicago location, I adjust for expenditures per pupil (measured at the school district level), school size, percentage Black in a school, percentage Hispanic in a school, percentage Asian in a school, percentage of a school’s population with limited English-speaking ability, percentage of students in school from low-income families, mobility rate

30


in a school, percentage taking the IGAP test (for the test score estimates), whether a school is in a unit district (relative to high school districts or elementary school districts), percentage urban in a school district, median household income in a school district, percentage of the adult population in a school district with at least 16 years of schooling (called “college”), and whether the school is located in a suburb of Chicago or in East St. Louis. These adjustments are made to control for the effects of family background, community background, and school resources. The variable called “mobility rate” is defined as the percentage of students who either transfer in or transfer out of a school during the school year. The variable called “low income” is defined as the percentage of students in a school who receive free or reduced price meals, live in families receiving public aid, live in foster homes supported with public funds, or live in institutions for neglected or delinquent children. Two data sources are merged for the estimates of achievement. The data on test scores, drop-out rates, the percentage taking the ACT, and the characteristics of schools are taken from the Illinois State Board of Education’s “1996 School Report Card Data.” The data from this source are at the school level, except for data on expenditures per pupil. Data on expenditures are measured at the district level; school-level data are not available. Data on the characteristics of school districts (median household income, the percentage of adults with 16 or more years of schooling, and the percentage urban) are taken from the United States Department of Education’s “School District Data Book” (1996). The schooldistrict data are used to adjust for variables that either are not measured very well at the school level or cannot be taken into account with the school-level data. For example, a variable such as percentage from low-income families that is available at the school level imprecisely measures the socioeconomic background of students in a school. Additional adjustments for educational attainment (percentage of adults with a college degree) and median household income in a school district enable one to take into account other factors that relate to family background because parents’ education and family income are not available at the school level. Selected


31

TABLE 1

Selected Summary Statistics (means) Variable Grade 3 Math Grade 6 math Grade 8 math Grade 10 math Grade 3 reading Grade 6 reading Grade 8 reading Grade 10 reading High school dropout (%) College bound (%) Expenditures per pupil ($) Grade 3 average class size Grade 6 average class size Grade 8 average class size Grade 10 average class size Average teacher’s salary ($) Low-income students (%)

All Illinois

Chicago

Chicago Suburbs

Downstate

277 263 263 257 238 234 223 219 6.5 63 5,947 23.4 24.4 23.9 20.3 40,984 26

224 213 220 182 180 181 189 162 15.5 59 6,941 23.1 23.9 24.4 23.1 43,867 83

310 300 302 286 270 266 252 238 3.7 67 6,486 24.4 25.3 24.6 20.9 45,092 16

292 283 289 265 258 259 247 228 5.1 60 4,776 22.6 23.7 22.7 18.4 34,782 27

SOURCE: Illinois State Board of Education (1996). NOTE: Data are weighted by school size. Downstate excludes the Chicago metropolitan area and the East St. Louis area.

summary statistics for the data set are provided (see Table 1). The data are presented for all public schools in Illinois and also for Chicago, suburbs of Chicago, and other locations in Illinois (called “downstate”). The data indicate much lower test scores and higher high school drop-out rates for Chicago. The data also indicate that the vast majority (83%) of students in Chicago schools are from low-income households, whereas a relatively small percentage of students in suburban and downstate schools are from low-income households (16% and 27%, respectively). THE RESULTS

In Table 2, ordinary least squares estimates of test scores in mathematics are presented by grade. The regression coefficients associated with the independent variables and their standard errors are reported. The ratio of the regression coefficient to its standard

32

URBAN EDUCATION / JANUARY 2001 TABLE 2

Ordinary Least Squares Estimates (and standard errors) of Math Scores by Grade Grade 3 Expenditures per pupil School size Black (%) Hispanic (%) Asian (%) Limited English (%) Low income (%) Mobility rate Taking test (%) Unit district Urban (%) Median income College (%) Chicago Chicago suburb East St. Louis Constant R2 F N

Grade 6

Grade 8

–.0001 .0019* .0025*** (.001) (.00097) (.0009) –.03*** –.015*** .001 (.01) (.004) (.003) –.60*** –.57*** –.57*** (.06) (.06) (.06) –.30*** –.17* –.22** (.11) (.10) (.09) .98*** .78*** 1.04*** (.21) (.19) (.17) –.54 –.35** –.38** (.19) (.17) (.17) –.73*** –.80*** –.82*** (.07) (.07) (.07) –.48*** –.38*** –.50*** (.09) (.09) (.09) –.56*** .17 .22** (.12) (.11) (.10) –4.17* –5.2** –5.2** (2.2) (2.2) (2.1) 11.3*** 3.7 –.35 (3.4) (3.1) (3.1) –.0006*** –.0006*** –.00026* (.0002) (.0002) (.00014) 134.8*** 146.5*** 116.0*** (14.0) (14.5) (14.3) 6.54 5.4 8.1* (4.4) (4.1) (4.3) –1.4 –.95 –7.3** (3.4) (3.3) (3.3) 14.8 49.2*** 7.0 (9.5) (8.4) (12.8) 390.0 299.6 287.7 .59 .67 .77 195 223 270 2,211 1,757 1,330

Grade 10 .001 (.001) .003 (.003) –.85*** (.09) –.57*** (.19) 1.09*** (.39) –1.17** (.49) –.21* (.12) –.38*** (.10) .26 (.17) 1.93 (3.9) .30 (4.4) .0002 (.0002) 137.7*** (23.0) –6.6 (7.4) 2.5 (5.0) –15.6 (20.0) 223.9 .70 88.5 618

*p < .10. **p < .05. ***p < .01.

error provides t statistics that indicate whether the coefficient is significantly different from zero. Other summary statistics are also presented in the table, including F statistics and the R-squared for the regression. The F statistic indicates whether at least some of the


33

independent variables are important determinants of the dependent variable. That is, the F statistic indicates whether the overall regression is significantly different from zero. The R-squared measures how much of the variation in the dependent variable is explained by the independent variables; it is often considered an indicator of the goodness of fit of the model. The coefficients for schools in Chicago are insignificant in three cases and positive and significant at the 10% level in one case (Grade 8 scores). In general, the results for Chicago schools suggest that the Chicago Public School System is as efficient as other public school systems in Illinois in terms of preparing students in mathematics. The other significant determinants of test scores in mathematics include negative Black, Hispanic, limited-English ability, mobility, school size (for Grades 3 and 6 only), unit district (for Grades 6 and 8), percentage taking test (only Grade 3), Chicago suburb (only Grade 8), and median income effects (for Grades 3, 6, and 8) and positive Asian, percentage taking test (only Grade 8), and college effects. Furthermore, in two cases (Grades 6 and 8), the coefficient on expenditures per pupil is positive and significant. The results in Table 2 also indicate that all of the regressions are highly significant, as indicated by the F statistics. Furthermore, more than half of the variation in the dependent variables is explained by the independent variables. This suggests a favorable goodness of fit for the models. Estimates of reading test scores by grade are presented in Table 3. The results for Chicago schools indicate no effect at the third-grade level, a positive effect at the 6th- and 8th-grade levels, and a negative effect at the 10th-grade level. All of the coefficients for suburban schools are insignificant. Once again, the results suggest that Chicago grade schools are as efficient, if not more efficient, as public schools elsewhere in Illinois in preparing students in reading after many background factors are taken into account. The other significant determinants of reading scores include negative school size (for grade schools only), Black, Hispanic (for Grade 10 only), limited English ability, mobility, unit district (for Grade 8), percentage taking test (Grade 3 only), and median income effects (for grade schools only) and positive Asian, percentage taking test (Grade 8 only), and college effects. For reading test scores, all of

34


Ordinary Least Squares Estimates (and standard errors) of Reading Scores by Grade Grade 3 Expenditures per pupil School size Black (%) Hispanic (%) Asian (%) Limited English (%) Low income (%) Mobility rate Taking test (%) Unit district Urban (%) Median income College (%) Chicago Chicago suburb East St. Louis Constant R2 F N

Grade 6

Grade 8

.001 –.0004 .001 (.001) (.0009) (.001) –.023*** –.016*** –.009*** (.004) (.003) (.003) –.48*** –.47*** –.48*** (.04) (.05) (.05) –.12 –.09 –.22 (.08) (.09) (.08) .76*** .83*** .50*** (.16) (.16) (.16) –.58*** –.68*** –.53*** (.15) (.15) (.15) –.83*** –.95*** –.73*** (.06) (.06) (.06) –.47*** –.43*** –.41*** (.07) (.08) (.08) –.40*** .10 .29*** (.09) (.10) (.09) –1.8 –.73 –4.5** (1.7) (2.0) (1.9) 2.98 4.3 –3.9 (2.6) (2.7) (2.9) –.0006*** –.0008*** –.0005*** (.0001) (.0001) (.0001) 135.1*** 139.9*** 115.7*** (11.0) (10.9) (13.3) –.20 9.3*** 17.3*** (3.4) (3.59) (4.0) –3.4 –3.7 –4.6 (2.7) (2.9) (3.0) 12.1 39.9*** 9.0 (7.5) (7.4) (11.9) 349.0 297.3 246.9 .71 .73 .72 328 293 213 2,211 1,758 1,330

Grade 10 .0001 (.001) –.001 (.002) –.57*** (.09) –.31* (.17) 1.12** (.36) –1.86*** (.46) –.14** (.11) –.50*** (.09) –.11 (.16) 1.9 (3.6) .01 (4.0) –.0002 (.0002) 145.5*** (21.0) –11.2* (6.8) –.68 (4.6) –26.7 (18.0) 236.1 .62 61.4 617

*p < .10. **p < .05. ***p < .01.

the coefficients for expenditures per pupil are insignificant. The F statistics for all of the estimates of reading scores indicate that the regressions are significant. Furthermore, the R-squared statistic indicates that more than 60% in the variance in 10th-grade test


35

scores is explained by the independent variables, whereas slightly more than 70% of the variance in grade school test scores is explained by the model. The results of estimates of the percentage taking the ACT and high school drop-out rates are presented in Table 4. For the percentage taking the ACT, schools in Chicago have a significant positive effect. Thus, students who make it to their senior year in high school in Chicago are more likely to take the ACT. The other significant determinants of the percentage taking the ACT include positive Asian and college effects and negative low-income and mobility effects. Although the F statistic indicates that the estimates of the percentage taking the ACT are significant, only a little more than a third of the variance in the dependent variable is explained by the model (the R-squared is .37). Schools in Chicago have a significant positive effect on the dropout rate (see Table 4). Furthermore, the magnitude of the Chicago school effect is relatively large—about five and one-half percentage points higher than the average after other background factors are taken into account. Thus, a relatively high percentage of students in Chicago public high schools do not graduate, after controlling for many other factors. The other significant determinants of the drop-out rate include positive limited English ability, low income, mobility, and urban effects and negative Asian and college effects. The F statistic for the drop-out rate estimate indicates that the regression is significant. Furthermore, the R-squared statistic indicates that a relatively large percentage of the variation in the drop-out rate (76%) is explained by the model. CONCLUSION

The results indicate that public grade schools in Chicago perform as well as grade schools elsewhere in Illinois in terms of preparing students in mathematics and reading, after many background factors are taken into account. However, a significantly higher percentage of high school students in Chicago do not complete high school. Furthermore, reading test scores in Chicago high schools are significantly lower. Thus, one might conclude that Chi-

36


Estimates (and standard errors) of Other High School Achievement Outcomes Taking ACT Expenditures per pupil School size Black (%) Hispanic (%) Asian (%) Limited English (%) Low income (%) Mobility rate Unit district Urban (%) Median income College (%) Chicago Chicago suburb East St. Louis Constant R2 F N

.0007 (.0005) .0002 (.001) –.03 (.04) –.11 (.08) .50*** (.17) –.31 (.22) –.23*** (.05) –.27*** (.04) 2.0 (1.7) –.98 (1.9) –.00016 (.00011) 56.1*** (10.1 ) 14.4*** (3.3) –2.4 (2.2) –2.5 (8.6) 61.4 .37 23.3 614

Drop-Out Rate .0002 (.0001) –.0001 (.0003) –.01 (.01) –.03 (.02) –.08* (.05) .11* (.06) .024* (.015) .30*** (.01) –.53 (.46) 1.53*** (.52) .00001 (.00003) –5.61*** (2.69) 5.48*** (.88) –1.11* (.59) 7.64*** (2.3) –.74 .76 130.8 619

*p < .10. **p < .05. ***p < .01.

cago grade schools are about as effective as public grade schools elsewhere in Illinois, whereas public high schools in Chicago are relatively less effective. This implies that strategies to improve education at one level in Chicago might not work at another level. It


37

could also imply that the positive effects of Chicago school reform have not had enough time to work at the high school level. To some extent, the more favorable results for grade schools might be the result of school reform in Chicago. School reform legislation that was enacted in 1988 and implemented in 1990 is resulting in more school-based management of education in Chicago. Schools now control most of the money that is allocated to them. Downes and Haeres (1994) and Hess (1999) show that after a decline in outcomes at the beginning of the reform period, Chicago school reform is improving student outcomes. Hess (1999) also makes the important point that although there have been improvements in Chicago, test scores are still very low and further improvement is needed. Improving test scores in Chicago is particularly important because recent studies suggest that earnings by young adults are increasingly linked to measures of cognitive ability such as test scores in mathematics (Murnane, Willett, & Levy, 1995). The results suggest that achievement levels are higher in smaller grade schools and grade schools that are not in unit districts. This implies that school achievement might be improved if grade schools in Chicago were smaller and not part of a larger system that includes high schools. The benefits of reducing the size of grade schools would have to be weighed against higher per unit costs that might be associated with a smaller size. Although it was not the focus of this article, my results suggest that at best, resources have modest positive effects on achievement in mathematics at the grade school level. I could not show any relationship between expenditures and student achievement at the high school level. One of the problems in estimating resource effects is that expenditures are not necessarily an exogenous determinant of achievement. To some extent, expenditures are endogenous with achievement. For example, schools and school districts with more at-risk students receive more resources. Thus, resources are partly a result of the ability of students in a school or school system. For this reason, relationships (and lack of relationships) between expenditures (and expenditure-related variables) should be interpreted with caution. In a related study, Sander (1999) examines this issue in more detail.

38


REFERENCES Coleman, J. S., Campbell, E. Q., Hobson, C. J., McPartland, J., Mood, A. M., Weinfield, F. D., & York, R. L. (1966). Equality of educational opportunity. Washington, DC: Government Printing Office. Committee for Educational Rights v. Thompson [Synopsis of the lawsuit]. (n.d.). Chicago: Hinshaw, Culbertson, Moelmann, Hoban and Fuller. Downes, T. A., & Haeres, J. L. (1994, October 26-27). An analysis of the Chicago school reform on student performance. In the Midwest Approaches to School Reform proceedings of the Federal Reserve Bank of Chicago conference. Hanushek, E. A. (1986). The economics of schooling. Journal of Economic Literature, 24, 1141-1177. Hedges, L. V., Laine, R. D., & Greenwald, R. (1994). Does money matter? Educational Researcher, 2, 5-14. Hess, G. A., (1999). Understanding achievement (and other) changes under Chicago school reform. Educational Evaluation and Policy Analysis, 21, 67-84. Illinois State Board of Education. (1996). 1996 school report card data. Springfield: Author. Kozol, J. (1991). Savage inequalities. New York: Crown. Murnane, R. J., Willett, J. B., & Levy, F. (1995). The growing importance of cognitive skills in wage determination. Review of Economics and Statistics, 77, 251-266. Sander, W. (1993). Expenditures and student achievement in Illinois. Journal of Public Economics, 52, 403-416. Sander, W. (1999). Endogenous expenditures and student achievement. Economics Letters, 64, 223-231. U.S. Department of Education. (1996). School district data book. Arlington, VA: The Mesa Group.