NATIONAL CENTER FOR EDUCATION STATISTICS Technical Report
January 2001
Technical Report and Data File User’s Manual For the 1992 National Adult Literacy Survey Irwin Kirsch, Kentaro Yamamoto, Norma Norris, Donald Rock, Ann Jungeblut, and Patricia O’Reilly Educational Testing Service
Anne Campbell
Diné College
Lynn Jenkins
Wordsworth Writing and Editing
Andrew Kolstad
National Center for Education Statistics
Martha Berlin, Leyla Mohadjer, Joseph Waksberg, Huseyin Goksel, John Burke, Susan Rieger, James Green, and Merle Klein Westat, Inc.
Peter Mosenthal
Syracuse University
Stéphane Baldi
American Institutes for Research
Andrew Kolstad, Project Officer National Center for Education Statistics
U.S. Department of Education Office of Education Research and Improvement
NCES 2001-457
U.S. Department of Education Richard W. Riley Secretary Office of Educational Research and Improvement C. Kent McGuire Assistant Secretary National Center for Education Statistics Gary W. Phillips Acting Commissioner The National Center for Education Statistics (NCES) is the primary federal entity for collecting, analyzing, and reporting data related to education in the United States and other nations. It fulfills a congressional mandate to collect, collate, analyze, and report full and complete statistics on the condition of education in the United States; conduct and publish reports and specialized analyses of the meaning and significance of such statistics; assist state and local education agencies in improving their statistical systems; and review and report on education activities in foreign countries. NCES activities are designed to address high priority education data needs; provide consistent, reliable, complete, and accurate indicators of education status and trends; and report timely, useful, and high quality data to the U.S. Department of Education, the Congress, the states, other education policymakers, practitioners, data users, and the general public. We strive to make our products available in a variety of formats and in language that is appropriate to a variety of audiences. You, as our customer, are the best judge of our success in communicating information effectively. If you have any comments or suggestions about this or any other NCES product or report, we would like to hear from you. Please direct your comments to: National Center for Education Statistics Office of Educational Research and Improvement U.S. Department of Education 1990 K Street, NW Washington, DC 20006–5574
January 2001 The NCES World Wide Web Home Page is: http://nces.ed.gov The NCES World Wide Web Electronic Catalog is: http://nces.ed.gov/pubsearch/index.asp Suggested Citation U.S. Department of Education. National Center for Education Statistics. Technical Report and Data File User’s Manual For the 1992 National Adult Literacy Survey, NCES 2001–457, by Irwin Kirsch, Kentaro Yamamoto, Norma Norris, Donald Rock, Ann Jungeblut, Patricia O’Reilly, Martha Berlin, Leyla Mohadjer, Joseph Waksberg, Huseyin Goksel, John Burke, Susan Rieger, James Green, Merle Klein, Anne Campbell, Lynn Jenkins, Andrew Kolstad, Peter Mosenthal, and Stéphane Baldi. Project Officer: Andrew Kolstad. Washington DC: 2001. For ordering information on this report, write U.S. Department of Education ED Pubs P.O. Box 1398 Jessup, MD 20794-1398 Or call toll free 1-877-4ED-PUBS Contact: Andrew Kolstad (202) 502-7374 E-mail:
[email protected]
TABLE OF CONTENTS CHAPTER 1: THE NATIONAL ADULT LITERACY SURVEY: AN OVERVIEW................................................1 1.1 Introduction ..................................................................................................................................1 1.2 Defining Literacy .........................................................................................................................3 1.3 The Sample...................................................................................................................................3 1.4 Weighting.....................................................................................................................................4 1.5 The Survey Instrument: Measuring Literacy................................................................................5 1.6 Field Operations ...........................................................................................................................7 1.7 Data Processing and Missing Data...............................................................................................8 1.8 Scaling and Proficiency Estimates ...............................................................................................9 1.9 Establishing Literacy Levels ........................................................................................................10 CHAPTER 2: SAMPLE DESIGN .......................................................................................................................11 2.1 Overview ......................................................................................................................................11 2.2 Sampling for the National Component.........................................................................................13 2.2.1 First-Stage Sample .......................................................................................................14 2.2.1.1 Westat’s master sample of PSUs..................................................................14 2.2.1.2 Selecting the sample of PSUs for the national component ..........................14 2.2.2 Second-Stage Sample—Selecting Census Blocks (Segments) ....................................21 2.2.2.1 Measures of size and sampling rates ............................................................22 2.2.2.2 Minimum segment size ................................................................................23 2.2.2.3 Segment sample selection ............................................................................24 2.2.2.4 TIGER maps.................................................................................................24 2.2.2.5 Listing sample segments ..............................................................................25 2.2.3 Third-Stage Sample—Selecting Housing Units...........................................................26 2.2.3.1 Within-segment sampling rate .....................................................................26 2.2.3.2 Overall probabilities of selection .................................................................28 2.2.3.3 Procedures for selecting missed structures and missed dwelling units ........28 2.2.4 Fourth-Stage Sample—Selecting Persons Age 16 or Older.........................................29 2.3 The Non-Incentive Sample...........................................................................................................30 2.4 Sampling for the State Literacy Surveys ......................................................................................30 2.4.1 Sample of PSUs............................................................................................................31 2.4.2 Sample of Segments .....................................................................................................31 2.4.3 Sample of Housing Units .............................................................................................32 2.4.4 Sample of Persons ........................................................................................................32 2.5 Weighted and Unweighted Response Rates.................................................................................32 2.6 Sampling for the Prison Survey....................................................................................................34 2.6.1 Sample of Correctional Facilities.................................................................................34 2.6.1.1 Sampling frame and selection of correctional facilities ...............................35 2.6.2 Selection of Inmates Within Facilities .........................................................................37 CHAPTER 3: WEIGHTING AND POPULATION ESTIMATES ...........................................................................39 3.1 Goals of Weighting ......................................................................................................................39 3.2 Calculating Sample Weights for the Household Population ........................................................41 3.2.1 Preliminary Steps in Weighting ...................................................................................41 3.2.2 Computing Base Weights.............................................................................................43 3.2.3 Nonresponse Adjustments and Poststratification .........................................................44 3.2.4 Compositing Data from the National and State Components ......................................45 3.2.4.1 Composite estimation procedure..................................................................45
i
TABLE OF CONTENTS — CONTINUED 3.2.4.2 Deriving the PSU design effect....................................................................48 3.2.4.3 Estimating composite factors .......................................................................49 3.2.5 Computing Final Weight—Poststratification Through Raking Ration Adjustments.............................................................................................................54 3.3 Replicated Weights for Variance Estimation in the Household Population.................................55 3.3.1 Household Sample Replication for the National Component ......................................57 3.3.2 Household Sample Replication for the State Component ............................................57 3.3.3 Final Household Sample Replication for the National and State Components............58 3.4 Calculating Sample Weights for the Prison Population ...............................................................58 3.4.1 Computing Inmate Base Weights.................................................................................58 3.4.2 Nonresponse Adjustments............................................................................................62 3.4.2.1 Facility nonresponse adjustment ..................................................................62 3.4.2.2 Inmate nonresponse adjustment ...................................................................63 3.4.3 Poststratification Procedures ........................................................................................64 3.4.4 Final Inmate Weights ...................................................................................................67 3.5 Replication Weights for Variance Estimation in the Prison Population ......................................68 CHAPTER 4: DEVELOPMENT OF THE SURVEY INSTRUMENTS ....................................................................70 4.1 Conceptual Framework ................................................................................................................70 4.2 The Scope of the Background Questionnaire...............................................................................72 4.2.1 General and Language Background .............................................................................73 4.2.2 Educational Background and Experiences ...................................................................73 4.2.3 Political and Social Participation .................................................................................74 4.2.4 Labor Force Participation.............................................................................................74 4.2.5 Literacy Activities and Collaboration ..........................................................................75 4.2.6 Demographic Information ............................................................................................75 4.2.7 Prison Survey Background Questionnaire ...................................................................76 4.2.8 Spanish Versions of the Questionnaires.......................................................................76 4.3 Development of the Simulation Tasks .........................................................................................76 4.3.1 Organizing Framework for Task Development............................................................77 4.3.2 Materials/Structures ..................................................................................................... 77 4.3.3 Adult Contexts/Content................................................................................................78 4.3.4 Processes/Strategies .....................................................................................................79 4.3.5 Task Difficulty .............................................................................................................87 4.3.6 Development of Scoring Guides ..................................................................................88 4.3.7 Assembling the Tasks for Administration....................................................................89 CHAPTER 5: THE HOUSEHOLD SURVEY .......................................................................................................93 5.1 Overview ......................................................................................................................................93 5.2 Listing ..........................................................................................................................................94 5.2.1 Staff Organization for Listing ......................................................................................94 5.2.2 Training Listers ............................................................................................................95 5.2.3 Listing Materials ..........................................................................................................95 5.2.4 The Listing Operation ..................................................................................................96 5.2.5 Quality Control Procedures..........................................................................................97 5.2.5.1 Quality control of listing sheets ...................................................................97 5.2.5.2 Quality control of the listing operation ........................................................97 5.3 Data Collection Instruments and Interviewer Materials...............................................................99 5.3.1 The Screener.................................................................................................................99 5.3.2 Interview Guides for Exercise Booklets.......................................................................101 5.3.3 Non-interview Report Forms .......................................................................................101
ii
TABLE OF CONTENTS — CONTINUED 5.3.4 Interviewer Manuals.....................................................................................................104 5.3.5 Field Aids .....................................................................................................................104 5.3.5.1 Aids used for locating and contacting respondents ......................................104 5.3.5.2 Aids used for obtaining respondent cooperation..........................................105 5.3.5.3 Aids used during the interview.....................................................................106 5.4 Field Organization and Training ..................................................................................................106 5.4.1 Field Organization........................................................................................................106 5.4.1.1 Lines of responsibility..................................................................................106 5.4.1.2 Interviewer recruitment ................................................................................107 5.4.2 Training ........................................................................................................................109 5.4.2.1 Supervisor training .......................................................................................109 5.4.2.2 Interviewer training......................................................................................110 5.4.2.3 Editor training ..............................................................................................114 5.5 Field Operations ...........................................................................................................................114 5.5.1 General Approach to the Field Effort...........................................................................115 5.5.2 Schedule and Production..............................................................................................116 5.5.3 Reporting Systems........................................................................................................117 5.5.3.1 Automated Survey Control System (ASCS) ................................................117 5.5.3.2 Interviewer reports to the supervisor............................................................118 5.5.3.3 Supervisor reports to the home office ..........................................................118 5.5.3.4 Home office staff reports to ETS and to NCES ...........................................119 5.6 Quality Control of Data Collection ..............................................................................................119 5.6.1 Introduction ..................................................................................................................119 5.6.2 Editing ..........................................................................................................................119 5.6.3 Validation.....................................................................................................................121 5.6.4 Observation ..................................................................................................................122 5.6.5 Supervisor Observations ..............................................................................................123 5.7 Response Rates.............................................................................................................................123 5.7.1 Reasons for Non-response............................................................................................125 5.7.2 Characteristics of Non-respondents .............................................................................126 5.7.3 Discussion ....................................................................................................................128 CHAPTER 6: THE PRISON SURVEY................................................................................................................131 6.1 Sample Design..............................................................................................................................131 6.2 Gaining Cooperation ....................................................................................................................132 6.3 Interviewer Selection and Training ..............................................................................................133 6.4 Data Collection.............................................................................................................................134 6.5 Quality Control.............................................................................................................................134 CHAPTER 7: PROCESSING THE DATA ...........................................................................................................136 7.1 Receipt Control ............................................................................................................................136 7.1.1 Screener........................................................................................................................136 7.1.2 Background Questionnaire...........................................................................................137 7.1.3 Exercise Envelope........................................................................................................137 7.2 Coding and Scoring......................................................................................................................138 7.2.1 Coding Background Questionnaires.............................................................................138 7.2.2 Scoring Simulation Tasks.............................................................................................139 7.3 Data Entry ....................................................................................................................................140 7.4 Editing and Quality Control .........................................................................................................141
iii
TABLE OF CONTENTS — CONTINUED
CHAPTER 8: ESTIMATING LITERACY PROFICIENCIES WITH AND WITHOUT COGNITIVE DATA ............142 8.1 The Normal Treatment of Missing Cognitive Data......................................................................143 8.1.1 Omitted Answers and Questions Not Reached ............................................................143 8.1.2 Statistical Imputation Through Scaling........................................................................144 8.2 Reasons Cognitive Data Were Missing/Not Reached..................................................................145 8.2.1 Non-interview Reports and Low Literacy Skills..........................................................151 8.2.2 Internal Evidence for the Validity of Reasons .............................................................153 8.3 Using ‘Reasons’ to Improve Treatment of Missing Cognitive Data............................................155 8.3.1 Five Logical Imputation Methods Considered.............................................................156 8.3.2 Five Methods Applied to 1991 Field Test Data ...........................................................158 8.3.3 The Method Selected....................................................................................................163 8.4 Final Evaluation ...........................................................................................................................164 CHAPTER 9: SCALING AND PROFICIENCY ESTIMATES ...............................................................................165 9.1 Scaling..........................................................................................................................................165 9.2 Scaling Methodology ...................................................................................................................168 9.2.1 The Scaling Model .......................................................................................................168 9.2.2 Design for Linking the 1992 Scales to the 1985 Scales ...............................................169 9.2.3 Item Parameter Estimation ...........................................................................................170 9.3 Proficiency Estimation Using Plausible Values...........................................................................177 9.3.1 Generating Proficiency Scores .....................................................................................177 9.3.2 Linking the 1992 Scale to the 1985 Scale ....................................................................182 9.3.3 Evaluation of Differential Group Performance ............................................................183 9.4 Statistical Tests.............................................................................................................................187 9.4.1 Analysis of Plausible Values........................................................................................187 9.4.2 Partitioning the Estimation Error Variance: A Numerical Example ............................188 9.4.3 Minimum Sample Sizes for Reporting Subgroup Results............................................190 9.4.4 Estimates of Standard Errors with Large Mean Squared Errors ..................................190 CHAPTER 10: THE ROLE OF INCENTIVES IN LITERACY SURVEY RESEARCH............................................191 10.1 Literature Review.......................................................................................................................191 10.2 The 1991 Field Test....................................................................................................................193 10.2.1 Field Test Design .......................................................................................................193 10.2.2 Summary of Field Test Results ..................................................................................194 10.2.3 Field Test Response Rates..........................................................................................196 10.2.4 Representation of the Target Population in the Field Test .........................................198 10.2.5 Relationship Between Incentive Level, Self-Selection, and Performance in the Field Test...................................................................................................................200 10.2.6 Survey Costs for the Field Test ..................................................................................208 10.2.7 Conclusions from the 1991 Field Test .......................................................................209 10.3 The 1992 Incentive Experiment .................................................................................................210 10.3.1 Sample Design for the 1992 National Adult Literacy Survey....................................210 10.3.2 1992 Incentive Experiment Design ............................................................................211 10.3.3 Analysis of Response Rates .......................................................................................212 10.3.3.1 Screener......................................................................................................213 10.3.3.2 Background questionnaire and exercise booklet ........................................214 10.4 Summary and Conclusion ..........................................................................................................217
iv
TABLE OF CONTENTS — CONTINUED
CHAPTER 11: EVALUATION OF SAMPLE DESIGN AND COMPOSITE ESTIMATION .....................................219 11.1 Estimation Methods....................................................................................................................220 11.1.1 Calculating Within- and Between-PSU Variance ......................................................222 11.2 The National Sample Design Evaluation ...................................................................................223 11.2.1 Computing the Design Effect.....................................................................................228 11.3 Composite Estimation ................................................................................................................237 11.3.1 Estimating Compositing Factors Using the National Adult Literacy Survey Data....238 11.3.2 Evaluating the National Adult Literacy Survey Compositing Factors.......................248 11.3.3 Compositing Factors for a Future Assessment of Adult Literacy ..............................248 CHAPTER 12: CONSTRUCT VALIDITY OF THE ADULT LITERACY SCALES ................................................262 12.1 Data and Models ........................................................................................................................263 12.1.1 Models for Four Restricted Adult Literacy Samples..................................................263 12.1.2 Models for Seven GED/Adult Literacy Samples .......................................................266 12.2 Method .......................................................................................................................................269 12.3 Results and Discussion...............................................................................................................271 12.3.1 Restricted Adult Literacy Samples.............................................................................271 12.3.2 GED/Adult Literacy Samples.....................................................................................275 12.4 Conclusions ................................................................................................................................278 CHAPTER 13: INTERPRETING THE ADULT LITERACY SCALES AND LITERACY LEVELS ..........................279 13.1 Background ................................................................................................................................279 13.2 The 1985 Young Adult Literacy Assessment.............................................................................280 13.2.1 Dimensionality of Literacy Skills ..............................................................................282 13.2.2 Difficulty of Literacy Tasks .......................................................................................284 13.2.3 Prose Comprehension Scale .......................................................................................285 13.2.4 Document Literacy Scale ...........................................................................................287 13.2.5 Quantitative Literacy Scale ........................................................................................290 13.3 Enhancing Understanding of Task Difficulty ............................................................................293 13.4 The 1990 Survey of the Literacy of Job-Seekers .......................................................................296 13.4.1 Prose Literacy.............................................................................................................297 13.4.1.1 Prose variables ...........................................................................................297 13.4.1.2 Examples of prose literacy tasks ................................................................299 13.4.1.3 Coding the 1985 and 1990 prose literacy tasks..........................................303 13.4.1.3.1 Type of match ............................................................................303 13.4.1.3.2 Type of information ...................................................................306 13.4.1.3.3 Plausibility of Distractors...........................................................307 13.4.1.3.4 Readability .................................................................................308 13.4.1.4 Codes for all 1985 and 1990 prose literacy tasks.......................................308 13.4.1.5 Validity evidence for the prose scale .........................................................310 13.4.2 Documents Literacy ...................................................................................................311 13.4.2.1 Document variables....................................................................................311 13.4.2.2 Examples of document literacy tasks .........................................................312 13.4.2.3 Coding the 1985 and 1990 document literacy tasks...................................314 13.4.2.3.1 Type of match ............................................................................314 13.4.2.3.2 Plausibility of distractors............................................................317 13.4.2.3.3 Type of information ...................................................................318 13.4.2.3.4 Structural complexity .................................................................318 13.4.2.4 Codes for all 1985 and 1990 document literacy tasks................................320 13.4.2.5 Validity evidence for the document scale ..................................................322
v
TABLE OF CONTENTS — CONTINUED 13.4.3 Quantitative Literacy..................................................................................................323 13.4.3.1 Quantitative variables.................................................................................323 13.4.3.2 Examples of quantitative literacy tasks ......................................................324 13.4.3.3 Coding the 1985 and 1990 Quantitative Literacy Tasks ............................325 13.4.3.3.1 Specificity of operation ..............................................................327 13.4.3.3.2 Type of calculation.....................................................................328 13.4.3.3.3 Type of match, plausibility of distractors, and structural complexity.................................................................................................329 13.4.3.4 Codes for all 1985 and 1990 quantitative literacy tasks.............................329 13.4.3.5 Validity evidence for the quantitative scale ...............................................330 13.4.4 Establishing Proficiency Levels .................................................................................331 13.5 The 1992 National Adult Literacy Survey .................................................................................336 13.5.1 Prose Literacy.............................................................................................................337 13.5.2 Document Literacy.....................................................................................................340 13.5.3 Quantitative Literacy..................................................................................................343 13.5.4 Levels of Prose, Document, and Quantitative Literacy..............................................346 13.6 Conclusion..................................................................................................................................347 CHAPTER 14: LITERACY LEVELS AND THE 80 PERCENT RESPONSE PROBABILITY CONVENTION..........348 14.1 Prose Literacy Tasks and Their Characteristics .........................................................................349 14.2 The Need for a Response Probability Convention .....................................................................352 14.3 Literacy Tasks and Literacy Levels............................................................................................357 14.3.1 Predictive Factors and RP80 Task Difficulty.............................................................358 14.3.2 Predictive Factors, Task Difficulty, and the Response Probability Convention ........360 14.3.3 Alternative Cut Points between Literacy Levels........................................................361 14.4 Distribution of Adults Among Alternative Literacy Levels.......................................................363 14.5 Selecting an Appropriate Response Probability Convention .....................................................367 14.6 Conclusions ................................................................................................................................369 CHAPTER 15: WORKING WITH SPSS AND SAS ...........................................................................................371 15.1 The Electronic Code Book and SPSS and SAS Control Statements..........................................371 15.2 Creating SPSS System Files.......................................................................................................372 15.3 Creating SAS System Files ........................................................................................................373 15.4 Computing the Estimated Variance of a Statistic Using Jackknife Methods in SPSS or SAS......375 REFERENCES ..................................................................................................................................................380 APPENDIX A: ESTIMATED ITEM PARAMETERS APPENDIX B: CONDITIONING VARIABLES APPENDIX C: GAMMAS APPENDIX D: RP80S AND ITEM PROBABILITIES X 100 APPENDIX E: NON-INTERVIEW REPORT FORM APPENDIX F: FORM FOR INTERVIEWER’S OBSERVATION ON THE EXERCISE BOOKLET AND THE OBSERVATIONS APPENDIX G: ENGLISH BACKGROUND QUESTIONNAIRE FOR HOUSEHOLDS
vi
TABLE OF CONTENTS — CONTINUED
APPENDIX H: ENGLISH BACKGROUND QUESTIONNAIRE FOR PRISONS APPENDIX I: DERIVED VARIABLES APPENDIX J: SPECIAL CODES FOR CONTINUOUS VARIABLES APPENDIX K: COUNTRY OF BIRTH CODES APPENDIX L: NOTES ON SCORING APPENDIX M: NOTES ON VARIABLES APPENDIX N: IMPACT OF TREATMENT ON DISTRIBUTION OF SCALE SCORES APPENDIX O: ESTIMATED COMPOSITE FACTORS FOR A SELECTED SET OF DEMOGRAPHIC VARIABLES FOR ILLINOIS, INDIANA, LOUISIANA, NEW JERSEY, NEW YORK, OHIO, PENNSYLVANIA, TEXAS, AND WASHINGTON APPENDIX P: ELECTRONIC CODE BOOK FOR WINDOWS USER’S MANUAL APPENDIX Q: STANDARD ERRORS FOR TABLE 8.2 APPENDIX R: STANDARD ERROR TABLES FOR CHAPTER 10
vii
Chapter 1 THE NATIONAL ADULT LITERACY SURVEY: AN OVERVIEW Lynn Jenkins, Wordsworth Writing and Editing (formerly with Educational Testing Service) Stéphane Baldi, American Institutes for Research 1.1 INTRODUCTION The Adult Education Amendments of 1988 required the U.S. Department of Education to submit a report to Congress defining literacy and measuring the nature and extent of literacy among adults in the nation. To satisfy these requirements, the National Center for Education Statistics (NCES) and the Division of Adult Education and Literacy planned a nationally representative household survey to assess the literacy skills of the adult population in the United States. In September 1989, NCES awarded a four-year contract for that purpose to Educational Testing Service (ETS) with a subcontract to Westat, Inc., for sampling and field operations. The National Adult Literacy Survey is the third and largest assessment of adult literacy funded by the Federal government and conducted by ETS. The two previous efforts included a 1985 household survey of the literacy skills of 21- to 25-year-olds, funded by the U.S. Department of Education, and a 1989-90 survey of the literacy proficiencies of job seekers, funded by the U.S. Department of Labor. In 1992, nearly 13,600 individuals age 16 and older, randomly selected to represent the adult population in this country, were surveyed in their homes. In addition, about 1,000 randomly selected adults age 16 through 65 were surveyed in each of 11 states that chose to participate in a concurrent State Adult Literacy Survey designed to produce state-level results comparable to the national data. In addition to the household samples, 1,147 inmates from 87 state and Federal prisons were randomly surveyed to represent the inmate population in the United States. Their participation helped to provide better estimates of the literacy levels of the total population and made it possible to report on the literacy proficiencies of this important segment of society. Each individual who participated in the National and State Adult Literacy Surveys was asked to provide background demographic information and to complete a booklet of literacy tasks. These tasks were carefully constructed to measure respondents’ ability to read and use a wide array of printed and written materials. The survey results comprise an enormous set of data that includes more than a million responses to the literacy tasks and background questions. More important than the size of the database, however, is the fact that it provides information that is essential to understanding this nation’s literacy resources. Specifically, the National Adult Literacy Survey data give policy makers, business and labor leaders,
1
educators, researchers, and citizens vital information on the condition of literacy in the United States. The survey results can be used to: • • • • • •
Describe the levels of literacy demonstrated by the adult population as a whole and by adults in various subgroups, including those targeted as “at risk;” Characterize adults’ literacy skills in terms of demographic and background information (such as reading characteristics, education, and employment experiences); Profile the literacy skills of the nation’s work force; Compare assessment results from the current study with those from the 1985 literacy survey of young adults; Interpret the findings in light of information-processing skills and strategies, so as to inform curriculum decisions concerning adult education and training; and Increase understanding of the skills and knowledge associated with living in a technological society.
This chapter describes the design for the 1992 National Adult Literacy Survey and gives an overview of the steps involved in its implementation, from the development of a working definition of literacy to the creation of edited data files. The major components of the implementation of the survey are presented here as a tool to help the reader gain an overview of the National Adult Literacy Survey without having to read each individual chapter. For more detailed or technical information, the reader is referred to the specific chapters of this technical report as well as to the booklet Assessing Literacy (Campbell, Kirsch, and Kolstad, 1992) and the initial report on the survey, Adult Literacy in America (Kirsch, Jungeblut, Jenkins, and Kolstad, 1993). The organization of this chapter is as follows: Section 1.2 provides an overview of the development of the working definition of literacy that underlies the National Adult Literacy Survey. Section 1.3 summarizes the stratified random sampling procedures used for the national, state, and prison components of the survey. Section 1.4 gives an overview of the use and computation of weights used in the 1992 National Adult Literacy Survey to permit inferences from persons included in the sample to the populations from which they were drawn. Section 1.5 discusses the development of cognitive and background questions in the survey instrument. Section 1.6 summarizes the field operations and data collection in the household and prison surveys. Section 1.7 describes the data processing operations, including data entry, validation, the treatment of missing data, and the creation of edited data files. Section 1.8 discusses the Item Response Theory (IRT) scaling model and the plausible values methodology used to score respondents’ performance to the items in the questionnaire. Section 1.9 discusses the establishment of literacy levels for the National Adult Literacy Survey.
2
1.2 DEFINING LITERACY Although few would deny the importance of literacy in today’s society, a shared belief in the value of literacy does not imply consensus on how to define and measure it. In fact, there are widely varying opinions about the skills that individuals need to function successfully in their work, in their personal lives, and in society, and about the ways in which these skills should be assessed. As a result, there have been widely conflicting diagnoses of the literacy problem in this country. A committee of experts from business and industry, labor, government, research, and adult education worked with ETS staff to develop the definition of literacy that underlies the National Adult Literacy Survey, as well as to prepare the assessment objectives that guided the selection and construction of assessment tasks. In addition to this Literacy Definition Committee, a Technical Review Committee was formed to help ensure the soundness of the assessment design, the quality of the data collected, the integrity of the analyses conducted, and the appropriateness of the interpretations of the final results. Drawing on the two earlier studies of adult literacy conducted by ETS and funded by the Federal government (Kirsch and Jungeblut, 1986; Kirsch, Jungeblut, and Campbell, 1992), the Literacy Definition Committee rejected the types of arbitrary standards—such as signing one’s name, completing five years of school, or scoring at a particular grade level on a school-based measure of reading achievement—that have long been used to make judgments about adults’ literacy skills. Through a consensus process, the committee adopted the following definition of literacy, initially developed for the 1985 young adult survey: Using printed and written information to function in society, to achieve one’s goals, and to develop one’s knowledge and potential. This definition of literacy extends beyond simple decoding and comprehension to include a broad range of skills that adults use in accomplishing many different types of literacy tasks associated with work, home, and community contexts. 1.3 THE SAMPLE The National Adult Literacy Survey was administered to three samples: 1) a national household sample, 2) household samples from 11 states, and 3) a national sample of prison inmates. Both the national and state household samples were based on four-stage, stratified sampling. The prison sample was based on two-stage sampling. While the national and state household samples were drawn using the same sampling strategy, they differed in two ways: blacks and Hispanics were oversampled only in the national sample, and the target population for the national sample consisted of adults age 16 or older while for the state sample the target population consisted of adults ages 16 to 64. Blacks and Hispanics were oversampled in the national sample based on the key objective of the national sample: to provide reliable statistics for the adult population along with the prespecified domains. The prespecified domains included a racial/ethnic domain
3
and an adults aged 65 and older domain. While the states wanted reliable statics, they were not concerned with the specific domains, and thus did not oversample them. The four sampling stages for the national and state samples were: (1) the selection of primary sampling units (PSUs) consisting of counties or groups of counties, (2) the selection of segments consisting of census blocks or groups of blocks, (3) the selection of households, and (4) the selection of age-eligible individuals. In the first stage of sampling, the PSUs were stratified according to census region, metropolitan status, percentage of black residents, percentage of Hispanic residents, and, whenever possible, per capita income. In the second stage of sampling, census blocks or groups of blocks within each PSU were selected with a probability proportional to the number of housing units. In the third stage, a list of all housing units was created. A list of all housing units within the boundaries of each segment was then selected. Households were selected with equal probability within each segment of census blocks or groups of blocks, except for White, non-Hispanic households in high-minority segments in the national component. Finally, in the fourth stage of sampling, one person was randomly selected from each household with fewer than four eligible members and two persons were randomly selected from each household with four or more eligible members, from a list of all age-eligible household members (age 16 or older for the national sample and age 16 to 64 for the state samples). The same stratification methods, PSU construction, sample design and instruments were used for both the national and state designs. In addition, at the request of the Office of Management and Budget, a subsample of 1,812 households drawn from the 2,064 segments in the national sample was randomly selected following the steps outlined above in order to yield approximately 1,000 respondents who would be administered the survey without a $20 incentive. This was done to be able to compare the incentive versus non-incentive response rates as well as assess the effect of incentives on response patterns. For the prison survey, the two sampling stages were (1) the selection of primary sampling units (PSUs), and (2) the selection of inmates within each PSU. In this case, PSUs consisted of state or Federal adult correctional facilities, which were selected with a probability proportional to size. In the second stage, inmates were selected with a probability inversely proportional to the number of inmates, up to 22 inmates in a facility. Chapter 2 provides a discussion of the sample design. 1.4 WEIGHTING Whenever various subsets of the population are sampled at different rates or have different rates of selection or response, weights are necessary in order to permit inferences from persons included in the sample to the populations from which they were drawn, as well as to have sample estimates reflect estimates of the larger population. For example, in the national component of the National Adult Literacy Survey, blacks and Hispanics were oversampled to ensure reliable estimates of literacy proficiencies and to permit analyses of the performance of different subpopulations. Furthermore, because only one person was selected in 4
households with fewer than four eligible members, members of households with only one eligible member had twice the chance of selection as members of households with two eligible members, and three times the chance of selection as those in households with three eligible members. In such cases, weights are necessary to prevent serious bias in the estimates. Specifically, in the National Adult Literacy Survey, weights were computed to accomplish the following five objectives: (1) to permit unbiased estimates, taking account of the fact that all persons in the population did not have the same probability of selection, (2) to combine the state and national samples in an efficient manner, (3) to bring data up to the dimensions of the population totals, (4) to use auxiliary data on known population characteristics in such a way as to reduce sampling errors, and (5) to minimize biases arising from differences between cooperating and non-cooperating persons in the sample. Differential probability of selection was corrected by computing base weights for all persons selected into the sample. For all three components (national, state, and prison), the base weight was calculated as the reciprocal of a respondent’s final probability of selection. Furthermore, to combine the state and national samples, composite weights were calculated for the respondents in the 11 state samples and the respondents in the national sample PSUs in the 11 states. Finally, to adjust for non-response, weights were adjusted through post stratification and raking to match 1990 census totals. Chapter 3 provides detailed information on the weighting procedures. 1.5 THE SURVEY INSTRUMENT: MEASURING LITERACY The Literacy Definition Committee endorsed the notion that literacy is neither a single skill suited to all types of texts, nor an infinite number of skills, each associated with a given type of text or material. Rather, as suggested by the results of the young adult and job seeker surveys, an ordered set of literacy skills appears to be called into play to accomplish diverse types of tasks. Accordingly, in addition to adopting the definition of literacy that guided the earlier young adult and job-seeker studies, the Literacy Definition Committee adopted three literacy scales—prose, document, and quantitative—to report the results of the surveys. Prose literacy involves the knowledge and skills needed to understand and use information from texts that include editorials, news stories, poems, and fiction; for example, finding a piece of information in a newspaper article, interpreting instructions from a warranty, inferring a theme from a poem, or contrasting views expressed in editorials. Document literacy concerns the knowledge and skills required to locate and use information contained in materials that include job applications, payroll forms, transportation schedules, maps, tables, and graphs; for example, locating a particular intersection on a street map, using a schedule to choose the appropriate bus, or entering information on an application form. Quantitative literacy involves the knowledge and skills required to apply arithmetic operations, either alone or sequentially, using numbers embedded in printed materials; for example, balancing a
5
checkbook, figuring out a tip, completing an order form, or determining the amount of interest from a loan advertisement. The prose, document, and quantitative scales were augmented in the current survey through the addition of new assessment tasks that took into account the following: • • • • •
Continued use of open-ended simulation tasks; Continued emphasis on tasks that measure a broad range of information-processing skills and cover a wide variety of contexts; Increased emphasis on simulation tasks that require brief written and/or oral responses; Increased emphasis on tasks that ask respondents to describe how they would set up and solve a problem; and Use of a simple, four-function calculator to solve selected quantitative problems.
Approximately 110 new assessment tasks were field tested, and 81 of these were selected for inclusion in the survey. These 81 new assessment tasks were added to a pool of 85 tasks that were administered in both the young adult and job-seeker assessments (Kirsch and Jungeblut, 1986a and 1992). Thus, the National Adult Literacy Survey consisted of a total of 166 assessment tasks. By administering a common set of assessment tasks in each of the three literacy surveys, it is possible to compare results across time and across population groups. No individual could be expected to respond to the entire set of 166 simulation tasks administered as part of the National Adult Literacy Survey. It was therefore necessary to adopt a survey design that would give each person participating in the study a subset of the total pool of literacy tasks, while at the same time ensuring that each of the 166 tasks was administered to a nationally representative sample of the adult population. Literacy tasks were assigned to blocks or sections that could be completed in about 15 minutes, and these blocks were then compiled into booklets in such a way that each block appeared in each position (first, middle, and last) and each block was paired with every other block. Thirteen blocks of simulation tasks were assembled into 26 booklets, each of which could be completed in about 45 minutes. During a personal interview, each survey participant was asked to complete one booklet. In addition to the time allocated for the literacy tasks, approximately 20 minutes were devoted to obtaining personal information from respondents. Major areas explored included background demographics, education, labor market experiences, income, and literacy-related activities. These background data help to improve understanding of the ways in which various characteristics are associated with demonstrated literacy skills. Trained interviewers surveyed some 13,600 adults age 16 and older, chosen to represent the household population nationwide. In addition to the national samples, approximately 1,000 adults ages 16 to 64 were assessed in each of the states that chose to participate in the State Adult Literacy Survey, a special study designed to provide state-level data comparable to the national results. California, Illinois, Indiana,
6
Iowa, Louisiana, New Jersey, New York, Ohio, Pennsylvania, Texas, and Washington conducted their surveys at the same time as the national survey. (One additional state, Florida, was surveyed at a later date.) To permit comparisons of the state and national results, the survey instruments administered to the state and national samples were identical. Finally, 1,147 inmates from 87 state and Federal prisons were surveyed. Because some questions included in the household survey were inappropriate for the prison population, a revised version of the background questionnaire was developed that included queries about current offenses, criminal history, and prison work assignments, as well as education and work force experiences. To ensure comparability with the national survey, the simulation tasks (tasks that simulate the demands that adults encounter when they interact with printed materials on a daily basis) given to the prison participants were the same as those given to the household survey population. A total of 26,091 adults gave, on average, over an hour of their time to complete the National Adult Literacy Survey instruments. Those who agreed to participate in the survey and completed as much of the assessment as their skills allowed were paid $20 for their time. Responses from the national, state, and prison samples were combined to yield the best possible performance estimates. Chapter 4 describes the development of the survey instrument. 1.6 FIELD OPERATIONS Field operations and data collection for the National Adult Literacy Survey were the responsibility of Westat, Inc. The literacy survey was conducted between February and August 1992 by more than 400 trained interviewers, some of whom were bilingual in English and Spanish. All components of the survey sample were worked simultaneously, including the national sample, the state sample, and the prison sample. The field organization was headed by the survey field director, who reported directly to the Westat project director and who was supported by four home-office field managers and 24 field supervisors located across the United States. Each supervisor was supported in the field by an editor who was responsible for completely editing each case received from the field. Interviewers were recruited directly based on Westat’s computerized field personnel file containing information on over 4,000 field staff who had worked for Westat in the previous three years. A total of 456 interviewers were recruited, of which 2 did not attend training and 2 were released at training. Training consisted of a 3-day in-person training program, preceded by home study. The administration of the national and state household surveys to respondents occurred in three overlapping stages: an initial phase, in which each area segment was assigned to an interviewer; a reassignment phase, in which incomplete interviews were given to another interviewer in the same PSU; and a special non-response conversion phase, in which the home office assembled a special traveling team of the most experienced interviewers to perform a non-response conversion effort. 7
For the survey of the prison population, 51 interviewers were recruited from among the household survey workforce. These interviewers received an additional 1-day, in-person training session emphasizing collecting data on criminal history and prison employment. Interviewers were required to perform a careful edit before leaving the facility because it was not possible to recontact the prisoners if errors were made. An automated management system tracked and recorded the progress of fieldwork throughout the interview phase. In addition, progress was monitored weekly through telephone conferences between field supervisors, Westat home office staff, and ETS staff. Quality control checks were performed throughout the field data collection period and took the form of careful editing of completed documents, validation of 10 percent of each interviewer’s closed-out cases, observations of interviews in person and by tape recordings, and observation of supervisors by the Westat home office and ETS staff. As a result of the careful design of the field operations, the response rates achieved were quite favorable. Eighty-one percent of eligible respondents for the combined state and national surveys answered the background questionnaire. Of those, 95.8 percent completed the booklet of literacy exercises. For the prison population, 85.6 percent completed the background questionnaire, and 96.2 percent of those completing the background questionnaire completed the exercise booklet. Chapters 5 and 6 document the field operations for the household and prison surveys respectively. 1.7 DATA PROCESSING AND MISSING DATA After performing quality checks on completed background questionnaires and exercise booklets, field supervisors shipped them to ETS where staff checked the contents of each shipment against the enclosed transmittal form serving as the packing list for the shipment. The background questionnaires were then given to coders who coded the open-ended items, and the exercise booklets were given to readers who scored the open-ended literacy items. Coding was performed by 20 individuals, 9 working on the background questionnaire and 11 on the exercise booklets, following coding guides developed by scoring supervisors. To check the accuracy of coding in the background questionnaire, items dealing with country of birth, language, wages, and date of birth were checked in 10 percent of the questionnaires by a second coder. In the exercise booklets, 20 percent of all booklets were checked by a second coder who performed a reliability check. The inter-reader reliability for booklets scored by two readers was 97 percent, a number comparing very favorably with the reliability for the 1985 young adult literacy assessment. The coded responses for the background questionnaire and exercise booklets were then recorded onto scannable answer sheets that were then scanned by ETS staff and transmitted to magnetic tape. The data were then transferred to a database on the main computer for editing and quality control. In a final stage, the data files were examined for nonexistent housing locations, illogical or inconsistent responses,
8
multiple responses, as well as to insure that the skip patterns had been properly followed and that all data errors had been resolved. In order to address the issue of missing data, several imputation methods were considered using field test data as well as non-interview report data collected by the interviewers. Three of the five imputation methods made no use of the non-interview report data and the remaining two were informed by the reasons found in the non-interview report. A series of analyses examined the extent to which using each of the five imputation methods affected overall literacy proficiency estimates. Because imputation methods which made no use of the non-interview report data tended to weaken the educational, income, and racial/ethnic differences in literacy scores, they were ruled out, leaving two viable imputation methods. After consulting with others and examining the analyses performed using the two remaining imputation methods, the Technical Review Committee and the Literacy Definition Committee advising the National Adult Literacy Survey project adopted an imputation method for dealing with missing responses. When a respondent failed to answer consecutive assessment tasks and cited a reason related to literacy skills (e.g., “I can’t read these tasks”), the missing tasks were assigned wrong answers. That is, they were scored as if the respondent had attempted and failed the tasks. The extensive processing of the data is detailed in Chapter 7. Chapter 8 provides a discussion of the missing data procedures. 1.8 SCALING AND PROFICIENCY ESTIMATES The scaling model used for the National Adult Literacy Survey is the three-parameter (3PL) model from item response theory (Birnbaum, 1968; Lord, 1980). This model estimates the probability that an individual will respond correctly to a particular task from a single domain of tasks as a function of a parameter characterizing the proficiency of that individual and three parameters characterizing the properties of a given task in terms of its sensitivity to proficiency, its difficulty, and its non-zero chance of correct response for a multiple-choice task. Item response theory (IRT) models are based on the assumptions of conditional independence (i.e., item response probabilities depend only on a measure of proficiency and the specified item parameters) and unidimensionality (i.e., performance on a set of items is accounted for by a single variable). Thus, a critical part of the data analysis involved the testing of these two assumptions in order to validate the accuracy and integrity of the results. Because in the National Adult Literacy Survey each respondent was administered relatively few items in a subject area scale, comparing scale scores based on the respondents’ responses to different questions would lead to seriously biased estimates of proficiency. To circumvent this problem, proficiency scores for respondents were estimated using plausible values methodology. Plausible values provide consistent estimates of population characteristics, even though they are not unbiased estimates of the proficiencies of the individuals with whom they are associated. Thus, plausible values are not test scores for individuals in the usual sense. They are merely an intermediate measure used to estimate population 9
characteristics. Chapter 9 discusses the scaling methodology as well as the calculation of proficiency estimates using plausible values methodology (Mislevy, Beaton, Kaplan, Sheehan, 1993). 1.9 ESTABLISHING LITERACY LEVELS As previously noted, the results of the National Adult Literacy Survey are reported using three scales: a prose scale, a document scale, and a quantitative scale. The literacy scales, each ranging from 0 to 500, provide a useful way to describe the various types and levels of literacy demonstrated by adults in the population as a whole and in different subpopulations. The scales used an item mapping procedure reflecting response probabilities (RP). Tasks were placed on the scale at the point at which a minimum of 80 percent (i.e., RP80) of respondents at a particular ability level could be expected to complete the task successfully. The scores on each literacy scale represent degrees of proficiency along that particular dimension of literacy. For example, a low score (below 200) on the document scale indicates that an individual has very limited skills in processing information from tables, charts, graphs, maps, and the like (even those that are brief and uncomplicated). On the other hand, a high score (above 375) indicates advanced skills in performing a variety of tasks that involve the use of complex documents. The literacy scales also make it possible to determine the relative difficulty of the literacy tasks included in the survey. In other words, just as individuals receive scale scores according to their performance in the assessment, the literacy tasks receive different scale values according to their difficulty, as determined by the performance of the adults who participated in the survey. The literacy tasks administered in the National Adult Literacy Survey varied widely in terms of materials, content, and task requirements, and thus in difficulty. A careful analysis of the range of tasks along each scale provides clear evidence of an ordered set of information-processing skills and strategies along each scale. To capture this ordering, each scale was divided into five levels that reflect this progression of information-processing skills and strategies: Level 1 (0 to 225), Level 2 (226 to 275), Level 3 (276 to 325), Level 4 (326 to 375), and Level 5 (376 to 500). By examining the tasks within each literacy level, it is possible to identify the types of materials or directives that are more or less difficult for various types of readers. Further, by examining the characteristics of individuals who performed at each literacy level, it is possible to identify factors associated with higher or lower proficiency in reading and using prose, documents, or quantitative materials. Chapter 13 summarizes the establishment of literacy levels for the National Adult Literacy Survey. Appendices A through R, respectively, contain information about: estimated item parameters, conditioning variables, gamma values, RP80s and item probabilities, the non-interview report form, interviewer’s observation guide, English background questionnaire for households, English background questionnaire for prisons, derived variables, codes for continuous variables, birth codes, scoring the variables, sample-specific variables, treatment distribution, estimated composite factors, the code book for windows, and standard errors for Chapters 8 and 10. 10
Chapter 2 SAMPLE DESIGN Leyla Mohadjer, Joseph Waksberg, Huseyin Goksel, and James Green, Westat, Inc.
2.1 OVERVIEW The National Adult Literacy Survey included the following three components: 1) a national household sample; 2) household samples from 11 states; and 3) a national sample of prison inmates. The national and state household components were based on a four-stage, stratified area sample with the following stages: (1) the selection of primary sampling units (PSUs) consisting of counties or groups of counties, (2) the selection of segments consisting of census blocks or groups of blocks, (3) the selection of households, and (4) the selection of age-eligible individuals. A single area sample was drawn for the national component, and 11 additional state-level area samples were drawn for the state component (i.e., California, Illinois, Indiana, Iowa, Louisiana, New Jersey, New York, Ohio, Pennsylvania, Texas, and Washington).1 The national and state samples differed in two important respects. In the national sample, Black and Hispanic individuals were sampled at a higher rate than the remainder of the population to increase their representation in the sample, whereas the state samples used no oversampling. Also, the target population for the national sample consisted of adults age 16 or older, whereas the target population for the state samples consisted of adults ages 16–64. As noted above, the first stage of sampling for all 12 household samples involved the selection of PSUs, which consist of counties or groups of counties. The PSUs were stratified according to census region, metropolitan status, percentage of Black residents, percentage of Hispanic residents, and, whenever possible, per capita income. The national component used a 101-PSU sample. The national frame of PSUs was used to construct individual state frames for the state components, and a sample of 8 to 12 PSUs was selected within each of the 11 states. All PSUs were selected with a probability proportional to the PSUs’ 1990 population. For the second stage of sampling, segments (census blocks or groups of blocks) within the PSUs were selected with a probability proportional to size, where the measure of size for a segment was a function of the number of year-round housing units within the segment. The oversampling of Black and Hispanic persons for the national component was carried out at the segment level, where segments were classified as high minority (segments with more than 25 percent Black or Hispanic residents) or low minority. The measure of size for high-minority segments was defined as the number of White, non-Hispanic households plus three times the number of Black or Hispanic households. High-minority 1
A state-level survey was later conducted in Florida, but the data are not included in this report. 11
segments were therefore oversampled at up to three times the rate of low-minority segments. As for all segments in the state components, the measure of size was simply the number of year-round housing units within the segment. One in seven of the national component segments was selected at random to be included in a “non-incentive” sample (see section 2.3 for more details). Respondents from the remaining segments in the national component received a monetary incentive for participation, as did all respondents in the state components. Data for respondents from the non-incentive segments were not included in the analyses reported by the National Center for Education Statistics, but are available as one of the three principal analysis files (household, prison, and non-incentive data). For the third stage of sampling, the selection of households within segments, Westat field staff visited all selected segments and prepared lists of all housing units within the boundaries of each segment, as determined by the 1990 census block maps. The lists were used to construct the sampling frame for households. Households were selected with equal probability within each segment, except for White, non-Hispanic households in high-minority segments in the national component. These households were sub-sampled after screening, so that the sampling rates for White, non-Hispanic persons would be about the same in the high-minority segments as in other segments. For the fourth stage of sampling, a list of age-eligible household members (age 16 or older for the national component, 16–64 for the state component) was constructed for each selected household. One person was selected at random from households with fewer than four eligible members, and two persons were selected at random from households with four or more eligible members. The interviewers were instructed to list the eligible household members in descending order of age. The interviewers then identified the one or two sample household members based on computer-generated sampling messages that had been attached to each questionnaire in advance. The sample design for the prison component involved two stages of selection. For the first stage of sampling, state or Federal correctional facilities were selected with a probability proportional to size, where the measure of size for a facility was equal to the size of the inmate population. The second stage involved the selection of inmates within each facility. Inmates were selected with a probability inversely proportional to the size of their facility’s inmate population (up to 22 inmates in a facility). Table 2-1 provides the sample sizes for all stages of sampling for the national and state components of the National Adult Literacy Survey. Section 2.2 provides a review of the four stages of sampling for the national component of the survey. A similar discussion of the state samples is presented in section 2.4. Section 2.5 presents weighted and unweighted response rates for the household component of the survey. Sections 2.3 and 2.6 describe the non-incentive sample design and the prison sample design, respectively.
12
Table 2-1. Sample sizes for the national and state components of the National Adult Literacy Survey Number Number Number of Number Number of Number of of persons of persons persons Component of PSUs segments* households+ screened interviewed assessed National and state incentive sample
210
3,733
43,783
30,806
24,944
22,107
National non-incentive sample
101
155
1,838
1,273
930
695
CA
20
405
4,917
3,371
2,665
2,143
IL
14
262
2,914
2,130
1,668
1,504
IN
15
215
2,361
1,755
1,441
1,368
IA
14
187
2,041
1,446
1,246
1,192
LA
10
188
2,270
1,460
1,192
1,087
NJ
16
243
2,790
1,821
1,317
1,111
NY
14
302
3,526
2,139
1,688
1,415
OH
17
246
2,691
1,984
1,568
1,510
PA
14
253
2,950
2,060
1,626
1,532
TX
16
316
3,833
2,681
2,209
1,834
WA
9
182
2,096
1,506
1,244
1,186
State samples**
& +
**
The numbers include segments with at least one dwelling unit selected into the sample. The numbers include the missed structures and units (refer to section 2.2.3.3) incorporated into the sample during the data collection. Numbers include the national sample cases in each state in addition to the individually selected state sample.
2.2 SAMPLING FOR THE NATIONAL COMPONENT The target population for the national component of the National Adult Literacy Survey consisted of adults age 16 or older in the 50 states and the District of Columbia who, at the time of the survey (February through August, 1992), resided in private households or college dormitories. The household component used a four-stage, stratified sample design. The first-stage sample was a sample of PSUs (counties or groups of counties) developed by Westat. In developing the sampling frame, the 3,141 counties and independent cities in the 50 states were grouped into 1,404 PSUs, from which a sample of 101 PSUs was selected for the household component. In the second stage of sampling, probability sampling was used to select a sample of 2,064 segments (census blocks or combinations of blocks) from the PSUs chosen during the first stage. The third stage of sampling involved the selection of 24,522 housing units from listings developed within the selected segments by the field listers. In the fourth
13
stage, age-eligible persons were chosen for interview and assessment from within selected households. The stages of sampling for the national component are described in greater detail in the following sections. 2.2.1 First-Stage Sample The first-stage sample was a sample of PSUs (counties or groups of counties) developed by Westat. 2.2.1.1 Westat’s master sample of PSUs In selecting the master sample, Westat used the 1990 census Public Law 94-171 (PL94) data tape file as the source of information (total and minority population sizes for each county) for stratification as well as to determine PSU size. The income data were based on the 1988 per capita income reported by the Bureau of Economic Analysis. In designing the Westat PSU sample, entire metropolitan statistical areas (MSAs) were treated as single PSUs; however, because of their size, the New York, Los Angeles, and Chicago MSAs were divided into three, two, and two PSUs, respectively. In New England, whole-county approximations of MSAs were used. Counties outside of MSAs were grouped to make PSUs (1) large enough to provide a sufficient sample size for most national surveys and (2) as internally heterogeneous as possible but still small enough that an interviewer could conveniently travel across the PSU. A total of 1,404 PSUs were constructed. All PSUs consisted of one or more contiguous counties, or contiguous counties and independent cities, and had minimum population sizes of 15,000. Additionally, all PSUs were completely contained within the boundaries of one of the four census regions. Master sample PSUs were stratified on the basis of the social and economic characteristics of the population, as reported in the 1990 census. Strata were of roughly equal size; they did not cross regions, and a stratum did not include both metropolitan and non-metropolitan PSUs. The following characteristics were used in stratifying the Westat PSUs (some explicitly and some implicitly, by ordering the PSUs and sampling systematically): • • • • •
Region of the country (four census regions); Whether or not the PSU was an MSA; Percentage of Black residents; Percentage of Hispanic residents; and Average income.
2.2.1.2 Selecting the sample of PSUs for the national component The sampling frame for the Westat PSU sample included Hawaii and Alaska, but neither of the Hawaii or Alaska counties were selected for the 100-PSU master sample. Honolulu MSA was added to the sample as the 101st PSU in the national sample. Westat adjusted the weights to correctly account for the inclusion of the Honolulu PSU in the sample. Table 2-2 shows the distribution of the population in the 101 PSUs selected for the household component of the survey. The measure of size for each PSU was equal to the 1990 population of the PSU. 14
Twenty-five PSUs were included in the sample with certainty on the basis of their sizes. Then 38 strata of approximately equal size were formed. Two PSUs were selected (without replacement), with probability proportionate to size, from each of the 38 strata. Among the multiple-PSU strata, 26 were MSA strata and 12 were non-MSA strata. Table 2-2. Proportion of U.S. population in PSUs selected for the national component by stratum type, total 1990 population, Black, and Hispanic Total 1990 Stratum type
PSU sample
Black
Hispanic
Number
%
Number
%
Number
%
76,349,843
30.7
12,304,548
40.0
11,769,950
52.7
Certainty MSA
Total in frame
Non-certainty MSA
Total in frame Not in sample In 101-PSU sample
116,764,722 75,474,068 41,290,654
47.0 30.4 16.6
12,823,091 8,115,899 4,707,192
42.8 27.1 15.7
8,444,362 5,387,275 3,057,087
37.8 24.1 13.7
Non-certainty non-MSA
Total in frame Not in sample In 101-PSU sample
55,595,308 54,058,657 1,536,651
22.4 21.7 0.6
4,858,421 4,742,122 116,299
16.2 15.8 0.4
2,139,747 2,072,580 67,167
9.6 9.3 0.3
248,709,873
100.0
29,986,060
100.0
22,354,059
100.0
Grand total
Table 2-3 contains a listing of the 101 PSUs in the national sample (certainty PSUs are in bold).
15
Table 2-3. National Adult Literacy Survey 101-PSU sample PSU
County and State
PSU
County and State
101
Boston, MA Essex Middlesex Norfolk Plymouth Suffolk
111
Buffalo, NY Erie
112
Bergen/Passaic, NJ Bergen Passaic
113
Newark, NJ Essex Morris Sussex Union
114
Monmouth/Ocean, NJ Monmouth Ocean
115
Atlantic City,NJ Atlantic Cape May
116
Philadelphia, PA/Camden, NJ Burlington, NJ Camden, NJ Gloucester, NJ Bucks, PA Chester, PA Delaware, PA Montgomery, PA Philadelphia, PA
117
Scranton/Wilkes-Barre, PA Columbia Lackawanna Luzerne Monroe Wyoming
118
Harrisburg, PA Cumberland Dauphin Lebanon Perry
119
Pittsburgh, PA Allegheny Fayette Washington Westmoreland
102
Pittsfield, MA Berkshire
103
Springfield, MA Hampden Hampshire
104
Providence, RI Bristol Kent Providence Washington
105
Newport, RI Newport
106
Nassau/Suffolk, NY Nassau Suffolk
107
Kings/Richmond, NY Kings Richmond
108
New York/Queens, NY New York Queens
109
Bronx/Putnam, NY Bronx Putnam Rockland Westchester
110
Rochester, NY Livingston Monroe Ontario Orleans Wayne
Certainty PSUs are in bold. 16
Table 2-3. 101-PSU sample – Continued PSU
County and State
PSU
County and State
120
Butler, PA Butler Lawrence
209
201
Steubenville, OH (Weirton,WV) Jefferson
202
Youngstown/Warren, OH Mahoning Trumbull
Indianapolis, IN Boone Hamilton Hancock Hendricks Johnson Marion Morgan Shelby
210 203
Akron, OH Portage Summit
Gary/Hammond, IN Lake Porter
211 204
Cleveland, OH Cuyahoga Geauga Lake Medina
Chicago, IL (CITY) Chicago City
212
Cook/DuPage/McHenry, IL (Chicago) Cook DuPage McHenry
213
Aurora/Elgin, IL Kane Kendall
214
Knox/Mercer, IL Knox Mercer
215
Peoria, IL Peoria Tazewell Woodford
216
St. Louis, MO/E. St. Louis, IL Clinton, IL Jersey, IL Madison, IL Monroe, IL St.Clair, IL Franklin, MO Jefferson, MO St. Charles, MO St. Louis City, MO St. Louis, MO
205
206
207
208
Cincinnati, OH/Dearborn, IN (Covington, KY) Dearborn, IN Clermont, OH Hamilton, OH Warren, OH Saginaw/Bay City/Midland, MI Bay Midland Saginaw Detroit, MI Lapeer Livingston Macomb Monroe Oakland St. Clair Wayne Fountain/Montgomery/Putnam,IN Fountain Montomery Putnam
Certainty PSUs are in bold. 17
Table 2-3. 101-PSU sample – Continued PSU
County and State
PSU
County and State
217
Pike/Ralls, MO Pike Ralls
225
Atchison/Jackson/Jefferson, KS Atchison Jackson Jefferson
218
Howard/Saline, MO Howard Saline
301
Washington, D.C./MD/VA District of Columbia Calvert, MD Charles, MD Frederick, MD Montgomery, MD PrinceGeorges, MD Arlington, VA Fairfax, VA Loudoun, VA PrinceWilliam, VA Stafford, VA Alexandria City, VA Fairfax City, VA Falls Church City, VA Manassas, VA Manassas Park, VA
302
Wilminton, DE/Cecil, MD New Castle, DE Cecil, MD
303
Baltimore,MD AnneArundel Baltimore County Baltimore City Carroll Harford Howard Queen Annes
304
Weirton, WV (Steubenville, OH) Brooke Hancock
305
Charlottesville, VA Albemarle Fluvanna Greene Charlottesville City
306
Norfolk/Virginia Beach,VA Gloucester James City York
219
Milwaukee, WI Milwaukee Ozaukee Washington Waukesha
220
Minneapolis/St. Paul, MN/WI Anoka, MN Carver, MN Chisago, MN Dakota, MN Hennepin, MN Isanti, MN Ramsey, MN Scott, MN St.Croix, MN Washington, MN Wright, WI
221
Iowa City, IA Johnson
222
Monona, IA/Thurston, NE Monona, IA Thurston, NE
223
Hall/Hamilton, NE Hall Hamilton
224
Cheyenne/Rooks, KS Cheyenne Decatur Graham Rawlins Rooks Sheridan
Certainty PSUs are in bold. 18
Table 2-3. 101-PSU sample – Continued PSU
307
County and State
PSU
County and State
Chesapeake City Hampton City Newport News City Norfolk City Poquoson Portsmouth City Suffolk City Virginia Beach City Williamsburg City
313
Nashville, TN Cheatham Davidson Dickson Robertson Rutherford Sumner Williamson Wilson
Johnson City, TN/Bristol, VA Carter, TN Hawkins, TN Sullivan, TN Unicoi, TN Washington, TN Scott, VA Washington, VA Bristol City, VA
314
Chattanooga, TN/Dade, GA Catoosa, GA Dade, GA Walker, GA Hamilton, TN Marion, TN Sequatchie, TN
315
Atlanta, GA Barrow Butts Cherokee Clayton Cobb Coweta DeKalb Douglas Fayette Forsyth Fulton Gwinnett Henry Newton Paulding Rockdale Spaulding Walton
316
Greene/Lincoln, GA Greene Lincoln Oglethorpe Wilkes
317
Wheeler/Toombs, GA Montgomery Toombs
308
Covington, KY (Cincinnati, OH) Boone Campbell Kenton
309
Fort Knox, KY Breckinridge Grayson Meade
310
Greensboro/Winston-Salem NC Davidson Davie Forsyth Guilford Randolph Stokes Yadkin
311
Albemarle, NC Montgomery Stanly
312
Fayetteville, NC Cumberland
Certainty PSUs are in bold.
19
Table 2-3. 101-PSU sample – Continued PSU
318
County and State
PSU
County and State
Treutlen Wheeler
328
Muskogee/McIntosh, OK McIntosh Muskogee
329
Dallas, TX Collin Dallas Denton Ellis Kaufman Rockwall
330
Anderson TX Anderson
331
Austin, TX Hays Travis Williamson
332
San Antonio, TX Bexar Comal Guadalupe
333
Houston, TX Fort Bend Harris Liberty Montgomery Waller
Tallahassee, FL Gadsden Leon
319
Tampa/St. Petersburg, FL Hernando Hillsborough Pasco Pinellas
320
Orlando, FL Orange Osceola Seminole
321
322
Miami/Ft. Lauderdale, FL Broward Dade Birminham, AL Blount Jefferson St. Clair Shelby Walker
323
Dothan, AL Dale Houston 334
324
Meridian, MS Lauderdale Newton
Big Spring, TX Howard
401
Franklin/Madison, AR Franklin Madison
Seattle, WA King Snohomish
402
Portland, OR Clackamas Multnomah Washington Yamhill
403
Missoula, MT Missoula
325
326
Pope, AR Pope
327
Shreveport, LA Bossier Caddo
Certainty PSUs are in bold. 20
Table 2-3. 101-PSU sample – Continued PSU
County and State
PSU
404
Boise City, ID Ada
413
Los Angeles/Long Beach, CA Los Angeles
414
Anaheim/Santa Ana, CA Orange
415
San Diego, CA San Diego
416
Douglas/Storey/Carson City, NV Douglas Storey Carson City
417
Las Vegas, NV Clark
418
Phoenix, AZ Maricopa
San Jose, CA Santa Clara
419
Tucson, AZ Pima
Merced, CA Merced
420
Cibola/Valencia, NM Cibola Valencia
421
Boulder, CO Boulder
422
Honolulu, HI
405
406
407
408
409
Elmore/Twin Falls, ID Elmore Twin Falls Sacramento, CA El Dorado Placer Sacramento Yolo San Francisco/Oakland, CA Alameda Contra Costa Marin San Francisco San Mateo
410
Fresno, CA Fresno
411
Riverside/San Bernardino, CA Riverside San Bernardino
412
County and State
Los Angeles City, CA Los Angeles City
Certainty PSUs are in bold. 2.2.2 Second-Stage Sample—Selecting Census Blocks (Segments) Within each PSU, area segments consisting of census blocks (or combinations of two or more adjacent census blocks) were selected with probability proportionate to size. A total of 2,064 segments were chosen, an average of 21 per PSU. The frame for defining and sampling segments was the 1990 PL94 data. The sample design requirements called for an average cluster size of about seven interviews (i.e., an average of about seven completed background interviews per segment). The sample of housing units within each segment was designed to account for attrition. Attrition was expected because, according to figures obtained from the 1990 census, approximately 10 percent of the housing units were probably vacant. Additionally, we expected a 10 percent screener refusal rate and a 15 percent background 21
questionnaire refusal rate. The sample of housing units selected within each segment was thus made equal to 11. In addition, a reserve sample of approximately 5 percent of the size of the main sample was selected and set aside in case of shortfalls due to unexpectedly high vacancy and nonresponse rates. 2.2.2.1 Measures of size and sampling rates Standard texts on sampling discuss measure of size in multistage designs for household surveys only in univariate situations. In effect, they describe how the total population can be used as the measure of size when sampling areas with probability proportionate to size, followed by sampling within each area at a rate proportionate to the reciprocal of the measure of size. A sample selected in this way has two desirable properties: (1) it is a self-weighting sample (i.e., all households are selected at the same rate), and (2) the interviewer workloads are approximately the same in all areas. The second property provides operational efficiency and results in lower variances than designs with variable workloads. The national sample design modified and adapted the theory for multivariate situations by establishing a measure of size that produced constant workloads among segments and, at the same time, produced constant (but separate) sampling rates for minorities and non-minorities within each of two strata. The following is a description of the derivation of measures of size for this survey. One of the requirements of the national design was to sample Black and Hispanic adults at a higher rate than the remainder of the population. Segments where 25 percent or more of the population consisted of Black and Hispanic adults were oversampled at a rate up to three times that of the remainder of the segments. The housing unit counts served as the measure of size for the low-minority segments (segments with less than 25 percent Black or Hispanic households). In high-minority segments, the measure of size was equal to the number of White, non-Hispanic households plus three times the number of Black and Hispanic households. In low-minority segments, the measure of size of a segment was equal to the number of households in the segment.
MOS2ij = H Oij + HMij mij
(1)
where MOS2ij =
measure of size for the ijth segment in the low-minority stratum.
HOij
=
number of “other” (i.e., non-minority) households in the jth segment in the ith PSU; and
HMij
=
number of minority (Black plus Hispanic) households in the jth segment in the ith PSU;
22
In high-minority segments, the measure of size was equal to MOS1ij =
HOij + 3HMij,
MOS1ij =
measure of size of the ijth segment in the high-minority stratum (the minority stratum is defined as segments in which the Black plus Hispanic population is 25 percent or more of the total population).
where
The sampling interval, I, was computed as
M MOS /P M MOS 1ij
I
2ij /Pi
i
1ij
2 ij
(2)
2064
where Pi
=
probability of selection of the ith PSU.
The segment selection probability in the ith PSU was thus MOS1ij for high-minority segments and MOS2ij I Pi
I Pi
for low-minority segments. It should be noted that the overall segment selection probability was independent of Pi. 2.2.2.2 Minimum segment size The screening sampling rate within a segment was 11/MOS2ij (in low-minority segments) and 33/MOS1ij (in high-minority segments). Thus, in the low-minority stratum
HOij
+
HMij 11.
(3)
HOij
+
3HMij 33.
(4)
HMij 11.
(5)
In the high-minority stratum
or
HOij /3
+
23
The actual segment sizes had to be 11 households in low-minority areas, 11 households in high-minority areas with 33 percent minorities, and 22 households in segments with 25 percent minorities.
2.2.2.3 Segment sample selection The first step in sampling segments was to extract block data from the PL94 file for the 101 PSUs in the sample. In the next step, blocks containing fewer than the minimum number of housing units required to select the third-stage sample were combined with other adjacent or nearby blocks to form the segments that served as second-stage sample units. Segments were sorted within each PSU according to the proportion of Black and Hispanic residents.2 A systematic sample of segments was then selected with probability proportional to size. The systematic selection provided implicit stratification according to the proportion of minority residents in the segments. The sample of 2,064 segments included 869 high-minority and 1,195 low-minority segments. 2.2.2.4 TIGER maps The National Adult Literacy Survey was one of the first sample surveys nationwide to use the Bureau of the Census’s Topologically Integrated Geographical Encoding and Referencing (TIGER) System file for the production of segment maps. Segment maps are essential features of an area sample; they define and describe the sample segments, permitting field interviewers to locate the areas and list the housing units within the segments. In the past, segment maps were produced by hand, with clerks outlining the segments manually on maps purchased from the Census Bureau. This operation was slow, costly, and somewhat error-prone. The maps were of diverse sizes, resulting in problems of filing and storage. Street names were difficult to read on many of the maps. The Census Bureau produced a system known as the TIGER file for the implementation of the 1990 census. The TIGER file digitized all intersections of geographic boundaries used in the 1990 census, including individual blocks. This information can be used to computer generate maps of selected blocks, combinations of blocks, or any other type of geography referred to in the census. Before the National Adult Literacy Survey began, Westat purchased a copy of the TIGER file and software to generate maps from the file and then developed additional software to facilitate its use for sample survey purposes. In the completely automated sampling process, sample blocks were selected from census summary tapes, and the block identifications were automatically fed into the TIGER file, which in turn generated the segment maps. This method of map production cost considerably less than the old method, was more accurate, and was much faster to implement. Because Westat developed much of the software, other useful features were included in the segment maps. For example, the maps were uniform in size, had sufficient detail to permit 2
A serpentine sort executes multiple sorts within a stratum such that bordering sample units are the most similar with respect to the sort variables. This is accomplished by reversing the sort order within the segment groups. 24
street names to be read, had convenient map numbers automatically inserted, included small-scale maps of larger areas, showed segment locations within broader areas, and included certain data (based on the 1990 census) for quality control. 2.2.2.5 Listing sample segments Westat field staff visited each sample segment and prepared a list of all housing units within the boundaries of the segment. (A total of 142 large segments from the national sample were subdivided before listing, with one part, or “chunk,” selected at random for listing.) Table 2-4 provides the distribution of segments in the national sample, by segment size. As noted earlier in this section, segments consisted of census blocks or combinations of two or more adjacent blocks that could be accessed without crossing over census tract boundaries. Therefore, if the segments did not contain enough households to reach the minimum size established for that type of segment (see section 2.2.2.2), the measure of size was considered to be equal to the minimum measure of size. Table 2-4. Distribution of segments in the national sample, by segment size* Dwelling Cumulative units Frequency Percent frequency 0-19 8 0.4 8 20-29 1 0.0 9 30-39 12 0.6 21 40-49 35 1.7 56 50-59 100 4.8 156 60-69 282 13.7 438 70-79 264 12.8 702 80-89 196 9.5 898 90-99 129 6.2 1,027 100-119 211 10.2 1,238 120-149 208 10.1 1,446 150-199 186 9.0 1,632 200-249 102 4.9 1,734 250-299 103 5.0 1,837 300-399 168 8.1 2,005 400-499 51 2.5 2,056 500-699 7 0.3 2,063 700-799 1 0.0 2,064
Cumulative percent 0.4 0.4 1.0 2.7 7.6 21.2 34.0 43.5 49.8 60.0 70.1 79.1 84.0 89.0 97.1 99.6 100.0 100.0
* The frequencies reported in this table are the actual numbers of dwelling units listed in the selected segments. Large segments were subdivided and one section was selected at random for listing.
25
2.2.3 Third-Stage Sample—Selecting Housing Units The third stage of sampling for the national component involved sampling households within the selected segments. After selection, households were screened to determine whether they included any eligible respondents. In the low-minority segments, any household with at least one person age 16 or older was included in the sample. In the high-minority segments, all minority households with at least one person age 16 or older were retained in the sample, but only one-third of nonminority households (with at least one person age 16 or older) were included in the sample. 2.2.3.1 Within-segment sampling rate The sampling rates within the low-minority segments were set to produce an average of 11 housing units per segment. In high-minority segments, the average was about 14 housing units. White, non-Hispanic households in high-minority segments were sub-sampled at a rate of about one-third, so that White, non-Hispanic adults from high-minority segments had the same overall sampling rate as those residing in low-minority segments. The within-segment sampling rate (i.e., the household sampling rate) in low-minority segments was
r2ij
11 MOS2ij
(6)
In high-minority segments, the sampling rate was r1ij
r2ij
33 for minority households MOS2ij
(7)
11 for other households. MOS1ij
(8)
If the number of housing units in the selected segments was the same in 1992 as in 1990, the number of selected households that remained in the sample for interview would be constant across all segments; that is, if in low-minority segments the number of households in segment ij was equal to HOij + HMij = MOS2ij, the sample size was equal to
11 x (HOij H Mij) MOS2ij
11.
(9)
26
In high-minority segments, the sample size was equal to no
11 x HOij MOS1ij
nM
33 x H Mij MOS1ij
no n
11 (HOij HOij
11 HOij
(10)
HOij 3H Mij 33 HMij HOij 3 HMij
3HMij )
3 HMij
11
where no is the number of non-minority households selected in a high-minority segment; nM is the number of minority households selected in a high minority segment.
The segment sizes would thus be constant, equal to 11. However, segment sizes for the screening sample varied in the high-minority stratum. The screening sample in each segment was the rate at which minorities were selected. The sampling yield for the screening sample was thus MOS
33 (H 33 (HOij HMij) Oij HMij)
MOS1ij HOij 3HMij
(11)
HOij 3HMij 2HOij
11 H 3H H 3H Oij Mij Oij Mij 2HOij
11 1 H 3H Oij Mij
2 HOij
11 ( 1 +
MOS1ij HOij MOS
+
3H mij MOS1ij
27
)
Since the cut-off point for the high-minority strata was 25 percent minorities, the proportion of minorities in a segment from a high-minority stratum ranged from 25 percent to 100 percent. Putting those values in the formula above gives a range for the screening sample of 11 to 22 households. In the national sample, 24,522 households were selected. The following table provides the distribution of the selected households by census region.
Number of households 4,676
Census region Northeast Midwest
5,051
South
9,340
West
5,455
Total
24,522
2.2.3.2 Overall probabilities of selection The overall probability of selection of households in low-minority segments was
P2
= Pi
11 11 MOS2 ij = I Pi I MOS2 ij
(12)
In high-minority segments, the overall probability of selection for nonminority households was equal to
P2 i
= Pi
33 1 11 MOS1ij = I Pi I MOS1ij 3
(13)
where I is the sampling interval. For minority households in high-minority segments, the overall probability was
P1i
= Pi
MOS1ij 33 33 = I Pi I MOS1ij
(14)
2.2.3.3 Procedures for selecting missed structures and missed dwelling units Entire structures may have been omitted from the initial segment listing, either because the lister made an error or because the structure was constructed in the interval between listing and interviewing. Additionally, listers may have missed dwelling units within a listed structure because they were instructed not to inquire about the number of units in most residential buildings in order to reduce listing costs. Instead, listers were told to list a structure that looked like a one-family residence as a one-family 28
residence. However, a smaller number of buildings that looked like one-family residences may have been converted to multi-family residences. To compensate for this problem and identify missed households, Westat instructed interviewers to conduct two quality control procedures at the time of data collection. These procedures are described below. Missed Structure Procedure. If the first dwelling unit on the completed listing sheet was selected for the sample, a segment recanvass to search for missed structures was conducted. If any missed structures were found, the dwelling units within each missed structure were selected if the number of units within the structure was less than or equal to 10. If the number of units was greater than 10, 10 dwelling units were selected at random. Missed Dwelling Unit Procedure. If the first (or only) dwelling unit on the completed listing sheet was selected for the sample, the interviewer inquired at the sample unit about any additional units in the building. If any missed dwelling units were found, then all missed units were selected if the number of missed dwelling units within the structure was less than or equal to 10. If the number of missed dwelling units within the structure was greater than 10, all missed units were listed and a sample was selected from the listing. The increase in the total number of assessments and the effects of differential weights were considered when determining the probabilities with which to select these dwelling units. The overall goal was to control the increase in the total number of assessments within a segment so that no more than (approximately) double the number of persons originally expected were selected in a segment. 2.2.4 Fourth-Stage Sample—Selecting Persons Age 16 or Older A list of household members was obtained during the screener interview conducted at each sample household. Interviewers listed the household members in descending order of age. A computer-generated sampling message attached in advance to each questionnaire contained instructions on which household members to choose for an interview. The following table illustrates a typical sampling message:
Number of eligible persons in household
Choose the following person for interview
1
First
2
Second
3
Second
4
First and third
5
First and fifth
6
Third and sixth
Etc.
29
Because the sampling messages varied from household to household, each household member had the same chance of selection within each size of household group. One adult was sampled randomly from households with fewer than four eligible persons. In households with four or more eligible persons, two adults were selected. The selection of two adults in households with four or more eligible persons prevented a substantial increase in variances due to high weights resulting from the selection of one person in households with large numbers of eligible persons. Because non-Black, non-Hispanic persons were undersampled in segments designated as high minority, each individual was classified into a race/ethnicity class during the screening interview so that the subsampling procedure for non-Black, non-Hispanic persons could be implemented. Because most U.S. households contain persons of the same race/ethnicity group, a race/ethnicity category was also assigned to each household and the subsampling procedure was carried out based on the race/ethnicity of the household. The household classification was based on the race/ethnicity of the person designated as the head of household, defined as the person who owns or rents the dwelling unit. If the screener respondent could not identify a head of household, the race/ethnicity of the first person listed on the household roster was used as the race/ethnicity of the household. This procedure made the sample screening and selection less complicated and reduced the chance of sample selection errors during the data collection. The subsampling of nonminority households in high-minority segments was carried out using a sampling message that was attached to the questionnaires for a randomly selected two-thirds of the households in high-minority segments. 2.3 THE NON-INCENTIVE SAMPLE At the request of the Office of Management and Budget, a subsample of segments was selected to produce about 1,000 completed interviews with respondents who were not offered the $20 incentive. A field test experiment carried out before the main survey showed lower response rates for the non-incentive group than for those who received incentives. The lower response rates were taken into account when selecting the segment sample for the non-incentive experiment. A subsample of 155 segments was selected randomly from the 2,064 segments in the national sample, including 65 high- and 90 low-minority segments. This subsample contained 1,812 households and was expected to yield approximately 1,000 completed interviews with respondents who received no incentives. The role of incentives is discussed in more detail in Chapter 10 of this report. 2.4 SAMPLING FOR THE STATE LITERACY SURVEYS The National Adult Literacy Survey provided an opportunity for state officials to request that supplementary adult literacy surveys be conducted within their states, to provide state-level estimates of adult literacy skills that are reliable, valid, and comparable to national estimates. A sample of about 1,000 30
interviewed persons was used to supplement the national sample in each of the 11 states participating in the program. This sample size was estimated to be sufficient to provide adequate precision for most anticipated analyses. Participants in the state component were selected through a process nearly identical to that used for the national component, where the units at each stage of sample selection represented a particular state rather than the entire United States. The two principal differences between the sample designs for the national and the state surveys were that (1) Black and Hispanic adults were not oversampled in the state surveys and (2) the respondent universe consisted of adults ages 16–64 (vs. adults age 16 and older for the national survey). 2.4.1 Sample of PSUs The first-stage primary sampling units, or PSUs, for a state consisted of geographic clusters of one or more adjacent counties within the state. With a few exceptions, the PSUs were identical to those used in the national sample. The exceptions were the national PSUs that crossed state boundaries, which were subdivided for the state sample. Each PSU was assigned to a stratum (i.e., groups of PSUs with similar characteristics) and one PSU was selected within each stratum. The following characteristics were used to stratify the PSUs: whether the PSU was within an MSA as defined for the 1990 census; the percentage of the population in the PSU who were Black and/or Hispanic; and the population size of the PSU. Per capita income was also used wherever possible. In some states, the number of strata that could be created precluded the effective use of all four stratifying characteristics. One PSU was selected from each stratum with a probability proportional to the PSU’s 1990 population. The number of sample PSUs per state varied from 8 to 12, with smaller numbers of PSUs in states with one or more very large PSUs that were chosen with certainty. 2.4.2 Sample of Segments The second-stage sampling units consisted of census blocks or groups of blocks within the selected PSUs. Adjacent blocks were combined whenever necessary to ensure that each segment had a minimum of 20 housing units per segment. In each state, 167 segments were selected across the PSUs. The selection was systematic and with probability proportional to size, where the measure of size was the number of year-round housing units within the segment. The sampling interval for the selection of segments, I, was computed as
M MOS /P 2ij
I
1 ij
167
31
i
(15)
where MOS2ij =
Pi
=
the measure of size for the jth segment in the ith PSU (note that this is equivalent to the low-minority segment measure of size in the national component) and probability of selection of the ith PSU.
The PL94 data tapes from the 1990 census were used to define the segments within each PSU. Segments were stratified according to the percentage of minority (Black and Hispanic) residents before selection. 2.4.3 Sample of Housing Units The third stage of sampling involved the selection of households within segments. Westat field staff visited the 167 selected segments and prepared a list of all housing units within the boundaries of each segment. Segment boundaries were determined by the 1990 census block maps (i.e., the TIGER maps). The segment listings were sent to Westat, where a sample of about 11 housing units was selected per segment. Interviewers visited these housing units, determined which were occupied, and obtained a roster of household members. The same quality control procedures as in the national sample were used to compensate for missed structures and missed dwelling units within listed structures. 2.4.4 Sample of Persons One or two adults ages 16–64 were selected from the list of household members obtained during the household screening. The selection procedure was similar to the one used in the national sample. One person was selected at random from households with fewer than four eligible members; two persons were selected from households with four or more eligible persons. Interviewers listed the eligible household members in descending age order. The interviewers then identified the one or two household members for interview based on computer-generated sampling messages that had been attached to each questionnaire in advance. 2.5 WEIGHTED AND UNWEIGHTED RESPONSE RATES Unweighted response rates are indicators of how well the survey operations were carried out. They are useful during the survey as part of the quality control process and at the completion of field work as a measure of success. However, weighted response rates are more appropriate in examining the potential effect of nonresponse on statistics. Because the literacy estimates are based on weighted data, weighted response rates are better clues to potential data quality problems. Table 2-5 provides the weighted and unweighted response rates for the survey. Note that for the National Adult Literacy Survey the weighted and unweighted response rates are almost identical. Chapter 3 includes a detailed discussion of the weighting procedures used in the National Adult Literacy Survey.
32
Table 2-5. Screener, background questionnaire, and exercise booklet response rates for the National Adult Literacy Survey, by respondent characteristics for all sample types Survey component and subgroup Unweighted (%) Weighted (%)* Screener
89.1
--
Background questionnaire All respondents
81.0
80.5
16-24 25-44 45-64 65+
85.0 82.8 78.7 77.4
85.5 82.3 78.1 74.9
Male Female
77.9 83.5
77.9 82.7
Race/ethnicity Hispanic Black, non-Hispanic White and other
81.7 84.6 80.2
82.3 84.0 79.9
Exercise booklet All respondents
95.9
95.9
16-24 25-44 45-64 65+
98.2 96.7 94.6 89.0
98.6 96.7 94.5
Male Female
95.7 96.0
95.6 96.2
Race/ethnicity Hispanic Black, non-Hispanic White and other
95.0 94.3 96.3
95.4 94.8 96.1
Education level Some or no high school High school graduate/GED** Some college or vocational education College graduate or advanced degree
94.0 95.4 96.7 97.1
93.9 95.3 97.0 97.0
Age
Sex
Age
Sex
*
**
The weighted response rates were calculated by applying the sampling weight to each individual to account for his/her probability of selection into the sample. Weighted response rates were computed only for screened households (the probability of selection is not known for persons in households that were not screened). GED = General Educational Development certificate
33
2.6 SAMPLING FOR THE PRISON SURVEY For the survey of the prison population, background interviews were completed with 1,147 persons. The survey used a two-stage sample design. The first-stage unit, or PSU, was a state or Federal adult correctional facility selected with probability proportional to size, where the measure of size was the size of the inmate population. The second-stage unit was an inmate within a sample facility. Inmates were selected with a probability inversely proportional to the facility’s population size, so that the product of the firstand second-stage selection probabilities would be constant. The selection rates were designed to produce an average of about 12 assessments per facility. In practice, this number varied because of differences between the anticipated and actual sizes of the inmate populations. Although the sample design was intended to provide a constant overall probability of selection across all inmates, inmate selection probabilities were lowered in a few facilities because of operational constraints. In facilities with high rates of population growth, the sample size to yield a constant selection probability exceeded the maximum allowable number of interviews (22). Because the sample sizes in these facilities had to be truncated to 22, the overall selection probabilities were lower. Sections 2.6.1 and 2.6.2 describe the procedures for selecting correctional facilities and inmates, respectively. 2.6.1 Sample of Correctional Facilities In the first stage of sampling, a sample of Federal and state adult correctional facilities was selected. The correctional units in multi-location facilities were sub-sampled, and one correctional unit was selected from each multi-location facility. It was estimated that, with a sample of approximately 15 inmates from each facility, a maximum of 96 facilities would be necessary to produce the required number of completed background interviews (1,000). This estimate was based on the assumptions that approximately 80 of 96 facilities (83 percent) would cooperate and that, on average, interviews would be completed with approximately 12 to 13 inmates in each of the cooperating facilities. However, early successes in gaining the cooperation of selected facilities indicated that response rates much higher than the anticipated 83 percent were likely. Therefore, a random subsample of eight facilities was deselected and set aside as a reserve sample. Of the 88 facilities selected for data collection, 87 (one of which was discovered to be two facilities) agreed to cooperate, and one facility was determined to be ineligible. The gain of one facility offset the loss of one facility due to ineligibility, making the number of eligible facilities 88. Therefore, it was not necessary to use the reserve sample. 2.6.1.1 Sampling frame and selection of correctional facilities The sampling frame for the correctional facilities was based on the 1990 census of Federal and state prisons. The data in the frame were updated to mid-1991. State adult correctional and Federal adult correctional facilities were extracted from the census file.3 3
The youth offender facilities is a category under the state adult prisons. 34
The sample of correctional facilities was drawn from the correctional facilities frame. The facilities in the frame were stratified on the basis of their characteristics using implicit stratification. That is, the facilities were placed in a sort order according to these characteristics and were selected systematically. The following variables were used in the sort: 1) State or Federal; 2) Region: Northeast, Midwest, South, West; 3) Sex of inmates: male only, both sexes, female only; and 4) Type of facility: a) For state facilities, the categories in the sort order were maximum and closed security; medium security; minimum security; classification, diagnostic, and reception center; medical facility and hospital; work-release/prerelease; and youthful offender facility. b) For Federal facilities, the categories in the sort order were U.S. penitentiary, Federal correctional institution, federal prison camp, metropolitan correctional center, federal detention center, metropolitan detention center, federal medical center, community correctional center, and other. The facilities were sorted first according to whether they were federal or state facilities; then by region, inmate gender composition within region, and type of facility within inmate gender composition; and, finally, by the size of the facility’s inmate population within type of facility. A serpentine sort order was used for the last three variables. That is, the direction of the sort for inmate gender composition alternated between region categories, and the direction of the sort for type of facility alternated between inmate gender composition categories. From this sorted list, the sample of facilities was drawn by taking a systematic sample with probabilities proportional to the number of inmates in the facility. The number of inmates in a facility was taken as its measure of size. The reserve sample of eight facilities was drawn by taking a systematic sample, with equal probabilities of selection, from the 96 sample facilities. Table 2-6 shows the numbers of correctional facilities in the sample (excluding the reserve units), as well as facilities and inmates in the sampling frame, by stratification variables.
35
Table 2-6. Number of facilities and inmates included in the survey of the prison population, by stratification variables* Sample
Sampling frame
Facilities Stratification variable Facility Type Total State Federal State Facilities Total Region Northeast Midwest South West Facility type Maximum security Medium security Minimum security Classification, Diagnostic, and Reception center Medical facility Work-release Prelease center Youthful offender Facility Sex of inmates Male only Both sexes Female only Federal facilities Total Region Northeast Midwest South West
Facilities
Inmates
Number
%
Number
%
Number
%
88 81 7
100.0 92.0 8.0
1,345 1,250 95
100.0 92.9 7.1
712,141 654,646 57,495
100.0 91.9 8.1
81
100.0
1,250
100.0
654,646
100.0
14 18 30 19
17.3 22.2 37.0 23.5
195 264 546 245
15.6 21.1 43.7 19.6
117,221 141,988 249,705 145,732
17.9 21.7 38.1 22.3
24 37 10 4
29.6 45.7 12.3 4.9
186 392 334 43
14.9 31.4 26.7 3.4
197,230 298,380 83,909 32,896
30.1 45.6 12.8 5.0
1 3
1.2 3.7
3 265
0.2 21.2
7,653 20,505
1.2 3.1
2
2.5
27
2.2
14,073
2.1
73 5 3
90.1 6.2 3.7
1,027 117 106
82.2 9.4 8.5
584,539 43,183 26,924
89.3 6.6 4.1
7
100.0
95
100.0
57,495
100.0
1 1 3 2
14.3 14.3 42.8 28.6
13 15 50 17
13.7 15.8 52.6 17.9
8,339 10,913 27,964 10,279
14.5 19.0 48.6 17.9
36
Table 2-6. Number of facilities and inmates included in the survey of the prison population, by stratification variables* – continued Sample
Sampling frame
Facilities Stratification variable Facility type U.S. penitentiary Federal correctional Institution Federal prison camp Metropolitan correctional center Federal detention center Metropolitan detention center Federal medical center Community correctional center Other Sex of Inmates Male Both sexes Female
Facilities
Inmates
Number
%
Number
%
Number
%
1 3
14.3 42.9
6 32
6.3 33.7
7,360 29,865
12.8 51.9
3 0
42.8 0.0
34 4
35.8 4.2
11,373 3,400
19.8 5.9
0
0.0
6
6.3
1,648
2.9
0
0.0
1
1.0
867
1.5
0
0.0
2
2.1
1,679
2.9
0
0.0
7
7.4
787
1.4
0
0.0
3
3.2
516
0.9
6 1 0
85.7 14.3 0.0
72 19 4
75.8 20.0 4.2
47,281 8,808 1,406
82.2 15.3 2.5
*
Excludes reserve sample.
2.6.2 Selection of Inmates Within Facilities An upper bound of 22 inmates per facility was used to determine the inmate sample sizes for the correctional facilities. This upper bound was dictated by the practical limits on interviewing a large number of inmates per facility. First, the expected inmate sample sizes for cooperating facilities were computed under a self-weighting design to yield a total of 1,500 inmates. If a facility’s expected sample size exceeded 22, it was truncated to 22, and the sample sizes for the other facilities were inflated to yield a total expected inmate sample of 1,500. This iterative process continued until there was no facility with an expected inmate sample size greater than 22, and the expected inmate sample sizes summed to 1,500 over all cooperating facilities. Because of the uncertainty concerning inmate response rates and their availability for interview, the sample of facilities was randomly divided into two waves. The first wave included 30 percent of the facilities. The outcomes of wave 1 (in terms of response rates and inmate availability) were used to set the sampling rates for wave 2.
37
The selection of inmates was conducted within each facility using a list of names obtained from facility administrators. The interviewers received forms to complete and instructions that they were required to follow when sampling inmates from the lists.
38
Chapter 3 WEIGHTING AND POPULATION ESTIMATES Leyla Mohadjer, John Burke, James Green, and Joseph Waksberg; Westat, Inc. 3.1 GOALS OF WEIGHTING Sample weights were produced for National Adult Literacy Survey respondents who completed the exercise booklet; those who could not start the exercises because of a language barrier, a physical or mental disability, or a reading or writing barrier; and those who refused to complete the exercises but had completed background questionnaires. Separate sets of weights were computed for the incentive and non-incentive samples (refer to section 2.3 for a description of the non-incentive sample). The purpose of calculating sample weights for the National Adult Literacy Survey was to permit inferences from persons included in the sample to the populations from which they were drawn, and to have the tabulations reflect estimates of the population totals. Sample weighting was carried out to accomplish the following five objectives: 1) To permit unbiased estimates, taking account of the fact that all persons in the population did not have the same probability of selection; 2) To combine the state and national samples in an efficient manner; 3) To bring data up to the dimensions of the population totals; 4) To use auxiliary data on known population characteristics in such a way as to reduce sampling errors; and 5) To minimize biases arising from differences between cooperating and non-cooperating persons in the sample. Objective 1 was accomplished by computing base weights for the persons selected into the sample. To produce unbiased estimates, different weights must be used for various subsets of the population, whenever these subsets have been sampled at different rates. Weighting was required to account for the oversampling of Black and Hispanic persons in high-minority segments of the national sample. Furthermore, the survey specifications called for the selection of one person in households with fewer than four eligible members and two persons in households with four or more eligible members. Using this approach, members of households with only one eligible member had twice the chance of selection of those in households with two eligible members, three times the chance of selection of those in households with three eligible members, etc. Weighting was needed in these situations to prevent potentially serious biases. The base weight was calculated as the reciprocal of a respondent’s final probability of selection. For the household sample, it was computed as the product of the inverse of probabilities of selection at the primary sampling unit (PSU), segment, household, and person levels. For the prison sample, the base
39
weight was equal to the reciprocal of the product of the selection probabilities for the facility and the inmate within the facility. Section 3.2.2 provides a summary of the base weight computation. The second objective of weighting was to provide composite weights for the respondents in the 11 state samples and the respondents in the national sample PSUs in the 11 states. The national and state components applied the same sampling procedures in terms of stratification method, PSU construction, sample design, and selection at the various stages of sampling. Furthermore, the same forms were used to screen households and to collect background information and literacy assessment data in the state and national surveys. To take full advantage of this comparability, the samples were combined to produce both state- and national-level statistics. The advantage of compositing the samples was the increased sample size, which improved the precision of both state and national estimates. It should be noted that composite estimates apply only to persons ages 16–64, because data for persons age 65 and older came only from the national sample. Section 3.2.4 describes the composite estimation procedures used for the National Adult Literacy Survey. For the household components, the post-stratified base weight was multiplied by a compositing factor that combined the national and state component data in an optimal manner, considering the differences in sample size and sampling error between the two components. Up to four different compositing factors were used in each of the 11 participating states, and a pseudo factor (equal to 1) was used for all persons age 65 and older and for national component records from outside of the 11 states. The product of the post-stratified base weight and the compositing factor for a record was the composite weight. A particular state analysis can include data from all respondents, age 16 and older, in that state. However, the sampling error for state estimates will increase with the inclusion of records for respondents over age 64, because these records came from the national component only. Objectives 3, 4, and 5 were accomplished in one step by adjusting for nonresponse through poststratification and raking1 to adjusted 1990 census totals. If every selected household had agreed to complete the screener, and every selected person had agreed to complete the background questionnaire and the exercise booklet, weighted estimates based on the data would be approximately unbiased (from a sampling point of view). However, nonresponse occurs in any survey operation, even when participation is not voluntary. The best approach to minimizing nonresponse bias is to plan and implement field procedures that maintain high cooperation rates. For example, the payment of a $20 incentive in the household survey and repeated callbacks for refusal conversion were very effective in reducing
1
Raking is a special kind of poststratification in which the weights of the adjustment cells are adjusted in such a way that the weighted sample marginal totals correspond to known population totals.
40
nonresponse, and thus nonresponse bias. However, because some nonresponse occurs even with the best strategies, adjustments are always necessary to avoid potential nonresponse bias. Although the data collection was carried out in 1992, adjusted 1990 census data were used for poststratification. Undercount rates estimated by the U.S. Bureau of the Census were applied to the 1990 census count to correct for the undercoverage of some population subgroups. It was concluded that the estimates would not have been improved by extrapolating 1990 census data to the 1992 estimates of the population. The composite weights were raked so that numerous totals calculated with the resulting full sample weights would agree with the 1990 census totals, adjusted for undercount. The cells used for the raking were defined to the finest combination of age, education level, and race/ethnicity that the data would allow. Raking adjustment factors were calculated separately for each of the 11 states and then for the remainder of the United States. Section 3.2.5 describes the details of the poststratification and raking approaches. Demographic variables that were critical to the weighting were re-coded and imputed, if necessary, before the calculation of base weights. Full-sample and replicate weights were calculated for each record to facilitate the computation of unbiased estimates and their standard errors. The full-sample and replicate weights for the household components were calculated as the product of a record’s post-stratified base weight and a compositing and raking factor. The weighting procedures were repeated for 60 strategically constructed subsets from the records in the sample to create a set of replicate weights for variance estimation using the jackknife method. The replication scheme was designed to produce stable estimates of standard errors for the national and 11 individual state estimates. The full-sample and replicate weights for the prison component were calculated as the product of a record’s base weight and a nonresponse and raking factor. The base weight was calculated as the reciprocal of the final probability of selection for a respondent, which reflected the two stages of sampling (sampling facilities and sampling inmates within facilities). The base weights were then adjusted for nonresponse to reflect both facility and inmate nonresponse. The resulting nonresponse-adjusted weights were then raked to agree with independent estimates for certain subgroups of the population. 3.2 CALCULATING SAMPLE WEIGHTS FOR THE HOUSEHOLD POPULATION 3.2.1 Preliminary Steps in Weighting The data used in weighting underwent edit, frequency, and consistency checks to prevent any errors in the sample weights. The checks were performed on fields required for data weighting and were limited to
41
records that required weights (i.e., records for respondents who completed the exercise booklet and those who failed to complete a screener. The consistency checks also helped to identify any unusual values. Listings were prepared of records with missing values in any of the fields used in weighting. The listings showed the entire record: the respondent’s identification number, age, date of birth (from the background questionnaire), sex, race/ethnicity, level of education, the race of the head of household, and the number of age-eligible members and respondents in the household. The printed listings were used to review the extent of missing data, identify the pattern of missing data, and prepare for imputation. The sex and race/ethnicity data from the screener and background questionnaire were also compared for consistency. Overall, these checks found little missing data and very few records with values that differed between the screener and the background questionnaire. Most of the fields required for data weighting (race/ethnicity of the head of household; sex, age, race/ethnicity and education of the respondent) were at finer levels of detail than were necessary for the later steps of weighting. The data in these fields were, therefore, collapsed to the required levels. Most of these fields were present in both the screener and the background questionnaire, thereby providing two measures of the same item. The background questionnaire measure was preferred for all items except the race of the head of household, which was collected only on the screener. For the few cases in which the background questionnaire measure was missing, the screener measure was generally available and was used as a direct substitute. Frequencies were prepared for each item after collapsing and making direct substitutions to gauge the magnitude of the imputation task. The amount of missing data remaining after substitution was small, making the imputation task fairly straightforward. The Westat imputation macro WESDECK was used to perform hot-deck imputation for particular combinations of fields that were missing. Imputation flags were created for each of the five critical fields to indicate whether the data were originally reported or were based on substitution or imputation via WESDECK. The imputed values were used only for the sample weighting process. Several special cases required attention before the calculation of base weights. In some dwelling units, the number of eligible household members exceeded nine, the maximum allowable number on preprinted labels used by the interviewers for respondent selection. In these instances, field staff provided the total number of eligible household members to the main office, where statisticians randomly selected respondents for interview and relayed this information back to the field staff. Detailed records indicated the PSU, segment number, total number of eligible household members, and number of respondents selected in each dwelling unit. This information was retrieved and attached to each of these records before the calculation of base weights.
42
Some additional dwelling units came into the sample as part of the missed structure and missed dwelling unit procedures (refer to section 2.2.3.3 for more information), which allow units that were missed in the segment listing activities to be included in the sample with a known probability of selection. All missed dwelling units within a segment were included unless the total number of missed units in the segment was unusually large, in which case a sample of missed dwelling units was taken. Detailed records indicated the PSU, segment, number of missed dwelling units selected, and total number of missed dwelling units whenever a sample of missed units was selected. This information was retrieved and attached to each of these records prior to the calculation of base weights. A few final checks were run before base weight calculation to ensure the availability and validity of all fields required by the base weights program (fields created for the special cases mentioned above and fields for the total number of age-eligible household members and the number of sample persons for each dwelling unit). A detailed description of base weight computation is provided in the next section. 3.2.2 Computing Base Weights A base weight was calculated for each record. The base weight was initially computed as the reciprocal of the product of the probabilities of selection at each stage of sampling (as given in section 2.2.3.2). The base weight reflected the probabilities of selection at the PSU, segment, dwelling unit, and respondent levels. The final base weight included adjustments to reflect the selection of the reserve sample (see section 2.2.2), the selection of missed dwelling units (see section 2.2.3.3), and the chunking process conducted during the listing of the segments (section 2.2.2.5), and to account for the subsample of segments assigned to the non-incentive experiment (section 2.3) and the sub-sampling of respondents within households (section 2.2.4). The base weight was given by
Wbij =
1 R k h i Ci S j Pij
(1)
where Pij
=
the initial probability of selection of household j in segment i;
R
=
the adjustment factor for the selection of the reserve sample;
k
=
the adjustment factor to reflect the sub-sampling of the non-incentive sample;
hi
=
the adjustment factor for the addition of missed structures and dwelling units in segment i;
Ci
=
the adjustment factor to reflect the chunking of the segments during the listing operation; and
43
Sj
=
the factor to reflect the sub-sampling of persons in household j with multiple eligible members.
Twelve respondents in the national sample had extremely high base weights resulting from various features of the design. The base weights of these respondents were trimmed down to about three times the mean value of the base weights to avoid unnecessary increases in variances of estimates from the National Adult Literacy Survey. 3.2.3 Nonresponse Adjustments and Poststratification Before compositing the national and state samples, the base weights for each sample were post-stratified separately to known population totals. This first-level poststratification provided sampling weights with lower variation and adjusted for nonresponse. Poststratification implicitly adjusts for unit nonresponse through adjustments to the weights of the responding units. Typically, the adjustments are made for subgroups of the sample that are likely to be quite different or for subgroups with high nonresponse rates. Poststratification is appropriate when population totals are known for the subgroups, or weighting classes, of the sample. For purposes of poststratification, the entire sample was partitioned into classes, with the classification based on available survey data from respondents. Each class contained sample persons with the survey characteristics provided below. The adjustment was then implemented within each weighting class. The national and state records were split into 45 mutually exclusive and exhaustive groups, according to the state the record came from, whether the record came from the national or a state sample, and whether the record came from a PSU that was included in the national sample with certainty. The 45 groups were defined as follows: Groups 1–11
State records from PSUs that were not selected with certainty for the national component, separated by state;
Groups 12–22
State records from PSUs that were selected with certainty for the national component, separated by state;
Groups 23–33
National records from one of the states participating in the state survey, from PSUs that were not selected with certainty for the national component, separated by state;
Groups 34–44
National records from one of the states participating in the state survey, from PSUs that were selected with certainty for the national component, separated by state; and
Group 45
National records from states not participating in the state survey.
44
State records were post-stratified separately from national records to provide a common base for applying the composite weighting factors. Population totals were calculated separately for each distinct group, based on 1990 census figures adjusted for undercount, thereby providing the control totals for poststratification. (More detail on poststratification totals is presented in section 3.2.5.) A post-stratified base weight was calculated for each person in the sample as follows:
WPShi
Wb hi
NTh
M Wb
(2)
nh
i 1
hi
where WPShi
=
the post-stratified base weight for the ith person record in the hth group;
Wbhi
=
the base weight for the ith person record in the hth group;
NTh
=
the population total for the hth group; and
nh
=
the number of respondents in the hth group.
3.2.4 Compositing Data from the National and State Components 3.2.4.1 Composite estimation procedure Composite estimates were developed so that National Adult Literacy Survey data could be used to produce both state and national statistics. The original plan was to consider the national and state samples as two separate surveys, so that national statistics would be prepared from the national sample only and state data would be prepared from the state samples only. Upon reconsideration, it was clear that sampling error would be reduced by combining the state and national samples for each state that participated in the state survey. The combined sample had the advantages of producing a single database for state and national statistics and improving precision. The method of combining data from the state and national samples is referred to as composite estimation. The composite estimation procedure and issues associated with the choice of composite weights for the national and state samples are discussed in the following sections. The composite estimator for the national/state sample is given by
Î iÎ st( 1 i) Î nt
45
(3)
where
Î
=
the composite estimate for variable Y in state i;
βi
=
the composite factor for state i (0 < βi < 1);
=
the estimate of Y coming from the state sample; and
=
the estimate of Y coming from the national sample.
Îst Înt
The variance of a composite estimator will be smaller than the variance of both the national and state estimates if appropriate composite factors are used. Optimal factors can be found when unbiased estimators exist for the two components and approximate estimates of their variances are available. It should be noted that a composite estimator will produce unbiased estimates for any value of βi. The optimum value of βi is the one that results in the lowest variance. However, there is generally only a slight loss in efficiency if a reasonable approximation of the optimum value of βi is used. In most practical situations (including the national and state components of the National Adult Literacy Survey), approximations are necessary because there is insufficient information available to provide the optimal value of βi when sample weights are produced. As stated earlier, the national and state samples were selected independently, and each could thus produce unbiased estimates of sub-domain statistics for persons 16–64 years of age. Therefore, factors could be derived to produce composite estimators with variances that were smaller than those of either of the two estimates. For statistic Y, the optimal composite factor for state i is
i
ˆ nt) V (Y ˆ nt ) V( Y
V ( Yˆ st)
(4)
where V(Înt) =
V(Îst) =
the variance of the estimate of Y coming from the national sample; and the variance of the estimate of Y coming from the state sample.
A different optimal value of βi might be found for each statistic of interest. However, data analyses would be complicated if item-specific values of βi were used, because items would not add up to totals, or totals derived by summing different items would not agree. Consequently, the goal for the National Adult Literacy Survey was to associate with each person in the sample a single compositing factor that, while not precisely optimal for any particular statistic, would be robust enough to enhance the precision of virtually all composited statistics. This objective was accomplished by focusing on aspects of the sample design that were likely to affect the variance, regardless of the choice of statistic. Under simple random sampling, the
46
variance of the estimator is inversely proportional to the sample size, and the expression for βi simplifies to the following:
i
nst nst
nnt
(5)
where nst
=
the number of respondents age 16-64 in the state sample; and
nnt
=
the number of respondents age 16-64 in the national sample.
Because of the complexity of the National Adult Literacy Survey sample design, it was useful to think of deriving βi in terms of the effective sample size, i.e., the actual sample size divided by the design effect. Three aspects of the survey design tended to inflate the design effect and thereby reduce the effective sample size: clustering, stratification, and the differential sampling rates used for Black and Hispanic adults. In both the national and state components, clustering occurred at the PSU and segment levels and, to a trivial extent, at the household level, where two respondents were sampled in a small proportion of households. Geographic clustering kept the cost of survey administration down but reduced the effective sample size because of within-PSU and within-segment intraclass correlations. For example, in the Current Population Survey, which has a PSU and segment sample design similar to that of the National Adult Literacy Survey, the within-PSU and within-segment intraclass correlations have been estimated to average about 0.00075 and 0.042, respectively (Train et al., 1978). It seemed reasonable to use these values as approximations of intraclass correlations for the national and state components of the National Adult Literacy Survey. Ordinarily, stratification enhances sample efficiency, but the national PSU sample was designed to optimize the precision of national estimates. As a result, stratum boundaries did not always conform with state boundaries; in fact, because PSUs sometimes contained counties from more than a single state, the measure of size used for PSU sample selection was not always optimal for producing state estimates. This aspect of the national design affected the variances of the state-level estimates coming from the noncertainty PSUs included in the national sample. (Note that stratum boundaries do not cause any problem for PSUs selected with certainty, because they are self-representing.) In the national sample, minority households were oversampled in segments containing a high proportion of Black and Hispanic households. This practice introduced variability in the weights and increased the design effect. Minority households were not oversampled in the state survey. A separate source of variability in weights for both the national and state samples was the within-household sampling of persons, although this variability was dampened somewhat by increasing the sample size to two persons in households containing four or more eligible adults.
47
To best reflect the influence of these design aspects on the effective sample size, distinct compositing factors were derived for up to four subsets of data in each participating state. Those subsets were defined according to (1) whether or not the data came from a PSU chosen with certainty for the national sample and (2) whether or not the respondent was Black or Hispanic. 3.2.4.2 Deriving the PSU design effect As mentioned in the previous section, the national PSU sample was not designed to maximize the efficiency of state-level estimates. To estimate the relative loss of efficiency for state data resulting from the inclusion of the national non-certainty PSUs, special tabulations were produced for each of the 11 participating states. The analysis was based on a variable that was likely to be correlated with literacy at the PSU level: the percentage of persons age 25 or older who had 0–8 years of schooling. Although the use of 1990 census data would have been preferable, only 1980 figures were available at the time. First, all possible PSU samples under the national sample design were enumerated, and the between-PSU variances were computed for the estimated percentage using a Taylor series approximation. This process was repeated for the state design. These variances, which are presented in the third column of Table 3-1, were used to calculate provisional compositing factors that would have been appropriate had no within-PSU sampling been performed. These compositing factors reflect the limitations of the national stratification procedures for producing efficient state estimates. The table shows that the national design was quite adequate for producing state estimates in California but was greatly deficient in Louisiana. Under the hypothesis that the national and state designs were equally efficient, another set of compositing factors, based strictly on the counts of PSUs (excluding the certainty PSUs in the national sample), was computed. These figures are presented in the fifth and sixth columns of Table 3-1. A factor similar to a design effect was computed by taking the quotient of the ratio of the state and national compositing factors derived using the two approaches:
Fij
Between PSU variance PSU count / (1 Between PSU variance) (1 PSU count )
(6)
National between-PSU variance State between-PSU variance / number of national PSUs number of state PSUs
This factor plays a role in calculating the effective sample size, as described in the next section.
48
3.2.4.3 Estimating composite factors For data collected in PSUs selected with certainty for both the national and state samples, the effective sample size was estimated as:
ijk
neff
nijk 1
(7)
(¯nijk 1) '1 Vw
2 ijk
where i
=
a participating state;
j
=
national or state sample;
k
=
minority (Black or Hispanic) or non-minority;
nijk
=
total number of respondents ages 16-64
Ýijk
=
mean number of respondents per segment;
ρ1
=
0.042, the intraclass correlation within segment, assumed to be equal to the Current Population Survey average and to be constant across states; and
2 Vw
=
the relvariance2 of the weights.
ijk
2
Relvariance, short for relative variance, is calculated by dividing the variance on an estimate by the squared value of the estimate.
49
Table 3-1. Between-PSU variance and provisional compositing factors for the National Adult Literacy Survey national and state PSU sample designs Provisional Provisional Data Between compositing PSU compositing State source variance* factors countt factors Fij** California National 0.000498 0.4644 4 0.5000 1.15 State 0.000432 0.5356 4 0.5000 1.00 Illinois National 0.001375 0.1735 3 0.3750 2.86 State 0.000289 0.8265 5 0.6250 1.00 Indiana National 0.000401 0.0865 4 0.2500 3.52 State 0.000038 0.9135 12 0.7500 1.00 Iowa National 0.001812 0.0324 2 0.1429 4.97 State 0.000061 0.9676 12 0.8571 1.00 Louisiana National 0.002499 0.0210 1 0.1000 5.19 State 0.000053 0.9790 9 0.9000 1.00 New Jersey National 0.000430 0.0000 4 0.2857 1.00 State 0.000000 1.0000 10 0.7143 1.00 New York National 0.000127 0.3964 2 0.3333 0.76 State 0.000083 0.6037 4 0.6667 1.00 Ohio National 0.000140 0.1703 5 0.2941 2.03 State 0.000029 0.8297 12 0.7059 1.00 Pennsylvania National 0.000214 0.2571 4 0.3333 1.44 State 0.000074 0.7429 8 0.6667 1.00 Texas National 0.001482 0.1715 4 0.3333 1.44 State 0.000307 0.8285 8 0.6667 1.00 Washington National 0.000390 0.0681 1 0.1111 1.71 State 0.000029 0.9319 8 0.8889 1.00 * Of the estimated percentage of persons 25 or older (1980) with 0-8 years of schooling. t Excluding National Adult Literacy Survey certainty PSUs. ** A design-effect-like factor descriptive of the relative inefficiency of the national PSU sample design for making state estimates
For data collected in other than the certainty PSUs included in the national sample, the effective sample size was estimated as
ijk
neff
nijk 1
¯ ijk 1) '2 Pijk Fij (¯nijk 1) '1 (m
50
(8) 2 Vw
ijk
where i
=
a participating state;
j
=
national or state sample;
k
=
minority (Black or Hispanic) or nonminority;
nijk
=
total number of respondents ages 16-64;
Ýijk
=
mean number of respondents per segment;
ρ1
=
0.042, the intraclass correlation within segment, assumed to be equal to the Current Population Survey average and to be constant across states
mijk
=
mean number of respondents per segment;
ρ2
=
0.00075, the intraclass correlation within PSU, assumed to be equal to the CPS average and to be constant across states;
Pijk
=
the proportion of respondents in non-certainty PSUs;
Fij
=
a design-effect-like factor descriptive of the relative inefficiency of the national PSU sample design for making state estimates; and
V2wijk =
the relvariance2 of the weights.
Then an estimate of the optimal composite factor for state i is given by
i(State)k
neff
(9)
i(State)k
neff
i( State)k
+
neff
i(National) k
i(National)k 1 i(State)k
neff
i(National) ( k
n eff
i(State) k
+
neff
i( National )k
Table 3-2 presents each of the quantities contained in the above formulas and the final compositing factors.
51
(10)
Table 3-2. Derivation of factors used to composite National Adult Literacy Survey national and state data State California
National Certainty PSU No
Race/ethnicity Black or Hispanic Other
Yes
Black or Hispanic Other
Illinois
No
Black or Hispanic Other
Yes
Black or Hispanic Other
Indiana
No
Black or Hispanic Other
Iowa
No
Black or Hispanic Other
Louisiana
No
Black or Hispanic Other
N. Jersey
No
Black or Hispanic Other
Yes
Black or Hispanic Other
Data source National State National State National State National State National State National State National State National State National State National State National State National State National State National State National State National State National State National State
Sample size 196 62 200 260 675 226 414 457 56 29 202 417 161 198 121 378 107 126 215 947 2 45 146 1027 80 315 55 718 132 209 163 535 15 28 38 103
Persons/ segment 3.5 2.1 3.7 4.7 13.0 7.5 8.3 15.2 4.3 1.8 5.3 6.1 5.2 5.0 4.8 7.0 5.4 3.1 5.8 5.9 1.0 1.7 7.3 6.2 4.7 3.4 4.2 5.5 4.3 3.4 3.5 4.6 3.0 4.7 5.4 4.9
Persons PSU 49.0 20.7 50.0 65.0 18.7 7.25 67.3 83.4 35.7 11.5 71.7 78.9 2.00 5.63 73.0 85.6 80.0 35.0 55.0 79.8 33.0 26.1 40.8 53.5 52
Pijk* 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.3 1.0 0.7 1.0 0.8 1.0 0.8 1.0 0.5 1.0 0.6 1.0 0.0 1.0 0.0 -
Fij* 1.2 1.0 1.2 1.0 2.9 1.0 2.9 1.0 3.5 1.0 3.5 1.0 5.0 1.0 5.0 1.0 5.2 1.0 5.2 1.0 1.0 1.0 1.0 1.0 -
Relvariance of weights 0.3305 0.0804 0.1393 0.1191 0.3666 0.1083 0.1177 0.1261 0.3629 0.1414 0.0844 0.0965 0.1764 0.1292 0.1243 0.0882 0.3943 0.1324 0.0628 0.1072 0.1837 0.2007 0.0997 0.1083 0.2808 0.1562 0.1222 0.1043 0.4060 0.1182 0.0917 0.1057 0.2381 0.1391 0.1346 0.0636
Effective design effect 1.48 1.14 1.30 1.32 1.87 1.38 1.42 1.72 1.54 1.18 1.41 1.37 1.35 1.30 1.29 1.34 1.67 1.22 1.45 1.35 1.19 1.23 1.63 1.38 1.74 1.27 1.47 1.33 1.57 1.22 1.23 1.26 1.32 1.29 1.32 1.23
Sample size 132.7 54.2 154.4 196.4 361.0 163.5 290.9 265.1 36.4 24.6 143.5 303.5 119.0 152.9 94.1 282.1 64.1 103.1 148.1 700.1 1.7 36.5 89.4 743.7 45.9 248.4 37.5 539.9 84.2 171.6 132.7 426.0 11.3 21.7 28.8 83.9
Compositing factor 0.7098 0.2902 0.4401 0.5599 0.6883 0.3117 0.5232 0.4768 0.5968 0.4032 0.3210 0.6790 0.4378 0.5622 0.2502 0.7498 0.3834 0.6166 0.1746 0.8254 0.0441 0.9559 0.1073 0.8927 0.1559 0.8441 0.0649 0.9351 0.3293 0.6708 0.2375 0.7625 0.3438 0.6562 0.2554 0.7446
Table 3-2. Derivation of factors used to composite National Adult Literacy Survey national and state data – continued State New York
National Certainty PSU No
Race/ethnicity Black or Hispanic Other
Yes
Black or Hispanic Other
Ohio
No
Black or Hispanic Other
Pennsylvania
No
Black or Hispanic Other
Yes
Black or Hispanic Other
Texas
No
Black or Hispanic Other
Yes
Black or Hispanic Other
Washington
No
Black or Hispanic Other
Data source National State National State National State National State National State National State National State National State National State National State National State National State National State National State National State National State
Sample size 69 24 154 370 275 170 186 317 158 138 309 871 25 52 309 704 60 64 79 210 235 272 250 497 194 145 155 320 13 55 99 1064
Persons/ segment 5.3 1.6 5.9 6.1 7.6 7.1 5.0 9.1 4.8 2.8 4.8 5.7 2.3 2.5 5.9 6.2 3.5 3.4 4.2 5.3 3.5 3.9 3.6 5.1 6.3 5.0 5.7 10.7 1.6 1.3 6.2 6.5
Persons PSU 34.5 6.00 77.0 92.5 31.6 11.5 61.8 72.6 6.25 7.43 77.3 88.0 58.8 34.0 62.5 62.1 13.0 6.88 99.0 133.0
* As defined in Section 3.2.4.3.
53
Pijk* 1.0 1.0 1.0 1.0 1.0 0.2 1.0 0.4 1.0 0.5 1.0 0.7 1.0 0.9 1.0 0.8 1.0 0.3 1.0 0.4
Fij* 0.8 1.0 0.8 1.0 2.0 1.0 2.0 1.0 1.4 1.0 1.4 1.0 2.4 1.0 2.4 1.0 1.7 1.0 1.7 1.0
Relvariance of weights 0.2721 0.2096 0.1035 0.1083 0.3344 0.1283 0.2343 0.1063 0.3722 0.1579 0.0962 0.1040 0.6318 0.1427 0.0818 0.1055 0.1565 0.1848 0.1581 0.8570 0.3547 0.0942 0.1670 0.0971 0.3709 0.1132 0.1429 0.1068 0.4044 0.1305 0.0945 0.1096
Effective design effect 1.47 1.24 1.35 1.39 1.61 1.38 1.40 1.44 1.58 1.23 1.35 1.32 1.69 1.21 1.37 1.37 1.26 1.28 1.29 1.26 1.56 1.24 1.39 1.30 1.59 1.28 1.34 1.51 1.45 1.15 1.44 1.38
Sample size 46.9 19.4 113.8 266.3 170.5 122.9 132.5 219.4 100.0 111.9 229.0 657.4 14.8 43.1 225.2 513.6 47.5 49.8 61.2 166.1 150.4 219.3 180.0 380.9 121.9 113.2 115.5 211.5 9.0 48.8 68.8 769.7
Compositing factor 0.7075 0.2925 0.2994 0.7006 0.5812 0.4188 0.3766 0.6235 0.4724 0.5277 0.2583 0.7417 0.2555 0.7445 0.3048 0.6952 0.4881 0.5119 0.2693 0.7308 0.4069 0.5932 0.3210 0.6790 0.5185 0.4815 0.3532 0.6468 0.1578 0.8422 0.0821 0.9179
3.2.5 Computing Final Weight—Poststratification Through Raking Ratio Adjustments Poststratification is commonly used in sample surveys to accomplish three purposes: (1) It generally reduces the sampling errors; (2) it is frequently an effective way of making nonresponse adjustments; and (3) it creates consistency with statistics from other studies. The National Adult Literacy Survey used a particular form of poststratification referred to as raking ratio adjustments. The final sampling weights were computed by raking the composited weights to known population totals. In poststratification, classes are formed from cross-tabulations of certain variables. In some instances, such cross-tabulations may lead to sparse cells, or population distributions may be known for the marginal but not the joint distributions for variables used to define the weighting classes. Weighting class adjustments based on small cell sizes can result in a large amount of variation in the adjusted weights. Raking ratio adjustments are useful for maintaining the weighted marginal distributions of variables used to define weighting classes. For this type of adjustment, population distributions are required for the marginal distributions of the weighting class variables and not for their joint distribution. An objective of raking ratio adjustments is to adjust the weights of cells in such a way that the marginal distributions for the weighted sample correspond to known population distributions. To illustrate the algorithm, consider a simple case of two variables that are cross-tabulated. Using an example from Kalton (1981), the marginal and joint distributions for the population and sample are as follows. Sample 1 2 … K
Total
1
q11 q12 … q1K
q1.
W2.
2
q21 q22 … q2K
q2.
WH1 WH2 … WHK
WH.
H
qH1 qH2 … qHK
qH.
W.1 W.2 … W.K
W..
Total
q.1 q.2 … q.K
q..
Population 1 2 … K
Total
1
W11 W12 … W1K
W1.
2
W21 W22 … W2K
H Total
The iterative procedure makes successive modifications to the weights until the process stabilizes. The algorithm used for raking in the National Adult Literacy Survey, and described by Kalton, first weights each cell in row h (h=1, …, H) by the factor Wh./qh.. The result is that the sum of the weighted cells for a given row h, Wh. q h.
∑
q hi , will be equal to Wh.. Because of the adjustments to the weights, the
h
54
column totals for the sample now become
∑
q hk
h
Wh. = q h.
∑ q′
hk
= q ′.k . At the second step in the
h
iterative procedure, the sampled units in each cell in column k (k=1, …, K) are weighted by the factor
W.k _ q ′.k . Then, the sum of the weights in a given column k is equal to W.k . At this point, the q ′.k values have been changed to
∑ q′ h
hk
Wh. = q ′ h.
∑ q′ ′
hk
= q ′ ′.k . The process now repeats with step one.
h
The procedure is completed when the process converges or, alternatively, is terminated after a prespecified number of iterations. The result is a set of adjusted weights that are then used for estimation. It has been shown that the raking ratio estimation procedure produces best asymptotically normal estimates under simple random sampling. At the same time, the procedure minimizes the adjustments to the sample weights based on one measure of closeness (Ireland & Kullback, 1968). Construction of weighting classes is an important consideration in poststratification, particularly when it is used as an adjustment for unit nonresponse. A purpose of using weighting classes is to bring together respondents and nonrespondents with similar characteristics not only for the variables defining the classes but also for variables that are unknown for nonrespondents only. The variables used to construct raking classes for the National Adult Literacy Survey were age, race/ethnicity, sex, education, and geographic indicators, i.e., metropolitan statistical area (MSA) vs. non-MSA for the 11 states and census region for the remainder of the United States. The 1990 census totals used for raking were adjusted separately by age, race/ethnicity, sex, and region of the country to account for undercoverage. The undercoverage rates used in this process were supplied by the U.S. Bureau of the Census. 3.3 REPLICATED WEIGHTS FOR VARIANCE ESTIMATION IN THE HOUSEHOLD POPULATION Variance estimation must take into account the sample design. In particular, the estimate of sampling variance for any statistic should account for the effects of clustering, the use of nonresponse and poststratification adjustments, and the component of sampling variability arising from the variation in the weights used to compute the statistic. Treating the data as a simple random sample will produce underestimates of the true sampling variability. The jackknife method can be used to estimate the variance for most statistics. Jackknifing estimates the sampling variability of any statistic Y, as the sum of components of variability that may be attributed to individual pairs of first-stage sampling units. The variance attributed to a particular pair is measured by estimating how much the value of the statistic would change if only one unit in the pair had been sampled. When using replication techniques such as jackknifing to calculate standard errors, it is
55
necessary to establish a number of subsamples (or replicates) from the full sample, calculate the estimate from each subsample, and sum the squared difference of each replicated estimate from the full-sample estimate. The 60 replicates formed for the National Adult Literacy Survey provided the degrees of freedom necessary for the production of stable estimates of variance. Variance estimation requires three steps: (1) forming the replicates, (2) constructing the replicate weights, and (3) computing estimates of variance for survey statistics. The formation of replicates is discussed in detail in sections 3.3.1 through 3.3.3. After the replicates had been formed, a replicate factor was constructed for each variance stratum. Let fijk(r) denote the rth replicate factor for the kth respondent in the jth variance unit in the ith variance stratum. Then, in general:
fijk(r)
2 if i r and j 1 0 if i r and j 2 1 if i gr
(11)
and the replicated base weight, Wbijk(r), was obtained as Wbijk(r) = Wbijk fijk(r) for r = 1, 2, …, 60. (A variation on this scheme, used for only non-certainty PSUs in the state component, is described in section 3.3.2.) After obtaining a person base weight for each replicate, all remaining full-sample weighting steps leading to the final person weight were performed on each replicate. By repeating the various weight adjustment procedures on each set of replicate base weights, the impact of these procedures on the sampling variance of the estimator Y is appropriately reflected in the variance estimator, v(Y). After the replicate weights had been constructed, the estimate of variance could easily be computed for any statistic. The statistic was computed 61 times, once using the full-sample weight and an additional 60 times using each of the 60 replicate weights. The variance estimate is the sum of the 60 squared differences between the estimate derived using the full-sample weight and the estimate derived using each of the 60 replicate weights. That is, the estimate of the variance of a statistic Y is,
60
v(Y) =
∑ (Y
r
- Y)
2
r=1
where Yr = the weighted estimate obtained using the rth replicate weight; and Y = the weighted estimate obtained using the full-sample weight.
56
(12)
The National Adult Literacy Survey pooled data from a nationally representative sample of 101 PSUs and from 11 independently selected state PSU samples. The threefold objective of the replication scheme was (1) to reflect the actual sample design of each sample; (2) to ensure the production of stable estimates of standard errors by having sufficient degrees of freedom for national estimates, individual state estimates, and regional estimates; and (3) to limit the total number of replicates so that variance estimation would not be prohibitively expensive. The general approach in setting up the replication was to devise an appropriate scheme for each component of the sample, the national sample and the 11 states, and then to collapse replicates to a reasonable number. 3.3.1 Household Sample Replication for the National Component The national sample contained 101 PSUs, 25 of which were selected with certainty. The remaining 76 PSUs were selected 2 per stratum using the Durbin method (1967), with probabilities proportional to size and with known joint probabilities. Ordinarily, replicates are formed by pairing first-stage sampled units, that is, segments are paired in PSUs selected with certainty and whole PSUs are paired in non-certainty strata. However, under the Durbin scheme, an unbiased estimate of variance can be obtained by treating PSUs in some non-certainty strata as if they had been chosen with certainty, that is, by pairing segments instead of whole PSUs. For the 101-PSU sample, the natural pairing led to 74 replicates. These replicates were examined carefully to see which contained data from any of the 11 participating states. In certainty PSUs where segments from a participating state had been paired to form a replicate, the segments were grouped into subsets and were paired within each subset to increase the number of replicates and hence the degrees of freedom of the state variance estimator. This procedure expanded the number of national sample replicates to 111. 3.3.2 Household Sample Replication for the State Component An independent sample of 8 to 12 PSUs was selected in each of the 11 participating states. The largest PSUs were taken with certainty. Within each state, the remaining PSUs were grouped into strata, and from each stratum a single PSU was sampled with probability proportional to size. In PSUs selected with certainty, segments were paired to form replicates. However, the segments were grouped into subsets and paired within each subset to increase the degrees of freedom. This procedure created from 2 to 8 replicates for each PSU chosen with certainty, with a total of 113 replicates across the 11 states. Ordinarily, non-certainty PSUs would be paired to form replicates so that, for instance, a state with n such PSUs would yield n/2 replicate pairs. With the goal of increasing the degrees of freedom, an alternative procedure was adopted. The same n PSUs were used to create n-1 replicates, as follows: The active part of each replicate contained data from exactly n-1 of the n PSUs, and the base weight was multiplied by n/(n-1) rather than the usual factor of 2. One randomly selected PSU was active in all n-1
57
replicates, and a successively different one of the remaining n-1 PSUs was inactive in each of the n-1 replicates. It was possible to create n replicates from the n PSUs, but only at the expense of a bothersome complication in the variance estimation formula. The applied method kept estimation consistent with the rest of the sample and created 54 replicates across the 11 states. 3.3.3 Final Household Sample Replication for the National and State Components A total of 278 replicates had been formed at this point: 111 from the national sample, 113 from PSUs chosen with certainty for the state samples, and 54 from non-certainty PSUs chosen for the state samples. These replicates reflected the actual design of each sample and provided sufficient degrees of freedom to produce stable estimates of variance for the nation, each state, and the four census regions. However, using 278 replicates to estimate variances would be computer intensive and expensive, while providing only a slight gain in the precision of the overall estimates. Therefore, the replicates were collapsed to 60, a much more realistic number. To preserve the total number of replicates for each state, replicates from the same state were never collapsed. As often as possible, the same constraint was used by region as well. Table 3-3 presents the results of the replication scheme, showing which replicates are active for the major sub-domains of analysis. 3.4 CALCULATING SAMPLE WEIGHTS FOR THE PRISON POPULATION The final inmate weight was constructed in four major steps. The first step was to construct the inmate base weight, which was the reciprocal of the overall probability of selection for each inmate. The second step was to adjust the inmate base weight for the one facility that did not cooperate, so that weighted estimates for inmates from cooperating facilities would also represent inmates from the non-cooperating facility. The third step was to adjust the inmate weight to compensate for not obtaining a completed background questionnaire for every inmate in the sample. The fourth step was to post-stratify the weight so that the weighted counts from the sample agreed with independent estimates for certain subgroups of the population. 3.4.1 Computing Inmate Base Weights The initial correctional facility sample consisted of 96 facilities, of which eight facilities were randomly selected and set aside as the reserve sample. The reserve sample was never used because the actual response rates were higher than those originally estimated for the sample of 96 facilities. The reduced sample of facilities was drawn by taking a systematic sample, with equal probabilities of selection, from a listing of all sample facilities in their initial selection order. The facility weight for the remaining 88 facilities in the sample was computed as a product of the reciprocal of the probability of the ith facility (PSU) being selected to the initial sample and the reciprocal of the probability of its not being selected to the reserved sample; that is:
58
Wbi =
1 1 Pi 88 / 96
(13)
where Wi =
the weight for the ith facility; and
Pi =
the probability of selection of the ith facility.
The inmate base weight is the reciprocal of the overall probability of selecting the jth inmate in the ith facility.
WI bij = Wbi
Ni ni
where Ni
=
the inmate population size for the ith facility; and
ni
=
the inmate sample size for the ith facility.
59
(14)
Table 3-3. Active replicates for sub-domains of the National Adult Literacy Survey analysis file Household sample
Replicate U.S.
Northeast
Midwest
South
West
California
Illinois
1
x
x
x
x
x
x
x
2
x
x
x
x
x
x
3
x
x
x
x
x
x
x
4
x
x
x
x
x
x
x
5
x
x
x
x
x
x
x
Indiana
x
New York
Ohio
Louisiana
New Jersey
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
Pennsylvania
Texas
Washington
Prison sample
Iowa
x
6
x
x
x
x
x
x
x
x
x
x
x
7
x
x
x
x
x
x
x
x
x
x
x
8
x
x
x
x
x
x
x
x
x
x
x
9
x
x
x
x
x
x
x
x
x
x
x
10
x
x
x
x
x
x
x
x
x
x
x
11
x
x
x
x
x
x
x
x
x
x
x
12
x
x
x
x
x
x
x
x
x
x
x
13
x
x
x
x
x
x
x
x
x
x
x
14
x
x
x
x
x
x
x
x
x
x
x
15
x
x
x
x
x
x
x
x
x
x
x
16
x
x
x
x
x
x
x
x
x
x
x
17
x
x
x
x
x
x
x
x
x
x
x
18
x
x
x
x
x
x
x
x
x
x
x
19
x
x
x
x
x
x
x
x
x
x
x
20
x
x
x
x
x
x
x
x
x
x
x
21
x
x
x
x
x
x
x
x
x
22
x
x
x
x
x
x
x
x
x
23
x
x
x
x
x
x
x
x
x
x
24
x
x
x
x
x
x
25
x
x
x
x
x
x
x
x
x
x
x
26
x
x
x
x
x
27
x
x
x
x
x
x
x
x
x
x
x
x
x
28
x
x
x
x
x
x
x
x
x
29
x
x
x
x
30
x
x
x
x
x
x
x
x
x
x
x
x
x
31
x
x
x
x
x
x
x
x
x
60
x
Table 3-3. Active replicates for sub-domains of the National Adult Literacy Survey analysis file – continued Household sample
Replicate U.S.
Northeast
Midwest
South
West
California
x
x
Illinois
Indiana
Iowa
Louisiana
New Jersey
x
New York
Ohio
x
x
Pennsylvania
Texas
Washington
Prison sample
32
x
x
x
x
33
x
x
x
x
x
x
x
x
34
x
x
x
x
x
x
x
x
35
x
x
x
x
x
x
36
x
x
x
x
x
x
x
x
37
x
x
x
x
x
x
x
x
38
x
x
x
x
x
x
x
x
x
39
x
x
x
x
x
x
x
x
x
x
x
x x
x
x
x
x
x
x
x
40
x
x
x
x
x
x
x
x
x
x
41
x
x
x
x
x
x
x
x
X
x
x
42
x
x
x
x
x
x
x
X
x
x
x
43
x
x
x
x
x
x
x
x
x
x
x
44
x
x
x
x
x
x
x
x
x
x
x
45
x
x
x
x
x
x
x
x
x
x
x
46
x
x
x
x
x
x
x
x
x
x
47
x
x
x
x
x
x
x
x
x
x
48
x
x
x
x
x
x
x
x
x
x
49
x
x
x
x
x
x
x
x
x
x
50
x
x
x
x
x
x
x
x
x
x
51
x
x
x
x
x
x
x
x
x
x
x
52
x
x
x
x
x
x
x
x
x
53
x
x
x
x
x
x
x
x
x
54
x
x
x
x
x
x
x
x
x
55
x
x
x
x
x
x
x
x
x
56
x
x
x
x
x
x
x
x
57
x
x
x
x
x
x
x
58
x
x
x
x
x
59
x
x
x
x
x
60 # active
x
x
x
60
60
60
57
59
32
23
18
20
20
61
29
22
25
20
22
x
x
19
45
3.4.2 Nonresponse Adjustments 3.4.2.1 Facility nonresponse adjustment Only one correctional facility did not cooperate. As described in section 2.6.1.1, the sample facilities were stratified on the basis of certain characteristics. Using this stratification scheme, the non-cooperating facility was classified as a state maximum security facility, in the southern region of the United States, with a male-only inmate population. To adjust for the non-cooperating facility, two nonresponse adjustment classes were constructed: (1) all facilities in the same sampling stratum (implicit stratum) as the noncooperating facility and (2) all remaining facilities. The facility nonresponse adjustment factor was computed for each nonresponse class as the ratio of the weighted (facility weight times the facility inmate population size) sum of all eligible sample facilities to the respondent facilities. That is, the nonresponse adjustment factor for the αth class, AFα, was computed as
AFα
M M
i J S( ) i J SR( )
Wb i Ni (15) Wb i N i
where Wbαi
=
the facility weight for the ith facility in the αth facility nonresponse adjustment class;
Nαi
=
the inmate population count for the ith facility in the αth facility nonresponse adjustment class;
S(α)
=
the collection of all eligible (cooperating and non-cooperating) sample facilities in the αth facility nonresponse adjustment class; and
SR(α) =
the collection of all cooperating facilities in the αth facility nonresponse adjustment class.
Table 3-4 presents the facility nonresponse adjustment factors for both nonresponse adjustment classes. Table 3-4. National Adult Literacy Survey correctional facility sample counts and facility nonresponse adjustment factor, by facility nonresponse adjustment classes Sample count Nonresponse Nonresponse Eligible Respondent adjustment class adjustment factor 1 8 7 1.122 2 80 80 1.000
62
3.4.2.2 Inmate nonresponse adjustment The inmate sample consisted of 1,340 inmates, of whom 1,147 completed background questionnaires. The main reason for adjusting the sampling weights was to remove potential bias on statistics of interest as a result of the inability to collect completed background questionnaires for all sample inmates. If the probability of nonresponse were independent of the statistics of interest, then no bias would arise. Therefore, the objective was to obtain adjustment classes such that the probability of nonresponse within each class was as independent of statistics of interest as possible. There are several alternative methods of forming the classes to achieve this result. For the prison sample, the classes were formed so that the variation in the response propensity within the classes was minimized. A set of potential predictive variables was selected for the response propensity. These variables had to be available for respondents and nonrespondents alike. They were •
State vs. Federal facility;
•
Region: Northeast, Midwest, South, West;
•
Sex of inmates: male only, both sexes, female only; and
•
Facility type: maximum security, medium security, minimum security, medical, all other.
To form the nonresponse adjustment classes, a technique similar to the automatic interaction detection type of algorithm was used. Pearson chi-square statistics were computed between the response and each one of the predictive variables. The predictor with the smallest p-value was selected as the "best" predictor. Then, the same process was applied within the subgroups of the population, defined by the levels of the "best" predictor chosen in the preceding step. This process was continued until no significant predictor was found or until a specified minimum class size had been reached. The procedure is stepwise and creates a hierarchical, tree-like structure. The inmate nonresponse classes are shown in Table 3-5. Table 3-5. National Adult Literacy Survey inmate sample counts and nonresponse adjustment factors, by inmate nonresponse adjustment classes Region
Facility type
Northeast and West
Maximum security and medical
Nonresponse adjustment Respondent factor 121 1.386
Sample counts
State/Federal facility
All
All
171
Northeast and West
All other
State
330
275
1.196
Northeast and West
All other
Federal
54
51
1.063
South and Midwest
Maximum security and medical
All
212
174
1.214
South and Midwest
Medium security
All
337*
302*
1.117
South and Midwest
Minimum security and other
All
235
224
1.051
*This class actually contained 338 and 303 responding inmates, with the additional unit representing one inmate who was selected into the sample twice from two different facilities. The number of records is adjusted here to be consistent with the number of records (1,147) receiving weights.
63
The inmate nonresponse adjustment factor for the hth nonresponse adjustment class, INRAFh, was computed as
AIh
M WI M WI
i J A(h)
i J AR(h)
bhi AFhi bhi AFhi
(16)
where WIbhi
=
the base weight for the ith inmate in the hth inmate nonresponse adjustment class;
AFhi
=
the facility nonresponse adjustment factor for the ith inmate in the hth nonresponse adjustment class;
A(h)
=
the collection of all sample inmates in the hth facility nonresponse adjustment class; and
AR(h) =
the collection of all sample inmates with completed background questionnaires in the hth facility nonresponse adjustment class.
3.4.3 Poststratification Procedures To reduce the mean square error of estimates, the weights were further adjusted so that the weighted totals obtained from the sample as estimates for certain subgroups of the population would be consistent with presumably more precise estimates available from external sources. Control totals were obtained from the U.S. Department of Justice’s Bureau of Justice Statistics and were partly based on data from the 1991 Survey of Inmates in State Correctional Facilities. Both sets of estimates were obtained from larger samples than the one utilized in this survey and thus were expected to have greater precision. Poststratification was intended to reduce nonresponse-related residual bias on the estimates and simultaneously to increase the precision of the post-stratified estimates. This beneficial effect on the variance was not restricted to the post-stratified variables. The precision of any substantive variable correlated with the post-stratified variables was also expected to improve. For the male inmates, the poststratification estimation utilized raking ratio estimation. The inmate nonresponse adjusted weights were alternately adjusted by an iterative process to provide consistency with the independent estimates of population by age and then by education within each race/ethnicity category. Table 3-6 shows the sample estimates for male inmates (before raking) and the independent control totals by age and by education within race/ethnicity categories.
64
Table 3-6. Comparison of National Adult Literacy Survey sample estimates (before raking) and independent control totals, by age and by education within ethnicity, for male inmates Sample Race/ethnicity Age or education Control total Size Estimate White and other Age 107,332 173 117,604 less than 30 167,488 255 175,019 30 or more Education 49 34,375 31,496 0-8 years 272 186,001 189,149 9-12 years 107 72,246 54,175 Some college Black Age 240 165,229 155,912 less than 30 210 145,130 164,931 30 or more Education 40 27,475 35,968 0-8 years 333 230,590 239,645 9-12 years 77 52,118 45,230 Some college Hispanic Age 76,144 107 6,400 less than 30 61,543 65,569 91 30 or more Education 59 41,256 34,035 0-8 years 109 76,144 77,758 9-12 years 30 20,289 15,176 Some college
Raking ratio estimation was used rather than a straightforward poststratification procedure because the cell sizes were too small to obtain stable estimates when age and education were cross-classified within race/ethnicity. Refer to section 3.2.5 for a detailed description of raking ratio estimation. Table 3-7 shows the raking ratio estimate and the adjustment factor for each adjustment class for the male inmates. The small adjustment factors for inmates with some college education could be related to the tendency of better educated inmates to be more cooperative. A similar pattern can be observed for inmates who were less than 30 years old.
65
Table 3-7. Raking ratio estimates and weight adjustment factors for male inmates in the National Adult Literacy Survey sample Adjustment cell Race/ethnicity Education Age Adjustment factor 1 White and other 0-8 years less than 30 0.859 2 30 or more 0.943 3 9-12 years less than 30 0.067 4 30 or more 1.061 5 Some college less than 30 0.700 6 30 or more 0.768 7 Black 0-8 years less than 30 1.130 8 30 or more 1.397 9 9-12 years less than 30 0.952 10 30 or more 1.177 11 Some college less than 30 0.741 12 30 or more 0.910 13 Hispanic 0-8 years less than 30 0.684 14 30 or more 0.950 15 9-12 years less than 30 0.892 16 30 or more 1.239 17 Some college less than 30 0.614 18 30 or more 0.853 One-dimensional poststratification was used for female inmates mainly because of the small sample size for this group. The poststratification adjustment factor for the gth poststratification adjustment class, PAIg, was
PAIg
Cg
M WI
i J E(g)
(17)
bgi AFgi AIgi
where Cg
=
the female inmate control total for the gth poststratification class;
E(g)
=
the collection of female respondent inmates in the gth poststratification class;
WIbgi
=
the inmate base weight for the ith inmate in the gth poststratification class;
AFgi
=
the facility nonresponse adjustment factor for the ith inmate in the gth poststratification class; and
AIgi
=
the inmate nonresponse adjustment factor for the ith inmate in the gth poststratification class.
The poststratification factors for the female inmates are shown in Table 3-8.
66
Table 3-8. Control totals and poststratification adjustment factors for female inmates in the National Adult Literacy Survey sample, by poststratification classes Poststratification Poststratification Race/ethnicity Sample size Control total adjustment cell factor 19 Black 30 19,465 0.906 20 All other 41 23,554 0.875 3.4.4 Final Inmate Weights Final inmate weights were obtained as a product of the inmate base weight, the facility nonresponse adjustment factor, the inmate nonresponse adjustment factor, and the raking/poststratification adjustment factor: Fwghαi
=
WIbghαi AFαi AIhi PAIgi
(18)
where WIbghαi =
the base weight for the ith inmate in the αth facility nonresponse adjustment class, the hth inmate nonresponse adjustment class, and the gth poststratification class;
AFαi
=
the facility nonresponse adjustment factor for the ith inmate in the αth facility nonresponse adjustment class;
AIhi
=
the inmate nonresponse adjustment factor for the ith inmate in the hth inmate nonresponse adjustment cell; and
PAIgi
=
the poststratification/raking adjustment factor for the ith inmate in the gth poststratification class.
Table 3-9 presents statistics for the sampling weights at each stage of weight adjustment. The table shows that the variation in the base weight was rather small and that nonresponse adjustments had only a trivial effect on the weight variation. The poststratification/raking increased the weight variation moderately. Despite the increase in weight variation, poststratification/raking usually decreases the variance of estimates for any characteristics that are correlated with the raked variables (Brackstone & Rao, 1979; Oh & Scheuren, 1978). The post-stratified/raked variables in this survey are known to be strongly correlated with many substantive characteristics. The poststratification procedure was effective in simultaneously reducing the residual nonresponse bias and the sampling variance.
67
Table 3-9. Statistics for the distribution of the weight-by-weight adjustment stage for the National Adult Literacy Survey incarcerated sample Facility Inmate Post-stratified Statistic Base weight nonresponse nonresponse raked weight adjusted weight adjusted weight Sample size 1,340 1,340 1,147 1,147 Mean 582.52 588.16 687.13 667.52 cv (%) 16.51 16.56 18.43 24.94 Minimum 5th Percentile Median 95th Percentile Maximum
110.22 491.47 593.20 680.51 1,012.37
110.22 491.47 596.27 700.67 1,012.37
115.89 530.58 684.30 877.48 1,682.92
110.29 458.49 644.83 937.89 1,785.87
3.5 REPLICATED WEIGHTS FOR VARIANCE ESTIMATION IN THE PRISON POPULATION The use of a complex sample design, adjustments for nonresponse, and poststratification procedures resulted in dependence among the observations. The application of the usual formulae of variance estimation, which were based on simple random sampling assumptions, would result in the underestimation of sampling variance in this survey. To estimate sampling variability, therefore, 45 jackknife replicates were formed to provide adequate degrees of freedom for the production of reliable estimates. The variance estimation was carried out in three steps: (1) the replicates were formed, (2) the replicate weights were computed, and (3) the estimates of the variances of the survey statistics were computed. The replicates were designed in accordance with the sample design. The 86 non-certainty facilities were placed in their sample selection order. Then, the facilities were paired consecutively, and each pair was assigned to a variance stratum. This process resulted in 43 variance strata. Within each variance stratum, one facility was assigned randomly to variance unit 1 and the other to variance unit 2. The two largest facilities in the sample were assigned to separate variance strata. These facilities were certainty selections and therefore their only contribution to the total variance was from within-facility sampling. Therefore, the inmate records within each facility were placed in their sample selection order and numbered sequentially. The odd-numbered inmates were assigned to one variance unit and the even-numbered inmates to the other. Thus, a total of 45 variance strata and 90 variance units were obtained. After the replicates had been formed, the replicate weights were constructed. A replicate factor was constructed for each variance stratum. If fijk(r) denotes the rth replicate factor for the kth inmate in the jth variance unit and the ith variance stratum, then 2 if i = r and j = 1 fijk(r) = 0 if i = r and j = 2 1 if i ≠ r 68
(19)
The rth replicate inmate base weight for the kth inmate in the ith variance stratum and the jth variance unit, WIbijk(r), was then obtained as WIbijk(r)
=
(20)
WIbijk fijk(r)
for r = 1, 2,........,45. After obtaining an inmate base weight for each replicate, all remaining full-sample weighting steps leading to the final inmate weight were performed on each replicate. For each replicate, a facility nonresponse adjustment factor, an inmate nonresponse adjustment factor, and a poststratification adjustment factor were computed, and these factors were then applied to the replicate inmate base weight to obtain 45 replicate final inmate weights. Replicate weights 46 through 60 were “inactive” for the prison sample and were set equal to the full-sample weight in the data file. The variance estimation procedures were similar to those used for the household sample, as described in section 3.3.
69
Chapter 4 DEVELOPMENT OF THE SURVEY INSTRUMENTS Anne Campbell, Diné College (formerly of Educational Testing Service)
One of the goals of the 1992 National Adult Literacy Survey was to relate the literacy skills of the nation’s adults to a variety of demographic characteristics and explanatory variables. To accomplish this goal, the survey included the administration of a background questionnaire as well as literacy simulation tasks. The next three sections describe the conceptual framework for the survey and the development of the background questionnaire and the literacy tasks. 4.1 CONCEPTUAL FRAMEWORK One of the major goals of the National Adult Literacy Survey (NALS) was to compare its results with those from other large-scale assessments of literacy that have been conducted during the past few years. These include two major surveys: 1) the 1985 Young Adult Literacy Assessment, conducted as a part of the National Assessment of Educational Progress (NAEP) and carried out by Educational Testing Service (ETS) and the Response Analysis Corporation under a grant from the National Center for Education Statistics (NCES; Kirsch and Jungeblut, 1986), and 2) the 1990 Workplace Literacy Survey, conducted by ETS under a contract from the Employment and Training Administration (Kirsch, Jungeblut, and Campbell, 1992). Thus, the conceptual framework for the National Adult Literacy Survey is based on the framework developed for the Young Adult Literacy Assessment and used again in the Workplace Literacy Survey. The foundation for the 1985 Young Adult Literacy Assessment, the 1990 Workplace Literacy Survey, and the 1992 National Adult Literacy Survey was the following definition of literacy: Using printed and written information to function in society, to achieve one’s goals, and to develop one’s knowledge and potential. This definition characterizes literacy by focusing on what adults do with printed and written information. It rejects an arbitrary standard, such as signing one’s name, completing five years of schooling, or scoring at the eighth grade level on a test of reading achievement. In addition, this definition goes beyond simply decoding and comprehending text and implies that the information-processing skills that adults use to think about content are part of the concept of literacy. The National Center for Education Statistics specified in its contract requirements for conducting the National Adult Literacy Survey that ETS appoint a Literacy Definition Committee to provide substantive expertise to guide the development and conduct of the survey. The Literacy Definition Committee recommended adopting the above definition of literacy, along with the three literacy scales
70
developed to report the results of the Young Adult Literacy Assessment as the framework for the National Adult Literacy Survey. Three literacy scales—prose literacy, document literacy, and quantitative literacy—were also used in the two preceding national surveys of literacy and represent distinct and important aspects of the ability to use printed and written information. Prose literacy consists of the knowledge and skills needed to understand and use information contained in prose texts, both expository and narrative. Expository prose consists of printed information in the form of connected sentences and longer passages that define, describe, or inform, such as newspaper stories or written instructions. Narrative prose tells a story, but is less frequently used by adults in everyday life than by school children, and did not occur as often in the texts presented in the prose literacy tasks. Prose varies in its length, density, and structure (e.g., use of section headings or topic sentences for paragraphs). Using information contained in prose texts, or prose literacy, means that people can locate information contained in prose in the presence of related, but unnecessary information, find all the information, integrate information from various parts of a passage of text, and write new information related to the text. Document literacy consists of the knowledge and skills required to locate and use information found in documents. Documents differ from prose text in that they are more highly structured. Documents consist of structured prose and quantitative information, in complex arrays arranged in rows and columns, such as tables, data forms, and lists (simple, nested, intersected, or combined), in hierarchical structures such as tables of contents or indexes, or in two-dimensional visual displays of quantitative information, such as graphs, charts, and maps. Using information contained in documents, or document literacy, means that people can locate information in documents, repeat the search as many times as needed to find all the information, integrate information from various parts of a document, and write new information as requested in appropriate places in a document, while screening out related, but inappropriate information. Quantitative literacy consists of the knowledge and skills needed to apply arithmetic operations, either alone or sequentially, to numbers embedded in printed materials. Quantities can be located in either prose texts or in documents. Quantitative information may be displayed in analog form in graphs, maps, or charts, or it may be displayed in digital form using whole numbers, fractions, decimals, percentages, or time units (hours and minutes). Using quantitative information contained in prose or documents, or quantitative literacy, means that people can locate quantities while screening out related, but unneeded information, repeat the search as many times as needed to find all the numbers, integrate information from various parts of a text or document, infer the necessary arithmetic operation(s), and perform the arithmetic operation(s) correctly.
71
The three literacy scales were measured with literacy tasks that simulate the demands that adults encounter when they interact with printed materials on a daily basis (simulation tasks). The tasks used to measure literacy along the three scales incorporate many features designed to demonstrate that adults can use information, including quantitative information, contained in texts and documents. The adoption of the definition of literacy and the three scales from the Young Adult Literacy Assessment facilitated implementing the goal of comparing the demonstrated literacy proficiencies of the national survey population with those of the populations from the two prior surveys. To ensure that valid comparisons could be made by linking the scales, a set of 85 tasks that were administered in the Young Adult Literacy Assessment and in the Workplace Literacy Survey were also planned to be included in the 1992 National Adult Literacy Survey. Still, new tasks needed to be developed because some of the old tasks had become dated and because a better balance of tasks among the three scales was needed (about two-thirds of the original tasks contributed to the document scale, leaving one-sixth of the tasks for the prose scale and one-sixth for the quantitative scale). Taking into consideration the definition of literacy and the three literacy scales, the Literacy Definition Committee established the following guidelines for developing new literacy tasks: • • • • •
Continued use of open-ended simulation tasks rather than multiple-choice questions; Continued emphasis on measuring a broad range of information-processing skills covering a variety of contexts; Increased emphasis on simulation tasks that require brief written and/or oral responses; Increased emphasis on tasks that focus on asking the respondent to describe how he or she would set up and solve the problem; and The use of a simple, four-function calculator to solve quantitative problems.
Using these guidelines, an additional 81 tasks were developed specifically for the 1992 National Adult Literacy Survey in order to complement and enhance the original set of 85 literacy tasks. In addition to the definition of literacy and the three literacy scales, the administration of a background questionnaire to collect demographic and background information was also carried over from the 1985 and 1990 assessments. This information, along with the information gathered from the simulation tasks, is important for interpreting and reporting the literacy results. 4.2 THE SCOPE OF THE BACKGROUND QUESTIONNAIRE The questionnaire was intended to provide data about the U.S. adult population, enhance understanding of the factors related to the observed distribution of literacy skills, and facilitate comparisons with previous studies. A modified version of the questionnaire was developed for the prison population, as some of the questions for the population at large were not relevant for this subgroup (see Appendix H). Both background questionnaires, but not the literacy tasks, were also translated into Spanish.
72
Two goals guided the development of the questionnaire: • •
To ensure the usefulness of the data by addressing issues of concern throughout the nation; and To ensure comparability with the Young Adult Literacy Assessment and the Department of Labor Workplace Literacy Survey by including some identical questions.
In keeping with these goals, the background questionnaire addressed the following broad issues: • • • • • •
General and language background; Educational background and experiences; Political and social participation; Labor force participation; Literacy activities and collaboration; and Demographic information.
4.2.1 General and Language Background By design, the survey is a study of English literacy proficiency. Projected demographic changes, however, point to a large and growing population of adults with limited English proficiency. It was likely, therefore, that little or no information from the simulation tasks in English would be available for these individuals and, thus, they could be characterized only from the information collected in the background questionnaire. In addition, many of the questions included in the category of general and language background were important in characterizing the sample of young adults in the 1985 Young Adult Literacy Assessment; and, in fact, the age at which English was learned was found to be a powerful variable in previous analyses of the data on young adults. In order to gather as much pertinent information as possible, the questions relating to respondents’ general and language background addressed the following: • • • • • • •
Country of birth; Education before coming to the United States; Language(s) spoken by others in the home; Language(s) spoken while growing up; Language(s) spoken now; Participation in courses for English as a second language; and Self-evaluation of proficiency in English and other languages.
4.2.2 Educational Background and Experiences Although “self-educated” individuals can still be found, formal education remains among the most important factors in the acquisition of literacy skills. Level of education is known to be an important predictor of demonstrated performance on the prose, document, and quantitative literacy scales across racial/ethnic groups. The questions addressing educational background and experiences were designed to provide data for descriptive and relational analyses as well as to address some specific issues. The questions collected information on the following:
73
• • • • • • •
Highest grade or level of education completed; Reasons for not completing high school; High school equivalency; Current educational aspirations; Types and duration of training received in addition to traditional school; Context, that is, school, home, or work, in which literacy activities were learned; and Physical, mental, or health conditions that may affect literacy skills.
4.2.3 Political and Social Participation People need to read, write, and calculate in order to accomplish important tasks not only at work and in school, but also at home and in their communities. The questions included under political and social participation make it possible to explore the kinds of free-time activities that adults engage in relative to demonstrated proficiencies. Information on the use of library services is important because libraries promote reading and often provide literacy programs. In addition, because an informed citizenry is essential to political participation, and because printed material is an important medium for conveying information on public issues, information was collected on how adults keep abreast of current events and public affairs. The questions in this section addressed the following: • • • •
Sources for obtaining information about current affairs; Television viewing; Use of library services; and Voting behavior.
4.2.4 Labor Force Participation There is widespread concern that the literacy skills of both our present and future work forces are not adequate for competing in the current global economy or for coping with our rapidly evolving technological society. The questions relating to labor force participation are based on standard labor force concepts widely used in economic surveys; they allow a variety of labor market activity and experience variables to be constructed. Combined with the data on the demonstrated literacy proficiencies of adults, the labor market variables make it possible to examine associations between literacy proficiencies and the labor market experiences of key subgroups. In addition, the questions included make it possible to link results to the Department of Labor literacy survey. The questions in this section addressed the following: • • • • •
Employment status; Weekly wages or salary; Weeks of employment for the last year; Annual wages or salary; and Industry and occupation.
74
4.2.5 Literacy Activities and Collaboration Questions relating to literacy activities and collaboration addressed several important issues. Some of the questions provided information about the types of materials—newspapers, magazines, books, and brief documents—that adults read, making it possible to investigate the relationship between the types of materials read and demonstrated literacy proficiencies. Another subset of questions asked about the frequency of particular reading, writing, and mathematics activities engaged in for personal use as well as for use on the job. By asking adults about the types of literacy practices they engage in specifically for work, analyses can relate on-the-job literacy practices to various occupational categories, education levels, and income levels. The issue of collaboration was addressed by questions that asked if a person received assistance when engaging in particular literacy activities. The questions in this section collected information on the following: • • • •
Newspaper, magazine, and book reading practices; Reading, writing, and mathematics activities engaged in for personal use; Reading, writing, and mathematics activities engaged in for work; and Assistance received from others with particular literacy activities.
4.2.6 Demographic Information The inclusion of demographic variables makes it possible to describe the adult population as well as to investigate the demonstrated literacy proficiencies of major subgroups of interest, such as racial/ethnic groups, males and females, and age groups, including those over the age of 64. In addition, the data allow for the investigation of such issues as the educational experiences of White, black, and Hispanic populations as well as their access to literacy related services; the educational experiences of different generations of adults; and the relationships of socioeconomic status and family background to literacy. The demographic information collected included the following: • • • • • • • •
Educational attainment of parents; Marital status; Number of people in family employed full time and part time; Sources of income other than employment; Family and personal income from all sources; Race/ethnicity; Age; and Sex.
75
4.2.7 Prison Survey Background Questionnaire Because many of the questions for the household population were not appropriate for a prison population, a more relevant version of the background questionnaire was developed incorporating questions from the 1991 Survey of Inmates of State Correctional Facilities, sponsored by the Bureau of Justice Statistics of the U.S. Department of Justice (see Appendix H). Most of the questions in the household survey questionnaire that dealt with general and language background and with literacy activities and collaboration remained in the incarcerated questionnaire. Many of the questions dealing with education, however, were either revised or replaced with questions from the 1991 inmate survey. These questions better reflected the educational experiences of inmates both prior to their incarceration and while in prison. The questions pertaining to political and social participation in the household questionnaire were replaced with questions from the 1991 inmate survey dealing with current offenses and criminal history. Some of the questions in the household questionnaire dealing with labor force participation were replaced with questions about inmates’ prison work assignments. Several questions dealing with family income and employment status of family members were dropped from the demographic section of the questionnaire. As a result of these changes, the questionnaire for the prison population addressed the following major topics: • • • • • •
General and language background; Educational background and experiences; Current offenses and criminal history; Prison work assignments and labor force participation prior to incarceration; Literacy activities and collaboration; and Demographic information.
4.2.8 Spanish Versions of the Questionnaires Because Spanish is the second most prevalent language in this country, both the household and prison background questionnaires were translated into Spanish and administered by bilingual interviewers. The non-English, non-Spanish language groups are not prevalent enough across the country as a whole to make other translations practical for conducting the survey. Because native Spanish speakers may not be able to complete the assessment’s simulation tasks in English, it was considered important to collect background information in order to understand the language background and literacy experiences of that group. Since the survey was intended to assess only the English literacy skills of the population, the simulation tasks were not offered in Spanish. 4.3 DEVELOPMENT OF THE SIMULATION TASKS This section describes the development of the new National Adult Literacy Survey tasks as well as the scope of the combined pool of existing tasks—that is, the original tasks plus the tasks newly developed for
76
the National Adult Literacy Survey. It also describes the process of grouping the tasks into blocks or sections and then assembling these blocks into booklets for administration. 4.3.1 Organizing Framework for Task Development The framework used to develop the National Adult Literacy Survey tasks reflects research conducted on the tasks from the 1985 Young Adult Literacy Assessment, particularly with respect to the processes and strategies involved in completing the tasks. Thus, the National Adult Literacy Survey tasks served to refine and extend the three existing literacy scales—prose, document, and quantitative literacy. In developing the tasks for the National Adult Literacy Survey, one goal was to complement the tasks that had been developed for the Young Adult Assessment. This meant including a diversity of stimulus materials and designing tasks that represented the broad range of skills and processes inherent in the three domains of literacy. Furthermore, the tasks were designed to assess a wide variety of skills reflecting the demands adults encounter in occupational, community, and home settings—skills that involve reading, writing, and computing. Because the tasks were meant to simulate the kinds of activities that people engage in when they use printed materials, they were open-ended. The underlying principle for the development of the National Adult Literacy Survey tasks was that demonstrated performance on any given task reflects interactions among the following: • • •
The structure of the stimulus material, e.g., exposition, narrative, table, graph, map, or advertisement; The content represented and/or the context from which the stimulus is drawn, e.g., work, home, community; and The nature of what the individual is asked to do with the material, i.e., the purpose for using the material, which guides the strategies needed to complete the task successfully.
4.3.2 Materials/Structures The stimulus materials selected for the tasks included a variety of structures or linguistic formats that adults encounter in their daily activities. The materials were reproduced in their original format. Most of the prose materials used in the survey were expository—that is, they describe, define, or inform—since much of the prose that people read is expository in nature; however, narratives and poetry were included as well. The expository materials included a diversity of linguistic structures, from texts that were highly organized both topically and visually to those that were loosely organized. They also included texts of varying lengths, from full-page magazine articles to short newspaper articles of several paragraphs. The document tasks were based on a wide variety of document structures, which were categorized as tables, charts and graphs, forms, maps, and miscellaneous documents. Tables included matrix documents in which information is arrayed in rows and/or columns, such as transportation schedules and lists or tables of information. Documents categorized as charts and graphs included pie charts, bar graphs,
77
and line graphs. Forms included any documents that required information to be filled in, and miscellaneous structures included such materials as advertisements and coupons. Because quantitative tasks involve performing arithmetic operations on numbers embedded in print, they were based on some kind of stimulus material. The materials for quantitative tasks included both prose and document structures as there are no structures that are unique to quantitative tasks. The majority of these tasks were based on document structures. Across the entire pool of tasks, the most prevalent structure used for tasks was tables—33 percent of the materials were tables (Table 4-1). While it may seem that there was a disproportionate number of tables, this particular structure comprises a wide range of materials that present information in matrix formats using words, numbers, pictures, and symbols. Thus, materials such as transportation schedules, menus, tables of contents, as well as tables of information, were categorized as tables. Table 4-1. Percentages of stimulus materials by categories of structures Percent of Tasks Structure Original in 1985 New in 1992 Exposition 6 15 Narrative and Poetry 1 5 Tables 23 10 Charts and Graphs 4 6 Forms 13 6 Maps 1 2 Miscellaneous 4 4
Total 21 6 33 10 19 3 8
4.3.3 Adult Contexts/Content Since adults do not read printed materials in a vacuum, but rather within a particular context or for a particular purpose, materials were used that represent a variety of contexts or content. Six adult context/content areas were identified as follows: • • • • • •
Home and family: interpersonal relationships, personal finance, housing, and insurance; Health and safety: drugs and alcohol, disease prevention and treatment, safety and accident prevention, first aid, emergencies, and staying healthy; Community and citizenship: community resources and being informed; Consumer economics: credit and banking, savings, advertising, making purchases, and maintaining personal possessions; Work: occupations, finding employment, finance, and being on the job; and Leisure and recreation: travel, recreational activities, and restaurants.
An attempt was made to include as broad a range of contexts and contents as possible and to select materials that would not be so specialized as to be familiar only to certain groups. This was to ensure that any disadvantages for people with limited background knowledge would be minimized.
78
Across the entire pool of tasks, 32 percent of the materials fell into the community/citizenship category (Table 4-2). While it may seem that this category is over-represented, it is a very broad category and includes such materials as news articles from newspapers and magazines, information from governmental agencies, transportation schedules, information from schools and colleges, and so on. Table 4-2. Percentages of tasks by categories of context/content Percent of Tasks Context/Content Original in 1985 New for 1992 Home/Family 7 7 Health/Safety 3 1 Community/Citizenship 12 20 Consumer Economics 11 5 Work 13 2 Leisure/Recreation 6 13
Total 14 4 32 16 15 19
The materials and contexts described above define the axes of the matrix in Table 4-3. This table illustrates that the tasks included in the assessment were based on a variety of materials from a variety of contexts. Each dot indicates that at least one task was included that was based on a particular kind of material from a particular context. For example, the row for the content area labeled health/safety contains two dots, one under exposition and one under tables. This means the assessment included tasks that were based on two types of materials, exposition and tables, related to the context of health/safety. Table 4-3. Task coverage by context or content and type of material Materials Context/Content Home/Family Health/Safety Community/ Citizenship Consumer Economics Work Leisure/Recreation
Exposition
9 9 9 9 9 9
Narrative/ Poetry
9
Tables
9 9 9
9 9 9
Charts/ Graphs
Forms
Maps
9
9
9
9 9 9
9 9
9
9
Miscellaneous
9
9 9 9
4.3.4 Processes/Strategies After the stimulus materials were selected, tasks were developed that simulated the way people would use the materials and required different strategies for successful task completion. Prose tasks were developed that involve three strategies for processing information: locating, integrating, and generating information. For locating tasks, readers must match information given in the question with either literal or synonymous information in the text (see Exhibit 4-1, “swimmer” tasks).
79
Exhibit 4-1. Example of prose locating task
Swimmer completes Manhattan marathon The Associated Press NEW YORK-University of Maryland senior Stacy Chanin on Wednesday became the first person to swim three 28-mile laps around Manhattan. Chanin, 23, of Virginia, climbed out of the East River at 96th Street at 9:30 p.m. She began the swim at noon on Tuesday. A spokesman for the swimmer, Roy Brunett, said Chanin had kept up her strength with “banana and honey” sandwiches, hot chocolate, lots of water and granola bars.” Chanin has twice circled Man-
hattan before and trained for the new feat by swimming about 28.4 miles a week. The Yonkers native has competed as a swimmer since she was 15 and hoped to persuade Olympic authorities to add a long-distance swimming event. The Leukemia Society of America solicited pledges for each mile she swam. In July 1983, Julie Ridge became the first person to swim around Manhattan twice. With her three laps, Chanin came up just short of Diana Nyad’s distance record, set on a Florida-toCuba swim.
Find the article “Swimmer completes Manhattan marathon” on page 2 of the newspaper provided and answer the following questions. 11. Underline the sentence that tells what Ms. Chanin ate during the swim.
12. At what age did Chanin begin swimming competitively?____
Of the original prose tasks, about one-third were locating tasks, and of the new prose tasks developed for the survey, about two-thirds were locating tasks. Of the total item pool—the original and new combined—slightly over half the tasks require readers to use locating strategies. Integrating tasks require readers to pull together two or more pieces of information located at different points in the text. None of the original prose tasks were integrating tasks, and of the new prose tasks developed for the survey, about one-fourth were integrating tasks. Generating tasks require readers not only to process information located at different points in the text, but also to go beyond that information by making broad, text-based inferences in order to produce new information (see Exhibit 4-2, “Dickinson” task) or by drawing on their knowledge about a subject (see Exhibit 4-3, “Wicker” task). Of the original prose tasks, about two-thirds were generating tasks. Of the new prose tasks developed for the survey, about one-tenth were generating tasks. Of the total item pool—the original and new combined—just under a third were generating tasks.
80
Exhibit 4-2. Example of prose generating task
The pedigree of honey Does not concern the Bee— A clover, any time, to him Is Aristocracy—
(Emily Dickinson)
11. What is the poet trying to express in this poem?_________________ __________________________________________________________ __________________________________________________________ __________________________________________________________
Exhibit 4-3. Example of prose generating task (reduced from original size)
Did U.S. know Korean jet was astray? THE COMPLICITY with government into which the press has sunk since Vietnam and Watergate has seldom been more visible than on the first anniversary of Soviet destruction of Korean Air Lines Flight 007. On Sept. 1, headlines, of course, reported the Reagan administration’s statements that the event had boosted, during the year, U.S. standing in the world relative to that of the U.S.S.R. But the press effectively ignored an authoritative article in The Nation (for Aug. 18-25) establishing to a reasonable certainty that numerous U.S. government agencies knew or should have known, almost from the moment Flight 007 left Anchorage, Alaska, that it was off course and headed for intrusion into Soviet air space, above some of the most sensitive Soviet military installations. Yet no agency, military or civilian, warned Flight 007 or tried to guide it out of danger; neither did the Japanese. As late as Aug.
28, in a briefing, a State Department spokesman claimed “no agency of the U.S. government even knew the plane was off course and was in difficulty until after it was shot down.” If that’s true, the author of The Nation’s article-David Pearson, an authority on the Defense Department’s World Wide Military Command and Control System, who spent a year researching his lengthy articleconcludes, “the elaborate and complex system of intelligence, warnings and security that the U.S. has built up over decades suffered an unprecedented and mind-boggling breakdown.” But Pearson shows in excruciating detail why its most unlikely there was any such “simultaneous failure of independent intelligence systems” of the Navy, army, Air Force, National Security Agency, Central Intelligence Agency “or the Japanese self-defense agency”all of which, he shows, had ability to track Flight 007 at various stages across the Pacific.
7RP :LFNHU What’s the alternative to the staggering idea of such a breakdown? That all these agencies deliberately chose not to guide the airliner back on a safe course, because its projected overflight of the Kamchatka Peninsula and Sakhalin Island would activate Soviet radar and air defenses and thus yield a “bonanza” of intelligence information to watching and listening U.S. electronic devices. Despite all administration protests to the contrary, the evidence Pearson presents raises this alternative at least to the high probability level. But Pearson does not assert as a fact that the United States, South Korea or both deliberately planned an intelligence mission for Flight 007; he concedes the
possibility that it simply “blundered” into sensitive Soviet air space, and the electronic onlookers for the United States decided on the spot to take intelligence advantage of the error-never dreaming the Russians would shoot down an unarmed airliner. But if the disaster happened that way, Pearson notes, two experienced pilots (nearly 20,000 flying hours between them) not only made an error in setting the automatic pilot but “sat in their cockpit for five hours, facing the autopilot selector switch directly in front of them at eye level, yet failed to see that it was set improperly.” Nor in all that time could they have used the available radar and other systems to check course and position. Pearson also presents substantial evidence that Soviet radar detection and communications systems over Kamchatka and Sakhalin were being jammed that night which would help account for their documented difficulty in catching up to Flight
007. He reconstructs electronic evidence too, to show that the airliner changed course slightly after passing near a U.S. RC-135 reconnaissance plane; otherwise it would have crossed Sakhalin far north of the point where a Soviet fighter finally shot it down. The jamming and course change, as detailed by Pearson, strongly suggest what he obviously fears: “that K.A.L. 007’s intrusion into Soviet airspace, far from being accidental, was well orchestrated,” with the Reagan administration, at some level, doing the orchestrating. Even if not, the deliberate silence-or shocking failure-of so many U.S. detection systems argue that President Reagan and the security establishment have greater responsibility for Flight 007’s fate than they admit-or that a complaisant press has been willing to seek. Copyright© 1984 by The New York Times Company. Reprinted by permission.
Find the article “Did U.S. know Korean jet was astray?” on the front page of the newspaper provided and answer the question below. 8. What argument is Tom Wicker making in his column?____________________ ________________________________________________________________ ________________________________________________________________ ________________________________________________________________
81
T he strategies required by document tasks also include locating, integrating, and generating information as well as cycling through information. For locating tasks, readers must match one feature or category of information given in the task with either identical or synonymous information in a document. (see E xhibit 4-4, “Social Security card” task). A bout two-thirds of the original document tasks and about two-thirds of the new document tasks were locating tasks. T hus, about two-thirds of the total document pool were locating tasks. E xhibit 4-4. E xample of document locating task 1. Here is a Social Security card. Sign your name on the line that reads “sign ature.”
Respondents were given a copy of a Social Security card to complete this task. [Note: T he critical element in scoring this task was not a proper signature, but successfully locating the place where the signature belongs.]
Cycling tasks require the reader to repeat the matching process by identifying all instances that satisfy a set of conditions stipulated in the question or directive (see E xhibit 4-5, “employment form” task). A bout one-ninth of the original document tasks, but none of the new document tasks were cycling tasks. Of the total document literacy pool, about one-tenth were cycling tasks. E xhibit 4-5. E xample document cycling task Y ou have gone to an employment center for help in finding a j ob. Y ou know that this center handles many different kinds of j obs. A lso, several of your friends who have applied here have found jobs that appeal to you. T he agent has taken your name and address and given you the rest of the form to fill out. Complete the form so the employment center can help you get a job. B irth date Height
Age
Sex: Male
Weight
Female
Health
L ast grade completed in school K ind of work wanted: Part-time
Summer
Full-time
Y ear-round
82
[Note: this document was scored as two tasks: one for entering all personal elements (birth date, age, sex, height, weight, health, and schooling) and another for entering the two features of the kind of work wanted. T he later task did not fit the IR T scale and was not included in figuring document literacy scale scores.]
To complete integrating tasks, readers must either match on two or more features located in different parts of the document or compare and/or contrast information (see Exhibit 4-6, “graph” task). About one-ninth of the original, and one-fourth of the new document tasks were integrating tasks. Of the total document pool, about one-seventh were integrating tasks. Exhibit 4-6. Example document integrating task
13. You are a marketing manager for a small manufacturing firm. This graph shows your company’s sales over the last three years. Given the seasonal pattern shown on the graph, predict the sales for Spring 1985 (in thousands) by putting an “X” on the graph.
As with generating tasks in the prose domain, generating tasks involving documents require readers to go beyond information in the document either by drawing on their knowledge of the subject or by making inferences to produce new information. About one-ninth of the original, and one-tenth of the
83
new document tasks were generating tasks. Of the total document pool, about one-tenth were generating tasks. Quantitative tasks require readers to perform arithmetic operations—addition, subtraction, multiplication, or division—either singly or in combination. Some quantitative tasks require readers to explain how they would solve a problem rather than just to produce a numerical answer, and others require the use of a simple, four-function calculator to solve the problem. Tasks can be more or less difficult for readers depending on the type of arithmetic operation involved, the ease of determining what operations were needed, and the ease of locating or identifying the appropriate numbers. Among the National Adult Literacy Survey tasks, the representation of numerical information associated with the quantitative tasks included whole numbers, decimals, percentages, fractions, and time (hours and minutes). Addition and subtraction tasks are usually considered the easiest operations (see Exhibit 4-7, “deposit slip” task). Of the original quantitative tasks, about one-fourth each involved the operations of addition and subtraction. Of the new quantitative tasks, about one-fifth were addition and somewhat more than one-fifth were subtraction tasks. Across the total quantitative pool, about one-fourth each were addition and subtraction tasks.
Exhibit 4-7. Example quantitative addition task Availability of Deposits )XQGV IURP GHSRVLWV PD\ QRW EH DYDLODEOH IRU LPPHGLDWH ZLWKGUDZDO 3OHDVH UHIHU WR \RXU LQVWLWXWLRQ¶V UXOHV JRYHUQLQJ IXQGV DYDLODELOLW\ IRU GHWDLOV
&UHGLWLQJ RI GHSRVLWV DQG SD\PHQWV LV VXEMHFW WR YHULILFDWLRQ DQG FROOHFWLRQ RI DFWXDO DPRXQWV GHSRVLWHG RU SDLG LQ DFFRUGDQFH ZLWK WKH UXOHV DQG UHJXODWLRQV RI \RXU ILQDQFLDO LQVWLWXWLRQ
3/($6( 35,17
&$6+ /,67 &+(&.6 %< %$1. 12
92 and YY < 100, IYEARS = IYEARS + 99 If YY = 92 and MM > 8, IYEARS = IYEARS + 99 If YY = 92 and MM < 9, IYEARS = IYEARS + 100 If YY > 77 and YY < 92, IYEARS = IYEARS + 100 DAGE = IYEARS EXCEPTIONS if all or part of birth date are missing: If day of birth only is missing (DD = 00), DD = 01 was used in the BDATE formula above. If month or year of birth are missing (MM = 00 or YY = 77), DAGE was set to age as provided on the Household Screener, if available; otherwise, DAGE was set to blanks.
DRACE (RACE/ETHNICITY F-9, F-10, AND F-11) 01 = Black not Hispanic 02 = Hispanic, Mexican 03 = Hispanic, Puerto Rican 04 = Hispanic, Cuban 05 = Hispanic, Central/South American or Other 06 = Asian 07 = American Indian/Alaskan Native 08 = Pacific Islander 09 = White 10 = Other If background question F-10 coded 'yes' then response to background question F-11 Hispanic origin was used. If background question F-10 coded 'no' then response coded for background question F-9 was used.
I- 2
DCBIRTH (COUNTRY OF BIRTH A-1) 1 = USA 2 = Spanish speaking countries 3 = European language countries 4 = Asian language countries 5 = All other countries 6 = Missing Spanish speaking countries 01 07 11 12 13 14 17 18 20 27
Argentina Bolivia Chili Columbia Costa Rica Cuba Dominican Republic Ecuador El Salvador Guatemala
29 41 44 48 49 53 58 65 66
Honduras Mexico Nicaragua Panama Peru Puerto Rico Spanish Uruguay Venezuela
31 35 37 42 43 46 51 52 54 56 59 60 69
Hungary Ireland Italy Netherlands New Zealand Norway Poland Portugal Russia Scotland Sweden Switzerland Yugoslavia
61 62 67
Taiwan (China) Thailand Vietnam
European language countries 02 03 05 08 09 10 15 16 21 22 23 24 25
Australia Austria Belgium Brazil Bulgaria Canada Czechoslovakia Denmark England Finland France Germany Greece
Asian language countries 30 39 40 50
Hong Kong Japan Korea Philippines
Other All other countries coded
I- 3
DAGEARR (AGE OF ARRIVAL IN THE USA: A-2 - AGE [COMPUTED FROM F-12]) Median of A-2 [Years lived in the U.S.] - Actual Age) A-2
1 to 5 years 6 to 10 years 11 to 20 years 16 to 20 years 21 to 30 years 31 to 40 years 41 to 50 years 51 or more
= 3 years = 8 years = 13 years = 18 years = 25 years = 35 years = 45 years = 51 years
DLANGHM (LANGUAGE SPOKEN IN HOME WHILE GROWING UP A-4) DLANGBS (LANGUAGE SPOKEN BEFORE STARTING SCHOOL A-5) 01 02 03 04 05 06 07 08 09 10
English only English & Spanish English & European English & Asian Spanish & Other Other Spanish only European only Asian only English & Other
"2nd Response" "1st Response"
None
Spanish
European
Asian
Other
English
1
2
3
4
10
Spanish
7
7
5
5
5
Other
6
5
8
9
6
English language 07
English
Spanish language 29
Spanish
I- 4
European languages 04 05 06 09 10 11 12 13 14 17 19 22
Czech Danish Dutch Finnish Flemish French Gaelic German Greek Hungarian Italian Latin
24 25 26 27 28 31 34 36 44 46 47
Norwegian Polish Portuguese Romanian Russian Swedish Ukrainian Yiddish Lithuanian Serbo-Croatian Slovak
35 41 43 45
Vietnamese Cambodian Lao, Laotian Philippine
Asian languages 03 20 21 32
Chinese Japanese Korean Thai
Other languages All other language codes DLANGRW (LANGUAGE FIRST LEARNED TO READ/WRITE A-6) 1 = English language 2 = Spanish language 3 = European languages 4 = Asian languages 5 = Other non-English languages 6 = Cannot read or write DLANGSP (LANGUAGE USUALLY SPOKEN NOW A-15) 1 = English language 2 = Spanish language 3 = European languages 4 = Asian languages 5 = Other non-English languages
I- 5
DBILING (NON-ENGLISH ORALITY A-8 PARTS A & B) 1 = Yes, non-English orality 2 = Not non-English oral very well or well
not well or not at all
understand
1
2
speak
1
2
DBILIT (NON-ENGLISH LITERACY A-8 PARTS C & D) 1 = Non-English literate 2 = Not non-English literate very well or well
not well or not at all
read
1
2
write
1
2
I- 6
DLINGL (ENGLISH ORALITY A-17 PARTS A & B) (A-16 in the Prison sample) 1 = Yes, English orality 2 = Not English orality very well or well
not well or not at all
understand
1
2
speak
1
2
DLITER (ENGLISH LITERACY A-17 PARTS C & D) (A-16 in the Prison sample) 1 = Yes, English literate 2 = Not English literate very well or well
not well or not at all
read
1
2
write
1
2
DWEEKWG (WEEKLY WAGE D-3) 0 = weekly wage could not be computed If D-3 per hour, multiply by number of hours indicated in D-4 If D-3 per day, multiply by number of days indicated in D-4 If D-3 per week, used amount as given If D-3 per two-week period, divided D-3 by 2 If D-3 per month, divided D-3 by 4.3 If D-3 per year, divided D-3 by 52
DWKYRWG (AVERAGE WEEKLY WAGE D-7) 0 = weekly wage could not be computed If D-7 per hour, multiply by number of hours indicated in D-8 If D-7 per day, multiply by number of days indicated in D-8 If D-7 per week, used amount as given If D-7 per two-week period, divided D-8 by 2 If D-7 per month, divided D-8 by 4.3 If D-7 per year, divided D-8 by 52
I- 7
DFAMINC (FAMILY INCOME F-7) Dollar amount indicated in question F-7 01 = Under $5,000 02 = $5,000 - $9,999 03 = $10,000 - $14,999 04 = $15,000 - $19,999 05 = $20,000 - $29,999 06 = $30,000 - $39,999 07 = $40,000 - $49,999 08 = $50,000 - $74,999 09 = $75,000 - $99,999 10 = $100,000 and over 11 = Refused and I don't know
DWKSWRK (NUMBER OF WEEKS WORKED LAST YEAR D-5 PARTS A & B) If D-5A equal to '1' weeks worked = 00 If D-5A equal to '2' weeks worked = value coded in D-5B If D-5A equal to '3' weeks worked = 52 D-5B equal to 88 omitted or 99 incomplete, weeks worked = 00
CLFSTAT (CURRENT LABOR FORCE STATUS D-1 AND D-2) D-1 Doing last week Working full-time Working part-time Working two or more jobs Unemployed, laid off With job but not at work With job, family leave In school Keeping house Retired Doing volunteer work Other
= 1 = 2 = 3 = 4 = 5 = 6 = 7 = 8 = 9 = 10 = 11
D-2 Have you looked for a job during past four weeks Yes = 1 No = 2
I- 8
New codes: 1 = Employed full-time (D-1 = 1 or 3) 2 = Employed part-time (D-1 = 2) 3 = Employed, not at work (D-1 = 5 or 6) 4 = Unemployed (D-1 = 4) 5 = Out of labor force (D-1 = 7, 8, 9, or 10) ANNEARN (ANNUAL EARNINGS D-7A WEEKLY WAGE, AND DWKSWRK WEEKS WORKED) ANNEARN = D-7A x DWKSWRK (weeks worked last year) LABFRES (LABOR FORCE RESERVES - DERIVED FROM CLFSTAT) 0 = Not labor force reserves 1 = Labor force reserves If age less than 65, and Current Labor Force Status equal to Unemployed OR with job but not at work and zero weeks worked last year then LABFRES = 1 If age greater than or equal to 65, OR if Current Labor Force Status is equal to 1, 2, or 3 OR if Weeks Worked is greater than 0 then LABFRES = 0 otherwise LABFRES = blank HEDENRN (CURRENTLY ENROLLED IN HIGHER EDUCATION B-5 AND B-6) 0 = Not currently enrolled 1 = Currently enrolled If B-5 equal '1' Yes currently enrolled and B-6 equal '2' Vocational, '3' Two year college, '4' Four or five-year college, or '5' Master's, PH.D or other advanced degree, HEDENRN = 1 If B-5 equal '2' No and B-6 equal '1' high school diploma, '6' Other or '7' None, HEDENRN = 0 BASCANY (EVER ENROLLED IN BASIC SKILLS PROGRAM B-7) 0 = No basic skills program (B-7 equal No (2)) 1 = Yes basic skills program (B-7 equal Yes (1))
I- 9
BASCONE (ENROLLED IN BASIC SKILLS PROGRAM IN THE LAST YEAR B-7 AND B-9) 0 = No basic skills program in the last year 1 = Yes, basic skills program within the last year If B-7 equal '1' and B-9 A,B,C, or D = '1' or '2' BASCONE = 1 If B-7 equal '2' BASCONE = 0
BASCFIV (TAKEN BASIC SKILLS PROGRAM WITHIN THE PAST 5 YEARS B-7 AND B-9) 0 = No basic skills program within the past 5 years 1 = Yes, basic skills program within the past 5 years If B-7 equal '1' and B-9 A,B,C, or D = '1','2' or '3' BASCONE = 1 If B-7 equal '2' BASCONE = 0
YRSLFEX (POTENTIAL YEARS OF LABOR MARKET EXPERIENCE AGE AND B-1) YRSLFEX = Actual age - years of schooling - 5 years of schooling B-1
YRSLFEX
Still in high school =0 Less than high school = grade specified, 8 if blank Some high school = grade specified, 11 if blank GED/High school = 12 years High school graduate = 12 years Vocational, trade or business = 13 years College: less than two years = 13 years College: associate's degree = 14 years College: two years or more, no degree = 15 years College graduate = 16 years Postgraduate/No degree = 17 years Postgraduate/Degree = 17 years Don't Know = blank Omitted = blank
I-10
POVLEVL (POVERTY LEVEL NUMBER OF HOUSEHOLD MEMBERS AND INCOME F-7) 0 = Not poor/near poor 1 = Poor/near poor If number of household members is greater than zero and household income is not missing Classified as poor/near poor household size equal 1 and income less than $8665 household size equal 2 and income less than $11081 household size equal 3 and income less than $13575 household size equal 4 and income less than $17405 household size equal 5 and income less than $20570 household size equal 6 and income less than $23234 household size equal 7 and income less than $26322 household size equal 8 and income less than $29506 household size equal 9 and income less than $34927 All others classified as not poor/near poor DERIVED VARIABLES FOR THE PRISON SAMPLE DIB0801: Derived months in educational program Derived from B-8 (months & years) BIB0801 contains the responses to "How long ..." BIB0802 contains the responses to "Specify ..." but are in different units depending whether BIB0801 indicated months or years DIB1101: Derived months in vocational program Derived from B-11 (months & years) BIB1101 contains the responses to "How long ..." BIB1102 contains the responses to "Specify ..." but are in different units depending whether BIB1101 indicated months or years DIC0301: Derived years since arrest Derived from C-3 (month and year arrested) BIC0301 contains the responses to "Month" BIC0302 contains the responses to "Year" These two items were converted to a date value, subtracted from the date value of the interview, and converted to years with two decimal places. DIC0401: Derived years since admission Derived from C-4 (month and year of prison admission) BIC0401 contains the responses to "Month" BIC0402 contains the responses to "Year"
I-11
These two items were converted to a date value, subtracted from the date value of the interview, and converted to years with two decimal places. DIC0601: Derived times on probation as a juvenile Derived from C-6, but C-5 could be used for improvement. BIC0501 may indicate "never" on probation, and could be used to fill in with valid data many of the blank elements that this variable now fails to count as zero. BIC0601 contains the responses to "How many times ..." BIC0602 contains the responses to "Specify ..." 1 = None 2 = 1 time 3 = 2 times 4 = 3-5 times 5 = 6+ times 6 = I don’t know DIC0701: Number of times on probation as an adult Derived from C-7, but C-5 could be used for improvement. BIC0501 may indicate "never" on probation, and could be used to fill in with valid data many of the blank elements that this variable now fails to count as zero. BIC0701 contains the responses to "How many times ..." BIC0702 contains the responses to "Specify ..." 1 = None 2 = 1 time 3 = 2 times 4 = 3-5 times 5 = 6+ times 6 = I don’t know DIC0901: No. of times served as a juvenile Derived from C-9, but C-8 could be used for improvement. BIC0801 may indicate "never" served time, and could be used to fill in with valid data many of the blank elements that this variable now fails to count as zero. BIC0901 contains the responses to "How many times ..." BIC0902 contains the responses to "Specify ..." 1 = None 2 = 1 time 3 = 2 times 4 = 3-5 times 5 = 6+ times 6 = I don’t know DIC1001: No. of times served as an adult Derived from C-10, but C-8 could be used for improvement. BIC0801 may indicate "never" served time, and could be used to
I-12
fill in with valid data many of the blank elements that this variable now fails to count as zero. BIC1001 contains the responses to "How many times ..." BIC1002 contains the responses to "Specify ..." 1 = None 2 = 1 time 3 = 2 times 4 = 3-5 times 5 = 6+ times 6 = I don’t know DIC1201: Derived years to release Derived from C-12, but C-11 could be used for improvement. BIC1101 may indicate that there is no release date, and could be used to change what is now counted as valid data to a missing value, since the release date is undefined for such prisoners. BIC1201 contains the responses to "Month" BIC1202 contains the responses to "Year" The date value of the interview was subtracted from the date value obtained by combining these two items, and then converting the result to years with two decimal places. OTHER = Valid responses DIC1401: Der. years to earliest possible release Derived from C-14, but C-13 could be used for improvement. BIC1301 may indicate that there is no possible release date, and could be used to change what is now counted as valid data to a missing value, since the possible release date is undefined for such prisoners. BIC1401 contains the responses to "Month" BIC1402 contains the responses to "Year" The date value of the interview was subtracted from the date value obtained by combining these two items, and then converting the result to years with two decimal places. OTHER = Valid responses DD03601: Derived hours assigned to job last week Derived from D-3 BD03601 contains a continuous version of this item. 1 = 0-19 hours 2 = 20-29 hours 3 = 30-39 hours 4 = 40 hours 5 = 41 + hours DID0501: Derived amount earned last month Derived from D-5 BID0501 contains a continuous version of this item.
I-13
1 = 1 - 19 dollars 2 = 20 - 29 dollars 3 = 30 - 39 dollars 4 = 40 - 59 dollars 5 = 60 + dollars DID1001: Derived number of months worked Derived from D-10, but D-9 could be used for improvement. DID0901 may indicate did not work for pay, and could be used to fill in with a valid zero many of the zero elements that this variable now considers to be missing. BID1001 contains the responses to "How many months ..." BID1002 contains the responses to "Specify ..."
RECODING OF DERIVED VARIABLES AND RE-RELEASE OF NEW NALS DATA FILES AND ELECTRONIC CODEBOOK A careful review of the released dataset uncovered some minor problems with the coding of some of the derived variables. To remedy this situation, the problematic variables were recoded to reflect a more accurate definition. As a result of this recoding, the 3 data files, household, prisoner, and non-incentive, were re-issued so as to reflect these coding changes. Below we provide a list of all derived variables that were recoded in 1999 in each of the three datasets. Detailed documentation on how these variables were recoded is included in the new release of the electronic codebook (version 3) under each variable’s description. Variables not rederived in version 3 maintain the same derivatives as in version 2. Household dataset: DAGE DRACE DCBIRTH DAGEARR DLANGRW DBILING DBILIT DLINGL DLITER DBUSIND DOCCUP DWKSWRK CLFSTAT LABFRES HEDENRN POVLEVL
derived age in years as of September 1, 1992 derived race/ethnicity derived country of birth age of arrival in the US derived language first learned to read/write derived bilingual orality derived biliteracy ability derived English orality (19a_b) derived English literacy (17c_d) derived business or industry code derived occupation codes derived number of weeks worked last year derived current labor force status derived labor force reserves derived currently enrolled in higher education derived poverty level
Prison dataset: DRACE DCBIRTH DAGEARR
derived race/ethnicity derived country of birth age of arrival in the US
I-14
DLANGRW DBILING DBILIT DLINGL DLITER DIB0801 DIB1101 DIC0301 DIC0401 DIC0601 DIC0701 DIC0901 DIC1001 DIC1201 DIC1401 DID1001
derived language first learned to read/write derived bilingual orality derived biliteracy ability derived English orality (19a_b) derived English literacy (17c_d) derived months in educational program derived months in vocational program derived years since arrest derived years since admission derived times on probation as a juvenile number of times on probation as an adult number of times served as a juvenile number of times served as an adult derived years to release derived years to earliest possible release derived number of months worked
Non-incentive dataset: DAGE DRACE DCBIRTH DAGEARR DLANGRW DBILING DBILIT DLINGL DLITER DBUSIND DOCCUP DWKSWRK CLFSTAT LABFRES HEDENRN
derived age in years as of September 1, 1992 derived race/ethnicity derived country of birth age of arrival in the US derived language first learned to read/write derived bilingual orality derived biliteracy ability derived English orality (19a_b) derived English literacy (17c_d) derived business or industry code derived occupation codes derived number of weeks worked last year derived current labor force status derived labor force reserves derived currently enrolled in higher education
In addition, due to a change in the definition of bilingual and biliterate ability, two new variables reflecting the new definitions were created (DTBILNG and DTBILIT) for each of the three datasets and appended at the end of each re-released dataset. Finally, it was discovered that the program codes generated by the electronic codebook for the nonincentive sample misread the position of the second records (starting with the variable SCOR100) by 34 columns. This problem has been resolved in the new version of the electronic codebook.
I-15
2-DIG OCCP
3-DIG OCCP
1
DERIVED OCCUPATION CODES (DOCCUP) & Detailed Census occupations (BLD1001)
43 63
ARCHITECTS/SURVEYORS Architects Surveyors & mapping scientists
44 45 46 47 48 49 53 54 55 56 57 58 59
ENGINEERS Aerospace engineers MetALLurgical+materials engineers Mining engineers Petroleum engineers Chemical engineers Nuclear engineers Civil engineers Agricultural engineers Electrical & electronic engineers Industrial engineers Mechanical engineers Marine & naval architects Engineers, n.e.c.
2
3
ALL 64 65 66 67 68
MATH/COMP SCIENTISTS Computer systems analysts & scientists Ops & systems researchers & analysts Actuaries Statisticians Mathematical scientists, n.e.c.
4
ALL 69 73 75 76 77 78 79 83
NATURAL SCIENTISTS Physicists & astronomers Chemists, excpt biochemists Geologists & geodesists Physical scientists, n.e.c. Agricultural & food scientists Biological & life scientists Forestry & conservation scientists Medical scientists
5
ALL 95
REGISTERED NURSES Registered nurses
6
ALL 84 85 86 87 88 89
HEALTH DIAGNOSTICS Physicians Dentists Veterinarians Optometrists Podiatrists Health diagnosing practitioners, n.e.c.
I-16
2-DIG OCCP
3-DIG OCCP
DERIVED OCCUPATION CODES (DOCCUP) & Detailed Census occupations (BLD1001)
7
ALL 96 97 98 99 103 104 105 106
OTHER HEALTH RELATED Pharmacists Dieticians Respiratory therapists Occupational therapists Physical therapists Speech therapists Therapists, n.e.c. Physicians assistants, n.e.c.
8
ALL 23
ACCOUNTANTS/AUDITORS Accountants & auditors
9
ALL 3 4 5 6 14
PUBLIC SECT EX.& MAN Legislators Chief execs & genl admins, pub admin Admins+Offcls, Public admin Administrators, protective service Admins: education & related fields
10
ALL 7 8 9 13 15 17 18 21 22
PRIVAT SECT EX.& MAN Financial managers Personnel & labor relations managers Purchasing managers Managers: mkting, advertising, pub rels Managers: medicine & health Managers: food service & lodging Managers: properties & real estate Managers: service organizations, nec Managers & administrators, n.e.c.
11
ALL 16 19 24 25 26 27 28 29 33 34 35 36 37
OTHER MANAGEMENT Postmasters & mail superintendents Funeral directors Underwriters Other financial officers Management analysts Personnel, training & labor rels specs Purchasing agents & buyers, farm prods Buyers, wholesale & retail, excpt farm Purchasing agents & buyers, n.e.c. Business & promotion agents Construction inspectors Inspectrs+complnce offcrs,excpt const Management related occs, n.e.c.
I-17
2-DIG OCCP
3-DIG OCCP
DERIVED OCCUPATION CODES (DOCCUP) & Detailed Census occupations (BLD1001)
12
ALL 113 114 115 116 117 118 119 123 124 125 126 127 128 129 133 134 135 136 137 138 139 143 144 145 146 147 148 153 154 155 156 157 158 159
TEACHERS Earth, environ, & marine sci teachers Biological science teachers Chemistry teachers Physics teachers Natural science teachers, n.e.c. Psychology teachers Economics teachers History teachers Political science teachers Sociology teachers Social science teachers, n.e.c. Engineering teachers Mathematical science teachers Computer science teachers Medical science teachers Health specialties teachers Business, commerce, & marketing teacher Agriculture & forestry teachers Art, drama & music teachers Physical education teachers Education teachers English teachers Foreign language teachers Law teachers Social work teachers Theology teachers Trade & industrial teachers Teachers, postsecondary, n.e.c. Postsecondary teachers, subj unspecifie Teachers, prekindergarten & kindergarte Teachers, elementary school Teachers, secondary school Teachers, special education Teachers, n.e.c.
13
ALL 163 164 165 166 167 169 173 174 175 176 177
OTHER PROFESSIONALS Counselors, educational & vocational Librarians Archivists & curators Economists Psychologists Social scientists, n.e.c. Urban planners Social workers Recreation workers Clergy Religious workers, n.e.c.
I-18
2-DIG OCCP
3-DIG OCCP
DERIVED OCCUPATION CODES (DOCCUP) & Detailed Census occupations (BLD1001)
178 179 183 184 185 186 187 188 189 193 194 195 197 198 199
Lawyers Judges Authors Technical writers Designers Musicians & composers Actors & directors Paintrs sculptrs craftrs & art prntmkrs Photographers Dancers Artists, performers & rel wrkrs, n.e.c. Editors & reporters Public relations specialists Announcers Athletes
14
ALL 213 214 215 216 217 218
ENGINEERING TECHNIC. Electrical & electronic technicians Industrial engineering technicians Mechanical engineering technicians Engineering technicians, n.e.c. Drafting occupations Surveying & mapping technicians
15
ALL 203 204 205 206 207 208
HEALTH TECHNICIANS Clin laboratory technolgsts & technicn Dental hygienists Health record technolgsts & technicn Radiologic technicians Licensed practical nurses Health technolgsts & technicns, nec
16
ALL 223 224 225
SCIENCE TECHNICIANS Biological technicians Chemical technicians Science technicians, n.e.c.
17
ALL 226 227 228 229 233 234 235
OTHER TECHNICIANS Airplane pilots & navigators Air traffic controllers Broadcast equipment operators Computer programmers Tool programmers, numer cntl Legal assistants Technicians, n.e.c.
18
ALL 253
SALES REPRESENTATIVE Insurance sales occupations
I-19
2-DIG OCCP
3-DIG OCCP
DERIVED OCCUPATION CODES (DOCCUP) & Detailed Census occupations (BLD1001)
254 255 256 257 258 259
Real estate sales occupations Securities & financl servs sales occups Advertising & related sales occups Sales occuptns, other business servcs Sales engineers Sales reps: mining, mfg & wholesale
19
ALL 243
SALES SUPERV.& PROPR Supervisors & prop, sales occupations
20
ALL 263 264 265 266 267 268 269 274 275 276 277 278 283 284 285
OTHER SALES RELATED Sales workers: motor vehicle & boats Sales workers: apparel Sales workers: shoes Sales workers: furniture, home furnshng Sales workers: radio, TV, hifi & applnc Sales workers: hardware, buildng supply Sales workers: parts Sales workers: other commodities Sales counter clerks Cashiers Street & door-to-door sales workers News vendors Demonstrators, promotrs & models: sales Auctioneers Sales support occupations, n.e.c.
21
ALL 375 376 377 378
ADJUSTORS AND INVEST Insurance adjustr, examiner & investgtr Investigatrs & adjustrs, exc. insurance Eligibility clerks, social welfare Bill & account collectors
22
ALL 308 309
COMPUTER EQUIP OPER. Computer operators Peripheral equipment operations
23
316 317 318 319 323
INFORMATION CLERKS Hotel clerks Transport ticket & reservation agents Receptionists Information clerks, n.e.c.
24
ALL 313
SECRETARIES Secretaries
25
ALL 314
STENOGR/TYPISTS Stenographers
I-20
2-DIG OCCP
3-DIG OCCP
DERIVED OCCUPATION CODES (DOCCUP) & Detailed Census occupations (BLD1001)
315
Typists
26
ALL 303 304 305 306 307
SUPERVISORS Supervisors: general office Supervisors: computer eqpmt operators Supervisors: financial records processg Chief communications operators Supervisors: distrib, sched, adj clerks
27
ALL 326 327 328 329 335 336 337 338 339 343 344 345 346 347 348 353 354 355 356 357 359 363 364 365 366 368 373 374 379 383 384 385 386 387 389
OTHER ADMIN. SUPPORT Correspondence clerks Order clerks Personnel clerks, excpt payroll+timekep Library clerks File clerks Records clerks Bookkeepers, acctng & auditing clerks Payroll & timekeeping clerks Billing clerks Cost & rate clerks Billing, posting & calc machine operatr Duplicating machine operators Mail preparing & paper handling mach op Office machine operators, n.e.c. Telephone operators Communications equipment operators Postal clerks, exc. postal service Mail carriers, postal service Mail clerks, exc. postal service Messengers Dispatchers Production coordinators Traffic, shipping & receiving clerks Stock & inventory clerks Meter readers Weighers, measurers, checkers, samplers Expediters Material record, sched & distrib clerks General office clerks Bank tellers Proofreaders Data-entry keyers Statistical clerks Teachers aides Administrative support occs, n.e.c.
28
ALL 553
CONSTRUCTION CRAFTS Supervisors: brickmasn, stonemsn & tile
I-21
2-DIG OCCP
29
3-DIG OCCP
DERIVED OCCUPATION CODES (DOCCUP) & Detailed Census occupations (BLD1001)
554 555 556 557 558 563 565 566 567 569 573 575 576 577 579 583 584 585 587 588 589 593 594 595 596 597 598 599
Supervisors: carpenters & reltd workers Supervrs: electrcns & powr transm instl Supervrs: paintrs, pprhangers, plasters Supervrs: plumbrs, pipefttrs, steamftrs Supervisors: construction, n.e.c. Brickmasons & stonemasons Tile setters, hard & soft Carpet instALLers Carpenters Carpenter apprentices DrywALL instALLers Electricians Electrician apprentices Electrical power instALLers & repairers Painters, construction & maintenance Paperhangers Plasterers Plumbers, pipefitters & steamfitters Plumber, pipeftr & steamftr apprentices Concrete & terrazzo finishers Glaziers Insulation workers Paving, surfacing & tamping equip opers Roofers Sheetmetal duct instALLers Structural metal workers Drillers, earth Construction trades, n.e.c.
ALL 503 505 507 508 509 514 515 516 517 518 519 523 525 526 527 529 533 534
OTH CRAFT/PREC.PROD Supervisors: mechanics & repairers Automobile mechanics Bus, truck, stationary engine mechanics Aircraft engine mechanics SmALL engine repairers Automobile body & releated repairers Aircraft mechanics, exc. engine Heavy equipment mechanics Farm equipment mechanics Industrial machinery repairers Machinery maintenance occupations Electronic repairers, comm & indus eqpt Data processing equipment repairers Household appliance & power tool repair Telephone line instALLers & repairers Telephone instALLers & repairers Misc electrc+electronc equipt repairers Heating, a/c, & refrigeration mechanics
I-22
2-DIG OCCP
3-DIG OCCP
535 536 538 539 543 544 547 549 613 614 615 616 617 628 634 635 636 637 639 643 644 645 646 647 649 653 657 658 666 667 668 669 674 675 677 678 679 683 684 686 687 688 689 693 694 695 696 699
DERIVED OCCUPATION CODES (DOCCUP) & Detailed Census occupations (BLD1001)
Camera, watch & musical instrmnt repair Locksmith & safe repairers Office machine repairers Mechanical controls & valve repairers Elevator instALLers & repairers Millwrights Specified mechanics & repairers, n.e.c. Not specified mechanics & repairers Supervisors: extractive occupations Drillers, oil well Explosives workers Mining machine operatives Mining occupations, n.e.c. Supervisors: production occupations Tool & die makers Tool & die maker apprentices Precision assemblers, metal Machinists Machinist apprentices Boilermakers Precisn grindrs, filrs & tool sharpenrs Patternmakers & model makers, metal Lay-out workers Precious stone & metals workers (Jewel) Engravers, metal Sheet metal workers Cabinet makers & bench carpenters Furniture & wood finishers Dressmakers Tailors Upholsterers Shoe repairers Misc precision apparel & fabric workers Hand molders & shapers, exc jewelers Optical goods workers Dental lab & medical appliance technicn Bookbinders Electrical & electronic equip assembler Misc precision workers, n.e.c. Butchers & meat cutters Bakers Food batchmakers Inspectors, testers & graders Adjusters & calibrators Water & sewage treatment plant operatrs Power plant operators Stationary engineers Misc plant & system operators
I-23
2-DIG OCCP
3-DIG OCCP
DERIVED OCCUPATION CODES (DOCCUP) & Detailed Census occupations (BLD1001)
30
ALL 803 804 806 808 809 813 814 823 824 825 826 828 829 833 843 844 848 849 853 855 856 859
TRANSPORT OPERATIVE Supervisors: motor vehicle operators Truck drivers Driver-sales workers Bus drivers Taxicab drivers & chauffeurs Parking lot attendants Motor transportation occupations, n.e.c Railroad conductors & yardmasters Locomotive operating occupations Railroad brake, signal & switch opers Rail vehicle operators, n.e.c. Ship captains & mates, exc fishng boats Sailors & deckhands Marine engineers Supervisors: material moving equipment Operating engineers Hoist & winch operators Crane & towing operators Excavating & loading machine operators Grader, dozer & scraper operators Industrial truck & tractor equip opers Misc material moving equip operators
31
ALL 783 784 785 786 787 789 793 795 796 797 798 799
FABRIC/ASSEM/INSPECT Welders & cutters Solderers & brazers Assemblers Hand cutting & trimming occupations Hand molding, casting & forming occups Hand painting, coating & decoratng occs Hand engraving & printing occupations Misc hand working occupations Productn inspectrs, checkrs & examinrs Production testers Production samplers & weighers Graders & sorters, exc agricultural
32
ALL 703 704 706 707 708 709 713 715
OTHER ASSEM/OPER/FAB Lathe & turning machine set-up operatrs Lathe & turning machine operators Punching & stamping press machine opers Rolling machine operators Drilling & boring machine operators Grindg, abradg, buffg & polishg mach op Forging machine operators Misc mtal, plastc, stone, glass work op
I-24
2-DIG OCCP
33
3-DIG OCCP
DERIVED OCCUPATION CODES (DOCCUP) & Detailed Census occupations (BLD1001)
717 719 723 724 725 726 727 728 733 734 735 736 737 738 739 743 744 745 747 748 749 753 754 755 756 757 758 759 763 764 765 766 768 769 773 774 777 779
Fabricating machine operators, n.e.c. Molding & casting machine operators Metal plating machine operators Heat treating equipment operators Misc metal & plastic processng machn op Wood lathe, routing & planing machn ops Sawing machine operators Shaping & joining machine operators Misc woodworking machine operators Printing press operators Photoengravers & lithographers Typesetters & compositors Misc printing machine operators Winding & twisting machine operators Knittng, loopng, tapng & weavng mach op Textile cutting machine operators Textile sewing machine operators Shoe machine operators Pressing machine operators Laundering & dry cleaning machine opers Misc textile machine operators Cementing & gluing machine operators Packaging & filling machine operators Extruding & forming machine operators Mixing & blending machine operators Separatg, filterg & clarifyg mach opers Compressing & compacting mach operators Painting & paint spraying machine opers Roasting & baking machn operators, food Washing, cleaning & pickling mach opers Folding machine operators Furnace, kiln & oven operatrs, exc food Crushing & grinding machine operators Slicing & cutting machine operators Motion picture projectionists Photographic process machine operators Miscellaneous machine operatrs, n.e.c. Machine operators, not specified
ALL 864 865 866 869 874 875 876 877
CLEAN EQUIP.HNDL/LAB Supervrs: handlrs, eqpm cleanrs, labors Helpers: mechanics & repairers Helpers: construction trades Construction laborers Production helpers Garbage collectors Stevedores Stock handlers & baggers
I-25
2-DIG OCCP
3-DIG OCCP
DERIVED OCCUPATION CODES (DOCCUP) & Detailed Census occupations (BLD1001)
878 883 885 887 888 889
Machine feeders & offbearers Freight, stock & material handlers, nec Garage & service station related occups Vehicle washers & equipment cleaners Hand packers & packagers Laborers, except construction
34
ALL 456 457 458 459 461 462 463 464 465 466 467 468 469
PERSONAL SERVICE OCC Supervisors: personal service occupatns Barbers Hairdressers & cosmetologists Attendants, amusement & recreatn facils Guides Ushers Public transportation attendants Baggage porters & bellhops Welfare service aides Family child care providers Early childhood teachers assistants Child care workers, n.e.c. Personal service occupations, n.e.c.
35
ALL 413 414 415 416 417 418 423 424
PUBLIC SAFETY Supervisors: fire & fire prevntn occs Supervisors: police & detectives Supervisors: guards Fire inspection & prevention occs Firefighting occupations Police & detectives, public servant Sheriffs, bailiffs & othr law enforcmnt Correctional institution officers
36
ALL 445 446 447
HEALTH SERVICES Dental assistants Health aides, exc. nursing Nursing aides, orderlies & attendants
37
ALL 404 405 406 407 425 426 427 433 434 435
OTHER SERVICES Cooks, private household Housekeepers & butlers Child care workers, private household Private household cleaners & servants Crossing guards Guards & police, exc. public service Protective service occs, n.e.c. Supervisors: food prep & service occpns Bartenders Waiters & waitresses
I-26
2-DIG OCCP
3-DIG OCCP
DERIVED OCCUPATION CODES (DOCCUP) & Detailed Census occupations (BLD1001)
436 438 439 443 444 448 449 453 454 455
Cooks Food counter, fountain & related occpns Kitchen workers, food preparation Assistants to waiters/waitresses Miscellaneous food preparation occuptns Supervisors: cleaning & bldg servc Maids & housemen Janitors & cleaners Elevator operators Pest control occupations
38
ALL 473 474 475 476 494 497
MANAGER/OPERATORS Farmers, exc. horticultural Horticultural specialty farmers Managers: farms, exc. horticultural Managers: horticultural specialty farms Supervisors: forestry & logging workers Captains & othr officers, fishing vessl
39
ALL 477 479 484 485 486 487 488 489 495 496 498
OTHER FARM/FISH/HUNT Supervisors: farm workers Farm workers Nursery workers Supervisors: related agricultural occs Groundskeepers & gardeners, exc. farm Animal caretakers, except farm Graders & sorters, agricultural prods Inspectors, agricultural products Forestry workers, except logging Timber cutting & logging occupations Fishers
40
ALL 903 904 905
MILITARY Commissioned officers & warrant officrs Non-commissioned offcrs & othr enlisted Military occupation, rank not specified
41
ALL 999
{MISSING} {DOES NOT APPLY}
42
ALL 999
NOT WORKED PAST 3 YR {DOES NOT APPLY}
43
ALL
NEVER WORKED
I-27
Appendix J SPECIAL CODES FOR CONTINUOUS VARIABLES
Appendix J: Special Codes for Continuous Variables NATIONAL ADULT LITERACY SURVEY J.1
INTRODUCTION
This appendix lists special codes and their meanings for continuous variables and noncognitive items in Blocks ID, WT and BQ. Continuous variables and items that are not shown here do not have special codes associated with them. For discrete variables and items, labels for all values can be found in the data set codebooks. The codes for background questionnaire item BLA0102, "Country of Birth," are provided in Appendix K; that item was designated continuous because the discrete numerical data included a large number of valid responses which could not be listed in the codebooks. Special codes that apply to variables and items in all samples are listed in Section J.2; special codes for variables and items which are found only in the Prison sample are presented in Section J.3.
J.2
SPECIAL CODES FOR CONTINUOUS FIELDS APPEARING IN ALL SAMPLES
Field Name
Special Code
Description
BQTIME
999
Missing
EXTIME
999
Missing
DAGEARR
0
Missing
DWEEKWG
0
Missing
DWKYRWG
0
Missing
BLA0102
0 99
Missing, Skipped In Error Missing
BLD0301
77777 88888 99999
Respondent Refused Missing Could Not Be Computed
BLD0801
77777 88888 99999
Respondent Refused Missing Could Not Be Computed
BLD0901
999
Missing
BLD1001
999
Missing
BD07502
77
Year of Birth Omitted
J- 1
Note: If the entire date of birth question had no response, Month of Birth (BD07501), Day of Birth (BLF1201) and BD07502 are all coded "00."
J.3
SPECIAL CODES FOR CONTINUOUS FIELDS THAT APPEAR ONLY IN THE PRISON SAMPLE
Field Name
Special Code
Description
DIB0801
0
Missing
DIB1101
0
Missing
DID1001
0
Missing
BIB0802
99
Missing
BIC0302
0
Missing
BIC0402
0
Missing
BIC0602
0
Missing
BIC0902
0
Missing
BIC1002
0
Missing
BIC1202
1991
Missing
BIC1402
1991
Missing
BD03601
888 999
BID0501
77777 88888 99999
BID1002
888 999
Don't Know, Not Specified Missing, Skipped In Error Respondent Refused Missing Could Not Be Computed Don't Know Missing
J- 2
Appendix K COUNTRY OF BIRTH CODES
Appendix K: Country of Birth Codes NATIONAL ADULT LITERACY SURVEY Table K-1 provides information about the valid data values and their associated labels for background questionnaire item A-1, Country of Birth (field name BLA0102).
Table K-1. Country codes 01 Argentina 02 Australia 03 Austria 04 Bahamas 05 Belgium 06 Bermuda 07 Bolivia 08 Brazil 09 Bulgaria 10 Canada 11 Chile 12 Colombia 13 Costa Rica 14 Cuba 15 Czechoslovakia 16 Denmark 17 Dominican Rep. 18 Ecuador
19 Egypt 20 El Salvador 21 England 22 Finland 23 France 24 Germany 25 Greece 26 Guam 27 Guatemala 28 Haiti 29 Honduras 30 Hong Kong 31 Hungary 32 India 33 Indonesia 34 Iran 35 Ireland 36 Israel
37 Italy 38 Jamaica 39 Japan 40 Korea 41 Mexico 42 Netherlands 43 New Zealand 44 Nicaragua 45 Nigeria 46 Norway 47 Pakistan 48 Panama 49 Peru 50 Philippines 51 Poland 52 Portugal 53 Puerto Rico 54 Russia
00 Skipped in Error
[No territory or country code was specified in BLA0102, although BLA0101 was coded "2" (U.S. Territory) or "3" (Other Country).]
K- 1
55 Saudi Arabia 56 Scotland 57 South Africa 58 Spain 59 Sweden 60 Switzerland 61 Taiwan (China) 62 Thailand 63 Turkey 64 Utd. Arab Emir. 65 Uruguay 66 Venezuela 67 Vietnam 68 Virgin Islands 69 Yugoslavia 70 United States 71 Other
Appendix L NOTES ON SCORING
Appendix L: Notes on Scoring NATIONAL ADULT LITERACY SURVEY L.1
INTRODUCTION
This appendix provides information about scoring. Four topics are covered: 1) the scoring for five cognitive items for which scoring keys could not be provided; 2) the reasons four cognitive items were omitted from scaling; 3) the method by which items were scored Right, Wrong, Omitted and Not Reached for input to the computer programs used for scaling; and 4) a table of exceptions to the general item scoring method, with information about the method applied to each exceptional item. L.2
SCORING FOR CERTAIN COGNITIVE ITEMS
For five cognitive items (AB60502, AB60503, AB70801, AB70802, and N120801), scoring keys are not provided, as the scoring could not be expressed as a correct value or range of values. Items AB60502 and AB60503 were scored together as a single item; AB70801 and AB70802 were also scored jointly. Scoring for N120801 simply could not be expressed as a range of values. Details on the scoring of each of these items are presented below. Refer to the codebooks for information about data values which occurred for the individual items.
AB60502 and AB60503
These items were scored jointly to produce a single score for input to the computer programs used for scaling. The combined scoring method was as follows: If the respondent scored "1" on either or both items, the score was Right; a "2" on both items was Wrong. Note: For all respondents, scores of "I Don't Know" (7), "Omitted" (8), and "Not Reached" (9), when given, applied to both items. See Section L.4 for scoring information about these categories.
AB70801 and AB70802
These items were scored jointly to produce a single score for input to the computer programs used for scaling. The combined scoring method was as follows: If the respondent scored "1," "2," "3," "4," "5," or "6" on AB70801, the score was Right; if not, a score of "1," 2," "3," "4," "5," or "6" on AB70802 was Wrong; otherwise, a "0" on both was Omit; also, an "8" (Newspaper Missing) on both was Not Reached, and a "99" on AB70801 was Not Reached. Note: Respondents with "I Don't Know" (77) or "Newspaper Missing" (8) scores received them for both items. The general scoring method for "I Don't Know," described in Section L.4, applies to this item.
N120801
The values "1" and "3" are the correct keys for this item.
L- 1
L.3
COGNITIVE ITEMS OMITTED FROM SCALING
IRT scaling parameters are not given for four cognitive items, AB40904, AB60503, AB70105 and AB70802. The reasons these items were omitted from scaling are provided below.
AB40904
This item was considered to measure a secondary trait, because a correct score on this item was dependent upon having received a correct score to the previous question, AB40901.
AB60503
This item was scored jointly with AB60502 to produce a single score for input to the scaling programs. The IRT parameters for the joint "item" are listed with item AB60502.
AB70105
Due to a poor fit with the IRT model, this item was excluded from calibration.
AB70802
This item was scored jointly with AB70801 to produce a single score for input to the scaling programs. The IRT parameters for the joint "item" are listed with item AB70801.
L.4
ITEM SCORING FOR SCALING PURPOSES -- GENERAL METHOD
For input to the computer programs used for IRT scaling, all possible item responses are assigned to one of four categories: Wrong (0), Right (1), Omitted (2), or Not Reached (3). This section explains the method applied to most of the NALS cognitive items to convert individual item data values to these four categories. Section L.5 contains information about exceptions to this general method. Refer to the codebooks for the data values possible for each individual cognitive item. The correct key(s) for the item were considered Right (1). Nonresponses which are followed by valid responses to other items in the same block (section) were considered Omitted (2). In the public-use files and in the codebooks, these nonresponses are designated by the special codes for "Omitted." Nonresponses which occur after the last item in a block (section) with a valid response were considered Not Reached (3). In the public-use files and in the codebooks, these nonresponses are designated by the special codes for "Not Reached." Multiple responses were considered Wrong (0). "I Don't Know" responses were considered Wrong (0). All other responses were considered Wrong (0).
L- 2
L.5
ITEM SCORING FOR SCALING PURPOSES -- SPECIAL CASES
For some items, the scoring for scaling purposes was more complex than the method described in Section L.4. In some cases, an item's potential data values could include values with meanings such as "Illegible," "Off Task," "Incomprehensible," or "Newspaper Missing." The numerical value associated with these categories varied from item to item; as an aid to the user, the associated scoring category for each of these data values is listed individually for each item in Table L-1. In other cases, items which had been administered in earlier literacy surveys were scored for scaling as they had been scaled previously, in order to make trend comparisons possible. All items that had to be scored by a method other than that described in Section L.4 are listed in Table L-1. The table shows to which of the four scoring categories each valid response to an item would be assigned. Boldface item responses in Table L-1 may be considered exceptions to the general method. Table L-1. Exceptions to general scoring method for scaling Item Field Name
Right (1)
Wrong (0)
Omitted (2)
Not Reached (3)
AB21101
1
2, 7
8
3, 9
AB21201
1
2, 7
8
3, 9
AB30501
1
2, 7
8
3, 9
AB30601
2, 3, 4
1, 7
77, 88
99
AB31201
2, 3
1, 7, 8
77, 88
99
AB40901
1
2, 6, 7
8
3, 9
AB40904
0, 1, 2, 3, 4, 5
6
8
9
AB50101
1
2, 7
8
3, 9
AB50301
2, 3
1, 7
77, 88
99
AB60201
2, 3, 4
1, 7, 8
77, 88
99
AB60502, AB60503
These items were combined for scoring. See Section L.2 for details.
AB60601
1
2, 7
8
3, 9
AB70104
1
2
7, 8
9
AB70105
1
2
7, 8
9
AB70801, AB70802
These items were combined for scoring. See Section L.2 for details.
L- 3
Table L-1. Exceptions to general scoring method for scaling — Continued Item Field Name
Right (1)
Wrong (0)
Omitted (2)
Not Reached (3)
AB71201
1
2, 7
8
3, 9
N081001
1, 2
3, 4, 7
8
5, 9
N090601
1
2, 7
8
3, 9
N090701
1
2, 7
8
3, 9
N090801
1
2, 7
8
3, 9
N090901
1
2, 3, 4, 5, 7
8
6, 9
N091001
1
2, 3, 4, 5, 6, 7, 77
88
8, 99
N110302
1
2, 3, 7
4, 8
9
N110303
1, 2
3, 4, 7
5, 8
9
N120901
1
2, 7
8
3, 9
N121001
1
2, 7
8
3, 9
N121101
1, 2
3, 7
8
4, 9
N130101
1
2, 3, 8, 77
4, 88
99
N130102
1
2, 8, 77
3, 88
99
N130103
1
2, 3, 8, 77
4, 88
99
N130104
1
2, 3, 8, 77
4, 88
99
L- 4
Appendix M NOTES ON VARIABLES
Appendix M: Notes on Variables NATIONAL ADULT LITERACY SURVEY
M.1
INTRODUCTION
This appendix provides sample-specific information for the users of certain variables. It is divided into sections for each sample.
M.2
MAIN (HOUSEHOLD) SAMPLE
The Taylor Series Linearization Variance Stratum and Paired Sampling Unit identifiers (TSLVSTR and TSLVPSU) can be used to estimate standard errors using a Taylor Series Linearization package such as SUDAAN. CAUTION: When using SUDAAN to produce standard errors for an individual state, the user should process the entire file and specify the state in a SUBPOPN statement, rather than processing a separate analysis file containing records for that state alone. This procedure is necessary because in some instances the National PSU sample design disregarded state boundaries in its stratification scheme.
M.3
PRISON SAMPLE
For the Taylor Series Linearization Variance Paired Sampling Unit identifier (TSLVPSU), "3" is a valid number for this sample.
M.4
NON-INCENTIVE SAMPLE
Some continuous variables in Block WT of the Non-incentive sample are provided in a different format than variables with the same names that appear in other samples. The user should note that the Person Base Weight (BASEWT), Step 1 and 2 Raking Factors (S1RAKFC and S2RAKFC), Final Weight (WEIGHT), and the sixty replicate weights (REPL01 - REPL60) have a field width of 11 in the Nonincentive sample data file. For other samples, these variables have a field width of 10. The number of implicit decimal places (4) is unchanged, however.
M- 1
Appendix N IMPACT OF TREATMENT ON DISTRIBUTION OF SCALE SCORES
Appendix N: Impact of Treatment on Distribution of Scale Scores NATIONAL ADULT LITERACY SURVEY by Andrew Kolstad, National Center for Education Statistics
Combined with other background information, there is strong evidence to support the notion that nonresponse to the cognitive tasks was not a random occurrence. Procedures to incorporate proficiency values for nonresponding individuals were established based on the field study and were thus in place before the National Adult Literacy Survey data analyses were conducted. The selected procedure used self-reported reasons for nonresponse to assign wrong answers to all the missing cognitive responses of adults when the reasons were related to literacy skills. If the reasons were unrelated to literacy, then no wrong answers were assigned. After the wrong answers were (or were not) assigned to the cognitive tasks that were presented each adult, normal scaling procedures for creating plausible values were applied. The normal procedure uses statistical imputation to estimate plausible values, even in the absence of any cognitive data. Table N-1 shows the average proficiency scores resulting from this procedure, based on the fullscale survey. Adults who completed the assessment had average proficiencies ranging from 279 to 285 on the three literacy scales. Because the missing responses of adults who were unable to complete the assessment for reasons related to literacy were treated as wrong answers, the average proficiency scores of these adults were considerably lower, ranging from 114 to 124. The proficiency scores of nearly all who were unable to complete the assessment were in the Level 1 range (below 225). Adults who stopped for other reasons or for unstated reasons had scores between those of the other two groups, ranging from 228 to 237. These adults were not found only in the lowest literacy level, but were distributed across the five levels. Those who were assigned wrong answers did not score zero on the three scales. Zero is the score value given to wrong answers, but the resulting scale scores of those given wrong answers were from 114 to 124, depending on the scale. Zero has no intrinsic meaning on the adult literacy scales, since each was derived from a latent variable with a mean of zero and a variance of one, transformed by adding a constant and multiplying by a fixed factor in order to eliminate negative numbers and decimal points in the resulting scale.
N- 1
Table N-1. Percentage and average proficiencies on each scale, by assessment completion status: Adults, 1992 Average proficiency Assessment completion status Prose Document Quantitative Percent literacy literacy literacy Total 100% 272 267 271 Completed assessment 88% 285 279 284 Did not completed assessment 6% 124 116 114 for literacy-related reasons Did not complete assessment 6% 237 228 231 for reasons unrelated to literacy N.1 The distribution of performance The method chosen for placing those with incomplete cognitive data on the literacy scale takes many factors into account, and as a result does not put those with missing data into a single point along the scales. The distributions on each scale are shown in Exhibits N-1, N-2, and N-3. These distributions were generated from a 3000-case subsample of the 1992 data, selected with a probability inversely proportional to the case weight, thereby providing a self-weighting subsample. The method of sample selection also took into account the segment from which the respondents were selected, so that the average number of adults selected from each segment fell from 6.9 to 1.3, considerably reducing any possible impact of clustering on the findings. The distributions were generated using a nonparametric method for estimating a density function, and can be thought of as a smoothed histogram (SAS Institute, 1991). In effect, the fixed intervals of the histogram are replaced by a moving window of selected width, within which each observation is smoothed. Two densities are plotted for each scale: one for the complete data, and one for the data leaving out any cases with less than sufficient cognitive data (for any reason). The area between the two curves represents the locations along the distributions where the partially or totally missing data appear. It is reassuring to note that the distributions are unimodal, and that the methods selected for treating the missing data did not assign all such cases to the same point along the literacy scales, and very few, if any, cases occurred at the zero point on the scales. Nevertheless, most of the cases with missing data occur in the bottom half of the distribution, so leaving out the cases with missing cognitive data would clearly have biased the findings of the survey in an upward direction.
N- 2
Exhibit N-1. Prose literacy distribution of U.S. adults, with and without missing cognitive data
Exhibit N-2. Document literacy distribution of U.S. adults, with and without missing cognitive data
Exhibit N-3. Quantitative literacy distribution of U.S. adults, with and without missing cognitive data
N- 3
N.2 Performance in the lowest literacy level Nearly all of those who were assigned wrong answers to tasks that had been left blank scored below 225 on the three scales (Level 1, the lowest of the five literacy levels). Some of those in Level 1 appear there through their own efforts, while others were helped to be there by the method used for treating missing cognitive data. Table N-2 presents the percentages of adults who performed at least one task correctly and their average proficiencies on each scale. Most adults in Level 1 performed at least one literacy task correctly. On the prose literacy scale, 72 percent of adults in Level 1 performed at least one task correctly, as did 83 percent on the document scale and 66 percent on the quantitative scale. A small proportion of adults in Level 1 failed to perform correctly any literacy task. This group constituted 26 percent of adults on the prose literacy scale, 11 percent of adults on the document literacy scale, and 21 percent of adults on the quantitative literacy scale. Wide disparities in average proficiencies occurred between those who performed at least one task correctly and those who did not. The average proficiencies of these adults (94 on the document scale, 110 on the prose scale, and 113 on the quantitative scale) were considerably lower than those of adults who did succeed with at least one task, whose scores were 182 on the document scale and 190 on the prose and quantitative scales. Some adults in Level 1 (6 to 8 percent) did not respond to any of the literacy tasks for reasons unrelated to their literacy skills or for unknown reasons. These persons could not be described as either meeting or failing to meet the demands of literacy tasks, so they are distinguished as a separate group. Their proficiencies were inferred from the performance of other adults with similar demographic backgrounds and fell in the middle range between the other two groups. Table N-2. Percentage who performed at least one task correctly and average proficiency on each scale: Adults in Level 1, 1992 Performance on literacy tasks
Total in Level 1 At least one literacy task correct No literacy tasks Correct No literacy task data
Prose literacy Average Percent proficiency 100% 173 72% 190
Document literacy Average Percent proficiency 100% 172 83% 182
Quantitative literacy Average Percent proficiency 100% 167 66% 190
21%
113
11%
94
26%
110
7%
177
6%
177
8%
159
Nearly all adults who correctly responded to at least one literacy task also completed the assessment. Still, some adults broke off the assessment after already having shown some initial success. Table 8-8 divides adults in Level 1 into four groups by crossing two variables: those who did not with who were successful with at least one task and those who were not successful with any task, crossed with those who completed the assessment (at least five literacy tasks) and those who did not. N- 4
Table N-3. Percentage who performed at least one task or no tasks correctly and average proficiency on each scale by provision of sufficient cognitive data: Adults in Level 1, 1992 Completion of literacy tasks
Total in Level 1 with at least one literacy task correct
Prose literacy Average Percent proficiency 100% 190
Document literacy Average Percent proficiency 100% 182
Quantitative literacy Average Percent proficiency 100% 190
Provided sufficient cognitive data Did not provided sufficient cognitive data
87%
196
83%
192
90%
194
13%
153
17%
132
10%
153
Total in Level 1 with no literacy tasks correct
100%
113
100%
94
100%
110
Provided sufficient cognitive data Did not provided sufficient cognitive data
14%
148
2%
-
14%
146
86%
107
98%
93
86%
98
Across the scales, from 83 to 90 percent of those in Level 1 who correctly responded to at least one task also completed the assessment. Their average scores ranged from 192 to 196. The remainder (10 to 17 percent) performed at least one task correctly before breaking off the assessment. Their average scores were much lower, ranging from 132 to 153. The population of adults who scored in Level 1 on each scale includes not only those who demonstrated the skills needed to succeed with at least some of the tasks in Level 1—who constituted the majority—but also those who were unable to succeed with any of the tasks in this level. Nearly all of those in Level 1 who failed to succeed with at least one literacy task also failed to complete the assessment, as shown in the following table. Most of these adults (from 86 to 98 percent) either did not start or broke off the assessment for literacy-related reasons, so that any literacy tasks that remained unanswered were treated as wrong answers. Their scores averaged from 93 to 107 across the scales. Some respondents completed the assessment, yet failed to respond correctly to any literacy tasks (2 to 14 percent of those in Level 1). These individuals had scores that averaged 148 on the prose scale and 146 on the quantitative scale; too few cases were available to estimate an average score on the document scale. The pattern of Level 1 proficiencies associated with various combinations of missing and incorrect answers show the consequences of including, rather than leaving out adults who were unable to complete the assessment for literacy-related reasons. In general, the very low scores of these individuals bring down the average for any group in which they are a significant component. Omitting these individuals from the assessment would have resulted in inflated estimates of the literacy skills of the adult population overall and particularly of certain populations.
N- 5
Appendix O ESTIMATED COMPOSITE FACTORS FOR A SELECTED SET OF DEMOGRAPHIC VARIABLES FOR ILLINOIS, INDIANA, LOUISIANA, NEW JERSEY, NEW YORK, OHIO, PENNSYLVANIA, TEXAS, AND WASHINGTON
Appendix O: Estimated Composite Factors For a Selected Set of Demographic Variables for Illinois, Indiana, Louisiana, New Jersey, New York, Ohio, Pennsylvania, Texas, and Washington NATIONAL ADULT LITERACY SURVEY Table O-1. NALS compositing analysis: Compositing factors* for average prose proficiency and literacy levels in Illinois sample, by Total Population, Gender, Race/Ethnicity, Education Level, Age, and Country of Birth
Demographic Subpopulations
National Certainty PSU?
State Compositing Factor: Beta Level 1 225 or lower
Level 2 226 to 275
Level 3 276 to 325
Level 4 326 to 375
Level 5 376 or higher
Overall Proficiency
yes no
0.5464 0.5441
0.6411 0.5579
0.3993 0.3956
0.5710 0.5155
0.7771 0.7481
0.4620 0.4690
yes no
0.5261 0.5115
0.5355 0.5159
0.4338 0.4316
0.5785 0.5576
0.7692 0.7818
0.4419 0.4458
yes no
0.6939 0.6275
0.5584 0.3878
0.6607 0.6077
0.7406 0.6847
0.6359 0.5412
0.5915 0.4836
yes no
0.4681 0.5954
0.4501 0.5715
0.4996 0.6300
0.2987 0.4091
0.7064 0.7983
0.2533 0.3666
Hispanic
yes no
0.4572 0.1358
0.5878 0.1829
0.5665 0.1987
0.8150 0.4118
0.5676 0.1764
0.4273 0.1222
Other
yes no
0.7161 0.6445
0.6489 0.5015
0.4957 0.4087
0.5865 0.4650
0.8113 0.7299
0.5974 0.5223
yes no
0.6695 0.5657
0.7686 0.6972
0.7239 0.6725
0.5581 0.4672
0.6011 0.5066
0.4673 0.3837
HS degree
yes no
0.5728 0.5975
0.4787 0.4390
0.5841 0.5307
0.6271 0.6061
0.7432 0.7279
0.5029 0.5495
Some college
yes no
0.5436 0.5377
0.6278 0.6090
0.4127 0.4694
0.6293 0.5945
0.8168 0.7805
0.5884 0.5699
College graduate
yes no
0.4919 0.4320
0.6718 0.6593
0.6155 0.5531
0.4793 0.4265
0.6922 0.6665
0.6104 0.5648
yes no
0.5398 0.4249
0.5100 0.3597
0.6016 0.5184
0.8138 0.7235
0.5723 0.5088
0.5422 0.5064
25 to 44 years
yes no
0.4551 0.4627
0.6861 0.6339
0.4475 0.4448
0.5318 0.4912
0.7820 0.7700
0.4593 0.4340
45 to 64 years
yes no
0.5845 0.5641
0.6562 0.5676
0.6628 0.5574
0.6718 0.5960
0.6148 0.5273
0.5796 0.5444
yes no
0.3770 0.1234
0.5388 0.1997
0.6225 0.2817
0.8245 0.5057
0.4339 0.1519
0.3319 0.1078
yes no
0.6014 0.6227
0.7062 0.6476
0.4441 0.4457
0.5519 0.5087
0.7610 0.7354
0.5469 0.5047
Total Population Total Gender Male
Female Race/Ethnicity Black
Education Level No HS degree
Age 16 to 24 years
Country of Birth Not USA
USA
* Compositing factors reflect imputation variance as well as sampling variance.
O- 1
Table O-2. NALS compositing analysis: Compositing factors* for average document proficiency and literacy levels in Illinois sample, by Total Population, Gender, Race/Ethnicity, Education Level, Age, and Country of Birth
Demographic Subpopulations
National Certainty PSU?
State Compositing Factor: Beta Level 1 225 or lower
Level 2 226 to 275
Level 3 276 to 325
Level 4 326 to 375
Level 5 376 or higher
Overall Proficiency
yes no
0.4290 0.3930
0.6861 0.6775
0.6988 0.6683
0.8946 0.8411
0.8489 0.8175
0.5364 0.5031
yes no
0.5108 0.4477
0.5965 0.5748
0.6610 0.6226
0.7680 0.7210
0.6388 0.6251
0.5300 0.5036
yes no
0.4633 0.4358
0.7341 0.6702
0.5312 0.5041
0.7538 0.6844
0.7811 0.7068
0.5746 0.4570
yes no
0.3925 0.5125
0.5264 0.6423
0.5123 0.6454
0.4193 0.5375
0.3105 0.4184
0.2740 0.3890
Hispanic
yes no
0.4332 0.1177
0.5660 0.1792
0.5604 0.1786
0.6526 0.2362
0.7372 0.3070
0.3840 0.1072
Other
yes no
0.5904 0.5253
0.6874 0.6485
0.7313 0.6618
0.8563 0.7461
0.8586 0.7929
0.6751 0.5804
yes no
0.5791 0.4608
0.6635 0.5516
0.7074 0.6135
0.6076 0.5152
0.5059 0.4102
0.5137 0.4185
HS degree
yes no
0.4638 0.4687
0.5379 0.5422
0.6224 0.5991
0.6427 0.6314
0.6378 0.6538
0.5199 0.5602
Some college
yes no
0.4313 0.3863
0.7843 0.7491
0.8861 0.8399
0.7278 0.6776
0.7936 0.7000
0.5991 0.5407
College graduate
yes no
0.5485 0.4989
0.6465 0.5839
0.4289 0.4275
0.7870 0.7673
0.7431 0.7392
0.6471 0.6328
yes no
0.6070 0.4895
0.8713 0.7474
0.8184 0.7523
0.6641 0.5420
0.6938 0.5964
0.5346 0.4744
25 to 44 years
yes no
0.2719 0.3039
0.4667 0.5065
0.5446 0.5647
0.8482 0.8179
0.8142 0.8091
0.5297 0.4918
45 to 64 years
yes no
0.6949 0.6155
0.6495 0.6073
0.6570 0.6083
0.7280 0.6745
0.8006 0.7514
0.6816 0.6210
yes no
0.4573 0.1584
0.5083 0.1772
0.6151 0.2613
0.7494 0.4115
0.6692 0.3233
0.4120 0.1445
yes no
0.4000 0.3925
0.6094 0.6291
0.6749 0.6510
0.8565 0.7948
0.8401 0.8088
0.6156 0.5433
Total Population Total Gender Male
Female Race/Ethnicity Black
Education Level No HS degree
Age 16 to 24 years
Country of Birth Not USA
USA
* Compositing factors reflect imputation variance as well as sampling variance.
O- 2
Table O-3. NALS compositing analysis: Compositing factors* for average quantitative proficiency and literacy levels in Illinois sample, by Total Population, Gender, Race/Ethnicity, Education Level, Age, and Country of Birth
Demographic Subpopulations
National Certainty PSU?
State Compositing Factor: Beta Level 1 225 or lower
Level 2 226 to 275
Level 3 276 to 325
Level 4 326 to 375
Level 5 376 or higher
Overall Proficiency
yes no
0.3068 0.3402
0.5788 0.5334
0.6494 0.5642
0.7088 0.6810
0.7533 0.7282
0.4386 0.4256
yes no
0.4756 0.4563
0.4434 0.4009
0.6978 0.6575
0.6905 0.6783
0.6715 0.6880
0.4060 0.3696
yes no
0.4973 0.4623
0.8474 0.7830
0.7255 0.6506
0.7160 0.6620
0.6886 0.5725
0.5707 0.4676
yes no
0.4519 0.5867
0.4857 0.6074
0.4256 0.5510
0.4028 0.5248
0.6921 0.7862
0.2449 0.3645
Hispanic
yes no
0.3776 0.0999
0.6876 0.2711
0.5199 0.1564
0.6559 0.2465
0.1936 0.0315
0.3911 0.1083
Other
yes no
0.5894 0.5670
0.6199 0.5329
0.6598 0.5089
0.6981 0.6180
0.7824 0.6927
0.6761 0.5547
yes no
0.3703 0.3197
0.6443 0.4763
0.4686 0.3997
0.5407 0.4836
0.4986 0.3850
0.3966 0.3539
HS degree
yes no
0.4596 0.5175
0.5509 0.5921
0.5104 0.4646
0.5099 0.4932
0.8620 0.8059
0.4106 0.4728
Some college
yes no
0.5083 0.4923
0.4545 0.4340
0.7222 0.6725
0.8476 0.8190
0.7199 0.6872
0.5858 0.5129
College graduate
yes no
0.3866 0.3426
0.4967 0.4376
0.5502 0.4768
0.4051 0.4046
0.6392 0.5911
0.7269 0.6694
yes no
0.5433 0.4865
0.6593 0.5335
0.5262 0.4297
0.4458 0.3586
0.6118 0.5525
0.5689 0.5415
25 to 44 years
yes no
0.3086 0.3678
0.6166 0.6058
0.6452 0.6162
0.7891 0.7679
0.7788 0.7651
0.3889 0.3758
45 to 64 years
yes no
0.6311 0.5629
0.5077 0.4353
0.6454 0.5420
0.7771 0.7383
0.6874 0.5751
0.6445 0.5891
yes no
0.3971 0.1318
0.3981 0.1287
0.5244 0.1920
0.8076 0.4857
0.6418 0.2838
0.4070 0.1389
yes no
0.4226 0.4307
0.5962 0.5681
0.7033 0.6425
0.7182 0.7100
0.7680 0.7516
0.4995 0.4883
Total Population Total Gender Male
Female Race/Ethnicity Black
Education Level No HS degree
Age 16 to 24 years
Country of Birth Not USA
USA
* Compositing factors reflect imputation variance as well as sampling variance.
O- 3
Table O-4. NALS compositing analysis: Compositing factors* for average prose proficiency and literacy levels in Indiana sample, by Total Population, Gender, Race/Ethnicity, Education Level, Age, and Country of Birth
Demographic Subpopulations
National Certainty PSU?
State Compositing Factor: Beta Level 1 225 or lower
Level 2 226 to 275
Level 3 276 to 325
Level 4 326 to 375
Level 5 376 or higher
Overall Proficiency
yes no
– 0.6940
– 0.7012
– 0.5557
– 0.6744
– 0.8524
– 0.6322
yes no
– 0.6771
– 0.6950
– 0.6075
– 0.7229
– 0.8837
– 0.6137
yes no
– 0.7480
– 0.5117
– 0.7403
– 0.8028
– 0.6783
– 0.6137
yes no
– 0.4634
– 0.4109
– 0.5432
– 0.2536
– 0.6827
– 0.2934
Hispanic
yes no
– 0.5928
– 0.7109
– 0.6971
– 0.8840
– 0.6900
– 0.5570
Other
yes no
– 0.7876
– 0.6771
– 0.5855
– 0.6474
– 0.8506
– 0.6942
yes no
– 0.8130
– 0.8827
– 0.8768
– 0.7461
– 0.7772
– 0.6798
HS degree
yes no
– 0.7610
– 0.6123
– 0.6950
– 0.7583
– 0.8487
– 0.7261
Some college
yes no
– 0.6494
– 0.7067
– 0.5867
– 0.6958
– 0.8425
– 0.6724
College graduate
yes no
– 0.4990
– 0.7193
– 0.6183
– 0.4934
– 0.7245
– 0.6300
yes no
– 0.6082
– 0.5229
– 0.6907
– 0.8404
– 0.6834
– 0.6892
25 to 44 years
yes no
– 0.6231
– 0.7689
– 0.6088
– 0.6566
– 0.8674
– 0.5924
45 to 64 years
yes no
– 0.7053
– 0.6991
– 0.6924
– 0.7289
– 0.6663
– 0.6895
yes no
– 0.5734
– 0.7172
– 0.7872
– 0.9105
– 0.6328
– 0.5260
yes no
– 0.7443
– 0.7527
– 0.5851
– 0.6400
– 0.8265
– 0.6319
Total Population Total Gender Male
Female Race/Ethnicity Black
Education Level No HS degree
Age 16 to 24 years
Country of Birth Not USA
USA
* Compositing factors reflect imputation variance as well as sampling variance.
– Not available.
O- 4
Table O-5. NALS compositing analysis: Compositing factors* for average document proficiency and literacy levels in Indiana sample, by Total Population, Gender, Race/Ethnicity, Education Level, Age, and Country of Birth
Demographic Subpopulations
National Certainty PSU?
State Compositing Factor: Beta Level 1 225 or lower
Level 2 226 to 275
Level 3 276 to 325
Level 4 326 to 375
Level 5 376 or higher
Overall Proficiency
yes no
– 0.5384
– 0.8045
– 0.7939
– 0.9030
– 0.8962
– 0.6551
yes no
– 0.6115
– 0.7301
– 0.7609
– 0.8314
– 0.7750
– 0.6641
yes no
– 0.5815
– 0.7806
– 0.6580
– 0.7966
– 0.8127
– 0.5886
yes no
– 0.3814
– 0.4867
– 0.5539
– 0.3711
– 0.2455
– 0.3043
Hispanic
yes no
– 0.5589
– 0.6889
– 0.6809
– 0.7624
– 0.8258
– 0.5111
Other
yes no
– 0.6856
– 0.7942
– 0.8061
– 0.8502
– 0.8913
– 0.7329
yes no
– 0.7218
– 0.8048
– 0.8416
– 0.7714
– 0.7023
– 0.6980
HS degree
yes no
– 0.6467
– 0.7198
– 0.7649
– 0.7719
– 0.8126
– 0.7243
Some college
yes no
– 0.4928
– 0.8200
– 0.8853
– 0.7600
– 0.7729
– 0.6409
College graduate
yes no
– 0.5663
– 0.6479
– 0.4982
– 0.8130
– 0.7897
– 0.6955
yes no
– 0.6635
– 0.8484
– 0.8615
– 0.7028
– 0.7524
– 0.6625
25 to 44 years
yes no
– 0.4533
– 0.6724
– 0.7288
– 0.8937
– 0.8939
– 0.6397
45 to 64 years
yes no
– 0.7372
– 0.7445
– 0.7389
– 0.7873
– 0.8478
– 0.7473
yes no
– 0.6375
– 0.6824
– 0.7748
– 0.8708
– 0.8196
– 0.6076
yes no
– 0.5171
– 0.7528
– 0.7603
– 0.8586
– 0.8773
– 0.6598
Total Population Total Gender Male
Female Race/Ethnicity Black
Education Level No HS degree
Age 16 to 24 years
Country of Birth Not USA
USA
* Compositing factors reflect imputation variance as well as sampling variance.
– Not available.
O- 5
Table O-6. NALS compositing analysis: Compositing factors* for average quantitative proficiency and literacy levels in Indiana sample, by Total Population, Gender, Race/Ethnicity, Education Level, Age, and Country of Birth
Demographic Subpopulations
National Certainty PSU?
State Compositing Factor: Beta Level 1 225 or lower
Level 2 226 to 275
Level 3 276 to 325
Level 4 326 to 375
Level 5 376 or higher
Overall Proficiency
yes no
– 0.4853
– 0.6825
– 0.7090
– 0.8089
– 0.8413
– 0.5861
yes no
– 0.6147
– 0.5648
– 0.7963
– 0.8104
– 0.8235
– 0.5314
yes no
– 0.6066
– 0.8639
– 0.7726
– 0.7861
– 0.7000
– 0.6008
yes no
– 0.5121
– 0.4725
– 0.4001
– 0.3724
– 0.6666
– 0.3146
Hispanic
yes no
– 0.5091
– 0.7889
– 0.6530
– 0.7716
– 0.2896
– 0.5265
Other
yes no
– 0.7276
– 0.7042
– 0.6883
– 0.7777
– 0.8264
– 0.7135
yes no
– 0.6010
– 0.7368
– 0.6896
– 0.7803
– 0.6915
– 0.6490
HS degree
yes no
– 0.6929
– 0.7626
– 0.6312
– 0.6752
– 0.8803
– 0.6648
Some college
Yes No
– 0.6031
– 0.5432
– 0.7565
– 0.8733
– 0.7707
– 0.6130
College graduate
yes no
– 0.4063
– 0.5047
– 0.5421
– 0.4749
– 0.6548
– 0.7258
yes no
– 0.6646
– 0.6882
– 0.6066
– 0.5409
– 0.7175
– 0.7183
25 to 44 years
yes no
– 0.5318
– 0.7488
– 0.7531
– 0.8647
– 0.8637
– 0.5294
45 to 64 years
yes no
– 0.6949
– 0.5840
– 0.6794
– 0.8380
– 0.7092
– 0.7158
yes no
– 0.5884
– 0.5894
– 0.6987
– 0.9008
– 0.7892
– 0.6002
yes no
– 0.5606
– 0.6888
– 0.7513
– 0.8143
– 0.8413
– 0.6255
Total Population Total Gender Male
Female Race/Ethnicity Black
Education Level No HS degree
Age 16 to 24 years
Country of Birth Not USA
USA
* Compositing factors reflect imputation variance as well as sampling variance.
– Not available.
O- 6
Table O-7. NALS compositing analysis: Compositing factors* for average prose proficiency and literacy levels in Louisiana sample, by Total Population, Gender, Race/Ethnicity, Education Level, Age, and Country of Birth
Demographic Subpopulations
National Certainty PSU?
State Compositing Factor: Beta Level 1 225 or lower
Level 2 226 to 275
Level 3 276 to 325
Level 4 326 to 375
Level 5 376 or higher
Overall Proficiency
yes no
– 0.8509
– 0.8339
– 0.7579
– 0.8200
– 0.9297
– 0.8134
yes no
– 0.7799
– 0.7688
– 0.7237
– 0.8021
– 0.9274
– 0.7426
yes no
– 0.8961
– 0.7318
– 0.8945
– 0.9227
– 0.8556
– 0.8168
yes no
– 0.7650
– 0.6631
– 0.8451
– 0.4387
– 0.8711
– 0.6559
Hispanic
yes no
– 0.8623
– 0.9149
– 0.9110
– 0.9710
– 0.9037
– 0.8379
Other
yes no
– 0.9165
– 0.8611
– 0.8070
– 0.8442
– 0.9438
– 0.8702
yes no
– 0.8874
– 0.9369
– 0.9427
– 0.8533
– 0.8712
– 0.8122
HS degree
yes no
– 0.9006
– 0.8151
– 0.8633
– 0.8981
– 0.9401
– 0.8838
Some college
yes no
– 0.7933
– 0.8382
– 0.7839
– 0.8134
– 0.9167
– 0.8150
College graduate
yes no
– 0.7260
– 0.8731
– 0.8117
– 0.7200
– 0.8738
– 0.8185
yes no
– 0.7597
– 0.6908
– 0.8313
– 0.9159
– 0.8325
– 0.8430
25 to 44 years
yes no
– 0.8191
– 0.8895
– 0.8070
– 0.8257
– 0.9447
– 0.7906
45 to 64 years
yes no
– 0.8413
– 0.8154
– 0.7964
– 0.8336
– 0.7898
– 0.8241
yes no
– 0.9337
– 0.9637
– 0.9749
– 0.9906
– 0.9480
– 0.9204
yes no
– 0.8793
– 0.8642
– 0.7706
– 0.7902
– 0.9129
– 0.7908
Total Population Total Gender Male
Female Race/Ethnicity Black
Education Level No HS degree
Age 16 to 24 years
Country of Birth Not USA
USA
* Compositing factors reflect imputation variance as well as sampling variance.
– Not available.
O- 7
Table O-8. NALS compositing analysis: Compositing factors* for average document proficiency and literacy levels in Louisiana sample, by Total Population, Gender, Race/Ethnicity, Education Level, Age, and Country of Birth
Demographic Subpopulations
National Certainty PSU?
State Compositing Factor: Beta Level 1 225 or lower
Level 2 226 to 275
Level 3 276 to 325
Level 4 326 to 375
Level 5 376 or higher
Overall Proficiency
yes no
– 0.7403
– 0.9095
– 0.9003
– 0.9525
– 0.9503
– 0.8192
yes no
– 0.7141
– 0.8176
– 0.8439
– 0.8910
– 0.8457
– 0.7734
yes no
– 0.8099
– 0.9116
– 0.8566
– 0.9179
– 0.9241
– 0.7985
yes no
– 0.6798
– 0.7307
– 0.8648
– 0.6046
– 0.3454
– 0.6547
Hispanic
yes no
– 0.8362
– 0.9032
– 0.8975
– 0.9320
– 0.9522
– 0.8096
Other
yes no
– 0.8664
– 0.9192
– 0.9245
– 0.9443
– 0.9602
– 0.8908
yes no
– 0.8231
– 0.8782
– 0.9079
– 0.8683
– 0.8199
– 0.8175
HS degree
yes no
– 0.8385
– 0.8785
– 0.9002
– 0.9057
– 0.9241
– 0.8833
Some college
yes no
– 0.6478
– 0.9003
– 0.9366
– 0.8639
– 0.8544
– 0.7761
College graduate
yes no
– 0.7759
– 0.8323
– 0.7282
– 0.9205
– 0.9084
– 0.8604
yes no
– 0.8026
– 0.9160
– 0.9308
– 0.8312
– 0.8655
– 0.8171
25 to 44 years
yes no
– 0.7043
– 0.8554
– 0.8824
– 0.9542
– 0.9571
– 0.8208
45 to 64 years
yes no
– 0.8473
– 0.8539
– 0.8536
– 0.8838
– 0.9136
– 0.8554
yes no
– 0.9464
– 0.9562
– 0.9726
– 0.9862
– 0.9795
– 0.9412
yes no
– 0.7261
– 0.8823
– 0.8785
– 0.9273
– 0.9375
– 0.8041
Total Population Total Gender Male
Female Race/Ethnicity Black
Education Level No HS degree
Age 16 to 24 years
Country of Birth Not USA
USA
* Compositing factors reflect imputation variance as well as sampling variance.
– Not available.
O- 8
Table O-9. NALS compositing analysis: Compositing factors* for average quantitative proficiency and literacy levels in Louisiana sample, by Total Population, Gender, Race/Ethnicity, Education Level, Age, and Country of Birth
Demographic Subpopulations
National Certainty PSU?
State Compositing Factor: Beta Level 1 225 or lower
Level 2 226 to 275
Level 3 276 to 325
Level 4 326 to 375
Level 5 376 or higher
Overall Proficiency
yes no
– 0.7198
– 0.8330
– 0.8360
– 0.9068
– 0.9243
– 0.7767
yes no
– 0.7456
– 0.6845
– 0.8535
– 0.8761
– 0.8876
– 0.6587
yes no
– 0.8242
– 0.9464
– 0.9064
– 0.9157
– 0.8637
– 0.8091
yes no
– 0.8343
– 0.7555
– 0.6802
– 0.6427
– 0.8613
– 0.7152
Hispanic
yes no
– 0.8142
– 0.9402
– 0.8921
– 0.9391
– 0.6370
– 0.8273
Other
yes no
– 0.8876
– 0.8755
– 0.8669
– 0.9114
– 0.9335
– 0.8809
yes no
– 0.7752
– 0.8038
– 0.8286
– 0.9002
– 0.7974
– 0.8195
HS degree
yes no
– 0.8671
– 0.9017
– 0.8272
– 0.8531
– 0.9538
– 0.8511
Some college
yes no
– 0.7601
– 0.7163
– 0.8599
– 0.9316
– 0.8737
– 0.7484
College graduate
yes no
– 0.6456
– 0.7305
– 0.7537
– 0.7118
– 0.8348
– 0.8754
yes no
– 0.8223
– 0.8257
– 0.7700
– 0.7171
– 0.8557
– 0.8630
25 to 44 years
yes no
– 0.7722
– 0.8875
– 0.8868
– 0.9423
– 0.9429
– 0.7498
45 to 64 years
yes no
– 0.8234
– 0.7304
– 0.7880
– 0.9133
– 0.7983
– 0.8437
yes no
– 0.9364
– 0.9370
– 0.9594
– 0.9894
– 0.9742
– 0.9393
yes no
– 0.7626
– 0.8358
– 0.8579
– 0.9077
– 0.9214
– 0.7934
Total Population Total Gender Male
Female Race/Ethnicity Black
Education Level No HS degree
Age 16 to 24 years
Country of Birth Not USA
USA
* Compositing factors reflect imputation variance as well as sampling variance.
– Not available.
O- 9
Table O-10. NALS compositing analysis: Compositing factors* for average prose proficiency and literacy levels in New Jersey sample, by Total Population, Gender, Race/Ethnicity, Education Level, Age, and Country of Birth
Demographic Subpopulations
National Certainty PSU?
State Compositing Factor: Beta Level 1 225 or lower
Level 2 226 to 275
Level 3 276 to 325
Level 4 326 to 375
Level 5 376 or higher
Overall Proficiency
yes no
0.5931 0.6327
0.6837 0.6783
0.4458 0.4834
0.6169 0.6073
0.8084 0.8094
0.5096 0.5455
yes no
0.5628 0.5967
0.5720 0.5703
0.4705 0.5023
0.6141 0.6267
0.7944 0.8094
0.4786 0.5318
yes no
0.7388 0.7502
0.6121 0.5933
0.7084 0.7107
0.7808 0.7693
0.6855 0.6728
0.6437 0.6555
yes no
0.6267 0.5188
0.6096 0.4792
0.6557 0.6025
0.4483 0.3076
0.8210 0.7351
0.3928 0.3505
Hispanic
yes no
0.4471 0.5868
0.5778 0.6718
0.5565 0.6872
0.8088 0.8644
0.5576 0.6650
0.4174 0.5635
Other
yes no
0.6825 0.7382
0.6117 0.6464
0.4558 0.5219
0.5472 0.5887
0.7856 0.8145
0.5583 0.6210
yes no
0.6797 0.7318
0.7768 0.8249
0.7331 0.7898
0.5695 0.6370
0.6122 0.6724
0.4789 0.5483
HS degree
yes no
0.7533 0.6848
0.6765 0.5705
0.7618 0.6611
0.7929 0.7166
0.8682 0.8080
0.6973 0.6329
Some college
yes no
0.4308 0.4942
0.5174 0.6015
0.3087 0.4308
0.5190 0.5692
0.7391 0.7979
0.4760 0.5599
College graduate
yes no
0.5539 0.5685
0.7242 0.7457
0.6724 0.6862
0.5414 0.5458
0.7426 0.7432
0.6677 0.6753
yes no
0.5702 0.5544
0.5407 0.5418
0.6307 0.6311
0.8317 0.8297
0.6021 0.6087
0.5725 0.5780
25 to 44 years
yes no
0.5530 0.5777
0.7640 0.7408
0.5453 0.5547
0.6271 0.5959
0.8416 0.8383
0.5571 0.5612
45 to 64 years
yes no
0.5415 0.5813
0.6158 0.6234
0.6227 0.6106
0.6321 0.6306
0.5726 0.5763
0.5365 0.5612
yes no
0.4392 0.4867
0.6019 0.6274
0.6809 0.7247
0.8588 0.8743
0.4979 0.5441
0.3913 0.4503
yes no
0.6365 0.6676
0.7361 0.7311
0.4811 0.4989
0.5884 0.5777
0.7870 0.7866
0.5834 0.5875
Total Population Total Gender Male
Female Race/Ethnicity Black
Education Level No HS degree
Age 16 to 24 years
Country of Birth Not USA
USA
* Compositing factors reflect imputation variance as well as sampling variance.
O-10
Table O-11. NALS compositing analysis: Compositing factors* for average document proficiency and literacy levels in New Jersey sample, by Total Population, Gender, Race/Ethnicity, Education Level, Age, and Country of Birth
Demographic Subpopulations
National Certainty PSU?
State Compositing Factor: Beta Level 1 225 or lower
Level 2 226 to 275
Level 3 276 to 325
Level 4 326 to 375
Level 5 376 or higher
Overall Proficiency
yes no
0.4762 0.5256
0.7256 0.7413
0.7374 0.7482
0.9113 0.9145
0.8718 0.8692
0.5834 0.6086
yes no
0.5477 0.5747
0.6315 0.6599
0.6934 0.7284
0.7933 0.8180
0.6723 0.6877
0.5667 0.6097
yes no
0.5186 0.5600
0.7750 0.7870
0.5857 0.5985
0.7925 0.7854
0.8166 0.8052
0.6276 0.6311
yes no
0.5521 0.4607
0.6795 0.5635
0.6670 0.5915
0.5793 0.4458
0.4620 0.3073
0.4186 0.3662
Hispanic
yes no
0.4232 0.5575
0.5560 0.6687
0.5503 0.6707
0.6433 0.7399
0.7292 0.8051
0.3744 0.5280
Other
yes no
0.5512 0.6371
0.6521 0.7167
0.6987 0.7426
0.8355 0.8655
0.8380 0.8587
0.6391 0.7032
yes no
0.5904 0.6656
0.6738 0.7230
0.7169 0.7661
0.6186 0.6916
0.5175 0.5824
0.5254 0.6016
HS degree
yes no
0.6633 0.5795
0.7261 0.6348
0.7896 0.6917
0.8038 0.7463
0.8004 0.7189
0.7114 0.6614
Some college
yes no
0.3252 0.3727
0.6979 0.7515
0.8317 0.8673
0.6295 0.6960
0.7096 0.7472
0.4870 0.5510
College graduate
yes no
0.6091 0.6191
0.7011 0.7288
0.4907 0.5255
0.8258 0.8345
0.7877 0.7912
0.7016 0.7305
yes no
0.6359 0.6274
0.8845 0.8844
0.8360 0.8360
0.6910 0.6905
0.7193 0.7154
0.5650 0.5583
25 to 44 years
yes no
0.3561 0.4117
0.5644 0.5951
0.6391 0.6367
0.8922 0.8872
0.8665 0.8630
0.6252 0.6385
45 to 64 years
yes no
0.6566 0.6744
0.6087 0.6170
0.6165 0.6378
0.6920 0.7140
0.7712 0.7714
0.6425 0.6580
yes no
0.5216 0.5721
0.5723 0.6013
0.6741 0.7084
0.7946 0.8229
0.7236 0.7618
0.4756 0.5361
yes no
0.4362 0.4882
0.6442 0.6635
0.7067 0.7169
0.8738 0.8771
0.8591 0.8543
0.6502 0.6508
Total Population Total Gender Male
Female Race/Ethnicity Black
Education Level No HS degree
Age 16 to 24 years
Country of Birth Not USA
USA
* Compositing factors reflect imputation variance as well as sampling variance.
O-11
Table O-12. NALS compositing analysis: Compositing factors* for average quantitative proficiency and literacy levels in New Jersey sample, by Total Population, Gender, Race/Ethnicity, Education Level, Age, and Country of Birth
Demographic Subpopulations
National Certainty PSU?
State Compositing Factor: Beta Level 1 225 or lower
Level 2 226 to 275
Level 3 276 to 325
Level 4 326 to 375
Level 5 376 or higher
Overall Proficiency
yes no
0.3488 0.4410
0.6244 0.6392
0.6915 0.6784
0.7465 0.7443
0.7870 0.7859
0.4860 0.5164
yes no
0.5126 0.5757
0.4802 0.5184
0.7281 0.7349
0.7212 0.7434
0.7033 0.7239
0.4421 0.4844
yes no
0.5525 0.5891
0.8739 0.8684
0.7673 0.7543
0.7588 0.7496
0.7339 0.7177
0.6238 0.6330
yes no
0.6112 0.5733
0.6430 0.5460
0.5856 0.4558
0.5627 0.4367
0.8108 0.7244
0.3821 0.3580
Hispanic
yes no
0.3680 0.5036
0.6787 0.7746
0.5098 0.6257
0.6467 0.7432
0.1873 0.2348
0.3814 0.5228
Other
yes no
0.5502 0.6398
0.5815 0.6368
0.6230 0.6495
0.6634 0.7036
0.7540 0.7854
0.6401 0.6934
yes no
0.3812 0.4759
0.6549 0.7033
0.4802 0.5614
0.5522 0.6140
0.5103 0.5563
0.4078 0.4982
HS degree
yes no
0.6594 0.6115
0.7363 0.6680
0.7036 0.6058
0.7031 0.5978
0.9343 0.9066
0.6133 0.5504
Some college
yes no
0.3964 0.4663
0.3461 0.4223
0.6229 0.6856
0.7795 0.8222
0.6202 0.6894
0.4733 0.5315
College graduate
yes no
0.4470 0.4624
0.5587 0.5733
0.6108 0.5918
0.4662 0.5197
0.6944 0.7083
0.7734 0.7821
yes no
0.5736 0.5818
0.6864 0.6984
0.5568 0.5577
0.4764 0.4684
0.6406 0.6526
0.5988 0.6079
25 to 44 years
yes no
0.3980 0.4637
0.7043 0.7069
0.7292 0.7277
0.8471 0.8414
0.8391 0.8361
0.4852 0.5082
45 to 64 years
yes no
0.5895 0.6159
0.4640 0.4704
0.6044 0.5937
0.7454 0.7589
0.6487 0.6243
0.6034 0.6436
yes no
0.4601 0.5109
0.4612 0.5021
0.5879 0.6239
0.8445 0.8663
0.6986 0.7371
0.4704 0.5253
yes no
0.4593 0.5127
0.6315 0.6485
0.7334 0.7171
0.7473 0.7423
0.7934 0.7902
0.5367 0.5437
Total Population Total Gender Male
Female Race/Ethnicity Black
Education Level No HS degree
Age 16 to 24 years
Country of Birth Not USA
USA
* Compositing factors reflect imputation variance as well as sampling variance.
O-12
Table O-13. NALS compositing analysis: Compositing factors* for average prose proficiency and literacy levels in New York sample, by Total Population, Gender, Race/Ethnicity, Education Level, Age, and Country of Birth
Demographic Subpopulations
National Certainty PSU?
State Compositing Factor: Beta Level 1 225 or lower
Level 2 226 to 275
Level 3 276 to 325
Level 4 326 to 375
Level 5 376 or higher
Overall Proficiency
yes no
0.3839 0.5603
0.4802 0.5563
0.2559 0.4107
0.4077 0.5158
0.6433 0.7517
0.3075 0.4855
yes no
0.4079 0.5318
0.4171 0.5305
0.3223 0.4524
0.4600 0.5741
0.6741 0.7965
0.3295 0.4698
yes no
0.5032 0.6298
0.3610 0.3728
0.4653 0.6088
0.5605 0.6799
0.4383 0.5333
0.3928 0.4826
yes no
0.3206 0.1762
0.3051 0.1465
0.3487 0.2447
0.1860 0.0739
0.5633 0.3459
0.1539 0.1041
Hispanic
yes no
0.2520 0.5215
0.3632 0.6463
0.3433 0.6348
0.6379 0.8501
0.3443 0.6218
0.2299 0.4824
Other
yes no
0.6385 0.6775
0.5641 0.5290
0.4077 0.4442
0.4983 0.4958
0.7507 0.7543
0.5096 0.5594
yes no
0.4431 0.6748
0.5662 0.7839
0.5074 0.7652
0.3316 0.5817
0.3719 0.6208
0.2563 0.4977
HS degree
yes no
0.4285 0.6589
0.3393 0.4917
0.4399 0.5804
0.4847 0.6585
0.6181 0.7710
0.3614 0.6170
Some college
yes no
0.3471 0.4613
0.4296 0.5478
0.2387 0.4321
0.4311 0.5088
0.6656 0.7326
0.3896 0.5067
College graduate
yes no
0.3993 0.3747
0.5844 0.6127
0.5236 0.4940
0.3873 0.3642
0.6070 0.6053
0.5183 0.5027
yes no
0.3136 0.4211
0.2884 0.3619
0.3704 0.5186
0.6299 0.7238
0.3426 0.5102
0.3156 0.5065
25 to 44 years
yes no
0.3403 0.4647
0.5743 0.6145
0.3333 0.4420
0.4122 0.4717
0.6889 0.7639
0.3440 0.4277
45 to 64 years
yes no
0.3624 0.6028
0.4354 0.5898
0.4427 0.5736
0.4527 0.6187
0.3920 0.5501
0.3578 0.5798
yes no
0.2299 0.5602
0.3656 0.7035
0.4486 0.7791
0.6986 0.9047
0.2744 0.6220
0.1968 0.5132
yes no
0.4651 0.6110
0.5807 0.6138
0.3153 0.4259
0.4152 0.4719
0.6472 0.7107
0.4102 0.4753
Total Population Total Gender Male
Female Race/Ethnicity Black
Education Level No HS degree
Age 16 to 24 years
Country of Birth Not USA
USA
* Compositing factors reflect imputation variance as well as sampling variance.
O-13
Table O-14. NALS compositing analysis: Compositing factors* for average document proficiency and literacy levels in New York sample, by Total Population, Gender, Race/Ethnicity, Education Level, Age, and Country of Birth
Demographic Subpopulations
National Certainty PSU?
State Compositing Factor: Beta Level 1 225 or lower
Level 2 226 to 275
Level 3 276 to 325
Level 4 326 to 375
Level 5 376 or higher
Overall Proficiency
yes no
0.2798 0.4068
0.5306 0.6879
0.5454 0.6754
0.8145 0.8410
0.7440 0.8183
0.3744 0.5137
yes no
0.3933 0.4630
0.4785 0.5935
0.5476 0.6405
0.6727 0.7347
0.5233 0.6419
0.4117 0.5242
yes no
0.2783 0.4483
0.5522 0.6728
0.3361 0.5111
0.5776 0.6796
0.6146 0.6990
0.3764 0.4528
yes no
0.2574 0.1442
0.3734 0.1949
0.3603 0.2359
0.2791 0.1287
0.1945 0.0704
0.1683 0.1089
Hispanic
yes no
0.2341 0.4820
0.3429 0.6210
0.3377 0.6110
0.4290 0.7041
0.5287 0.7773
0.1996 0.4366
Other
yes no
0.5023 0.5645
0.6063 0.6851
0.6559 0.6933
0.8067 0.7671
0.8096 0.8127
0.5927 0.6146
yes no
0.3508 0.5701
0.4365 0.6628
0.4871 0.7164
0.3782 0.6242
0.2869 0.5257
0.2933 0.5303
HS degree
yes no
0.3261 0.5300
0.3943 0.6036
0.4797 0.6530
0.5015 0.6828
0.4962 0.7119
0.3771 0.6241
Some college
yes no
0.2529 0.3086
0.6187 0.6886
0.7764 0.7959
0.5441 0.6132
0.6319 0.6184
0.4001 0.4633
College graduate
yes no
0.4548 0.4377
0.5567 0.5348
0.3403 0.3832
0.7174 0.7237
0.6652 0.6888
0.5574 0.5886
yes no
0.3756 0.4879
0.7250 0.7497
0.6371 0.7527
0.4351 0.5432
0.4688 0.5958
0.3091 0.4717
25 to 44 years
yes no
0.1873 0.3139
0.3507 0.5133
0.4247 0.5614
0.7752 0.8094
0.7301 0.8041
0.4102 0.4870
45 to 64 years
yes no
0.4792 0.6391
0.4282 0.6380
0.4363 0.6385
0.5196 0.7004
0.6186 0.7705
0.4638 0.6477
yes no
0.2936 0.6167
0.3378 0.6623
0.4408 0.7621
0.5960 0.8658
0.4995 0.8125
0.2569 0.5937
yes no
0.2775 0.3840
0.4734 0.6130
0.5447 0.6284
0.7747 0.7731
0.7517 0.7849
0.4800 0.5101
Total Population Total Gender Male
Female Race/Ethnicity Black
Education Level No HS degree
Age 16 to 24 years
Country of Birth Not USA
USA
* Compositing factors reflect imputation variance as well as sampling variance.
O-14
Table O-15. NALS compositing analysis: Compositing factors* for average quantitative proficiency and literacy levels in New York sample, by Total Population, Gender, Race/Ethnicity, Education Level, Age, and Country of Birth
Demographic Subpopulations
National Certainty PSU?
State Compositing Factor: Beta Level 1 225 or lower
Level 2 226 to 275
Level 3 276 to 325
Level 4 326 to 375
Level 5 376 or higher
Overall Proficiency
yes no
0.1863 0.3649
0.4154 0.5407
0.4893 0.5598
0.5572 0.6850
0.6123 0.7321
0.2878 0.4388
yes no
0.3602 0.4796
0.3309 0.4187
0.5890 0.6699
0.5807 0.6952
0.5592 0.7069
0.2979 0.3878
yes no
0.3066 0.4734
0.7127 0.7774
0.5414 0.6433
0.5298 0.6587
0.4969 0.5602
0.3726 0.4662
yes no
0.3066 0.2267
0.3362 0.1912
0.2844 0.1375
0.2657 0.1280
0.5465 0.3338
0.1481 0.1118
Hispanic
yes no
0.1953 0.4358
0.4682 0.7344
0.3023 0.5846
0.4327 0.7181
0.0876 0.2303
0.2044 0.4547
Other
yes no
0.5013 0.6094
0.5332 0.5682
0.5760 0.5348
0.6183 0.6503
0.7158 0.7190
0.5937 0.5862
yes no
0.1876 0.4217
0.4157 0.5893
0.2573 0.5119
0.3162 0.6032
0.2810 0.5048
0.2052 0.4623
HS degree
yes no
0.3223 0.5862
0.4069 0.6574
0.3683 0.5161
0.3678 0.5516
0.7774 0.8307
0.2804 0.5454
Some college
yes no
0.3157 0.4213
0.2711 0.3708
0.5371 0.6045
0.7129 0.7701
0.5343 0.6253
0.3870 0.4314
College graduate
yes no
0.3021 0.2919
0.4040 0.3801
0.4566 0.3960
0.3186 0.3690
0.5489 0.5339
0.6464 0.6130
yes no
0.3166 0.4880
0.4298 0.5386
0.3019 0.4305
0.2386 0.3566
0.3803 0.5554
0.3394 0.5424
25 to 44 years
yes no
0.2161 0.3825
0.4982 0.6007
0.5289 0.6080
0.6979 0.7598
0.6849 0.7589
0.2821 0.3753
45 to 64 years
yes no
0.4087 0.5912
0.2941 0.4616
0.4238 0.5592
0.5849 0.7620
0.4705 0.5867
0.4228 0.6201
yes no
0.2452 0.5720
0.2460 0.5730
0.3523 0.6805
0.6743 0.8945
0.4692 0.7742
0.2530 0.5850
yes no
0.2966 0.4234
0.4597 0.5461
0.5773 0.6011
0.5949 0.6831
0.6561 0.7269
0.3651 0.4628
Total Population Total Gender Male
Female Race/Ethnicity Black
Education Level No HS degree
Age 16 to 24 years
Country of Birth Not USA
USA
* Compositing factors reflect imputation variance as well as sampling variance.
O-15
Table O-16. NALS compositing analysis: Compositing factors* for average prose proficiency and literacy levels in Ohio sample, by Total Population, Gender, Race/Ethnicity, Education Level, Age, and Country of Birth
Demographic Subpopulations
National Certainty PSU?
State Compositing Factor: Beta Level 1 225 or lower
Level 2 226 to 275
Level 3 276 to 325
Level 4 326 to 375
Level 5 376 or higher
Overall Proficiency
yes no
– 0.5993
– 0.5867
– 0.4530
– 0.5725
– 0.7909
– 0.5396
yes no
– 0.5716
– 0.6046
– 0.5032
– 0.6289
– 0.8411
– 0.5043
yes no
– 0.6469
– 0.3681
– 0.6498
– 0.7287
– 0.5652
– 0.4827
yes no
– 0.3799
– 0.3208
– 0.4602
– 0.1859
– 0.5974
– 0.2283
Hispanic
yes no
– 0.7310
– 0.8264
– 0.8146
– 0.9366
– 0.8068
– 0.6941
Other
yes no
– 0.7028
– 0.5585
– 0.4724
– 0.5353
– 0.7821
– 0.5960
yes no
– 0.7032
– 0.8062
– 0.8064
– 0.6209
– 0.6603
– 0.5451
HS degree
yes no
– 0.6486
– 0.4559
– 0.5433
– 0.6298
– 0.7547
– 0.6120
Some college
yes no
– 0.5886
– 0.6268
– 0.5295
– 0.6208
– 0.7739
– 0.5899
College graduate
yes no
– 0.4911
– 0.7260
– 0.6084
– 0.4972
– 0.7414
– 0.6337
yes no
– 0.5336
– 0.4193
– 0.6264
– 0.7850
– 0.6255
– 0.6529
25 to 44 years
yes no
– 0.5086
– 0.6631
– 0.4940
– 0.5408
– 0.8029
– 0.4673
45 to 64 years
yes no
– 0.6216
– 0.5902
– 0.5813
– 0.6343
– 0.5566
– 0.6030
yes no
– 0.4291
– 0.5869
– 0.6740
– 0.8507
– 0.4905
– 0.3827
yes no
– 0.6665
– 0.6499
– 0.4883
– 0.5339
– 0.7547
– 0.5194
Total Population Total Gender Male
Female Race/Ethnicity Black
Education Level No HS degree
Age 16 to 24 years
Country of Birth Not USA
USA
* Compositing factors reflect imputation variance as well as sampling variance.
– Not available.
O-16
Table O-17. NALS compositing analysis: Compositing factors* for average document proficiency and literacy levels in Ohio sample, by Total Population, Gender, Race/Ethnicity, Education Level, Age, and Country of Birth
Demographic Subpopulations
National Certainty PSU?
State Compositing Factor: Beta Level 1 225 or lower
Level 2 226 to 275
Level 3 276 to 325
Level 4 326 to 375
Level 5 376 or higher
Overall Proficiency
yes no
– 0.4168
– 0.7361
– 0.7135
– 0.8415
– 0.8468
– 0.5471
yes no
– 0.4841
– 0.6308
– 0.6579
– 0.7444
– 0.6925
– 0.5494
yes no
– 0.4725
– 0.6855
– 0.5683
– 0.7107
– 0.7292
– 0.4555
yes no
– 0.2940
– 0.3881
– 0.4824
– 0.2815
– 0.1741
– 0.2354
Hispanic
yes no
– 0.6947
– 0.8054
– 0.7958
– 0.8586
– 0.8991
– 0.6505
Other
yes no
– 0.5784
– 0.7231
– 0.7323
– 0.7636
– 0.8396
– 0.6271
yes no
– 0.5776
– 0.6904
– 0.7446
– 0.6483
– 0.5674
– 0.5584
HS degree
yes no
– 0.5063
– 0.5948
– 0.6432
– 0.6472
– 0.7217
– 0.6020
Some college
yes no
– 0.4017
– 0.7523
– 0.8220
– 0.6703
– 0.6641
– 0.5381
College graduate
yes no
– 0.5672
– 0.6263
– 0.5084
– 0.8187
– 0.8121
– 0.6963
yes no
– 0.5853
– 0.7761
– 0.8192
– 0.6194
– 0.6880
– 0.6193
25 to 44 years
yes no
– 0.3447
– 0.5781
– 0.6475
– 0.8308
– 0.8429
– 0.5094
45 to 64 years
yes no
– 0.6336
– 0.6687
– 0.6512
– 0.7029
– 0.7859
– 0.6561
yes no
– 0.4967
– 0.5468
– 0.6584
– 0.7902
– 0.7174
– 0.4640
yes no
– 0.4010
– 0.6827
– 0.6703
– 0.7751
– 0.8178
– 0.5368
Total Population Total Gender Male
Female Race/Ethnicity Black
Education Level No HS degree
Age 16 to 24 years
Country of Birth Not USA
USA
* Compositing factors reflect imputation variance as well as sampling variance.
– Not available.
O-17
Table O-18. NALS compositing analysis: Compositing factors* for average quantitative proficiency and literacy levels in Ohio sample, by Total Population, Gender, Race/Ethnicity, Education Level, Age, and Country of Birth
Demographic Subpopulations
National Certainty PSU?
State Compositing Factor: Beta Level 1 225 or lower
Level 2 226 to 275
Level 3 276 to 325
Level 4 326 to 375
Level 5 376 or higher
Overall Proficiency
yes no
– 0.3800
– 0.5756
– 0.5973
– 0.7403
– 0.7807
– 0.4824
yes no
– 0.4933
– 0.4401
– 0.7096
– 0.7333
– 0.7645
– 0.4079
yes no
– 0.4968
– 0.7931
– 0.6827
– 0.7089
– 0.5803
– 0.4718
yes no
– 0.4305
– 0.3797
– 0.3178
– 0.2886
– 0.5774
– 0.2547
Hispanic
yes no
– 0.6571
– 0.8745
– 0.7832
– 0.8695
– 0.4429
– 0.6748
Other
yes no
– 0.6430
– 0.6052
– 0.5732
– 0.6988
– 0.7502
– 0.5996
yes no
– 0.4622
– 0.5853
– 0.5587
– 0.6861
– 0.5539
– 0.5216
HS degree
yes no
– 0.5710
– 0.6579
– 0.4732
– 0.5356
– 0.7842
– 0.5472
Some college
yes no
– 0.5256
– 0.4552
– 0.6682
– 0.8204
– 0.6934
– 0.5020
College graduate
yes no
– 0.4047
– 0.4973
– 0.5536
– 0.4706
– 0.6510
– 0.7177
yes no
– 0.6085
– 0.5946
– 0.5291
– 0.4729
– 0.6595
– 0.6859
25 to 44 years
yes no
– 0.4297
– 0.6496
– 0.6472
– 0.7956
– 0.7968
– 0.4041
45 to 64 years
yes no
– 0.5890
– 0.4767
– 0.5677
– 0.7747
– 0.6004
– 0.6135
yes no
– 0.4445
– 0.4456
– 0.5653
– 0.8357
– 0.6775
– 0.4565
yes no
– 0.4503
– 0.5831
– 0.6516
– 0.7533
– 0.7803
– 0.5285
Total Population Total Gender Male
Female Race/Ethnicity Black
Education Level No HS degree
Age 16 to 24 years
Country of Birth Not USA
USA
* Compositing factors reflect imputation variance as well as sampling variance.
– Not available.
O-18
Table O-19. NALS compositing analysis: Compositing factors* for average prose proficiency and literacy levels in Pennsylvania sample, by Total Population, Gender, Race/Ethnicity, Education Level, Age, and Country of Birth
Demographic Subpopulations
National Certainty PSU?
State Compositing Factor: Beta Level 1 225 or lower
Level 2 226 to 275
Level 3 276 to 325
Level 4 326 to 375
Level 5 376 or higher
Overall Proficiency
yes no
0.5376 0.6075
0.6328 0.6120
0.3908 0.4605
0.5622 0.5842
0.7709 0.7970
0.4532 0.5408
yes no
0.5142 0.5795
0.5236 0.6010
0.4222 0.5069
0.5668 0.6315
0.7606 0.8375
0.4301 0.5132
yes no
0.6860 0.6708
0.5493 0.4207
0.6524 0.6619
0.7334 0.7371
0.6273 0.5927
0.5825 0.5222
yes no
0.4661 0.5759
0.4481 0.5464
0.4976 0.6143
0.2970 0.3843
0.7047 0.7828
0.2517 0.3512
Hispanic
yes no
0.0939 0.8437
0.1493 0.9025
0.1386 0.8946
0.3516 0.9663
0.1391 0.8934
0.0841 0.8241
Other
yes no
0.6926 0.6649
0.6228 0.5087
0.4675 0.4292
0.5589 0.4815
0.7934 0.7438
0.5699 0.5486
yes no
0.5977 0.7040
0.7090 0.8049
0.6579 0.7924
0.4808 0.6154
0.5250 0.6545
0.3914 0.5348
HS degree
yes no
0.5925 0.6572
0.4989 0.4791
0.6036 0.5677
0.6458 0.6487
0.7583 0.7667
0.5231 0.6176
Some college
yes no
0.5293 0.5924
0.6143 0.6485
0.3988 0.5223
0.6158 0.6407
0.8080 0.8016
0.5744 0.6115
College graduate
yes no
0.4753 0.4716
0.6571 0.7028
0.5997 0.5909
0.4628 0.4715
0.6779 0.7132
0.5946 0.6093
yes no
0.5359 0.4326
0.5061 0.3611
0.5979 0.5257
0.8114 0.7269
0.5684 0.5168
0.5383 0.5177
25 to 44 years
yes no
0.4875 0.5382
0.7133 0.7018
0.4797 0.5238
0.5639 0.5760
0.8033 0.8224
0.4916 0.5059
45 to 64 years
yes no
0.4828 0.6565
0.5588 0.6391
0.5661 0.6322
0.5759 0.6771
0.5143 0.6054
0.4778 0.6396
yes no
0.2677 0.1820
0.4136 0.2938
0.4989 0.3796
0.7394 0.6271
0.3164 0.2206
0.2308 0.1565
yes no
0.5939 0.6739
0.6997 0.6800
0.4365 0.5000
0.5442 0.5562
0.7553 0.7705
0.5391 0.5463
Total Population Total Gender Male
Female Race/Ethnicity Black
Education Level No HS degree
Age 16 to 24 years
Country of Birth Not USA
USA
* Compositing factors reflect imputation variance as well as sampling variance.
O-19
Table O-20. NALS compositing analysis: Compositing factors* for average document proficiency and literacy levels in Pennsylvania sample, by Total Population, Gender, Race/Ethnicity, Education Level, Age, and Country of Birth
Demographic Subpopulations
National Certainty PSU?
State Compositing Factor: Beta Level 1 225 or lower
Level 2 226 to 275
Level 3 276 to 325
Level 4 326 to 375
Level 5 376 or higher
Overall Proficiency
yes no
0.4203 0.4407
0.6784 0.7379
0.6913 0.7235
0.8912 0.8612
0.8443 0.8540
0.5276 0.5630
yes no
0.4989 0.5010
0.5849 0.6390
0.6503 0.6725
0.7594 0.7589
0.6278 0.6945
0.5181 0.5628
yes no
0.4541 0.4868
0.7268 0.7094
0.5220 0.5683
0.7468 0.7297
0.7747 0.7497
0.5655 0.4963
yes no
0.3906 0.4868
0.5243 0.6171
0.5102 0.6334
0.4173 0.5098
0.3087 0.3901
0.2724 0.3712
Hispanic
yes no
0.0860 0.8266
0.1383 0.8927
0.1356 0.8892
0.1878 0.9234
0.2566 0.9471
0.0713 0.7954
Other
yes no
0.5628 0.5461
0.6627 0.6804
0.7085 0.6866
0.8418 0.7423
0.8443 0.8059
0.6498 0.5939
yes no
0.5022 0.5925
0.5912 0.6929
0.6394 0.7437
0.5317 0.6508
0.4288 0.5617
0.4366 0.5601
HS degree
yes no
0.4840 0.5223
0.5580 0.6036
0.6412 0.6533
0.6611 0.6688
0.6563 0.7201
0.5400 0.6161
Some college
yes no
0.4173 0.4295
0.7744 0.7771
0.8801 0.8527
0.7163 0.7060
0.7840 0.7194
0.5851 0.5768
College graduate
yes no
0.5321 0.5433
0.6312 0.6156
0.4128 0.4788
0.7757 0.8008
0.7303 0.7842
0.6318 0.6752
yes no
0.6032 0.4956
0.8695 0.7462
0.8161 0.7569
0.6606 0.5457
0.6904 0.6025
0.5306 0.4858
25 to 44 years
yes no
0.2983 0.3682
0.4991 0.5928
0.5765 0.6576
0.8641 0.8555
0.8330 0.8567
0.5619 0.5545
45 to 64 years
yes no
0.6018 0.6801
0.5515 0.7016
0.5596 0.6896
0.6398 0.7407
0.7271 0.8144
0.5869 0.6967
yes no
0.3372 0.2314
0.3844 0.2654
0.4911 0.3647
0.6436 0.5250
0.5499 0.4283
0.2973 0.2065
yes no
0.3926 0.4277
0.6020 0.6850
0.6680 0.6904
0.8526 0.8069
0.8359 0.8338
0.6082 0.5741
Total Population Total Gender Male
Female Race/Ethnicity Black
Education Level No HS degree
Age 16 to 24 years
Country of Birth Not USA
USA
* Compositing factors reflect imputation variance as well as sampling variance.
O-20
Table O-21. NALS compositing analysis: Compositing factors* for average quantitative proficiency and literacy levels in Pennsylvania sample, by Total Population, Gender, Race/Ethnicity, Education Level, Age, and Country of Birth
Demographic Subpopulations
National Certainty PSU?
State Compositing Factor: Beta Level 1 225 or lower
Level 2 226 to 275
Level 3 276 to 325
Level 4 326 to 375
Level 5 376 or higher
Overall Proficiency
yes no
0.2993 0.3916
0.5701 0.5928
0.6413 0.6209
0.7014 0.7428
0.7467 0.7834
0.4299 0.4912
yes no
0.4637 0.5102
0.4317 0.4556
0.6876 0.7162
0.6802 0.7379
0.6609 0.7599
0.3946 0.4231
yes no
0.4881 0.5130
0.8425 0.8141
0.7180 0.7010
0.7085 0.7166
0.6806 0.6174
0.5616 0.5087
yes no
0.4498 0.5721
0.4837 0.5840
0.4236 0.5292
0.4009 0.5005
0.6903 0.7693
0.2434 0.3554
Hispanic
yes no
0.0695 0.7943
0.2131 0.9335
0.1176 0.8754
0.1901 0.9262
0.0287 0.6088
0.0733 0.8046
Other
yes no
0.5618 0.6024
0.5929 0.5565
0.6340 0.5170
0.6738 0.6450
0.7626 0.7079
0.6508 0.5631
yes no
0.3013 0.4518
0.5705 0.6120
0.3927 0.5467
0.4633 0.6499
0.4218 0.5467
0.3253 0.4989
HS degree
yes no
0.4797 0.5813
0.5708 0.6607
0.5306 0.4997
0.5300 0.5488
0.8713 0.8109
0.4303 0.5490
Some College
yes no
0.4939 0.5410
0.4402 0.4774
0.7105 0.7027
0.8400 0.8405
0.7081 0.7197
0.5718 0.5469
College graduate
yes no
0.3710 0.3829
0.4802 0.4775
0.5338 0.5240
0.3892 0.4491
0.6238 0.6312
0.7136 0.7026
yes no
0.5394 0.4949
0.6558 0.5345
0.5223 0.4357
0.4420 0.3664
0.6081 0.5594
0.5650 0.5528
25 to 44 years
yes no
0.3370 0.4449
0.6467 0.6782
0.6743 0.6825
0.8099 0.8187
0.8003 0.8176
0.4202 0.4414
45 to 64 years
yes no
0.5317 0.6354
0.4063 0.5229
0.5471 0.6185
0.6982 0.8032
0.5934 0.6513
0.5460 0.6579
yes no
0.2846 0.1936
0.2855 0.1927
0.3997 0.2808
0.7171 0.6027
0.5197 0.3897
0.2930 0.2014
yes no
0.4150 0.4722
0.5887 0.6080
0.6967 0.6791
0.7119 0.7579
0.7624 0.7899
0.4917 0.5420
Total Population Total Gender Male
Female Race/Ethnicity Black
Education Level No HS degree
Age 16 to 24 years
Country of Birth Not USA
USA
* Compositing factors reflect imputation variance as well as sampling variance.
O-21
Table O-22. NALS compositing analysis: Compositing factors* for average prose proficiency and literacy levels in Texas sample, by Total Population, Gender, Race/Ethnicity, Education Level, Age, and Country of Birth
Demographic Subpopulations
National Certainty PSU?
State Compositing Factor: Beta Level 1 225 or lower
Level 2 226 to 275
Level 3 276 to 325
Level 4 326 to 375
Level 5 376 or higher
Overall Proficiency
yes no
0.4401 0.5373
0.5381 0.5241
0.3025 0.3884
0.4647 0.4871
0.6946 0.7312
0.3590 0.4638
yes no
0.4365 0.4793
0.4458 0.4720
0.3484 0.4018
0.4892 0.5184
0.6993 0.7619
0.3559 0.4216
yes no
0.5817 0.6184
0.4368 0.3544
0.5443 0.6008
0.6365 0.6734
0.5172 0.5214
0.4704 0.4667
yes no
0.3481 0.4209
0.3319 0.3765
0.3773 0.4901
0.2054 0.2305
0.5934 0.6485
0.1707 0.2503
Hispanic
yes no
0.3100 0.5074
0.4320 0.6081
0.4107 0.6972
0.7014 0.8387
0.4118 0.5200
0.2847 0.3974
Other
yes no
0.6462 0.6655
0.5723 0.5395
0.4158 0.4324
0.5066 0.4973
0.7568 0.7536
0.5179 0.5445
yes no
0.5186 0.5620
0.6386 0.7063
0.5824 0.7027
0.4018 0.4783
0.4449 0.5139
0.3181 0.3984
HS degree
yes no
0.4450 0.5887
0.3545 0.4258
0.4565 0.5157
0.5014 0.5947
0.6338 0.7175
0.3770 0.5421
Some college
yes no
0.3858 0.4927
0.4708 0.5691
0.2704 0.4570
0.4724 0.5373
0.7016 0.7445
0.4298 0.5288
College graduate
yes no
0.5020 0.4574
0.6807 0.6859
0.6250 0.5779
0.4894 0.4533
0.7008 0.6929
0.6200 0.5917
yes no
0.3372 0.3991
0.3110 0.3336
0.3958 0.5032
0.6546 0.7026
0.3672 0.4998
0.3393 0.5048
25 to 44 years
yes no
0.3967 0.4470
0.6324 0.5864
0.3893 0.4234
0.4720 0.4461
0.7384 0.7482
0.4007 0.4052
45 to 64 years
yes no
0.4427 0.5640
0.5188 0.5561
0.5261 0.5410
0.5361 0.5849
0.4740 0.5155
0.4378 0.5414
yes no
0.3307 0.4358
0.4881 0.5645
0.5737 0.6900
0.7932 0.8386
0.3848 0.5096
0.2885 0.3980
yes no
0.4847 0.5859
0.5998 0.5883
0.3325 0.4007
0.4344 0.4462
0.6650 0.6888
0.4294 0.4490
Total Population Total Gender Male
Female Race/Ethnicity Black
Education Level No HS degree
Age 16 to 24 years
Country of Birth Not USA
USA
* Compositing factors reflect imputation variance as well as sampling variance.
O-22
Table O-23. NALS compositing analysis: Compositing factors* for average document proficiency and literacy levels in Texas sample, by Total Population, Gender, Race/Ethnicity, Education Level, Age, and Country of Birth
Demographic Subpopulations
National Certainty PSU?
State Compositing Factor: Beta Level 1 225 or lower
Level 2 226 to 275
Level 3 276 to 325
Level 4 326 to 375
Level 5 376 or higher
Overall Proficiency
yes no
0.3289 0.3807
0.5877 0.6675
0.6021 0.6518
0.8470 0.8215
0.7856 0.8005
0.4302 0.4872
yes no
0.4216 0.4063
0.5078 0.5407
0.5764 0.5894
0.6979 0.6889
0.5525 0.5900
0.4404 0.4718
yes no
0.3462 0.4390
0.6287 0.6614
0.4100 0.5058
0.6525 0.6701
0.6864 0.6893
0.4531 0.4368
yes no
0.2817 0.3411
0.4027 0.4499
0.3892 0.5023
0.3046 0.3397
0.2146 0.2269
0.1863 0.2619
Hispanic
yes no
0.2895 0.3494
0.4103 0.5311
0.4047 0.4879
0.5005 0.6379
0.5993 0.6777
0.2495 0.3517
Other
yes no
0.5107 0.5447
0.6142 0.6628
0.6634 0.6829
0.8118 0.7743
0.8147 0.8126
0.6007 0.6057
yes no
0.4225 0.4542
0.5119 0.5416
0.5625 0.6136
0.4516 0.5243
0.3526 0.4166
0.3598 0.4261
HS degree
yes no
0.3409 0.4589
0.4104 0.5304
0.4964 0.5842
0.5182 0.6227
0.5129 0.6424
0.3930 0.5541
Some college
yes no
0.2857 0.3312
0.6572 0.7063
0.8040 0.8035
0.5851 0.6291
0.6697 0.6310
0.4407 0.4832
College graduate
yes no
0.5585 0.5260
0.6557 0.6071
0.4389 0.4584
0.7937 0.7876
0.7507 0.7642
0.6562 0.6598
yes no
0.4011 0.4642
0.7459 0.7183
0.6615 0.7383
0.4617 0.5174
0.4956 0.5756
0.3325 0.4641
25 to 44 years
yes no
0.2272 0.3017
0.4079 0.5017
0.4849 0.5485
0.8147 0.7914
0.7752 0.7916
0.4699 0.4616
45 to 64 years
yes no
0.5625 0.6063
0.5114 0.6017
0.5196 0.6032
0.6018 0.6688
0.6939 0.7438
0.5473 0.6138
yes no
0.4075 0.4508
0.4577 0.4877
0.5661 0.6354
0.7094 0.8045
0.6229 0.7321
0.3639 0.4693
yes no
0.2936 0.3584
0.4931 0.5884
0.5642 0.6034
0.7881 0.7529
0.7661 0.7667
0.4996 0.4831
Total Population Total Gender Male
Female Race/Ethnicity Black
Education Level No HS degree
Age 16 to 24 years
Country of Birth Not USA
USA
* Compositing factors reflect imputation variance as well as sampling variance.
O-23
Table O-24. NALS compositing analysis: Compositing factors* for average quantitative proficiency and literacy levels in Texas sample, by Total Population, Gender, Race/Ethnicity, Education Level, Age, and Country of Birth
Demographic Subpopulations
National Certainty PSU?
State Compositing Factor: Beta Level 1 225 or lower
Level 2 226 to 275
Level 3 276 to 325
Level 4 326 to 375
Level 5 376 or higher
Overall Proficiency
yes no
0.2241 0.3454
0.4726 0.5131
0.5472 0.5272
0.6135 0.6624
0.6658 0.7115
0.3376 0.4151
yes no
0.3876 0.4302
0.3573 0.3666
0.6171 0.6153
0.6089 0.6481
0.5879 0.6637
0.3230 0.3382
yes no
0.3777 0.4636
0.7730 0.7680
0.6185 0.6340
0.6074 0.6523
0.5756 0.5456
0.4491 0.4512
yes no
0.3334 0.4562
0.3643 0.4302
0.3101 0.3640
0.2905 0.3379
0.5769 0.6315
0.1644 0.2653
Hispanic
yes no
0.2444 0.3852
0.5400 0.6590
0.3662 0.5920
0.5042 0.8300
0.1135 0.1179
0.2552 0.4480
Other
yes no
0.5096 0.5785
0.5414 0.5578
0.5841 0.5488
0.6261 0.6421
0.7225 0.7186
0.6017 0.5857
yes no
0.2382 0.3514
0.4906 0.4326
0.3193 0.4258
0.3850 0.5265
0.3460 0.3745
0.2591 0.3968
HS degree
yes no
0.3371 0.5121
0.4231 0.5838
0.3840 0.4519
0.3835 0.4797
0.7888 0.7985
0.2941 0.4664
Some college
yes no
0.3528 0.4475
0.3052 0.3923
0.5782 0.6221
0.7458 0.7843
0.5754 0.6441
0.4272 0.4500
College graduate
yes no
0.3963 0.3674
0.5069 0.4631
0.5602 0.5031
0.4148 0.4337
0.6485 0.6166
0.7349 0.6912
yes no
0.3403 0.4795
0.4563 0.5121
0.3251 0.4123
0.2587 0.3416
0.4060 0.5453
0.3640 0.5427
25 to 44 years
yes no
0.2600 0.3734
0.5587 0.5810
0.5887 0.5848
0.7465 0.7420
0.7349 0.7426
0.3338 0.3552
45 to 64 years
yes no
0.4914 0.5559
0.3680 0.4260
0.5069 0.5262
0.6632 0.7337
0.5540 0.5554
0.5058 0.5849
yes no
0.3496 0.4308
0.3506 0.4279
0.4737 0.5195
0.7740 0.8248
0.5939 0.6355
0.3591 0.4511
yes no
0.3133 0.3972
0.4794 0.5195
0.5964 0.5759
0.6137 0.6609
0.6736 0.7061
0.3836 0.4372
Total Population Total Gender Male
Female Race/Ethnicity Black
Education Level No HS degree
Age 16 to 24 years
Country of Birth Not USA
USA
* Compositing factors reflect imputation variance as well as sampling variance.
O-24
Table O-25. NALS compositing analysis: Compositing factors* for average prose proficiency and literacy levels in Washington sample, by Total Population, Gender, Race/Ethnicity, Education Level, Age, and Country of Birth
Demographic Subpopulations
National Certainty PSU?
State Compositing Factor: Beta Level 1 225 or lower
Level 2 226 to 275
Level 3 276 to 325
Level 4 326 to 375
Level 5 376 or higher
Overall Proficiency
yes no
– 0.8765
– 0.8587
– 0.7978
– 0.8601
– 0.9465
– 0.8524
yes no
– 0.8315
– 0.8525
– 0.7959
– 0.8618
– 0.9576
– 0.7963
yes no
– 0.9013
– 0.7357
– 0.9063
– 0.9349
– 0.8685
– 0.8185
yes no
– 0.7669
– 0.7446
– 0.7970
– 0.6002
– 0.8971
– 0.5728
Hispanic
yes no
– 0.7954
– 0.8691
– 0.8658
– 0.9539
– 0.8512
– 0.7594
Other
yes no
– 0.9027
– 0.8084
– 0.7760
– 0.8057
– 0.9292
– 0.8557
yes no
– 0.9085
– 0.9454
– 0.9442
– 0.8718
– 0.8898
– 0.8319
HS degree
yes no
– 0.9125
– 0.8327
– 0.8782
– 0.9084
– 0.9480
– 0.8973
Some college
yes no
– 0.9200
– 0.9078
– 0.8991
– 0.9133
– 0.9443
– 0.8952
College graduate
yes no
– 0.7330
– 0.9049
– 0.8132
– 0.7425
– 0.9070
– 0.8391
yes no
– 0.8550
– 0.7768
– 0.8968
– 0.9476
– 0.8979
– 0.9136
25 to 44 years
yes no
– 0.8080
– 0.8726
– 0.7965
– 0.8122
– 0.9407
– 0.7682
45 to 64 years
yes no
– 0.9205
– 0.9052
– 0.9052
– 0.9245
– 0.8950
– 0.9163
yes no
– 0.8897
– 0.9327
– 0.9592
– 0.9817
– 0.9193
– 0.8690
yes no
– 0.9023
– 0.8860
– 0.8128
– 0.8328
– 0.9308
– 0.8228
Total Population Total Gender Male
Female Race/Ethnicity Black
Education Level No HS degree
Age 16 to 24 years
Country of Birth Not USA
USA
* Compositing factors reflect imputation variance as well as sampling variance.
– Not available.
O-25
Table O-26. NALS compositing analysis: Compositing factors* for average document proficiency and literacy levels in Washington sample, by Total Population, Gender, Race/Ethnicity, Education Level, Age, and Country of Birth
Demographic Subpopulations
National Certainty PSU?
State Compositing Factor: Beta Level 1 225 or lower
Level 2 226 to 275
Level 3 276 to 325
Level 4 326 to 375
Level 5 376 or higher
Overall Proficiency
yes no
– 0.7587
– 0.9316
– 0.9200
– 0.9558
– 0.9617
– 0.8460
yes no
– 0.7568
– 0.8611
– 0.8698
– 0.9071
– 0.8940
– 0.8137
yes no
– 0.8191
– 0.9152
– 0.8762
– 0.9264
– 0.9324
– 0.8022
yes no
– 0.6999
– 0.7968
– 0.8075
– 0.7162
– 0.6060
– 0.5935
Hispanic
yes no
– 0.7547
– 0.8508
– 0.8413
– 0.8937
– 0.9237
– 0.7216
Other
yes no
– 0.8452
– 0.9201
– 0.9168
– 0.9135
– 0.9508
– 0.8623
yes no
– 0.8531
– 0.9036
– 0.9242
– 0.8852
– 0.8450
– 0.8410
HS degree
yes no
– 0.8536
– 0.8956
– 0.9169
– 0.9117
– 0.9388
– 0.8915
Some college
yes no
– 0.7972
– 0.9422
– 0.9534
– 0.9101
– 0.8926
– 0.8570
College graduate
yes no
– 0.7958
– 0.8268
– 0.7990
– 0.9378
– 0.9482
– 0.8896
yes no
– 0.8771
– 0.9420
– 0.9584
– 0.8893
– 0.9181
– 0.9002
25 to 44 years
yes no
– 0.6915
– 0.8603
– 0.8928
– 0.9455
– 0.9557
– 0.7925
45 to 64 years
yes no
– 0.9178
– 0.9385
– 0.9282
– 0.9402
– 0.9636
– 0.9286
yes no
– 0.8804
– 0.9013
– 0.9453
– 0.9786
– 0.9670
– 0.8959
yes no
– 0.7438
– 0.9104
– 0.8987
– 0.9328
– 0.9504
– 0.8276
Total Population Total Gender Male
Female Race/Ethnicity Black
Education Level No HS degree
Age 16 to 24 years
Country of Birth Not USA
USA
* Compositing factors reflect imputation variance as well as sampling variance.
– Not available.
O-26
Table O-27. NALS compositing analysis: Compositing factors* for average quantitative proficiency and literacy levels in Washington sample, by Total Population, Gender, Race/Ethnicity, Education Level, Age, and Country of Birth
Demographic Subpopulations
National Certainty PSU?
State Compositing Factor: Beta Level 1 225 or lower
Level 2 226 to 275
Level 3 276 to 325
Level 4 326 to 375
Level 5 376 or higher
Overall Proficiency
yes no
– 0.7425
– 0.8592
– 0.8644
– 0.9323
– 0.9449
– 0.8149
yes no
– 0.7796
– 0.7310
– 0.8931
– 0.9109
– 0.9326
– 0.7066
yes no
– 0.8327
– 0.9504
– 0.9178
– 0.9287
– 0.8730
– 0.8132
yes no
– 0.7681
– 0.7744
– 0.7304
– 0.7082
– 0.8900
– 0.5771
Hispanic
yes no
– 0.7296
– 0.9058
– 0.8372
– 0.9071
– 0.5159
– 0.7488
Other
yes no
– 0.8901
– 0.8568
– 0.8153
– 0.9030
– 0.9165
– 0.8408
yes no
– 0.7792
– 0.8593
– 0.8388
– 0.8969
– 0.8380
– 0.8153
HS degree
yes no
– 0.8776
– 0.9153
– 0.8409
– 0.8732
– 0.9543
– 0.8688
Some college
yes no
– 0.8800
– 0.8334
– 0.9109
– 0.9603
– 0.9250
– 0.8319
College graduate
yes no
– 0.6717
– 0.7384
– 0.7608
– 0.7674
– 0.8469
– 0.8748
yes no
– 0.8920
– 0.8761
– 0.8511
– 0.8246
– 0.9098
– 0.9250
25 to 44 years
yes no
– 0.7746
– 0.8793
– 0.8726
– 0.9358
– 0.9381
– 0.7236
45 to 64 years
yes no
– 0.9033
– 0.8645
– 0.9004
– 0.9604
– 0.9140
– 0.9100
yes no
– 0.8800
– 0.8828
– 0.9115
– 0.9789
– 0.9407
– 0.8886
yes no
– 0.7834
– 0.8582
– 0.8873
– 0.9340
– 0.9410
– 0.8349
Total Population Total Gender Male
Female Race/Ethnicity Black
Education Level No HS degree
Age 16 to 24 years
Country of Birth Not USA
USA
* Compositing factors reflect imputation variance as well as sampling variance.
– Not available.
O-27
Appendix P ELECTRONIC CODE BOOK FOR WINDOWS USER’S MANUAL
Appendix P: Electronic Code Book for Windows User’s Manual NATIONAL ADULT LITERACY SURVEY When the Electronic Code Book for Windows (ECBW) is opened, the user will see the main window containing variable names and label names for the 1992 National Adult Literacy Survey. At the top of the table is a menu, and several buttons, which allows users to move throughout the ECBW and obtain the information needed. A description of all of the menu options is provided below. Shortcuts and toolbar buttons are underlined and described within the menu text.
File Menu The File menu options provide the user with the ability to move and export information about the variables selected from the main window. •
Output. The Output option of the File menu allows the user to export the codes that have been selected from the table. The user may export this information to one of the following types of files: 1. SAS-PC Code—Allows the user to create a file that tells a SAS program how to read the data; 2. SPSS—Allows the user to create a file that tells a SPSS program how to read in the data. The data in this file is compressed; 3. Code Book Text—Allows the user to create a file that contains information about the variables selected (frequency, labels, etc.); 4. Tag File—Allows the user to create a file that contains the variables selected so that items do not have to be re-selected each time the user enters the program. 5. AccessDB—Allows the user to create an Access database file that contains the variables selected.
•
View Output. The View Output option allows the user to look at the output file created
•
Import Tag Files. The Import Tag Files option allows the user to recall a previously created tag file.
•
Set Up. The Set Up option tells the user what directory the files are in and where the files are located.
•
Exit. Used to exit the ECBW.
P-1
Move Menu The Move menu allows the user to move between variable names and labels within the ECBW. The menu options are: • • • • • •
Top of List Prev Section (previous section) Next Section Bottom of List Prev File (previous file) Next File
The six black arrow buttons located on the toolbar can also be used to move within ECBW without accessing the Move menu. Move Button Options: Arrow Button Left/Right arrows with double lines: Left/Right arrows with single line: Left/Right single arrows:
Function Move to top of list/bottom of list Move to previous file/next file Move to previous section/next section
Tag Menu The Tag menu provides options for selecting and deselecting items. •
Tag/Untag Items. The Tag Items option in the Tag menu allows the user to select variables within the ECBW once they are highlighted with the mouse. The Untag Items option allows the user to deselect individual items. Users may also select a variable by simply clicking in the box located next to each variable. To deselect the user must click the box a second time.
•
Clear All Tags. The Clear All Tags option erases all the checked boxes in order to make a new selection of items.
•
Previous Tag/Next Tag. The Previous Tag and Next Tag options allow the user to move back and forth between the selected variables. Users may also move between tags by clicking on the blue and red arrow buttons located on the toolbar.
P-2
View Menu Once a variable is selected, the user may choose to obtain a more detailed description of the variable. The View menu provides this information. •
Description. When chosen, this option produces a Description/Frequency Window. Users may also reach the Description/Frequency Window by double clicking on the selected variable.
•
Description/Frequency Window. This window has two options. 1. View Description. The header over the window provides the user with information about the survey and where the information came from. The text inside the window provides a description of the variable (parameters, how it was derived, etc.). 2. View Frequency. Provides the user with the code, frequency, and percent category label of the variable selected. To exit the window, the user must click on the X button in the top right corner of the Description/Frequency Window.
•
Tagged Items. This option in the View menu allows the user to create a list of the items that have been tagged/checked from the main window. The user may also create a list of tagged items by clicking on the toolbar button that looks like a sheet of paper.
Search Menu When the Search menu is selected, a search window will appear. This window allows the user to search the Code Book by variable, label, or description in a forward or backward direction. To exit this window, the user must click on the X button in the top right corner of the search window. This window can also be accessed by clicking on the magnifying glass button located on the toolbar.
Help Menu The Help menu (located in the far right hand corner of the main window) provides the user with information pertaining to the ECBW. •
Contents. The user may search for information on a certain topic by selecting the Contents option. Once selected, a new window will appear which allows the user to select a topic of interest.
P-3
Help topics include:
System overview The ECB Main Window System Requirements How to Tag/Untag a Variable How to View Descriptions/Frequencies How to Create SAS/SPSS Program Code, Codebook Text, Tag and Access Database Files How to Navigate Through Variables How to View Tagged Variables How to Import Tag Files How to Search for Text How to Change the System Setup
•
Search—If the user is unsure of which topic to select, or would prefer to search for specific words and phrases in the Help feature, the user must click on the Search button at the top of the Help screen, and follow the given instructions. The user may choose to perform either an Index search, or a Find search by clicking on the tabs located at the top left of the Search Window.
•
Back—The Back button brings the user back to main topics page, once a topic has been selected and viewed.
•
Print—Allows the user to print out selections of the Help Manual for hard copy.
•
Other Menu Options: File, Edit, Bookmark, Options, Help. These features allow the user to further manipulate the information in the Help Manual. Certain pages may be copied, saved to another file, bookmarked or annotated, if needed.
To exit the Help window, the user must click on the X button located on the top right corner of the Help window. The user is able to keep this window open while working within the ECBW by shrinking it on the desktop. To minimize the window, the user must click the dashed line button, also located in the top right corner of the Help window (to the left of the X). To enlarge the window, the user must click on the view window button, which is located to the left of the close window (X) button when minimized at the bottom of the screen.
P-4
Appendix Q STANDARD ERRORS FOR TABLE 8.2
Appendix Q: Standard Errors for Table 8.2 NATIONAL ADULT LITERACY SURVEY Corrected with Taylor Series Sufficient cognitive data present
Insufficient cognitive data present
Reasons Related to Literacy
Reasons unrelated to literacy
Total
Total Age 16 to 64 65 and older Race/ethnicity Hispanic Asian Black White Other or missing Language spoken while growing up English Language other than English Missing data English spoken while growing up Hispanic Asian Black White English not spoken while growing up Hispanic Asian Black White Education 0 to 8 years 9 to 12 years HS grad or GED Some college College degree Other or Missing
NonEnglish Language
Mental Disability
Reading or writing difficulty
Total
Refused
Physical Disability
Other Unknown No answer
0.005
0.003
0.002
0.001
0.002
0.004
0.002
0.002
0.001
0.004 0.015
0.002 0.008
0.002 0.005
0.001 0.004
0.001 0.006
0.003 0.014
0.002 0.008
0.001 0.008
0.001 0.005
0.015 0.032 0.008 0.005 0.037
0.015 0.028 0.006 0.002 0.035
0.015 0.027 0.003 0.000 0.029
0.001 0.007 0.003 0.001 0.000
0.004 0.004 0.004 0.002 0.011
0.005 0.016 0.006 0.004 0.016
0.003 0.007 0.004 0.003 0.006
0.002 0.012 0.004 0.002 0.011
0.003 0.008 0.003 0.002 0.012
0.004 0.017 0.052
0.002 0.015 0.008
0.000 0.014 0.000
0.001 0.003 0.003
0.002 0.006 0.007
0.004 0.006 0.051
0.003 0.004 0.000
0.002 0.004 0.000
0.001 0.004 0.051
0.009 0.042 0.008 0.005
0.006 0.000 0.005 0.002
0.004 0.000 0.000 0.000
0.001 0.000 0.003 0.001
0.004 0.000 0.004 0.002
0.008 0.042 0.006 0.004
0.005 0.016 0.004 0.003
0.002 0.037 0.004 0.002
0.004 0.016 0.003 0.002
0.019 0.039 0.082 0.031
0.019 0.037 0.085 0.027
0.020 0.036 0.087 0.017
0.001 0.010 0.019 0.011
0.006 0.006 0.000 0.018
0.007 0.011 0.019 0.016
0.004 0.007 0.014 0.011
0.003 0.005 0.003 0.011
0.005 0.009 0.014 0.010
0.019 0.011 0.006 0.004 0.005
0.014 0.006 0.003 0.002 0.001
0.014 0.005 0.002 0.001 0.001
0.005 0.003 0.001 0.001 0.001
0.010 0.004 0.002 0.001 0.000
0.013 0.009 0.005 0.003 0.004
0.007 0.005 0.004 0.003 0.003
0.009 0.005 0.002 0.002 0.001
0.007 0.004 0.002 0.001 0.002
0.014
0.012
0.012
0.003
0.002
0.005
0.004
0.003
0.003
Q- 1
Appendix R STANDARD ERROR TABLES FOR CHAPTER 10
Appendix R: Standard Error Tables for Table 10 NATIONAL ADULT LITERACY SURVEY Chapter 10
(se's of proportions based on experimental design)
Table 10-2. Standard error of response rate $0
$20
$35
overall
Background questionnaire Response rate N SE
Background questionnaire (t-tests)
0.786
0.827
0.844
0.836
730
740
818
1558
0.015
0.014
0.013
0.009
t of row difference $0-$20
Exercise booklet Response rate N SE
$20-$35 0.928
0.979
0.98
0.980
574
612
690
1302
0.011
0.006
0.005
0.004
Exercise questionnaire (t-tests)
R-1
2 0.9
t of row difference $0-$20/$35
4.5
$20-$35
0.1
Table 10-3. Standard errors (SE) (based on assumed equal distribution of cases across all incentive situations within a row) Background Questionnaire
Exercise Booklet
$0
$20
$35
overall
$0
$20
$35
overall
0.015
0.014
0.013
0.009
0.011
0.006
0.005
0.004
N
331
331
331
331
287
287
287
287
Response rate
0.8
0.884
0.902
0.867
1
1
1
1
0.038
0.030
0.028
0.019
NA
NA
NA
NA
959
959
959
959
807
807
807
807
Response rate
0.784
0.873
0.86
0.842
0.944
0.983
0.996
0.976
25-44 (SE)
0.023
0.019
0.019
0.012
0.014
0.008
0.004
0.005
534
534
534
534
423
423
423
423
Response rate
0.749
0.79
0.833
0.792
0.896
0.958
0.962
0.94
45-64 (SE)
0.032
0.031
0.028
0.018
0.026
0.017
0.016
0.012
253
253
253
253
206
206
206
206
Response rate
0.837
0.758
0.831
0.814
0.934
0.978
0.953
0.952
65-74 (SE)
0.040
0.047
0.041
0.024
0.030
0.018
0.026
0.015
N
160
160
160
160
130
130
130
130
Response rate
0.86
0.736
0.842
0.813
0.743
0.97
0.919
0.876
0.048
0.060
0.050
0.031
0.066
0.026
0.041
0.029
51
51
51
51
23
23
23
23
Response rate
0.545
0.357
0.462
0.451
1
0.33
1
0.875
Missing
0.121
0.116
0.121
0.070
NA
0.170
NA
0.069
Total age (SE)
16-24 (SE) N
N
N
75+ (SE) N
t $0-$20
t $0-$35
0.00
1.72
3.01
0.92
-1.28
-1.61
-1.12
R-2
2.15
2.52
1.96
-0.10
-0.26
-0.49
t $0-$20
t $0-$35
2.43
3.58
2.02
2.18
1.26
0.48
3.19
2.25
Table 10-3. Standard errors — Continued (based on assumed equal distribution of cases across all incentive situations within a row) Background Questionnaire
Exercise Booklet
$0
$20
$35
overall
N
1769
1769
1769
Response rate
0.787
0.811
White (SE)
0.017
t $0-$20
t $0-$35
$0
$20
$35
overall
1769
1448
1448
1448
1448
0.853
0.819
0.934
0.975
0.979
0.964
0.016
0.015
0.009
0.011
0.007
0.007
0.005
226
226
226
226
187
187
187
187
Response rate
0.754
0.883
0.833
0.827
0.864
0.985
0.971
0.949
Black (SE)
0.050
0.037
0.043
0.025
0.043
0.015
0.021
0.016
161
161
161
161
143
143
143
143
Response rate
0.857
0.914
0.894
0.888
0.917
1
1
0.969
Hispanic (SE)
0.048
0.038
0.042
0.025
0.040
NA
NA
0.014
N
69
69
69
69
62
62
62
62
Response rate
0.8
0.958
0.84
0.87
0.909
1
1
0.98
0.083
0.042
0.076
0.040
0.063
NA
NA
0.018
63
63
63
63
36
36
36
36
Response rate
0.667
0.571
0.571
0.603
0.917
0.667
1
0.9
Missing
0.103
0.108
0.108
0.062
0.080
0.136
NA
0.050
t $0-$20
t $0-$35
Race/ethnicity
N
N
Other (SE) N
1.03
2.08
0.93
1.69
-0.64
R-3
2.96
1.20
0.58
0.35
-0.64
3.07
3.45
2.63
2.21
-1.59
Table 10-3. Standard errors — Continued (based on assumed equal distribution of cases across all incentive situations within a row) Background Questionnaire $0
$20
$35
overall
Exercise Booklet t $0-$20
t $0-$35
$0
$20
$35
overall
68
68
68
68
1
0.955
1
0.985
Still in HS (SE)
NA
0.044
NA
0.015
N
358
358
358
358
Response rate
0.721
0.78
0.789
0.767
No HS diploma (SE)
0.041
0.038
0.037
0.022
486
486
486
486
Response rate
0.867
0.949
0.966
0.894
HS diploma (SE)
0.027
0.017
0.014
0.014
553
553
553
553
Response rate
0.923
0.952
0.973
0.951
Some college (SE)
0.020
0.016
0.012
0.009
345
345
345
345
Response rate
0.952
0.962
0.982
0.965
College degree (SE)
0.020
0.018
0.012
0.010
66
66
66
66
Response rate
0.467
1
0.65
0.645
Missing
0.106
NA
0.102
0.059
t $0-$20
t $0-$35
Education N
N
N
N
N
Note: NA indicates no variation in (100% response rate) in cell
R-4
1.06
1.23
2.58
3.27
Table 10-3. Standard errors for exercise booklet (no high school vs. high school)
SE
No High Sch
High School
t of row
$0
0.041
0.027
2.980
$20
0.038
0.017
4.055
$35
0.037
0.014
4.428
Table 10-3. Standard errors for exercise booklet (some college vs. college) Some College SE
College
t of row
t $20-$35
$0
0.020
0.020
1.036
Some College
0.420
$20
0.016
0.018
0.420
College
1.063
$35
0.012
0.012
0.523
R-5
Table 10-3. Standard errors (minority population equals black and Hispanic population combined)
Background questionnaire Response Rate HisBl SE
SE HisBlack
Response Rate White
SE White
t of row
$0
0.797
0.035
0.787
0.017
0.257
$20
0.896
0.027
0.811
0.016
2.708
$35
0.858
0.031
0.853
0.015
0.158
t$20-$35 HisBlack
0.929
Exercise booklet Response Rate HisBl SE
SE HisBlack
Response Rate White
SE White
t of row
$0
0.887
0.03
0.934
0.011
1.46
$20
0.992
0.009
0.975
0.007
1.46
$35
0.984
0.012
0.979
0.007
0.332
Note: HisBl represents the combination of the Black and Hispanic poulations
R-6
t $20-$35 HisBlack
0.53
t $0-$20 3.33
Table 10-3. Standard errors (ages 16-64 combined and 65-75+ combined)
Background questionnaire Response Rate 16-64 SE
SE 16-64
Response Rate 65-75+
SE 65-75+
t of row
$0
0.777
0.017
0.844
0.031
1.91
$20
0.851
0.0145
0.758
0.037
2.6
$35
0.86
0.014
0.833
0.032
0.762
Exercise booklet Response Rate 16-64 SE
SE 16-64
Response Rate 65-75+
SE 65-75+
t of row
$0
0.941
0.011
0.86
0.033
2.34
$20
0.979
0.006
0.975
0.015
0.229
$35
0.987
0.005
0.94
0.022
2.031
T-tests for background questionnaire compared to exercise booklet Response Rate 16-64
Response Rate 65-75+
$0
3.011
0.361
$20
8.085
5.707
$35
8.471
2.742
R-7
Table 10-4. Standard errors for initial cooperation rate, by incentive level: 1991 Field test Incentive
$0
$20
$35
566
603
684
Response rate
0.126
0.066
0.051
Incomplete (SE)
0.014
0.010
0.008
N
t of $0-$20 t of $0-$35 t of $25$35 -3.48
-4.60
-1.14
Table 10-5. Standard errors for initial cooperation rate, by incentive level: Older adults 1991 field test
Incentive
$0
$20
$35
N
135
88
116
Response rate
0.24
0.09
1
0.037
0.031
0.028
Incomplete (SE)
t of $0-$20 t of $0-$35 t of $25$35 -3.14
R-8
-3.04
0.24
Table 10-7. Standard errors for prose scale Total Std Error Total Sex Male Female Race/ethnicity White Black Hispanic Education Still in H.S. < HS Some HS GED/HSEQ HS diploma Some college College degree Age 16-20 21-25 26-31 32-45 46-64 65+ Income
