the amount of money spent on the Expanded Food and Nutrition Education ... Education Program (EFNEP) is one of the largest federally funded nutrition education ... The total. EFNEP budget for each state/territory is available as well. All data ...
Returns to Scale and the Effectiveness of Money Spent on the Expanded Food and Nutrition Education Program
Ranju Baral George C. Davis Wen You
Selected Paper for Presentation at the Agricultural and Applied Economics Association’s 2011 AAEA&NAREA Joint Annual Meetings, Pittsburgh, Pennsylvania, July 24-26, 2011
Ranju Baral is a Ph.D. candidate, George C. Davis a Professor, and Wen You an Assistant Professor, all in the Department of Agricultural and Applied Economics, Virginia Tech, Blacksburg VA 24061.
Copyright © 2011 by Ranju Baral, George C. Davis, and Wen You. All rights reserved. Readers may make verbatim copies of this document for non-commercial purposes by any means, provided that this copyright notice appears on all such copies.
2
Abstract In operation for more than 40 years and now in all 50 states and 6 territories, the Expanded Food and Nutrition Education Program has become a cornerstone in US nutrition education. The aim of the program is to assist limited resource audiences to acquire the knowledge, skills, attitudes, and changed behavior necessary for nutritionally sound diets, and to contribute to their personal development and improvement of the overall family diet and nutritional well-being. However, very little is known about the effectiveness of this program, especially at the national level. The purpose of this research is to determine the effectiveness of money spent on the Expanded Food and Nutrition Education Program in satisfying its stated goals for adult participants. Data from all states and territories participating in the program for the years 2000-2006 are utilized in a non-linear seemingly unrelated regression framework to estimate returns to scale and related cost measures. Controlling for participant and program characteristics, the amount of money spent on the Expanded Food and Nutrition Education Program has a positive and significant impact on two of three federal outcome indices used to measure outcomes for adults. Larger-funded programs (states/territories) are relatively more cost efficient than smaller programs.
3
Returns to Scale and the Effectiveness of Money Spent on the Expanded Food and Nutrition Education Program
Nutrition education programs are a common policy tool for improving nutrition and consequently public health. Lower socioeconomic status (SES) populations are usually targeted because of the well-established positive relationship between SES and health quality (e.g., Marmot and Wadsworth 1999). The Expanded Food and Nutrition Education Program (EFNEP) is one of the largest federally funded nutrition education programs in the United States and is administered by the United States Department of Agriculture (USDA) to youth and adults (General Accounting Office 2004). In operation for more than 40 years and now in all 50 states and 6 territories, the EFNEP has become a cornerstone in US nutrition education (USDA 2009a). The aim of the EFNEP is to assist limited resource audiences to “acquire the knowledge, skills, attitudes, and changed behavior necessary for nutritionally sound diets, and to contribute to their personal development and improvement of the overall family diet and nutritional well-being” (2009a). The EFNEP is administered as a series of lessons at the county level through the USDA Cooperative Extension Service. Presently, the USDA spends about $66 million per year on the EFNEP (USDA 2009b). However, there is no national level information on the effectiveness of this federal level appropriation. The USDA does reports basic EFNEP “impact data” annually, where impact is measured as the percentage of participants showing improvement from a pre-test to a
4 post-test on questions related to food resource management practices, nutrition practices, and food safety practices (USDA 2009a). The data suggest most participants tend to improve in these domains. However, these are only summary statistics with no multivariate analysis correlating these results with dollar expenditures or other program characteristics. A few more sophisticated analyses have appeared in the peer reviewed literature (e.g., Arnold and Sobal 2000; Dickin, Dollahite, Habicht 2005; Dollahite and Pierce 2003). The common approach in this literature has been to study a subset of participants in a single state in a single year that graduated from the EFNEP and analyze changes in nutrition knowledge or behavior. The general finding is that the EFNEP improves nutrition knowledge and behavior. While these results are encouraging and suggestive, these studies are not designed to answer three fundamental questions: (1) Does the money spent on the EFNEP contribute to the stated objectives of the program at the national level? (2) What are the returns to scale associated with the EFNEP money in achieving its objectives at the national level? (3) How much does it cost to increase the number of participants satisfying the objectives of the program at the national level? Using the USDA impact data for all states and US territories for seven consecutive years (2000–2006), but in conjunction with several available covariates, we answer these questions using a nonlinear multivariate regression model. Results show that after controlling for participant and program characteristics, the amount of money spent on the EFNEP has a positive and significant impact on two of the three outcome indices used by
5 USDA. One index shows increasing returns to scale and another shows constant returns to scale associated with the EFNEP budget. We also find that larger programs are relatively more cost efficient and the cost of adding an additional person satisfying the program’s objective (i.e., the marginal cost) can range from $375 to $695 on average, depending on the outcome index. These findings are the first to quantify the budget effects at the national level and also help in identifying potential sources where cost effective information may be gleaned. Limitations are discussed in the conclusions. Methods USDA collects data on the EFNEP by state/territory through its Nutrition Education Evaluation and Reporting System (NEERS) (USDA 2009c). The NEERS is a software program that allows administrators at the county and state level to collect and report data to USDA in aggregate form. Though the EFNEP is administered to youth and adults (age 18 and over), we focus on adult participants because the data and the methodology for collecting data from adults is standardized across all states/territories and therefore the adult data is more continuous, consistent, and reliable than for youth. Given the structure of the program, the budget effects will potentially be affected by participant and program characteristics so the analysis must control for these potential confounding factors. In general, the NEERS collects data on participant characteristics (e.g., age, education, and income), program characteristics (e.g., type of lesson, type of instructor) and most importantly responses to 10 required behavioral checklist questions for adults. The total EFNEP budget for each state/territory is available as well. All data is for all 50 states and 6 US territories for the fiscal years 2000 to 2006.
6
Dependent Variables We use the same measurements developed by USDA as our dependent variables. Specifically, the USDA uses 10 behavioral checklist questions to form the basis for evaluating the success of the EFNEP. The 10 behavioral checklist questions were developed over a span of five years from 1993 to 1997 and involved several phases of development that assured content and face validity (see Anliker, Willis, and Montgomery 1998 for details). The major phases were (i) a committee formed the questions based on reviews of other existing and submitted instruments, (ii) feedback was solicited on the instrument from the EFNEP Coordinators in all 50 states, (iii) more feedback on the instrument was solicited from a larger pool of specialists, (iv) an Expert Panel met in Washington DC to revise the instrument, (v) the revised instrument was subjected to focus group testing and tested for reading level (determined it was at the 6th grade reading level), (vi) based on feedback from the focus group, the instrument was revised and pilot tested in seven states, (vii) statistical analysis was conducted of the pilot tested instrument and the instrument was checked for validity and internal reliability, (viii) additional minor revisions were made based on the validity and reliability analysis to improve and finalize the instrument. The 10 behavioral questions are given in table I and responses are based on a 5 point Likert scale (e.g., 1 = do not do, 2 = seldom, 3 = sometimes, 4 = most of the times, 5 = almost always). Participants answer these behavioral questions both pre and post EFNEP participation. USDA uses subsets of the 10 questions to create three indices to measure different aspects of nutrition knowledge: the Food Resource Management
7 Practices (FRMP) index is constructed from answers to questions 1–4, the Nutrition Practices (NP) index is constructed from answers to questions 1 and 7–10, and the Food Safety Practices (FSP) index is constructed from answers to questions 5 and 6. Improvement in an index occurs when the Likert score on at least one of the index questions increases from the pre-test to the post-test. Our data are measured in terms of the percentage of participants showing an improvement for each index and, consequently, each index ranges from 0 to 100. Table II lists the variables definitions and summary statistics. On average, 83% of the participants improved in the FRMP index, 88% of the participants improved in the NP index, and 64% of the participants improved in the FSP index. The corresponding standard deviations are FRMP 9%, NP 6%, and FSP 11%.
Covariate Explanatory Variables Budget: The variable budget is defined as the annual budget allocation (in US $) for each state and US territory. The average amount over all observations (i.e., states/territories and years) is $1,034,542 with a standard deviation of $908,877 (table II). We hypothesize that controlling for the other covariates, as the budget increases the index scores will improve. Number of Participants and Participant Characteristics: We include in the analysis the number of participants as a control variable. The average number of participants is about 2,880 with a standard deviation of 4,200 (table II). We included several sociodemographic categorical characteristics of the participants as control variables: gender, race/ethnicity, age, educational level, household income, and place of residence (table II).
8 These socioeconomic control variables are typical of those found in the literature on health behavior (e.g., Pollard, et al. 2009). On average, females constitute 89% and whites and blacks about 65% of the participants. Most of the participants (~68%) fall in the 20 to 39 year age range and most (~63%) have income below 100% of the poverty line. Unfortunately, most of the participants (~72%) did not provide information on their level of education, but for those who did the largest percentage is for those with education only through some high school (~23%). The majority of participants (~43%) reside in central cities with populations over 50,000. Program characteristics: Program characteristics may be important in terms of behavior change, so the analysis includes types of lessons and types of instructors as program control characteristics. EFNEP offers four types of lessons. The largest percentage of participants attend group lessons (~73%), followed by individual lessons (~21%), group and individual lessons (~6%) and other lesson types (~1%). Three types of instructors are involved in the program: professionals, paraprofessionals, and volunteers. The professionals train the paraprofessionals and volunteers, and provide technical support. The paraprofessionals and volunteers are mostly from the local area; thus, they are expected to be best suited to deliver lessons to local participants. Volunteers constitute the largest percentage of instructors (~85%), followed by paraprofessionals (~13%) and professionals (~2%).
Data analysis There are three dependent variables: the FRMP index, the NP index, and the FSP index. Each is restricted to the range 0 to 100% and, to account for this restricted range,
9 we use a logistic functional form for each equation (Kmenta 1997). Mathematically, the three equations to estimate have the form
(1) Yi = 100 / (1 + e −α0 i −α1i ln B −α 2 i X ) + ε i
i = FRMP, NP, FSP ,
where Y denotes the nutrition index, ln B is the natural log of the budget, X the vector of control variables, ε the error term with an expected value of zero and the αs are conformable parameters. In economics, equation (1) represents a simple indirect production function, which is the natural economic construct for conducting cost-return or ‘cost-benefit’ type analysis (Shephard 1974). An important feature of this model is that it is nonlinear and allows for increasing, constant, or decreasing returns to scale in the budget. Specifically, the change in the index for a one percent change in the budget (i.e., a one unit change in the natural log of B, ln B) is the marginal budget effect, or mathematically from equation (1), (2) MBEi = ∂Y / ∂ ln B = α1i × ⎡⎣100 × e −α0 i −α1i ln B −α 2 i X / (1 + e −α0 i −α1i ln B −α 2 i X ) 2 ⎤⎦ The sign of the marginal budget effect is determined by the sign of the parameter on the budget, α1i, because the term in brackets is always positive. The magnitude of this effect will vary depending on the value of the budget and other covariates (i.e., the point of evaluation). Furthermore, because the dependent variable is a percentage and the budget is expressed in natural logs, simple calculus reveals that this marginal effect is equal to the ratio of average cost (ACi) to marginal cost (MCi) or (3) MBEi = ACi / MCi
i = FRMP, NP, FSP .
The average cost (ACi) is defined as the cost per person showing an improved index score and the marginal cost (MCi) is defined as the cost to produce one more participant with an improved index score. Basic economics indicates if ACi > MCi and therefore MBEi >
10 1, there are increasing returns to scale, if ACi = MCi (MBEi = 1) there are constant returns to scale, and if ACi < MCi (MBEi < 1) there are decreasing returns to scale. The average cost is easily calculated from available data. However, the marginal cost cannot be estimated without an estimate of the marginal budget effect MBEi via equation (3). Because the error terms across equations are likely correlated, we use the non-linear seemingly unrelated regression (NLSUR) method (Wooldridge 2002) for model estimation using the statistical software STATA v10. As many of the explanatory variables are categorical percentages, a reference set of categories must be chosen to avoid the perfect collinearity problem (the more general version of the ‘dummy variable trap’). The reference case here is the percentage of participants under the age of 20, with income less than 50% of the poverty level, education up to the high school level, residing in a central city with a population over 50,000, participated in a group lesson, and instructed by a volunteer. Gender is excluded from the estimation model as ~90% of the sample is female, so the variation is quite limited. Finally, because of repeated observations by states/territories, we use a cluster-robust covariance matrix to improve efficiency as there is likely correlation in the errors over time for the same state/territory (Woolridge 2002). Results
The parameter estimates and their corresponding p-values are given in table III and are only briefly discussed as the marginal effects and cost estimates are the central focus of the analysis.
11 Budget Consistent with our hypothesis, the parameter estimate for the budget is positive and significant at conventional levels in the Food Resource Management Practices (FRMP) and the Nutrition Practices (NP) index equations (FRMP: 0.135, p-value = .02 and NP: 0.102, p-value = .08). However, the parameter estimate for the budget is not significantly different from zero in the Food Safety Practices (FSP) index equation (0.0003, p-value = .99).
Control Variables Very few of the participant characteristics are significant. We focus on those that are significant in more than one equation. As the percentage of participants with incomes over the 150% of the poverty line increases, the NP index (0.051, p-value =.02) and FSP index (0.028, p-value = .05) both increase. Though still positive, this effect is not significant in the FRMP index (0.035, p-value = .15). When the percentage of participants who did not report their age increases relative to the percentage of younger participants (i.e., those of age less than 20), all indices decrease (FRMP: –0.022, p-value = .09; NP: –0.022, p-value = .04: FSP: –0.016, p-value = .10). The only other control variable that is significant in more than one equation is the town size variable (Town2) for a town with a population less than 10,000 and rural non-farm area. Relative to the town base (Town5), those in a town with a population less than 10,000 and rural nonfarm scored higher on all three indices (FRMP: 0.012, p-value =
