Food Safety Practices Assessment Tool: An ... - Wiley Online Library

Food Safety Practices Assessment Tool: An Innovative Way to Test Food Safety Skills among Individuals with Special Needs Elena T. Carbone, Stanley E. Scarpati, and Lori F. Pivarnik

Abstract: This article describes an innovative assessment tool designed to evaluate the effectiveness of a food safety skills curriculum for learners receiving special education services. As schools respond to the increased demand for training students with special needs about food safety, the need for effective curricula and tools is also increasing. A Food Safety Education for High School and Transition Special Needs Students curriculum served as the basis upon which our assessment tool was developed. The project was a collaborative effort by food safety and education professionals in Connecticut, Rhode Island and Massachusetts. This USDA-funded initiative emerged from teacher-generated data that identified critical gaps in food safety knowledge and skills among students with disabilities (SWD) receiving special education services. As an adjunct to this curriculum, a Food Safety Practices Assessment Tool was developed to: 1) conduct observations of students as they demonstrate food safety practices, and 2) use this information to design classroom-based learning activities that are aligned with students’ Individual Education Plans (IEP). Pilot data suggest that the tool is valid and reliable for use in a kitchen-based setting. This is the first known tool of its kind to test food safety skills of individuals with special needs in a real-world environment. Further testing is needed to determine the usefulness of the tool for broader audiences.

The Centers for Disease Control and Prevention (CDC) estimates that foodborne diseases cause approximately 48 million illnesses, 130,000 hospitalizations, and 3,000 deaths each year (Scallan and others 2011a; Scallan and others 2011b). Federal agencies such as the US Dept. of Agriculture (USDA), US Food and Drug Administration (FDA), and CDC have responded to this public health issue by developing and implementing guidelines and interventions to address the need for highly qualified food safety workers. Education initiatives such as FightBac!® (Partnership for Food Safety Education 2009) and Be Food Safe (United States Dept. of Agriculture Food Safety Inspection Service 2008) have been developed for consumers and food service personnel to increase knowledge about food handling practices that can reduce risk of illness. While these initiatives reflect the critical importance of food safety knowledge through education and training for the general public (United States Dept. of Health and Human Services 2009), little is known about the effectiveness of training and testing programs on participants in general, and even less among employees with unique learning characteristics. An often over looked segment of this population are students with disabilities (SWD), particularly high school and transition students MS 20120551 Submitted 4/16/2012, Accepted 9/18/2012. Author Carbone is with Univ. of Massachusetts, Dept. of Nutrition, 100 Holdsworth Way, Chenoweth Lab, Amherst, MA 01003–9282. Author Scarpati is with Univ. of Massachusetts, School of Education (Emeritus). Author Pivarnik is with Univ. of Rhode Island, Nutrition and Food Sciences Dept. Direct inquiries to author Carbone (Email: [email protected]) c 2012 Institute of Food Technologists® doi: 10.1111/j.1541-4329.2012.00161.x

with special educational needs. For the purposes of this article, students receiving special education services are identified as those who have a disability based on either a state or federal definition and for whom appropriate diagnostic and evaluation teams have concluded that special education services are needed to make reasonable academic progress in school (Individuals with Disabilities Education Improvement Act 2004). These students are placed on Individual Education Plans (IEP) that guide their school activities and identify the services they need to access the general school curriculum. Transition students are those individuals receiving special education services who have IEP objectives beginning at age 14 that specifically address the skills and services they will need to be successful when they leave high school (Individuals with Disabilities Education Improvement Act 2004). The food safety skills used as the basis of the Food Safety Practices (FSP) Assessment Tool described in this article are examples of IEP-based transition objectives, specifically designed to meet industry-approved food safety requirements.

Students with Disabilities (SWD) in Food Service National data collected by the federal government (Sanford and others 2011) point to the food service industry as a major player in the employment success of post secondary SWD. Among this group, 17% worked in food service, 11% worked as a skilled laborer, and 10% were employed as a cashier. Youth were significantly more likely to work in food service than in most other types of jobs (Sanford and others 2011). Vol. 12, 2013 r Journal of Food Science Education 7

Food safety practices assessment tool. . . Since food service is among the most viable employment options for SWD, handling foods safely is a critical pre-employment skill. Demonstration of knowledge as a preventive strategy for reducing foodborne illness risk is also fundamental to the FDA Food Code (United States Dept. of Health and Human Services 2009). Since 1997, the FDA Food Code has required demonstration of knowledge of foodborne disease along with food handling procedures determined to reduce the risk of food related illnesses. Given the special learner characteristics of SWD, reliance on traditional paper and pencil evaluations might not provide useful information about certain skills (Elliott and Thurlow 2006). Real-life demonstrations of safe food handling practices can result in more beneficial assessment data of student skill development (Stiggins 1987). A Food Safety Education for High School and Transition Special Needs Students curriculum served as the basis upon which our assessment tool was developed (Madaus and others 2010). The project was a collaborative effort by food safety and education professionals in Connecticut, Rhode Island and Massachusetts. This USDA-funded initiative emerged from teacher-generated data that identified critical gaps in food safety knowledge and skills among students with disabilities (SWD) receiving special education services. As an adjunct to this curriculum, the FSP Assessment Tool was developed to: 1) conduct reliable observations of students as they demonstrate food safety practices and 2) use this information to design classroom activities that are in keeping with students’ IEPs. This is the first known tool of its kind to test food safety skills of individuals with special needs in a real-world setting.

performance is established. In this case, it would assume to be set at 100%.

Methods Once the study was approved by the Univ. of Massachusetts (UMass) Institutional Review Board and a setting with high school SWD was selected to pilot test the tool, an observation study was designed in three phases. In phase I, an “Observation Template” was created to establish a framework against which all observed data were compared. Design of the template adhered to the philosophy and guidelines of what is commonly referred to as Universal Design for Learning (UDL) (Hehir 2009). UDL assumes all learners have unique cognitive learning styles and that a “one size fits all” approach ignores these differences and is incapable or inflexible of meeting the individual needs of many students, especially those with disabilities. The template was also designed in consultation with the Culinary Arts Manager around four existing lunch meals to reflect a variety of foods prepared by the students (see Figure 1). Organization centered around the four core practices for safe food handling established by Fight Bac!: 1) Clean, 2) Separate, 3) Cook, and 4) Chill (Partnership for Food Safety Education 2009). Using these four general categories, specific safe food handling skills were identified and, for each of the four meals selected for inclusion in the pilot test, the possible opportunities for these behaviors to be demonstrated were recorded. After consulting with the Culinary Arts Manager, we knew for example, if using a cutting board was part of the preparation process and how many times it should be used for each particular meal selected. In this way a realistic Meal #1

Performance Assessment and Food Safety Skills

Egg salad sandwich with lettuce, tomato and mayonnaise

Validity of the FSP Assessment Tool was based on observing Celery with Ranch dip students demonstrating what they know or what they can do. Cole slaw with carrots Canned pear halves In contrast to assessments that limit how students respond by reBrownie with peanut butter chips quiring them to select a response in a multiple choice or true or false fashion, performance assessments require students to conMeal #2 struct their responses while they are active participants in the assessment process. Students are therefore responsible for constructing Chicken noodle soup their responses and creating an answer or a product rather than Fish tenders with tartar sauce simply filling in a blank, selecting a correct answer from a list, or Onion rings Tossed salad deciding whether a statement is true or false. The focus of perFresh pineapple formance assessment is on measuring students’ learning as part of Fruit sherbet real kitchen-based activities representing cooking situations that demand attention to safe food handling practices. The appeal of Meal #3 performance assessment is that it is criterion-referenced. This approach illustrates how well students are performing on specific Clam chowder All white meat chicken nuggets with dipping sauce goals or standards, rather than how their performance compares Greek tortellini salad to a norm group of students nationally or locally, and is thereDinner roll fore intended to show individual growth and inform instruction. Apples Finally, using performance assessment with SWD when handling Oatmeal raisin cookie food refers to their specific skills and competencies in relation to a continuum of agreed upon standards of proficiency or excelMeal #4 lence and requires professional rater judgment in its design and interpretation (Haney and Madaus 1989; Stiggins 1987). Beef tacos with salsa Information gathered on the FSP Tool not only provides criFat free sour cream terion performance indicators, or benchmarks, of what students Green salad Brown rice know (and need to improve upon), but it can also provide useful Peaches insights into instructional design needs, materials development, Apple strudel and define opportunities for success. For example, hand washing involves several steps; therefore, a student may successfully demonstrate all or a portion of those steps. A benchmark for adequate Figure 1–Observed lunch menu items 8 Journal of Food Science Education r Vol. 12, 2013

Available on-line through ift.org

Food safety practices assessment tool. . . baseline was obtained, against which future observations could be compared. The draft template was sent to the project Advisory Committee for review and comment. After the template was revised, the next step (Phase II) was to establish an observation system. Safe food handling behaviors were observed and recorded using both a time sampling and an interval recording system. Intervals are pre-established time segments during which each of the four food safety behaviors were likely to be demonstrated. We established a 5 to 10 minute time frame for the intervals once the template was designed and the possible opportunities for each behavior to occur were identified. A time sampling methodology was used to collect data about the degree to which students demonstrated the selected food safety skills (Primavera and others 1997). Because this approach involves intermittent observation, the observer is able to study a behavior without having to set an amount of time aside to observe continually. Thus, it is a practical way to estimate the overall occurrence of a behavior. Time sampling is similar to interval recording in that the observation time is divided into intervals (in this case 5 to 10 minutes); however, in time sampling, the behavior is recorded only if it occurs at the end of the time period when the observer is required to check if the behavior has been demonstrated or not. For instance, if proper washing of fruits and vegetables is being observed, it would be recorded as occurring/not occurring at the end of each interval. When the specified amount of time has expired, the observer(s) looks at the student and determines whether or not the behavior is occurring. Time sampling is appropriate for behaviors that are longer in duration (such as proper washing of fruits and vegetables). If the behaviors are of shorter duration (such as checking proper cooking temperature of a meat product), the time sampling method would be adjusted to an interval recording method where behaviors are designated as either “occurring” or “not occurring” during each interval. In either case, total occurrences of the specified behavior are summed for each period, divided by the number of recording intervals, and multiplied by 100 to establish a percentage. For example, if washing fruits and vegetables occurred four times during an eight segment recording interval, a value of 50% would be assigned to this behavior: (4 occurrences/8 segments) = 50%. In Phase III, pilot “template” observation data were collected for each of the four meals selected during Phase I. A number of specific guiding principles and practices were adhered to throughout the data collection process (see Table 1) (adapted in part from Patton 2002). Three to six students were observed as a group preparing and/or cooking meals during each of the 60-minute meal preparation periods pre-established by the facility. Two authors (Carbone and Scarpati) independently observed the students. Check marks were made on the FSP Assessment Tool to indicate if a food safety skill had been demonstrated.

Reliability Inter-rater or inter-coder reliability examines the extent to which different observers using the same instrument get equivalent results (Singleton and others 1993). Inter-rater reliability checks the stability of data collected with the observational forms and should be completed on at least three separate occasions during the initial data collection phase if possible (Singleton and others 1993). In this study, two independent raters (Carbone and Scarpati) used the FSP Assessment Tool at the same time, on the same students, based on the same observable behavior. Both raters were knowledgeable about proper use of the instrument and in the food safety Available on-line through ift.org

Table 1–Principles and practices informing data collection. Principles Observation procedures should not be a distraction from instruction.

Practices Clearly describe the observation procedure.

Observation should not take undue time away from instruction.

Clearly define the behavior/s to be observed.

Data collection should provide a learning experience for students.

Establish a scoring and recording system.

Results should lead to frequent changes and improvements in instructional process.

Create an opportunity for observers to learn and practice the observation technique.

The process should not create undo anxiety for students.

Establish a behavioral recording system that is best suited to monitor the behavior/s.

The process should encourage cooperation.

Clearly define the schedule of when behavior/s will be observed and the observation intervals (periods).

The process should respect the privacy and dignity of students.

Create an opportunity to observe and record students’ behavior. Conduct reliability testing. Graphically represent the data.

(Adapted in part from Patton 2002)

behaviors and skills prior to implementation. Inter-rater reliability (that is, percent of agreement) was calculated as follows: Agreements × 100 Agreements + Disagreements For example, if Rater #1 and Rater #2 agreed that hand washing was observed a total of 8 times over a 10-minute interval session but disagreed about its occurrence during two intervals, the overall reliability rating would be 80% {8/8 + 2 = 80%}. Data collection and systematic behavior observation procedures used in this study provided an objective basis for making reliable decisions and documenting student behaviors and performance.

Results and Discussion A total of 15 students (9 male, 6 female) between the ages of 15–19 years old agreed to participate in the pilot testing of the FSP Assessment Tool. All participants had an Individual Education Plan (IEP) and attended a high school for students with learning, emotional, and behavioral disabilities. Their specific learner characteristics were aligned with state and federal definitions of these disabilities and included average or above intellectual ability, achievement deficits, selective attention difficulties, and emotional reactions to situations that might interfere with their learning and/or the learning of others. No other specific student data were available due to privacy issues and legal constraints regarding access to IEP information. Four different lunch meals were observed over a nine-week period. Each observation session took approximately one hour. This observation study was exploratory in nature; data were collected over time in an iterative fashion, with each observation session informing the next. Therefore, changes were made to both the content and format of the FSP Assessment Tool throughout this pilot study. After the first observation (meal #1), for instance, we discovered the need to add a space for “other” food safety practices Vol. 12, 2013 r Journal of Food Science Education 9

Food safety practices assessment tool. . . Table 2–Observation data for student-based food safety practices. Food Safety Category and Observable Practices Clean Wash hands properly Clean prep surfaces Pre-wash vegs/fruits Wash boards/utensils Clean spills Other (rinsed lids) Practice Performance Rate (%)

Meal #1a c Exp Obsd %e 4 3 75 2 2 100 4 3 75 1 1 100 f NA NA NA 1 1 100 83

Expc 6 2 4 1 NA NA

Chill Keep refrigerated foods at proper temp Check freezer temp Chill hot foods properly Practice Performance Rate (%)

Expc 2

Obsd 1

%e 50

Expc 2

Obsd 2

NA NA

NA NA

NA NA 50

NA 1

Separate Separate cutting boards Separate utensils for meat/fish Proper storage of meat/fish Practice Performance Rate (%)

Expc NA

Obsd NA

%e NA

NA

NA

NA

Overall Rates (%)b

Meal #3a c Exp Obsd %e 6 5 83 3 2 67 1 1 100 2 2 100 1 1 100 NA NA NA 85

Expc 5 3 4 2 2 NA

%e 100

Expc 1

Obsd 1

%e 100

Expc 2

Obsd 1

%e 50

71

NA 1

NA 100 100

NA 2

NA 1

NA 50 67

NA 1

NA 1

NA 100 67

NA 75 73

Expc 1

Obsd 1

%e 100

Expc 1

Obsd 1

%e 100

Expc 2

Obsd 2

%e 100

100

NA

1

1

100

1

1

100

1

1

100

100

NA

NA

1

1

100

1

1

100

1

1

100

100

100

100

Cook Use thermometer to check internal cooking temp Other (cook eggs until yolks/whites are firm) Practice Performance Rate (%)

Expc

Obsd

%e

Expc

Obsd

%e

Expc

Obsd

%e

Expc

Obsd

%e

NA

NA

NA

1

0

0

1

0

0

NA

NA

NA

0

1

1

100

NA

NA

NA

NA

NA

NA

NA

NA

NA

100

Other (personal hygiene) Wear proper hair restraint Wear gloves Demonstrate proper care of cuts Demonstrate proper tasting practices Practice Performance Rate (%)

Expc

Obsd

%e

Expc

Obsd

%e

Expc

Obsd

%e

Expc

Obsd

%e

3

2

67

6

5

83

5

5

100

5

4

80

84

3 NA

3 NA

100 NA

6 NA

6 NA

100 NA

5 NA

4 NA

80 NA

5 NA

5 NA

100 NA

95 NA

2

1

50

1

1

100

NA

NA

NA

NA

NA

NA

67

90

88

Meal #2 Obsd %e 6 100 2 100 2 50 1 100 NA NA NA NA 85

100

100

100

0

75

92

Meal #4 Obsd %e 3 60 1 33 3 75 2 100 1 50 NA NA 63

0

90

81 70 69 100 67 100 78

33

a Meals for which inter-rater reliability checks were performed. b Overall performance rates reflect data for each food safety practice across all meals observed. c Exp = expected number of observed practices. d Obs = actual number of observed practices. e Total observed/total expected = % f NA = not applicable (not observed).

in the Clean and Cook categories. We also added a new category (Other) to reflect personal hygiene practices that were not reflected in the four Fight Bac! practices. There was also a need to add more space on the observation form to better accommodate note taking. Two observers (Carbone and Scarpati) independently observed food safe skills during meal preparation opportunities. Inter-rater reliability rates were calculated for two meals (#1 and #3). For meal #1, inter-rater reliability rates were as follows: Clean-90% agreement; Chill -100% agreement; Separate-N/A; Cook-100% agreement; and Other-90% agreement. For meal #3, overall agreements were: Clean-95%; Chill-100%; Separate-100%; Cook-100%; and Other-90%. Reliabilities were relatively easy to establish given the clear and well defined safe food handling behaviors, the discrete nature of their occurrence, and the moderate to low rates at which they occurred. Therefore, reliability checks were only taken for two meals. All behaviors were equally weighted and the benchmark for adequate performance was set at 100%. General observation data for each food safety practice demonstrated during the pilot study are shown in Table 2. These data suggest that cooking and chilling foods, and cleaning practices had the lowest overall performance

10 Journal of Food Science Education r Vol. 12, 2013

rates. Specific food safety practices with the poorest performance rates were cleaning spills and work surfaces, and pre-washing vegetables/fruits (Clean), keeping foods that need to be refrigerated at the appropriate temperature and chilling hot foods properly (Chill). Proper tasting practices were also inconsistently demonstrated (Other). Overall, properly separating foods and utensils and personal hygiene practices were more consistently demonstrated. An example of more complete observation data from meal #1, with specific comments and suggested corrective actions, are provided in in the case study example Figure 2. The assessment technique and instrument described here provides educators working with SWD with a useful way to collect information about student performance and effective instruction. This information can help reveal inconsistencies in instructional techniques that may not otherwise present themselves in traditional paper and pencil evaluations. In the context of current educational reform and its emphasis on outcomes, accurate measures of student performance are critical to making informed decisions about students, teachers, and schools (US Dept. of Education No Child Left Behind Act 2001). For SWD in particular, performance outcomes are inherent in their IEPs and reflect the effectiveness of the curriculum. Useful and accurate performance data


Food safety practices assessment tool. . .

On Monday, July 16th, the following lunch menu is planned for the High School in Middletown, USA. o Egg salad sandwich with lettuce, tomato and mayonnaise o Celery with Ranch dip o Cole slaw with carrots o Canned pear halves o Brownie with peanut butter chips

It’s 8:30 a.m. and Peter, the food service manager, is in the kitchen with three of his staff: Hannah and Bob, the two salad prep cooks, and Roman, the line cook. Because it’s so hot outside, a cold entrée is planned for lunch. Hannah and Bob get busy preparing the vegetables and fruits. They need to clean and sanitize the work surface areas, then wash the celery, carrots, lettuce and tomatoes, slice the tomato, chop the celery, and shred the cabbage. Meanwhile, Roman is peeling and chopping eggs that he hard boiled the day before for the egg salad. He has already checked to make sure the yolks and whites are firm; next he will add the mayonnaise, some dry mustard and seasonings to make the egg salad. Peter is overseeing things and cutting the brownies for dessert. Later he will portion out the pear halves from the cans in the storage area. With a menu like this, basic food safety practices of proper cleaning, chilling and sanitation practices can be observed. It is expected that all food service workers will properly wash their hands prior to starting their work in the kitchen. They should all be wearing a hair restraint and gloves. Hannah and Bob are working with fresh produce, all of which should be washed prior to prep work. After the tomatoes and celery are cut, the cutting board and knives should be cleaned,sanitized and properly stored. The cabbage will be shredded using a food processor, which needs to be put into the dishwasher after use. The mayonnaise for the cole slaw and egg salad should remain in the refrigerator until needed. After the cole slaw is made, it should be covered and stored in the refrigerator. When the eggs are peeled and mixed with the dressing and flavorings, the egg salad should also be covered and stored in the refrigerator until it’s ready to be served. The cans of pears need to be cleaned, opened and portioned out right before service. The Ranch dressing dip should be refrigerated after opening and remain refrigerated until right before serving. Figure 2–Meal #1 case study example


Vol. 12, 2013 r Journal of Food Science Education 11


OBSERVATION FORM FOR STUDENT-BASED FOOD SAFETY PRACTICES MEAL #1 EXAMPLE Date

July 16, 2012

Start Time

8:30 AM

Observer E.Carbone, S. Scarpati End Time

9:30 AM

Initials of Students being Observed: HC, RW, RO a

Number of Students Observed 3 Location: Kitchen

Assigned Role/s of each Student: 2 salad prep cooks, 1 line cook Menu (to be completed prior to observation): Egg salad sandwich with lettuce, tomato and mayonnaise Celery with Ranch dip Cole slaw with carrots Canned pear halves Brownie with peanut butter chips Record one tally mark for every occurrence of each practice observed. Observable Check if Tally Total (observed)/ Comments Practice Practice (observed) Total (expected) Observed =%

Food Safety Category

CLEAN

•

wash hands properly - before working - after handling foods

√

1,1,1

3/4 = 75%

•

clean prep work surfaces

√

1,1

2/2 = 100%

•

pre-wash fresh fruits/ vegetables

√

1,1,1

3/4 = 75%%

•

wash cutting board and utensils

√

1

1/1 = 100%

•

clean spills

•

other( rinsed off lids of pear cans)

1

1/1 = 100%

N/A

Corrective Actions

One student did not wash his hands after taking a call on his cell phone.

Follow up with student regarding use of cell phones in the kitchen area.

Celery was not washed prior to chopping it.

Remind students of the need to wash all fresh produce.

b

√

Overall Performance Rate: 10 (total observed) /12(total expected) Overall performance rate = 83%

Figure 2–Continued




CHILL

•

keep all foods that need refrigeration (cheese, milk, eggs, deli meats, etc) at the appropriate temperature

•

check freezer temperature

•

chill hot foods properly

√

1

Mayonnaise was left out on the counter.

1/2 = 50%

N/A

N/A

Overall Performance Rate: 1(total observed) /2 (total expected) Overall performance rate = 50%

SEPARATE

•

use a separate cutting board for raw meat or properly wash the board between uses

N/A

•

use separate utensils for meat/fish

N/A

•

demonstrate proper storage of meat/fish

N/A

Overall Performance Rate: N/A

COOK

•

use thermometer to check internal cooking temperature

•

other (cooked eggs until yolks/whites are firm)

N/A

√

1

1/1 = 100%

Figure 2–Continued




Overall Performance Rate: 1(total observed) /1 (total expected) Overall performance rate = 100%

OTHER (Personal hygiene practices)

•

wear proper hair restraint

√

1,1

2/3 = 67%

•

wear gloves

√

1,1,1

3/3 = 100%

•

demonstrate proper care of cuts

N/A

•

demonstrate proper tasting practices

√

1

1/2 = 50%

One student was not wearing proper hair restraint.

Remind student of the need to wear hair restraint.

One student used the same spoon twice to taste the egg salad.

Review proper tasting practices.

Overall Performance Rate: 6 (total observed) /8(total expected) Overall performance rate= 75% a

b

Students’ names and initials have been changed; N/A = Not Applicable (not observed)

Additional Comments: Review of these data indicates that certain food safety practices need additional instruction or review. Overall Performance Rates for observed food safety practices ranged from 50% for Chilling foods to 75% for Other (personal hygiene practices), to 83% for Cleaning. Instruction would therefore focus on specific practices in need of improvement, including properly cleaning vegetables, hand washing, keeping foods adequately chilled, wearing a hair restraint, and proper tasting practices. NOTE: Some practices may be assigned that may not be observable in all individuals (Ex: checking freezer or refrigerator temperatures, proper storage of foods, etc). Figure 2–Continued

generated from the Tool fits well into the outcomes sections of IEPs and it is an accurate way to monitor students’ progress while not overwhelming teachers and other practitioners with yet another “test” to give to students. Focus of this study was on the development of the tool and its feasibility of use. A limited number of meals and students were observed; therefore, generalizability of the data should be viewed with caution. However, the findings do provide information around which instruction can be built. For example, review of the pilot data suggest that properly separating cutting boards and utensils is a food safety behavior that was consistently practiced,


while the other behaviors may need to be targeted for additional instruction. The FSP Assessment Tool can reveal a wide range of observable food safety practices. Identification of problem areas can easily be made and corrective actions taken. The Tool can also be tailored to accommodate the needs of the particular facility. In addition to the small sample size, a caveat we offer is the same for any assessment that it is a sample of student performance across a specified time frame and context, and that it is not expected to measure every aspect of a specific safe food activity. Context played an important role when we designed the observation form.


Food safety practices assessment tool. . . Table 3–Observation session organization. Procedure Identify food safe behaviors Define food safe behaviors

Define events related to the target behaviors

Establish measurement strategies

Collect, analyze, graph data

Example Clean, Chill, Separate, Cook Specify what each behavior will “look like” when observed (for example, What do you see when hands are washed, or foods are separated?) Behaviors should clearly observable, measurable and specific. Identify events that might influence the defined behaviors (e.g, time of day, kitchen organization, menu instructions, teacher behavior, number of students in kitchen, etc.). Determine how to measure and record food safe behaviors (e.g, as descriptive narrative, checklist or interval recording techniques). Establish inter-rater reliability. Compile results based on measurement strategy. Prepare results to inform curriculum.

curriculum, and it uses frequent measures to determine progress and inform instruction. Teachers and classrooms will benefit from establishing high expectations while presenting meaningful and appropriate activities for all students. For more information about UDL, visit the Center for Applied Special Technology (CAST), Universal Design for Learning Research and Demonstration Center (www.cast.org).

Conclusion Finally, this assessment approach, while specifically tailored to a safe food curriculum, can be readily adapted to many performance skills that are part of any well-rounded food management training curriculum. Our intention was to present a logical, valid and reliable way of determining the effectiveness of a safe food instruction program by observing SWD demonstrate specific skills in real time across real food handling situations. With more accurate assessment data and better informed instruction, SWD are more likely to gain access to valuable employment in food service and ultimately better access to a community and productive life.

Acknowledgements

This study was funded by a grant from the USDA Integrated Research, Education and Competitive Grants Program under agency We kept in mind that observers would likely operate in a variety award #2005–51110-03275. This study was assigned Contribution of kitchens. Some will be large enough to allow for easy and fluid #5141 by the USDA at the Univ. of Rhode Island, Agricultural avenues for observers to move around while checking for safe Experiment Station. food skills, while others might be small and restrict observer(s) to specific locations while collecting data. Our pilot test location was small, with equipment (for example, ovens,) and work spaces (for References example, prep tables) not in close proximity to sinks and wash areas. Elliott, JL, Thurlow ML. 2006. Improving test performance of students with As observers, we were restricted to one general location to prevent disabilities on district and state assessments. 2nd ed. Thousand Oaks, CA: interference with the natural work flow. Despite these restraints Corwin. p. 232. we were able to collect sufficient data in a reliable manner. Haney W, Madaus G. 1989. Searching for alternatives to standardized tests: Many aspects of recording the occurrences of a skill make it Whys, whats, and whithers. Phi Delta Kappan 70:683–87. user friendly because it rarely interferes with classroom activities Hehir T. 2009. Policy foundations of universal design for learning. In: DT, Gravel JW, Schifter LA, editors. A policy reader in universal and a simple “yes” or “no” check may be all that is needed. Gordon design for learning. Cambridge, MA: Harvard Education Press. p. 35–45. Furthermore, occurrence data produces a count that can easily be Individuals with Disabilities Education Improvement Act (IDEA) of 2004. transferred to a graph or other sheet and interpreted to monitor [Accessed 2012 Feb 15]. Available from: http://idea.ed.gov. progress. A major challenge, however, to observing specific food Madaus JW, Pivarnik L, Patnoad M, Scarpati S, Richard N, Hirsch D, safety skills within the general categories described here (that is, Carbone E, Gable R. 2010. Teaching food safety skills to students with Clean, Chill, Separate, Cook) is that they are likely to be “low disabilities. TEACHING Exceptional Children, March/April;42(4):44–51. frequency behaviors.” In other words, it would be unreasonable Partnership for Food Safety Education. 2009. [Accessed 2012 Feb 15]. Available from: http://www.fightbac.org. to expect them to occur very often. For example, it is logical to Patton MQ. 2002.Qualitative research & evaluation methods. 3rd ed. assume that if washing a cutting board is not an expected skill for Thousand Oaks, CA: Sage Publications. 598 p. that observation period, then it would not be demonstrated. If Primavera LH, Allison DB, Alfonso VC. 1997. Measurement of dependent it was demonstrated, it would likely only occur once, or possibly variables. In:, Allison DB, Franklin RD, Gorman BS, editors. Design and twice during that time. The FSP Assessment Tool was designed for analysis of single-case research. Mahway, NJ: Lawrence Erlbaum Associates. observing skills as they are exhibited and the sample of individuals p. 41–92. C, Newman L, Wagner M, Cameto R., Knokey AM, Shaver D. being observed was relatively small. To maximize the accuracy of Sanford 2011. The Post-High School Outcomes of Young Adults With Disabilities the observation tool, we offer several guidelines to prepare and up to 6 Years After High School. Key Findings From the National Longitudinal Transition Study-2 (NLTS2) (NCSER 2011–3004). Menlo organize an observation session (see Table 3). Park, CA: SRI International. [Accessed 2012 Feb 15]. Available from: www.nlts2.org/reports/. Scallan E, Griffin PM, Angulo FJ, Tauxe RV, Hoekstra RM. 2011a. Universal Design and Assessment illness acquired in the United States—unspecified agents. As noted earlier, the FSP Assessment Tool was developed in ad- Foodborne Emerging Infectious Diseases [Accessed 2012 Feb 16]. Available from: herence to the guiding principles of Universal Design for Learning http://wwwnc.cdc.gov/eid/article/17/1/pdfs/p2–1101.pdf. (UDL) (Hehir 2009). Many traditional assessments, such as writ- Scallan E, Hoekstra RM, Angulo FJ, Tauxe RV, Widdowson M-A, Roy SL, ten classroom tests, are not designed in this fashion and therefore Jones JL, Griffin PM. 2011b. Foodborne illness acquired in the United have to provide accommodations (for example, more time, special States—major pathogens. Emerging Infectious Diseases [Accessed 2012 Feb 16]. Available from: http://wwwnc.cdc.gov/eid/article/17/1/pdfs/ materials, etc.) in order to allow SWD fair and equitable access p1–1101.pdf. to the test information. In contrast, the FSP Assessment Tool Singleton RA, Straits BC, Straits MM. 1993. Approaches to Social Research. is flexible and applicable to all students engaged in a safe food 2nd ed. New York: Oxford Univ. Press. 605 p.



Food safety practices assessment tool. . . Stiggins RJ. 1987. Design and development of performance assessments. Educational Measurement: Issues and Practices 6(1):33–42.

http://www.fda.gov/Food/FoodSafety/RetailFoodProtection/FoodCode/ FoodCode2009/default.htm. United States Department of Education. No Child Left Behind Act of 2001. United States Department of Agriculture Food Safety Inspection Service. [Accessed 2012 Feb 26]. Available from 2008. Be food safe. [Accessed 2012 Feb 15]. Available from: http://www2.ed.gov/nclb/landing.jhtml http://www.befoodsafe.gov. United States Department of Health and Human Services. 2009. Food Code, www.cast.org: Center for Applied Special Technology (CAST), Inc. Wakefield, MA; 2012 [Accessed 2012 Feb 26]. Available from: Recommendations of the US Public Health Service, Food and Drug www.cast.org. Administration, [Accessed 2012 Feb 15]. Available from:

