An evaluation of the sensitivity and specificity of energy expenditure ...

European Journal of Clinical Nutrition (2003) 57, 455–463 ß 2003 Nature Publishing Group All rights reserved 0954–3007/03 $25.00 www.nature.com/ejcn

ORIGINAL COMMUNICATION An evaluation of the sensitivity and specificity of energy expenditure measured by heart rate and the Goldberg cut-off for energy intake: basal metabolic rate for identifying mis-reporting of energy intake by adults and children: a retrospective analysis MBE Livingstone1*, PJ Robson1, AE Black2, WA Coward3, JMW Wallace1, MC McKinley1, JJ Strain1 and PG McKenna1 1 Northern Ireland Centre for Diet and Health, University of Ulster, Coleraine, Co Londonderry, Northern Ireland, UK; 29 Birch Close, Cambridge, UK; and 3MRC Human Nutrition Research, Elsie Widdowson Laboratory, Cambridge, UK

Objective: To identify adults and children as under- (UR), acceptable (AR), or over-reporters (OR) of energy intake (EI) using energy expenditure measured by doubly labelled water (DLW) (EEDLW), and to use this as a reference to determine the sensitivity and specificity of (i) EE measured by heart rate (EEHR), and (ii) the Goldberg cut-off technique for classifying subjects into the same categories. Design: Retrospective analysis of a dataset comprising concurrent measurements of EEDLW, EEHR, basal metabolic rate (BMR), and EI by weighed record (EIWR ) on 14 adults and 36 children. EI by diet history (EIDH) was also measured in the children only. EIWR:EEDLW provided the reference definition of subjects as UR, AR or OR. Three strategies for classifying mis-reporters based on EEHR and Goldberg cut-offs were then explored. Sensitivity and specificity were calculated respectively as the proportion of UR and non-UR correctly identified. Results: Approximately 80% of all subjects were AR. For EIWR and EIDH respectively, the sensitivity of EEHR was 0.50 and 1.00, and specificity was 0.98 and 1.00. Although designating subjects as having low, medium or high activity levels (EEHR:BMRmeas) and calculating cut-offs based on appropriate WHO physical activity level PALs did not change sensitivity, specificity dropped to 0.98 (EIWR) and 0.97 (EIDH). Cut-offs based on a PAL of 1.55 reduced sensitivity to 0.33 (EIWR) and 0.00 (EIDH), but specificity remained unchanged. The sensitivity of all cut-offs based on physical activity level (PALs) for EIWR was 0.50 (adults) and 0.25 (children). Conclusions: If the precision of EEHR was improved, it may be useful for identifying mis-reporters of EI. European Journal of Clinical Nutrition (2003) 57, 455 – 463. doi:10.1038=sj.ejcn.1601563 Keywords: adults; children; validity; heart-rate; energy; intake; dietary surveys

Introduction *Correspondence: Dr MBE Livingstone, Northern Ireland Centre for Diet and Health, University of Ulster, Coleraine, Co Londonderry, Northern Ireland, UK. E-mail: [email protected] Guarantor: Dr MBE Livingstone. Contributors: MBEL was the principal author and was responsible for executing the original adult and child cohorts. PJR participated in the conceptualisation, data analysis, drafting and critical review of the manuscript. AEB formulated the specific hypothesis and critically reviewed the manuscript. WAC was responsible for over-seeing the original doubly labelled water analysis and critically reviewed the manuscript. JMWW, McMcK, JJS and PGMcK participated in conceptualisation and helped with the drafting and critical review of the manuscript. Received 18 January 2002; revised 2 May 2002; accepted 17 June 2002

The independent validation of energy intake (EI) data by doubly labelled water (DLW) measurements of energy expenditure (EE) has conclusively demonstrated that systematic under-reporting in dietary surveys, due to under-eating and=or under-recording, is pervasive and is evident across the entire range of energy expenditures (Schoeller, 1990; Black et al, 1993; Black, 1997, 1999). However, while DLW measurements of EE have been paramount in highlighting the existence of bias in dietary surveys, the cost of the technique precludes its routine use in the screening of EI data, particularly in large epidemiological surveys. Reported EI can also be evaluated against presumed energy requirements (Goldberg et al, 1991; Black, 2000a).

Mis-reporting of energy intake MBE Livingstone et al

456 This procedure, known as the Goldberg cut-off technique, has demonstrated a widespread tendency to underestimation in large national dietary surveys from several countries as well as in many smaller surveys (Heywood et al, 1993; Fogelholm et al, 1996; Briefel et al, 1997; Lafay et al, 1997; Price et al, 1997; Pryer et al, 1997; Rothenberg et al, 1997; Voss et al, 1998). However, this technique was devised to evaluate the overall bias towards under-reporting at the group level. Although its use has been extended to identify under-reporting at the individual level, the cut-off is limited by low sensitivity, as it only identifies about 50% of underreporters (Black, 2000b). Furthermore, it can make no distinction between varying degrees of mis-reporting. If the complex nature of biased reporting is to be unravelled, and its effects minimized in the analysis of dietary data, it is vital that individuals who have provided data of poor validity are identified. To improve sensitivity in the detection of mis-reporting in dietary surveys, some attempt should be made to establish values for EE, against which reported EI can be evaluated. To date, five studies have compared reported EI directly with values for EE derived from a variety of techniques other than EEDLW (Johnson et al, 1994; ¨ rtzinger et al, 1997; Charlton & de Vries et al, 1994; Ko Lambert, 1999; Goris & Westerterp, 1999). Unfortunately, the sources of error and bias of these techniques, when used as validation tools for screening EI data, are unknown. If, as appears likely, researchers increasingly turn to these more feasible and cost-effective ways of measuring EE in dietary surveys, it is vital that their sensitivity and specificity for detecting dietary mis-reporting is established at the outset. The present study examines data from two studies in adults (Livingstone et al, 1990a, b) and children (Livingstone et al, 1992a, b) in which EE was measured concurrently by

both DLW and HR monitoring. In the same subjects, EI was measured by 7 day weighed record (WR; adults and children) and diet history (DH; children only). The present analysis uses the direct comparison of EI and DLW measurements of EE to classify individuals as acceptable, under- or overreporters (the reference definition). The sensitivity and specificity of two different strategies for placing individuals into the same categories were then explored: (i) EE measured by HR monitoring; and (ii) Goldberg cut-offs based on PALs for differing intensities of physical activity.

Subject and methods The dataset The dataset comprised individual data from four previously published studies of 14 free-living adults (Livingstone et al, 1990a, b) and 36 children aged 7, 9, 12, and 15 y (Livingstone et al, 1992a, b). The adult subjects were a sub-sample from a large randomly selected community dietary survey, and the children were recruited through schools that were selected to represent the range of socio-economic status. The age distribution and physical characteristics of the subjects are presented in Table 1. These studies were approved by the Queen’s University Ethical Committee (Livingstone et al, 1990a, b) and by the Ethical Committee of the University of Ulster (Livingstone et al, 1992a, b).

Energy intake Weighed dietary record. All subjects kept a WR for 7 consecutive days of all individual items of food and fluid consumed, together with the weights of leftovers. Parents of the 7- and 9-y-old children reported the food intakes of their

Table 1 Description of the subjects Height (m)

Weight (kg)

BMI (Kg=m2)

Age group 7y Total Males Females

n

Mean (SD)

Mean (SD)

Mean (SD)

Mean (SD)

11 6 5

7.5 (0.3) 7.5 (0.3) 7.8 (0.3)

1.23 (0.07) 1.26 (0.07) 1.20 (0.05)

24.5 (5.0) 25.4 (6.6) 23.5 (2.5)

16.0 (1.7) 15.8 (2.2) 16.3 (1.2)

9y Total Males Females

9 5 4

9.3 (0.2) 9.3 (0.2) 9.4 (0.5)

1.35 (0.07) 1.35 (0.09) 1.34 (0.05)

31.6 (7.2) 30.2 (9.4) 33.4 (3.8)

17.2 (2.4) 16.1 (2.5) 18.4 (1.0)

12 y Total Males Females

10 5 5

12.4 (0.3) 12.7 (0.3) 12.5 (0.4)

1.55 (0.08) 1.52 (0.09) 1.58 (0.08)

44.4 (5.8) 43.8 (7.3) 45.1 (4.7)

18.5 (1.6) 18.8 (2.0) 18.2 (1.1)

15 y Total Males Females

6 3 3

15.5 (0.3) 15.4 (0.4) 15.6 (0.4)

1.65 (0.12) 1.72 (0.09) 1.57 (0.11)

53.0 (9.6) 50.7 (6.4) 55.4 (13.2)

19.8 (4.4) 17.2 (2.3) 22.4 (4.7)

Adults Total Males Females

14 9 5

31.1 (6.8 29.4 (6.4) 34.2 (6.9)

1.72 (0.10) 1.78 (0.07) 1.62 (0.06)

73.5 (16.6) 81.9 (13.8) 58.3 (8.4)

24.6 (3.8) 26.0 (3.8) 22.2 (2.6)

BMI body mass index

European Journal of Clinical Nutrition

Age (y)


457 children while subjects aged 12 and 15 y took a greater responsibility for measurement of their own food intake in co-operation with their parents. Subjects were issued with dietary scales, a log book for recording foods and fluids eaten at home, or prepared at home for eating elsewhere, a pocket notebook for recording foods and fluids obtained and eaten away from home, and written instructions which included examples of completed forms. On the day before recording started, each subject was given a detailed explanation and demonstration of the cumulative weighing technique. Subjects were instructed to record brand names of foods and to provide a complete description of methods of food preparation and cooking and recipes for composite dishes. The records were used to adjust for losses during cooking. For foods and fluids eaten away from home, a description of the food, place of purchase and price were requested. Subjects were visited on a least four occasions during the weighing period and on the day after completion of weighing to monitor progress and to check log books for completeness and accuracy. The intake of metabolizable energy was calculated from food tables using a computerized data base. Diet history. The usual food intake of the children only was also assessed by DH interview conducted with the child and=or parent. Potential carryover memory effects between reported EI by diet history (EIDH) and weighed record (EIWR) were minimized by conducting the DH interviews either 2 – 4 weeks before or after the completion of the WR. To eliminate inter-interviewer bias and effects, all interviews were conducted by one investigator. Information was obtained on the usual meal and snack pattern of the subject, including place of consumption, usual foods consumed during the week and on weekends, and detailed descriptions of these foods including methods of preparation and portion size. Amounts of foods and fluids consumed were estimated by means of photographs of known portion weights of foods supplemented with the use of common household cups, glasses and dishes. School meal intakes were assessed by obtaining a typical week’s menu with weights from the school meals staff and asking the child to indicate how much of these meals they usually ate. Self-service school cafeteria meals were assessed by asking the child which foods he or she usually selected and how much they normally ate. Typical portion sizes were subsequently obtained from the supervisory staff. Energy intakes were calculated as described above. Details of each dietary methodology are fully documented in the source references (Livingstone et al, 1990b, 1992b).

Total energy expenditure Doubly labelled water method. Total EE by the DLW method was measured over 15 days in the adults and over 10 – 15 days in the children (Livingstone et al, 1990a, b, 1992a, b). The DLW technique, together with methods of calculation, validation studies and estimates of potential errors, has been fully described elsewhere (International

Dietary Energy Consultative Group, 1990; Speakman, 1997). After collection of a predose urine sample, each subject was dosed orally with 0.05 g 2H2O and 0.15 g (adults) or 0.125 g (children) H18 2 O=kg body weight. Aliquot samples of urine were collected 5 h postdose and each day at a known time for 15 days in the adults and 10 – 15 days in the children. Isotope analysis of postdose urine samples was made in duplicate, corrected for background amounts and carbon dioxide production rate calculated by using the multipoint method (Coward, 1988). The mean respiratory quotient required to estimate EE from carbon dioxide production was calculated from food quotients measured by WR in the adults and DH in the children (Black et al, 1986). Proporation of error analysis yielded an average standard error for the estimates of EEDLW of 6 2.6% (adults) and 2.8 1.0% (children). Heart-rate monitoring. In all subjects, total EE by the FlexHR method was measured over two to four separate days concurrently with the assessment of EEDLW. The procedures are described in detail in the source references (Livingstone et al, 1990a, 1992a). Briefly, the estimation of EEHR involved measurements of basal metabolic rate (BMR), resting metabolic rate (RMR), individually determined heart rate (HR) – oxygen consumption (VO2) regression lines, and minute-byminute daytime HR recordings in free-living conditions. To determine the individual HR – VO2 regression line, five calibration points were obtained by simultaneous measurement of VO2 and recording of HR under standardized conditions. Calibration points were obtained for the following activities carried out in sequence: supine, sitting quietly, standing quietly (resting activities) and two exercise activities. HR was monitored for two to four separate days in the freeliving situation with a cardiofrequency meter (Sport Tester PE3000; Polar Electro, Kempele, Finland). Daytime HR was recorded at 1 min intervals up to a maximum recording time of 16 h, at which time stored information was retrieved and the memory reprogrammed. Twenty-four-hour EE from HR was calculated as follows. A FLEX HR was calculated as the mean of the highest HR for the resting activities and the lowest HR of the exercise activities. RMR was calculated as the mean of the VO2 for the resting activities. EE for the 16 h of HR recording was determined as follows. For periods of the daytime when HR fell below FLEX HR, EE was calculated as RMR. For the remainder of the time, when HR was above FLEX HR, EE was derived from the minute-by-minute recorded HR by reference to the subject’s regression line for the VO2 corresponding to the HR. Twenty-four-hour EE was computed by summing the estimated EE by HR monitoring and EE at night. The latter was assumed to be equal to the measured BMR. Basal metabolic rate. BMR was measured (BMRmeas) by indirect calorimetry using a ventilated hood apparatus (Datex Metabolic Monitor; Datex Instrumentation CorporaEuropean Journal of Clinical Nutrition


458 tion, Helsinki, Finland). In the adult subjects, a classical BMR was measured in which subjects spent the previous night in a metabolic laboratory and BMR was measured at 1 min intervals for 30 – 45 min under standardised conditions immediately upon waking and with minimal physical disturbance. Children were brought to the laboratory early in the morning in a fasted state and allowed to rest quietly for 20 min before BMR was measured. BMR was also estimated (BMRest) for each subject from the appropriate Schofield equation based on height and weight (Schofield et al, 1985).

Identifying dietary data of poor validity: reference definition Subjects were identified as acceptable reporters (AR), underreporters (UR), or over-reporters (OR) from their ratio of EIWR:EEDLW (adults and children) and EIDH:EEDLW (children only), according to whether the individual’s EI:EE ratio was within, below or above the 95% confidence limits of agreement between the two measurements. The 95% confidence limits of agreement between EIWR or EIDH and EEDLW were calculated as 95%CL ¼ 2

ððCV2wEI =dÞ

2

þ ðCVwEE Þ Þ

where d is the number of days of diet assessment and CVwEI and CVwEE are the pooled mean coefficients of variation in EI (by WR or DH) and EEDLW, respectively. The CVwEI in EIWR for the adults, for the children and for the total group was 27, 23 and 24%, respectively. For the purpose of this analysis, the EI data measured by DH were treated as 7 day records and assumed to have a pooled mean CVwEI of 23% (Bingham, 1987; Nelson et al, 1989). The coefficient of variation (CVwEE) for EEDLW was taken as 8.2% (Black & Cole, 2000). This equation defined AR by WR as having an EIWR:EEDLW ratio within the range 0.74 – 1.26 (adults), 0.76 – 1.24 (children) and 0.75 – 1.25 (total group). UR and OR were defined as having an EIWR:EEDLW ratio less than or greater than the minimum and maximum values of the appropriate ranges, respectively. AR by DH (children only) were defined as having an EIDH:EEDLW ratio in the range 0.76 – 1.24, UR as < 0.76 and OR as > 1.24. Sensitivity and specificity The classification into ar, ur and or, according to the ratios defined above, provided the baseline or reference definition. Subjects were then classified as acceptable reporters (ar), under-reporters (ur), and over-reporters (or) according to three strategies described below. 1

2

Cut-offs for ar, ur and or were calculated from the 95% confidence limits of agreement between EIWR or EIDH and EEHR. Subjects were classified according to their EI:BMRest and EI:BMRmeas ratio and the upper and lower cut-offs were calculated by the Goldberg equation for n ¼ 1, assuming an energy requirement for a sedentary lifestyle of


3

1.55BMR. The principles of the Goldberg cut-off and the statistical derivation of the equation have been described in detail elsewhere (Goldberg et al, 1991; Black, 2000a, b). The revised factors to be used in the equation proposed by Black (2000a, b) were applied in calculating the cut-offs. Each subject’s value for EEHR was expressed as the PAL (EEHR:BMRmeas) and subjects were allocated to low, medium and high levels of activity as defined by the appropriate age – sex physical activity level (PAL) values from the WHO recommended energy requirements (FAO=WHO=UNU, 1985). Upper and lower cut-offs were calculated for these individual PAL values using the Goldberg equation for n ¼ 1.

The sensitivity of each of these three strategies for detecting UR was calculated as the proportion of UR correctly identified, while the specificity was calculated as the proportion of non-UR correctly identified. The percentage of all subjects mis-classified by each strategy was also calculated.

Results The plots of EIWR (adults and children) and EIDH (children only) against EEDLW are shown in Figures 1 and 2. The dotted lines which indicate the 95% confidence limits of agreement for EIWR:EEDLW ( 25%) and EIDH:EEDLW ( 24%) provide the reference definition of AR, UR and OR by the respective dietary assessment methods. Approximately 80% of both adults and children provided acceptable reports of their EIWR, while under-reporting (14% adults; 11% children) was more prevalent than over-reporting. Comparison of mis-reporting by WR and DH (children only) shows that over-reporting was more prevalent by DH (17% DH vs 6% WR), while under-reporting was more prevalent by WR (11% WR vs 3% DH) (Table 2). Table 3 shows the sensitivity and specificity of the first strategy for evaluating reported EI, namely a direct compar-

Figure 1 Energy intake assessed by seven day weighed record (WR EI) against energy expenditure measured by doubly labelled water (DLW EE) for the total group (n ¼ 50). Dotted lines indicate upper and lower 95% confidence limits of agreement between energy intake and energy expenditure.


459 EIWR:EEHR or EIDH:EEHR. The numbers in brackets are subjects who have been mis-classified. The major impact of the wider 95% confidence limits of agreement for EIWR:EEHR, compared with EIWR:EEDLW, was to reduce the apparent extent of under-reporting by 50% in both adults and children. However, the specificity of EEHR for identifying non-UR by WR was good, with only one adult subject and none of the children being misclassified as a ur rather than an ar. On the other hand, the sensitivity and specificity of EEHR for identifying both UR and non-UR by DH (children only) were excellent. When cut-offs based on a blanket PAL of 1.55 were applied to the EIWR data, specificity also remained high (1.00 adults; 0.97 children) for cut-offs based on BMRest and was perfect (1.00) when based on BMRmeas (Table 4). The sensitivity of this strategy in the adult subjects was similar to that of EEHR, with only 50% of subjects who under-reported by WR being correctly identified. However, sensitivity of the cut-offs was particularly poor when applied to the EI reports of the children. Only 25% of the children who under-reported by WR and none of the UR by DH were correctly identified using this strategy. In the third strategy, the subjects were assigned to low, medium and high activity levels according to their own

Figure 2 Energy intake assessed by diet history (DH EI) against energy expenditure measured by doubly labelled water (DLW EE) for the children only (n ¼ 36). Dotted lines indicate upper and lower 95% confidence limits of agreement between energy intake and energy expenditure.

ison of EI with EE measured by HR monitoring. It shows the number of AR, UR and OR, as defined by EIWR:EEDLW or EIDH:EEDLW , falling into each category of acceptable, underand over-reporters (designated by, ar, ur and or) as assessed by Table 2 . . . of EIWR or

DH

and EEDLW Diet history (DH)

Weighed dietary record (WR) Children

Adults

Total

Children

Reporting category

n

%

n

%

n

%

n

%

Acceptable reporters (AR) Over-reporters (OR) Under-reporters (UR) Total

30 2 4 36

83.3 5.6 11.1 100.0

12 0 2 14

85.7 0.0 14.3 100.0

42 2 6 50

84.0 4.0 12.0 100.0

29 6 1 36

80.6 16.7 2.8 100.0

The numbers of subjects classified as acceptable-, over-, and under-reporters (AR, OR, UR) by the direct comparison of EIWR or DH and EEDLW

Table 3

The numbers of UR, AR and OR as defined by the reference of EIWR

or DH:EEDLW

and as classified by EIWR

or DH:EEHR

Reference clasisication by EI:EEDLW UR

AR

OR

Classification by EI:EEHR Lower cut-off

Upper cut-off

ur

ar

ur

ar

or

ar

or

Mis-classified (%)

Sensitivity

Specificity

EIWR:EEHR Total group (n ¼ 50) Adults (n ¼ 14) Children (n ¼ 36)

0.68 0.60 0.72

1.32 1.40 1.28

3 1 2

(3) (1) (2)

(1) (1) 0

40 11 29

(1) 0 (1)

(1) 0 (1)

1 0 1

12 14 11

0.50 0.50 0.50

0.98 0.92 1.00

EIDH:EEHR Children (n ¼ 36)

0.72

1.28

1

0

0

28

(1)

0

6

3

1.00

1.00

UR and ur, under-reporters; AR and ar, acceptable reporters; OR and or, over-reporters. EI, energy intake; EE, energy expenditure, DLW, doubly labelled water; HR, heart rate monitoring; WR, weighed record; DH, diet history. Numbers in brackets are mis-classified subjects.



460 Table 4 The numbers of UR, AR and OR as defined by the reference EIWR or DH:EEDLW and as classified by EI:BMR using a blanket PAL of 1.55 and either a measured or estimated BMR to calculate the Goldberg cut-off Reference clasisication by EI:EEDLW UR

AR

OR

Classification by EI:BMR Lower cut-off

Upper cut-off

ur

ar

ur

ar

or

ar

or

Mis-classified (%)

Sensitivity

Specificity

EIWR:BMRest Total group (n ¼ 50) Adults (n ¼ 14) Children (n ¼ 36)

1.05 1.04 1.05

2.29 2.31 2.29

2 1 1

(4) (1) (3)

(1) 0 (1)

41 12 29

0 0 0

0 0 0

2 0 2

10 7 11

0.33 0.50 0.25

0.98 1.00 0.97

EIWR:BMRmeas Total group (n ¼ 50) Adults (n ¼ 14) Children (n ¼ 36)

1.08 1.07 1.08

2.23 2.24 2.22

2 1 1

(4) (1) (3)

0 0 0

41 12 29

(1) 0 (1)

(1) 0 0

1 0 2

12 7 11

0.33 0.50 0.25

1.00 1.00 1.00

EIDH:BMRest Children (n ¼ 36)

1.05

2.28

0

(1)

0

27

(2)

0

6

8

0.00

1.00

EIDH:BMRmeas Children (n ¼ 36)

1.09

2.21

0

(1)

27

(2)

(1)

5

11

0.00

1.00

UR and ur, under-reporters; AR and ar, acceptable reporters; OR and or, over-reporters. EI, energy intake; EE, energy expenditure, DLW, doubly labelled water; BMR, basal metabolic rate; WR, weighed record; DH, diet history. Numbers in brackets are mis-classified subjects.

measured PAL (EEHR:BMRmeas). Table 5 presents the WHO PAL values for the three activity levels for each age – sex group and the boundaries selected for assigning subjects to each level (the midpoints between the WHO PAL values). Table 5 Activity levels and PAL ranges for low, medium and high activity levels for each age – sex group Subjects

PALa

Activity level

PAL rangeb

Adult males

1.55 1.78 2.10

Low Medium Heavy

< 1.665 1.665 – 1.94 > 1.94

Adult females

1.56 1.64 1.82

Low Medium Heavy

< 1.60 1.60 – 1.73 > 1.73

7-, 9- and 12-y-old males

1.54 1.75 1.96

Low Medium Heavy

< 1.645 1.645 – 1.855 > 1.855

7-, 9- and 12-y-old females

1.48 1.68 1.88

Low Medium Heavy

< 1.58 1.58 – 1.78 > 1.78

15-y-old males

1.6 1.82 2.04

Low Medium Heavy

< 1.71 1.71 – 1.93 > 1.93

15-y-old females

1.46 1.66 1.86

Low Medium Heavy

< 1.56 1.56 – 1.76 > 1.76

a

PAL (physical activity level) values taken from FAO=WHO=UNU (1985). Boundaries calculated as the mid-points of FAO=WHO=UNU (1985) values.

b


Table 6 shows the effect of applying the upper and lower cutoffs based on these PAL values to scrutinize the EI data. Only data based on BMRest are presented, as the sensitivity and specificity were not improved by calculations based on BMRmeas. Relative to the use of cut-offs based on a single measured PAL of 1.55, the application of cut-offs based on the PALs of the subjects did not alter the proportion of adults or children who were mis-classified. However, they were fully sensitive to the extent of under-reporting by WR in the adult subjects, albeit there was some loss of specificity. In contrast, while the cut-offs based on activity levels offered no improvement in sensitivity (0.25) when applied to the EIWR of the children, they were fully sensitive to correctly identifying UR by DH.

Discussion At the outset it is acknowledged that the small study sample used in this analysis precludes any definitive conclusions concerning the efficacy of EEHR as an independent validity check on mis-reporting of EI data. Unfortunately, the expense and other problems of conducting such validation studies will inevitably constrain the numbers that can be studied, and hence, the generality of the conclusions for dietary surveys. The merit of this analysis lies in drawing attention to the possible pitfalls that could arise when applying validation techniques of unknown sensitivity and specificity for detecting mis-reporting of EI. With this caveat in mind, the results of this retrospective study on randomly selected subjects demonstrated that EEHR


461 Table 6 The numbers of UR, AR and OR as defined by the gold standard EIWR or DH:EEDLW and as classified by the cut-off for EI:BMRest using values for low, medium and high activity (calculated from EEHR:BMRmeas) as the reference PAL Clasisication by EI:EEDLW UR

AR

OR

Classification by EI:BMRest ur

ar

ur

ar

or

ar

or

Mis-classified (%)

Sensitivity

Specificity

EIWR:BMRest Total group (n ¼ 50) Adults (n ¼ 14) Children (n ¼ 36)

3 2 1

(3) 0 (3)

(1) (1) 0

41 11 30

0 0 0

(1) 0 (1)

(1) 0 (1)

10 7 11

0.50 1.00 0.25

0.98 0.92 1.00

EIDH:BMRest Children (n ¼ 36)

1

0

(1)

27

(1)

(1)

(5)

8

1.00

0.97

UR and ur, under-reporters; AR and ar, acceptable reporters; OR and or, over-reporters. EI, energy intake; EE, energy expenditure, DLW, doubly labelled water; BMR, basal metabolic rate; PAL, physical activity level; WR, weighed record; DH, diet history. Numbers in brackets are mis-classified subjects.

had only half the sensitivity of EEDLW for detecting UR by WR, although its specificity remained largely uncompromised. Relative to EEDLW, EEHR was equally effective for detecting UR and non-UR by the DH method (children only). The loss of sensitivity for screening EIWR can be largely attributed to the higher within-subject coefficient of variation in the measurements of EE by HR monitoring (13.3, 17.6, 11.2% for the total group, adults and children respectively) relative to DLW, which was taken as 8.2% (Black & Cole, 2000). In the original studies (Livingstone et al, 1990a, 1992a), EEHR and EEDLW showed good agreement at the group level, but individual estimates of EEHR ranged from 722 to þ 52% of corresponding EEDLW values. The reasons for these discrepancies have been discussed in detail in the source references (Livingstone et al, 1990a, 1992a) and include limited sampling periods for the estimation of HREE, inappropriate Flex-HR definition and=or unrepresentative calibration data. In particular, it is most unlikely that 2 – 4 days of EEHR, as was measured in these subjects, would provide a representative estimate of habitual EEHR. Undoubtedly, the CVwEE would be reduced and the sensitivity of the method enhanced, if the precision of HR monitoring could be improved. Inevitably, more lengthy periods of EEHR assessment would have practical and resource implications for researchers who intend to exploit this methodology as a validation tool. However, failure to extend the assessment periods would mean that EEHR would offer little advantage over the original Goldberg equation based on a blanket PAL of 1.55. Another constraint associated with estimating EEHR is the requirement for individual calibration of HR vs VO2, necessitating appropriate metabolic facilities for measuring respiratory gas exchange. Thus, assessment of EEHR for screening EI in large dietary surveys may simply not be a viable option. However, with the proviso that representative measures of EEHR must be obtained to maximize sensitivity,

the method could prove an extremely effective validation tool in small studies where individual data have much greater influence on the results and conclusions (Black et al, 1997). An alternative application of HR monitoring, more suited to larger population samples, is to dispense with the assessment of EEHR in favour of simply defining patterns of physical activity as low, medium or high. In default, subjects in this analysis were classified into their respective activity levels based on a comparison of their measured EEHR:BMRmeas ratio with WHO PALs since this is a better measure of ‘true’ activity level as measured by HR. Overall, this strategy proved as sensitive for screening EIWR data as that based on individual measurements of EEHR and more sensitive than that based on a blanket PAL of 1.55. Nevertheless, it is apparent that cut-offs based on activity levels are of differential sensitivity, given that they identified all of the adult UR but 75% of the children who under-reported by WR went undetected. Clearly, the efficacy of this strategy hinges on being able to select a suitable PAL value for each activity level. Even then, differences in sensitivity will emerge depending on the extent to which a selected PAL at any one level is close to the true value. For example, the actual mean EEHR:BMRmeas of the adult subjects assigned to each activity level were 1.42, 1.68 and 2.30 for the men and 1.56, 1.65 and 1.78 for women while the corresponding WHO PAL values used to derive the cut-offs were 1.55, 1.78 and 2.10 (men), and 1.56, 1.64 and 1.80 (women). Fortuitously, these cut-offs proved to be fully sensitive in the adult group but the penalty was a reduced specificity. Even if subjects can be correctly classified into their respective activity levels, the application of a single cut-off to encompass the range of EE within a given activity level will vary the probability between subjects that they will be (mis)classified as a UR, AR or OR. Consequently, while the cut-offs based on the physical activity levels clearly have more merit than those European Journal of Clinical Nutrition


462 based on a blanket cut-off of 1.55, and should be used in preference, it is with the caveat that they are most unlikely to fulfill the criterion of full sensitivity as shown in this small sample of adults. The differential sensitivity of cut-offs is particularly well exemplified in the dietary reports of the children. Most attempts at identifying mis-reporting of EI by children have applied the same strategies as those used to scrutinize the EI data of adults. Intuitively this seems logical, but as the present analysis has shown, there are inherent problems in doing so. Of the strategies explored, EEHR was the most sensitive and specific indicator of mis-reporting by WR in children and it is reasonable to assume that the detection of UR would improve if due cognisance was taken of the need to obtain more representative estimates of EEHR. The use of cut-offs based on a blanket PAL of 1.55 and physical activity levels was much more problematic because application of each of these strategies was only sensitive to 25% of UR by EIWR, even though their specificity was perfect. It is perhaps not surprising that cut-offs based on a blanket PAL of 1.55 were so insensitive for evaluating the EIWR in a paediatric population. A PAL of 1.55 was originally devised to evaluate the overall bias to under-reporting in a sedentary adult population, although in theory the cut-off values for a sample size of n ¼ 1 can be used to identify UR at the individual level (Goldberg et al, 1991; Black, 2000a, b). Consequently, its application for identifying individual mis-reporters in paediatric groups cannot be advocated. Of equal concern is that application of age-sex specific cut-offs based on three levels of physical activity did not improve the sensitivity of the technique, suggesting that the WHO recommended PAL levels for these age-sex-activity groups may be inappropriate for this purpose (Torun et al, 1996). In the present state of knowledge, therefore, all cut-offs based on assumed PAL levels used for screening the EI data of children should be applied with caution and with full acknowledgement of the pitfalls involved. There is some evidence from these data to suggest that EEHR and cut-offs based on physical activity levels may be more sensitive in detecting UR by EIDH. In marked contrast, a cut-off based on a PAL of 1.55 was totally insensitive to EIDH underreporting. However, EIDH was assessed only in the children and, as this analysis shows, the issue with these data was of over-reporting, not under-reporting. Therefore, no conclusions about the efficacy of these strategies for detecting UR by DH in this, or any other group, could be justified at present. The final issue is whether sensitivity and specificity of cutoffs based on assumed PAL values would be improved if BMR was not estimated but measured directly. The data presented in Table 4 show no differences in the sensitivities based on BMRest or BMRmeas, although the detection of non-UR was marginally improved. Therefore, any advantages of BMRmeas, relative to the resource implications of making such measurements, do not justify inclusion of BMRmeas in large European Journal of Clinical Nutrition

epidemiological studies. However, BMRmeas in small-scale studies could help to avoid misclassification of subjects (Black et al, 1997). The identification of mis-reporters of EI at the individual level must underpin any attempt to assess the causes and consequences of mis-reporting in dietary surveys. This is a formidable challenge which can only be addressed by direct comparison of EI with EE, or in default, by using cut-offs for EI:BMR based on the physical activity level of individual subjects. However, as this analysis has shown, currently available techniques such as EEHR may be able to identify only gross bias. The implications are salutary. Before any technique for estimating EE comes into common use for validating EI, its sensitivity and specificity for detecting misreporters needs to be carefully evaluated, not just assumed. Otherwise, indiscriminate use of instruments of unknown sensitivity and specificity will only generate more confusion in an already confused area. Acknowledgements We thank the subjects who took part in these studies. References Bingham S (1987): The dietary assessment of individuals; methods, accuracy, new techniques and recommendations. Nutr. Abstr. Rev. 57, 705 – 742. Black AE, Prentice AM, Goldberg GR, Jebb SA, Bingham SA, Livingstone MBE and Coward WA (1993): Measurements of total energy expenditure provide insights into the validity of dietary measurements of energy intake. J. Am. Diet. Assoc. 93, 572 – 579. Black AE (1997): Under-reporting of energy intake at all levels of energy expenditure: evidence from doubly labelled water studies. Proc. Nutr. Soc. 56, 121A. Black AE (1999): Small eaters or under-reporters? In Progress in Obesity Research, ed. G Ailhaud & B Guy-Grand pp 223 – 228. Paris: John Libbey. Black AE (2000a): Critical evaluation of energy intake using the Goldberg cut-off for energy intake: basal metabolic rate. A practical guide to its calculation, use and limitations. Int. J. Obes. Relat. Metab. Disord. 24, 1119 – 1130. Black AE (2000b): The sensitivity and specificity of the Goldberg cutoff for EI:BMR for identifying diet reports of poor validity. Eur. J. Clin. Nutr. 54, 395 – 404. Black AE & Cole TJ (2000): Within- and between-subject variation in energy expenditure measured by doubly-labelled water technique: implications for validating reported dietary energy intake. Eur. J. Clin. Nutr. 54, 386 – 394. Black AE, Prentice AM and Coward WA (1986): Use of food quotients to predict respiratory quotients for the doubly-labelled water method of measuring energy expenditure. Hum. Nutr. Clin. Nutr. 40C, 381 – 391. Black AE, Bingham SA, Johansson G & Coward WA (1997): Validation of dietary intakes of protein and energy against 24 hour urinary N and DLW energy expenditure in middle aged women and retired men: comparison with validation against presumed energy requirements. Eur. J. Clin. Nutr. 51, 405 – 413. Briefel RR, Sempos CT, McDowell MA, Chien S & Alaimo K (1997): Dietary methods research in the third National Health and Nutrition Examination Survey: under-reporting of energy intake. Am. J. Clin. Nutr. 65, S1203 – S1209. Charlton KE & Lambert EV (1999): Validation of a food frequency questionnaire in older South Africans. S.A. Med. J. 89, 184 – 189. Coward WA (1988): The doubly-labelled-water (2H2 18O) method: principles and practice. Proc. Nutr. Soc. 47, 209 – 218.

Mis-reporting of energy intake MBE Livingstone et al de Vries JH, Zock PL, Mensink RP, Katan MB (1994): Underestimation of energy intake by 3-d records compared with energy intake to maintain body weight in 269 nonobese adults. Am. J. Clin. Nutr. 60, 855–860. FAO=WHO=UNU (1985): University. Report of a joint expert consultation. Energy and protein requirements. WHO Technical Report Series no. 724 Geneva: WHO. Fogelholm M, Ma¨ nnisto S, Vartiainen E & Pietinen P (1996): Determinants of energy balance and overweight in Finland 1982 and 1992. Int. J. Obes. Relat. Metab. Disord. 20, 1097 – 1104. Goldberg GR, Black AE, Jebb SA, Cole TJ, Murgatroyd PR, Coward WA & Prentice AM (1991): Critical evaluation of energy intake data using fundamental principles of energy physiology. 1. Derivation of cut-off values to identify under-recording. Eur. J. Clin. Nutr. 45, 569 – 581. Goris AHC & Westerterp KR (1999): Underreporting of habitual food intake is explained by undereating in highly motivated lean women. J. Nutr. 129, 878 – 882. Heywood P, Harvey PJW & Marks GC (1993): An evaluation of energy intake in the 1983 Australian National Dietary Survey of Adults. Eur. J. Clin. Nutr. 47, 604 – 606. International Dietary Energy Consultative Group (IDECG) (1990). The doubly-labelled water method for measuring energy expenditure: technical recommendations for human applications, ed. AM Prentice. Vienna: IAEA=IDECG NAHRES-4. Johnson RK, Goran MI & Poehlman ET (1994): Correlates of overand underreporting of energy intake in healthy older men and women. Am. J. Clin. Nutr. 59, 1286 – 1290. ¨ rtzinger I, Bierwag A, Mast M & Mu ¨ ller MJ (1997): Measurements Ko of energy intake using a 7-day dietary record and a diet history in non-obese subjects. Ann. Nutr. Metab. 41, 37 – 44. Lafay L, Basdevant A, Charles M-A, Vray M, Balkau B, Borys J-M, Eschwege E & Romon M (1997): Determinants and nature of dietary underreporting in a free-living population: the Fleurbaix Laventie Ville-Sante´ (FLVS) Study. Int. J. Obes. Relat. Metab. Disord. 27, 567 – 573. Livingstone MBE, Prentice AM, Coward WA, Ceesay SM, Strain JJ, Mckenna PG, Nevin GB, Barker ME & Hickey RJ (1990a): Simultaneous measurement of free-living energy expenditure by the doubly labeled water method and heart-rate monitoring. Am. J. Clin. Nutr. 52, 59 – 65.

Livingstone MBE, Prentice AM, Strain JJ, Coward JJ, Black AE, Barker ME, McKenna PG & Whitehead RG (1990b): Accuracy of weighed dietary records in studies of diet and health. Br. Med. J. 300, 708 – 712. Livingstone MBE, Coward WA, Prentice AM, Davies PSW, Strain JJ, McKenna PG, Mahoney CA, White JA, Stewart CM & Kerr MJ (1992a): Daily energy expenditure in free-living children: comparison of heart-rate monitoring with the doubly labeled water (2H218O) method. Am. J. Clin. Nutr. 56, 343 – 352. Livingstone MBE, Prentice AM, Coward WA, Strain JJ, Black AE, Davies PSW, Stewart CM, McKenna PG & Whitehead RG (1992b): Validation of estimates of energy intake and diet history in children and adolescents. Am. J. Clin. Nutr. 56, 29 – 35. Nelson M, Black AE, Morris JA & Cole TJ (1989): Between- and within-subject variation in nutrient intake from infancy to old age: estimating the number of days required to rank dietary intake with desired precision. Am. J. Clin. Nutr. 50, 156 – 167. Price GM, Paul AA, Cole TJ & Wadsworth MEJ (1997): Characteristics of the low-energy reporters in a longitudinal national dietary survey. Br. J. Nutr. 77, 833 – 851. Pryer J, Vrijheid M, Nichols R, Kiggins M & Elliott P (1997): Who are the ’Low energy reporters’ in the dietary and nutritional survey of British adults? Int. J. Epidemiol. 26, 146 – 154. Rothenberg E, Bosaeus I & Steen B (1997): Evaluation of energy intake estimated by a diet history in three free-living 70 year old populations in Gothenburg, Sweden. Eur. J. Clin. Nutr. 51, 60 – 66. Schoeller DA (1990): How accurate is self-reported dietary energy intake? Nutr. Rev. 48, 373 – 379. Schofield WN, Schofield C & James WPT (1985): Basal metabolic rate. Hum. Nutr. Clin. Nutr. 39C(Suppl 1), S1 – S96. Speakman JR (1997): Doubly Labelled Water: Theory and Practice. London: Chapman and Hall. Torun B, Davies PSW, Livingstone MBE, Paolisso M, Sackett, & Spurr GB (1996): Energy requirements and dietary energy recommendations for children and adolescents 1 to 18 years. Eur. J. Clin. Nutr. 50(Suppl 1), S37 – S81. Voss S, Kroke A, Klipstein-Grobusch K, Boeing H (1997): Obesity as a major determinant of underreporting in a self-administered food frequency questionnaire: results from the EPIC-Potsdam Study. Z. Ernahrungswiss. 36, 229–236.

463
