Psychometric Testing and Refinement of the Support ...

Journal of Nursing Measurement. Volume 12. Number 3, Winter 2004

Psychometric Testing and Refinement of the Support Needs Inventory for Parents of Asthmatic Children Christine Toye Linda J. Kristjanson Edith Cowan University Churchlands, Australia

Mardhie E. Coleman Murdoch University Mandurah, Australia

Hendrika Maltby Edith Cowan University Churchlands, Australia, and University of Vermont Burlington, VT

Glenda Jackson Edith Cowan University Joondalup, Australia This study refined the Support Needs Inventory for Parents of Asthmatic Children (SNIPAC) (Coleman, Maltby, Kristjanson, & Robinson, 2001) to produce a more parsimonious tool to assess the importance and meet the support needs of parents of children with asthma. The original tool was completed by 145 parents of 199 children with asthma, and 74 of these also provided test-retest responses. Internal consistency reliability, construct validity, and stability over time were assessed and refinements were made. Internal consistency reliability of the revised 20-item tool ranged from .77 to .95 for the three subscales of the Parent's Priority Scale (PPS), and .92 for the full PPS. Cronbach's alphas ranged from .74 to .90 for the three subscales of the Parent's Fulfillment Scale (PFS) and was .91 for the full scale. Factor analysis results of the PPS were compatible with the tool's conceptual framework. The revised 20-item tool demonstrated adequate psychometric properties in most areas. This tool may be used for research or clinical screening without imposing undue burden on parents. Further work is required to establish the tool's stability over time. Keywords: assessment tool; asthma; parents; reliability; support; validity

A

sthma is a complex disease process involving biochemical, immune, infectious, endocrine, and psychological factors (Anderson, Anderson, & Glanze, 1994). It is an episodic, reversible, obstructive airway disease that results in smooth muscle contraction of the airways, characterized by mucous hyper-secretion, mucosal edema, cell infiltration, and epithelial desquamation (National Asthma Campaign, 1993). However, asthma is a phenomenon that is more complex than its medical definition suggests. As is © 2004 Springer Publishing Company

179

180

Toye et al.

true with many other diseases, asthma has a social dimension. There is increasing recognition among researchers and clinicians that the social and medical dimensions of the illness are linked (Nocon & Booth, 1991). Juniper and associates (1996) have reported that children with asthma are troubled not only by symptoms, but by the physical, social, educational, and emotional impairments that they experience in response to their asthma as well. The impact of this illness on parents, and the extent to which they are able to help their children respond to demands of the illness, are critical components in understanding the social dimension of this disease. Parental needs for support merit investigation because it is known that feeling well supported by others (i.e., perceiving social support to be adequate) can minimize the negative impact upon health during a stressful situation (Turner, 1992). If parental support needs are not met, the child's needs are less likely to be addressed, increasing the possibility that they will experience exacerbations of the disease process. In addition to the suffering associated with this scenario, economic and social costs to the community, accrued because of the provision of care to these children, would also increase. Before health care providers can address the support needs of parents of children with asthma, it is important that these needs be systematically assessed. A search of the literature has revealed no published instruments specifically designed for assessing either the relative importance of the support needs of this parent group or the extent to which these needs are met. Tools designed for use with different parent groups may not be appropriate for use with this population. For example, the Family Support Scale (Dunst, Jenkins, & Trivett, 1984) measures how helpful various sources of social support are to families of young children with or at risk of developmental delay. Similarly, the Support Functions Scale (Dunst & Trivette, 1988) and the Parent Needs Inventoty (Robinson & DeRosa, 1980) are tools primarily designed to assess support needs of children with a disability. Parents of disabled children tend to provide constant high levels of care and may risk social isolation because of this (Britner, Morog, Pianta, & Marvin, 2003). The scenario for parents of children with asthma is different. These parents are likely to provide a lower level of ongoing care but also need to respond to acute, and sometimes life-threatening, exacerbations of the disease. Family assessment tools for use when children are ill, rather than disabled, were reviewed in 2000 by Neabel, Fothergill-Bourbonnais, and Dunning. The nine tools examined assessed either family functioning or family needs for support, but only one addressed family needs when there are chronically ill children (i.e., the Family Needs Tool developed by Rawlins, Rawlins, and Homer in 1990). Whereas this tool can (appropriately) be used with parents of children with asthma, some items, such as those addressing bowel/bladder problems and surgery, are clearly irrelevant for this parent group. Moreover, a tool designed specifically to assess the support needs of parents of children with asthma might be expected to cover the assessment of these needs more comprehensively. The Support Needs Inventory for Parents of Asthmatic Children (SNIPAC) (Coleman et al., 2001) was developed as a tool to specifically assess the support needs of parents of children with asthma. The development of this tool is summarized in this article, prior to detailing a study conducted to further test and refine the instrument, conducted at Edith Cowan University in Westem Australia. The aims of the study were to (a) produce a more parsimonious tool that adequately covered the conceptual domain identified during earlier, qualitative work, and (b) ensure that the tool demonstrated satisfactory psychometric properties. The refined tool could then be used without burdening parents unduly, either for research in the area or for screening parents to assess requirements for a more thorough needs assessment.

Testing and Refining the SNIPAC

181

BACKGROUND AND CONCEPTUAL FRAMEWORK Coleman and colleagues (2001) initially undertook a descriptive phenomenological study to examine the experiences of parents of children with asthma and their consequent support needs. These data provided a basis for questionnaire development. Interviews with 15 families who had asthmatic children were conducted. Data analysis indicated that parents experience economic hardships, lack of support by extended family, worries about safety, and uncertainty about the future. The support needs of parents varied according to the individual; age of the child; length of time since diagnosis; type, amount, level, and pace of information provided; and support from family, friends, and the community. Six categories of support needs emerged from this early phase of the work: economic support, extended family support, health professional support, community support, educational support, and internal family support. Coleman and colleagues (2001) validated the thematic analysis through focus groups with mothers to determine if the interpretation of the interview data accurately reflected their original experiences. During these focus groups, the mothers identified additional support needs, so the analysis was revisited. Six, more inclusive, categories replaced the original ones derived only from the family interviews. The newly identified categories of support needs were for 1. support bolstering the parent's spirit; 2. informational support assisting the parent in the caregiver role; 3. knowledge of care available for the child in the community, outside the home or healthcare setting (e.g., from teachers); 4. partnering by health care professionals in the provision of health care for the child; 5. support allowing the parent to meet his or her own care needs; and 6. support allowing lifestyle planning, includingfinancialsupport. A broader overarching conceptual framework was also subsequently defined, entitled "Leaming to Be a Parent of a Child With Asthma," based upon the results of this descriptive phenomenological study. The conceptual framework described the phases of challenges to competency that parents experience as they learned to care for their asthmatic children (Maltby, Kristjanson, & Coleman, 2003). The impact of this illness on parents and the extent to which they were able to help their children respond to demands of asthma were critical components in understanding the effect of this disease on parental competency. The framework is shown in Figure 1 and demonstrates relationships among

Parenting competence challenged: "NAMING ASTHMA"

Parentmg competence uncertamty: "TAKING ON THE REALITY"

Figure 1. Leaming to be a parent of a child with asthma.

Parenting competence reclaimed: "GETTING ON WITH IT"

182

Toye et al.

three main concepts: "Parenting Competence Challenged: Naming Asthma," "Parenting Competence Uncertainty: Taking on the Reality," and "Parenting Competence Reclaimed: Getting on With It." The six categories of support needs that shaped the questionnaire were viewed as consistent with potential support needs that parents may have at any stage as they learn to become a parent of a child with asthma. These support needs may vary in priority and in degree of fulfillment. Therefore, the questionnaire was structured in two scales (the Parents' Priority Scale [PPS] and the Perceived Fulfillment Scale [PFS]) and based upon the six categories of support needs.

PROCEDURES FOR INSTRUMENT DEVELOPMENT Items related to each of the six categories designated after the focus groups were generated from the qualitative data. These items used the informants' words wherever possible to incorporate content validity into the developing instrument. Results were shared with participants who provided confirmation of the categories and the emerging conceptual framework. To ensure that the tool was clearly written, that items were clustered appropriately, and that content validity was demonstrated, a panel of experts, six parents of children with asthma, were mailed the SNIPAC (Coleman et al., 2001), instructions, and a response format. Percent agreement was used to determine if items were acceptable. A preset agreement of five of six parents was used (at least 83% agreement) as the decision rule for retaining an item. The panel members were asked to evaluate each item and indicate if it was clear. They were also asked to determine whether the items belonged together and whether each item belonged in the selected subscale set. Space was provided for the panel members to comment further. All items met the preset criterion of 83% agreement for clarity and agreement that the item belonged in the subscale set. Content validity of items was determined by asking panel members to read each item and the title of the subscale to determine whether the title of the subscale was appropriate for the set of items. Panel members also rated whether each item was a type of support need, defined as a requirement for assistance by parents in response to their efforts to care for their child with asthma, using a Yes/NoAJnsure response option with space for comment. Items all met the preset criterion of 83% acceptability. Finally, raters indicated if any items on the scale were redundant and if any were missing from the scale and should be added, again using a Yes/No/Unsure response option with space for comments. A few minor suggestions for wording improvements were mentioned, the preset criterion was met, and no evidence of redundancy was noted.

DESCRIPTION, ADMINISTRATION, AND SCORING OE THE INSTRUMENT The first draft of the SNIPAC (Coleman et al., 2001) comprised 48 items, each a statement describing a support need. The tool was formatted as two scales (i.e., two sets of response options were listed beside each item), the PPS and the PFS. The PPS measured the relative importance of each of the support needs. Responses were Not Important (scored as 1), Somewhat Important (2), Average Importance (3), Very

Testing and Refining the SNIPAC

183

Important (4), and Extremely Important (5). These ratings were explained at the beginning of the questionnaire, as a heading to the responses for the PPS. Ratings (i.e., numbers 1-5) were listed beside each item and respondents were asked to circle the appropriate number. The PFS measured the extent to which each of the needs was met. PFS responses were required to be ticks placed in blank columns headed Met (scored 3), Partly Met (2), and Unmet (1). These columns were to the right of the columns used for the PPS so that parents would read each item, assess the priority of the need first, and then indicate the extent to which it was met. The proposed subscales contained the same items for the PPS and the PFS. These subscales corresponded to the six categories of support needs identified after the focus groups: Personal Support Needs (8 items). Information Support Needs (22 items). Community Support Needs (4 items), Medical Support Needs (6 items). Self Care Support Needs (5 items), and Lifestyle Support Needs (3 items). Summed overall scores for the PPS demonstrated the importance attributed to these support needs overall. Summed PPS subscale scores demonstrated the importance attributed to the different categories of support needs. Summed scores for the PFS identified the extent to which the overall support needs relevant to parents of children with asthma were met. Summed PFS subscale scores demonstrated the extent to which the categories of needs were met. Whether used in the research or the clinical context, subscale and total scale scores would be useful overall indicators of the importance of needs and of the extent to which they were met. In both instances, individual item scores might also be examined to interpret findings.

METHODS EOR TESTING AND REFINING THE TOOL Following the receipt of ethical approval from the institutional ethics committee of the administering university for all procedures, and from the ethics committee of a children's hospital for the distribution of some questionnaires by staff of the emergency department, we conducted a survey to obtain data for assessment of the psychometric properties of the tool. Parents of children with asthma were recruited throughout Western Australia via media; notices sent to schools and swimming clubs; questionnaires distributed by community nurses, school nurses, and the emergency department nurses; and directly from swimming pools hosting the statewide swimming program for children with asthma. On each occasion, access to parents was negotiated via the official contact person. Each survey pack included a demographic data collection form, information sheets about the study, SNIPAC questionnaire (Coleman et al., 2001), and the consent form required by one of the ethics committees. To allow assessment of the stability of the instrument over time, two questionnaires were included in initial questionnaire packs with two return envelopes and a letter explaining the test-retest process. Initial exploratory analyses included the documentation of sample characteristics. The data analysis process for tool reliability and validity testing used only time 1 (Tl) data except when stability over time was assessed. Validity and reliability testing used PPS data in preference to PFS data because the PPS was theoretically more likely to elicit consistent responses (i.e., the extent to which needs are met may vary from day to day, depending upon contact with others, but the relative importance of needs is likely to be a more enduring construct). As it was desirable to obtain a parsimonious tool, refinement involved item deletions. Initial decisions to delete items took into account items with exceptionally high

184

Toye et al.

rates of missing PPS or PFS responses. Thereafter, all deletions were prompted by the findings of PPS analyses but were automatically made from both scales. Although the construct validity of the PFS was not assessed, the affect of any deletions on the conceptual domain of this scale was evaluated throughout the refinement process, with a view to ensuring that this domain remained intact.

Approaches to Reliability Assessment Internal Consistency. A baseline assessment of the internal consistency reliability of both scales was carried out at the beginning of the data analysis process. We calculated internal consistency reliability estimates using Cronbach's alpha coefficient. Internal consistency reliability assessment was also conducted (for the PPS) as a part of tool refinement, a coefficient of greater than or equal to .70 being preset as the acceptable criterion for reliability of the scale (Nunnally & Bernstein, 1994). As suggested by Carmines and Zeller (1979), the criterion set for item-to-total correlations demanded that at least 50% of retained item scores correlated with total subscale scores in the range .30 to .70. For inter-item correlations, the criterion was that scores of retained items should correlate with 50% or more of other item scores in the range .30 to .70. At the end of the refinement process, we again assessed the internal consistency reliability of both the PPS and PFS. Stability. The time interval between the completion of the two copies of the questionnaire was to be 24 hours, a brief interval to prevent contaminadon of assessment of stability of the tool by changes in the child's illness. It is essential when testing the stability of an instrument over time that the researchers select a time interval that does not allow for changes in the phenomenon during this fime interval (De Vellis, 1991; Streiner & Norman, 2000). The 24-hour time interval between test and retest time points has been used previously with this type of tool without association with a memory effect (Kilpatrick, Kristjanson, & Tataryn, 1998) and was judged to be appropriate given potential changes in parents' perceptions over a short time period. We computed correlations between time 1 and time 2 PPS and PFS data to assess the stability of the instrument over time, using the intraclass correlation coefficient (ICC) and only including items retained in the refined instrument. Nunnally and Bernstein (1994) suggest that a correlation of at least .70 may be considered to be satisfactory in such an assessment.

Approaches to Validity Assessment We needed to assess the construct validity of the SNIPAC (Coleman et al., 2001), establishing whether the categories of items designated during the construction of the tool did, in fact, demonstrate its true (subscale) structure. We also needed to re-cluster the items appropriately if an alternate structure became evident and to delete items that were not essential to the conceptual domain in the given structure. To do this, we conducted a factor analysis of the PPS using principal components analysis. Criteria for factor analysis estimates were that factors included should have eigenvalues of greater than one and be supported by results of a scree plot analysis. The scree plot would show a gradual decrease in eigenvalues before a plateau and only factors identified prior to the plateau would be considered valid (Cattell, 1966). Factor loadings should be of at least .40 with differences of at least .15 for items loading on different factors. Missing responses were not imputed for the purpose of the analysis process; rather, the reasons underlying missing responses were carefully considered as part of the decision-making process around the retention or deletion of items.

Testing and Refining the SNIPAC

185

RESULTS Sample We obtained a convenience sample of 145 parents of children with asthma at time 1. Of these parents, 74 also provided time 2 responses. Although a case-to-variable ratio of 5 to 1 is recommended, according to Aleamoni (1973, 1976), a sample size of at least 100 is sufficient to allow the required statistical preliminary examination of the properties of the instrument and should provide reasonably stable results. Parents participating in the study were predominantly mothers (n - 134, 92.4%). Seventy-three participants (50.3%) reported completing post-secondary education courses, and 41 of these (28.3% of the sample) indicated that they held baccalaureate degrees. Nine participants (6.1%) had not progressed past Grade 8 in their formal education. Parents' ages varied widely and are summarized in Table 1. Fifty-three (36.6%) parents reported having more than one child in the family with asthma. The highest number of children with asthma in any one family was four. There were 199 children with asthma referred to by their parents in the questionnaires (5 being aged older than 20 years). These children included 92 (46.2%) females and 107 (53.8%) males. Thirty-five (18.4%) of these children were said by their parents to have severe asthma, 68 (35.8%) moderate asthma, and 90 (47.4%) mild asthma. The general practitioner (GP) was considered to be the main medical provider for the asthmatic child(ren) in the family in 114 cases (78.6%). Respiratory consultants were identified as the main medical providers in 14 cases (9.7%), and pediatricians were nominated in 11 cases (7.6%). There was a wide range both in the number of years these children had suffered with asthma and in their ages (Table 1). We collected information about children's medication using the demographic data collection form. Medications used to treat asthma in children were categorized as "puffer" bronchodilators, bronchodilators administered via a nebulizer, long-acting (oral) bronchodilators, preventative medications, antihistamines, and oral steroids. Approximately 50% of the children were taking preventative medication, and almost all of them were using bronchodilators in puffer form. Five percent of the children were taking long-acting bronchodilators and 9.5% were taking oral steroids. More than 25% of the children were taking combinations of bronchodilators and preventatives, but only 2.5% were taking antihistamines (Table 2).

TABLE 1. Participants' and Children's Demographic Characteristics Characteristic Age of participant Age of child 1 with asthma Age of child 2 with asthma Age of child 3 with asthma Number of years with asthma (child 1) Number of years with asthma (child 2) Number of years with asthma (child 3)

M

SD

Range

37.26 8.78 7.57 6.51 5.68 5.41 6.23

7.00

4.45 3.96 6.93 4.11 3.89 6.59

21-57 1-25 1-22 0-21 0-25 0-22 0-21

139 143 43 7 137 43 8

186

Toye et al.

o c u

I dai

PU

p p p p p >o C3 •a ci22222 ^ o o o o --< T3

I

p p P IT) p O O —< O O

O O CN -^ O

p vd

TS

O ON

.a

ON CN 0 0 —•

IT)

O CN

O ^

vq CN

OS

lU

o

o en II

j2 2

•2

2.-S

o

ca •a o xio

1

"3

u

O X)

o X> N

-3

o ^ -

2 g o

00 Sd -S „

I (2 z §J 1(i:' ^- 0.90

111 118 128

TABLE 6. Test-Retest Correlations for the 20-Item Parents' Priority Scale Reliable Alliance Subscale

Parent Assistance Subscale

Family Role Assistance Subscale

Whole Scale

0.26*

0.86*"

0.60***

0.53***

Intraclass

Note. V < . 0 5 . ***/7