FEATURE
Pressure Ulcer Prevention Algorithm Content Validation: A Mixed-methods, Quantitative Study
O
D
Lia van Rijswijk, MSN, RN, CWCN; and Janice M. Beitz, PhD, RN, CS, CNOR, CWOCN, CRNP, FAAN Abstract
N
Translating pressure ulcer prevention (PUP) evidence-based recommendations into practice remains challenging for a variety of reasons, including the perceived quality, validity, and usability of the research or the guideline itself. Following the development and face validation testing of an evidence-based PUP algorithm, additional stakeholder input and testing were needed. Using convenience sampling methods, wound care experts attending a national wound care conference and a regional wound ostomy continence nursing (WOCN) conference and/or graduates of a WOCN program were invited to participate in an Internal Review Board-approved, mixed-methods quantitative survey with qualitative components to examine algorithm content validity. After participants provided written informed consent, demographic variables were collected and participants were asked to comment on and rate the relevance and appropriateness of each of the 26 algorithm decision points/steps using standard content validation study procedures. All responses were anonymous. Descriptive summary statistics, mean relevance/appropriateness scores, and the content validity index (CVI) were calculated. Qualitative comments were transcribed and thematically analyzed. Of the 553 wound care experts invited, 79 (average age 52.9 years, SD 10.1; range 23–73) consented to participate and completed the study (a response rate of 14%). Most (67, 85%) were female, registered (49, 62%) or advanced practice (12, 15%) nurses, and had >10 years of health care experience (88, 92%). Other health disciplines included medical doctors, physical therapists, nurse practitioners, and certified nurse specialists. Almost all had received formal wound care education (75, 95%). On a Likert-type scale of 1 (not relevant/appropriate) to 4 (very relevant and appropriate), the average score for the entire algorithm/all decision points (N = 1,912) was 3.72 with an overall CVI of 0.94 (out of 1). The only decision point/step recommendation with a CVI of ≤0.70 was the recommendation to provide medical-grade sheepskin for patients at high risk for friction/shear. Many positive and substantive suggestions for minor modifications including color, flow, and algorithm orientation were received. The high overall and individual item rating scores and CVI further support the validity and appropriateness of the PUP algorithm with the addition of the minor modifications. The generic recommendations facilitate individualization, and future research should focus on construct validation testing.
T
O
U
D
PL
Index: Ostomy Wound Management 2015;61(4):48–57
IC
Keywords: validation studies, algorithms, pressure ulcer, prevention, adult
E
fforts aimed at reducing barriers to implementing evidence-based protocols of patient care have continued unabated since the Institute of Medicine’s1 (IOM) document Crossing the Quality Chasm highlighted the imperative for patients to receive care based on the best available scientific knowledge and that care should not vary illogically from clinician to clinician or from place to place. Since that time, many
E AT
Potential Conflicts of Interest: This study was supported by a research grant from Convatec Inc (Skillman, NJ) to Holy Family University School of Nursing and Allied Health Professions, Philadelphia, PA.
review and observational studies have shown implementing pressure ulcer prevention (PUP) protocols of care, including standardization of pressure ulcer (PU)-specific interventions and documentation, reduces their prevalence in acute care,2,3 long-term care,3-5 and home care populations.6 However, for a wide variety of reasons, knowledge translation of PU best practice recommendations into practice remains a challenge.2 The
Ms. van Rijswijk is a Mentor, W. Cary Edwards School of Nursing, Thomas Edison State College, Trenton, NJ; a doctoral student, Department of Nursing, West Chester University, West Chester, PA; and Clinical Editor, Ostomy Wound Management. Dr. Beitz is a Professor of Nursing, School of Nursing, Rutgers University-Camden, Camden, NJ. Please address correspondence to: Lia van Rijswijk, MSN, RN, CWCN, 210 S. Chancellor Street, Newtown, PA 18940; email:
[email protected]. 48
OSTOMY WOUND MANAGEMENT® APRIL 2015
www.o-wm.com
VALIDATING A PRESSURE ULCER PREVENTION ALGORITHM
O
D
perceived quality and usability of the research or the guideline itself is an important barrier to its implementation. Processes used for the development of guidelines have varied considerably. Most are developed by groups or panels and not evaluated or tested by stakeholders, resulting in sometimes conflicting recommendations.7 For example, in a review8 of 5 wound care guidelines developed by and/or for physicians it was observed many are difficult to evaluate by clinicians not versed in guideline appraisal and ratings for many development aspects were very low. End-user concerns about the relevance or validity of guidelines can be addressed when developed using a process more closely aligned with recent IOM guidelines by soliciting input from relevant stakeholders through a rigorous established process toward establishing validity.7 Usability is facilitated when large amounts of information can be captured in a step-by-step process or algorithm.9 Recently, a PUP algorithm was developed based on a systematic literature review and face validation by wound care experts.10 Following an on-admission risk assessment using a valid and reliable instrument, the end user is guided toward modifiable risk factors included in the Braden Scale score and interventions designed to address them (see Figure 1). Using a systematic review of published evidence from 2007 to 2013 and Strength of Recommendation Taxonomy (SORT),11 the authors developed a 1-page algorithm with 26 distinct steps/decision points. Each step/recommendation was developed with study quality ratings for all identified publications and resultant strength of recommendation. As part of the external review and validation process, a face validation was conducted among 12 wound care experts. In a process reported by Lynn12 and Waltz and Bausell,13 experts rated content for validity using a 4-point Likert scale (1 = not relevant/appropriate, 4 = very relevant/appropriate). The algorithm’s overall mean score was 3.6 (SD 0.8) with a content validity index (CVI) of 0.89 (out of a possible 1), indicating strong content validity. Qualitative comments were analyzed, and minor revisions were made to the algorithm. However, additional stakeholder input and testing were needed to determine the content validity of this algorithm. The purpose of this prospective, descriptive study was to examine the content validity of the algorithm using a larger sample size of interdisciplinary wound care experts.
Ostomy Wound Management 2015;61(4):48–57
N
T
O
Key Points • Many studies have shown pressure ulcer (PU) prevalence can be reduced by standardizing interventions and documentation, but barriers to implementing guidelines of care persist. • Following development and initial validation of an evidence-based PU prevention algorithm for adults, additional stakeholder input and testing were needed. • The authors conducted a mixed-methods content validation study involving 79 multidisciplinary wound care experts. • The calculated content validity index was high (0.94 out of 1), and minor suggestions for change were included in the final algorithm revision. • Guidelines of care should be evidence-based and tested.
U
D
wound care background; 2) able to speak, read, and write in English; and 3) willing to review the algorithm and provide input, abiding by a confidentiality expectation. Wound care background meant substantive (>5 years) wound care experience and/or formal wound care education, preferably with board certification in wound care. Participants did not receive financial compensation. Health care clinicians attending a national interdisciplinary conference, a regional conference for wound, ostomy, and continence nurses (WOCN), and/or graduates of a WOCN program were invited to participate. Data collection occurred over a period of 6 months. A total of 553 health care providers were invited to participate. Conference attendees were invited via email before the conference or with a posting or personal invitation during the conferences. Graduates of a WOCN program received an emailed invitation. If they agreed to participate, they were emailed a copy of the consent form; the algorithm and study instrument were sent via regular mail. The consent form could be returned via email to ensure confidentiality of the actual survey responses. Volunteers were asked to send the completed instruments via regular mail to the researchers within 1 week, but the mail was monitored for a period of 6 months. Ethical considerations. Following IRB approval procedures, study volunteers were asked to read a consent form and provide written informed consent. Signed consent forms were collected and the algorithm surveys were distributed or sent by mail. Participants were asked to return consent forms within 1 week. No participant identification or linkage between consent and data form was possible. All consent forms and survey responses were collected and stored in a locked file cabinet of the first author. Instrumentation. The data collection survey was comprised of a paper-pencil instrument consisting of an 18-item demographic data form, the content validation questionnaire
E AT
IC
www.o-wm.com
PL
Methods Design. A mixed-methods, quantitative survey design with qualitative components was used to obtain content validation data for the PUP algorithm. Institutional Review Board (IRB) approval was obtained from Holy Family University (Philadelphia, PA). Sample and setting. Wound care providers were invited to participate in the study using convenience sampling methods. Sample inclusion criteria were relatively broad to encourage participation by a wide range of providers. Criteria included: 1) licensed health care professional or clinical researcher with
APRIL 2015 OSTOMY WOUND MANAGEMENT®
49
FEATURE
O
D
containing 26 statements matched to each of the 26 decision points/steps of the PUP algorithm, and a final segment with 2 open-ended questions asking for overall comments about the algorithm and the research process itself. The content validation survey asked participants to review the entire algorithm and then read the statements related to each of the decision points/steps and rate their level of agreement with the relevance (appropriateness) of the item. A 4-point rating scale was used as per Lynn12 and Waltz and Bausell13: 4 = very relevant and appropriate: 3 = relevant but needs minor alteration; 2 = unable to assess relevance without revision; 1 = not relevant/ appropriate. For each statement, participants were asked to add written comments considering omission(s), suggest changes to improve clarity/succinctness, present an alternative, and provide literature references if possible. Data collection procedures. At the national meeting, participants were directed by signage to an adjacent room at a specified time. Following attention to ethical considerations and a brief oral introduction about the history of the algorithm development and purpose of the study, participants signed and returned the consent forms and received the algorithm and the survey response form. Participants reviewed the algorithm then provided validation ratings and narrative comments. All participants completed and returned the survey before leaving the room. Attendees at the WOC regional nursing conference and graduates of the WOC nursing program received the same introduction in writing in addition to the consent form, algorithm, and data collection instrument and were asked to complete and return them during the meeting or via regular mail. The data collection process took approximately 45 minutes. Time to complete the study for those who mailed the forms could not be verified, but no comments were added about excessive time. Data analysis. All variables were coded and entered into SPSS® Version 19.0 (IBM, New York, NY) for analysis. Descriptive summary statistics were calculated for all demographic variables. Mean scores and the CVI were calculated for each of the 26 individual algorithm components and the entire algorithm. CVI was calculated by grouping very relevant/relevant (ratings 3 and 4) and not relevant/unable to assess relevance (ratings 1 and 2). The proportion of items rated 3 and 4 was used to calculate the CVI; validity was indicated by a score of >0.70 (scale 0 to 1.0).14,15 Qualitative comments regarding individual decision statements/steps and overall processes were transcribed and thematically analyzed using qualitative data reduction techniques. Qualitative comments about individual decision steps were substantive, and a frequency count method was utilized to give meaning for most commonly identified themes.
N
T
O
U
D
The majority (67, 85%) of participants were female, average age 52.9 years (SD 10.1; range 23–73), and practicing in the United States (95%). Most were registered nurses (49, 62%) or advanced practice (12, 15%) nurses. Other health disciplines included medical doctors, physical therapists, nurse practitioners, and certified nurse specialists (see Table 1). The majority of participants spoke English only (61, 77%) and had received their health care education in the US (71, 90%); 70 (89%) had a baccalaureate degree or above. Almost all participants had received formal wound care education (75, 95%); >75% of nurses were board-certified in wound care. Most participants were well experienced with health care; 77 (92%) had 10 or more years’ experience. Geographically, most participants were from the northeastern US (53, 68%) practicing in both urban (30, 38%) and suburban (46, 58%) settings. Only 13 (17%) of respondents practiced in a rural area. Fifty-eight (nearly 75%) encountered more than 10 patients weekly who had or were at risk for PUs. The top 3 types of wounds most commonly managed included PUs (67, 89%), lower extremity ulcers (61, 77%), and diabetic foot ulcers (42, 53%) (see Table 1). Quantitative analysis. The calculated average item relevance/appropriateness score for the entire algorithm/all decision points (1,912 ) was 3.72, with an overall CVI of 0.94 (out of 1). The only decision point/step recommendation with a CVI of ≤0.70 was the recommendation to provide medicalgrade sheepskin for patients with activity/mobility limitations who are at high risk for friction/shear (see Table 2). Otherwise, quantitative data analysis supports the algorithm components and inherent decision processes are appropriate. Qualitative analysis. Respondents’ comments about the PUP algorithm were both positive and negative (see Table 3 and Table 4). Qualitative analysis of overall comments generated themes (see Table 3) regarding algorithm strengths such as simplicity, good use of color, necessity, and flexibility. Other potential uses for patient education, quality improvement, possibility of individualization, and aligning with other guidelines or algorithms were identified. Negative themes included complexity, confusion with color use, omission of components, support surface clarification, education issues, and problems with specifics such as directions. All participants provided written comments, necessitating a more quantitative approach to the data analysis; a decision was made to generate frequency counts by item to suggest more imperative themes (see Table 4 for items with a frequency count >5). Four items received the largest number of comments. For admission assessment and documentation, 12 study participants indicated the word usually should be removed from the “within 24 hours” statement. Timing and content of staff, caregiver, and patient education also were concerning. Twenty-nine participants questioned the recommendation regarding the use of medical-grade sheepskin. Finally, 15 participants commented that the need to obtain a dietitian consult should be included for patients with a lessthan-optimal nutritional status.
E AT
OSTOMY WOUND MANAGEMENT® APRIL 2015
IC
50
PL
Results Demographics. Of the 553 providers invited, 79 consented to participate and completed the study (a response rate of 14%). All common wound care-associated disciplines were represented.
www.o-wm.com
VALIDATING A PRESSURE ULCER PREVENTION ALGORITHM
Table 1. Study participant data (N = 79) Professional degree
Table 1. Study participant data (N = 79) n (%)
n (%) a
a
MD
6 (8)
PT
6 (8)
≥30
39 (48)
RN
49 (62)
25–29
10 (13)
NP/CNS
12 (15)
20–24
10 (13)
Other/not reported
6 (7)
15–19
10 (13)
10–14
4 (5)
5–9
3 (4)
Years of health care experience
D
Wound education 45 (57)
Wound care symposia/conferences
14 (18)
Burn wound education (Advanced Burn Life Support)
4 (5)
O
WOCN programs
Practice facility size
Wound certification type WCC Other
6 (8)
O
CWS
27 (37)
N
CWOCN
Master’s degree State of practice
19 (24)
100–199
16 (20)
≥10
58 (74)
2 (3)
7–9
6 (8)
4–6
8 (10)
4 (5) 30 (38)
1–3
2 (2)
Three most commonly managed wounds
28 (36) 12 (15)
b
Pressure ulcers
67 (89)
Lower extremity ulcers
61 (77)
Diabetic foot ulcers
42 (53)
28 (35)
Surgical wounds
25 (44)
New York
10 (13)
Traumatic wounds
12 (15)
California
5 (6)
Virginia
3 (4)
U
Pennsylvania
200–299
10 (13)
D
PhD, EdD, MD
T
Baccalaureate
27 (34)
Average number of weekly patients treated who are at risk for or who have pressure ulcer
Level of highest education Diploma/AND degree
≥300 beds
b
this recommendation is good (A-strength level of evidence) and the algorithm can be used in other countries, the recommendation was not removed but included in smaller print and with an “if available” footnote. By design, algorithms are ideal for specifying appropriate management strategies, communicating complex series of conditional statements, and helping the transfer of research into clinical practice, but they are not exhaustive.9 The qualitative comment results of this and other content validation studies16 suggest a constant tension between clinician’s need for easy-to-follow, simple directions and more details and guidance. On the one hand, study participants were pleased with the easy-to-follow steps and were interested in algorithm pocket guides, yet for a number of action steps they would have liked more details. Following a careful review of all quantitative and qualitative results, several minor algorithm modifications were made. Specifically, concerns about information flow, design, and colors centered mainly on the admission assessment regarding current or recent history of limited mobility that was preceded by “Not at risk and intact skin.” The latter was removed
E AT
IC
www.o-wm.com
Total 100% — multiple responses
PL
Discussion The history of the development of the PUP algorithm via systematic review and face validation were described in an earlier publication.10 The current study provided content validation data and an overview of strengths and areas of challenge. The rating scores (average score 3.72 out of 4) and CVI (0.94 out of 1) of the PUP algorithm for use in adults were strong, suggesting the components were appropriate to the purpose of the instrument. Only 1 practice recommendation, the recommendation to use medical-grade sheepskin for patients with activity/mobility limitations and/or high risk for friction or shear, received a low score. This recommendation also received a low appropriateness score in the earlier face validation study.10 Ironically, this was one of a handful of recommendations based on the results of several high-quality studies and an overall A strength of recommendation based on several, mostly Australian, studies. Study participants were concerned about this recommendation because most practitioners in the US are not familiar with medical-grade sheepskin and may have used synthetic sheepskins only. Because the literature-based quality of the research underlying
a
APRIL 2015 OSTOMY WOUND MANAGEMENT®
51
FEATURE
Table 2. Content validation results: individual item score averages and content validity index (CVI)a Item #/Decision statement/step
N
Average
SD
CVI
1. On admission, assess and document pressure ulcer (PU) risk using valid and reliable instrument (eg, Braden Scale) – usually within 24 hours
77
3.79
.55
0.96
79
3.81
.56
0.94
77
3.60
.69
0.90
4. Assess for current or recent history of limited mobility
77
3.64
.6
0.93
5. If not at risk and no current or recent history of limited mobility, repeat PU risk and skin assessment as per setting protocol
76
3.72
.62
0.93
6. If not at risk but with current/recent history of limited mobility, include patient in at-risk protocol
76
3.55
.81
0.88
7. If results of assessment show patient is at risk for PU and/or has actual alteration in skin integrity, assess and document activity/mobility limitations, high risk for friction/shear.
75
3.83
.53
0.96
If activity/mobility limitations, high risk for friction/shear assessed and intervention is consistent with overall goals of patient care: 7.a Use high-quality foam or support surface for bed and chair
79
3.65
.69
0.92
7.b Implement regular repositioning schedule
76
3.89
0.35
0.99
O
D
2. On admission, assess and document a complete skin assessment 3. Provide staff, caregiver, and patient education related to assessment, documentation, and interventions. Following admission assessment, if patient not at risk and with intact skin:
N
T
O
7.c Use transfer device(s)
3.78
0.5
0.96
79
3.80
0.44
0.99
7.e Maintain head of bed at 0.70 = generally valid 52
OSTOMY WOUND MANAGEMENT® APRIL 2015
www.o-wm.com
VALIDATING A PRESSURE ULCER PREVENTION ALGORITHM
Table 3. Themes related to overall algorithm content/design Subtheme
Indicator statements
Positive themes; strengths of algorithm
Simple, uncomplicated design
I found algorithm one of easier ones to follow. This algorithm is not overly busy. I found it easy to follow
Flexibility of algorithm
Given medicolegal ramifications, any pressure ulcer prevention must be flexible. This does a good job
Good use of color
I like use of color boxes – helps understanding
Necessary
On the right track, think definitely needed
Caregiver/patient reminders/education
Pocket cards. Good visual reminder
Linkage to decision tree or guidelines
The Convatec Solutions is an excellent evidence-based practice guideline. Could be linked to decision tree with appropriate surface for state of PU
Potential for individualization
Could include more detail and perhaps stages and include products as per facility
Possible to link to digital images and documentation
Could be combined with digital images and documentation to keep check on outcomes/improvements
Patient education
Handouts designed to coordinate could add to patient education program. Helps standardize educational materials
O
D
Theme
Potential related supports/uses
N
T
O
D
Tracking defaulters Too busy
Color issues
Color coding
Color key would be helpful. Add meaning of colors on page at bottom Colors confusing as to what they indicate. Red means not stop? Color doesn’t seem significant
Infection
Under assessment, include infection – raises risk of nonhealing
Shearing
Where is protection for shearing? (eg, barrier films?) Add shearing as etiology
Education issues
Timing of education
Thought you state education of patient/caregiver first, education must be an intervention after patient is identified at risk
Support surfaces
Medical-grade sheepskin
Where is the evidence for medical-grade sheepskin? Sheepskins not appropriate; money and care of them
Terminology
Use NPUAP language – Pressure redistribution
Clear directions
Admission assessment – omit “usually” leaves times of initial assessment open to interpretation
Incontinence management
Specify Foley, condom catheter, fecal management system, or rectal bag
Missing specifics
www.o-wm.com
E AT
Lacking components
Would turn algorithm so it is vertical and combine some boxes
IC
Color usage
Too busy for many patients/caregivers. Easier to follow if expanded. Graphics or flow are a little confusing
PL
Orientation
U
Complex design
Keep check on defaulters. May be a nurse, doctor, ICU or OR staff. Algorithm could get an alert and feedback
APRIL 2015 OSTOMY WOUND MANAGEMENT®
53
FEATURE
Table 4. Individual items with thematic frequencies >5 (potential N = 79) Item decision step/statement
1
On admission, assess and document pressure ulcer Eliminate “usually” within 24 hours. Include (PU) risk, using valid and reliable instruments (eg, just “within 24 hours” Should be within 8 Braden Scale) – usually within 24 hours hours
12
3
Provide staff, caregiver, and patient education related to assessment, documentation and interventions
Lack of clarity of education focus who gets it? Who gives it? Is this to use the algorithm? When is the education given?
18
Following admission assessment, if patient not at risk and with intact skin: Assess for current or recent history of limited mobility
Clarity issues. What do you mean by recent? What do you mean by limited? Is mobility the same as activity?
6
If not at risk but with current/recent history of limited mobility, include patient in at-risk protocol
Is limited mobility the only trigger? Other high-risk conditions? Lack of clarity – need more description
7
D
Item Number
4
O
6
Identified themes
Frequency count
N
If activity/mobility limitations, high-risk for friction/ shear assessed and intervention is consistent with overall goals of patient care: use high-quality foam or support surface for bed and chair
Terminology issues. What is high-quality foam?
6
7D.
Elevate heels (pillows/devices)
Terminology: Better to use “float” or “offload” heels
10
7E.
Maintain head of bed at
