Basic Psychometric Properties of the Transfer Assessment Instrument ...

Archives of Physical Medicine and Rehabilitation journal homepage: www.archives-pmr.org Archives of Physical Medicine and Rehabilitation 2013;94:2456-64

ORIGINAL ARTICLE

Basic Psychometric Properties of the Transfer Assessment Instrument (Version 3.0) Chung-Ying Tsai, MS,a,b Laura A. Rice, PhD,c Claire Hoelmer, BS,e Michael L. Boninger, MD,a,b,d,e Alicia M. Koontz, PhDa,b,d From the aHuman Engineering Research Laboratories, Veterans Affairs Pittsburgh Healthcare System, Pittsburgh; bDepartment of Rehabilitation Science and Technology, University of Pittsburgh, Pittsburgh, PA; cDepartment of Kinesiology and Community Health, University of Illinois, Champaign, IL; and dDepartment of Physical Medicine and Rehabilitation and eDepartment of Bioengineering, University of Pittsburgh, Pittsburgh, PA.

Abstract Objectives: To refine the Transfer Assessment Instrument (TAI 2.0), develop a training program for the TAI, and analyze the basic psychometric properties of the TAI 3.0, including reliability, standard error of measurement (SEM), minimal detectable change (MDC), and construct validity. Design: Repeated measures. Setting: A winter sports clinic for disabled veterans. Participants: Wheelchair users (NZ41) who perform sitting-pivot or standing-pivot transfers. Intervention: Not applicable. Main Outcome Measures: TAI version 3.0, intraclass correlation coefficients, SEMs, and MDCs for reliable measurement of raters’ responses. Spearman correlation coefficient, 1-way analysis of variance, and independent t tests to evaluate construct validity. Results: TAI 3.0 had acceptable to high levels of reliability (range, .74e.88). The SEMs for part 1, part 2, and final scores ranged from .45 to .75. The MDC was 1.5 points on the 10-point scale for the final score. There were weak correlations (r range,
.13 to .25; P>.11) between TAI final scores and subjects’ characteristics (eg, sex, body mass index, age, type of disability, length of wheelchair use, grip and elbow strength, sitting balance). Conclusions: With comprehensive training, the refined TAI 3.0 yields high reliability among raters of different clinical backgrounds and experience. TAI 3.0 was unbiased toward certain physical characteristics that may influence transfer. TAI fills a void in the field by providing a quantitative measurement of transfers and a tool that can be used to detect problems and guide transfer training. Archives of Physical Medicine and Rehabilitation 2013;94:2456-64 ª 2013 by the American Congress of Rehabilitation Medicine

People with lower limb paralysis rely on their upper limbs to complete essential activities of daily living (ADL) such as transfers, manual wheelchair propulsion, and pressure-relief activities. Performance of these skills often creates excessive loads on their arms, leading to upper extremity pain and injuries.1,2 A full-time manual wheelchair user performs transfers approximately 14 to 18 times per day.3 These transfers also involve potentially injurious Supported by the VA Rehabilitation Research & Development Services Merit Review Project (grant no. B6789C), the National Science Foundation (grant no. 0849878), and the National Institute on Disability and Rehabilitation Research (grant nos. H133N060019, H133N110011). The contents of this article do not represent the views of the Department of Veterans Affairs or the United States Government. No commercial party having a direct financial interest in the results of the research supporting this article has conferred or will confer a benefit on the authors or on any organization with which the authors are associated.

shoulder positions (eg, extreme combinations of shoulder flexion, internal rotation, and abduction) and exposure to high internal joint forces.4-6 Further, the transfer process has been associated with an increased likelihood of falls that can result in serious injuries.7 Therefore, using the best transfer skills possible is critical for wheelchair users to minimize the risk of secondary injuries and maintain a high level of independence with ADL. Transfer training protocols and skills evaluation vary widely in clinics.8 No criterion standard or uniform way to train and evaluate a wheelchair user’s transfer skills exists. Transfer training and evaluation have been largely subjective rather than scientific.9 Only 1 study10 has assessed transfers in a clinical setting. Harvey et al10 designed an assessment tool used by physiotherapists to evaluate how much assistance the subject needs in 6

0003-9993/13/$36 - see front matter ª 2013 by the American Congress of Rehabilitation Medicine http://dx.doi.org/10.1016/j.apmr.2013.05.001

Transfer Assessment Instrument 3.0 fundamental mobility tasks, including supine to sit transfers, completing a horizontal transfer, completing a vertical transfer, pushing a manual wheelchair on the flat ground, pushing on ramps, and negotiating curbs. Each task is evaluated using a 6point scale, with a score of “1” indicating the subject needs total assistance, and “6” indicating the subject is independent with the task.10 The tool only assesses how much assistance with transfers is needed and does not evaluate transfer skill and quality in detail. For example, the scale cannot evaluate whether subjects positioned their wheelchair and body appropriately during the transfer. Hence, in an attempt to standardize the way clinicians evaluate and measure transfer skill, we developed the Transfer Assessment Instrument (TAI) and an educational program that teaches clinicians how to administer it. The TAI identifies harmful aspects of transfers in wheelchair users. The TAI is an objective and quantifiable measure of transfer technique.11 Items falling into 3 general subdomainsdpreparation, conservation of techniques, and exhibition of controldwere developed based on evidence-based practice guidelines,12 current knowledge of the literature,13 and best clinical practice related to transfers. The tool was designed to allow all participants to achieve high scores, regardless of the extent of disability. The TAI takes less than 10 minutes to complete and requires no specialized equipment or testing tools. McClure et al11 used the TAI version 2.0 (TAI 2.0) to evaluate wheelchair users’ transfer skills and found that the TAI 2.0 had acceptable interrater reliability (intraclass correlation coefficient [ICC]Z.64) and acceptable to good intrarater reliability for 2 of 3 raters (ICCZ.74 and .89). The other rater had poor intrarater reliability (ICCZ.35). Concurrent validity was tested by comparing TAI scores to a single global rating of transfer quality by an expert clinician untrained on the TAI. Correlation coefficients between TAI scores and the global rating scale ranged from .19 to .69, indicating that the potential for the TAI corresponded well to the expert clinician’s rating of transfer; however, the spread in rater scores suggested that TAI 2.0 needed to be further refined.11 Rater feedback on TAI 2.0 alluded to ambiguities in the interpretation of a few items and a need for a more in-depth training on how to properly administer the tool. Also, item analysis revealed that all raters exhibited poor reliability on 8 of the items. The purpose of this study was to refine TAI 2.0 based on the initial results, repeat reliability testing, and evaluate basic measurement properties of the refined version of TAI (3.0). We hypothesized that after tool refinements and development of an indepth training program, reliability and the standard error of measurement (SEM) would improve. To evaluate construct validity of the scale, we hypothesized that subject factors (age, sex, body mass index, type of disability, duration of wheelchair use, number of transfers per day, upper extremity pain, arm strength, balance) would have no bearing on the TAI score.

List of abbreviations: ADL BMI ICC MDC N/A SCI SEM TAI WUSPI

activities of daily living body mass index intraclass correlation coefficient minimal detectable change not applicable spinal cord injury standard error of measurement Transfer Assessment Instrument Wheelchair User’s Shoulder Pain Index

www.archives-pmr.org

2457

Methods Refinement of TAI 2.0 and clinician training program In order to refine TAI 2.0, the 2 items that had poor inter- and intrarater reliability shown in our previous study11 were removed based on clinician feedback and statistical results (average intrarater ICCsZ.42 and .23, respectively; average interrater ICCsZ.35 and .13, respectively). Of the remaining 27 items, 13 were reworded to clarify meaning and remove ambiguities in their interpretation. Therefore, TAI 3.0 has more succinctly worded items than TAI 2.0. Appendix 1 shows the current version of the TAI (3.0). In addition to refinements of item wording, a revised comprehensive training program was developed based on successful adult education theories.12,13 The training program included both text and video-based descriptions of each item, and vignettes. The text instruction describes in detail the meaning of each item, specific items to look for during the transfer, and how different transfer situations should be scored. Decision trees were used for complex items. Videos of transfers were used to give examples of both high- and low-scoring scenarios. In addition, general recommendations were provided including where to stand when observing different aspects of the transfer and instructions to give participants to ensure scoring accuracy. The vignettes containing pictures of different transfer postures allowed clinicians to observe a set of transfers and practice using the tool.

Participants Subjects were recruited from the 2011 National Disabled Veterans Winter Sports Clinic in Snowmass, Colorado, and signed a consent form approved by the Veterans Affairs Pittsburgh HealthCare System Institutional Review Board. Each subject met the following inclusion criteria: (1) using a wheelchair for at least 1 year, (2) able to sit upright for at least 4h/d, (3) older than 18 years, (4) use a wheelchair more than 40h/wk, and (5) English speaking. The exclusion criteria were (1) current or recent history of pressure ulcers in the last year, and (2) able to stand unsupported.

Testing protocol Before testing, 4 raters were educated on how to use TAI 3.0 using the new training program. A clinician skilled in using the TAI ensured that each rater completed the training, and provided individualized instruction on any items they may have had trouble scoring. The raters’ clinical experience ranged from 1 to 16 years. Three of the raters were licensed physical therapists, and the other was a physical medicine and rehabilitation physician. Participants’ general demographic information, transfer and wheelchair type were recorded. The Wheelchair User’s Shoulder Pain Index (WUSPI) was used to measure the participants’ shoulder pain intensity.14 The subjects’ grip strength (Jamara) in a sitting position and elbow flexor and extensor strength (MicroFET2.0b) in supine were measured on the nondominant side.15 The Modified Functional Reach Test served as a quantified measure of trunk stability.16 The strength and trunk balance measures were repeated 3 times and averaged as the subject’s final measurement. Muscle spasticity of the nondominant upper extremity was determined by passively moving the participants’

2458 limbs through the range of motion to detect the resistance, and grading it on the Modified Ashworth Scale.17 Participants were then asked to perform 4 transfers to and from a height-adjustable mat table with their habitual approaches. If they needed assistance for transfers, they were permitted to use a transfer device (eg, transfer board or lift) or ask their caregivers or one of the raters to provide assistance. While participants performed transfers, 4 raters used TAI 3.0 to score and evaluate their transfer skills. All participants were asked to return 4 to 72 hours later11,18 to perform the transfer and evaluation portion of the protocol a second time (session 2 for testing intrarater reliability).

Data analysis The TAI contains 2 parts and 3 scoresdthe part 1 score, the part 2 score, and the final score. Part 1 comprises 15 items and is scored as follows: “yes,” 1 point; “no,” 0 points; or “not applicable” (N/ A), which means a removed item. Part 1 is completed after each transfer, and item scores are averaged together (4 scores in this study) to produce a single representative item score. The part 1 score is the summation of each item’s score multiplied by 10 and then divided by the number of applicable items, ranging from 0 to 10.11 The items in part 2 are completed after all transfers have been performed. The 12 items in part 2 are scored on a Likert scale ranging from 0 (strongly disagree) to 4 (strongly agree). The part 2 score is the summation of each item’s score multiplied by 2.5 and then divided by the number of applicable items, resulting in a range of scores from 0 to 10. The final score of the TAI is the average of the part 1 and part 2 scores. Data analysis was performed using SPSS 19.0 software.c Descriptive statistics were calculated for the subjects’ demographic data, strength, trunk stability, and TAI scores. The Shapiro-Wilk and Brown-Forsythe tests were used to determine the normality and homogeneity of variance of the data. The ICCs between raters and within each rater in each item, part 1 and part 2, and final TAI scores were calculated to assess reliability. ICCs higher than .8 were considered strong, between .6 and .79 were acceptable, between .4 and .59 were moderate, and lower than .4 were weak.11,19,20 Each item’s SEM (SEM Z SD[1er]1/2, where SD is the standard deviation of the dataset, and r is the reliability coefficient) was calculated.21 The intrarater SEM for each item was averaged across all 4 raters’ SEM for each item. Minimal detectable change (MDC) based on the 95% confidence interval was also analyzed (MDC Z 1.96*21/2*SEM). To assess whether the refinements to the TAI and training program to the raters made a difference, intra- and interrater ICCs, SEMs, and MDCs for each item’s score, as well as part 1, part 2, and final scores, were compared between TAI 3.0 and TAI 2.0 by using descriptive statistics. Spearman rank correlation coefficients were used to describe the strength of the relationships between the subjects’ demographic data, strength variables, trunk stability, and the final scores of rater 1 on the TAI 3.0. Rater 1 had the highest intrarater ICCs and was chosen as a standard for comparison. The subjects’ demographic data included age, type of disability, duration of wheelchair use in years, estimated number of transfers performed per day, and the total index scores of the WUSPI. To compare disability types with the TAI score, we created 5 disability categories based on our sample (table 1): tetraplegia (nZ8), high paraplegia (T2-7; nZ7), low paraplegia (T8 to L4; nZ14),22

C-Y Tsai et al Table 1

Subjects’ demographic characteristics

Variable Sex Men Women Type of transfer Sitting pivot Standing pivot Dependent without lift Device assisted Type of disability Tetraplegia High paraplegia (T2-7) Low paraplegia (T8 to L4) Multiple sclerosis & brain injury LMN injuries & amputee BMI Underweight (BMI