Journal of Plastic, Reconstructive & Aesthetic Surgery (2016) 69, 290e293
LETTER FROM AMERICA
A PROM primer What is a PROM? Ask a typical American high school girl, “What is a prom”? She will tell you that it is the final social event of high school life which is what some in the UK call a “leaver’s ball”. Ask the average American hand therapist to define the acronym PROM and she will tell you that it represents passive range of motion, in contrast to AROM or active range of motion. In fact, PROM also stands for Patient Reported Outcome Measures. Patients have been reporting their outcomes for decades. When I was a fellow at Johns Hopkins, the staff patients were treated by the house staff practice without any direct faculty supervision. Indeed, the tradition was that the faculty had to request the resident’s permission to scrub on a staff, as opposed to private, patient. Needless to say, some of our results were mediocre, at best. Typically, when we presented our post-operative images, we would include the caveat, “The patient was pleased with the result.” When I was a plastic surgery resident under Ralph Millard, I quickly learned that he had a different philosophy. The resident who stated, “The patient was pleased with the result” was guaranteed to incur Millard’s wrath. For teaching purposes, he had little interest in whether or not the patient was pleased. He only cared how the resident felt about his result. But God help the resident who was self-satisfied rather than self-critical. Millard’s philosophy was on the seal that he designed for his division, “Semper investigans, nunqum perficiens”, “Always searching, never perfecting”. Even today, too many manuscripts submitted to our journals have the sentence in their conclusion, “Most patients having [our procedure] were satisfied.” However, these manuscripts have no patient satisfaction data so the conclusion is unsupported.1
Oneeoff scales There are concrete means of measuring patient satisfaction, but any measurement must be not just accurate, but also reproducible. For example, one concrete scale measures how a patient compensated a plastic surgeon beyond his fee. At one end of the scale are the extremely satisfied
patients, like Millard’s extremely wealthy, pleased and generous patients who manifested their satisfaction with their facelifts by funding a professorial chair for him and fellowships for plastic surgeons to study under him. At the other end of the scale are the angry patients who have physically assaulted or even murdered their plastic surgeons. While this concrete scale accurately measures patient satisfaction, mercifully it is not frequently reproduced.
A very brief early history of plastic PROMs More than a decade ago, British Plastic Surgery was early to the PROM when this journal published its first paper to recognize the need for PROMs.2 The following year the same academic group published its first PROM.3 Weighing in on this side of the pond was that 800-pound gorilla and arbiter of safety and effectiveness, the US Food and Drug Administration (FDA). In 2006, it decreed that, as part of its evaluation of gel-filled breast implants, PROMs would be necessary, and FDA issued its own guidance on what comprised acceptable PROMs.4 PROMs are sweeping the world of evidence-based medicine and have generated acronyms including DASH,a POSHA,b MHQ,c Breast-Q and Face-Q. New PROMs are arriving at a rate so fast and furious that databases are unable to keep current. Astute JPRAS readers know that PBOT is an acronym for a PROM that is a post-bariatric outcome tool.5 But google “PBOT” and “Acronym Finder” and you will see that the sole science/medicine definition is “Permian Basin Onshore Technology (conference)”.6
The ABC’s of PROMs Authors e do not try this at home Over six years ago a systematic review of plastic surgery PROMs found that 68% of the articles analyzed used onea b c
Disabilities of the Arm Shoulder and Hand. Patient Outcomes Of Surgery-Hand/Arm. Michigan Hand Outcomes Questionnaire.
http://dx.doi.org/10.1016/j.bjps.2015.09.021 1748-6815/ª 2015 British Association of Plastic, Reconstructive and Aesthetic Surgeons. Published by Elsevier Ltd. All rights reserved.
Letter from America off PROMs.7 Sadly, the practice continues. In a typical paper, authors analyze satisfaction with an ordinal scale from one to four representing poor, fair, good and excellent. They analyze patient satisfaction with t-tests to reject the null hypothesis. This is statistically inadvisable when done for the small cohorts found in most articles as it increases the probability of having a false positive error.8 Authors who desire to create new PROMs are cautioned to obtain guidance from experts in psychometrics and to use the latest PROM quality checklist to measure their PROM.9,10
Readers e facts about PROMs Terminology taxonomy and definitions Knowing what words to use, what the words mean and how classification of words used by psychometricians is important for readers in order to understand how PROMs effect
291 plastic surgery. These are some of the most relevant words and their meanings: 1. Instrument e A PROM is a type of instrument used for measuring what the patient is reporting. 2. Validated e Shorthand for stating that a PROM has demonstrated that it fulfills a set of quality measures. This should not be confused with validity as discussed below.11 3. Domains e A PROM’s domains include reliability, validity, and responsiveness. In 2010, an international group of psychologists, epidemiologists, statisticians and clinicians published their findings and recommendations on what were the appropriate definitions, terminology and taxonomy for PROMS.12 Their Venn diagram (Figure 1) that shows how these domains relate to each other, what are their measurement properties and what features are contained in each domain.
Figure 1 The COSMIN (COnsensus-based Standards for the selection of health Measurement Instruments) taxonomy of relationships of measurment properties; adapted from reference.12
292 4. MCID e Minimum clinically important difference. For example, although a study may have a statistically significant difference in PROM scores between two cohorts, it may not be clinically important.13 Not only may the MCID differ between one PROM and another, but also the MCID for any PROM may be different if it is used to measure outcomes of different diagnoses. For example, the MCID for carpal tunnel syndrome was different than the MCID of rheumatoid disease when both were measured with the MHQ.14
Letter from America takes time with telephone calls and even the specialist’s presence in the emergency department. All too often, I have treated patients who were seen in an emergency department weeks earlier and discharged with little more than a splint and a list of hand specialists, none of whom were contacted by the emergency department physician, which would have taken time. Faster emergency department discharge yields higher consumer satisfaction scores. However, the PROM does not measure the delay that was created in the patients’ receiving definitive care. If patients are to be considered consumers, then the advice for any economic decision-making applies d caveat emptor.
Avoid over PROMising
Competing interests 1. There is no perfect PROM. A systematic review suggested that none of the most commonly used hand PROMS adequately met quality ratings for reliability, validity and responsiveness.15 2. There is no “one size fits all” PROM. Using the wrong PROM is akin to using a dermatome blade in a facelift for undermining. Just because a dermatome blade is a cutting instrument does not mean that it is the appropriate instrument. For example, while the DASH is a frequently used hand PROM, a recent clinimetric study questioned its use to measure outcomes in patients with Dupuytren’s Disease.16 3. If patients are asked to complete too many PROMs, then survey response fatigue can set in resulting in incomplete or invalid responses.17 For example, as part of a prospective cross-sectional study seeking to determine the validity of abbreviated questionnaires, the patients were requested to complete four questionnaires, the SHAI-,18 PCS-13, PHQ-9, DASH, and an ordinal pain scale. The authors had to impute missing answers, which is consistent with survey response fatigue.18 4. PROMs are measures of patient satisfaction, but they are not measures of the quality of care. The emergency medicine literature is replete with studies that reiterate this message.19
PROM perversions e the misuse of PROMs in America When the Affordable Care Act (Obamacare) was enacted, a portion of hospitals’ reimbursement from the US Government became driven by PROMs.20 Logic dictates that in order to maximize their reimbursement, hospitals would attempt to increase their consumer satisfaction rates. The telling feature of these PROMs is their name “Consumer Assessment of Healthcare Providers & Systems (CAHPS)”; note that the noun consumer rather than patient is used.21 Some hospitals contract with for-profit corporations to provide physicians to staff the emergency department. These contracts require that the physicians achieve a certain level of consumer satisfaction. Obviously, one way that emergency department physicians can improve consumer satisfaction is by decreasing the time that patients spend in the emergency department. A perverse way to decrease a patient’s time spent in the emergency department is by not contacting the on-call specialist since that
None.
Ethical approval N/A.
Funding None.
References 1. Freshwater MF. Plastic surgery spin d suppressing clarity. J Plast Reconstr Aesthet Surg 2015;68:292e4. 2. Cano SJ, Browne JP, Lamping DL. Patient-based measures of outcome in plastic surgery: current approaches and future directions. Br J Plast Surg 2003;57:1e11. 3. Cano SJ, Browne JP, Lamping DL, Roberts AHN, McGrouther DA, Black NA. The patient outcomes of surgery-Hand/Arm (POSHand/Arm): a new patient-based outcome measure. J Hand Surg Br 2004;29:477e85. 4. U.S. Food and Drug Administration. Guidance for industry. Patient reported outcome measures: use in medical product development to support labeling claims. 2006. Available at, http://www.fda.gov/downloads/Drugs/GuidanceCompliance RegulatoryInformation/Guidances/ucm071975.pdf [accessed 27.08.15]. 5. Al-Hadithy N, Welbourn R, Aditya H, Stewart K, Soldin M. A preliminary report on the development of a validated tool for measuring psychosocial outcomes for massive weight loss patients. J Plast Reconstr Aesthet Surg 2014;67:1523e31. 6. What does PBOT stand for? Acronym Finder http://www. acronymfinder.com/Science-and-Medicine/PBOT.html [accessed 09.09.15]. 7. Clapham PJ, Pushman AG, Chung KC. A systematic review of applying patient satisfaction outcomes in plastic surgery. Plast Reconstr Surg 2010;125:1826e33. 8. Freshwater MF, Theopold C, Potter S, Dempsey M, O’Shaughnessy M. A randomised controlled trial of absorbable versus non-absorbable sutures for skin closure after open carpal tunnel release. J Hand Surg Eur 2012;37:350e3. J Hand Surg Eur Vol. 2012 Sep;37(7):705; author reply 705e6. 9. Streiner DL, Norman GR, Cairney J. Health measurement Scales: a practical guide to their development and use. 5th ed. New York: Oxford University Press. 10. Mokkink LB, Terwee CB, Patrick DL, et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement
Letter from America
11.
12.
13.
14.
15.
16.
17.
instruments: an international Delphi study. Qual Life Res 2010; 19:539e49. Sinno H, Dionisopoulos T, Slavin SA, Ibrahim A, Chung KC, Lin SJ. The utility of outcome studies in plastic surgery. Plast Reconstr Surg Glob Open 2014;2:e189. Mokkink LB, Terwee CB, Patrick DL, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol 2010;63:737e45. Freshwater MF. Botulinum toxin for scars d can it work, does it work, is it worth it? J Plast Reconstr Aesthet Surg 2013;66: e92e3. Shauver MJ, Chung KC. The minimal clinically important difference of the Michigan hand outcomes questionnaire. J Hand Surg Am 2009;34:509e14. van de Ven-Stevens L, Munneke M, Terwee CB, Spauwen PH, van der Linde H. Clinimetric properties of instruments to assess activities in patients with hand injury: a systematic review of the literature. Arch Phys Med Rehabil 2009;90:151e69. Rodrigues J, Zhang W, Scammell B, et al. Validity of the disability of the Arm, Shoulder and Hand patient-reported outcome measure (DASH) and the Quickdash when used in Dupuytren’s disease. J Hand Surg Eur Vol 2015 Aug 25. pii: 1753193415601350. [Epub ahead of print]. Egleston BL, Miller SM, Meropol NJ. The impact of misclassification due to survey response fatigue on estimation and identifiability of treatment effects. Stat Med 2011;30: 3560e72.
293 18. Bot AG, Becker SJ, van Dijk CN, Ring D, Vranceanu AM. Abbreviated psychologic questionnaires are valid in patients with hand conditions. Clin Orthop Relat Res 2013;47:4037e44. 19. Farley H, Enguidanos ER, Coletti CM, et al. Patient satisfaction surveys and quality of care: an information paper. Ann Emerg Med 2014;64:351e7. Elsevier. 20. Government Printing Office. Patient Protection and affordable care act [PL 111-148] and healthcare and education reconciliation act of 2010 [PL 111-152]. US Government Printing Office Web site; March 23, 2010. Available at, http://www.gpo.gov/ fdsys/pkg/PLAW-111publ148/pdf/PLAW-111publ148.pdf [accessed 17.09.15]. 21. Consumer Assessment of Healthcare Providers & Systems (CAHPS) https://www.cms.gov/Research-Statistics-Data-andSystems/Research/CAHPS/index.html [accessed 17.09.15].
M. Felix Freshwater* Voluntary Professor of Surgery, University of Miami School of Medicine, 9155 S Dadeland Blvd, Suite 1404, Miami, FL 33156-2739, USA *Tel.: þ1 305 670 9988. E-mail address:
[email protected] 17 September 2015
