An Introduction to Propensity Score Matching

Different Methods of Controlling for Confounders in Retrospective Case/Control Studies

Aran Canes Sr. Analytic Consultant, Cigna Corporation, October 2015

Brief History of Controlling for Covariates • •

• • •

Retrospective exact matching of case/control studies has been discussed in the literature dating back to the 1940s A rigorous theoretical basis for matching was first developed in the 1970s by Cochrane and Rubin who focused attention on establishing the Average Treatment Effect (ATE) Since then many methods to control for confounders have been proposed Propensity Score Matching was introduced by Rubin and Rosenbaum in 1983 and is the most-widely used approach Prominent among other methods are Mahalanobis Metric Matching and Coarse Exact Matching

Why Do We Need To Use One Of These Methods? •

•

• •

Often we want to examine the effect of a particular treatment on a population but, for various reasons, cannot randomly sort participants into case and controls. For example, a health insurer only has access to retrospective data on the efficacy of two similar medications and cannot conduct a double blind clinical trial on its customers. The matching methods are techniques which allow one to simulate the rigor of a clinical trial without actually conducting one. Of course, these methods are not limited to health care analytics and can be used in a wide variety of settings.

Randomized Trials and Retrospective Matching are Instances of a Larger Class of Experiments • • •

•

These techniques can be thought of as a subset of a larger set of experiments which includes randomized trials Randomized trials and retrospective matching are similar in that both test the effect of a treatment on a sample population They differ in the assignment mechanism—randomized trials assign observations to case and control randomly while retrospective matching assigns observations based on confounding variables which must be discovered The key part of all successful retrospective matching is then to successfully recover the variables which led to an observation being either a case or control

Why Not Regression Analysis? •

A regression using confounders as the predictor variables along with the treatment effect can be performed along with these methods and is a good technique to validate the results • But these approaches offer several advantages compared to performing a regression: A) The dependent variable may not follow a normal distribution, precluding ordinary least squares B) Confounder terms can have interactions with one another and with the treatment effect C) Unlike regression coefficients, simple means or frequencies can succinctly summarize the effect of the treatment.

What is Similar Between These Methods? All three approaches need to follow these steps: 1) 2) 3) 4)

Identify all potential confounders that could have an effect on the decision to be in either case or control Identify which of these variables has, or is likely to have, an effect on the outcome Match observations from the case and control groups which exhibit similarities across these confounders Compute statistics on the outcome of interest on the matched case/control population

What is Different in These Methods? • The primary difference in all these methods is on how the observations are matched: 1) Propensity Score Matching (PSM) matches the observations on the result of a logistic regression which assesses the likelihood, given the covariates, of being in either the case or control population 2) Mahalanobis Metric Matching matches the observations based on the Mahalanobis distance between each case and control 3) Coarse Exact Matching aggregrates (or coarsens) the covariates to a tolerance set by the researcher. An exact match is then performed on the coarsened variables.

Real World Example • • •

•

Eliquis is one of a number of Novel Oral Anti-coagulants used to treat atrial fibrillation Historically, the medication of choice for this condition has been Warfarin but Warfarin has the potential to cause serious side effects Clinical trials have indicated that Eliquis has improved clinical outcomes to Warfarin—a reduction in the number of strokes and fewer cases of adverse bleeding The task is to try to see whether the results of the clinical trials also show up in retrospective claims data. Do we see improvements in safety and efficacy for Eliquis vis-à-vis Warfarin? Are there corresponding reductions in total medical cost?

Pre-Matched Population Characteristics Eliquis Number of Customers

Warfarin

194

518

Age

< 17 18 to 24 25 to 34 35 to 44 45 to 64 65 or Older

0% 0% 1% 3% 58% 38%

0% 0% 1% 4% 53% 41%

Region

Midwest Northeast Other South West

10% 13% 24% 44% 9%

20% 6% 30% 31% 13%

Females

24%

32%

Males

76%

68%

4.90

6.69

Gender Mean Retrospective Risk Number

P-Value

0.6791