What is the difference between an alpha level and a p-value

What is the difference between an alpha level and a p-value? Science sets a conservative standard to meet for a researcher to claim that s/he has made a discovery of a real phenomenon. The standard is the alpha level, usually set of .05. Assuming that the null hypothesis is true, this means we may reject the null only if the observed data are so unusual that they would have occurred by chance at most 5 % of the time. The smaller the alpha, the more stringent the test (the more unlikely it is to find a statistically significant result). Once the alpha level has been set, a statistic (like r) is computed. Each statistic has an associated probability value called a p-value, or the likelihood of an observed statistic occurring due to chance, given the sampling distribution. Alpha sets the standard for how extreme the data must be before we can reject the null hypothesis. The p-value indicates how extreme the data are. We compare the p-value with the alpha to determine whether the observed data are statistically significantly different from the null hypothesis: If the p-value is less than or equal to the alpha (p< .05), then we reject the null hypothesis, and we say the result is statistically significant. If the p-value is greater than alpha (p > .05), then we fail to reject the null hypothesis, and we say that the result is statistically nonsignificant (n.s.).

EXAMPLE: In the 1980s, there was a streak of home runs in baseball. It was the decade of the home run. Why? How do we test the “show me the money” hypothesis using a bivariate approach? If they are trying to hit home runs, they are more likely to strike out because they are swinging harder. Correlation between home runs and strike outs is r = 0.70 (real data).

How do we test this hypothesis using a multivariate approach? 2007 Salary is the criterion variable. 2006 Number of home runs is one predictor variable. 2006 Batting average (very low correlation with home runs). 2006 Bat quality (moderately correlated with home runs and batting average) A simple regression is Y = a + bX. A multiple regression is simply: Y = a +b1X1 + b2X2 + …+ bnXn Ichiro Salary = a + b1Home + b2Batting + b3BatQuality X is an unique predictor score Y is criterion variable 1.

Size of the b’s reflects the relative importance of each predictor. If number of home runs is more predictive of salary than batting average, b1 should be greater than b2.

Salary = a + b1Home + b2Batting + b3BatQuality 2.

, where b1 > b2.

R2 of regression including home runs should be greater than R2 of regression including batting average.

Salary = a + b1Home + b2BatQuality R2 = 0.18 Salary = a + b1Batting + b2BatQuality R2 = 0.03 The only difference between the first and second set is whether home runs versus batting average is included. Therefore, home runs are a better predictor of salary than batting average, when controlling for bat quality.