Calculating and Interpreting the p-values for Functions, Pathways and Lists in IPA Overview The p-value associated with a function or a pathway in Global Functional Analysis (GFA) and Global Canonical Pathways (GCP) is a measure of the likelihood that the association between a set of focus genes in your experiment and a given process or pathway is due to random chance. The smaller the p-value the less likely that the association is random and the more significant the association. In general P-values less than .05 indicate a statistically significant, non-random association. The p-value is calculated using the right-tailed Fisher Exact Test. In this method, the p-value for a given process annotation is calculated by considering (1) the number of focus genes that participate in that process and (2) the total number of genes that are known to be associated with that process in the selected reference set. The more focus genes involved, the more likely the association is not due to random chance, and thus the more significant the p-value. Similarly, the larger the total number of genes known to be associated with the process, the greater the likelihood that a association is due to random chance, and the p-value accordingly becomes less significant. In short, the p-value identifies statistically significant over-representation of focus genes in a given process. Over-represented functional or pathway processes are processes which have more focus genes than expected by chance („right-tailed‟). For example: Experiment #1: 5 focus genes related to hematopoesis 50 genes related to hematopoesis total in the reference set Experiment #2: 5 focus genes related to hematopoesis 10 genes related to hematopoesis total in the reference set In this case the p-value for the hematopoesis process in Experiment #2 would be more significant (i.e. smaller value) than the p-value for hematopoesis in Experiment #1because of the greater over-representation of the focus genes in the relevant set of genes related to hematopoesis.
CONFIDENTIAL
[email protected] IPA 8.5
http://www.ingenuity.com
1
Calculating the P-value There are several ways to calculate p-values, such as the binomial test, Fisher‟s exact test, etc. The Fisher‟s exact test is appropriate in this case because it works well with large result sets, but also works well for cases with a small number of expected results (less than 10), cases which are sometimes encountered when assigning functional annotations to networks. Furthermore, Fisher‟s exact test is a computationally efficient method of calculating a hypergeometric distribution. The Fisher‟s exact test starts with the 2X2 contingency table as shown below (this example is for calculating the p-value for Global Functional Analysis, the calculations for Local Functional Analysis and Global Canonical Pathways are similar):
The probability of observing and individual combination, is, according to Fisher: P= (row1! row2! col1! col2!)/ (N! n11! n12! n21! n22!) Using this formula, we can calculate a table containing all the possible combinations of n11, n12, n21, n22 with the same row and column sums. Because the row and column sums remain fixed, it turns out that the only number we can freely choose is n 11. The pvalue corresponding to a particular occurrence of n11 is the sum of all probabilities associated with overlaps (subnetwork and annotation genes) greater than or equal to n11.
CONFIDENTIAL
[email protected] IPA 8.5
http://www.ingenuity.com
2
Benjamini-Hochberg Multiple Testing Correction p-Value IPA now provides you with the option of performing a corrected p-value calculation to identify the most significant results in IPA‟s Functional, Canonical Pathway, My Pathway and List Analyses. This calculation returns corrected p-values based on the BenjaminiHochberg method of accounting for multiple testing, and enables you to control the error rate in analysis results and focus in on the most significant biological functions associated with your genes of interest. Benefit: In IPA, a multiple-testing corrected p-value can be calculated using the BenjaminiHochberg method [see Reference], and displayed for the Functions, Canonical Pathways, My Pathways and List Analyses. The Benjamini-Hochberg procedure is a widely used, computationally efficient way to allow users to control the rate of false discoveries in statistical hypothesis testing. A corrected p-value P can be interpreted as an upper bound for the expected fraction of falsely rejected null hypotheses (i.e. the hypothesis that the function/data set association is just random) among all functions with p-values smaller than P. For example, let us assume that you consider all functions with corrected p-values smaller than 0.01 as being significant. With a corrected p-value threshold of 0.01, you can expect that the fraction of false positives among the significant functions is less than 1%. Reference: Y. Benjamini and Y. Hochberg. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. Roy. Statist. Soc. Ser. B 57, 289300 (1995).
Use Case/ Recommendation to Users: The question of when to use the corrected or uncorrected p-value calculation depends on the type of question you are asking when running a Functional Analysis or Canonical Pathway Analysis. For example: If you are asking whether a *particular* function or pathway is significant in relation to your dataset, then the uncorrected p-value calculation (Fisher‟s Exact Test p-value) is appropriate. It measures how likely the observed result (the number of overlapping genes) would be if the association was just random. If this is very unlikely (i.e. the p-value is below the threshold) then the function is said to be significant.
CONFIDENTIAL
[email protected] IPA 8.5
http://www.ingenuity.com
3
If you are not asking about a specific pathway – but instead want to identify all significant functions from a relatively large set of functions or pathways, in relation to your dataset, then the corrected p-value is the more appropriate measure. That is – the Benjamini-Hochberg method is recommended if you are trying to determine the significant subset from a large set, and you don‟t have any information about significant associations ahead of time. In this case the threshold p-value gives you information about how many “false positives” (i.e. functions falsely identified as being significant) you can maximally expect among the significant functions. For example, if your p-value threshold is 0.01 and there are 100 “significant” functions with a p-value below this threshold, you can expect that at most, on average, one of them was falsely identified as being significant. The threshold p-value corresponds to the “false discovery rate” which is 1% in this example. If it happens that no pathways or functions come up as significant it means that there is no set of pathways with a false discovery rate corresponding to your threshold. For example, if you run a Functional Analysis in IPA and focus on functions with a corrected p-value of
