Emergencias 2017;29:126-135
REVIEW ARTICLE
Reliability and validity of the modified Allen test: a systematic review and metanalysis Óscar Romeu-Bordas1, Sendoa Ballesteros-Peña2,3 The objective was to evaluate the reliability and validity of the modified Allen test in screening for collateral circulation deficits in the palm and for predicting distal hand ischemia. We performed a systematic review of the literature indexed in 6 databases. We developed a search strategy to locate studies comparing the Allen test to Doppler ultrasound to detect circulation deficits in the hand, studies assessing the incidence of ischemic events on arterial puncture after an abnormal Allen test result, and studies of Allen test interobserver agreement. Fourteen articles met the inclusion criteria. Nine assessed the validity of the test as a screening tool for detecting collateral circulation deficits. From data published in 3 studies that had followed comparable designs we calculated a sensitivity of 77% and specificity of 93% for the Allen test. Four studies that assessed the ability of the test to predict ischemia did not predict any ischemic hand events following arterial puncture in patients with abnormal Allen test results. A single study assessing the test’s reliability reported an interobserver agreement rate of 71.5%. This systematic review and metanalysis allows to conclude that the Allen test does not have sufficient diagnostic validity to serve as a screening tool for collateral circulation deficits in the hand. Nor is it a good predictor of hand ischemia after arterial puncture. Moreover, its reliability is limited. There is insufficient evidence to support its systematic use before arterial puncture. Keywords: Hand. Collateral circulation. Ultrasonography. Doppler. Reliability. Validity.
Validez y fiabilidad del test modificado de Allen: una revisión sistemática y metanálisis El objetivo de esta revisión se centra en evaluar la validez y fiabilidad del test modificado de Allen como prueba de cribado para déficits en la circulación colateral palmar y como predictor de isquemia distal de la mano. Para ello, se realizó una revisión sistemática de la literatura a través de 6 bases de datos. Se elaboraron protocolos de búsqueda para localizar estudios que comparasen el test de Allen con la ecografía Doppler para detectar déficits de la circulación colateral de la mano; que evaluasen la incidencia de eventos isquémicos cuando la arterioclisis se realizaba ante un test de Allen anormal y que estudiasen la concordancia interobservador del test. En total, 14 artículos cumplieron los criterios de búsqueda. Nueve estudios evaluaron la validez del test como prueba de cribado para déficits de circulación colateral. En base a 3 estudios de metodología análoga, se calculó una sensibilidad y especificidad del test del 77% y 93%, respectivamente. Cuatro estudios que valoraban la capacidad del test para predecir isquemia no objetivaron ningún evento isquémico de la mano tras arterioclisis cuando el test de Allen era anormal. Un único estudio evaluó la fiabilidad del test, determinando una concordancia interobservador del 71,5%. Esta revisión sistemática y metanálisis permite concluir que el test de Allen no presenta una adecuada validez diagnóstica como prueba de cribado de déficits de la circulación colateral de la mano, ni resulta un buen predictor de isquemia de la mano tras una punción arterial. Además, presenta una limitada fiabilidad, por lo que no existen evidencias que sustenten su realización sistemática previa a una punción arterial.
Authors affiliation: Organización Sanitaria Integrada de Barrualde-Galdakao, Bizkaia, Spain. 2 Organización Sanitaria Integrada de Bilbao-Basurto, Bizkaia, Spain. 3 Universidad del País Vasco/Euskal Herriko Unibertsitatea, Bizkaia, Spain. 1
Contribution of authors: All authors have confirmed their authorship in the author's responsibilities documents publication agreement and assignment of rights to EMERGENCIAS. Corresponding author: Sendoa Ballesteros Peña Facultad de Medicina y Enfermería. Universidad del País Vasco/Euskal Herriko Unibertsitatea (UPV/EHU) Bº Sarriena, s/n. 48940 Leioa, Bizkaia, Spain. E-mail:
[email protected] Article information: Received: 12-8-2016 Accepted: 6-10-2016 Accepted: 21-3-2017 Editor in charge: Guillermo Burillo-Putze, MD, PhD.
Palabras clave: Mano. Circulación colateral. Ultrasonografía. Doppler. Fiabilidad y validez.
Introduction Hand ischemia is an adverse effect derived from radial artery puncture in the process of obtaining a blood sample (arterial blood gas analysis) or performing a catheterization for diagnostic or therapeutic purposes. Although symptomatic arterial occlusion or permanent ischemic injury of the hand are rare phenomena (incidence of hand ischemia has been documented in less than 0.2%) 1,2, as a pre-measurement of an arterial puncture, for to assess the quality of the collateral circulation of the ulnar artery to the palmar arch and to predict the risk of ischemic complications, the Allen test is routinely performed.
126
Although there are variations in the literature on the Allen test procedure, the classic and mostly extended manoeuvre is that proposed by Bedford 3 in the 1970s (and modified from the first description by Edgar Allen), which consists of asking the patient to clench the hand in a fist and after compressing the radial artery and the ulnar at the same time to verify that the palm of the hand acquires a pale colour. Subsequently, the ulnar artery is released, while continuing to compress the radial artery. If the normal coloration of the palm is not restored within 10-15 seconds, the test is considered abnormal (positive) and constitutes a contraindication for radial artery puncture because it considers the risk of palmar ische-
Romeu-Bordas O, et al. Emergencias 2017;29:126-135
mia due to a deficit in the collateral circulation of the hand. Although the biomedical literature and current clinical practice manuals recommend systematically performing the modified Allen test (MAT) in procedures that include arterial puncture4,5, the low incidence of ischemic complications in the hand, lack of homogeneity In the criterion to establish the cut-off point of the test (that is, the time it takes to recover the colour of the hand) and the doubts based on the validity of the test as a screening test 6 have raised the debate about its relevance and degree of recommendation7. In this line of discussion, we present the objectives of this work, which focus on: 1) evaluating MAT as a screening test for deficits in the collateral circulation of the hand; 2) to evaluate the MAT as a predictor of distal ischemia of the hand, and 3) to examine the reliability of the Allen test.
Method A systematic review of the scientific literature was carried out. The following electronic databases were consulted: Medline, Scopus, Web of Science, EMBASE, Cochrane plus and CINAHL. Strategies were designed for the search in Medline (Table 1), which were later adapted for the other bibliographic databases selected. The article selection process included publications in English and Spanish, with full text access, indexed until April 2016 on observational studies, validity studies of diagnostic tests and clinical trials in which: 1) the MAT was compared without another joint technique) with Doppler ultrasonography in the evaluation of palmar collateral circulation; 2) the ischemic complications were assessed after radial artery punctures when the MAT was abnormal, and 3) the inter- and intraobserver concordance of the MAT was studied. Those studies that used only other variants other than the previously described classical MAT (for example, those using complementary electromedical instruments, such as pulse oximeters or echographs) and bibliographical reviews, clinical cases and opinion articles were excluded. Two reviewers independently and in parallel evaluated the titles and abstracts of all studies identified using the search strategy. Then, those studies that appeared to meet the specified selection criteria were read in full and evaluated for inclusion by the two independent reviewers. Disagreements between the two reviewers were resolved after discussion and when there was no consensus, external experts were consulted. Finally, an inverse search was carried out as a secondary strategy, and the bibliography of articles considered of interest was analysed. After the research and selection of studies, the following data were obtained: author and date, type of study, study/sample population, intervention performed (and characteristics of the MAT), description of the technique of gold standard if there was, main results and conclusions of the authors.
Table 1. Search strategies for each of the objectives used in Medline and exported to other databases Search for studies evaluating… Allen's test as a screening tes for deficits in the collateral circulation of the hand Allen test as predictor of distal hand ischemia Allen Test Reliability
Search strategy used in Medline ((collateral hand circulation) AND doppler) AND hand/blood supply[MeSH Terms] ((allen test[Title/Abstract]) AND safety[Title/Abstract]) AND complications[Title/Abstract]) ((allen test[Title/Abstract]) AND reliability[Title/Abstract]) AND observer [Title/Abstract]
The extraction was carried out by a reviewer and was checked for a second reviewer. When there were doubts or discrepancies, they were resolved by consensus after a combined review of the researchers. The methodological quality of the validity studies of selected diagnostic tests was assessed using the QUADAS-29 questionnaire and the rest of the studies were classified based on the criteria proposed by the Agency for the Evaluation of Medical Technology of Catalonia (AATMC)10. In parallel, in order to study the diagnostic validity of MAT as a screening test for collateral circulation deficits of the hand, a meta-analysis was proposed that included those articles in which the MAT was compared with a cut-off point of 10 seconds with a titration Doppler ultrasound (gold standard) performed on the thumb’s main artery. These criteria were determined after an assessment of the heterogeneity of the studies and the possibility of existence of threshold effect. They responded to the fact that most of the studies reviewed chose the 10 seconds as the maximum time for reperfusion of the hand after the MAT and, secondly, because it is the first finger of the hand that most frequently presents a dominant radial irrigation, and therefore, the one that should be used as an indicator8. Data on sensitivity, specificity and predictive values were extracted (or calculated based on the information provided in the articles); The 95% confidence intervals (95% CI) were estimated and the main results were synthesized by the graphical representation of forest-plot figures. In order to measure the weighted effect of all the studies, a variable effects model was chosen using the OpenMeta program.
Results The search in the six electronic databases of biomedical literature resulted in the identification of 49 studies comparing MAT with Doppler ultrasonography as a reference test to evaluate the permeability of the cubitar-palate arches. After reading the title and summary of the articles, 13 records were selected and after a critical reading of the full text, four: 2 were eliminated for not comparing the Allen test with Doppler ultrasound; one for not performing the Dop-
127
Romeu-Bordas O, et al. Emergencias 2017;29:126-135
pler ultrasonography in the whole of the sample and to perform a MAT with support of the pulse oximetry, and 1 because it is the description of a clinical case. Of the 9 articles finally included, 6 were eliminated for the accomplishment of the meta-analysis: one was excluded for not defining the criteria to consider an abnormal ultrasound result; 4 because they had a cut-off point in the MAT that differed from the previously stipulated one, and the remainder because, despite having a cut-off point of 10 seconds, the echographic technique was not performed on the main artery of the thumb (Figure 1). For the evaluation of ischemic complications by radial artery punctures with abnormal MAT, 39 studies were initially selected. After reading the titles or abstracts, 4 observational studies were selected for their relevance to the research question (Figure 2). The search strategy designed to assess the reliability of MAT managed to recover 10 studies. After reading the titles or abstracts, 2 were selected for their relevance to the topic, but after reading the full text one was discarded as a narrative revision (Figure 3). The secondary inverse search strategy did not locate
Number of records identified in searches (n = 80)
any relevant documents for any of the three research questions, so a total of 14 studies were finally completed, all in English. For a better understanding of the themes, the characteristics of the selected studies and the description of the results of the review are organized according to each of the three objectives.
Allen's modified test as screening test for deficits in the collateral circulation of the hand The methodology of the selected studies was based on the comparison of the MAT as a predictor of poor hand collateral circulation versus a non-invasive diagnostic reference test, the Doppler ultrasound. Abnormal results on Doppler ultrasonography were considered when no arterial flow was detected after manual radial artery occlusion. Six articles 11-16 were presented as diagnostic evaluation studies. The remainder were of the analytical type17-19, but in two of the cases17,19 the published information allowed for calculations of diagnostic validity (sensitivity, specificity and predictive values). A high proportion of studies with low risk of bias and concerns about applicability
Number of additional records identified in other sources (n = 0)
Total number of duplicate records deleted (n = 31) Total number of single screening records (n = 49)
Total number of records deleted (n = 36)
Total number of full-text articles analysed to decide eligibility (n = 13)
Total number of studies included (n = 9)
Total number of excluded full-text articles (n = 4)
Total number of studies included in meta-analysis (n = 3) Figure 1. Selection process of studies comparing Allen's modified test with Doppler ultrasonography in the assessment of palmar collateral circulation.
128
Romeu-Bordas O, et al. Emergencias 2017;29:126-135
Number of records identified in searches (n = 39)
Number of additional records identified in other sources (n = 0)
Total number of duplicate records deleted (n = 26) Total number of single screening records (n = 13)
Total number of full-text articles analysed to decide eligibility (n = 4)
Total number of studies included (n = 4)
Total number of records deleted (n = 9)
Total number of excluded full-text articles (n = 0)
Figure 2. Selection process of studies evaluating ischemic complications after radial artery punctures when Allen's modified test was abnormal. to the research question (Table 2 and Figure 4) were observed in the evaluation of the quality of the articles included. Several studies 1,15,16 also evaluated the use of the pulse oximeter as a support tool in the Allen test, and Kohonen et al.12 used plethysmography in addition to ultrasonography as a diagnostic confirmation test. There was a wide variability in the consideration of the compression time of the ulnar and radial arteries for the Allen test, with the cut-off point being established in 10 seconds11,13,17-19 (Table 3). Heterogeneity was also observed when choosing the reference artery to evaluate the collateral circulation of the hand using the Doppler ultrasound technique, and the dorsal artery of the thumb was used in half of the studies11,14,16,18,19. A single study18 recommended, after ultrasonically detecting a greater decrease in arterial flow after manual radial artery occlusion in patients with abnormal MAT compared to the rest, that radial artery puncture should not be performed in the event of an abnormal MAT result, due to the theoretical risk of ischemia. Two studies concluded that MAT was valid as a primary screening test for ischemia risk, but it was recommended to use other techniques complementary to the Allen test either jointly13 or later confirmation12. The remaining studies questioned its validity as a single test11,14-17,19, noting the need to use other more objective screening tests, such as ultrasound. In a step
further, the work developed by Yokoyama et al.19 confirmed the validity of Doppler ultrasonography by performing confirmatory angiography (invasive gold standard) in a small number of patients (n = 11). In parallel, using studies that established a maximum time of 10 seconds to recover the colour of the hand after manual compression of the radial artery and then performed a confirmatory Doppler ultrasonography on the main artery of the thumb11,13,19 a meta-analysis of diagnostic validity was performed. The results (Figure 5) show a sensitivity of 76.9% (95% CI 46-93.8) and a specificity of 92.6% (CI 95% 88.4 95.4) and positive and negative predictive values of 35.7% (95% CI 19.3-55.9) and 98.7% (95% CI 95.999.7), respectively, for MAT as a screening test for hand collateral circulation deficit. There was a statistically significant heterogeneity between studies following the global calculation of specificity, which can be explained by the small number of studies included in the analysis (which also makes subgroup analysis impossible). This limits the validity of the results grouped in that section.
Allen's modified test as a predictor of distal hand ischemia The studies aimed at the evaluation of ischemic complications after radial artery puncture (during an-
129
Romeu-Bordas O, et al. Emergencias 2017;29:126-135
Number of records identified in searches (n = 10)
Number of additional records identified in other sources (N = 0)
Total number of duplicate records deleted (n = 5) Total number of single screening records (n = 5)
Total number of records deleted (n = 3)
Total number of full-text articles analysed to decide eligibility (n = 2)
Total number of excluded full-text articles (n = 1)
Total number of studies included (n = 1)
Figure 3. Selection process of studies evaluating the reliability of the modified Allen test. giography, coronary revascularization surgery or intraoperative monitoring) focused on the incidence rate of adverse effects after transradial extremity catheterization Distal upper limbs with an abnormal MAT result (Table 4). From a methodological point of view, these are unicentric follow-up studies with a discrete sample size (considering the low incidence of adverse effects resulting from arterial punctures) and a quality of evidence that can be classified as regular (level VI) according to the Criteria of the AATMC. Three studies20-22 compared the effects of arterial puncture between patients with normal and abnormal MAT and no signs of ischemia were detected in any of the two groups of patients, although Maniotis et al.20 evidenced cases of thrombosis in the artery radial in the subclinical
presentation in a similar proportion in patients with normal and abnormal MAT results. In the RADAR study21, the results of arterial catheterization were evaluated as a function of the time required until palmar reperfusion using serial measurements of capillary lactate on the thumb, without observing differences between the groups and no clinically significant findings. All authors concluded that MAT was not a good predictor of ischemia after arterial puncture and questioned the contingency of arterial puncture to obtain a normal result in MAT. On the other hand, the study by Abu-Omar et al.23 was designed to assess the effectiveness of MAT along with Doppler ultrasound. In this case, MAT was first performed and then, in those hands that had obtained an abnormal result in the test, a confirmatory ul-
Tabla 2. Quality of the studies of diagnostic validity of the palmar collateral circulation deficit of the modified Allen test Study
Risk of bias Patient Selection
Al-Metwalli, et al. ☺ Kohonen, et al. ☹ Agrifoglio, et al. ☺ Greenwood, et al. ☺ Ruengsakulrach, et al. ¿? Yokoyama, et al. ¿? Jarvis, et al. ¿? Michel-Cherqui, et al. ☺ Glavin, et al. ¿? ☺ Low risk; ☹ High risk; ¿? Uncerain risk.
130
Test index
☺
¿? ¿?
☹ ☺ ¿? ☺ ☺
¿?
Concern about the applicability of results Reference test
☺ ☺ ☺ ☺ ☺ ☺ ☺
¿? ¿?
Flow and time
☺ ☺ ☺ ☺ ☺ ¿? ☺
¿? ¿?
Patient Selection
☺ ☺ ☺ ☺ ☺ ☺ ☺ ☺ ☺
Test index
☺ ☺ ☺ ☺ ☺ ☹ ¿? ☹ ¿?
Reference test
☺ ☺ ☺ ☹ ☺ ☺ ☺
¿? ¿?
Romeu-Bordas O, et al. Emergencias 2017;29:126-135
Low
Flow and times
High
Uncertain
Reference test
Test index
Patient selection
Proportion of studies with low, high and uncertain RISK OF BIAS
Proportion of studies with low, high and uncertain CONCERNS ABOUT THE APPLICABILITY
Figure 4. Evaluation of the quality of the studies of diagnostic validity of the palmar collateral circulation deficit of the modified Allen test. trasound. Only arterial puncture was performed in those extremities with normal results in the first or second screening. Although the authors concluded that MAT could be used as a simple and rapid screening test before proceeding to an arterial puncture, they also pointed out the need to confirm the abnormal results of the test by ultrasound, since otherwise most patients with an abnormal test result would be unnecessarily deprived of transradial intervention.
The reliability of Allen's modified test The reliability of the MAT has been calculated in a single study where the interobserver agreement of the results obtained by 4 examiners who tested in 200 hands of 100 healthy volunteers was evaluated24. There was unanimity in 71.5% of the cases evaluated as normal MAT, but in no case did all the observers agree on an abnormal result. Their authors concluding that the MAT could only be considered an acceptable screening test if the abnormal results were confirmed by another complementary test.
Discussion MAT is a simple and inexpensive tool, but it also has important limitations: it requires the cooperation of the patient for its correct performance and does not provide information about the vascular anatomy of the hand. In addition, there is no unanimous agreement on the time of adequate arterial compression; the interpretation of the test results is not very objective and the procedure of the technique can generate false positives (due to a forced hyperextension of the wrist during the test) or false negatives (due to an insufficient compression of the radial artery)25. The results of this review indicate that, although the MAT
still constitutes the reference test to evaluate the collateral circulation of the hand before an arterial puncture, its usefulness is seriously questioned, its use being rejected or advised the search for more objective confirmatory techniques, since the information it provides does not seem relevant as to whether or not a radial artery puncture can be performed safely for the patient. It has been estimated that radial artery occlusion is a not uncommon complication following an arterial puncture, with a variable incidence according to studies ranging from less than 1% to 33%26. Even so, some authors have considered that incidence rates may have been underestimated due to the subclinical nature of the process27. However, few cases of hand ischemia secondary to an arterial puncture have been reported in medical literature 28-30, some of which have even occurred when MAT showed normal results31,32. In our study, none of the studies reviewed have described events of hand ischemia after radial artery puncture. The cases of postpunctional thrombosis detected have been presented subclinically, confirming the low incidence of serious adverse effects arising from transradial interventions, and raise doubts about the clinical importance of transient or permanent occlusion of the radial artery. These data seem to be related to the fact that the majority of the population have complete palm groves (more than 80% and 90% of the population have, respectively, superficial and deep palm arches). However, the vascular anatomy of the hand is complex and may present high individual variability, such as anastomosis that allows hand irrigation in case one of the predecessor arteries is interrupted by flow33,34. Because of the dynamic nature of the collateral circulation, people with normal palmar arches should be able to compensate for occlusion of the irrigation of the radial or ulnar artery. Recently, a study based on angiogra-
131
Romeu-Bordas O, et al. Emergencias 2017;29:126-135
Table 3. Main characteristics of the studies comparing Allen's modified test (without another joint technique) with Doppler ultrasonography (gold standard) in the assessment of palmar collateral circulation First author Country, year (type of study) Al-Metwalli et al. Saudi Arabia, 2014. [Diagnostic validity]
Population
Intervention/Comparison
Results
42 healthy volunteers (84 hands) randomly selected.
MAT with 10" cut-off point vs. DU on the artery of the thumb. He also evaluated plethysmography. MAT with cut-off of 6'' vs DU on arterial of the thumb. He also performed analysis using joint plethysmography to MAT.
Abnormal MAT in 19% and abnormal DU in 3.6%. Sensitivity: 100%; Specificity: 83.9%; PPV: 18.7%; NPV: 100%. The utility of MAT was limited, given its low PPV.
Kohonen et al. Finland, 2007. [Diagnostic validity]
145 patients
