Ultrasound Obstet Gynecol 2014; 43: 170–175 Published online 22 December 2013 in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/uog.12509
Optimal caliper placement: manual vs automated methods B. YAZDI*, P. ZANKER*, P. WAGNER*, J. SONEK†, K. PINTOFFL‡, M. HOOPMANN* and K. O. KAGAN* *Department of Obstetrics and Gynaecology, University of Tuebingen, Tuebingen, Germany; †Fetal Medicine Foundation USA, Dayton, OH, USA; Division of Maternal Fetal Medicine, Wright State University, Dayton, OH, USA; ‡GE Medical Systems Kretztechnik GmbH & Co OHG, Zipf, Austria
K E Y W O R D S: abdominal circumference; automated measurements; biometry; biparietal diameter; femur length
ABSTRACT Objective To examine the inter- and intraoperator repeatability of manual placement of calipers in the assessment of basic fetal biometric measurements and to compare the results with those of an automated caliper placement system. Methods We used stored ultrasound images of 95 normal fetuses between 19 and 25 weeks’ gestation. Five operators (two experts, one resident and two students) were asked to measure the biparietal diameter (BPD), occiptofrontal diameter (OFD), abdominal circumference (AC) and femur length (FL) twice manually and twice automatically. For each operator, the repeatability of manual and automated measurements was assessed using intraoperator SD. For the assessment of interoperator repeatability, the mean of the four manual measurements by the two experts for each fetus was used as the gold standard. The relative bias of the manual measurements of the three non-expert operators and the operatorindependent automated measurement were compared with the gold standard measurement by mean and 95% CI. Results In 89.5% of the 95 cases, the automated measurement algorithm was able to obtain appropriate measurements of BPD, OFD, AC and FL. Intraoperator SDs for the manual measurements ranged between 0.15 and 1.56, irrespective of the experience of the operator. For the automated biometric measurement system, there was no difference between the measurements of each operator. Regarding interoperator repeatability, the mean difference between the manual measurements of the two students, the resident and the gold standard was between −0.10 and 2.53 mm. The automated measurements tended to be closer to the gold standard, but the difference in bias in automated vs manual measurements did not reach statistical significance.
Conclusion In about 90% of cases, it was possible to obtain basic biometric measurements with an automated system. The use of automated measurements resulted in a significant improvement in intraoperator repeatability, but measurements were not significantly closer to the gold standard of expert examiners. Copyright 2013 ISUOG. Published by John Wiley & Sons Ltd.
INTRODUCTION Fetal size is assessed by measuring basic biometric parameters, such as biparietal diameter (BPD), occipitofrontal diameter (OFD), head circumference (HC), abdominal circumference (AC) and femur length (FL). These measurements are used to estimate fetal weight and, when employed in a longitudinal fashion, to assess the rate of fetal growth. There are essentially two types of error that affect the performance of biometric measurements: inappropriate sonographic plane and inappropriate caliper placement. Interpretation of measurements is influenced by the normal biological variability in the size of various fetal parts and the regression equation that is used for a particular measurement. These errors are magnified in the case of fetal weight estimation, because multiple fetal measurements are utilized. Numerous studies have been published comparing the accuracy of sonographic fetal weight estimation with actual birth weight1 – 5 . The mean absolute error between these two parameters has been found to vary considerably, depending to some extent on the size of the neonate (≤ 1500 g, ≤ 2500 g, 2500–4000 g and ≥ 4000 g). However, the mean absolute error of the estimated fetal weight can be as low as 5–8%1 – 5 . Various approaches have been employed in an attempt to improve the accuracy of estimating fetal weight. For example, a recent study utilized morphometric data (HC,
Correspondence: Prof. K. O. Kagan, University of Tuebingen, Calwerstrasse 7, 72076 Tuebingen, Germany (e-mail:
[email protected]) Accepted: 30 April 2013
Copyright 2013 ISUOG. Published by John Wiley & Sons Ltd.
ORIGINAL PAPER
Manual vs automated caliper placement AC, FL) that were acquired postnatally in average-forgestational-age neonates in estimation of fetal weight on prenatal ultrasound; the mean absolute percentage error (APE) between the estimated fetal weight and the actual birth weight in this study was reduced to 4.8%6 . It is possible that automated ultrasound image acquisition and caliper placement would lead to a reduction in operator-dependent error, thereby improving the accuracy of prenatal estimation of fetal weight. While an automated system identifying the appropriate plane for biometric measurement is not available routinely, many ultrasound systems do contain software algorithms that provide automated placement of the calipers to measure BPD, OFD, AC and FL. In this study we examined the inter- and intraoperator repeatability of manual placement of calipers for these measurements and compared the results to those of an automated caliper placement system.
METHODS This study was performed at the Department of Prenatal Medicine at the University of Tuebingen, Germany. At this unit, BPD, OFD, AC and FL are measured routinely at each prenatal ultrasound examination in the second and third trimesters. The images and ultrasound reports are stored in an electronic database (Viewpoint, GE Healthcare, Solingen, Germany) and images are also stored in the ultrasound machine itself. We reviewed all second-trimester examinations that were performed with a GE Voluson E8 (GE Healthcare) ultrasound machine during the month of September 2012, analyzing the images that were saved in the database of the machine itself (so that previously stored measurements could be deleted). We included in the study only those examinations that were performed between 19 and 25 weeks’ gestation, established by last menstrual period dates confirmed by ultrasound or based on firsttrimester crown–rump length measurement. Fetuses with anomalies were excluded. Maternal weight and age were also extracted from the electronic medical records. Five operators were involved in this study; two were senior consultants with extensive experience in fetal medicine and prenatal sonography, two were medical students, who were instructed how to perform biometric measurements prior to the study, and one was a fourth-year resident with basic experience in obstetric ultrasound. For each set of stored images, each operator measured manually, in a standardized fashion, fetal BPD, OFD and FL (all as a distance between two points) and AC (with the ellipse measurement mode). Each operator then repeated these measurements after allowing sufficient time to elapse to minimize bias (at least 1 week). Using the automated software, each operator then performed the measurements again, twice, having been instructed not to adjust the calipers that were placed on the image by the automated system.
Copyright 2013 ISUOG. Published by John Wiley & Sons Ltd.
171
All numeric displays onscreen were covered so that the operators were blinded to the actual measurements. They were also blinded to the results obtained by the other operators and to their own previous results. Measurements were recorded from a separate screen, which was connected to the ultrasound machine. Whilst obtaining his measurements, Consultant 2 (K.O.K.) evaluated subjectively the performance of the automated system by checking whether the target (head, abdomen and femur) was identified correctly and whether the calipers were placed appropriately. Those cases in which the target was not identified properly (i.e. the calipers were placed on a completely different structure, such as the placenta or the uterine wall instead of the appropriate fetal body part, or the automated algorithm failed to identify any structure at all) were excluded from all further analysis. The automated measurement algorithm SonoBiometry (GE Healthcare, Zipf, Austria) is designed to identify the fetal head, abdomen and femur on a frozen ultrasound image and provides automated measurement of BPD, OFD, AC and FL. The structure-finding algorithm interrogates the displayed image, which can be optimized by using post-processing parameters such as dynamic contrast, gamma curves and different speckle reduction imaging and zoom levels. However, for the purposes of this study, further post-processing optimization by the operators was not allowed; images were used exactly as stored except for the fact that previously stored measurements were deleted. The study was approved by the local ethics committee.
Statistical analysis For each operator, repeatability of manual and automated BPD, OFD, AC and FL measurements was assessed by intraoperator SD, calculated as the SD of the differences √ between each pair of measurements divided by 2. Since the operators were not allowed to change the caliper placement performed by the automated system, for these measurements this in fact represents the repeatability of the automated system itself. For the assessment of interoperator repeatability, the mean of the four manual measurements obtained by the two experts for each fetus was used as the gold standard. The relative bias of the manual measurements of the three non-expert operators and the operator-independent automated measurement were compared with the gold standard measurement using means and 95% CIs as well as 95% limits of agreement. Each set of four biometric measurements from each operator was used to calculate the estimated fetal weight (EFW) based on the following Hadlock formula7 : EFW = 10ˆ(1.3596 + 0.0064 × HC + 0.0424 × AC + 0.174 × FL + 0.00061 × BPD × AC – 0.00386 × AC × FL). Head circumference (HC) was calculated as: HC = 2.325 × (OFDˆ2 + BPDˆ2)ˆ1/2). The effect of measurement variability on estimated fetal weight was assessed by evaluating the mean absolute
Ultrasound Obstet Gynecol 2014; 43: 170–175.
Yazdi et al.
RESULTS A search of the database identified 95 cases that met our inclusion criteria. Their median gestational age was 21.3 (interquartile range (IQR), 20.6–22.1) weeks of gestation. The median maternal age was 32.5 (IQR, 29.7–36.3) years and maternal weight was 62.0 (IQR, 58.0–71.0) kg. As evaluated subjectively by Consultant 2 (K.O.K.), in 85 (89.5%) of the 95 cases, the automated measurement system was able to obtain an appropriate biometric measurement of BPD, OFD, AC and FL. In 10 cases (10.5%), the system did not identify the location of the AC or the FL at all.
Intraoperator repeatability of biometric measurements The intraoperator SDs of the manual measurements of BPD, OFD, AC and FL were between 0.15 and 1.56, irrespective of the experience of the operator. All of the automated biometric measurements were the same for each fetus, resulting in an intraoperator SD of 0 (Figure 1). Table 1 shows the 95% limits of agreement between pairs of measurements, for both manual and automated measurements, for each operator.
Interoperator repeatability of biometric measurements The mean difference between the manual measurements of the two students, the resident and the gold standard was between −0.10 and 2.53 mm. They were similar irrespective of the training level of the operator. The automated measurements, especially of BPD and AC, were closer than were the manual measurements to the gold standard (bias of automated measurements: BPD, –0.17 mm; OFD, –0.06 mm; AC, 0.49 mm; FL, –0.03 mm). However, these differences were not statistically significant compared with the manual measurements (Figure 2). Table 2 shows the 95% limits of agreement.
Effect of measurement variability in manual and automated biometric measurements on fetal weight estimation The mean APE of the two fetal weight estimations based on manual measurements was similar for each operator, and ranged between 0.7 and 1.6%. For the automated measurement system, mean APE was 0.
Copyright 2013 ISUOG. Published by John Wiley & Sons Ltd.
2.0 1.8 1.6 1.4 1.2 1.0 0.8 0.6 0.4 0.2 0.0 A C1C2 R S1 S2
A C1C2 R S1 S2
A C1C2 R S1 S2
A C1C2 R S1 S2
Biparietal diameter
Frontooccipital diameter
Abdominal circumference
Femur length
Figure 1 Intraoperator SD of automated (A) and manual biometric measurements by five operators with different experience in fetal medicine (Consultants 1 and 2 (C1 and C2), Resident (R) and Students 1 and 2 (S1 and S2)). Measurements were performed on ultrasound images of 95 normal fetuses between 19 and 25 weeks’ gestation. 8.0 6.0 Mean difference from gold standard (95% CI)
percentage error (APE = (|EFW1 – EFW2|)/ mean × 100). In order to evaluate the intraoperator repeatability, for each operator and each case, the APE was calculated based on comparison of the two sets of manual measurements and the automated measurements. For interoperator repeatability, the mean of the two manual measurements of the three non-experts was compared with the gold standard of the two experts by the APE method. In the same way, the operator-independent automated measurements were compared with the gold standard.
Intraoperator SD (mean and 95% CI)
172
4.0 2.0 0.0 −2.0 −4.0 −6.0 −8.0 A R S1S2
A R S1S2
Biparietal diameter
FrontoAbdominal occipital circumference diameter
A R S1S2
A R S1S2 Femur length
Figure 2 Mean difference from the gold standard, defined as the mean of the manual measurements of two expert operators, of manual and automated biometric measurements by three operators with different experience in fetal medicine (Resident (R) and Students 1 and 2 (S1 and S2)). Measurements were performed on ultrasound images of 95 normal fetuses between 19 and 25 weeks’ gestation. Error bars show 95% CI.
With respect to interoperator repeatability, for manual measurements the mean APE was between 2.1 and 2.4%. For automated measurements the mean APE was 2.1% (Table 3).
DISCUSSION In this study, we have demonstrated that an automated system can generate appropriate measurements of the fetal head, abdomen and femur in about 90% of cases. As expected, use of this technique resulted in a significant improvement of intraoperator repeatability compared
Ultrasound Obstet Gynecol 2014; 43: 170–175.
Manual vs automated caliper placement
173
Table 1 Intraoperator repeatability of manual and automated measurement of basic fetal biometric measurements Mean difference of two measurements in mm (95% LOA) Operator Manual measurement Student 1 Student 2 Resident Consultant 1 Consultant 2 Automated measurement
BPD
OFD
0.05 (−0.42 to 0.51) 0.07 (−0.35 to 0.5) −0.05 (−2.05 to 1.95) −0.01 (−0.55 to 0.52) 0.12 (−0.86 to 1.11) 0
−0.14 (−0.91 to 0.63) 0.01 (−1.3 to 1.31) 0.07 (−1.15 to 1.3) 0.02 (−0.9 to 0.95) 0.14 (−1.24 to 1.52) 0
AC −0.05 (−3.69 to 3.6) −0.43 (−4.63 to 3.78) 0.45 (−3.88 to 4.79) 0.04 (−1.87 to 1.95) 0.80 (−3.16 to 4.76) 0
FL −0.06 (−0.61 to 0.49) 0.05 (−0.35 to 0.46) −0.03 (−0.95 to 0.88) 0.02 (−0.78 to 0.81) 0.02 (−0.78 to 0.81) 0
Intraoperator repeatability was assessed as mean difference between the two measurements of each operator with 95% limits of agreement (LOA). In each case, all automated measurements assessed by the five operators were the same. AC, abdominal circumference; BPD, biparietal diameter; FL, femur length; OFD, occiptofrontal diameter. Table 2 Interoperator repeatability of manual and automated measurement of basic fetal biometric measurements Mean difference between manual or automated measurements and gold standard mean (95% LOA) Operator Manual measurement Student 1 Student 2 Resident Automated measurement
BPD
OFD
AC
FL
−1.06 (−2.27 to 0.15) −0.90 (−4.01 to 2.21) −0.60 (−1.88 to 0.67) −0.17 (−0.96 to 0.61)
−0.38 (−2.92 to 2.15) −0.38 (−2.49 to 1.73) −0.10 (−1.62 to 1.43) −0.06 (−2.39 to 2.26)
1.93 (−4.60 to 8.47) 1.76 (−3.09 to 6.61) 2.53 (−3.35 to 8.40) 0.49 (−4.29 to 5.27)
−0.14 (−1.38 to 1.11) −0.12 (−1.59 to 1.34) −0.26 (−1.15 to 0.62) −0.03 (−1.50 to 1.44)
Manual measurements of both consultants were considered as the gold standard. Manual measurements of the other operators and automated measurements were compared with the gold standard by means and 95% limits of agreement (LOA). AC, abdominal circumference; BPD, biparietal diameter; FL, femur length; OFD, occiptofrontal diameter. Table 3 Intra- and interoperator repeability of fetal weight estimation using manual and automated measurement of biparietal and occipitofrontal diameters, abdominal circumference and femur length APE (%) of fetal weight estimation (mean (SD)) Operator Manual measurement Student 1 Student 2 Resident Consultant 1 Consultant 2 Automated measurement
Intraoperator repeatability
Interoperator repeatability
1.2 (1.0) 1.3 (1.2) 1.6 (1.4) 1.4 (1.0) 0.7 (0.6) 0 (0)
2.4 (2.0) 2.1 (1.8) 2.2 (2.0) Gold standard 2.1 (2.3)
Manual measurements of both consultants were considered as the gold standard. Differences between the measurements and the gold standard are expressed as mean absolute percentage error (APE) and SD.
with manual caliper placement, but measurements were not significantly closer to the gold standard of expert examiners. Whether the use of automated caliper placement will lead to an improvement in the prenatal weight estimation process is still uncertain. However, if a new regression model were computed based on a dataset consisting of only automated measurements and if this weight estimation formula were then used prospectively with automated measurements, it is possible that the accuracy
Copyright 2013 ISUOG. Published by John Wiley & Sons Ltd.
of fetal weight estimation would improve. In order to assess the feasibility of this approach, Yu et al.8 used automated measurements of 215 fetuses prior to delivery to compute a new regression model for estimation of fetal weight. They demonstrated that the mean APE was reduced from 6.7% with manual measurements to 4.7% with the automated approach. These results are consistent with those of Kehl et al.6 , who computed a regression formula for fetal weight estimation based on postnatal measurements in term neonates. The mean APE with this formula was 4.8%, which apparently represents the remaining error related to the regression itself after having eliminated the operator-dependent measurement error. Nonetheless, it must be kept in mind that the most important elements of fetal weight estimation include sonographic acquisition of the correct plane and use of an appropriate regression model. In comparison, the accuracy of caliper placement plays a relatively minor role. Whichever formula and method for caliper placement is used for prenatal weight assessment, the fact remains that appropriate images of the head, abdomen and femur need to be obtained manually and are, therefore, operatordependent. In this study we have shown that fetal measurements can be standardized using an automated system, but they are no closer to the gold standard than are measurements using the manual approach. A few studies have focused on the intra- and interobserver repeatability of manual biometric measurements. Perni et al10 . reported good agreement between the biometric measurements of five sonographers. Sarris
Ultrasound Obstet Gynecol 2014; 43: 170–175.
Yazdi et al.
174
et al.11 showed that with increasing gestational age, the variability of manual measurements increased as absolute values. However, when using z-scores, the variability remained constant throughout pregnancy. In an earlier study12 , this group demonstrated that even among experienced operators, theoretical and practical training results in improvement of measurement consistency. In their investigation of the usefulness of an automated biometric measurement algorithm, Pashaj et al.9 found that in about 40% of cases, the automated measurements were within 5% of the manual measurements. However, in about 20% of cases, the automated measurements were incorrect. The proportion of measurements that were inadequate in their study is much higher than that in our study. One possible explanation for this discrepancy is that we used measurements in only mid to late second-trimester fetuses, whereas Pashaj et al. used data from pregnancies between 11 and 40 weeks’ gestation. Several factors that reduced the ability of the automated system to place calipers correctly were mentioned in their study, many of which become more prevalent in the third trimester. These include the presence of particulate material in the amniotic fluid, oligohydramnios, difficult position of the fetal head and abdomen and acoustic shadowing cast by fetal bones. Theoretically, automated caliper placement systems could be used as a tool to assess the quality of manual caliper placement; if automated caliper placement could be used as the gold standard, the difference between manual and automated measurements could provide a metric for quality assurance. However, in about 10% of cases in our study, the automated algorithm did not identify the displayed fetal body parts correctly. Furthermore, even in cases in which the head, abdomen and femur were identified properly, the quality of the automated measurements was no higher than that of the same measurements obtained by an experienced operator and the bias of automated measurements was not significantly different from the bias of manual measurements of inexperienced operators. Automation has also been introduced to measure fetal structures other than basic biometry, for example firsttrimester nuchal translucency thickness measurement; in this case, automation appears to decrease the operatordependent variation13 – 15 . This benefit of automation was most evident in measurements obtained by nonexpert operators, helping to place their measurements closer to those obtained by experts. The same algorithm was used to measure the intracranial translucency and again it was observed that automation resulted in further standardization of the measurement and in a significant reduction in operator dependency16,17 . As this particular software basically measures a black space between two white lines, it may be applicable in measuring other similar structures, such as the lateral ventricles or the cavum septi pellucidi18 . Some limitations of our study must be noted. One is that we used second-trimester rather than term data. However, in severe growth restriction, when the importance of
Copyright 2013 ISUOG. Published by John Wiley & Sons Ltd.
accurate biometric measurements is paramount, the size of the head, abdomen and femur are often closer to that of second-trimester than term fetuses. However, in contrast to normal late second-trimester pregnancies, growth-restricted pregnancies are often accompanied by oligohydramnios, which may impair automated caliper placement. The second major limitation is the fact that the study was done on stored images. Therefore, the intra- and interoperator variability does not include the variability of appropriate image acquisition. Finally, the gold standard was based on four measurements by two experts and compared to the two measurements of each of the non-experts and the automated system. This simplified approach may have resulted in underestimation of the SD of the expert measurements. However, even after adjusting for this statistical weakness, the differences between the current and the statistically correct approach is minute and would not be expected to be of any clinical significance. Automated algorithms tend to improve the standardization of fetal measurements and help to overcome operator dependency. However, the biggest advance in automation would be an algorithm capable of reproducibly selecting the correct sonographic plane for a particular fetal measurement, followed by accurate caliper placement. In conclusion, in this study we have shown that in about 90% of cases it was possible to obtain appropriate measurements of the fetal head, abdomen and femur using an automated system. The use of automated measurements resulted in significant improvement of the intraoperator repeatability, but measurements were not significantly closer to the gold standard of expert examiners.
REFERENCES 1. Abele H, Hoopmann M, Wagner N, Hahn M, Wallwiener D, Kagan KO. Accuracy of sonographic fetal weight estimation of fetuses with a birth weight of 1500 g or less. Eur J Obstet Gynecol Reprod Biol 2010; 153: 131–137. 2. Siemer J, Hilbert A, Hart N, Meurer B, Goecke T, Schild RL. A new sonographic weight formula for fetuses. Ultraschall Med 2009; 30: 47–51. 3. Dudley NJ. A systematic review of the ultrasound estimation of fetal weight. Ultrasound Obstet Gynecol 2005; 25: 80–89. ¨ 4. Siemer J, Hilbert A, Wolf T, Hart N, Muller A, Schild RL. Gender-specific weight estimation of fetuses between 2,501 and 3,999 g -- new regression formulae. Fetal Diagn Ther 2008; 24: 304–309. 5. Hoopmann M, Abele H, Wagner N, Wallwiener D, Kagan KO. Performance of 36 different weight estimation formulae in fetuses with macrosomia. Fetal Diagn Ther 2010; 27: 204–213. ¨ 6. Kehl S, Schmidt U, Spaich S, Schild RL, Sutterlin M, Siemer J. What are the limits of accuracy in fetal weight estimation with conventional biometry in two-dimensional ultrasound? A novel postpartum study. Ultrasound Obstet Gynecol 2012; 39: 543–548. 7. Hadlock FP, Harrist RB, Carpenter RJ, Deter RL, Park SK. Sonographic estimation of fetal weight. The value of femur length in addition to head and abdomen measurements. Radiology 1984; 150: 535–540. 8. Yu CKH, Smith GCS, Papageorghiou AT, Cacho AM, Nicolaides KH. An integrated model for the prediction of preeclampsia using maternal factors and uterine artery Doppler
Ultrasound Obstet Gynecol 2014; 43: 170–175.
Manual vs automated caliper placement
9.
10.
11.
12.
13.
velocimetry in unselected low-risk women. Am J Obstet Gynecol 2005; 193: 429–436. Pashaj S, Merz E, Petrela E. Automated ultrasonographic measurement of basic fetal growth parameters. Ultraschall Med 2013; 34: 137–144. Perni SC, Chervenak FA, Kalish RB, Magherini-Rothe S, Predanic M, Streltzoff J, Skupski DW. Intraobserver and interobserver reproducibility of fetal biometry. Ultrasound Obstet Gynecol 2004; 24: 654–658. Sarris I, Ioannou C, Chamberlain P, Ohuma E, Roseman F, Hoch L, Altman DG, Papageorghiou AT. International Fetal and Newborn Growth Consortium for the 21st Century (INTERGROWTH-21st). Intra- and interobserver variability in fetal ultrasound measurements. Ultrasound Obstet Gynecol 2012; 39: 266–273. Sarris I, Ioannou C, Dighe M, Mitidieri A, Oberto M, Qingqing W, Shah J, Sohoni S, Al Zidjali W, Hoch L, Altman DG, Papageorghiou AT. International Fetal and Newborn Growth Consortium for the 21st Century. Standardization of fetal ultrasound biometry measurements: improving the quality and consistency of measurements. Ultrasound Obstet Gynecol 2011; 38: 681–687. Moratalla J, Pintoffl K, Minekawa R, Lachmann R, Wright D, Nicolaides KH. Semi-automated system for measurement
Copyright 2013 ISUOG. Published by John Wiley & Sons Ltd.
175
14.
15.
16.
17.
18.
of nuchal translucency thickness. Ultrasound Obstet Gynecol 2010; 36: 412–416. Abele H, Hoopmann M, Wright D, Hoffmann-Poell B, Huettelmaier M, Pintoffl K, Wallwiener D, Kagan KO. Intraand interoperator reliability of manual and semi-automated measurement of fetal nuchal translucency by sonographers with different levels of experience. Ultrasound Obstet Gynecol 2010; 36: 417–422. ¨ B, Pintoffl K, Wright D, Kagan KO, Abele H, Yazdi B, Boer Hoopmann M. Intraoperator and interoperator repeatability of manual and semi-automated measurement of increased fetal nuchal translucency according to the operator’s experience. Prenat. Diagn 2011; 31: 1229–1233. Karl K, Kagan KO, Chaoui R. Intra- and interoperator reliability of manual and semi-automated measurements of intracranial translucency. Ultrasound Obstet Gynecol 2012; 39: 164–168. Chaoui R, Nicolaides KH. Detecting open spina bifida at the 11–13-week scan by assessing intracranial translucency and the posterior brain region: mid-sagittal or axial plane? Ultrasound Obstet Gynecol 2011; 38: 609–612. Abele H, Babiy-Pachomow O, Hoopmann M, Schaelike M, Kagan KO. The cavum septi pellucidi in euploid and aneuploid fetuses. Ultrasound Obstet Gynecol 2013; 42: 156–160.
Ultrasound Obstet Gynecol 2014; 43: 170–175.