Quality Control Tools and Best Practices For Multi-site ...

Quality Control Tools and Best Practices For Multi-site Neuroimaging Data Management
 L. C. MacIntyre1,S. Das1,C. Makowski1,T Glatard1,C. Rogers1, Z. Mohades1, P. Kostopoulus1,D. MacFarlane1,
 C. Madjar2,R. Gnanasekaran1,V. Fonov1, J. Stirling1,L. Collns1, A.C. Evans1
 1McGill




Center for Integrative Neuroscience, Montreal Neurological Institute, McGill University, Montréal, Qc, Canada, 2Douglas Mental Health University Institute

WWW.LORIS.CA


Introduction


Results


•  For over 20 years the MNI has served as the data coordinating centre (DCC) for numerous neuroimaging projects. •  The DCC provides both software (LORIS and CBRAIN) and procedural expertise. •  We will review the comprehensive best practices and procedures validated through several multi-site longitudinal studies, most notably the NIH Pediatric Development Project and the Infant Brain Imaging Study.


 LORIS tools and best practices have been employed in:

Figure 3 LORIS’ Imaging Browser allows the investigator to view candidate images and include/exclude images on the basis of several QC feedback parameters

IBIS-1 project has provided an application for the lifecycle of this QC platform’s capabilities featuring the following metrics:

Methods 
 We will identify both the work flow stages of multi-site neuroimaging studies, and the stage specific QC protocols and modules employed to ensure a robust data set. (Fig 1 and Fig 2)

QC Imaging Work Flow (Fig. 1) Figure 4 LORIS’ Protocol Violation module categorizes scans based on the protocol violated and subsequently allows those with appropriate permissions to resolve these issues.

Registra)on   and  Data   Acquisi)on  

•  Harmonization of images across multipile sites •  Phantom-based MRI corrections •  Comprehensive MOP

•  •  •  • 

DICAT – DICOM anonymizer tool Imaging uploader Protocol violation module (Fig 4) Caveat emptor flags

•  •  •  • 

Imaging QC module (fig 3) Radiological Reviews (Fig 5) Feedback Module Review of scans violating protocol (Fig 4)

Scan  Inser)on   into  DB  

Visual    QC  and   review  of  Raw   Images  

Processed   Images  

Study  Wide   Data   Dissemina)on      

Public  Data   Sharing  and   Open  Science  

•  •  •  •  •  •  • 

Inter and intra site variability assessments Automated SNR and CNR ration checking Visual QC Statistical processing and validation pipelines Cross sectional comparisons Outlier checking Multivariate modeling of derived measures

•  •  •  • 

Data query tool Shared query feature External ID generator Imaging statistics module (Fig 8)

Registra)on   and  Data   Acquisi)on  

Data  Entry  

Data  Valida)on  

Study  Wide   Data   Dissemina)on  

Public  Data   Sharing  and   Open  Science  

Figure 5 LORIS’ Radiological Review module allows radiologists to independently review candidate scans, leave feedback and flag conflicts between reviews to include/ exclude a given candidate.

•  Repository specific external ID generator tool •  Defacing algorithms and PII scrubbing tool (ie. DCM headers) •  Data Archive Module

QC Behavioural Work Flow (Fig. 2)

Figure 6 LORIS’ Conflict Resolver module allows any data entry errors occurring between initial data entry and double data entry to be flagged and resolved.

•  Automated eligibility checks •  Examiner Certification •  Training module

•  •  •  • 

Structural Imaging QC: >4000 volumes Diffusion Imaging QC: >90% QC dataset coverage to date Conflict Resolver: 0.5% error detected and corrected from manual data input Imaging Protocol Violations: 5% of scans were flagged for possible violations Certification of Examiners: >600 registered certifications on >150 examiners

Conclusion The combination of long-standing procedural expertise and LORIS tools comprise a powerful Quality Control platform for neuroimaging studies. The IBIS multisite network has completed a full lifecycle employing this platform for the detection and removal of tens of thousands of errors from its publishable dataset. This comprehensive validated model provides a impactful approach for leveraging neuroimaging and clinical data quality for greater confidence in scientific research results.

References


•  Automated Range Checks •  Automated scoring and validation for derived variables •  Data entry completion stages

•  Double data entry •  Conflict Resolver Module (Fig 6) •  Behavioral feedback module •  Interac)ve  sta)s)cs  module  (outlier  checking,  distribu)on   etc..) •  Reliability module (Fig 7) •  Instrument/measurement comparison flag •  Data integrity module

>130 sites >500 instruments >75,000 variables >30 TBs of imaging datasets

Figure 7 LORIS’ Reliability Module with configurable instrument and question specific thresholds for quantification of cross-examiner assessment reliability.

Data query tool Data Dictionary Enrollment Report and demographics module Comprehensive publication and analysis archivie module

Das, Samir, Tristan Glatard, Leigh C. MacIntyre, Cecile Madjar, Christine Rogers, Marc-Etienne Rousseau, Pierre Rioux, et al. 2016. “The MNI Data-Sharing and Processing Ecosystem.” NeuroImage 124 (Pt B): 1188–95. Ducharme, Simon, Matthew D. Albaugh, Tuong-Vi Nguyen, James J. Hudziak, J. M. Mateos-Pérez, Aurelie Labbe, Alan C. Evans, Sherif Karama, and Brain Development Cooperative Group. 2016. “Trajectories of Cortical Thickness Maturation in Normal Brain Development - The Importance of Quality Control Procedures.” NeuroImage 125 (January): 267–79. Evans, Alan C., and Brain Development Cooperative Group. 2006. “The NIH MRI Study of Normal Brain Development.” NeuroImage 30 (1): 184–202. Orban, Pierre, Cécile Madjar, Mélissa Savard, Christian Dansereau, Angela Tam, Samir Das, Alan C. Evans, et al. 2015. “Test-Retest Resting-State fMRI in Healthy Elderly Persons with a Family History of Alzheimer’s Disease.” Scientific Data 2 (October): 150043. Wolff, Jason J., Hongbin Gu, Guido Gerig, Jed T. Elison, Martin Styner, Sylvain Gouttard, Kelly N. Botteron, et al. 2012. “Differences in White Matter Fiber Tract Development Present from 6 to 24 Months in Infants with Autism.” The American Journal of Psychiatry 169 (6): 589–600.

Acknowledgments
 •  External IDs are generated for open science repositories   •  PII removal •  Data archive module

Figure 8 LORIS’ Statistics module allows users to see at a glance QC status by modality, cohort and site.

Supported by the Irving Ludmer Family Foundation, the LUDMER Centre for Neuroinformatics and Mental Health, McGill University, Foundation, the Government of Canada, the Canada Fund for Innovation and the National Institue of Health