Sci., Liverpool John Moores University, Liverpool L3 3AF, U.K.. 1. Introduction. This study assesses the accuracy of a pattern recognition approach to NMR ...
Quantification of uncertainty in tissue characterisation with NMR Spectra P.J. G. Lisboa, A. Vellido, H. Aung, W. El-Deredy, Y.Y.B. Lee and S.P. J. Kirby School of Comp. and Math. Sci., Liverpool John Moores University, Liverpool L3 3AF, U.K.
1. Introduction
Accuracy of tissue assignment (%)
This study assesses the accuracy of a pattern recognition approach to NMR Spectroscopy for tissue characterisation, taking account of a requirement for confidence in the class assignments. NMR Spectroscopy opens a unique window into the biochemistry of tissue and is, therefore, increasingly used as a non-invasive diagnostic aid for the characterisation of in vivo tissue and for the quantification of tissue extracts.
9498
.
-
-
m
9796-
95-
The complexity in the spectra makes it attractive to apply automated methods of analysis, such as Linear Discriminant Methods (LDA) [1] and Neural Networks (NN) [2], since these methods are, in principle, unbiased of prior clinical expectation. The accuracy of the characterisation of tumour tissue in [1,2] is considerable. However, the variability in the detailed spectral across different acquisitions, patients and clinical sites, reduces the ultimate sensitivity that can be achieved.
10
20
30
40
50
60
70
80
90
100
80
90
100
Probability benchmark
(a) Reject ratio (%)
The focus of this study is the trade-off between reducing the uncertainty of outcome predictions, and rejection of sample assignments. 2. Methodology A set of 83 high frequency spectra, originally described in [3], was labelled using statistical and neural network methods. In an earlier study [2], the spectra from 8 rat tumour and normal tissue extracts were characterised using a neural network optimised within a Bayesian statistical framework [4]. This procedure automatically selected 8 metabolic indicators to maximise the predictive power of the neural network. Whenever a class label is assigned to a new sample, there is an element of uncertainty in that assignment. This uncertainty can be controlled by requiring that the probability of the assignment not occurring by chance exceeds a pre-set benchmark value. Neural network Bayesian models are constructed separately for each tissue type vs. the rest, and the predicted variance of the class assignments is used to estimate its reliability. In addition, an alternative neural network model was constructed, which assigns a posterior probability of class membership to each of the 8 classes, within a single model. The posterior probability estimates are then directly thresholded against a pre-set probability benchmark. Tissue assignments which failed to meet a probability benchmark, were rejected. The rest were accepted and their predictive accuracy measured. 3. Results and discussion Figure 1 shows the performance of class label assignments measured by 11-fold cross-validation tests, as the pre-set posterior probability of making each assignment is stepped up from 5% to 99%. For these high resolution spectra, it is clear that optimal
accuracy in automatic class labelling can be achieved with around 10% sample rejection. Similar results were obtained single multi-
class neural network model.
'0
2o
30
40
50
60
70
Probability benchmark
(b) Figure 1. Trade-off between classification accuracy (a) and rejection rate (b) of a neural network model predicting posterior probabilities of class assignment for NMR spectra, aggregated over 8 rat tumour and normal tissues. 4. Conclusion The accuracy claimed for automatic tissue assignment from NMR Spectra is sustained when account is taken of the statistical uncertainties involved, resulting in surprisingly low rejection rates for high field spectra. Following [5], future work will extend these results to in vivo data. 5. References [1] Preul, M.C., Caramanos, Z., Collins, D.L., Villemure, J.-G., Leblanc, R., Olivier, A., Pokrupa, R. and Arnold, D.L.Nat. Med. , 2, 323, 1996. [2] Lisboa, P.J. G., Kirby, S.P.J. , Vellido, A., Lee, Y.Y.B. and ElDeredy, W. NMR Biomed. , in press. [3] Howells, S.L., Maxwell, R.J., Peet, A.C. and Griffiths, J.R. Mag. Res. Med. 28, 214, 1992. [4] MacKay, D.J. Neural Comp., 4, 448, 1992. [5] Lisboa, P.J. G., Vellido, A., El-Deredy, W. and Auer, D. Proc. ESMRMB,224, 1997.
