Automatic Detection of Laryngeal Pathologies in Running Speech Based on the HMM Transformation of the Nonlinear Dynamics Carlos M. Travieso1, Jesús B. Alonso1, J.R. Orozco-Arroyave2, Jordi Solé-Casals3, and Esteve Gallego-Jutglà3 1
Signals and Communications Department, Institute for Technological Development and Innovation in Communications, University of Las Palmas de Gran Canaria, Campus University of Tafira, 35017, Las Palmas de Gran Canaria, Las Palmas, Spain {ctravieso,jalonso}@dsc.ulpgc.es 2 DIEyT, Universidad de Antioquia, Medellín, Colombia
[email protected] 3 Digital Technologies Group, University of Vic, Sagrada Família 7,08500 Vic, Spain {jordi.sole,esteve.gallego}@uvic.cat
Abstract. This work describes a novel system for characterizing Laryngeal Pathologies using nonlinear dynamics, considering different complexity measures that are mainly based on the analysis of the time delay embedded space. The model is done by a kernel applied on Hidden Markov Model and decision of the Laryngeal pathology/control detection is performed by Support Vector Machine. Our system reaches accuracy up to 98.21%, improving the current reported results in the state of the art in the automatic classification of pathological speech signals (running speech) and showing the robustness of this proposal. Keywords: Nonlinear Dynamic parameterization, Pathological voice detection, Hidden Markov Models, Laryngeal Pathologies, Speech signal.
1
Introduction
For several years, voice pathology detection has been addressed by means of acoustic, Cepstral and perturbation analysis of voice. Good results have been obtained in sustained voices [1] and also in continuous speech [2]. However, these features present stability and accuracy problems in cases where the estimation of the pitch period is not possible, such as those where the level of the pathology is severe [3]. The nonlinear behavior of the vocal fold vibration was demonstrated in previous works [4, 5]; the research community is now working in nonlinear dynamics (NLD) for the development of methods to perform the automatic and accurate detection of voice pathologies. This problem can be addressed mainly from two different points: the evaluation of sustained phonations and the evaluation of continuous speech. In the evaluation of sustained phonations using NLD features, the state of the art reports accuracy levels of 99.69% considering a set of six NLD features such as first T. Drugman and T. Dutoit (Eds.): NOLISP 2013, LNAI 7911, pp. 136–143, 2013. © Springer-Verlag Berlin Heidelberg 2013
Automatic Detection of Laryngeal Pathologies in Running Speech
137
minimum of the mutual information (FMMI), correlation dimension (Dc), first-order Renyi block entropy, second-order Renyi block entropy and Shannon entropy [6]. More recently, Vaziri et al. [7] report a success rate of 94.44% considering only the correlation dimension. Additionally, there are recent works that consider the evaluation of sustained voices by mixing acoustic, Cepstral, perturbation and NLD features, reporting accuracy levels of 98.23% [8]. For continuous speech, it is already demonstrated that it contains more information about different variations of pitch period, rhythm, intonation and other suprasegmental features [9]. In [2] the authors report accuracy levels of 96.3% considering fourteen Mel Frequency Cepstral Coefficients (MFCC), energy content, harmonics to noise ratio (HNR), normalized noise energy (NNE), glottal to noise excitation ratio (GNE) and the first derivates of each feature, building a representation space with 36 dimensions estimated over a sub set of the Massachusetts Eye & Ear Infirmary (MEEI) database that includes 117 pathological and 23 healthy speech signals. In [10] the authors consider all of the speech recordings in the MEEI database and perform the automatic detection of the laryngeal pathologies in running speech signals by means of different implementations of the jitter, reporting accuracy rates of up to 94.82%. Recently, in [11] different statistics from NLD features are considered for characterizing continuous speech signals obtained from a sub set of the MEEI database. The experiments reported in that work are performed over 396 speech recordings, 360 from people with different laryngeal pathologies and 36 with healthy voices. The results reported by the authors indicate accuracy rates of up to 95%. With the aim of performing the automatic classification of pathological speech signals and healthy recordings, the application of different NLD features is introduced in this paper considering speech recordings from a sub set of the MEEI database. Four complexity measures are implemented for the characterization of the speech signals: correlation dimension (Dc), Lyapunov exponent (λ ), Hurst exponent ( ) and Lempel-Ziv complexity ( ). The features are modeled by a Hidden Markov Model (HMM) [12] and a kernel applied to HMM. Finally, a Support Vector Machine (SVM) [13] is used as classifier for identifying the pathology. The paper is organized as follows: Section 2 includes the description of NLD features. Section 3 provides the details about our classification system. In Section 4, the database, the experimental methodology, results and comments are shown. Finally, in Section 5, general conclusions of this work are presented.
2
Nonlinear Dynamics (NLD) Characterization
Some works in the state of the art demonstrate the existence of nonlinear dynamics in the voice production process and analyze its capability for the automatic detection of pathologies [6, 13]. The different complexity measures which were implemented for the automatic detection of laryngeal pathologies in Spanish vowels and words will be described in the following sections. This measures have not studied from medical
138
C.M. Travieso et al.
point of view, but it will be demonstrated its good behavior for Laryngeal Pathologies. The goal of this parameterization doesn’t depend on pitch [15], and it has invariance of the age and gender. 2.1
Correlation Dimension Dc
In an Euclidean space with dimensionality d, a volume measure V can be described V . To describe by “longitude” L measurements, such that VαLd, or equivalently dα L
Dc, it is necessary to introduce the concept of “correlation sum” in the state space (ε), which can quantify the number of points xi that are correlated with the others inside an sphere with radius ε. Intuitively, this sum can be interpreted as the probability of having pairs of points in a trajectory in the attractor inside the same sphere of radius ε. This event can be described by a uniform distribution, so it is possible to define the expression for the correlation sum using the Heaviside function, as in Equation 1: lim
Θ
1
(1)
0, 0 and is the Euclidean distance between every 1, z 0 pair of points inside the sphere of radius ε. In [16], Grassberger and Procaccia demonstrated that (ε) represents a volume measure, thus the correlation dimension is defined by Equation 2. where Θ z
lim
log
(2)
In the process of the estimation of Dc it is necessary to draw the figure log( (ε)) vs log(ε). The slope of the obtained straight line after a linear regression for the small values of ε is the correlation dimension Dc. A proper estimation of Dc must guarantee that the embedding dimension complies withthe expression = 2Dc + 1 [17]. 2.2
Maximum Lyapunov Exponent
This feature represents the average divergence rate of neighbor trajectories in the state space. Due to its robustness to noisy and short term signals, its estimation in this work has been developed according to the algorithm proposed by Rosenstein, et al. [18]. In this algorithm, once again the nearest neighbors to every point in the trajectories must be estimated. In this case, a neighbor must fulfill a temporal separation greater than the period of the time series, to be considered as a nearest neighbor. Considering every pair of neighbors on each trajectory as the representation of the initial conditions of the phenomena, the maximum Lyapunov exponent is estimated as the average separation rate of the nearest neighbors in the embedding space.
Automatic Detection of Laryngeal Pathologies in Running Speech
139
Applying the Oseledec’s theorem [19], it is possible to state that two points in the attractor are separated at a rate of d(t) = C·eλmaxt, where λmax is the maximum Lyapunov exponent, d(t) is the average divergence taken at the time t, and C is a normalization constant. Considering that the distance between the jth pair of nearest neighbors approximately diverge at a rate of λmax, it is possible to obtain the expression ln(dj(i)) = ln(Cj)+λmax(iΔt),where λmax is the slope of the average line that appears when such expression is drawn on a logarithmic plane. 2.3
Hurst Exponent
This parameter allows to analyze the long term dynamics of a system, stating the possible long term dependencies of the different elements in a given time series. The estimation of H for a time series x(n) with n = 1, 2, ...,N, is based on the rank scaling method, proposed by Hurst, et al. in [20]. Hurst demonstrated that the relation between the variation rank of the signal R, evaluated in a segment, and the standard deviation of the signal S is given by =cTH, where c is a scaling constant, T is the duration of the segment and H is the Hurst exponent. A value of H=0.5 indicate a completely uncorrelated series (Brownian time series), meaning that there is no correlation between any element and a future element and there is a 50% probability that future return values will go either up or down. A value of H in the range 0 < H < 0.5 exists for time series with "anti-persistent behaviour", meaning that a single high value will probably be followed by a low value. Finally, a value of H in the range 0.5 < H < 1 indicates positive autocorrelation, that is the time series is trending. If there is an increase from tn-1 to tn there will probably be an increase from tn to tn+1. Of course, the same rule applies for decreases, where a decrease will tend to follow a decrease. 2.4
Lempel-Ziv Complexity
As LZC is used for estimating the complexity of computer algorithms, it can also be used for the estimation of complexity in time series. Its implementation consists in finding the number of different “patterns” present in a given sequence. The algorithm only considers binary strings; for the practical case, it is necessary to assign 0 when the difference between two successive samples is negative; and 1 for the case when the difference is positive or null. The estimation of the LZC is based on the reconstruction of a sequence X by means of the copying and insertion of symbols in a new sequence. Considering the sequence X = x1, x2 ,... xn, it is analyzed from left to right, the first bit of the string is taken by default as initial point. The variable S is defined to store the bits that have been inserted, i.e. at the beginning S only has x1. The variable Q is defined for the accumulation of the bits that have been analyzed from left to right in the bit stream. On each iteration, the union of S and Q, denoted by SQ is generated. When the sequence Q does not belong to the string SQπ, which is the result of eliminating the last bit in the stream SQ, the insertion of the bits in the subset of symbols finishes. The value of LZC will be the number of subsets used for the representation of the original signal [21].
140
3
C.M. Travieso et al.
Classification System
This part has been divided on two subparts: (i) the use direct of NLD characterization on Hidden Markov Models (HMM) [12]; (ii) the transformation of HMM as parameterization (HMMK), and its classification by means of SVM [12]. The proposed classification system is based on the HMMs [12]. An HMM is a string of states q, jointed with a stochastic process which takes values in an alphabet S which depends on q. These systems evolve in time passing randomly from one state to another and issuing in each moment a random symbol of the S alphabet. When the system is in the state qt-1 = i, it has a probability aij of moving to the state qt = j in the next instant of time and the probability bj(k) of issuing the symbol ot = vk in time t. Only the symbols issued by the state q are observable, nor the route or the sequence of states q. That is why the HMM obtain the appellative “hidden”, since the Markov process is not observable. In this investigation, we have worked with a Bakis HMM, also called left to right, which is particularly appropriate for sequences. The Bakis HMM is especially appropriate for sequential sound data because the transitions between states are produced in a single direction. Therefore, it always advances during the transitions of its states, providing the ability to keep a certain order in this type of models with respect to the observations produced where the temporary distance among the most representative changes [22]. Finally, the HMM model used has been configured with a range from 5 to 20 states and 32 symbols per state, following the recommendations from [22]. The next step is the transformation of HMM probabilities, relating to the approach of the Kernel building [13]. With this goal, the aim is to merge the probability given by the HMM to the given discrimination by the classifier based on SVM. This score calculates the gradient with respect to HMM parameters, in particular, on the probabilities of emission of a vector of data x, while it is found in a certain state q∈ {1,..,N}, given by the matrix of symbol probability in state q (bq(x)), as it is indicated in equation 3;
P ( x / q , λ ) = bq ( x )
(3)
If the derivative of the logarithm of the previous probability is calculated (gradient calculation), the HMM kernel is obtained, whose expression is given by [22];
∂ ξ ( x, q ) log P( x / q, λ ) = − ξ (q ) ∂P( x, q) bq (x )
(4)
Details for the previous equation can be found in [21]. In our case, and using Discrete HMM (DHMM), ξ(x,q) represents the number of times that it is localized in a state q, during the generation of a sequence, emitting a certain symbol x [22]. And ξ(q) represents the number of times which it has been in q during the process of sequence generation [22]. These values are directly obtained from the forward backward algorithm, applied to DHMM by [10]. The application of this score (UX) to the SVM is given by the expression of equation 5, using the technique of the natural gradient;
Automatic Detection of Laryngeal Pathologies in Running Speech
U X = ∇ P ( x , q ) log P ( x / q , λ )
141
(5)
where UX defines the direction of maximum slope of the logarithm of the probability of having a certain symbol in a state. Finally, the final decision will be done by Support Vector Machine (SVM) [13]. SVM is based on a bi-class system, in other words only two classes are considered. In particular for this present work, we have worked with 2 classes, pathological and control classes [13].A linear kernel has been used in our SVM.
4
Experimental Methodology
In this section, the used database, the experiments and the results will be shown. 4.1
Sound Collections
With the aim of eliminating balance problems between the classes, a total of 72 recordings of the “rainbow passage” which are included in the MEEI database are randomly chosen: 36 of the voice recordings are from patients with a variety of voice impairments such as organic, neurological, and traumatic disorders, and the remaining 36 are from healthy people. The speech samples are captured using a condenser microphone in a sound-proof booth and the distance between the microphone and the speakers is 15 cm. The original frequency sampling is 44100 Hz with a resolution of 16 bits, and they are down sampled at 25kHz using the CSL system model 4300. 4.2
Experiments and Results
NLD features described in Section 2 are calculated for each time window of 30 ms, forming four feature vectors (one per feature) per voice recording. A 20% hold-out cross validation were implemented, and it was repeated 10 times in order show the accuracy by mean and standard deviation in the Table 1. We used 20% of samples for training and the rest of samples for testing. Two classification approaches were evaluated, HMM with NLD features and SVM with the HMM transformation of NLD features (see Table 1). Table 1. Accuracy for NLD features for our classification systems
Number of states 5 10 15 20
HMM accuracy 79.24% ± 14.41 88.69% ± 4.09 84.73% ± 9.42 86.50% ± 6.19
Linear SVM accuracy 96.82% ± 6.36 97.42% ± 6.44 98.21% ± 2.36 96.23% ± 4.59
Looking at the experiments, it can be observed that the transformation of the NLD features reaches better accuracy than the use of HMM with NLD characterization directly. It is also interesting to note the decrease of the standard deviation when using
142
C.M. Travieso et al.
transformation of the NLD features, obtaining the smallest value of 2.36 for the better classification accuracy of 98.21%. Therefore, jointing HMM transformation and nonlinear features is a good solution to the automatic detection of laryngeal pathologies in voice. If we compare our results with the state of the art ones we observe that our proposal works better. Hence it can be an option to be considered for the detection of laryngeal pathologies.
5
Conclusions and Future Work
In this work we propose a novel strategy for an automatic detection of laryngeal pathologies based on a fusion of HMM transformation of the nonlinear parameters and the use of a SVM classification system. After experiments, the accuracy reached up to 98.21%, applying a hold-out cross validation. Laryngeal voices have been detected using NLD, achieving better accuracy rates than other systems that are based on acoustic features and only NLD features. According to the results presented in this work, NLD features and its HMM transformation can be used for automatic detection of laryngeal pathologies. The comparison versus the state of the art, for references, which have used the same database, show the improvement of our approach based on the parameterization and its classifier. In [2, 9, 10-11], authors have used temporal information (pitch, etc…), MFCC, jitter and statistics from NLD, but their accuracy rates don’t reach the success of our proposal. The next step will be to increase the number of experiments and different databases and different pathologies for verification approach and to use another SVM kernel. Acknowledgments. This work has been supported by funds with reference “e-Voice” from “Cátedra Telefónica - ULPGC 2013”. Besides, this work has been partially supported by the University of Vic under the grant R904, and under a predoctoral grant from the University of Vic to Mr. Esteve Gallego-Jutglà, ("Amb el suport de l'ajut predoctoral de la Universitat de Vic"). Juan Rafael Orozco Arroyave is under grants of “Convocatoria 528 para estudios de doctorado en Colombia, generación del bicentenario, 2011”financed by COLCIENCIAS.
References [1] Hadjitodorov, S., Mitev, P.: A computer system for acoustic analysis of pathological voices and laryngeal diseases screening. Medical Engineering & Physics 24(6), 419–429 (2002) [2] Godino, J.I., Fraile, R., Sáenz, N., Osma, V., Gómez, P.: Automatic detection of voice impairments from text-dependent continuous speech. Biomedical Signal Processing and Control 4(3), 176–182 (2009) [3] Zhang, Y., Jiang, J.J.: Acoustic analyses of sustained and continuous voices from patients with laryngeal pathologies. Journal of Voice 22(1), 1–9 (2008) [4] Titze, L.R.: Principles of Voice Production. Prentice Hall, Englewood Cliffs (1994)
Automatic Detection of Laryngeal Pathologies in Running Speech
143
[5] Giovanni, A., Ouaknine, M., Guelfucci, R., Yu, T., Zanaret, M., Triglia, J.M.: Nonlinear behavior of vocal fold vibration: the role of coupling between the vocal folds. Journal of Voice 13(4), 456–476 (1999) [6] Henríquez, P., Alonso, J.B., Ferrer, M.A., Travieso, C.M., Godino, J.I., Díaz, F.: Characterization of Healthy and Pathological Voice Through Measures Based on Nonlinear Dynamics. IEEE Transactions on Audio, Speech, and Language Processing 17(6), 1186–1195 (2009) [7] Ghazaleh, V., Farshad, A., Roozbeh, B.: Pathological assessment of patients’ speech signals using nonlinear dynamical analysis. Journal of Computers in Biology and Medicine 40(1), 54–63 (2010) [8] Arias, J.D., Godino, J.I., Sáenz, N., Osma, V., Castellanos, G.: Automatic detection of pathological voices using complexity measures, noise parameters, and mel-Cepstral coefficients. IEEE Transactions on Bio-medical Engineering 58(2), 370–379 (2011) [9] Fourcin, A., Abberton, E.: Hearing and phonetic criteria in voice measurement: clinical applications. Logopedics Phoniatrics Vocology 33(1), 35–48 (2007) [10] Vasilakis, M., Stylianou, Y.: Voice pathology detection based eon short-term jitter estimations in running speech. Folia Phoniatrica et Logopaedica 61(3), 153–170 (2009) [11] Orozco-Arroyave, J.R., Vargas-Bonilla, J.F., Alonso-Hernández, J.B., Ferrer-Ballester, M.A., Travieso, C.M., Henríquez, P.: Voice pathology detection in continuous speech using nonlinear dynamics. In: Proceedings of the 11th IEEE International Conference on Information Science, Signal Processing and their Applications (ISSPA), pp. 1030–1033 (2012) [12] Rabiner, L.R.: A tutorial on Hidden Markov models and Selected Applications in Speech Recognition. Proceedings of the IEEE 77(2), 257–286 (1989) [13] Taylor, J.S., Cristianini, N.: Support Vector Machines and other kernel-based learning methods. Cambridge University Press, Cambridge (2000) [14] Shaheen, A., Roy, N., Jiang, J.J.: Nonlinear dynamic analysis of disordered voice:the relationship between the correlation dimension (D2) and pre-/post-treatment changein perceived dysphonia severity. Journal of Voice 24(3), 285–293 (2010) [15] Jiang, J.J., Zhang, Y., McGilligan, C.: Chaos in Voice, From Modeling to Measurement. Journal in Voice 20(1), 2–17 (2006) [16] Grassberger, P., Procaccia, I.: Measuring the strangeness of strange attractors. Physica D 9, 189–208 (1983) [17] Abarbanel, H.D.I.: Analysis of observed chaotic data. Institute of Nonlinear Science (1999) [18] Rosenstein, M.T., Collins, J.J., De Luca, C.J.: A practical method for calculatinglargest Lyapunov exponents from small data sets. Physica D 65, 117–134 (1993) [19] Oseledec, V.A.: A multiplicative ergodic theorem. Lyapunov characteristic numbers fordynamical systems. Transactions of Moscow Mathematic Society 19, 197–231 (1968) [20] Hurst, H.E., Black, R.P., Simaika, Y.M.: Long-term storage: an experimental study, 1st edn., Constable, London (1965) [21] Kaspar, F., Shuster, H.G.: Easily calculable measure for complexity of spatiotemporalpatterns. Physical Review A 36(2), 842–848 (1987) [22] Briceño, J.C.: Metodología para la Identificación de Formas mediante las Transformación Markoviana de su Contorno. Ph.D. Thesis. University of Las Palmas de Gran Canaria (2013)
