Entropy, entropy rate, and pattern classification as tools ... - IEEE Xplore

1282

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 48, NO. 11, NOVEMBER 2001

Entropy, Entropy Rate, and Pattern Classification as Tools to Typify Complexity in Short Heart Period Variability Series Alberto Porta*, Stefano Guzzetti, Nicola Montano, Raffaello Furlan, Massimo Pagani, Alberto Malliani, and Sergio Cerutti, Senior Member, IEEE

Abstract—An integrated approach to the complexity analysis of short heart period variability series ( 300 cardiac beats) is proposed and applied to healthy subjects during the sympathetic activation induced by head-up tilt and during the driving action produced by controlled respiration (10, 15, and 20 breaths/min, CR10, CR15, and CR20 respectively). The approach relies on: 1) the calculation of Shannon entropy (SE) of the distribution of patterns lasting three beats; 2) the calculation of a regularity index based on an entropy rate (i.e., the conditional entropy); 3) the classification of frequent deterministic patterns (FDPs) lasting three beats. A redundancy reduction criterion is proposed to group FDPs in four categories according to the number and type or of heart period changes: a) no variation (0V); b) one variation (1V); and c) two like variations (2LV); 4) two unlike variations (2UV). We found that: 1) the SE decreased during tilt due to the increased percentage of missing patterns; 2) the regularity index increased during tilt and CR10 as patterns followed each other according to a more repetitive scheme; and 3) during CR10, SE and regularity index were not redundant as the regularity index significantly decreased while SE remained unchanged. Concerning pattern analysis we found that: a) at rest mainly three classes (0V, 1V, and 2LV) were detected; b) 0V patterns were more likely during tilt; c) 1V and 2LV patterns were more frequent during CR10; and d) 2UV patterns were more likely during CR20. The proposed approach based on quantification of complexity allows a full characterization of heart period dynamics and the identification of experimental conditions known to differently perturb cardiovascular regulation. Index Terms—Autonomic control, conditional entropy, corrected conditional entropy, heart rate variability, pattern analysis, regularity index, Shannon entropy, symbolic dynamics. Manuscript received January 10, 2001; revised July 05, 2001. Asterisk indicates corresponding author. *A. Porta is with the Dipartimento di Scienze Precliniche, Laboratorio Interdisciplinare Tecnologie Avanzate LITA di Vialba, Universita’ degli Studi di Milano, LITA di Vialba, Via G.B. Grassi 74, 20157 Milano, Italy (e-mail [email protected]). S. Guzzetti and R. Furlan are with the Centro Ricerche Cardiovascolari CNR, Medicina Interna II, Ospedale L. Sacco, Universita’ degli Studi di Milano, 20157 Milan, Italy. N. Montano is with the Dipartimento di Scienze Precliniche, Laboratorio Interdisciplinare Tecnologie Avanzate LITA di Vialba, Universita’ degli Studi di Milano, LITA di Vialba, 20157 Milano. M. Pagani is with the Dipartimento di Scienze Precliniche, Universita’ degli Studi di Milano, LITA di Vialba, Milan, Italy and Medicina Interna I, Ospedale L. Sacco, Universita’ degli Studi di Milano, 20157 Milan, Italy. A. Malliani is with the Dipartimento di Scienze Precliniche, Laboratorio Interdisciplinare Tecnologie Avanzate LITA di Vialba, Universita’ degli Studi di Milano, LITA di Vialba, 20157 Milano and also with the Centro Ricerche Cardiovascolari CNR, Medicina Interna II, Ospedale L. Sacco, Universita’ degli Studi di Milano, 20157 Milan, Italy. S. Cerutti is with the Dipartimento di Bioingegneria, Politecnico di Milano, 20157 Milan, Italy. Publisher Item Identifier S 0018-9294(01)09141-8.

I. INTRODUCTION

T

HERE is an increasing interest in evaluating complexity of the short term cardiovascular control via heart period variability analysis [1]–[6]. Heart period variability is the result of the activity of vasomotor and respiratory centers [7], [8], of baroreflex and chemoreflex closed loop regulation [9], [10] of cardiovascular reflexes mediated by vagal and sympathetic afferences [11], and of vascular autoregulation [12]. All these mechanisms act over similar but not coincident frequencies (below 0.5 Hz) contributing to the complexity of the signal. The increased strength of the links among these mechanisms (usually weakly interacting) or the dominant action of one of these mechanisms taking priority over the others determines the reduction of complexity (i.e., the increase of regularity and predictability) and this is considered a marker of pathology [13]. Complexity analysis can be performed through the evaluation of entropy and entropy rate. Entropy [e.g., Shannon entropy (SE)] calculates the degree of complexity of the distribution of the samples of a signal. The largest entropy is obtained when the distribution is flat (the samples are identically distributed). On the contrary, if some values are more likely (e.g., the sample distribution is Gaussian), the entropy decreases. Usually, in the analysis of heart period variability, entropy is not calculated directly over the samples of the series but over patterns of length (i.e., ordered sequences of samples). In this case, entropy measures the complexity of the pattern distribution as a function of . Voss et al. [14] found a decrease of the SE of patterns lasting three beats in patients after myocardial infarction and correlated this decrease to the risk of malignant arrhythmias and sudden cardiac death. Cysarz et al. [15] found an inverse relationship between the SE of binary patterns and the mean RR interval. Differently, entropy rate (e.g., approximated entropy or conditional entropy) provides a global index of complexity of previous the distributions of the samples conditioned to 2 was samples as a function of . Approximated entropy at found useful to detect infants at risk of sudden infant death syndrome [16]. Conditional entropy (CE) was utilized by Porta et al. [17] to evaluate the decrease of complexity during the sympathetic activation produced by tilt and parasympathetic blockade induced by atropine. Usually, in heart period variability analysis entropy and entropy rates have not been utilized in association [14]–[17] even though they provide different information. Indeed, entropy depends on whether there are some patterns more present than

0018–9294/01$10.00 © 2001 IEEE

PORTA et al.: TOOLS TO TYPIFY COMPLEXITY IN SHORT HEART PERIOD VARIABILITY SERIES

others, but it does not provide any information about the dynamical relationship among patterns (i.e., the rule linking a pattern to the next one), while entropy rate evaluates whether there is a repetitive sequence of patterns, thus quantifying regularity of the signal [18], [19]. Moreover, as complexity indexes do not give any indication about the pattern type, the classification tools might provide the complementary information by labeling the patterns, the frequency, and the recurrency of which are responsible for generating complexity. The aim of this study is to propose an integrated approach to the analysis of complexity of heart period variability. Therefore, an entropy (i.e., SE), an entropy rate (i.e., CE) and a method of pattern classification are jointly utilized. Entropy and entropy rate indexes are compared with verify whether they provide non redundant information even over short data series, while deterministic patterns are classified to find out whether privileged categories do exit and depend on experimental conditions. The analysis was performed on data derived from healthy human subjects during two experimental conditions known to perturb cardiovascular regulation by inducing a sympathetic activation (head-up tilt) and by forcing heart period variability to oscillate at the imposed respiratory rate (controlled respiration at different respiratory rates).

1283

tern (MP)[14]. The percentage of MP (MP%) is calculated by dividing the amount of MPs by the number of all possible patterns (i.e., ). As the distribution of the patterns depends on and , MP% is also a function of and . C. Shannon Entropy SE is defined [20] as (2) represents the probability of the pattern apwhere proximated by its sample frequency and the sum is extended to all different patterns . SE is an index describing the shape of the distribution of the patterns . Indeed, SE is large if the dispatterns are identically distributed tribution is flat (all the and the series carries the maximum amount of information). On the contrary, SE is small if there is a subset of patterns more likely, while others are missing or infrequent (e.g., in a Gaussian distribution). Obviously, the SE is derived directly from the PDF of . D. CE and Regularity Index The CE is defined [20] as

II. COMPLEXITY MEASURES A. Coarse Graining and Pattern Construction is first Each sample of the series normalized by subtracting the mean and, then, divided by the standard deviation, thus obtaining the series expressed in adimensional units. Next, the full range ) is divided into a fixed number of dynamics (i.e., of values (the quantization levels) labeled with numbers from . The quantization procedure performs a coarse zero to graining of the dynamics with a resolution equal to , thus rendering the signal a sequence of symbols from the limited alphabet of symbols . From the symbolic series , patterns of delayed samples (also referred to as words in terms of symbolic . dynamics) are constructed as As the time delay between samples is equal to one, sents an authentic feature of signal (i.e., a wavelet).

repre-

B. Probability Density Function of the Patterns The wavelet

can be codified in decimal format as

(1) again a series of inthus rendering the sequence of wavelets with ranging teger numbers . With this definition, the from zero to probability density function (PDF) of provides the distribution of the patterns . It reports the sample frequency of each as a function of the pattern decimal code . If the pattern is never detected in , then it is referred to as missing pat-

(3) is the probability of the pattern , where is the probability of the most recent symbol of the pat(i.e., ) when the previous ones (i.e., tern ) are given. The inner sum can be interpreted as the SE calculated over the distribution of the conditioned by the patterns and performed symbols over all different symbols , the outer sum is performed over . If the signal is fully repetitive, it is all different patterns possible to find a value of

such that

(i.e.,

are completely predictable from the patterns the symbols ) and . If the signal is an identically disgiven tributed white noise, the conditional distributions of are flat independently of (i.e., the past samples never reduce the uncertainty in the prediction of the next sample), . If the white noise is not thus producing a identically distributed, the CE is again constant but the value . On the contrary, if repetitive patterns is smaller than embedded in noise are present, the CE is not constant but given decreases as soon as the conditional distributions of exhibit a sharp peak (i.e., the past samples are useful to predict future dynamics). As an effect of its definition, CE is like SE but a not a measure of the incidence of patterns follow each measure of how much regularly the patterns can be calculated also as other in the series. (4) where of the series

and represent the SE of the PDF obtained by considering patterns of length and

1284


respectively. Unfortunately, when CE is estimated over can be the result of a short data segments, given the pattern , thus producing a decrease unique of CE with regardless of the type of dynamics[19]. If this , the 0. For phenomenon occurs for every pattern example, null CE values can be observed at very small in short samples) of white noise. In this specific case, segments ( is only an effect of an unreliable estimate of due to the shortness of the data sequence. If the might be found and a length sequence is increased, several different from one might be measured. Porta et al. [19] propose a function referred to as corrected CE (CCE) dealing with this bias. It is defined as (5) is the SE of the distribution of the quantized where and is the percentage of patterns found series only once in the data set. Equation (5) substitutes the erroneous due to its unique appearcertainty associated to the pattern ance with the maximum uncertainty detectable in the series . Therefore, the CCE: 1) remains constant measured by in case of white noise; 2) decreases to zero in case of a fully periodic signal; and 3) exhibits a minimum if repetitive patterns , are embedded in noise. If the CCE is normalized by the normalized CCE (NCCE) ranges from zero to one. Therefore, based on the detection of the NCCE minimum, an index [17]. of regularity is defined as It ranges from one (maximum regularity) and zero (maximum complexity). III. METHODS A. Experimental Protocol and Beat-To-Beat Variability Series Extraction Fifteen healthy young subjects (ten women and five men; age range: 24–32 years) were enrolled in this study. All the subjects had no sign of organ or systemic disease. No one was taking any medication and each subject was instructed to avoid beverages containing alcohol or caffeine in the day preceding recordings. Studies were performed between 09:00 a.m. and 11:00 a.m. All the subjects were carefully instructed and gave their written consent. The protocol was approved by our Institute’s Review Board. The protocol included four recordings lasting 10 min in supine position with the subject placed on an electrically driven tilt table. After 30 min allowed for stabilization, the first recording was obtained at rest while the subject was breathing spontaneously. The following three recordings in supine position were obtained while the subject was controlling his breathing rate at 10, 15, and 20 breaths/min according to a metronome (CR10, CR15, and CR20, respectively). All the subjects were trained to follow the metronome and adjust their tidal volume in response to changes of the respiratory rate. Finally, the last recording was obtained during spontaneous breathing after the subject was passively moved to an upright

80 position by electrically rotating the tilt table. No episode of syncope was observed. Electrocardiogram (ECG) signal (lead II) and the respiratory signal (via a nasal thermistor) were stored on a magnetic tape (Racal Recorder, Southampton, U.K.). Next, signals were played back from the tape and sampled at 300 Hz by means of a 12 bits analog-to-digital board (National Instruments, Austin, TX). All the analyzes were carried out on a PC by ad hoc C-code programs. The QRS complex of the ECG signal was detected when the first derivative of the ECG exceeded a user-defined fraction of the maximum derivative. A parabolic interpolation was performed on the R peak. The R peak was located at the maximum of the parabola. The RR interval was defined as the time interval between two consecutive R peaks. The detections were carefully checked. Missed detections were manually inserted, while detections of peaks of noise were deleted. Only isolated ectopic beats were occasionally observed. If an ectopic beat was detected, the RR intervals ending or starting with the ectopic beat were substituted with a linear interpolation between the most adjacent RR intervals defined by normal R waves. The respiratory rate was derived by spectral analysis from the respiratory signal to verify the ability of the subject to follow the metronome. B. Evaluation of Complexity Indexes The reliability of the estimate of PDF increases while in. creasing the length of data sequence with respect to Unfortunately, increasing caused a lack of stationarity. Therewas maintained low by choosing small values for fore, the pattern length and for the number of quantization levels. should However, the number of detected patterns in order be larger than the number of possible patterns to allow patterns to be found several times. Therefore, as series of 300 cardiac beats were considered, SE and MP% were caland . Differently, culated with as the regularity index was derived by using a minimization procedure over , the pattern length was not a priori fixed (CCE was necessary to avoid the unreliability of the CE estimate), while is again fixed to six. C. Surrogate Data Approach and Deterministic Pattern Detection Due to the small number of quantization levels utilized to codify the dynamics, some patterns might be more frequent than others only by chance. In order to exclude this possibility a surrogate data approach was followed [21]. Fifteen surrogate data series were obtained from the original one by randomly shuffling the samples according to 15 different white noise realizations, thus completely destroying the original power spectrum but maintaining sample distribution. A complexity index (SE, was comMP% or ) calculated over the original series by pared with those obtained from the surrogate series using the formula [21] (6) and where dard deviation of

represented the mean and the stanover the 15 surrogate series. If ,


1285

4, 4) or (3, 3, 1)]; 3) patterns with two like variations [2LV, the three symbols formed an ascending or descending ramp, e.g., (1, 2, 4) or (5, 4, 3)]; and 4) patterns with two unlike variations [2UV, the second symbol was larger or smaller than the remaining ones forming a peak or valley, e.g., (3, 4, 3) or (3, 0, 2)]. The choice of this strategy for pattern redundancy reduction was to group all possible patterns in four categories characterized by different frequency contents: 0V category collected patterns characterized only by very slow frequencies (the pattern was constant); 1V class clustered patterns with some slow and very slow frequencies as both a plateau and a ramp were present; 2LV category grouped patterns typified only by slow frequencies without slower frequencies as the plateau was not codified; and 2UV class collected patterns dominated by faster frequencies as they actually represented peaks or valleys. E. Statistical Analysis One way ANOVA (Bonferroni test) was utilized to test differences between rest and all the other experimental conditions. A was considered significant. IV. RESULTS A. Complexity Measures in Short Beat-To-Beat Variability of RR Interval

Fig. 1. Examples of patterns belonging to 0V [first row: (4, 4, 4) and (1, 1, 1)], 1V [second row: (3, 4, 4) and (3, 3, 1)], 2LV [third row: (1, 2, 4) and (5, 4, 3)] and 2UV [forth row: (3, 4, 3) and (3, 0, 2)] categories.

could not derive from a white noise with the same sample . distribution of the original series Moreover, in order to allow a reliable classification of the even on short data segpatterns lasting three beats ments, the PDF of patterns calculated over the original segment was compared with the PDF obtained from the surro. If, for a given pattern, was larger gate series then that specific pattern was than not detected by chance in the series and, therefore, it was referred to as deterministic pattern. D. Classification of Frequent Deterministic Patterns and Redundancy Reduction A deterministic pattern observed with a probability larger than 0.04 in the original series, was defined as frequent deterministic pattern (FDP). Three-beat FDP patterns were reduced into the following four families (Fig. 1): 1) patterns with no variation [0V, all the three symbols were equal forming a three-beat plateau, e.g., (4, 4, 4) or (1, 1, 1)]; 2) patterns with one variation [1V, two symbols were equal and consecutive forming a two-beat plateau, while the remaining one was different, e.g., (3,

An example of heart period variability series at rest, during tilt, CR10, CR15 and CR20 are depicted in Fig. 2(a)–(e), respectively. It was easy to recognize deterministic, repetitive patterns with different durations (the slowest ones during tilt and progressively faster from CR10 to CR20). However, the duration and the amplitude of these patterns changed thus preventing the strict repetition of the same pattern (this phenomenon was evident even during tilt and CR10). Also the presence of irregular oscillations with a period larger than the duration of these patterns contributed to the complexity of the series (more evident at rest and during CR15 and CR20). Quantized series maintained these features [Fig. 2(f)–(j)]. Fig. 3 depicts the PDFs calculated over the series of Fig. 2. At rest, the PDF [Fig. 3(a)] revealed that some patterns were more frequently detected (the peaks had different heights) and others were missing (the sample frequency was 0). During tilt [Fig. 3(b)], the frequency of some patterns largely increased (some peaks emerged) as well as the number of missing patterns. During CR10, CR15, and CR20, PDFs [Fig. 3(c)–(e)] were not different from those at rest. Fig. 4 shows the CCE functions calculated over the series of Fig. 2. At rest [Fig. 4(a)], the RR interval was regular (the CCE exhibited a minimum) but the CCE minimum was largely different from zero (i.e., the regularity index was different from 1). During tilt and CR10 [Fig. 4(b) and (c), regularity increased (the CCE minimum decreased], while it was unchanged during CR15 and CR20 [Fig. 4(d) and (e)]. Fig. 5 shows an RR interval series at rest [Fig. 5(a)] and its surrogate series [Fig. 5(b)]. Fig. 5(c) depicts the PDFs calculated over the original and surrogate series (open and filled bars, respectively). PDF obtained from original series was different from that obtained from its surrogate series. Indeed, several peaks were larger and missing patterns were more present. However, PDF calculated over the surrogate data was not flat (some patterns could be frequently detected only

1286


Fig. 2. Beat-to-beat series of heart period variability in a healthy young subject at rest (a), during tilt (b), CR10 (c), CR15 (d) and CR20 (e). Their relevant quantized series are depicted below (f,g,h,i,j). Slow oscillations (at 0.1 Hz) are clearly evident in (b), while high-frequency rhythms are present in (a), (c), (d), (e), progressively faster from (c) to (e). Quantized series maintain these features.

Fig. 3. PDFs calculated on the series of Fig. 2. In all the experimental conditions the PDF is not flat, thus indicating that some patterns are more frequently detected than others. SE is 4.02, 3.30, 3.99, 4.36, and 4.57, while MP% is 62.96, 79.63, 64.35, 52.78, and 45.37, at rest, during tilt, CR10, CR15, and CR20, respectively. The SE is significantly reduced during tilt (b).

Fig. 4. CCEs calculated on the series of Fig. 2. In all the experimental conditions the CCEs exhibit a minimum, thus detecting a certain level of regularity. The CCE minimum is deeper during tilt (b) and CR10 (c). The regularity index is 0.175, 0.482, 0.292, 0.154, and 0.117 at rest, during tilt, CR10, CR15, and CR20, respectively.

by chance). Fig. 5(d) depicts CCEs calculated over the original and surrogate series of Fig. 5(a) and (b). The CCEs calculated over the original (dotted line) and the surrogate data (solid line) were different: the former exhibited a deep minimum, while the latter was completely flat. The results of complexity analysis are summarized in Table I. The complexity indexes calculated over all the original series from those obtained from their surwere different rogates in all the experimental conditions. During tilt, SE de, while MP% increased . Under creased

controlled respiration SE and MP% did not change. Regularity index increased during tilt and CR10 , while it was not influenced by CR15 and CR20. B. Classification of FDPs in Short Beat-To-Beat Variability of RR Interval The procedure to detect FDPs is described in Fig. 6. It shows the superposition of a PDF of an original RR interval series at rest (open bars) and the average PDF plus two times the standard deviation calculated over 15 surrogate realizations (filled


Fig. 6.

1287

Description of the procedure to detect FDPs. A FDP is found if the

PDF calculated over the original series (open bars) overcomes av[PDF ] + 2 sd[PDF ] calculated over 15 realizations of surrogate data (filled bars) and if PDF is larger than 0.04 (dotted line). Only three FDPs are detected. 1

PERCENTAGE

OF

TABLE II EXPERIMENTAL SUBJECTS EXHIBITING FREQUENT DETERMINISTIC PATTERNS (FDP)

Fig. 5. Original beat-to-beat variability of heart period at rest (a) and its surrogate series (b). The PDFs calculated over the original beat-to-beat series of heart period at rest (open bars) and over its surrogate series (filled bars) are depicted in (c). The PDF obtained from the surrogate data is not flat, thus indicating that several patterns can be repetitive only by chance. SE is very different (3.947 and 4.607 over original and surrogate series, respectively) as well as MP% (64.81 and 42.13, respectively). The CCEs calculated over the original (dotted line) and surrogate (solid line) series are depicted in (d). The CCE calculated over the surrogate series is completely flat (the regularity index is 0.269 and 0.006 over the original and surrogate series, respectively). TABLE I RESULTS OF THE COMPLEXITY ANALYSIS

bars). The open bar was higher than filled bar when the pattern was more frequently detected in the original than in surrogate data. That pattern was referred to as deterministic pattern. If the sample frequency of the deterministic pattern was larger than 0.04 (the dotted line), it was referred to as FDP. Three FDPs are detectable in Fig. 6. Table II reports FDPs and their class of belonging according to the adopted redundancy reduction criterion (first and second column, respectively) and percentage of experimental subjects

that exhibit that particular pattern while varying the experimental condition (the remaining columns). Only FDPs found in more than two subjects and at least in one experimental condition were reported. Therefore, if a FDP was missing in the first column of Table II, this means that it was strongly unlikely in all the experimental conditions. A hyphen was reported in Table II when the relevant FDP was found in less than three subjects in that specific experimental condition. At rest FDPs mainly belonged to three classes (i.e., 0V, 1V, and 2LV) and the 1V class was privileged. During tilt two classes were detected (i.e., 0V and 1V) and the 0V class was more important. During CR10 and CR15, two classes were identified: 1V and 2LV classes during CR10 and 0V and 1V classes during CR15. During CR20 the 2UV class was found with the classes 0V and 1V. Table III reports the percent change of the number of subjects that exhibit FDPs belonging to the class indicated

1288


TABLE III PERCENT CHANGES OF THE NUMBER OF SUBJECTS EXHIBITING FREQUENT DETERMINISTIC PATTERNS BELONGING TO THE CONSIDERED CLASS

could be covered in an irregular fashion, thus explaining a high approximated entropy in presence of a small SE. Also Fig. 5 illustrates this concept. Indeed, the distribution pattern obtained from a random surrogate data (Fig. 5(c), filled bars) is not flat (a large amount of missing patterns are present, thus limiting the number of patterns actually present and the SE) but the regularity index is null (the CCE is flat, Fig. 5(d), solid line). B. Complexity Analysis of Heart Period Variability

During tilt, we observed a dramatic increase of number of subjects exhibiting patterns relevant to the 0V class, while the 2LV and 2UV classes diminished their importance. During CR10, 1V, and 2LV patterns were detected more frequently, while the number of subjects characterized by 2UV patterns largely decreased. During CR15, the increase of the importance of 1V and 2LV classes was less remarkable than during CR10. Also the reduction of the number of subjects exhibiting 2UV patterns was less important. During CR20, the importance of the 0V, 1V, and 2LV categories decreased while the number of subjects characterized by 2UV patterns dramatically increased. V. DISCUSSION The proposed approach to the complexity analysis of heart period variability recommends: 1) to jointly consider entropy and entropy rate indexes to quantify two different aspects of complexity; 2) to perform pattern classification to clarify the type of patterns contributing to the generation of complexity. A. Entropy and Entropy Rate Indexes Provide Different Information SE is a measure of the complexity of the pattern distribution. The presence of peaks in the pattern distribution (relevant to patterns more frequently detected) or valleys (relevant to missing or less frequent patterns) determines the decrease of the SE with respect to its maximum value provided by a flat distribution. Therefore, the SE correlates with the percentage of missing patterns. The CE [17] and the approximated entropy [16] provide measures of the complexity of the dynamical relationship between a pattern and the next one. If the temporal sequence of the patterns is fully regular (i.e., the patterns follow each other in a repetitive periodic way), the CE is zero. On the contrary, the maximum value of the CE is found when no relationship between a pattern and the next one is found (the sequence of pattern is completely random). Due to these different definitions, measures of complexity based on entropy (e.g., SE) and entropy rates (e.g., approximated and CE) provide different information. Indeed, a significant increase in the regularity index (a decrease of CE) does not imply a decrease in the SE, as it can be observed during CR10. It appears that heart period variability during CR10 is characterized by the same amount of patterns found at rest (during CR10 MP% is not significantly different from rest) but the temporal sequence of patterns is more regular and predictable (i.e., the next pattern is selected in a less random fashion). Conversely, a significant decrease of the SE does not imply a decrease in the entropy rate (e.g., the approximated entropy) as it was observed by Palazzolo et al. [3] in dogs during standing. It appears that the limited number of frequent patterns

At rest, short variability series of heart period exhibit a large and a level of regunumber of missing patterns , thus evidencing that larity largely different from heart period variability contains a limited amount of patterns but the temporal sequence of these patterns is not fully regular (Table I). During tilt, the decrease of the SE and the increase of missing patterns and of the regularity index suggest that the decrease in complexity is the effect of a presence of a smaller number of patterns, the temporal sequence of which is more predictable. During CR10, the increase in regularity not associated with a decrease of the SE and of the number of missing patterns demonstrates that the same amount of patterns found at rest forms a more repetitive sequence of patterns. During CR15 and CR20 the SE, the number of missing patterns and the regularity index do not change, thus measuring the same level of complexity calculated at rest both in terms of complexity of the pattern distribution and of the relationship linking a pattern to the next one. In a previous paper [17], we attributed the increase of regularity found during tilt to the synchronizing action of the sympathetic activation on mechanisms modulating heart period in the low-frequency (LF) band (from 0.04 to 0.14 Hz) and to the limiting action over the high-frequency oscillation (HF, at the respiratory rate), thus reducing the number of temporal scales present in heart period variability. As a new finding, this study points out that the sympathetic activation is also able to reduce complexity through a reduction of the number of different patterns constituting heart period variability. In a previous paper [17], we attributed the frequency-dependent effect of controlled respiration on regularity of heart period (only slow breathing rates produce an increase of regularity) to the tidal volume (it decreases while increasing the respiratory rate). Indeed, slow breathing rates are more effective in stimulating cardiac and pulmonary low pressure receptors and in inducing respiratory-related changes on arterial pressure able to produce heart rate variations via the baroreflex feedback. As a new finding, this study indicates that controlled respiration leaves unchanged the number of patterns composing heart period variability and, therefore, the flexibility of the cardiovascular regulatory system even in presence of a more regular alternation among patterns produced by slower breathing rates. Pathological conditions (e.g., at the level of low pressure receptors or neural pathways) could reduce the ability of slower breathing rates to regularize heart period. C. Pattern Classification in Heart Period Variability Series The SE provides a measure of the complexity of the pattern distribution but does not furnish any indication about the type


of patterns detected in the series. Indeed, if the dynamics of the two series are characterized by different patterns with identical sample frequencies, the SE is equal. Therefore, different dynamics may exhibit equal SE, thus rendering necessary a pattern classification to understand which patterns are actually involved in generating the complexity. Usually pattern analysis in heart period variability series is based on an a-priori definition of the patterns to be detected [1], [22]. This approach needs to define the physiological correlate of the searched wavelet (e.g., the tachicardia at the onset of physical activity [22]) to ensure the search for deterministic and meaningful patterns. In this study, we propose a new approach based on the identification of the most frequent features directly on heart period variability series. This approach exploits the pattern classification performed to calculate the sample frequency of each pattern necessary to derive both the Shannon and conditional entropies. This approach requires: 1) to validate the deterministic nature of the pattern; 2) to discard unlikely patterns; 3) to provide a redundancy reduction criterion to group the detected patterns into a limited number of categories; and 4) to a-posteriori search for physiological correlates of categories. To validate the deterministic nature of a pattern, we use a surrogate data approach. When a coarse graining procedure is applied to convert the original series in a sequence of symbols belonging from a very limited alphabet (six symbols in this application), it is mandatory to perform surrogate data analysis in order to check whether some patterns are frequent only by chance. Unlikely deterministic patterns are discarded by setting a threshold on the sample frequency (0.04 in this study), thus classifying only FDPs (Table II) and focusing only on wavelets capable to describe a large amount of the dynamics of the series. This choice provides the first redundancy reduction criterion utilized in this study. A second redundancy criterion is applied to group FDPs in four categories based on the number and type of symbol variations. We find out that heart period variability series have different compositions in terms of the four defined categories (Table II) and changes of the importance of the four classes allow to distinguish the considered experimental conditions (Tables II and III). The different weight of the four classes can be interpreted by considering the different frequency content of the heart period variability under the considered experimental conditions [23], [24]. Indeed, 0V patterns are features of slow waves (e.g., LF oscillations) while 2LV and 2UV patterns are fragments of faster waves (e.g., HF oscillations). At rest mainly three classes (0V, 1V and 2LV) are detected due to the contemporaneous presence of LF and HF oscillations. During tilt, the large increase of number of subjects showing 0V patterns and the important decrease of the number of subjects exhibiting 2LV and 2UV patterns can be explained in terms of the increased presence of LF waves and a decreased presence of HF oscillations. During CR10, the number of subjects showing 0V patterns decreases, while 2LV patterns are more likely as a result of the absence of LF oscillations and of the presence of HF oscillations. In addition, during CR10, 2UV patterns are largely less likely as respiratory sinus arrhythmia is stable and not too fast Hz). During CR15 the composition of heart rate vari(

1289

ability in terms of the defined four categories is similar to that observed at rest. During CR20, features characterizing fast frequencies (2UV patterns) are more likely, while 0V patterns relevant to slow oscillations are also present, thus demonstrating the rise of oscillations slower than HF rhythms. Even though sensible to the frequency content of the series (i.e., to the linear features of the signal), the proposed classification method cannot be considered as another approach to perform power spectral analysis. Indeed, the approach takes into account also nonlinear features and completely decomposes the series in patterns not limited to be portions of sinusoids. At the moment, we lack the necessary physiological knowledge to transform the detected patterns or categories into physiological correlates and to combine the detected patterns or categories forming longer, nonlinear and complex waveforms. However, several findings not detectable trough conventional methods and typically related to nonlinear features, can be derived from the classification of FDPs (Table II): 1) during tilt, all the patterns belonging to the 0V class are frequently detected (only (0, 0, 0) is unlikely), thus pointing out the importance of amplitude modulations in this experimental condition; 2) during CR10, increasing ramps [i.e., (2, 3, 4)] are not found as frequently as decreasing ramps [i.e., (4, 3, 2)], thus suggesting an asymmetry in the respiratory sinus arrhythmia; 3) patterns describing abrupt changes in the heart period ([i.e., variations larger than one quantization level like in (1, 3, 3), (5, 4, 1), or (5, 3, 5)] are not repetitively found, although they are present, thus largely contributing to the generation of heart period complexity; and 4) during CR15 and CR20, patterns, that are composed by quantization levels below three [e.g., (3, 2, 2)], are unlikely, thus suggesting a more regular dynamics at larger heart periods. VI. CONCLUSION As indexes based on entropy (e.g., SE) and entropy rate (e.g., CE) provide different information, they should be utilized in association to evaluate complexity of heart period variability. Moreover, as complexity measures based on entropy calculation do not provide any indication about the pattern types involved in the generation of a specific level of complexity, measures of complexity should be accompanied by a pattern classification procedure. Pattern classification, when based on surrogate data approach and redundancy reduction criteria, help to identify physiological conditions characterized by the activation of different mechanisms responsible for cardiovascular regulation. In presence of physiological correlates of the detected patterns, the proposed approach can be extremely selective in searching for a specific status and in performing a waveform analysis. For example, in the study of arrhythmias, where specific and well codified sequences of cardiac beats are clearly related to malignant events, this tool may be extremely useful. REFERENCES [1] H. Bettermann, D. Amponsah, D. Cysarz, and P. Van Leeuwen, “Musical rhythms in heart period dynamics: A cross-cultural and interdisciplinary approach to cardiac rhythms,” Amer. J. Physiol., vol. 277, pp. H1762–H1770, 1999.

1290


[2] N. Iyengar, C.-K. Peng, R. Morin, A. L. Goldberger, and L. A. Lipsitz, “Age-related alterations in the fractal scaling of cardiac interbeat interval dynamics,” Amer. J. Physiol., vol. 271, pp. R1078–R1084, 1996. [3] J. A. Palazzolo, F. G. Estafanous, and P. A. Murray, “Entropy measures of heart rate variation in conscious dogs,” Amer. J. Physiol., vol. 274, pp. H1099–H1105, 1998. [4] S. M. Pikkujamsa, T. H. Makikallio, L. B. Sourander, I. J. Raiha, P. Puukka, J. Skytta, C.-K. Peng, A. L. Goldberger, and H. V. Huikuri, “Cardiac interbeat interval dynamics from childhood to senescence. Comparison of conventional and new measures based on fractals and Chaos theory,” Circulation, vol. 100, pp. 393–399, 1999. [5] J. S. Richman and J. R. Moorman, “Physiological time-series analysis using approximate entropy and sample entropy,” Amer. J. Physiol., vol. 278, pp. H2039–H2049, 2000. [6] S. Vikman, T. H. Makikallio, S. Yli-Mayry, S. M. Pikkujamsa, A.-M. Koivisto, P. Reinikainen, K. E. J. Airaksinen, and H. V. Huikuri, “Altered complexity and correlation properties of RR interval dynamics before the spontaneous onset of paroxysmal atrial fibrillation,” Circulation, vol. 1000, pp. 2079–2084, 1999. [7] J. L. Feldman, J. C. Smith, H. H. Ellenberger, C. A. Connelly, G. Liu, J. J. Greer, A. D. Lindsay, and M. R. Otto, “Neurogenesis of respiratory rhythm and pattern: Emerging concepts,” Amer. J. Physiol., vol. 259, pp. R879–R886, 1990. [8] G. Preiss and C. Polosa, “Patterns of sympathetic neuron activity associated with Mayer waves,” Amer. J. Physiol., vol. 226, pp. 724–730, 1974. [9] B. Anderson, R. A. Kenney, and E. Neil, “The role of the chemoreceptors of the carotid and aortic regions in the production of Mayer waves,” Acta Physiol Scand, vol. 20, pp. 203–220, 1950. [10] A. C. Guyton and J. H. Harris, “Pressoreceptor-autonomic oscillation: A probable cause of vasomotor waves,” Amer. J. Physiol., vol. 165, pp. 158–166, 1951. [11] A. Malliani, Principles of Cardiovascular Neural Regulation in Health and Disease. Norwell, MA: Kluwer Academic, 2000. [12] J. U. Meyer, L. Limdbom, and M. Intiglietta, “Coordinated diameter oscillations at arteriolar bifurcation in skeletal muscle,” Amer. J. Physiol., vol. 253, pp. H568–H573, 1987. [13] A. L. Goldberger, “Non-linear dynamics for clinicians: Chaos theory, fractals and complexity at the bedside,” Lancet, vol. 347, pp. 1312–1314, 1996. [14] A. Voss, J. Kurths, H. J. Kleiner, A. Witt, N. Wessel, P. Saparin, K. J. Osterziel, R. Schurath, and R. Dietz, “The application of methods of nonlinear dynamics for the improved and predictive recognition of patients threatened by sudden cardiac death,” Cardiovasc. Res., vol. 31, pp. 419–433, 1996. [15] D. Cysarz, H. Bettermann, and P. Van Leeuwen, “Entropies of short binary sequences in heart period dynamics,” Amer. J. Physiol., vol. 278, pp. H2163–H2172, 2000. [16] S. M. Pincus, “Heart rate control in normal and aborted-SIDS infants,” Amer. J. Physiol., vol. 33, pp. R638–R646, 1993. [17] A. Porta, S. Guzzetti, N. Montano, M. Pagani, V. K. Somers, A. Malliani, G. Baselli, and S. Cerutti, “Information domain analysis of cardiovascular variability signals: Evaluation of regularity, synchronization and co-ordination,” Med Biol Eng Comput, vol. 38, pp. 180–188, 2000. [18] S. M. Pincus and A. L. Goldberger, “Physiological time-series analysis: What does regularity quantify,” Amer. J. Physiol., vol. 266, pp. H1643–H1656, 1994. [19] A. Porta, G. Baselli, D. Liberati, N. Montano, C. Cogliati, T. GnecchiRuscone, A. Malliani, and S. Cerutti, “Measuring regularity by means of a corrected conditional entropy in sympathetic outflow,” Biol. Cybern., vol. 78, pp. 71–78, 1998. [20] A. Papoulis, Probability, Random Variables and Stochastic Processes. New York: McGraw-Hill, 1984. [21] J. Theiler, S. Eubank, A. Longtin, and J. Galdrikian, “Testing for nonlinearity in time series: The method of surrogate data,” Physica D, vol. 58, pp. 77–94, 1992. [22] D. Roach, P. Malik, M. L. Koshman, and R. Sheldom, “Origins of heart rate variability: Inducibility and prevalence of a discrete, tachicardic event,” Circulation, vol. 99, pp. 3279–3285, 1999. [23] T. E. Brown, L. A. Beightol, J. Kob, and D. L. Eckberg, “Important influence of respiration on human RR interval power spectra is largely ignored,” J. Appl. Physiol., vol. 75, pp. 2310–2317, 1993. [24] M. Pagani, F. Lombardi, S. Guzzetti, O. Rimoldi, R. Furlan, P. Pizzinelli, G. Sandrone, G. Malfatto, S. Dell’Orto, E. Piccaluga, M. Turiel, G. Baselli, S. Cerutti, and A. Malliani, “Power spectral analysis of heart rate and arterial pressure variabilities as a marker of sympatho-vagal interaction in man and conscious dog,” Circ. Res., vol. 59, pp. 178–193, 1986.

Alberto Porta was born in 1964. He graduated in electronic engineering from Politecnico di Milano, Milan, Italy, in 1989. He received the Ph.D. degree in Biomedical Engineering at Politecnico di Milano, Milano, in 1998. He was a Research Fellow on Automatic Control and System Theory at the Dipartimento di Elettronica per l’Automazione, Universita’ di Brescia, Brescia, Italy, from 1989 until 1994. Since 1999, he has been with the Dipartimento di Scienze Precliniche, Universita’ degli Studi di Milano, Milan, Italy. Since 1999, he has taught a course on biomedical instrumentation at the Dipartimento di Elettronica per l’Automazione, University of Brescia, Brescia, Italy. His primary interests include time series analysis, nonlinear dynamics, system identification, and modeling applied to cardiovascular control mechanisms.

Stefano Guzzetti was born in 1954. He graduated in medicine and surgery from the University of Milan, Milan, Italy, in 1979 (magna with laude). He was specialized in cardiology in 1981 and in medical statistics in 1984 at the University of Milan. He is presently Senior Registrar with the Division of Internal Medicine II at the “L. Sacco” Hospital, Milan, Italy. He is the Referent of the “Heart Failure” outpatient clinic at the “L.Sacco” Hospital. He published more than 60 scientific papers in the areas of the neural regulation of cardiovascular functions with particular attention to the neural physiopathological mechanisms involved in heart failure.

Nicola Montano was born in 1963. He graduated in medicine at the University of Milan, Milan, Italy, in 1988. He received the Ph.D. degree in clinical physiopathology from the University of Milan in 1994. He is presently Established Investigator of Internal Medicine at the University of Milan. His research interests are in the field of neural control of cardiovascular function and are mainly related to recordings of neural autonomic activities in animal and humans and to the relationship between neural and cardiovascular oscillatory patterns.

Raffaello Furlan was born in 1954. He graduated in medicine from the University of Milan, Milan, Italy. Presently, he is Senior Registrar in Internal Medicine and Nontenure Professor of the postgraduate course in clinical psychology at University of Milan, since 1991. From 1977 to 1984, he took part in animal studies aimed at evaluating the role of cardiac and vascular sympathetic excitatory reflexes regulating systemic arterial pressure. Since 1982, he is involved in clinical studies aimed at assessing the changes in the neural mechanisms controlling the cardiovascular system in different physiological and pathophysiological conditions including the gravitational stimulus, physical exercise, athletic training, shift-work, syncope, dysautonomia, active ulcerative colitis, and others. During a sabbatical at the Clinical Research Center of the Vanderbilt University, Nashville, TN, in 1995, he was External Advisor for a program project grant aimed at clarifying the pathophysiology of chronic orthostatic intolerance, which was funded the following year. He is in charge of the “Syncope and Orthostatic Intolerance Unit “ which is part of an international web, organized by the National Dysautonomia Research Foundation (NDRF), linking centers with recognized expertise in the different forms of dysautonomia.


Massimo Pagani is Professor of Internal Medicine at the University of Milano Medical School, Milan, Italy. His main interests are neural control of the circulation, examined in intact conscious animals and men, and computer applications to assess invasively and noninvasively hemodynamic performance of the heart and of the vasculature. He has contributed to clinical applications of computer analysis of cardiovascular variabilities in vaious fields as health promotion in ambulant patients, prognostic evaluation in coronary artery disease, and telematics application to arterial hypertension treatment.

Alberto Malliani was born in 1935. He graduated in medicine from the University of Siena, Siena, Italy in 1959. From 1965 to 1967, he was Fellow of the Public Health Service at Columbia University, New York. During the early 1970s, he was Visiting Professor at universities in Utah and Texas. He is presently Full Professor of Internal Medicine at the University of Milan and is the head of the Department of Internal Medicine II at the “L. Sacco” Hospital. He has published more than 200 full papers that have appeared in the most prestigious journals and in textbooks such as Handbook of Physiology and Textbook of Pain. Among these articles, editorials in Circulation, the American Heart Journal, the British Heart Journal, and Cardiovascular Research and other minor Journals. His articles have received more than 6500 quotations in current literature. Dr. Malliani received the award for “Student of the Year” in the Faculty of Medicine from the University of Siena in 1959. In 1980, the European Society for Clinical Investigation awarded Dr. Malliani the “Mack-Foster Award”. He is President of the Società Italiana di Medicina Interna. He was President of the XXI International Congress of Neurovegetative Research. He is a member of the Editorial Board of Circulation, of the Autonomic Neruroscience and of other minor journals.

1291

Sergio Cerutti (M’81–SM’97) is Professor and Chairman of Biomedical Engineering (BME) of the Department of Biomedical Engineering at the Polytechnic University, Milan, Italy. He is Chairman of the Bachelor Degree (Diploma Universitario) in BME, and Chairman of the Master Degree (Laurea) in BME at the same University. He has also taught courses at a graduate level on biomedical signal processing at engineering faculties (Milano and Roma) and at specialization schools of medical faculties (Milano and Roma). Dr. Cerutti was an Elected Member of IEEE-EMBS AdCom (Region 8) from 1993–1996. He has been member of the International Committee of the various Annual Conferences of EMB Society since 1988. He is a member of the Steering Committee of the IEEE-EMBS Summer School on Biomedical Signal Processing; he was the local organizer of the I and III IEEE-EMBS Summer Schools held in Siena, Italy, on July 1995 and on June 1999, respectively. He is a Member of Regional Conference Committee of IEEE-EMBS. He is an Associate Editor of IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING and serves on the EditorialBoard of various other scientific international journals. In the last ten years he has published more than 200 papers in the area of biomedical engineering [main topics: biomedical signal processing (ECG, blood pressure signal and respiration, cardiovascular variability signals, and EEG and evoked potentials) and cardiovascular modeling, neurosciences, medical informatics, etc].