Ordinal Networks - Reconstructing continuous

Ordinal Networks Reconstructing continuous dynamical systems from time series with discrete transition graphs

Michael Small Complex Systems Group School of Mathematics and Statistics The University of Western Australia Mineral Resources CSIRO

Preamble

A dynamical system is a triple (T, M, ϕ) where ϕ : U ⊂ T × M → M, T is an additive monoid and M is a set. Think of T as time and M as space. The operator ϕt (z) defines the evolution of a point z ∈ M forward by time t. To make T sufficiently “time-like” we require: • ϕ0 (z) = z for all z ∈ M • ϕt+s (z) = ϕt ◦ ϕs (z) for all z ∈ M and t, s ∈ T Hence, ϕt (·) is the dynamical evolution operator. Typically, M will be a smooth manifold that is locally diffeomorphic to Rn .

1

An Experiment

Let Φ(z0 ) = {ϕt (z0 )|t ∈ T} be the trajectory for an initial condition z0 1 Let h : M → R be a measurement function and suppose that we can only observe xi = h(ϕiκ (z0 )) for i ∈ Z+ ∪ {0}. We call κ the sampling rate of our experiment. What can the time series {xi }Ni=1 tell us about ϕ?

1 We can, of course, think about trajectories backwards in time as well — and to do so is more typical.

Moreover, we

have yet to prove that Φ(z0 ) is well defined, but suppose for now that it is. 2

Takens’ Embedding Theorem (1981) The map xi 7→ (xi , xi+1 , . . . x(i+m−1) ) =: vi is an embedding of a compact manifold with dimension d ∈ Z+ (m = 2d + 1) if h : M → R is C2 and “generic”2 . Moreover, evolution of vi are diffeomorphic to the dynamics of ϕ. Corollary (Sauer, York, and Casdagli, 1981) Takens’ embedding theorem also holds for d ∈ R+ with the condition that m ≥ 2d + 1. Hence, a fractal attractor A ⊂ M can be reconstructed from a time series generated from a trajectory lying on that attractor.

2 Sufficiently well coupled between the d-dimensional variables and the theorem is then true almost always.

3

Delay reconstruction of experimental data

Suppose that a time series, as defined above, is the output of a deterministic and stationary (autonomous) dynamical system. Techniques to reconstruct the underlying attractor are now well established3 and widely used. This is usually achieved by estimating two parameters: embedding dimension m, and embedding lag τ and applying the map xi 7→ (xi , xi−τ , . . . x(i−(m−1)τ ) =: vi Can we do better?

3 See M. Small Applied Nonlinear Time Series Analysis, World Scientific, 2003.

4

Table of contents

1. Recurrence networks 2. Forbidden sequences 3. Ordinal Networks 4. Experiments

5

Recurrence networks

Recurrence

Poincaré Recurrence Theorem (Poincaré, 1890; Carathéodory, 1919) For a volume preserving dynamical system with continuous evolution operator ϕ and bounded trajectories, then for any open set V ⊂ M there exists orbits that come arbitrarily close to V infinitely often. More generally, and attractor A ⊂ M of a dynamical system is a set of points to which trajectories converge — and in particular, strange attractors are the objects of central interest in the study of chaotic dynamics.

6

Recurrence Plots The recurrence matrix R(ϵ) is an N × N matrix with { 1 d(vi , vj ) < ϵ Rij = 0 otherwise where ϵ > 0 is a threshold and d(·, ·) is the usual Euclidean metric4 . An alternative definition5 of the recurrence matrix R(k) is to set { ∑ 1 k̸=i,j I (d(vi , vk ) < d(vi , vj )) < k Rij = 0 otherwise That is, we either choose neighbours within an ϵ-ball, or we select the k-nearest neighbours. 4 Marwan, Romano, Thiel, Kurths Physics Reports (2007) 438: 5 Eckmann, Kamphorst, Ruelle Europhysics Letters (1987) 5:

237 973–977. 7

a = 0.386

a = 0.398

a = 0.400

a = 0.411

8

Recurrence Networks

How can we bring networks6 in on the act? Clearly, we could treat A ≡ R(k) (or R(ϵ)) as an adjacency matrix:

a = 0.386

6 Apologies to all graph theorists:

a = 0.398

a = 0.400

a = 0.411

in what follows graph, network, node, vertex, edge and link are pairwise synonymous. 9

10

10

10

Motif superfamily periodic flow chaotic flow hyper-chaotic flow noisy periodic flow chaotic map

Compute the relative frequency of subgraphs of size n to create an order sequence of connected graphs of n nodes.7 But, can we do better?

hyper-chaotic map noise 7

Xu, Zhang, Small, Proceedings National Academy of Sciences USA (2008) 105: 19601-19605 11

Why does this work?

043101-11

A. Khor and M. Small

Observe that certain motifs are less likely to occur in a 1-dimensional manifold — these motifs become increasingly common as the local dimension of the dynamics increases. Chaos 26, 043101 (2016)

FIG. 9. Original time series above and their reconstructions below after being converted to k-nearest neighbour networks and then applying the inversion algorithm provided by Hirata et al. From left to right: chaotic Lorenz attractor (r ¼ 10, b ¼ 8/3, q ¼ 28); period 2 R€ossler attractor (a ¼ 0.1, b ¼ 0.1, c ¼ 6); period 6 R€ossler attractor (a ¼ 0.1, b ¼ 0.1, c ¼ 12.6); and chaotic R€ossler attractor (a ¼ 0.1, b ¼ 0.1, c ¼ 18).

pathological and is related to an under sampling of state vectors as opposed to a problem with the inversion algorithm. The period 6 attractor exhibits a very important dual stripped nature, with two very close strips remaining side by side for three periods. The distinction between the lines is particu8larly hard to see on the peak, as the plotting function is drawing straight lines between successive states on either side of 9the bump, obscuring the true dual stripped nature. On the other hard, the distinction is most evident at the furthermost region of the original attractor, where the two strips momentarily separate to reveal the fact that they are indeed two side by side and not one.

Hirata at al. 8 provided a constructive proof that a distance matrix (diffeomorphic to the original) could be obtained from a ϵ-ball recurrence plot. Time information added trajectories. Khor and Small9 extended this to recurrence networks showing there is !-recurrence contain no k-neighbour information about the system, although this is clearly not the case based upon the extensive empirical evidence. The resolution to this apparent paradox a metric which would map k-neighbours to ϵ-balls. lies in the examination of what information !-recurrence networks, as well as k-nearest neighbour networks, are truly capturing. Networks examine local structure of the systems.

Hirata, Horai and Aihara Eur. Phys. J.: Spec. Top. (2008) Each state vector shares an164 edge13–22 with only those other state in its immediate neighbourhood, either via a fixed Khor and Small Chaos (2016) 26 043101.vectors !-ball or by seeking k nearest neighbours. This fact can be exploited by forcing states that would normally be far away in a Euclidean sense to be close to one another by curving space.

12

Forbidden sequences

Logistic map

Consider the logistic map xi+1 = F(xi ) = λxi (1 − xi ) for xi ∈ [0, 1] and 0 < λ ≤ 4. What is the possible ordering (i.e. sorted into relative magnitude) of x, F(x), F2 (x) and F3 (x)?

13

Logistic map

Consider the logistic map xi+1 = F(xi ) = λxi (1 − xi ) for xi ∈ [0, 1] and 0 < λ ≤ 4. What is the possible ordering (i.e. sorted into relative magnitude) of x, F(x), F2 (x) and F3 (x)?

13

Logistic map

Consider the logistic map xi+1 = F(xi ) = λxi (1 − xi ) for xi ∈ [0, 1] and 0 < λ ≤ 4. What is the possible ordering (i.e. sorted into relative magnitude) of x, F(x), F2 (x) and F3 (x)?

13

Logistic map

Consider the logistic map xi+1 = F(xi ) = λxi (1 − xi ) for xi ∈ [0, 1] and 0 < λ ≤ 4. What is the possible ordering (i.e. sorted into relative magnitude) of x, F(x), F2 (x) and F3 (x)?

13

Logistic map

Consider the logistic map xi+1 = F(xi ) = λxi (1 − xi ) for xi ∈ [0, 1] and 0 < λ ≤ 4. What is the possible ordering (i.e. sorted into relative magnitude) of x, F(x), F2 (x) and F3 (x)?

13

Ordinal Networks

Ordinal Networks Let {xi }Ni=1 denote the scale time series and choose a window size w ∈ Z+ . As before vi := (xi , xi+1 , xi+2 , . . . , xi+w−1 ) and denote by πi ∈ Sw a permutation of the integers 1, 2, . . . , w with the property that xi+k−1 (the k-th element of vi ) is the pii (k)-th largest element of {xi , xi+1 , . . . , xi+w−1 }.

4213

4123

I.e. πi encodes the rank ordering of the coordinates of vi . Each element in the time series xi is now associated with a particular window vi and a corresponding permutation πi . Construct a graph with each vertex associated with a permutation, and connect vertices if the corresponding windows occur in succession (non-overlapping). 14

G. 1: An illustration of the process for transforming a univariate time series into an ordinal network: (a) Take t first embedding vector with lag ⌧ and dimension m, (b) find its ordinal pattern which is the amplitude rank of it lements, (c) repeat the process for all embedding vectors from the time series, and (d) define a network A with 15 ode for each unique ordinal pattern, and edges assigned between nodes based on the temporal succession of ordin

Each ordinal sequence is a wedge of space bounded by hyperplanes bounded by xi = xj Associate with each ordinal label πi the set Qi = {(x1 , x2 , . . . xw )|xj is the(πi )j −th largest}. Then w! ∪

Qi = M and

Q◦j ∩ Q◦i = 0 iff i ̸= j.

i

Recall that the trajectory of z0 is observed via an experiment as . . . , ϕ−2κ (z0 ), ϕ−κ (z0 ), ϕ0 (z0 ), ϕκ (z0 ), ϕ−2κ (z0 ), . . . and so we can build the bi-infinite sequence {sn }∞ −∞ = {sn ∈ Sw |ϕnκ ∈ Qsn } and then Wij

=

µ (Qi ∩ ϕ−κ Qj )

for measure µ. W is the adjacency matrix of the weighted network, A the binary version (i.e. Aij = 1 iff Wij ̸= 0 and Aij = 0 otherwise). 16

Entropies Knowing that Wij = µ (Qi ∩ ϕ−κ (Qj )) it now reasonable to define the entropy (in one of several ways): • Permutation entropy hPE µ =−

1 ∑ Prob (π) log Prob (π) w − 1 π∈S w

• Conditional permutation entropy hCE µ =−

∑ ∑

Prob (ξ = ϕκ (π)) log

π∈Sw ξ∈Sw

Prob (ξ = ϕκ (π)) Prob π

• Topological entropy log [λmax (A)] → hTP 17

Periodic dynamics

“Clearly” periodic dynamics of maps of period p will lead to cyclic chain graphs of length (at most) p. Periodic dynamics of flows is complicated by sampling rate and the necessarily much larger w and hence w! possible symbols.10

What about chaotic dynamics? Apart from entropy, what more can be said?

10 Although, typically a much larger number of forbidden patterns too.

18

Nullity of A Computation of log Null(A) from time series as a function of the bifurcation parameter. 1 0.8

x$

0.6 0.4 0.2 0 3.5

3.6

3.7

3.8

3.9

4

r Logistic map 19

Nullity of A Computation of log Null(A) from time series as a function of the bifurcation parameter. 1.5 1

x$

0.5 0 -0.5 -1 -1.5 1.05

1.1

1.15

1.2

1.25

1.3

1.35

1.4

a Hénon map 19

Nullity of A Computation of log Null(A) from time series as a function of the bifurcation parameter. 2 1.5

x$

1 0.5 0 -0.5 0.65

0.7

0.75

0.8

0.85

7 Ikeda Map 19

Maximal Lyapunov Exponent Let u = (u1 , u2 , . . .) be the right eigenvector of the largest non-negative eigenvalue of the transpose of the transition matrix (i.e. normalise by incoming neighbours) and define the matrix P by µ(Qi ∩ ϕ−κ Qj ) uj Pij = . µ(Qj ) ui with associated left eigenvector vp . The largest Lyapunov exponent11 is12 LLE = −

w! ∑ i,j=1

( vP,i Pij log

µ(Qi ∩ ϕ−κ Qj ) µ(Qj )

)

11 Erstwhile, the sum of positive Lyapunov exponents. 12 Froyland.

Nonlinearity (1999) 12 79-101 20

h7P E (m) hC7 E (m) hSE 7 (m) Ordinal-based MLE Theoretical

1.5 1 0.5 0 0

5

10

15

2.5 h7P E (m) ^ P E (m) h 7

2

hC7 E (m) ^ C E (m) h 7 hSE 7 (m) ^ SE (m) h 7

1.5

Ordinal-based MLE Analytical Approx.

1 0.5 0

20

0

5

2.5 hP7 E (m) hC7 E (m) hSE 7 (m) Ordinal-based MLE Theoretical

2 1.5 1 0.5 0 0

5

10

m

10

15

2.5 h7P E (m) ^ P E (m) h 7

2

hC7 E (m) ^ C E (m) h 7

hSE 7 (m) ^ SE (m) h 7 Ordinal-based MLE Analytical Approx.

1.5 1 0.5 0

20

0

5

m max Lyapunov Exponent

max Lyapunov Exponent

m

15

20

2.5 hP7 E (m) hC7 E (m) hSE 7 (m) Ordinal-based MLE Theoretical

2 1.5 1 0.5 0 0

5

10

m

10

15

20

m max Lyapunov Exponent

2

max Lyapunov Exponent

max Lyapunov Exponent

max Lyapunov Exponent

2.5

15

20

2 hP7 E (m) ^ P E (m) h 7

hC7 E (m) ^ C E (m) h 7 hSE 7 (m) ^ SE (m) h 7 Ordinal-based MLE Analytical Approx.

1.5

1

0.5

0 0

5

10

15

20

m

21

Experiments

Ventricular Arrhythmia (Electrocardiogram)

Fig. 3.5 and 3.7

22

Electrocardiogram (RR-intervals)

Fig. 3.9 and

23

Epileptic Seizure detection and prediction (Electroencephalogram)

Fig. 4.9.

24

Epileptic Seizure detection and prediction (Electroencephalogram)

Fig. 4.9 and 4.13.

25

Acknowledgements

26

Questions?

26