NONLINEAR DYNAMICS AND CHAOS LECTURE 1 - University of ...

LECTURE 1: INTRODUCTION

1. Introduction

NONLINEAR DYNAMICS AND CHAOS

2. Maps 3. Flows 4. Fractals and Attractors

Patrick E McSharry Systems Analysis, Modelling & Prediction Group

5. Bifurcations 6. Quantifying Chaos

www.eng.ox.ac.uk/samp

7. Nonlinear Time Series Analysis

[email protected]

8. Nonlinear Modelling and Forecasting

Tel: +44 20 8123 1574

9. Real-World Applications

Trinity Term 2007, Weeks 3 and 4 Mondays, Wednesdays & Fridays 09:00 - 11:00

10. Weather Forecasting 11. Biomedical Models 12. Time Series Analysis Workshop

Seminar Room 2 Mathematical Institute University of Oxford

c 2007 Patrick McSharry – p.1 Nonlinear dynamics and chaos °

Suggested Reading


Relevant journal articles

Strogatz, S. H., Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry, and Engineering, Reading, MA: Addison-Wesley (1994)

Yorke, J. and Li, T. Y., Period Three Implies Chaos, American Mathematical Monthly 82:985-992 (1975)

Eubank, S., and D. Farmer, An introduction to chaos and randomness. In Jen, E. (Ed.), 1989 Lectures in Complex Systems. Santa Fe Institute Studies in the Sciences of Complexity, Lecture Vol. II, pp. 75-190. Reading, MA: Addison-Wesley, (1990)

May, R., Simple mathematical models with very complicated dynamics. Nature 261: 459-467 (1976)

Ott, E. and Sauer, T. and Yorke, J. Coping with Chaos, J. A. John Wiley & Sons, New York (1984) Ott, E., Chaos in Dynamical Systems, Cambridge: Cambridge University Press (1993)

Packard, N. and Crutchfield, J. and Farmer, J. D. and Shaw, R., Geometry from a time series, Phys. Rev. Lett. 45: 712-716 (1980) Crutchfield, J. P, N. H. Packard, J. D. Farmer, and R. S. Shaw. (1986) Chaos, Scientific American 255:46-57 (1996)

Schuster, H.G., Deterministic Chaos: An Introduction, VCH, (1995). Wiggins, S., Introduction to Applied Nonlinear Dynamical Systems and Chaos, Springer (1990) Kantz, H. and Schreiber, T., Nonlinear Time Series Analysis, Cambridge Univ. Press (1997) Abarbanel, H.D.I, Analysis of Observed Chaotic Data, Springer, (1996)



Sources of information

Suggested special topics

Journals: Physica D Physical Review Letters Physical Review E Physics Letters A International Journal of Bifurcation and Chaos

Look for evidence of low-dimensionality in real time series Investigate the predictability of a real time series using nonlinear methods

Online discussion groups: sci.nonlinear www.jiscmail.ac.uk/lists/allstat.html www.jiscmail.ac.uk/lists/timeseries.html Websites: www.societyforchaostheory.org (Society for chaos theory) www.physionet.org (MIT-Harvard biomedical database and tools) www.comdig.org (Complexity Digest)

Available time series: electronic circuits lasers sunspot record electricity data (demand, price, grid frequency) weather data (temperature, precipitation, wind speed) economic data (GDP, inflation, interest rates, unemployment) financial data (stockmarket tick data, S&P500, Dow Jones, FTSE100) biomedical signals (ECG, EEG, blood pressure, respiration, blood gases) Construct and investigate a nonlinear model of a particular system Compare nonlinear methods for forecasting, (e.g. RBFs vs. local linear) Develop new methods for classification of health/disease using biomedical signals TISEAN package Matlab software


Special topic instructions


Linear analysis Definition of linearity: L(ax) = aL(x)

Should be one week of work

L(x + y) = L(x) + L(y)

Marking scheme:content 20 pts presentation 5 pts

Principle of superposition: If x and y are solutions, then z = ax + by is also a solution Advantages of linear models: Often have analytical solutions A large body of historical knowledge for helping with model specification and estimation Less parameters - smaller chance of overfitting Can employ Fourier spectral analysis and associated techniques

Two copies of final report required

Disadvantages of linear models: Real-world systems are usually nonlinear Linearity is a first order approximation and neglects higher orders Stanislaw Ulam: nonlinear science is like non-elephant zoology In practice, while underlying dynamics may be nonlinear, observed data may only provide sufficient resolution for linear models Need relevant null hypothesis tests for nonlinearity (surrogate data)



Normal distributions

Dynamical systems I Variables

Linear

n=1

Growth, decay, equilibrium

Mean x0 and variance σ 2 : −(x − x0 )2 exp p(x) = √ 2σ 2 2πσ 2 1

»

–

Easily manipulated: if x ∼ N (0, σx2 ) and y ∼ N (0, σy2 ), then x + y ∼ N (0, σx2 + σy2 ) Central limit theorem: sum of a large number of IID random variables (with finite mean and variance) is normally distributed For linear models: Normal distributions are preserved by principle of superposition Normally distributed forecast errors: Maximum likelihood gives least squares Useful for calculating prediction intervals Problems for nonlinear systems: Use of normal distributions neglects possibility of asymmetric distributions Fat tailed distributions imply larger probability of worse case scenarios (risk management)

Exponential growth

Fixed points

RC circuit

Bifurcations

Radioactive decay

Overdamped systems, relaxational dynamics Logistic equation for single species

n=2

All higher order moments are given in terms of x0 and σ

Nonlinear

Oscillations Linear oscillator

Pendulum

Mass and spring

Anharmonic oscillator

RLC circuit

Limit cycles

2-body problem

Biological oscillators Predator-prey cycles Nonlinear electronics (van Der Pol)

n≥3

Chaos Civil engineering

Strange attractors (Lorenz)

Electrical engineering

´ 3-body problem (Poincare) Chemical kinetics Iterated maps (Feigenbaum) Fractals (Mandelbrot) Forced nonlinear oscillators (Levison, Smale)


Dynamical systems II Variables

Linear

nÀ1

Collective phenomena


Napolean’s Army’s Russian Campaign Nonlinear

Coupled harmonic oscillators

Coupled nonlinear oscillators

Solid-state physics

Lasers, nonlinear optics

Molecular dynamics

Nonequilibrium statistical mechanics

Equilibrium statistical mechanics

Nonlinear solid-state physics Heart cell synchronisation Neural networks Economics

Continuum

Waves and patterns

Spatio-temporal complexity

Elasticity

Nonlinear waves (shocks, solitons)

Wave equations

Plasmas

Electromagnetism (Maxwell)

Earthquakes

¨ Quantum mechanics (Schrodinger, Heisenberg)

General relativity (Einstein)

Heat and diffusion

Quantum field theory

Acoustics

Reaction-diffusion, biological waves

Viscous fluids

Fibrillation Epilepsy Turbulent fluids

Map drawn by the French engineer Charles Joseph Minard in 1861 to show the tremendous losses of Napolean’s army during his Russian Campaign of 1812

Life c 2007 Patrick McSharry – p.11 Nonlinear dynamics and chaos °


Happiness time evolution

Happiness versus GDP

From Culture and Subjective Well-being, edited by Ed Diener & Eunkook M. Suh (2002) From Culture and Subjective Well-being, edited by Ed Diener & Eunkook M. Suh (2002) c 2007 Patrick McSharry – p.13 Nonlinear dynamics and chaos °

Time series from nonlinear systems


Mathematical characterisation

Time series: variables are recorded as a function of time A

Steady state: a constant solution of a mathematical equation e.g. Homeostasis: relative constancy of the internal environment with respect to variables such as blood sugar, blood gases, blood pressure and pH. Control mechanisms constrain variables to narrow limits. e.g. Following a hemorrhage, reflex mechanisms quickly restore blood pressure to equilibrium values.

B

C

Oscillations: periodic solutions of mathematical equations e.g. Heartbeat, respiration, sleep-wake cycles and reproduction Irregular activity: intrinsic fluctuations, can even be present when external parameters are relatively constant. Two distinct mathematical descriptions: noise and chaos

D

Noise: Variability cannot be linked with any underlying stationary or periodic process e.g. Fluctuating environment: eating, exercise, rest and posture affects heart rate, blood pressure, blood-sugar levels and insulin levels e.g. Respiratory Sinus Arrhythmia (RSA): heart rate increases during inspiration

E

F

Chaos: Irregularity that arises in a deterministic system Chaos can exist without influence of external noise

G



Noise versus Chaos

An exponential distribution, but not a Poisson process 2

Inter-event time sequences (e.g. heartbeat, neurons firing)

1.5

Believed to be a random process

ti+1

Simplest model for such a random process is a Poisson process: Probability of event to occur in a very short time increment dt is Rdt The probability R is independent of the previous history Probability of two or more events occurring during dt is negligible (Rt)k −Rt e k! st 1) following

(Poisson distribution)

Probability that interval between event and (k +

event is

R(Rt)k −Rt e k!

0.5

0

0

0.1

0.2

0.3

0.4

0.5 ti

0.6

0.7

0.8

0.9

1

90

100

0.9

1

ln2/3 1.4 1.2

Probability of k events in time interval t is Pk (t) = pk (t) =

1

1 ti

0.8 0.6

(PDF of Poisson process)

0.4 0.2

Average time between events is 1/R and variance is 1/R2

0

Nonlinear map, ti+1 = −(1/R) ln |1 − 2 exp(−Rti )| also gives p(t) = Re−Rt

0

10

20

30

40

50 i

60

70

80

0

0.1

0.2

0.3

0.4

0.5 t

0.6

0.7

0.8

4

Observation of an exponential probability density is not sufficient to identify a Poisson process!

3.5 3 2.5

Use recurrence plots (e.g. ti+1 versus ti ) to identify structural equations

p(t)

2 1.5 1 0.5 0


Determinism and Predictability


Poincaré

After Newton people believed in a deterministic, and hence, predictable Universe “Given for one instant an intelligence which could comprehend all the forces by which nature is animated and the respective situation of the beings who compose it—an intelligence and sufficiently vast to submit these data to analysis—it would embrace in the same formula the movements of the greatest bodies of the universe and those of the lightest atom; for it, nothing would be uncertain and the future, as the past, would be present before its eyes.” [P.S. Laplace, 1814] “Laplacian dream” excludes stochastic laws of physics Laplace acknowledged that we would never achieve the “intelligence” required—a tacit appreciation that deterministic systems might not, in practice, be predictable deterministic 6= predictable

After tackling the 3-body problem Poincarè identified the phenomenon of sensitive dependence on initial conditions (SDIC), this provided a definition of “chaos” “If we knew exactly the laws of nature and the situation of the universe at the initial moment, we could predict exactly the situation of that same universe at a succeeding moment. But even if it were the case that the natural laws had no longer any secret for us, we could still only know the initial situation approximately. If that enabled us to predict the succeeding situation with the same approximation, that is all we require, and we should say that the phenomenon had been predicted, that is is governed by laws. But it is not always so; it may happen that small differences in the initial conditions produce very great ones in the final phenomena. A small error in the former will produce an enormous error in the latter. Prediction becomes impossible, and we have the fortuitous phenomenon.”[H. Poincaré, 1903]

Laplace saw probabilities as a way to describe our ignorance of a deterministic system Analytic expediency means most mathematics revolves around linear systems



Lorenz’s butterfly effect

Sensitive dependence on initial condition

50

20

45

15 40 35

10

30 z

5

25 20

x(t) 0

15

−5

10 5

−10 0 −20

−15

−15

−10

−5

0 x

5

10

15

20

Perfect model and perfect knowledge of observational uncertainty Predictability varies with position

−20

0

1

2

3

4 time [secs]

5

6

7

8


Chaos


Two faces of Chaos

Advent of digital computer allowed numerical investigation of nonlinear equations

The word chaos refers to disorder and extreme confusion

Lorenz found SDIC in a numerical model of the atmosphere and constructed the “Lorenz system” to illustrate the effect in a simple system [1963]

To a scientist, it implies “deterministic disorder”

Yorke and Li coined the word “chaos” in 1975

On the contrary, a chaotic deterministic system is, in principle, perfectly predictable

May demonstrates chaos in the one-dimensional Logistic map in 1976

The sensitive dependence of the system dynamics to the initial conditions (SDIC) implies that, in reality, any error in specifying the initial condition will lead to an erroneous prediction

Chaos becomes trendy, “Chaos” is published by James Gleick, 1987 Claims of chaos in the brain, heart, economy, stockmarket, ... Investigations of nonlinear dynamical systems, claims of chaos are played down!

This might suggest that a chaotic system should be unpredictable

Laplace suggests using probabilistic predictions to overcome the problems, of diverging trajectories, posed by chaotic systems Chaos is sometimes used as a scapegoat: meteorologists blame chaos for inaccurate predictions when it is often model inadequacy that is at fault Important research: separating model inadequacy (structural and parametrical errors) from effects of observational uncertainty



Deterministic versus Stochastic

Deterministic versus Stochastic

stochastic system

Deterministic:

xt+1 = axt + εt

εt : N (0, 1)

deterministic system

Everything is described by a single point in state space. This description completely determines the future.

Stochastic:

Knowledge of present states does not determine the evolution of future states.

Can you tell if a system is stochastic or deterministic?

yt+1 = ayt + σzt

Should a system be modelled as stochastic or deterministic?

zt+1 = 4zt (1 − zt )

High dimensional deterministic chaotic system might be modelled as a stochastic system State space trajectory of an autonomous, deterministic system never crosses itself Sources of forecast uncertainty: Uncertainty in the initial condition Model inadequacy (parametrical and structural)


Introduction to Dynamical Systems


Properties of dynamical systems

A state is an array of numbers that provides sufficient information to describe the future evolution of the system. If m numbers are required, then these form an m-dimensional state vector x. The collection of these state vectors defines an m-dimensional state space. The rule for evolving from one state to another may be expressed as a discrete map or a continuous flow: map xt+1 = F (xt ) ˙ flow x(t) = f (x(t)) Fixed point of a map: x0 = F (x0 ) Fixed point of a flow: x˙ 0 = f (x0 ) = 0

Determinism: trajectories should not diverge when going forward in time Invertibility: A dynamical system is invertible if each state x(t) has a unique predecessor x(t − 1). This implies that trajectories should never merge. Thus continuous deterministic flows are always invertible! Maps derived from flows (Poincaré maps) are also invertible Reversibility: if the dynamical system obtained by the transformation t → −t is equivalent to the original one Invariance under coordinate transforms: an invariant of a dynamical system represents a fundamental property of that dynamical system, e.g. dimensions and Lyapunov exponents System invariant offer a means of summarising the behaviour of a particular system: (e.g. health and disease)

Non-autonomous system: x˙ = f (x, t) Autonomous system: x˙ = f (x) If the non-autonomous nature is due to periodic terms it can be made autonomous Dissipative flow: ∇ · f < 0 implies contracting state space volume

Hamiltonian systems are non-dissipative or conservative, preserving state space volume (Liouville theorem) c 2007 Patrick McSharry – p.27 Nonlinear dynamics and chaos °


Simple Pendulum

Fixed Points

An example of a simple two-dimensional dynamical system

Consider the fixed point of a flow f (x0 ) = 0

From Newtons’s second law, knowledge of the forces, position and velocity are sufficient to determine future motion

Let x(t) = x0 + ε(t) ε˙ = f (x0 + ε)

Pendulum (constrained to move in the plane)

ε˙ = f (x0 ) + Dx f (x0 )ε + O(||ε||2 )

˙ Dynamics fully specified by the displacement angle θ(t) and the angular velocity θ(t)

ε˙ ≈ Dx f (x0 )ε = Jε

˙ State vector given by x(t) = [θ(t), θ(t)] Let m be the mass of the pendulum

J is Jacobian matrix of partial derivatives

g is the acceleration due to gravity

The solution is ε(t) = eJt ε0

l is the length of the pendulum Let λi be (distinct) eigenvalues of J

Tangential restoring force due to gravity: −mg sin θ Tangential force due to angular acceleration: mlθ¨

P −1 JP = Λ

In the absence of friction, dynamics are governed by d θ dt d ˙ θ dt

=

θ˙

=

−

where Λii = λi and Λij = 0 if i 6= j Let ε = P y so y = eΛt y0

g sin θ l c 2007 Patrick McSharry – p.29 Nonlinear dynamics and chaos °

Classification of fixed points

NONLINEAR DYNAMICS AND CHAOS LECTURE 1 - University of ...

NONLINEAR DYNAMICS AND CHAOS LECTURE 1 - University of ...

Suggest Documents

NONLINEAR DYNAMICS AND CHAOS ... - University of Oxford

nonlinear dynamics and chaos strogatz.pdf