# Mathematical Software - Chaotic Systems - Tutorial

## Chaotic Systems - Introduction

### Chaotic Systems - Definitions

A criterion of chaotic behavior of the system in time is exponential instability of phase trajectories and a fractal dimension of its attractor. The latter means that a chaotic attractor demontrates a geometric complexity.

The time evolution of the state of a system with a finite number of degrees of freedom is described by either a system of ordinary differential equations

dxi / dt = fi(x1, …, xN, α1, … αk)

or discrete maps

 xin+1 = fi(x1n, …, xNn, α1, … αk) i = 1,2, … N.

where xi(t) (or xin) are variables uniquely describing the system state which can be seen as a point in state space ℜN (phase point). αj denote system parameters and fi are, in general, non-linear functions.

For time-continuous systems the system state and the evolution operator are specified for any time moment.

For discrete time map the system state and the evolution operator are defined only at discrete time moments.

If the evolution operator depends implicitly on time, the corresponding system is autonomous e. g.

dx/dt = F(x)

otherwise i. e. it contains external forces depending explicitly on time the system is called nonautonomous e. g.

dx/dt = F(x) + U(t).

The set of all possible states of the system is called its phase space of dimension N. If xi are dynamical variables not functions of some variables yj and N is finite this dynamical system is called a lumped system. When a system is infinite-dimensional then it is called a distributed system. As a rule yj variables of such a system denote spatial coordinates. Distributed systems are often represented by a system of partial differential equations.

The motion of a phase point, starting from some initial point x(t0) and followed as t → ±∞, represents a phase trajectory (or phase orbit) which shows the time evolution of a state of the system.

A set of characteristic phase trajectories in the phase space represents the phase portrait of the dynamical system.

For conservative dynamical systems, the volume in phase space occupated by phase points is conserved during time evolution while for dissipative dynamical systems the volume is usually contracted due to energy dissipation.

Graphical profiles of phase portraits even for three-dimensional systems are difficult to study thus it is helpful to use the Poincare method that constitute a tool allowing to simplify phase portraits.
In a poincare map, finite sequences of points (periodic orbits of the map) correspond to closed curves (limit cycles of the initial system) and infinite sequences of points correspond to non-periodic trajectories.

### Solutions of Dynamical Systems

There are a few limit sets for dynamical systems realize in the phase space. They can be equivalently classified according to

• types of limit sets of dynamical systems,

• or the stability property of phase trajectories belonging to limit sets.

#### Limit Sets

##### Attractor

Attractor definitions:

• … a set A is called … invariant … if PhitA = A.
Neighbourhood of a set A is an open set U containing the closure of A. i.e. A together with all its limits points including boundary points. …
A closed invariant set A is called an attracting set if there exists its neighbourhood U such that Phit(x) → A for all x ∈ U and t → ∞. A maximal U satisfying this definition is called basin attraction of A. … An attracting set containing an everywhere dense orbit is called an attractor A. (Malinetsky and Potapov, 2000, pp. 76-77)

• An attractor is the least set to which almost all orbits of a dynamic system from some area of non-zero volume tend.
A set of points in a phase space from which a system gets to a certain attractor is called basin of attraction. (Bezruchko and Smirnov, 2010, pp. 32)

##### Repeller

If points of U belong to A in the limit t → -∞, set A is called repeller. It means that points belonging to U tend to A backward in time.

If U is a sum of two subsets U = V ∪ W and points belonging to V tend to L forward in time whereas points belonging to W go to L backward in time then L is called a saddle (or a saddle limit set). The sets V and W are the stable and unstable manifolds of the saddle, respectively.

These sets listed-above (attractor, repeller or saddle) can exist only in a phase space of a dissipative dynamical system (a system with friction in physics). For conservative systems (friction-free systems in physics) the limit set is of a center-type i.e. for which U = A.

##### Separatrices

Separatrices divide the phase space into regions with principally different behavior of phase trajectories.

#### Stability Property

##### Stability of equilibrium states
• Singular points (or fixed points, equilibrium points) are points of equillibrium where a state of a dynamical system remains constant for arbitrary long time.

• Other types of equilibrium state can be divided into two different classes with respect to stability property of motion. To identify stable or unstable behavior analysis of the Lyapunov characteristic exponents (LCF) spectrum is needed.

• stable focus and stable node (attractor) - according to the Lyapunov criterion the LCF spectrum has only negative exponents that ensures asymptotical stability in all directions as it is required

• unstable focus and unstable node (repeller) - positive exponents in the LCF spectrum give unstable behavior in all directions

• saddle - there are both positive and negative exponents in the LCF spectrum

##### Stability of periodic states

A limit cycle is a closed curve, an image of a motion repeating itself with some period T (one of the LCE spectrum exponents is always zero). It can be an attractor (other exponents are negative), a repeller (other exponents are positive) or a saddle (other exponents have different signs).

##### Stability of quasi-periodic states

A toroidal limit set is an image of a quasi-periodic motion (e.g. two-dimensional torus has two characteristic periods whose ratio is an irrational number); k-dimensional torus has LCF spectrum of quasi-periodic trajectories with k zero exponents. It is an attractor if all other exponents are negative or repeller if they are positive. If LCF spectrum has, besides zero, both negative and positive exponents it is called saddle.

##### Stability of chaotic states

A fractal set concetrated in a bounded area of a phase space being an image of chaotic oscillations is called a strange attractor. A chaotic trajectory (belonging to attractor, repeller or saddle set) is always unstable at least in one direction. It means that LCF spectrum of chaotic system has always at least one positive Lyapunov exponent. Phase trajectories starting from close initial points in the basin of attraction tend to the attractor but they are separated on it.

##### Solution vs. Dimension

A phase space of dimension D = 1 can contain only points of equilibrium, D = 2 points of equilibrium and limit cycles, D ≥ 3 all types of limit sets.

### Time - Delay Techique

Let us consider a scalar time series {ζi}i=1N, ti = i Δt. One can construct the following D-dimensional model state vectors from the time series

{ xi = (ζi, ζi+m,…, ζi +(D-1)m) } i=1N-(D-1)m

where D is the model dimension and the time delay is τ = m Δt. In that way an observable ζ on a set M (i. e. where the considered motion takes place) of a dimension d in the phase space is mapped into a vector x on ℜD by an unique mapping

Ψ: M → ℜD

The space ℜD containing the image Ψ(M) is called embedding space. An equivalent description of the dynamics is achieved for sure only if the state vectors are constructed in the space of the dimension

D > 2d

This is the sufficient condition (formulated by Takens) but not necessary. It means that for the appropriate choice of the dynamical variable(s) describing the motion a good reconstruction can be obtained even for lower D.

Selections of model dimension D and the time delay τ are described below.

### Chaotic Systems - False Neighbours Technique

It gives an integer-valued estimate of the attractor dimension. Phase trajectory reconstructed in the space of sufficient dimension D must not exhibit self-intersections.
Let us consider a scalar time series {ζi}i=1N, ti = i Δt. The optfinderML program finds a single nearest neighbour for each vector xk

D = 1: xk = ζk
D = 2: xk =(ζk, ζk+m)
etc.

where D is the assumed model dimension and τ = m Δt is the time delay. After increasing D by 1 optfinderML program decides which neighbours are false and which ones are true. Then the ratio of the number of the false neighbours to the total number of the reconstructed vectors is plotted versus D. If the relative number of self-intersections decreases rapidly at some value than this value is taken as a trial model dimension.
Alternatively, the model dimension, where embedding of the original phase trajectory is achieved, can be find together with the correlation dimension making use of the Grassberger & Procaccia approach that is employed in the Tools-Chaotic Analysis-Embed & Correlation Dimension panel.

### Chaotic Systems - First Zero of the Autocorrelation Function

When time-delay reconstruction technique is applied than the time delay value must be set. If this value is too small it appears that components of a reconstructed vector are almost equal and reconstruction is useless. On the other hand, when the time delay is too large than vector coordinates are too distant and they become uncorrelated. Thus information of the geometrical structure of the reconstructed atractor is lost.
There are a few methods of choosing the value of the time delay. The most populat method is based on the autocorrelation function. Having computed autocorrelation function the time delay τ is taken as its first zero.
An user should only load autocorrelation function and the optfinderML program computes the time delay.

## Characteristics of Attractors

Phase portraits are characterized by a number of quantitative measures:

### Attractor's Geometrical Measures

Geometrical structure of chaotic (strange) attractors requires new measures called fractal dimensions:

• capacity - a limit set in D-dimensional phase space is covered with D-dimensional balls (or cubes etc) of ε radius (or edge-size). Capacity of a set is defined as the limit (if it exists) of the expression

DF = - limε → 0 ln(N(ε))/ln(ε)

where N(ε) is the minimal number of balls (cubes) sufficient to provide covering. Fractal sets mostly have a non-integer dimensions and they can be embedded into spaces of dimensions equals the smallest integer exceeding DF.

• generalized dimension of Renyi Dq - a limit set is partitioned into N non-empty cubes (balls) of size ε. A time spent by a representative point at the i-th ball (cube) is denoted as pi. It is a normalized density of points in a ball i.e. an estimate of the probability of a visitation to a ball. Then generalized dimension is defined as

 N(ε) Dq = 1/(q-1) limε → 0 ln ( ∑ pqi ) / ln(ε) i = 1

when q = 0 the generalized dimension is called capacity; if q → 1 information dimension; and at q = 2 correlation dimension. In the latter case, the quantity p2 is interpreted as a probability to find two representative points within the i-th ball.

• correlation dimension - is based on the correlation integral defined as
 N N C(ε) = 2/(N(N-1)) ∑ ∑ Θ(ε - || x(ti) - x(tj) ||). i=1 j=i+1

where Θ is the Heavyside function (Θ = 0, s ≤, 0; Θ = 1, s > 0) and ||.|| is a norm of a vector.

C(ε) ≈ A εD2

when ε → 0. Thus the correlation dimension D2 can be taken from the plot ln(C(ln(ε))).

### Attractor's Dynamical Measure

As a dynamical measure often are used Lyapunov exponents. They measure the speed of divergence or convergence of phase trajectories in the D-dimensional phase space and hence they characterize stability of the motion. The set of values Λi where i = 1, … D in descending order is called spectrum of Lyapunov exponents. Each value estimate a perturbation ε of a representative point after time interval τ from the phase trajectory since its initial deviation ε0 was weak. Let us denote ε(t0) = ε0. These quantities are given by the ratios of an ellipsoid semi-axis length to an initial radius of a sphere:

Λi = (1/τ) ln(||εi(t0 + τ)||/||ε0||)

in the limit where τ→∞ and ||.|| denotes the norm of a vector. The time evolution of ε can be expressed as

dε(t)/dt = A(t) ε(t)

where ε ∈ ℜD and A is a matrix of an order D. The solution reads

ε(t0 + Δt) = M(t0, Δt) ε0

where M is a matrix of an order D:

 t0 + Δt M(t0, Δt) = exp ( ∫ A(t')dt' ) t0

the matrix exponent is defined as formal expansion in a power series.

The singular value decomposition of the matrix M reads

M = U ΣVT.

where U (that is built from left singular vectors) and V (right singular vectors) are orthogonal matrices whereas Σ is diagonal. Action of the matrix M on the vector ε0 parallel to an i-th right singular vector vi leads to the expression

ε(t0 + τ) = σi ||ε0|| ui

where σi is an i-th singular value of the Σ matrix. Now ε vector is parallel to an i-th left singular vector. When one of singular values of M exceeds 1 in absolute value, then an initial perturbation rises for some directions. The local Laypaunov exponent shows how a perturbation changes

λi(t0, τ) = 1/τ ln(σi).

For a non-singular matrix M the orthogonal vectors u and v are of unit length. In the limit τ → ∞, it gives the expression for the Lyapunov exponents Λi presented-above.

## Machine Learning - OptFinderML

Package for machine learning - OptFinderML.