Introduction to computational methods in science and engineering ...

1

Introduction to computational methods in science and engineering using MATLAB Dr. rer. nat. Hans-Georg Matuttis University of Electro-Communications, Department of Mechanical and Control Engineering Chofu Chofugaoka 1-5-1 Tokyo 182-8585 Japan http://www.matuttis.mce.uec.ac.jp/, but 1. When something is unclear, ask me, not you neighbor, who is busy himself. Ask as much questions as you need. 2. The Script will be downloadable from http://www.matuttis.mce.uec.ac.jp/ or from the E-Learning system. You can read it online or print it out. If you print out more than 100 pages, you have to submit an application (signed by me) for more printout pages. Reading the script does not replace attending the lecture. 3. The homework is exercise to learn programming, you cannot learn programming by reading the script. 4. Learn to use the online-help from MATLAB 5. For the credit: Presence in the Lecture Number of Points in the programming homework and the E-Learning System

Getting Started 0.1

Why ”Computational Methods”

Most problems in Science and Engineering differ from undergraduate problems in the respect that no ”closed solutions” exist: Whereas there is a closed solution (solution function) for the harmonic oscillator with viscous damping, x¨ +

δ x˙$ !2 "#

+ω02 x = 0

Viscous damping

%

x(t) = A exp(−δt) exp ±i

&

ω02

−

'

δ2t

there is no closed solution for the harmonic oscillator with sliding friction (see graphics to the right) m¨ x + sgn (v) µFN +kx = 0, sgn(v) = !

"#

$

Sliding friction

v , |v|

but solutions can only be given piecewise for ranges where the friction is constant: %

'

π x(t) = −(∆x − x0 ) sin ωt + − x0 2 % ' π x(t) = −(∆x − 3x0 ) sin ωt + + x0 2

T 2

for

0≤t≤

for

T µFN ≤ t ≤ T, x0 = 2 k

For forces which are not linear in x and its derivatives, in general not even piece-wise solutions can be given. Other problems for which no solutions exists are problems with many degrees of freedom (e.g. planet systems), or flow problems. For the technically important fields of structural analysis and fluid mechanics, most results are nowadays obtained by computer simulations.

4 Fluid mechanics: Flow around a sphere with increasing Reynolds number/ flow speed: Analytical solutions exist only for the Stokes flow problem. Vortices Vortex Street Stokes Turbulence

0.2

New MAC-Installation

Since 4/2006, the exercises-room is equipped with MAC-computers instead of the old SUN-Unix-Workstation-Terminals. Software can either be started via the WindowIcons in the Applications-Directory, or via the command-line terminal, which gets started by clicking on the X-Icon. MATLAB can be started by clicking on the MATLABIcon. It is recommended for the Course to use the EMACS-Editor. Because the current MAC-OSX-Operating-System is based on the Unix-Operating system, the following comments on Unix are useful, last not least, because UNIX-commands (for directory listings, previewing of graphics, removal of unneeded data etc.) can be used from the MATLAB-prompt via ! as escape-sequence. WARNING! The new MAC-Installation allows the teacher to view the screen and the currently active programs in each student terminal. Applications which are unneeded for the lesson can be terminated from the teacher-console.

0.3

UNIX-Workstations

The MAC-computers in the terminal-room cannot be used remotely. If students want to login remotely from outside the terminal-room, the access is possible to the SUNcluster which has the name sun.edu.cc.uec.ac.jp, which consists neither of MAC’s nor PC’s, but UNIX-Workstations. The login (also possible from the MAC-computers) has to be via the secure shell and login by setting the X-terminal is possible as ssh -X sun.edu.cc.uec.ac.jp or ssh -Y sun.edu.cc.uec.ac.jp (the option -Y or -X depends on the version of the operating system one is logging on from). UNIX was originally written to be used from a commando-prompt window, not from GUI/Window systems. It is advisable to be able to use the original UNIXcommands. If you like UNIX and would like a UNIX-like environment on your PC, install the free CYGWIN-package, www.cygwin.com. Some survival-UNIX-commands:

0.3. UNIX-WORKSTATIONS

5

copy the file source to the file destination if destination exists, overwrite it with source cp -r source destination like copy, -r means recursive, works also with directories pwd display current directory mv source destination rename the file source to the file destination cd change directory ps x display existing jobs and their job-id kill job-id kills the job with the number job-id ls list directory content ls -d list all directories in the current directory ls -lrt list all files with information about their size, in the order in which the have been created, the newest ones a the end ls [a-c]* b list all files in this directory with names beginning with a,b or c find . -name thisname -print look for the file or directory thisname in the current directory and in all the subdirectories find . -name ’*arg*’ -print display all files and directories in in the current directory and in all the subdirectories which contain the string arg in their name fgrep asdf *.txt look for all lines which contains the string asdf in all files which have the extension .txt. UNIX is a multi-user multi-process operating system, so several uses can run commands at the same time on a single computer. It is also to move jobs in the background when starting them so that they don’t block the command prompt by appending the ampersand &. If a program was started in the foreground and blocks the prompt, it can be pushed into the background via CNTRL-Z and the execution will be interrupted. If in the same terminal window bg is typed, the execution is resumed. (Of course, this does not work with programs which have an input prompt in the foreground, like MATLAB). Some special directories: ./ the current ../ the directory below ~/ the users login direcdirectory the current directory tory Some special characters: * any string ? any single character [a-f] all of the characters of a,b,c,d,e,f The recommendation for this course is to work in a directory which is dedicated to MATLAB alone, and this directory is not the root-directory of the account. The name ml should do just fine: cp

source destination

mkdir ml cd ml

6

0.4 0.4.1

MATLAB Introduction: Interpreters and Compilers

In general, for the programming projects with high numerical complexity, it will be the best to develop the algorithms in MATLAB. MATLAB is, like BASIC or symbolic computer languages like MAPLE, MATHEMATICA, MACSYMA and REDUCE, an interpreter language, i.e. the language commands are translated into processor instructions. Nevertheless, MATLAB is not a symbolic language, but performs all calculations numerically, i.e. with floating point numbers.1 The language can be used either from a command prompt or as a functional (or object-oriented) programming language. In compiler languages like FORTRAN, C or PASCAL, the program is fully translated into processor instructions before execution. If errors occur at runtime, the memory contents is difficult to analyze, usually only with the help of a debugger, which may alter the program execution and memory layout up to the point that some errors cannot reproduced. The debuggers properties vary much more than the language itself. In MATLAB, after a program crash, the data are still accessible in MATLAB’s memory and can be analyzed using the commands from the MATLAB-language itself. Interpreters allow fast program development. As a rule, their execution times are higher than those of compiler languages, but during program development, usually the compile time is more costful than the actual runtime. In MATLAB, when complex builtin functions are initialized via small commands, like a matrix inversion, very often the advantage in speed for the compiler languages is negligible. Many programming languages have a whole zoo of data types. MATLAB’s elementary elementary data type is the complex matrix. (Recently, MATLAB also offers more kinds of data types, but we will not use them in this course). Variables can be processed up to the point where they take a complex value. Variables which are used as indices must nevertheless have an integer value. Because it is not possible to declare variables in MATLAB, is refuses to process variables which are not initialized. In FORTRAN77, for example, it was possible to use variables which were neither declared nor initialized, and which assumed the value 0 at the moment they were used.

0.4.2

Getting started

The MATLAB-Interpreter is started on our installation by by typing matlab at the command prompt, which starts the MATLAB-desktop. If you are busy and you don’t want to see the splash-screen (MATLAB-Commercial) at the program start, use matlab -nosplash 1

The symbolic package available with MATLAB is basically MAPLE with a MATLAB-Interface.

0.4. MATLAB

7

Basic Commands: edit starts the MATLAB-Editor with Syntax-highlighting of MATLABcommands. You can use any editor you like to write MATLAB-files, but the line-end may vary between operating systems and may lead to trouble clear empties the memory clear a clear the variable a from the memory who displays the variables which have been assigned help gives help concerning a specific topic help help tells you how to use the help function lookfor looks for a word in the help files, useful if you are looking for a command according to context, but are not sure about the command name disp(a) displays the value of the variable a disp(’a’) displays just the string, a. rand random number generator, will be used a lot to initialize data format format the output, format compact suppresses output of empty lines, format short forces the rounding of the output to eight digits, but the computations are still performed with full precision % comment sign ls displays the current working directory of MATLAB, i.e. the directory for which MATLAB can access the files directly cd changes the current working directory of MATLAB The MATLAB-desktop is written in JAVA (another interpreter-based programming language), which has still some stability problems2 , so the desktop crashes relatively often. If you don’t want to work with the desktop to avoid unnecessary crashes, but want to write the programs in a Unix-editor you know, you can also start MATLAB with the command-prompt only as

matlab -nosplash -nodesktop

To get an idea why the JAVA-Interface of MATLAB crashes so often, see the internal memo from SUN from http://www.internalmemos.com/memos/memodetails.php?memo id=1321 2

8 Special Characters: ! escape sequence, allows to use UNIX-Commands like cd, pwd from the MATLAB-prompt [...] 1. vector brackets referring to the value of the entry, [1 2 3] is a vector with the entries 1, 2 and 3. 2. brackets referring to the output arguments of functions. (...) 1. Brackets referring to the indices of a vector, a(3) is the third element of the vector a 2. brackets referring to the input arguments of a function. ... three dots mark the end of a line which is continued in the next line ; has no syntactical function like in C but is only used to suppress the output of the operation √ i, j stands for the complex increment −1, but can also be overwritten for other uses. pi is indeed 3.1415.... , divide commands, when several command lines should be written in the same editor line : divide loop variables, lower_bound:stepwidth:upper_bound, WARNING ! lower_bound,stepwidth,upper_bound only displays the variables lower_bound, stepwidth, upper_bound As a first reference, Kermit Sigmons MATLAB primer at http://math.ucsd.edu/∼driver/21d-s99/matlab-primer.html can be recommended. It gives a short overview over available commands, but it is a good idea to get used to the builtin help-function of matlab (just type help from the prompt). For most purposes, the internal help is sufficient. Manuals for MATLAB are available, but there is not much information which ones needs beyond the builtin help command on a daily basis, except the references to the algorithms used. This is a huge difference to e.g. MATHEMATICA, where the algorithms are ”secret”. Beware, in contrast e.g. to FORTRAN, MATLAB is case sensitive, ABC is not the same as abc. If you used the same variable names in lower case and upper case in the same programs, you will run into trouble anyway. Information about a public-domain clone of MATLAB, OCTAVE, can be found at www.octave.org. Control statements are usually terminated via the end command, no matter whether it is an if statement or a for loop: a=2 b=3 for i=1:10 if (a>b) i disp(’a>b’) end else disp(’a1) myeps=0.5*myeps; myepsp1=1+myeps; end myeps Other builtin functions which are convenient to get ideas about the feasibility of some numerical algorithm are realmax, the largest representable floating point number, and realmin, the smallest floating point number which is larger than 0. All these functions eps, realmin, realmax are implementation dependent, i.e. their result may be different on different computer models, because the mathematical operations are ”wired” in a different way on the chip. The actual number of valid digits of mantissa and exponent are usually not defined in

3.3. CHECKING FOR EQUALITY

33

language-Standards, so that the IEEE-Standard (IEEE= Institute of Electrical and Electronics Engineers) uses for double precision S EEEEEEEEEEE FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF 0 1 11 12 63 with the sign S, the exponents E and Mantissa digits F. whereas CRAY used something like S EEEEEEEEEEEEEEEEE FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF 0 1 17 18 63 which due to the lower accuracy and other idiosyncrasies in rounding, has now totally vanished. For most numerical computations, double precision is sufficient, and the errors with single precision computations will be too large. Additionally to double precision, many manufacturers offered Real*16/Quadruple Precision, which usually will be considerably slower than double precision. Modern compilers and Processors, as the Pentium4 and the G4/G5 allow faster computations if the compiler options allow rounding for double precision, so that the results will be considerably less accurate than double precision/16 digits, but still more accurate than single precision/8 digits. That 4/8 Byte are used does not mean that all compiler functions operate on these data types are computed with the full accuracy, correct ”up to the last Bit”. The IEEE-Standard defined that all results have to be given in such a way that only the last/ least significant bit is rounded. Because this can become quite costly, one can usually choose compiler options which offer higher accuracy, but not so good performance, or faster but less accurate code. The errors elf-implemented routines may suffer from additional errors which will be discussed in the next sections.

3.3

Checking for Equality

Concerning what has been said in this section about accuracy, there are some things one can do with integers which one should not do with floating point numbers. For a start, do not check floating point numbers for equality if (a==b) but check for equality with a certain error #. Be sure whether you need the absolute error n=10 epsilon=10^{-n} if (abs(a-b) asin(1.000000000001) ans = 1.5708e+00 - 1.4143e-06i so that the computation will be continued with a complex part. In such cases, the input should always be checked with an if-statement whether it conforms to the expectations. There is also an IEEE-Standard which defines such ”exceptions”, e.g. what should be done if e.g. a number is divided by 0. The result is stored in a bit-pattern which is outputted as NaN, ”Not a Number”. MATLAB is a bit more sophisticated. For a start, it gives the ”correct” result for the division, ±∞ : > 4/0 warning: division by zero ans = Inf > -2/0 warning: division by zero ans = -Inf for Inf, the usual ”rules” apply, but some cases are different:

3.5. ERRORS

35

> Inf+3 ans = Inf > Inf+Inf ans = Inf > Inf/Inf ans = NaN > Inf-Inf ans = NaN When tested for equality via the ==-Operator, one Idiosyncrasy is, that Infinity is always equal to Infinity in MATLAB, but NaN is always unequal to NaN: > 4==4 ans = 1 > Inf==Inf ans = 1 > NaN==NaN ans = 0 and for tests for NaN, the isnan-Function must be used: > isnan(4) ans = 0 octave:17> isnan(NaN) ans = 1 octave:18> To test which numbers are the largest and smallest, the MATLAB-Functions realmin and realmax can be used. Because Inf, -Inf and NaN must are represented as Floating-Point patterns in MATLAB, there are about three to four Bit-patterns less available in MATLAB than in Compilers for e.g. FORTRAN or C which don’t use Inf and Nan. Because the Bit-Pattern of the largest Numbers are used, the largest represented floating-point number is smaller than in the compilers.

3.5

Errors

As we have seen in the previous chapter, the representation of real numbers as floating point approximation leads intrinsically to rounding errors. In the following, we will treat additional sources of error which occur in the evaluation of algebraic equations.

3.5.1

Truncation error

Function evaluation Many mathematical expressions are defined as an infinite process, for example the exponential function is x x2 x3 exp(x) = 1 + + + ... (3.1) 1! 2! 3!

36

CHAPTER 3. NUMERICAL ANALYSIS I

The error which results when e.g. the infinite series is instead computed with only a finite number of operations, i.e. truncated after a finite step is called the truncation error. In fact, if in a given interval a function f (x) is given by the infinite polynomial series with series coefficients a1 , a2 , a3 . . . , a∞ , if the series is truncated after n steps in a given interval, an approximation f (x) =

n ( i=0

a%i xi + O(xn+1 )

(3.2)

can be found which has a smaller error than the truncated series using the coefficients of the infinite series ai . Such a series is called an ”n-th order approximation of f (x),” which often makes use of the expansion of the function in terms of Orthogonal polynomials1 Whereas the exponential function exp(−x) is defined in the infinite series with the coefficients exp(−x) =

∞ ( −xn

n=0

n!

,

the best finite approximation in the interval 0 ≥ x ≥ −ln2 to exp(−x) with 10 digits is exp(x) = a0 + a1 · x1 + a2 · x2 + a3 · x3 + a4 · x4 + a5 · x5 + a6 · x6 + a7 · x7 + ε(x) with ε(x) ≤ 2 × 10−10 and the coefficients are given in Tab.3.2. Be aware that the coefficients for the truncated polynomial approximation depend on the interval for which the approximation should be used, to minimize the error.

n an 0 1.00000 00000 1 -0.99999 99995 2 0.49999 99206 3 -0.16666 53019 4 0.04165 73475 5 -0.00830 13598 6 0.00132 98820 7 -0.00014 131261

1/n! 1.000000000000000000 -1.000000000000000000 0.500000000000000000 -0.166666666666666657 0.041666666666666664 -0.008333333333333333 0.001388888888888889 -0.000198412698412698

Table 3.2: Coefficients for the Polynomial approximation of exp(−x) in the interval 0 ≤ x ≤ ln2 (middle column) and the corresponding coefficients of the infinite Taylor series. In practice, many transcendental functions f (x) which are introduced in elementary classes of mathematical lessons are numerically better approximated by other approximations than polynomial approximations, e.g. by making explicit use of divisions, which can itself mimic operations of infinite order in x, either via Pade-Approximation (quotient of two polynomial expressions) or via continued fractions. An effective strategy, especially with periodic functions, is argument reduction, so that one does not have to compute the Taylor series for large x, but for a small x near the origin Chap. 22, Handbook of Mathematical Functions, M. Abramowitz, I. Stegun, National Bureau of Standards. 1

3.5. ERRORS

37

by either shifting the periodic functions like sin, cos into the interval between [0, π/4], or by decomposing the function into a product of an integer argument and an non-integer argument, like in the case of the exponential function, where one computes exp(x) = exp(m + f ) = exp(m) ∗ exp(f ), m integer, |f | < 1. Many approximation of transcendental functions can be found in Abramovitz/Stegun2 . Other examples for truncation error can be found in other series expansion methods, e.g. Fourier series truncated after a certain number of coefficients, or Pade approximations, where an analytical function +∞ ai xi f (x) = +i=1 ∞ i i=1 bi x is approximated by the truncated Pade approximation +

3.5.2

Rounding error

n a ˜ i xi f˜(x) = +i=1 . m ˜ i i=1 bi x

Because we have only a finite number of digits available, when we try e.g. in Octave to compute 5/9, we get > format long > 5/9 ans = 0.555555555555556 > So, first of all, it is not necessary to input 5./9. like in FORTRAN when one wants to use floating point numbers. On the other hand, one sees that the periodic fraction which is the result must be rounded to 16 decimal digits. Therefore, when we compute the exponential function in the following program, % Example for rounding error in computing transcendental functions clear format compact format long x=-20.5 n_iter=100 myexp=0. for i=0:n_iter-1 % Compute the Taylor-series for the exp-function % x!=gamma(x+1) myexp=myexp+x^i/gamma(i+1) end exp(x) return 2

Handbook of Mathematical Functions, M. Abramowitz, I. Stegun, National Bureau of Standards.

38

CHAPTER 3. NUMERICAL ANALYSIS I

we obtain myexp=-4.422614950123058e-07 as a result, instead of the correct exp(x) = 1.250152866386743e − 09. As we see, the result is so ”very” wrong that not even the sign of the result is correct, we get a negative value for a computation which should always give positive values. The problem is also not the range of the numbers, because the smallest number representable in MATLAB precision is ≈ 10−300 , much smaller than the correct result, 10−9 . The problem is also not a truncation error, as we are still trying to add taylor contributions, even the result does not change any more after the 95th iteration. The problem is that we try to add something which is smaller than the last digit of the summation. There are possibilites to circumvent such kinds of problems which will be explained later in the lecture.

3.5.3

Catastrophic cancellation

Even for the last bit of a floating point function evaluation in double precision, which gives about 16 digits accuracy, the 17th digit is of course wrong. The subtraction of numbers of nearly equal size shifts these invalid digits in front, so that the results For expressions like a=cos(x)^2-sin(x)^2 this gives dubious results whenever the argument x is a multiple of π/4, with arbitrary number of canceled digits. The problem can simply be circumvented by using the trigonometric identity cos(2x) = cos(x)2 − sin(x)2 so that a = cos(2x)

(3.3)

always gives the result with the accuracy of the compilers evaluation of the cos- evaluation.

3.6

Good and bad ”directions” in Numerics

The following integral is positive because, because the integrand is positive in the whole integration interval [0,1]: En =

,

1

xn ex−1 dx,

n = 1, 2, . . .

0

From partial Integration we obtain a relation between En and En−1 , which can be used to iteratively compute En if we have E0 given: ,

0

1

n x−1

x e

dx = x e

-1 , - −

n x−1 -

= 1−n

,

0 1

0

1

nxn−1 ex−1 dx

0

xn−1 ex−1 dx

En = 1 − nEn−1 ,

n = 2, . . .

3.6. GOOD AND BAD ”DIRECTIONS” IN NUMERICS In a REAL*8 implementation, we obtain with E1 = exp(−1) the result to the right. We can be sure that at least E18 , and therefore all the following solutions are wrong, because the result should not become negative in the first place. Because the Ei should fall monotonically, in the iteration 1 − nEn the term nEn is approaching 1 in the iteration, and the correct information in the iteration is quickly annihilated, so that only the erroneous ”last digits” survive. If we use instead the reordered equation En = 1 − nEn−1 , n = 2, . . . 1 − En so that En−1 = , n = ∞ . . . 3, 2 n

(3.4) (3.5)

we can approximate the starting value: En = ≤ =

, ,

1

xn ex−1 dx

(3.6)

xn dx

(3.7)

0 1

0 xn+1 -1

39

E1 E2 E3 E4 E5 E6 E7 E8 E9 E10 E11 E12 E13 E14 E15 E16 E17 E18 E19 E20

0.367879441171442 0.264241117657115 0.207276647028654 0.170893411885384 0.145532940573080 0.126802356561519 0.112383504069363 0.100931967445092 0.09161229299417073 0.08387707005829270 0.07735222935878028 0.07177324769463667 0.06694777996972334 0.06273108042387321 0.05903379364190187 0.05545930172957014 0.05719187059730757 -0.02945367075153627 1.559619744279189 -30.19239488558378

(3.8) n + 1 -0 1 = , (3.9) n+1 which shows that for ”very large n, ”En is very small”, and take E21 ≈ 0 as an educated guess, so that E20 = 0.5. The output for this iteration E20 → E0 is given on the left, on the right the output is overlayed with the output of the iteration E0 → E20 . E1 0.3678794411714423 E1 0.3678794411714423 0.2642411176571154 E2 0.2642411176571154 E2 0.2642411176571153 0.2072766470286539 E3 0.2072766470286539 E3 0.207276647028654 0.1708934118853843 E4 0.1708934118853843 E4 0.170893411885384 0.1455329405730786 E5 0.1455329405730786 E5 0.1455329405730801 0.1268023565615286 E6 0.1268023565615286 E6 0.1268023565615195 0.1123835040692999 E7 0.1123835040692999 E7 0.1123835040693635 0.1009319674456008 E8 0.1009319674456008 E8 0.1009319674450921 E 0.09161229298959281 E9 0.09161229298959281 0.09161229299417073 9 0.083877070104072 E10 0.083877070104072 E10 0.0838770700582927 0.07735222885520793 E11 0.07735222885520793 E11 0.07735222935878028 0.0717732537375049 E12 0.0717732537375049 E12 0.07177324769463667 0.06694770141243632 E13 0.06694770141243632 E13 0.06694777996972334 0.06273218022589153 E14 0.06273218022589153 E14 0.06273108042387321 0.05901729661162711 E15 0.05901729661162711 E15 0.05903379364190187 0.05572325421396629 E16 0.05572325421396629 E16 0.05545930172957014 0.0527046783625731 E17 0.0527046783625731 E17 0.05719187059730757 0.05131578947368421 E18 0.05131578947368421 E18 -0.02945367075153627 0.0250000000000000 E19 0.025 E19 1.559619744279189 0.5000000000000000 E20 0.5 E20 -30.19239488558378

40

CHAPTER 3. NUMERICAL ANALYSIS I

As can be seen, the second iteration with the ”wrong” starting value converges against the right end-value exp(−1), whereas the first iteration with the ”right” starting value converges against a wrong result. This shows the ”art” of numerical computing, which is, to obtain a correct end result with a ”good” routine and a wrong starting value, instead of obtaining a wrong end result with a correct starting, but a ”bad” routine. It will be later become obvious in this course that integration is always the ”good” direction in numerical computing, which can decrease initial errors, whereas the differentiation is the bad direction, which can increase initial errors. This is in contrast to manual calculation, where the differentiation is easier to treat than integration.

3.7 3.7.1

Calculus and order of Methods Taylor Approximation revisited

Many functions can be approximated by a polynomial series ∞ f (x) =

(

ai xi

i=1

in such a way, so that around the point x0 , for the ν−th derivative f ν (x), we have f (x) =

∞ ( f (ν)(x0 )

ν=1

ν!

(x − x0 )ν .

2

sin(x) 1st Order 3rd Order 5th Order 7th Order

0 −2 −5

0

5

2

cos(x) 0th Order 2nd Order 4th Order 6th Order

0

For the functions exp(t), sin(t), cos(t), the Taylor series is given below: −2 exp(t) = sin(t) = cos(t) =

∞ n ( t

n=0 ∞ (

n=0 ∞ (

n=0

n!

=1+

t t2 t3 + + + ... 1! 2! 3!

(−1)n

t2n+1 t3 t5 = t − + − ... (2n + 1)! 3! 5!

(−1)n

t2n t2 t4 t6 = 1 − + − + ... 2n! t! 4! 6!

−5

0

5

6 exp(x) 0th Order 1st Order 2nd Order 3rd Order

4 2 0

−5

0

5

If we truncate the (for transcendental functions infinite) series after a finite number of terms, we obtain the Taylor approximation. The the evaluation of a Taylor approximation, e.g. of fourth order with the coefficients a, b, c, d, e f (x) = a + bx + cx2 + dx3 + ex4 series can be done in an efficient and in an inefficient way. Using the above formula directly, we can write f(x)=a+b*x+c*x*x+d*x*x*x+e*x*x*x*x

3.7. CALCULUS AND ORDER OF METHODS

41

so that we need four additions and ten multiplications. If we use brackets around the expression in a skilled way, four additions and four multiplications are sufficient: f(x)=a+(b+(c+(d+e*x)*x)*x)*x It is easy to write down the derivative of the above polynom as f(x)=b+(2*c+(3*d+4*e*x)*x)*x 8

In MATLAB, the evaluations of polynoms is implemented with the function polyval, the derivative with polyder, but the order of the coefficients is the oppo6 site from the above example, and the graph can be seen on the right: clear, format compact P=[1 0 -1] x=linspace(-3,3,100); y=polyval(P,x); P_deriv=polyder(P); y_deriv=polyval(P_deriv,x); plot(x,y,’-’,x,y_deriv,’--’) legend(’f(x)=x^2-1’,’d/dx f(x)=2x’) grid axis image

4

2

0

Typical functions which cannot be approximated by Tayler-series are functions with a ”jump”, like the sign−2 function sign(x) = x/|x|. Because the Taylor-series is an infinite series, one needs comparatively many terms to obtain a good approximation. If convergence is sought only on a finite inter- −4 vall, the Tchebicheff-approximation, which minimizes the error over a finite interval, is usually a much better approximation for the same number of terms. −6

3.7.2

2

f(x)=x −1 d/dx f(x)=2x

−2

0

2

Integration I

In the same way that many transcendental functions can be represented by an infinite Taylor series but approximated as a finite polynomial series in x, integrals and derivatives can be approximated by replacing the infinitely small differential dx by the finite difference ∆x, and the error can be expressed as a power of ∆x, as in the approximation of transcendental functions by finite power series. The simplest method to numerically integrate an integral I=

,

b

a

f (x)dx

(3.10)

42

CHAPTER 3. NUMERICAL ANALYSIS I

consist by the simple evaluation of the corresponding Riemann-sum as I (1) = ∆x

( i

f (xi ), f (xi ) = {a, a + ∆x, . . . b − 2∆x, b − ∆x},

(3.11)

where b − a is a multiple of ∆x, and the integration points are spaced equidistantly.3 I (1) means that the method is of first order in ∆x, the error is of the order of ∆x2 . Numerical integration is sometimes called ”quadrature”, maybe from the time where the integral was approximated numerically by drawing squares under the graph, and this box counting was the first ”non-analytical” quadrature. As an example, let us compute the integral √ √ , b πerf(b) πerf(a) 2 exp(−x )dx = − , 2 2 a which is a bit unintuitive because its needs the error function erf to be represented analytically. With the integration bounds of [0, 1] the integral is with about 15 digits accuracy ,

1

exp(−x2 )dx = .7468241328124270.

0

Now let us approximate this integral with the rectangle midpoint rule, where we replace the integral with a Riemann sum with n corner points and we will evaluate the function in the middle of the n − 1 inIntegration with Rectangle Midpoint Rule tervals of equal width h with the functon 1 evaluated at the middle instead of the left 0.9 or right end of the integration interval: 0.8 clear format long n=101 % n odd ! dx=1/(n-1) % stepsize xrect=[dx/2:dx:1-dx/2]; yrect=exp(-xrect.*xrect); sum(yrect)*dx ,

b

a

.

0.7 0.6 0.5 0.4 0.3 0.2 0.1

f (x)dx = h· f (x0 )+f (x1 )+f (x2 ) . . .+f (xn )

/

0 −0.4

−0.2

0

0.2

0.4

0.6

0.8

1

1.2

1.4

For 100 intervals / 101 corner points, the result 0.74682719849232 is correct up to 5 digits or the rectangle midpoint rule , amazing, as we only used 100 point. In our MonteCarlo evaluation of π, we needed of the order of 10000 points when we wanted an accuracy of only two digits. Very often, numerical integration methods are introduced not by using the rectangle midpointrule, but the trapeze-rule, which is slightly more complicated than the midpoint rule, as each integration interval must be approximated as a trapeze: ,

b

a

f (x)dx =

/ h . · f (x0 ) + 2f (x1 ) + 2f (x2 ) + . . . + f (xn ) 2

There are methods which don’t chose the points equidistantly, but optimize the choice of points so that the most accurate approximation is obtained with the minimum number of points. 3

3.7. CALCULUS AND ORDER OF METHODS Instead of evaluating the left and right bound of each interval, we will count the function values between the upper and lower integration bounds once, the function values of the upper and lower bound only half. This corresponds to the summation over the existing trapeze areas:

43 Integration wit Trapeze Rule

1 0.9 0.8 0.7 0.6 0.5

0.4 clear format long 0.3 n=101 % n odd ! 0.2 dx=1/(n-1) % stepsize 0.1 xtrap=[0:dx:1]; 0 ytrap=exp(-xtrap.*xtrap); −0.4 (sum(ytrap)-.5*(ytrap(1)+ytrap(n)))*dx

−0.2

0

0.2

0.4

0.6

0.8

1

1.2

1.4

Surprisingly, the result of 0.74681800146797 is one digit less accurate than the result with the midpoint rule, though the program was more complicated, because we had to think about a proper way to implement the trapeze shape for each interval. If we think about a graph with mostly negative curvature, the trapeze rule will end with an approximation which is constantly below the true function value. For the rectangles of the midpoint rule are partly above, partly below the graph, so that there is error compensation already within one interval. In the rectangle midpoint rule, we have chosen the quadrature points in the middle of the interval. If we would have choosen the values for the function evaluation at the left/right boundary of each interval, we would have obtainted 0.74997860426211/ 0.74365739867383, considerably less accurate than the rectangle midpoint rule. It can be shown4 that the

Integral over convex curve

midpoint rule has an accuracy of 1 3 %% h f (xi ), 24 i

(3.12)

Trapeze rule underestimated

Rec. Midp. rule overestinated

better than the trapeze rule which has an accuracy of 1 3 %% h f (xi ), 12 i

(3.13)

so it is surprising that textbooks usually introduce numerical quadrature via the trapeze rule. Because both formulae are correct up to the second power of hi , and the error is of the third power, they are called formulae of ”second order”. More accurate, accurate to third order, is the composite Simpson-Rule S(f ), which makes use of combining the rectangle midpoint rule R(f ) and the trapez rule T (f ). When we compare the integral for the midpoint rule and for the trapeze rule, we see that in our integration G.E. Forsythe, M. Malcom, C. Moler, Computer Methods for Mathematical computations, Prentice Hall 1977 4

44

CHAPTER 3. NUMERICAL ANALYSIS I

intervall with a convex function, the trapeze rule allways gives a too small result, the midpoint rule gives allways a too large result. Therefore, if we average R(f ) and T (f ), we will get a better result than R(f ) and T (f ) alone. Because the error for R(f ) (Eqn. 3.12) is twice as large as the error for T (f ) (Eqn. 3.13), we should not take the direct average 12 R(f ) + 12 T (f ), but the weighted average so that the error of both rules cancels: 2 1 S(f ) = R(f ) + T (f ). 3 3 Its error can be shown5 to be of the order of 1 4 %%%% h f (xi ). 2880 i For our example with the integral from 0 to 1 over exp(−x2 ), we obtain 0.746824132817537 as the result, instead of the exact 0.7468241328124270 . . . . In our second-order formulae, we tried to approximate the graph with straight lines and integrated the area below the curve. A parabola is determined by 3 points, and therefore one can also try to approximate the graph via a parabola instead of a straight line to obtain a Simpson rule directly by supplying three integration points for each interval. It is therefore necessary to have an odd number of integration points, and the direct derivation of the Simpson rule can be done for an integration interval with length 2h and by inserting the Taylor expansion of the function f (x) with the ν−th derivatives f (ν) around the point x0 so that f (x) =

∞ ( f (ν)(x0 )

ν=0

ν!

Simpson-Rule

(x − x0 )ν

instead of the function f (x) itself yields:

,

0

2h %

,

2h

f (x)dx =

0

%

%%

'

f (h) + f (h) · x + f (h) · x + . . . dx ≈

.

,

0

2h %

.

2

/

f (h) + f (0) − f (2h)/(2h) · x +

f (0) − 2f (h) + f (2h)/(2h)

2

/

2

·x

'

g(x)

f(x)

dx =

h 0 h 2h 3h 4h (f (0) + 4f (h) + f (2h)) 3 Using this formula for the single interval of length h, we can compose the formula for the whole integration over [a, b] with the integration point from x0 = a to x2n = b : ,

b

f (x)dx =

a

The MATLAB-Program is

/ h. f (x0 ) + 4f (x1 ) + 2f (x2 ) + 4f (x3 ) + . . . + f (x2n ) 3

G.E. Forsythe, M. Malcom, C. Moler, Computer Methods for Mathematical computations, Prentice Hall 1977 5

3.7. CALCULUS AND ORDER OF METHODS

45

clear format long,format compact n=101 % n odd ! dx=1/(n-1) % Stepsize xsimp=[0:dx:1]; ysimp=exp(-xsimp.*xsimp); (4*sum(ysimp(2:2:n))+2*sum(ysimp(3:2:n-1))+ysimp(1)+ysimp(n))*dx/3 which gives the Result 0.74682413289418, slightly worse than the composed Simpson rule. In the following table, we compare the errors of the different orders by underlining the correct digits and introduce the Big-O-notation Method Exact Rectangle, left endpoint Rectangle, right endpoint Rectangle, midpoint Trapeze rule Simpson Composite Simpson

Integral 01 2 0 exp(−x )dx 0.7468241328124270 . . . 0.74997860426211 0.74365739867383 0.74682719849232 0.74681800146797 0.74682413289418 0.746824132817537

Order correctness O(h∞ ) O(h) O(h) O(h2 ) O(h2 ) O(h3 ) O(h3 )

Several conclusions can be drawn from viewing the above table, which hold also for other numerical methods which have an intrinsic truncation error: 1. If a formula if of n−th order and a discretization of 1/100 of the interval length are used, for a first order implementation, the error is about 1/100=1 %, for the second order method it will be about 1/10.000 and for the third order method it will be 1/1000000. (Of course, the prefactors in the order also have to be taken into consideration). 2. Therefore, it is not always necessary to increase the number of discretization steps to obtain a more accurate result. The change from first order to second order in the rectangle rule resulted from just switching the integration points by h/2. 3. If the theoretical accuracy cannot be reached, it is necessary to consider whether 1. The function under consideration does not fulfill the necessary criteria (smoothness etc) or 2. to search for the error in the program resulting from incorrect prefactors, intervals with incorrect bounds etc. If a formula of second order gives results with an error proportional 1/(number of points), then the interval bounds are usually determined wrongly. 4. Be aware that it is not possible to integrate functions analytically if their integral has no solution due to divergence etc ..... In this section, we have discussed the error resulting from the integration over a whole interval. This is also called a global error, in contrast to the error which occurs in the approximation of the single interval. Numerical methods suffering from truncation error vary depending on whether the global error is the same as the local error, whether the global error is larger than

46

CHAPTER 3. NUMERICAL ANALYSIS I

the local error (many solvers for differential equations which do not conserve energy) or the global error is smaller than the local error (error compensation as in the case of the Simpson Integration.). One generally should be very careful in using a method with Relative Accuracy for Integrating exp(−x*x) between 0 and 1 low order accuracy and a small 10 Rectangular left and right boundary time step. First of all, for 10 many problems, such a method Trapeze Rule can become quite time con10 suming. Furthermore, the Rectangular Midpoint Rule 10 more function evaluations occur, the more rounding errors 10 are accumulated. The diagram to the right shows the 10 Simpson cost-performance diagram, the 10 number of time steps plotted with respect to the accu10 racy. Cost-performance diagrams vary depending on the 10 10 10 10 10 10 10 evaluated functions. As can Number of Integration Points be seen, beyond 1000 integration points, the accuracy of the Simpson method has already reached the limit of 16 digits of the double precision accuracy and therefore the integral evaluation cannot be increased further by increasing the number of integration points. 0

−2

−4

−6

−8

−10

−12

−14

−16

0

1

2

3

4

5

There are integration formulas which are easier to use than the numerical approximations of the Riemann sum we introduced here, which are called Newton-Coates formulae. The Midpoint rule is called an ”open Newton-Coates formula”, because the endpoints of the integration intervall are not evaluated, the formulae for which the endpoints must be evaluated, are alled ”closed Newton-Coates formulae”. The following table shows the Taylor expansion for a single interval of length h for Newton-Coates formulae of different order with the corresponding error term: Name Midpoint Trapez–Rule Simpson Simpson 3/8 Bode

Integral Formula 0 x2 f (x)dx = h[ 12 f1 + 12 f2 ] 0x1 x3 f (x)dx = h[ 13 f1 + 43 f2 + 13 f3 ] 0x1 x4 f (x)dx = h[ 38 f1 + 98 f2 + 98 f3 + 38 f4 ] 0x1 x5 14 64 24 64 x1 f (x)dx = h[ 45 f1 + 45 f2 + 45 f3 + 45 f4 +

14 45 f5 ]

Error Term +O(h3 f %% ) +O(h5 f (4) ) +O(h5 f (4) ) +O(h7 f (6) )

Carrying the error compensation in formulas with truncation error further to higher orders, by combining low order methods as in the case of the composite Simpson rule so that a higher-order method results, is called ”Romberg integration”. If the limit for ”infinitely high orders” is taken, this is called Richardson-extrapolation, and these ideas can also be applied to differentiation and the solution of numerical differential equations.

3.7. CALCULUS AND ORDER OF METHODS

3.7.3

47

Differentiation I

In the same way one can derive the Newton-Coates formulae for integrals from their Taylor expansion in the previous section, one can derive formulae for the derivatives using the Taylor expansion6 . Such approximations are often called finite difference formulas, as they approximate the differential with the finite difference. For a data which take the value fi−2 , fi−1 , fj , fj+1 , fj+2 at equidistant points, we get the following finite difference schemes for first order derivatives.: Name: Forward–Difference Backward–Difference 3–point symmetric 3–point asymmetric 5–point symmetric

Finite difference scheme: (fi+1 − fi )/∆x (fi − fi−1 )/∆x (fi+1 − fi−1 )/(2∆x) (−1.5fi + 2fi+1 − .5fi+2 )/∆x (fi−2 − 8fi−1 + 8fi+1 − fi+2 )/(12∆x)

Leading error ∆xf %% (x)/2 −∆xf %% (x)/2 ∆x2 f %%% (x)/6 −∆x2 f %%% (x)/3 −∆x4 f %%%%% (x)/3

Note that the leading coefficients in front of the fi−2 , fi−1 , fj , fj+1 , fj+2 have to add up to 0. For second order derivatives, similar schemes are be written down in the following table and again the coefficients add up to 0: Name: 3–point symmetric 3–point asymmetric 5–point symmetric

Finite difference scheme: (fi−1 − 2fi + fi+1 )/(∆x2 ) (fi − 2fj+1 + fi+2 )/(∆x2 ) (−fi−2 + 16fi−1 − 30fi + 16fi+1 − fi+2 )/(12∆x)

Leading error ∆x2 f %%%% (x)/12 ∆x2 f %%% (x) −∆x4 f %%%%%% (x)/90

In contrast to numerical integration, which smoothes out errors via error compensation, numerical differentiation ”roughens up” the solution. If high accuracy is desired, there are usually better solutions than computing the derivatives directly via finite difference schemes. The graph to the right shows the numerical integral of 1−

,

x

0

sin(y)dy = 1 − cos(x)

and the numerical differential of sin(x), d sin(x) = cos(x) dx with some additional noise of 1 % in sin(x). The above graph was produced with the following program: clear format compact nstep=200 x=linspace(0,4*pi,nstep); dx=mean(diff(x)); idx=1/dx; y=sin(x-dx/2)+0.01*(rand(size(x))-.5); subplot(3,1,1) 6

Clive A.J. Fletcher, Computational Techniques for Fluid Dynamics, Vol.1, 2nd. ed. Springer 1990

48

CHAPTER 3. NUMERICAL ANALYSIS I 1

sin(x)+−0.005*rand d/dx sin(x) cos(x) 1−int(sin(x))

0.5 0 −0.5 −1 0

2

4

6 absolute error

8

10

12

8

10

12

8

10

12

0.1 0.05 0 −0.05

cos(x)−d/dx*sin(x) cos(x)−1+int(sin(1)

−0.1 0

2

4

6 relative error

(cos(x)−d/dx*sin(x))/cos(x) (cos(x)−1+int(sin(1))/cos(x)

4 2 0 −2 0

2

4

6

plot(x,y,’-.’,x(1:nstep-1),diff(y)*idx,... x(1:nstep-1),cos(x(1:nstep-1)),’:’,... x(1:nstep-1),1-cumsum(y(1:nstep-1))*dx,’--’) axis tight legend(’sin(x)+-0.005*rand’,’d/dx sin(x)’,’cos(x)’,’1-int(sin(x))’) subplot(3,1,2) plot(x(1:nstep-1),diff(y)*idx-cos(x(1:nstep-1)),... x(1:nstep-1),1-cumsum(y(1:nstep-1))*dx-cos(x(1:nstep-1)),’:’) legend(’cos(x)-d/dx*sin(x)’,’cos(x)-1+int(sin(1)’) title(’absolute error’) axis tight subplot(3,1,3) plot(x(1:nstep-1),((diff(y)*idx-cos(x(1:nstep-1)))./cos(x(1:nstep-1))),... x(1:nstep-1),(1-cumsum(y(1:nstep-1))*dx-cos(x(1:nstep-1)))... ./cos(x(1:nstep-1)),’:’) legend(’(cos(x)-d/dx*sin(x))/cos(x)’,’(cos(x)-1+int(sin(1))/cos(x)’) axis tight title(’relative error’) return Both the differential and the integral should give cos(x), but the differential is so noisy that the result deviates visibly from the exact solution. The integral over the noisy result gives nevertheless a smooth curve. This is again a case of a ”good” and a ”bad” direction of numerical computing, as we encountered before by rewriting the iterative computation of the

3.7. CALCULUS AND ORDER OF METHODS

49

equation into

En = 1 − nEn−1 (numerically unstable) En = 1 − nEn−1 (numerically stable).

As can be seen, for the differentiation and its inverse operation, the integration, in numerical analysis, the differentiation consists the ”bad” direction, ”integration” is the good direction. In numerical analysis, integrals, also of higher order, can usually be computed with sufficient precision, in contrast to derivatives, whereas in analytical calculations, it is usually always possible to compute differentials, but very often the computation of closed forms for integrals is problematic. Exercises: 1. Write a program which produces floating point numbers for base 2 and mantissa 4, as well as for base 4 with mantissa 2. a) Chose the exponent so that both number systems are roughly comparable. b) Plot the position of the numbers. c) Compare both number systems: Which number system can be supposed to have the better roundoff-properties. 2. Write a program which computes the exponential function exp(x) using the Taylor series and one program which computes the exponential function by evaluating the integer part of x using powers of the Euler number e and the non-integer-part using the taylor series. For which size of the arguments become the

Chapter 4 Graphics 4.0.4

Initializing and manipulating vectors

Instead of using for loops for setting up vectors and matrices, it is convenient in MATLAB to use the implicit loops provided by the colon operator : and brackets for the array constructor [] >> a=[3:6] a = 3 4

5

6

For step-sizes different from one the stepsize can be specified as [lower_bound:stepsize:upper_bound] like in >> a=[3:.5:6] a = 3.0000 3.5000

4.0000

4.5000

5.0000

5.5000

6.0000

This is different from loops in FORTRAN and C, where the stepsize is added as the third argument for a loop statement. Whereas the colon operator notation using : constructs a vector with a given lower and upper bound for a given stepsize, [lower_bound:stepsize:upper_bound], if instead of the stepsize the number of points is known, it is more convenient to use the linspace-function >> a=linspace(3,6,7) a = 3.0000 3.5000

4.0000

4.5000

5.0000

5.5000

There is also a function which gives vectors in logarithmic spacing >> b=logspace(1,1000,3) b = 10 Inf In >> b=logspace(1,4,4) b = 10 100

1000

10000

6.0000

52

CHAPTER 4. GRAPHICS

If several vectors should be concatenated, this can be done with the brackets for the arrayconstructor [] >> c=[1 3] c = 1 3 >> c=[4 c b] c = Columns 1 through 6 4 Column 7 10000

1

3

10

100

1000

After a lot of vector operations, one usually one also needs functions which give informations about the vectors used. The most elementary function, which displays information about variables, is >> who Your variables are: a

ans

b

c

The length of a vector is displayed by >> length(a) ans = 7 but this function makes no difference between column- and row vectors. For information on higher dimensions, one has to use the function >> size(a) ans = 1 7 Vector elements can be accessed either via the for loops like in other programming languages like in >> for i=1:length(b) f(i)=2*b(i) end f = 20 f = 20 200 f = 20 200 f = 20 200

2000 2000

20000

53 or via the colon-notation with : and round brackets so that for a vector >> c=.2:.2:1.2 c = 0.2000 0.4000

0.6000

0.8000

1.0000

1.2000

the assignment of the second to the fourth element to a vector g can be written as >> g=c(2:4) g = 0.4000

0.6000

0.8000

The whole of a vector can be assigned without specifying the bounds like in >> h=c(:) h = 0.2000 0.4000 0.6000 0.8000 1.0000 1.2000 If the vector from a lower bound up to the end should be assigned, this can be done via the the end statement in round brackets together with the colon operator : >> v=c(4:end) v = 0.8000 1.0000

1.2000

Functions which operate on vectors are usually defined in the ”canonical” way, that means in a way in which one expects the function to work. The functions prod and sum acting on a vector behave in the way one expects, e.g. they give as a result the product and sum of the vector elements. Whereas prod and sum are acting on vectors and give a scalar as a result, the functions cumsum and cumprod which computed the cumulated sum and the cumulated product give a vector as a result >> cumsum(1:5) ans = [1 3 6

10

15]

One must be careful with the use of ”multiplicative” operators *, / and ^, which are in MATLAB in general interpreted in the sense of numerical linear algebra, so that columnand line- operations must match. If one wants to use these operators elementwise, one should use their ”elementwise” variants which are preceded by a ”.”, as in .*, ./ and .^.

54

4.1

CHAPTER 4. GRAPHICS

Setting up and manipulating Matrices

Matrices can be manipulated in the same way as vectors, preferably with the colon operator : and the brackets for the array-constructor []. Some elementary builtin MATLAB matrix functions will be explained here because they make matrix construction easier. The ones function sets up a matrix with ones as every element, as usual in MATLAB a single matrix sets up a square two-dimensional matrix >> ones(2) ans = 1 1 1 1 For non-square matrixes, two indices have to be specified, where the first is the columns-index, and the second is the row index, for example >> ones(3,2) ans = 1 1 1 1 1 1 The zeros function behaves in the same way as the ones function, only that it sets up matrixes with 0 as every element >> zeros(2,3) ans = 0 0 0 0

0 0

In linear algebra, the identity matrix is very important, and therefore the unit matrix in MATLAB is named eye, eyedentity / identity >> eye(3) ans = 1 0 0

0 1 0

0 0 1

It may be surprising, but the identity-matrix is also defined for non-square matrices, as the following example shows >> eye(2,5) ans = 1 0 0 1

0 0

0 0

0 0

Another important matrix function is the constructor for the random matrix

4.1. SETTING UP AND MANIPULATING MATRICES >> rand(2,4) ans = 0.8214 0.4447

0.6154 0.7919

0.9218 0.7382

55

0.1763 0.4057

Matrices can then be constructed via matrix functions alone like in >> c=ones(2)-eye(2) c = 0 1 1 0 or with the help of the matrix constructor brackets [] so that >> b=zeros(2) b = 0 0 0 0 >> d=[2 3 4 5] d = 2 3 4 5 >> e=[c b d c] e = 0 1 1 0 2 3 4 5

0 0 0 1

0 0 1 0

A very convenient function similar to linspace in one dimension which can be used to set up arguments for functions in higher dimensions is the meshgrid-function which the functionality is as follows: >> [X,Y] = meshgrid(1:3,10:14) X = 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 Y = 10 10 10 11 11 11 12 12 12 13 13 13 14 14 14

56

4.2 4.2.1

CHAPTER 4. GRAPHICS

Graphs and Visualization Visualizing Vectors

The elementary command in MATLAB for plotting functions etc. is the plot command. Plots can be shown in the plotting window either alone or as one of many sub-plots, like in the following example >> x=[.1:.1:.5] x = 0.1000 0.2000 >> y=[20:-4:1] y = 20 16 12

0.3000

8

0.4000

0.5000

4

>> subplot(2,2, 1) >> plot(y) >> subplot(2,2,2) >> plot(x,y) which displays on the screen (note the differnt scale on the x-axis) 20

20

15

15

10

10

5

5

0

1

2

3

4

5

0 0.1

0.2

0.3

0.4

0.5

Plots of vectors can be done either by plotting the vector directly or by specifying two vectors, the first will be taken as the x-axis. If the vector length does not match, MATLAB issues an error message and stops the program execution. The plots are automatically done in the sub-plot which has been called last. A subtle way of plotting it the plot of a vector of complex numbers. If you have a complex vector c, you can get the real part x and the imaginary part y via x=real(c) y=imag(c) The command plot(c) has then the same effect as plot(x,y) which means that the imaginary part is plotted versus the real part. If a new plotting window should be opened, this can be done via the figure command, the first window is built is figure(1) command which is automatically executed if no plotting

4.3. VISUALIZING ARRAYS

57

window is open, figure(2) opens a second plotting window and so on. Plots are done in the window for which the figure command was called last. There is a wide variety of ways to influence graph annotation in MATLAB 0.3

% example for graph anotation subplot(2,2,1) x=[.1:.1:.5] plot(x,x.*x,x,x.*x.*log(x)) xlabel(’Xaxis’) ylabel(’Yaxis’) titel(’Plot anotations’) text(.14,.2,’any label here’) legend(’x^2’,’x^2*log(x)’)

any label here

Yaxis

0.2 0.1

x2 x2*log(x)

0 −0.1 −0.2 0.1

0.2

0.3 Xaxis

0.4

0.5

The legend created by label can be moved with the mouse. MATLAB graphics can be saved in various styles (Postscript, encapsulated Postscript, JPEG, .....) via the print command. The line-style (full lines, dotted lines, symbols) can be changed via the arguments in the plot command 0.5

plot(x,log(x),... x,x,’:’,... x,x.^2,’+’,... x,x.^3,’*-’)

0 −0.5

To look at a drawing in higher resolution, use the zoom command, aim with the mousepointer at the region which should be zoomed and click the left mousebuttom (the right mousebuttom unzooms the region again).

4.3

−1 −1.5 −2 −2.5 0.1

0.2

0.3

0.4

0.5

Visualizing Arrays

As an example in this section, we will use the the Rosser matrix >> rosser ans = 611 196 196 899 -192 113 407 -192 -8 -71 -52 -43 -49 -8 29 -44

-192 113 899 196 61 49 8 52

407 -192 196 611 8 44 59 -23

-8 -71 61 8 411 -599 208 208

-52 -43 49 44 -599 411 208 208

-49 -8 8 59 208 208 99 -911

29 -44 52 -23 208 208 -911 99

If a matrix is displayed with the plot command, the lines are each plotted as a vector, as in the example for plot(rosser) below on the right.

58

CHAPTER 4. GRAPHICS

Arrays are plotted as arrays with the verb—mesh—-command, which plots the data in a wire-frame-type of graph, as below on the right. The view command can be used to set a different viewing angle for three-dimensional plots. It is also possible to change the viewpoint interactively via the rotate3d command by pointing with the mouse on the frame and pulling the frame of the 3D-graph. 1000

800 1000

600

400

500

200 0

0 −500

−200

−400

−1000 8

−600

6 4

−800 2

−1000

0

1

4.4

2

3

4

5

6

7

8

1

2

3

4

5

6

7

8

Analyzing systems via plotting

In the following, a linear, an exponential function, a hyperbola, a logarithmic and an inverse square root are plotted via the following program, in linear, x- and y logarithmic as well as double logarithmic scale: clear format compact x=[linspace(0.1,10,100)]; subplot(2,2,1) plot(x,x,’-’,x,exp(x),’--’,x,1./x,’:’,x,log(x),’-.’,x,1./sqrt(x),’-+’) axis([0 10 -3 20 ]) title(’linear plot’) legend(’x’,’exp(x)’,’1/x’,’log(x)’,’1/sqrt(x)’) subplot(2,2,2) semilogx(x,x,’-’,x,exp(x),’--’,x,1./x,’:’,x,log(x),’-.’,x,1./sqrt(x),’-+’) axis([0.1 10 -3 20 ]) title(’semilogarithmic in x-direction’) legend(’x’,’exp(x)’,’1/x’,’log(x)’,’1/sqrt(x)’,2) subplot(2,2,3) semilogy(x,x,’-’,x,exp(x),’--’,x,1./x,’:’,x,log(x),’-.’,x,1./sqrt(x),’-+’) axis([0.1 10 .01 20000 ]) title(’semilogarithmic in y-direction’) legend(’x’,’exp(x)’,’1/x’,’log(x)’,’1/sqrt(x)’,2)

4.4. ANALYZING SYSTEMS VIA PLOTTING

59

subplot(2,2,4) loglog(x,x,’-’,x,exp(x),’--’,x,1./x,’:’,x,log(x),’-.’,x,1./sqrt(x),’-+’) axis([0.1 10 .01 20000 ]) title(’logarithmic ’) legend(’x’,’exp(x)’,’1/x’,’log(x)’,’1/sqrt(x)’,2)

semilogarithmic in x−direction

linear plot

20

20 x exp(x) 1/x log(x) 1/sqrt(x)

15 10

15 10

5

5

0

0 0

2

4

6

8

x exp(x) 1/x log(x) 1/sqrt(x)

10

10

−1

semilogarithmic in y−direction 4

10

2

10

1

10

logarithmic 4

10

x exp(x) 1/x log(x) 1/sqrt(x)

x exp(x) 1/x log(x) 1/sqrt(x)

2

10

0

0

10

10

0

10

10

−2

2

4

6

8

10

10

−2

10

−1

0

10

1

10

Many systems in science and mathematics can be better understood by just plotting typical properties in different scales. Logarithmic, linear, exponential and power laws can be found in nature, and are easily identifiable by plotting the data in difference scales.

60

4.4.1

CHAPTER 4. GRAPHICS

Linear Plots

Typical ”linear” plots result from linear ”response functions”, the simplest is probably Hooks law, which is sketched in the drawing to the right. It is a ”linear” law, and in the dynamical situation, when the spring is pulled with a force in a certain frequency, the elongation changes with the same frequency. Such a linear response is not a matter of course. There are ”nonlinear systems” which respond with e.g. ”frequency-doubling” to an external stimulation, like in the case where a high intensity red laser beam going into a target comes out as a blue laser beam (blue light with twice the frequency of the red light).

Crystall target Red Beam Blue Beam LASER 4.4.2

Logarithmic Plots

If the y-axis of a plot is choosen logarithmically, exponential curves appear as straight lines, so that logarithmic plots allow to identify exponential behavior. Typical examples for exponential behavior are time evolution plots. Radioactive decay and the increase of the GDP in Economies are examples for such a time evolution. Below, the increase in the Dow Jones Industrial Stock Index is shown. The curves in linear plot are bent, but more or less straight in a logarithmic plot. It is a matter of ongoing debate whether this reflects rather the exponential increase in the strength of the US-Economy or just the exponential inflationary effects.

If instead of the y-axis the x-axis is choosen in logarithmic scale, logarithmic curves become straight lines. Logarithmic curves grow slower than linear curves. Typical examples for logarithmic behavior are animal senses. Light and sound are perceived on a logarithmic

4.4. ANALYZING SYSTEMS VIA PLOTTING

61

scale, i.e. sound is not ”twice as loud” if the pressure of the sound wave is ”twice as high”, but ”102 as hight”.

4.4.3

Double logarithmic Plots

Double logarithmic plots allow to identify ”power laws”, functions of the form xr , where r does not necessarily have to be integer. Power laws are usually found in nature when systems suffer from ”finite size effects”.

The Gutenberg-Richter, which states that the probability of earthquakes of magnitude x is proportional to a function of 1/x r is an example where a system (the tectonic plates) create earthquakes which are up to a maximum size, the size of the plate itself. The curves of a power law can be written as a superposition of exponential curves, like in the following program clear format compact a=linspace(.0,100,1000) la=length(a) x=linspace(0.1,1000,100); y=zeros(size(x)); for i=1:la y=y+exp(-.1*a(i).*x); end loglog(x,y) This means that a power-law is found in a system if there are many different scales, which contribute to an exponential phenomenon and on each size scale (variable a in the above example) there is a different prefactor in the exponential law. In the same way as the curve of a power law can be written as a superposition of exponential curves, a Lorentzian can be written as a superposition of Gaussian curves. Another example for power-laws is the 1/f -noise in many technical applications. I its very often found in system where a seemingly continuous process is a result of discrete processes, and the deviation from the mean causes the noisy fluctuations.

62

4.5 4.5.1

CHAPTER 4. GRAPHICS

Specialized Plots and specialized styles Graph Properties

The axis of a graph can be easily modified with the axis-command. axis image defines the same length unit for x− and y−axis. axis([xmin xmax ymin ymax]) defines the minimal coordinates for x− and y−axis respectively. Because MATLAB usually chooses the axis so as to end the axis at values for multiples of 1, 10, 100, it is sometimes necessary to set axis tight so that the axes terminate at the extremal values of the plot. Apart from the axis, a grid can be specified by using grid. Sometimes it is necessary to modify picture properties like the axis labels etc from the default values chosen by MATLAB. For the program clear format compact x=linspace(0,2*pi,10); y=sin(x); h=plot(x,y) g=axes get(h) get(g) the plot is defined as a variable, and these variable can be displayed with the get command. The entries for h and g can then be directly modfied using the set-command, by specifying the object-name, h, g, the property to modify, e.g. Color and the new value, e.g. set(g,’Color’,[.5 .5 .5]) Another possible usage, if one already knows the property name, is e.g. set(gca,’XTickLabel’,{’One’;’Two’;’Three’;’Four’}) which labels the first four tick marks on the x-axis and then reuses the labels until all ticks are labeled. The labels can be positioned like set(gca,’XTickLabel’,{’1’;’10’;’100’})

4.6. MATLAB-OUTPUT INTO A FILE

4.5.2

63

Including Images

The command image puts a default-image (in GIF-Format) on the graphics-screen. In general, graphics of nearly any format can be read and displayed using name=imread(’name.gif’) image(name) The date in the variable name can then be manipulated like a usuall MATLAB-array.

4.6

MATLAB-output into a file

Very often, one wants to save some output of MATLAB onto a file to include it in other documents. For short output, it is simplest under a window-system to copy the desired lines with the mouse into an editor. If the output becomes too long, one can use the command diary on and MATLAB will then output not only to the screen, but also in the file diary . If one wants to redirect the output in a file with a special name, one can use the command diary(’special_filename’) To end the output in the diary, use the command diary off If you want to include the output in a LaTeX document and preserve the ”Computer-outputlook”, you can use the \begin{document} \end{\document} style, all the program examples in this scriptum are produced in such a way.

4.7

Graphics to include into documents

If you want to save the graphics on the MATLAB-Graphics screen as a file that should be included in a text (for e.g. LaTeX or WORD), you have to use the print-command, with the syntax print -dFORMAT FILENAME

64

CHAPTER 4. GRAPHICS

The following table introduces some graphics-formats print -dps2 name.ps

print -dpsc2 name.ps print -deps2 name.eps print -depsc2 name.eps print -djpeg name.jpg If the orientation should

Black-white Postscript-output, Graphics which can be directly plotted on as (Postscript-) printer. All graphics are on a single page in the paper format. Like above, but in color Black-white encapsulated POSTSCRIPT-output, can be included in word-processor-programs like LaTeX. Like the above, but in color. Output in jpeg-format which can be included in Word-processors like Word or for Internet-pages. be changes e.g. to portrait, one can use the MATLAB-command

orient

portrait

4.8

Including Encapsulated Postscript in LaTeX

If you want to include jpeg in GUI-based word-processors, you can use the corresponding menus and/or the mouse. LaTeX, probably the most widely used word-processing program in the sciences, is not GUIbased. A (not so) short introduction about the various commands in various languages can be found on ftp://ctan.tug.org/tex-archive/info/lshort/. LaTeX is rather a ”textprogramming-Language”, which converts a file name.tex (the ”program”) into a file name.dvi (the device-independent-file) which can then be converted into general formats like Postscript via dvips -o name.ps name which produces a postscript-file. These postscript-formats can then e.g. by the command ps2pdf be used to transform the Postscript-format into PDF-format (Adobe-Acrobat). If you want to prepare a LaTeX-document with the corresponding graphics, you have to load a ”package” with the software for including graphics. A widely used package is the epsfigpackage, so whereas for a conventional LaTeX-report, the header looks like \documentclass[twoside,12pt]{report} the header must contain \documentclass[twoside,12pt]{report} \usepackage{epsfig} if postscript-graphics should be included. Graphics can then be included with \epsfig=filename,width=??,height=??,angle= where either width or height must be given, the angle can also be left away. Here are some examples: \epsfig{file=graphiken/circle_square.eps,width=2cm}

r=1

4.9. PROCESSING GRAPHICS

65

\epsfig{file=graphiken/circle_square.eps,height=2cm}

r=1

\epsfig{file=graphiken/circle_square.eps,height=2cm,angle=-90}

r=1

\epsfig{file=graphiken/circle_square.eps,height=2cm,width=4cm}

r=1

In principle, all postscript-files should be includable in Latex, but some Programs produce postscript-output which is not compatible with LaTeX. Under UNIX, one can use the command ps2epsi name.ps name.epsi to convert a file name.ps to a file name.epsi which corresponds to the encapsulated postscript interchange format.

4.9

Processing Graphics

If you have graphics in another format than postscript, like *.jpeg or *.gif files, which you want to include in LaTeX, you have to convert them into postscript with some other software. One of the most widely used programs for this task under UNIX is xv, which allows to load graphics in one format and to save it in another format, for example xv name.jpg will load the program name.jpg. Pressing the right button of the mouse will make a menu appear, and the graphics can be saved as Postscript by choosing the appropriate menu (SAVE → FORMAT → POSTSCRIPT).

Chapter 5 Linear Algebra Usually, one learns about linear algebra in the first year of study, but often, one needs it much later, when one has forgotten most of it already. MATLAB means Matrix LABoratory and its first version was written by Cleve Moler so that his students could learn linear Algebra more easily. General documentation of MATLAB can also be found on http://www.mathworks.com/ access/helpdesk/help/techdoc/matlab.shtml

5.1 5.1.1

Matrix Manipulation Matrix commands

The diagonal of a matrix can be extracted with the diag command in the following way: A = 0.520109 0.510104 0.010375

0.340012 0.326988 0.782090

0.470293 0.636776 0.900370

> diag(A) ans = 0.52011 0.32699 0.90037 If the input of the diag-command is a vector, diag constructs a matrix with the vector on the diagonal, a typical example how commands are overloaded in MATLAB: > b=[3 5 7] b = 3 5 7 > A=diag(b) A =

68 3 0 0

CHAPTER 5. LINEAR ALGEBRA 0 5 0

0 0 7

The operator for the matrix-transpose is the accent ’: > A=rand(2) A = 0.66166 0.48661 0.69184 0.39113 > B=A’ B = 0.66166 0.48661

0.69184 0.39113

Because MATLAB knows the difference between column- and row-vectors, the transposeoperator ’ can also be used to transform column- into row-vectors and vice versa: > v=[1 2 3 4 5] v = 1 2 3 4 5 > u=v’ u = 1 2 3 4 5 For complex-valued matrices, the ’-operator gives the Hermitian conjugate matrix: > H=rand(3)+sqrt(-1)*rand(3) H = 0.59574 + 0.89043i 0.91601 + 0.87663i 0.71691 + 0.73996i 0.31324 + 0.44034i 0.38660 + 0.13756i 0.33661 + 0.71527i

0.19920 + 0.74066i 0.19254 + 0.85119i 0.29184 + 0.58186i

> G=H’ G = 0.59574 - 0.89043i 0.91601 - 0.87663i 0.19920 - 0.74066i

0.38660 - 0.13756i 0.33661 - 0.71527i 0.29184 - 0.58186i

0.71691 - 0.73996i 0.31324 - 0.44034i 0.19254 - 0.85119i

The commands which extract the upper/lower trigonal matrix are triu/tril:

5.2. MATRIX PRODUCTS > A A = 0.951650 0.109170 0.667123

0.084814 0.585341 0.528991

69

0.208357 0.562931 0.860920

> tril(A) ans = 0.95165 0.10917 0.66712

0.00000 0.58534 0.52899

0.00000 0.00000 0.86092

> triu(A) ans = 0.95165 0.00000 0.00000

0.08481 0.58534 0.00000

0.20836 0.56293 0.86092

If the columns or rows should be flipped, i.e. if their order should be inverted, this can be done with the commands fliup and fliplr, ”flip up down” and ”flip left right”: > fliplr(A) ans = 0.73180 0.40541 0.55208 0.79014 > flipud(A) ans = 0.79014 0.55208 0.40541 0.73180 These two commands can be used to form a transposition for a complex matrix, which is not the hermitian conjugate: > A=rand(2)+sqrt(-1)*rand(2) A = 0.839504 + 0.572899i 0.466803 0.086815 + 0.252680i 0.132638 > B=fliplr(flipud(A)) B = 0.132638 + 0.086518i 0.086815 0.466803 + 0.675260i 0.839504

5.2

+ 0.675260i + 0.086518i

+ 0.252680i + 0.572899i

Matrix Products

For matrices and vectors, there are a lot of ways products can be computed. There is no difference between vectors and matrices, a vector is just a matrix with only one row or column. The simplest form is the elementwise product, which uses the operator .*:

70

CHAPTER 5. LINEAR ALGEBRA

> u=[1 2 3 4] u = 1 2 3 4 > v=[5 6 7 8] v = 5 6 7 8 > u.*v ans = 5 12

21

32

The inner product for a row-vector u and a column-vector w is computed with the operator *: > u=[1 2 3 4] u = 1 2 3 4 > w=[1 1 2 2]’ w = 1 1 2 2 > u*w ans = 17 If instead of u*w we compute w*u, the result is the outer product > w*u ans = 1 2 1 2 2 4 2 4

3 3 6 6

4 4 8 8

Matrices can be treated in the same way as vectors with elementwise multiplication .* or multiplication in the sense of linear algebra: > A=[1 2 > 3 4] A = 1 2 3 4 > B=[1 -1

5.2. MATRIX PRODUCTS

71

> -2 2] B = 1 -1 -2 2 > A*B ans = -3 -5

3 5

> A.*B ans = 1 -2 -6 8 A matrix-vector product is performed like this: > A=[1 2 > 3 4] A = 1 2 3 4 > v=[1 > 2] v = 1 2 > A*v ans = 5 11 Matlab also has the Kronecker-Product > u=[1 2 3 4] u = 1 2 3 4

1

as a builtin function,

> v=[5 6 7 8] v = 5 6 7 8 > kron(u,v) ans = 5 6 7

8

10

12

14

16

15

18

21

24

20

24

28

32

72

CHAPTER 5. LINEAR ALGEBRA

> kron(u,v’) ans = 5 10 15 6 12 18 7 14 21 8 16 24

20 24 28 32

Whereas the elementwise matrix product computed with .∗ is commutative, of course the matrix product computed with ∗ is not commutative.

5.3

Repetition of elementary linear Algebra

The angle between vectors to vectors v and w for any finite can be computed via their inner/scalar-product · as |v · w| √ cos θ = √ . v·v· w·w For the inner/ scalar-product, we have the Cauchy–Schwartz–Inequality |v · w| ≤

√

v·v·

√

w · w.

Vectors for which the scalar product is 0 are called orthogonal. Whereas orthogonality of two vectors v and w can be defined in ”theoretical” mathematics as the property that their scalar product is zero, v · w = 1, in numerical mathematics it is necessary to define orthogonality in a way so that possible rounding errors are taken into account, as the following example shows > w=[sqrt(3) sqrt(3)] w = 1.73205080756888 1.73205080756888 > v=[sqrt(3) -sqrt(3)] v = 1.73205080756888 -1.73205080756888 > v*w’ ans = -9.64636952420157e-17 Obviously the last result should be ”exactly” zero, but due to the rounding errors in the computation, there is a finite error. How the definition of orthogonality can be applied in such a way that rounding errors are taken into account can be seen in the next section about the rank of matrices.

5.3.1

Size and Rank of Matrices

The size of a matrix A can be computed with the size-command:

5.3. REPETITION OF ELEMENTARY LINEAR ALGEBRA

73

> size(A) ans = 2 2 size gives a two-row vector as an answer, the number of columns is size(A,2), the number of rows size(A,1). Also the length of columns / rows for a column/ row vector v can be computed with size(v,2) / size(v,1) The rank of a matrix is in ”theoretical” linear algebra the number of linear independent rows/columns. Because the definition of linear independence is equivalent to the definition of orthogonality, we will use the rank computation as the criterion for the orthogonality. The rank of a matrix can be computed with MATLABS’srank command. How the rank command works, will be explained later in the section about the singular value decomposition, along how one should choose the optional threshold in MATLABS’s rank command. First let us review some theorems about the rank of matrices1

5.3.2

Some Theorems on the rank of matrices

• The outer product of two matrices of two vectors always gives a matrix of rank 1. Example: • Random matrices have ”nearly always” full rank, i.e. the rank of a matrix constructed with rand is the same as the number of the columns/rows, and if the number rows/columns larger than the number of columns/rows, we have rank(A)=min(size(A)): > A=rand(3,4) A = 0.23382 0.43570 0.79868 0.34546 0.66927 0.71192

0.42862 0.69142 0.15419

0.97961 0.74305 0.11667

> rank(A) ans = 3

• Square matrices which have a rank smaller than their number of columns/rows are called singular. They cannot be inverted, and systems of linear equations where the equations form a singular matrix cannot be solved. Their determinant vanishes. • The Rank of a matrix does not change through transposition, complex or hermitian conjugation. • The product of non-singular matrices as the same rank as the matrices themselves: > A=rand(3,4) A = 1

Roger A. Horn, Charles R. Johnson, Matrix Analysis Cambrigde University Press 1991

74

CHAPTER 5. LINEAR ALGEBRA 0.476924 0.061165 0.139677

0.068071 0.885041 0.708093

0.420827 0.027155 0.489577

> B=rand(3) B = 0.908288 0.703948 0.245781 0.950685 0.942011 0.726192

0.363589 0.097344 0.064962

> C=B*A C = 0.52703 0.18896 0.50276

0.94231 0.92705 0.75283

0.57935 0.17690 0.44795

0.883968 0.417966 0.978820

1.45301 0.70990 1.19982

> rank(A) ans = 3 > rank(B) ans = 3 > rank(C) ans = 3 • For rank-deficient matrix, the rank of the product matrix is the same as that of the matrix with the lowest rank: > A=rand(2,4) A = 0.200421 0.795092 0.838726 0.220597

0.896583 0.018236

0.454798 0.018493

> A(3,:)=A(2,:) A = 0.200421 0.795092 0.838726 0.220597 0.838726 0.220597

0.896583 0.018236 0.018236

0.454798 0.018493 0.018493

> B=rand(3) B = 0.94359 0.31700 0.50896 0.36833 0.48172 0.19705

0.20635 0.40063 0.42594

> C=B*A C = 0.62806

0.85555

0.86569

0.43882

5.3. REPETITION OF ELEMENTARY LINEAR ALGEBRA 0.74695 0.61907

0.57430 0.52044

0.47035 0.44326

75

0.24569 0.23061

> rank(A) ans = 2 > rank(B) ans = 3 > rank(C) ans = 2

5.3.3

Rank-Inequalities

• For A ∈ Mm,n we have rankA ≤ min(m, n) • When a column or a row of a matrix are deleted, the rank of the resulting matrix cannot be larger than the rank of the original matrix. • For A ∈ Mm,k , B ∈ Mk,n we have rankA + rankB − k ≤ rankAB ≤ min(rankArankB) • For A, B ∈ Mm,k , rank(A + B) ≤ rankA + rankB • For A ∈ Mm,k , B ∈ Mk,p , C ∈ Mp,n we have rankAB + rankBC ≤ rankB + rankABC.

5.3.4

Norms of a Matrix

Every Matrix-Norm can also be used as a vector-norm, but not vice versa. Therefore, we explain here only definitions for matrix-norms. Analogous to real and complex Scalars, one wants to use something like an absolute value also for matrices. Something which behaves like an ”absolute value” under addition is the Norm of a matrix.

Properties of Matrix–Norms • ||A|| ≥ 0 (Non-negativity) • ||A|| = 0 if A = 0 • ||cA|| = |c| · ||A|| for all real and complex c (homogeneity) • ||A + B|| ≤ ||A|| + ||B|| (triangle inequality) • ||AB|| ≤ ||A|| · ||B||(sub-multiplicativity)

76

CHAPTER 5. LINEAR ALGEBRA

Definitions for Norms In MATLAB, all of the following norms exist, and they can be computed via the function norm(x) and if necessary further arguments. Name Definition MATLAB–Function ∗ 1. Spectral Norm ||A||2 maximal Eigenvalue of A A norm(A), norm(A,2) 2. 1–Norm ||A||1 maximal row sum of A norm(A,1) 3. ∞–Norm ||A||∞ maximal column sum of A norm(A,’inf’) & + ∗ 4. Frobenius–N. ||A||Fro ( Ai,j Aj,i ) norm(A,’fro’)

5.3.5

Determinant of a Matrix

The norm only fulfills sub-multiplicativity, i.e. the norm of a matrix product is equal or smaller than the product of the norms of the factors. An ”absolute value” which fulfills the multiplicativity is the determinant, which can be computed in matlab via det(A): det A · det B = det(A · B)

Further properties of the matrix are:

• The exchange of two adjacent columns/rows inverts the sign of the matrix: > A=rand(3) A = 0.209224 0.413728 0.106481 0.192283 0.291095 0.436435

0.212479 0.074438 0.508115

> det(A) ans = -0.0017939 > B=[A(:,2) B = 0.413728 0.192283 0.436435

A(:,1) A(:,3)] 0.209224 0.106481 0.291095

0.212479 0.074438 0.508115

> det(B) ans = 0.0017939 • The determinant of the identity matrix is one, independent of its dimension. • Never use the Cramer Rule or the Jacobi expansion for the computation of a determinant, is is wasteful and numerically instable. • The numerically most suitable computation method for determinants is the so-called LU-decomposition, where the matrix A is decomposed as a product of an lower triangular matrix L with 1’s on the diagonal and an upper triangular Matrix U as L · U = A.

5.3. REPETITION OF ELEMENTARY LINEAR ALGEBRA

77

The determinant of A is therefore the product of the diagonal entries of U. Rowand Column-Permutations, so called pivoting, increases the numerical accuracy of the decomposition, for details, see [Gol89]. The MATLAB-command which computes the matrix determinant via LU-decomposition is det.

5.3.6

Matrix inverses

Nonsingular square matrices are inverted by the inv command. The elementwise division of one matrix by another in MATLAB is written as A./B, where all entries of the divisor matrix must be )= 0 to avoid an error message. This is totally different from the MATRIX division A/B, which corresponds to the multiplication of matrix A with the inverse of matrix B: > A=rand(2) A = 0.29975 0.85007 0.88812 0.33290 > B=rand(2) B = 0.89979 0.72370 0.53648 0.97567 > A/B ans = -0.33410 1.40492

1.11909 -0.70090

> C=inv(B) C = 1.9926 -1.4780 -1.0956 1.8376 > A*C ans = -0.33410 1.40492

1.11909 -0.70090

Because the product C*A does not necessarily the same result as the product A*C, there is also the ”right division” of a matrix, which with the above matrices gives > C*A ans = -0.71537 1.30362 > B\A

1.20181 -0.31963

78

CHAPTER 5. LINEAR ALGEBRA

ans = -0.71537 1.30362

1.20181 -0.31963

If one tries to invert a singular matrix, MATLAB gives a result (usually wrong), and issues an error message: > A=[1 1 > 1 1] A = 1 1 1 1 > inv(A) warning: inverse: matrix singular to machine precision, rcond = 0 ans = 1 1 1 0 > B=inv(A) warning: inverse: matrix singular to machine precision, rcond = 0 B = 1 1 1 0 > B*A ans = 2 2 1 1

5.4

How many matrix products are possible

A matrix product is computed using three indices i, j, k, aij =

(

bik ckj

k

Therefore, there are 6 possible orders to to program the loops, but basically, there are only two possibilities: clear format long n=20 b=randn(n).*10.^(16*randn(n)); c=randn(n).*10.^(16*randn(n));

5.4. HOW MANY MATRIX PRODUCTS ARE POSSIBLE tic % Version 1: Dot-Product a1=zeros(n); for j=1:n for i=1:n for k=1:n a1(i,j)=a1(i,j)+b(i,k)*c(k,j); end end end toc

tic % Equivalent to a2=zeros(n); for j=1:n for i=1:n a2(i,j)=b(i,:)*c(:,j); end end toc tic % Version 2: Daxpy-Product a3=zeros(n); for j=1:n for k=1:n for i=1:n a3(i,j)=a3(i,j)+b(i,k)*c(k,j); end end end toc tic % equivalent to a4=zeros(n); for j=1:n for k=1:n a4(:,j)=a4(:,j)+c(k,j)*b(:,k); end end toc return

79

80

CHAPTER 5. LINEAR ALGEBRA

We have also included the tic and toc command to profile the time used for a matrix multiplication. It can be seen that MATLAB performs much faster if the inner loop is evaluated using the :-notation. The first version of the matrix-matrix multiplication has a inner vector product as a ”Kernel”, the inner part of the routine. The second version of the Matrix-multiplication has a kernel which can be written as (y = a(x + (y , an operation where the left side in words is ”A X Plus Y”, for which often the acronym SAXPY or DAXPY (S for single, D for double precision) is in use. It turns out that both operations are numerically equivalent, and both need 2l3 floating point operations (multiplications and additions). It is common to give the speed of computers by how many Floating Point Operations Per Second (Flops) then can perform. Modern PC’s are in the range of a few hundreds MFlops, Workstations are nowadays in the GFlops-Range, and the Earth Simulator, a Supercomputer near Yokohama, can to about 4 TeraFLOPS. Using programs to test the speed of Computers is called benchmarking.

5.5 5.5.1

Matrix Inverses again How to solve a linear system by hand

The standard way to solve a linear system of k equations with k unknowns, Ax = b, with the unknowns in the vector x and the right-hand-side b      

a1,1 a1,2 · · · a1,k a2,1 a2,2 · · · a2,k .. .. .. .. . . . . ak,1 ak,2 · · · ak,k

is to rewrite the system in ”augmented form”      

     

x1 x2 .. . xk

a1,1 a1,2 · · · a1,k a2,1 a2,2 · · · a2,k .. .. .. .. . . . . ak,1 ak,2 · · · ak,k





    =    

 - b - 1  - b2  - .  - .  - .  - bk

b1 b2 .. . bk

     

and transform the matrix and the right-hand-side vector b via elementary row- and columnoperations (subtracting multiples of some rows from other rows) to upper triangular form, where all the elements below the diagonal are 0:      

a ˜1,1 a ˜1,2 · · · a ˜1,k 0 a ˜2,2 · · · a ˜2,k .. .. .. .. . . . . 0 0 ··· a ˜k,k

-

˜b1 ˜b2 .. . ˜bk

     

5.5. MATRIX INVERSES AGAIN

81

The solutions for x1 , x2 , . . . xk can then be computed via back-substitution as xk = b/˜ ak,k / . ˜k−1,k xk /˜ ak−1,k−1 xk−1 = ˜bk−1 − a

xk−2 = xi =

.

/

˜bk−2 − a ˜k−2,k−1 xk−1 − a ˜k−2,k xk /˜ ak−2,k−2 



k ( 1 ˜ bi − a ˜i,j xj  a ˜i,i j=i+1

This scheme of eliminating elements so that a triangular coefficient matrix survives for which the unknowns can be computed in a trivial way is called Gaussian elimination. As an example 

in augmented form









9 3 4 x1 7       4 3 4   x2  =  8  1 1 1 x3 3 

-



9 3 4 -- 7    4 3 4 - 8 . 1 1 1 - 3

We start by interchanging the first and the last row 

-



1 1 1 -- 3    4 3 4 - 8 . 9 3 4 - 7

Next, we subtract 4times the first row from the second row and nine times the first row from the last row:   1 1 1 -3   0 - −4  .  0 −1 0 −6 −5 - −20 Finally, we add -6 times the second row to the last row, and obtain the triangular system 

1 1 1  0  0 −1 0 0 −5

 - 3  - −4  , - −4

from which we can compute the unknowns successively as x= − 4/5, x= 4 and x= − 1/5.

5.5.2

The numerical variants: LU-decomposition

For numerical purposes, the two steps, reduction to a triangular system (elimination step), and backward substitution (solution step), are often split up in two routines. A common collections of subroutines for the computation of numerical linear algebra is the LINPACKpackage, which includes matrix inversions and orthogonalization methods for real and complex matrices. MATLABS routines for linear algebra are basically routines from LINPACK, and Cleve Moler, the inventor of MATLAB was also a co-author of LINPACK.

82

CHAPTER 5. LINEAR ALGEBRA

Numerically, the Gaussian elimination is usually implemented as a LU-decomposition, a factorization of the Matrix A of the system Ax = b into an upper trigonal matrix U and lower trigonal matrix L, so that LU = A so that the solution can again be computed in a trivial way. In MATLAB, the LU-factorization can be computed via the lu command, for example as > a a = -1.0688456296920776 0.0473232455551624 -0.5952438712120056

0.5834664106369019 -0.6955339908599854 -0.0617007017135620

-0.0174380335956812 -0.2883380949497223 -1.1060823202133179

> [l,u]=lu(a) l = 1.000000000000000 -0.044275098518011 0.556903499136249

0.000000000000000 1.000000000000000 0.577325122175704

0.000000000000000 0.000000000000000 1.000000000000000

u = -1.068845629692078 0.000000000000000 0.000000000000000

0.583466410636902 -0.669700958047086 0.000000000000000

-0.017438033595681 -0.289110165605131 -0.929460456605607

The solution of a linear system Ax = b can be computed in MATLAB with the slashcommand, which is not only the division ”from the left” for scalars, 5/4 ans = 1.25000000000000 > 5\4 ans = 0.800000000000000 but also for matrices. The Algebraic meaning is A\B = A−1 · B,

A/B = A · B −1 ,

and for matrices (Remember that MATLAB means MATRIX Laboratory), this is not necessarily the same. The solution for Ax = b can be obtained by formally dividing through A from the left, Ax = b → A\Ax = A\b → x = A\b. The solution of the triangular system, including with testing whether Ax is really equal b, can then be programmed in the following way:

5.5. MATRIX INVERSES AGAIN

83

> A=rand(3) A = 0.63356 0.98480 0.60858

0.25786 0.13788 0.76457

0.71159 0.62761 0.90059

> b=rand(3,1) b = 0.81931 0.61835 0.14195 > x=A\b x = -0.74296 -2.36996 2.67169 > A*x ans = 0.81931 0.61835 0.14195 In LINPACK, the elimination step is called factoring (because the LU-decomposition produces two factors, L and U), and the Double precision GEneral matrix FActoring is therefore called DGEFA. The solution/substitution of the system is DGESL , SL for solution. There exists also a LINPACK-benchmark, which sets up matrices in a well-defined way and computes the matrix inverses, then computes the number of floating point operations and the time and then computes the Flop-rate. In this way, the speed of computers has been evaluated for decades.2

5.5.3

Matrix inversion

A matrix inversion can be computed in the same way as the solution of a linear system, which we see that if we write the problem as

AA

−1



  = E, with the identity-matrix E =   



1 0 ··· 0 0 1 ··· 0   .. .. . . ..  . .  . .  0 0 ··· 1

http://www.netlib.org/benchmark/linpackd in FORTRAN, but also available in other languages, the results in http://www.netlib.org/benchmark/linpackd/performance.ps 2

84

CHAPTER 5. LINEAR ALGEBRA

where the columns of the identity matrix E have the role of the right-hand side b and the columns of A−1 are the unknowns x. It is now clear why it is advantageous to use the LU-decomposition, as it allows the simultaneous solution of systems with arbitrary many columns on the right hand side. After the factoring is completed, the solution step for computing the inverse of a l×l matrix takes l times as many steps as the solution of the system for a single column right-handside. We can see this by using the flops-command which was available with old versions of MATLAB (before version 6), which measured the number of floating point iterations, and the example program on the right. For 150 × 150 matrices, the number of FLOPS necessary for the solution of the linear system is 2419042, for the computation of the matrix inverse it is 6907967. This means that the number of FLOPS required for the matrix inversion is about three times as much than the solution of the linear system. For the solution of the linear system, the highest computational cost is actually the factoring, not the backward substitution, and we can see that the backward substitution takes twice as much operations as the factoring itself.

5.5.4

clear format compact n=150 A=randn(n); b=randn(n,1); flops(0) x1=A\b; flops flops(0) x2=inv(A)*b; flops return >> n = 150 ans = 2419042 ans = 6907967

Accuracy of the matrix inversion

Up to now, we have not discussed the error in matrix inversions. As we have not used any order of approximation, it is clear that there will be neither truncation error nor discretization error, and only rounding errors have to be taken into consideration. As a test case for the matrix inversion, let us consider the matrix A= which has the inverse −1

A

=

)

)

1 1 1−$ 1

*

1/$ −1/$ −(1 + $)/$ 1/$

*

If we compute the inverse in double precision, for # = 10−8 , we obtain for A= 1.000000000000000 0.999999990000000

1.000000000000000 1.000000000000000

the inverse 99999999.4975241 -99999998.4975241

-99999999.4975241 99999999.4975241

5.6. EIGENVALUES

85

instead of the expected result −1

A

=

)

108 −108 −100000001 108

*

.

What went wrong? The numerical parameter which describes how accurate a matrix inversion can be computed, or a linear system can be solved, is the condition number κ, which is implemented in MATLAB’s cond-function. The condition number for a matrix A is defined as the norm of the Matrix |A| divided by the norm of the inverse matrix |A−1 |, or, if |A|/|A−1 | < 1, then as the inverse κ = |A−1 |/|A|. There is a heuristic, which says that if the condition-number κ of a matrix A is ≈ 10k , for a matrix inversion about k digits will be lost in accuracy. four our above matrix with # = 10−8 , the condition number is κ = 4 · 10−8 . as the error is about 0.5 for a matrix for which the entries are of the order of 108 , we see that the predictions of the Heuristic are quite accurate. We have discussed that there are two possible implementations of the matrix-multiplication, the DOT and the DAXPY-product. The LU-decomposition can formally written as an operator O acting on the original matrix A, so that formally O · A = L · U.

In other words, the LU-decomposition is a very special matrix-matrix multiplication, and therefore there a two variants. The conventional DAXPY-variant, which is widely treated in textbooks on numerical analysis, is implemented in MATLAB, and in Packages like LINPACK, LAPACK, NAG and visual numerics. The rather rarely mentioned DOT-variant, for which depending on the implementations Names like Doolittle, Crout or Crout-Doolittle are used, is basically only used in NUMERICAL RECIPES, a compendium of numerical routines, where none of the authors has a background in analysis. A DDOT-routine in MATLAB (not shown, because I don’t want anybody to use it) for the above problem produced the result 1.99999999 -0.99999999

-1.00000000 1.00000000

and the error was not in the eight digit, but already in the first digit! The scalar product as a kernel introduces rounding errors which cannot be predicted with the conventional formula using the condition number.

5.6

Eigenvalues

The eigenvalues can be computed in MATLAB via the eig command. For a random matrix A, we obtain the eigenvalues as >> A=randn(2) A = 0.5181 -1.2274 0.8397 0.1920 >> eig(A) ans = 0.3551 + 1.0020i 0.3551 - 1.0020i

86

CHAPTER 5. LINEAR ALGEBRA

so one can see that the eigenvalues of a real square matrix are in general not real. For a symmetric matrix, we see that

>> A=A+A’ A = 1.0363 -0.3876 >> eig(A) ans = 1.2167 0.2035

-0.3876 0.3840

we obtain real eigenvalues. Formally, the eigenvalues λi of a matrix A are often introduced as the roots of the characteristic polygyon of A,

det(A − λE) = 0,



  E=  



1 0 ··· 0 0 1 ··· 0   . .. .. . . ..  . .  . .  0 0 ··· 1

For a diagonal 2 × 2 matrix, det

))

a 0 0 b

*

−λ

)

1 0 0 1

**

= (a − λ)(b − λ) = 0,

we see that the solutions for λ are exactly a and b. In other words, the eigenvalues of a diagonal matrix are the diagonal matrix entries themselves. As we have seen above, the eigenvalues of a real symmetric diagonal matrix are real. If we look at the characteristic polynomial, we see that for an upper triangular matrix, the off-diagonal elements vanish in the characteristic polynomial, so also for a trigonal matrix the eigenvalues are exactly the diagonal elements:

A = 1.0363 0 >> eig(A) ans = 1.0363 0.3840

-0.3876 0.3840

5.6. EIGENVALUES

5.6.1

87

What to do with eigenvalues

For a non-diagonal matrix A, the ”action” of the non-diagonal matrix can usually be replaced ”somehow” by the ”action” of one or more eigenvalues. One example for such an ”action” is the multiplication of a matrix with a vector. We if we multiply a vector iteratively with a matrix A and compute of the norm of the vector, for example with the program on the right, we find that after several iterations the length of the iterated vector become the absolute value of the largest eigenvalue of the matrix: ans = -0.23606797749979 norm_of_v = norm_of_v = norm_of_v = norm_of_v = norm_of_v = norm_of_v = norm_of_v = norm_of_v =

5.6.2

4.23606797749979 4.12310562561766 4.23570259468110 4.23606684261261 4.23606797397526 4.23606797748884 4.23606797749976 4.23606797749979 4.23606797749979

clear format compact format long v=[1 1]; v=v/norm(v); A=[1 2 2 3]; eig(A) for i=1:8 v=A*v; norm_of_v=norm(v) v=v/norm(v); end

Diagonalization and Eigenvectors )

*

1 2 Now, if we have a matrix A = we can call eig not only to compute the eigen2 3 values, but using two output-arguments in constructor-brackets [], we can also obtain the eigenvectors as >> [u,l]=eig(A) u = 0.85065080835204 -0.52573111211913 l = -0.23606797749979 0

0.52573111211913 0.85065080835204 0 4.23606797749979

88

CHAPTER 5. LINEAR ALGEBRA

(the eigenvectors l are then not outputted as a vector, but as a diagonal matrix). In our above example, where we iteratively multiplied the vector v with the matrix A, the end result for v is >> v v = 0.52573111213781 0.85065080834049 which is the right column of u, and therefore the eigenvector to the larger eigenvalue λ2 = 4.23606797749979. In other words, out iterative multiplication of a vector to a matrix is a way to find the largest eigenvalue and the eigenvector corresponding to this largest eigenvector, and in the literature, this method is often called the power method, because it corresponds to multiplying a power of A onto v : An v− > (umax The matrix u which contains the eigenvectors is at the same time the transformation which transforms A onto diagonal form, so that %

u Au =

)

−0.23606797749979 0 0 4.23606797749979

*

The matrix u is called a unitary transformation

5.6.3

Computing the characteristic polynomial

As we have used Newton-Raphson Iteration for the computation of roots of polynomials, one could think that this would also be a good method to compute the eigenvalues from the characteristic polynomial det(A − λE) = 0. Actually, this is not the case. The numerical algorithm for the computation of the eigenvalues, which will not be elaborates here, makes use of all of the l × l matrix entries, whereas the solution of the characteristic polynomial only makes use of l coefficients computed from the l × l matrix entries, so again we loose significant information as in the example for the intersection computation of ellipses via the fourth order polynomial. On the contrary, instead of computing eigenvalues via roots it is usually feasible to compute the roots of a polynomial by rewriting it as the corresponding eigenvalue problem. First let us divide the polynomial P˜ (x) P˜ (x) = a ˜ k xk + a ˜k−1 xk−1 . . . a ˜0 = 0 by the leading coefficient a ˜k , so that our equation looks like P (x) = xk + ak−1 xk−1 . . . a0 = 0.

5.6. EIGENVALUES

89

Then one can set up the so-called companion matrix CP for P (x) e.g. as 

CP

   =   

0 0 .. .

1 0 .. .

0 1 .. .

··· ··· .. .

0 0 .. .

0 0 0 ··· 1 −a0 −a1 −a2 · · · −ak−1



    ,  

and the eigenvalues of CP are the roots of the polynomial P (x). For example, the polynomial P (x) = x3 − 2x2 − 5t + 6 = 0 has the roots x1 = 1, x2 = −2, x3 = 3. If we set up the companion matrix as C = 0 0 -6

1 0 5

0 1 2

We obtain as the eigenvalues of the companion matrix C > eig(C) ans = 3.0000 1.0000 -2.0000

5.6.4

Stability Analysis

Eigenvalues play an important role in stability analysis, i.e. in the analysis whether a numerical problem is ”stable” or not. Instability usually results from eigenvectors which are larger than 1 (or, in some cases, different from 1). As an example of how eigenvalues enter in the solution of problems, let us look at the example problem for the ordinary differential equations: function dydt = f(t,y) % necessary parameters as global variables global D global omega0 % velocity component dydt(1)=-omega0^2 * y(n,2)- 2*D*y(n,1) ; % position component dydt(2)=y(n,1); return This can also be written in matrix notation as

90

CHAPTER 5. LINEAR ALGEBRA

function dydt = f(t,y) % necessary parameters as global variables global D global omega0 % velocity component dydt=[- 2*D -omega0^2 0 1 ]* y; % position component return )

*

−2D −ω 2 Obviously, the Matrix A = has Eigenvalues, and the integration step is given 0 1 by At., so in fact errors in the time integration can be analyzed by analyzing the eigenvalues of Adt. For this harmonic oscillator, the Matrix is trigonal and the eigenvalues are obviously (−2Ddt, dt) , and are therefore constant in time, so the problem can be analyzed also purely analytical. For more complicated problems, like for the ordinary differential equations of the Lorentz attractor dx dt dy dt dz dt

= σ(y − x) = rx − y − xz = xy − bz,

with real constants σ, r, b the matrix of the ordinary differential equation is obviously a nondy dz linear function, because the time evolution ( dx dt , dt , dt ) cannot be written as a product of a matrix A independent of x, y, z and the vector x, y, z, like in the case of the harmonic oscillator. The classical way to analyze the stability of such a system is to ”linearize” the matrix, usually a risky business, because the linearized matrix is not guaranteed reproduce the full behavior of the non-linear system. The ”modern” approach is to simply perform the time integration and output representative values for the eigenvalues of the matrix 



−σ σ 0   B =  r −1 −x  . 0 x −b

Now let us solve the Lorentz-Model with constant stepsize using the Euler method and let us plot the Eigenvalues of the matrix. We know already that the Euler-method is bad, so our solution will be inaccurate, but it will be much more interesting to implement the Euler-method in two different ways and see how the two solutions diverge from each other. % Compute the Lorenz-Model clear,format compact n=20; r=60; b=8/3; sigma=10;

5.6. EIGENVALUES

91

t_max=1.3 dt=0.01 % diverges with this timestep: dt=0.011; ndt=round(t_max/dt); x=zeros(ndt,1); mateig=zeros(ndt,1); x(1)=1; y=x; z=x; bild=0; k=[1 1 1]’; k(:,2:ndt)=zeros(3,ndt-1); prop=[ -sigma sigma 0, 0 -1 0, 0 0 -b]; % L\"osung der DGL direkt for i=1:ndt-1 dx=sigma*(y(i)-x(i))*dt; dy=(x(i)*(r-z(i))-y(i))*dt; dz=(x(i)*y(i)-b*z(i))*dt; x(i+1)=x(i)+dx; y(i+1)=y(i)+dy; z(i+1)=z(i)+dz; end % Loesung der DGL mit Matrix% Vektor-Multiplikationen for i=1:ndt-1 prop(2,1)=(r-k(3,i)); prop(3,1)=(k(2,i)); k(:,i+1)=dt*(prop*k(:,i))+k(:,i); mat_eig(i+1)=max(abs(eig(prop*dt))); end subplot(4,1,1) plot3(x(1:ndt),y(1:ndt),z(1:ndt)); subplot(4,1,2) plot3(k(1,1:ndt),k(2,1:ndt),k(3,1:ndt)); subplot(4,1,3) plot3(k(1,1:ndt)-x(1:ndt)’,... k(2,1:ndt)-y(1:ndt)’,... k(3,1:ndt)-z(1:ndt)’); subplot(4,1,4) plot(mat_eig) This is the first surprise, that two implementations of the Euler-method don’t give ”numerical identical results”, and the difference increases if we increase the maximal time. The next

92

CHAPTER 5. LINEAR ALGEBRA

surprise comes when we increase the timestep from dt=0.01 to dt=0.013. We can see that then the Solution and the maximal eigenvalues start to diverge, and if the maximal time is taken longer, the program even crashes because it reaches infinity. Here we have found the property of the solution of differential equations, that the eigenvalues of the corresponding matrix times the time step may not become larger than 1, or the solution does not converge any more. The eigenvalue-spectrum obtained from the Euler-method is also representative for the eigenvalues which we would obtain from higher-order Methods like Runge-Kutta, which are itself only a ”sophisticated” concatenation of Euler-steps with different step-size.

5.6.5

Eigenvalue condition number

As in the case of the matrix inversion, there is a parameter which tells how accurately the condition of the eigenvalues could be performed. In MATLAB, the function which gives the eigenvalue condition number (different from the condition number cond for matrix inverses) is called condeig, and it gives the inverses of the eigenvectors of the matrix.

Chapter 6 Ordinary differential equations For ordinary differential equations there is a closed theory about which solution method should be applied in which case. In the case of ordinary differential equations, the total differential imposes additional constraints on the solution so that the numerical equations can be satisfied more easily. In contrast, Partial differential equations are much more difficult to treat numerically, because the boundary conditions impose certain constraints on the solution method, so that in the case of nonlinear equations, the optimal choice for a solution strategies is far from obvious.

6.1 6.1.1

Reference Example Newton’s equations of motion

Ordinary differential equations play a an important role in science and engineering, and maybe the most central equation is Newton’s equation of motion, which relates a time- , velocity and position-dependent force F (x, x, ˙ t), mass m and a F (x, x, ˙ t)x = ma. Rewriting the equation with the second derivative of the position x ¨, we get F (x, x, ˙ t) = m¨ x, which due to the second derivation of x is called a ordinary differential equation of second order. In general, it can be shown that n ordinary differential equations of order m can be rewritten into n · m coupled differential equations of first order. For the case of Newtons equations of motion, this can be done by introducing the velocity v as the derivative of x so that F (x, x, ˙ t) = mv, ˙ v = x. ˙ Because standard texts in numerical analysis prefer to deal with first order differential equations, is is importand to understand the latter form.

94

6.1.2

CHAPTER 6. ORDINARY DIFFERENTIAL EQUATIONS

Linear oscillator

2 x, which For simplicity, we set the mass m = 1. If the force takes the form −2Dxomega ˙ 0 corresponds to a linear spring with linear damping, the equation of motion takes the form

−2Dx˙ − ω02 = x ¨. The solution of this equation with the Damping term 2D and the frequency of the undamped oscillation ω0 is x(t) = x0 exp(−Dt) cos(ωD t), ωD =

&

ω02 − D2 .

Though there are solution schemes to solve second order equations directly, it is usually simpler to solve equation of second order by reducing them to a system of coupled first order equations. For our problem, we introduce the velocity v and its time derivative v¨ = a, so this leads to the system of first order equations v˙ = −2Dx˙ − ω02 x,

x˙ = v.

It is customary in the mathematical community to introduce the vector y = vector symbol( ) and to rewrite the equation as

8v9 x

(without

d y = F (y, t), dt where F (y, t) becomes a vector-valued function with the time t and the vector y as argument.

6.2

The Euler Method

When faced with a differential for an numerical implementation, intuitively one first wants to replace the differential operator d by the finite difference ∆ as dy y(t + ∆t) − y(t) ≈ dt ∆t so that the first order approximation in the solution of one time-step ti with value y(ti ) and the Function value Fi = F (yi , ti ) to the next is y(ti + ∆t) ≈ y(ti ) + Fi ∆t, which is called Euler method. clear format compact D=.2 , omega0=1 % Damping and Force constant x0=1 , v0=0 %Initial conditions

6.2. THE EULER METHOD

95

dt=0.1, t0=0, t_max=30 % time-step, start-time, end time y(1,1)=v0 y(1,2)=x0 t(1)=t0 n=1 while (t(n)