Lecture 4. Matrix Equations and Linearity Principle - Wright State ...

Lecture 4. Matrix Equations and Linearity Principle Recall that a vector in Rm consists of m ordered real numbers, and that a m £ n matrix A = [~a1 ~a2 ::: ~an ]m£n consists of n ordered vectors ~a1 ; ~a2 ; :::; ~an in Rm ; where 2 3 2 3 2 3 a11 a12 a1n 6 a21 7 6 a22 7 6 a2n 7 6 7 6 7 6 7 ~a1 = 6 .. 7 ; ~a2 = 6 .. 7 ; :::;~an = 6 .. 7 : 4 . 5 4 . 5 4 . 5 am1 am2 amn We may view a m £ n matrix as a vector in Rmn as 2 a11 6 2 3 6 ... 6 ~a1 am1 6 ~a2 7 6 6 7 6 6 6 .. 7 = 6 a12 4 . 5 6 .. 6 . ~an 6 a 4 m2 .. .

3

7 7 7 7 7 7: 7 7 7 7 5

In this way, matrix addition, subtraction, and scalar multiplication can be de…ned in the way analogous to vectors, and enjoy all the same properties as for vectors. We shall discuss later in the next lecture about matrix operations. However, matrix notation has its own advantage over vector notation. ² Products of matrices and vectors Let A = [~a1 ~a2 ::: ~an ] be a m £ n matrix whose ith column is 2 3 a1i 6 a2i 7 m 7 ~ai = 6 4 ::: 5 2 R ; ami

i = 1; 2; :::; n; and ~x a (unknown) vector in Rn : 2

a11 a12 6 a21 a22 A=6 4 ::: ::: am1 am2

3 ::: a1n ::: a2n 7 7 ; ::: ::: 5 ::: amn m£n

1

2

6 6 ~x = 6 4

x1 x2 .. . xn

3 7 7 7 5

n£1

:

Notice that the number of columns in A = the number of rows in ~x: The product A~x is de…ned as the linear combination of ~a1 ;~a2 ; :::; ~an with the weight x1 ; x2 ; :::; xn ; i.e., 2 3 x1 n 6 x2 7 X 6 7 A~x = [~a1 ; ~a2 ; :::;~an ] 6 .. 7 = xi~ai = x1~a1 + x2~a2 + ::: + xn~an : 4 . 5 i=1 xn In particular, the system of linear equation, or its equivalent vector equation x1~a1 + x2~a2 + ::: + xn~an = ~b can now be rewritten as Matrix equation: A~x = ~b: If A is 1 £ n matrix, i.e., A = [a1 ; a2 ; :::; an ] ; then 2 3 x1 6 x2 7 6 7 A~x = [a1 a2 ::: an ] 6 .. 7 = x1 a1 + x2 a2 + ::: + xn an 4 . 5 xn 2 3 2 3 a1 x1 6 a2 7 6 x2 7 6 7 6 7 = 6 .. 7 ¢ 6 .. 7 4 . 5 4 . 5 an xn Example 4.1 (a) ·

(b)

2

1 2 ¡1 2 5 3

¸

2

3 · ¸ · ¸ · ¸ 4 1 2 ¡1 4 3 5=4 +3 +7 2 5 3 7 · ¸ · ¸ 1 ¢ 4 + 2 ¢ 3 + (¡1) ¢ 7 3 = = 2¢4+5¢3+3¢7 44

3 2 2 ¡2 3 2 3 2 1 6 8 7 6 0 1 74 5 6 6 8 4 ¡5 2 0 5 3 = 1 4 ¡5 2 0 1 2 0

3

3 2 ¡2 3 7 6 0 7 6 1 7+36 7 6 5 4 2 5+24 0 1 2

2

2

3

2

3 2 7 6 10 7 7=6 7 5 4 1 5 7

Note that for product (b), one can also redo it using "dot product" as 2 3 3 2 2 3 2 3 2 1 2 1 £ ¤ 6 2 ¡2 3 4 3 5 7 6 4 ¡2 5 ¢ 4 3 5 6 7 6 6 7 6 6 2 23 7 6 2 3 3 2 23 6 7 6 8 1 6 £ 7 6 2 3 ¤ 1 6 6 4 0 5¢4 3 5 2 ¡2 3 2 3 6 8 0 1 4 3 5 7 7 6 6 8 7 1 6 7 6 0 1 2 6 74 3 5 = 6 2 3 7 = 6 2 1 3 22 3 6 7 6 4 ¡5 2 0 5 1 6 £ ¤ 1 7 6 ¡5 2 6 7 6 4 5 4 5 4 0 1 2 3 7 6 2 ¢ 3 5 6 ¡5 2 0 6 7 6 6 2 23 7 6 2 0 3 2 23 6 7 6 0 1 6 £ 7 6 ¤ 1 4 5 4 4 5 4 5 4 0 1 2 3 1 ¢ 3 5 2 2 2 The matrix-vector product has the following PROPERTIES:

3

7 7 7 7 7 7 2 7 7 7 6 7=6 7 4 7 7 7 7 7 7 7 5

3 2 10 7 7 1 5 7

A (~x + ~y ) = A~x + A~y ; A (¸~x) = ¸A~x (A + B) ~x = A~x + B~x; (¸A) ~x = ¸(A~x) These relations can be summarized so-called linearity principle: (aA + bB) (¸~x + ±~y ) = a¸A~x + a±A~y + b¸B~x + b±B~y : ² Matrix Equations As we indicated above, the system of linear equation, or its equivalent vector equation x1~a1 + x2~a2 + ::: + xn~an = ~b can now be rewritten as Matrix equation: A~x = ~b; where matrix A has columns ~a1 ; ~a2 ; ::: ~an ; i.e., A = [~a1 ~a2 ::: ~an ] : This is the advantage of using matrix notation: simpli…cation. It looks as if Ax = b: Theorem 4.1 Let A = [~a1 ~a2 ::: ~an ] be a m £ n matrix whose ith column is ~ai 2 Rm ; i = 1; 2; :::; n; and ~x an unknown vector in Rn : The following statements are equivalent: 1. For each vector ~b 2 Rm ; the vector equation A~x = ~b has a solution. 2. Each vector ~b 2 Rm is a linear combination of ~a1 ; ~a2 ; :::;~an ; the columns of A: 3

3. Column vectors of A span the entire space Rm : 4. The matrix A has a pivot position in each row. Proof. (1) =) (2) : Suppose the …rst statement is true. Then (1) implies that for each ~b; there exists a solution xi = ¸i ; i = 1; :::; n such that ¸1~a1 + ¸2~a2 + ::: + ¸n~an = ~b: Therefore, each ~b is a linear combination of ~ai : (2) =) (3) : Since each vector ~b in Rm is a linear combination of ~a1 ;~a2 ; :::; ~an , by de…nition, each ~b in Rm is in the subset spanned by ~a1 ;~a2 ; :::;~an ; i.e., Rm ½ Span f~a1 ; ~a2 ; :::; ~an g : On the other hand, it is obvious that Span f~a1 ; ~a2 ; :::;~an g ½ Rm : Therefore, Span f~a1 ; ~a2 ; :::;~an g = Rm (3) =) (4) : Suppose (3) is true but (4) is NOT true. then the reduced Echelon form of A has at least one zero row, i.e., A is row equivalent to E by a series of elementary row reductions: 2 3 ¤ ¤ ::: ¤ 6 ::: ::: ::: ::: 7 7 A!E=6 4 ¤ ¤ ::: ¤ 5 0 0 ::: 0 We now consider the augmented matrix 2 ¤ ¤ · ¸ 6 ::: ::: .. E .~em = 6 4 ¤ ¤ 0 0

::: ::: ::: :::

3 ¤ 0 ::: ::: 7 7; ¤ 0 5 0 1

(1)

and reverse completely all the row reductions we did from A to E: (We will show an example later.) This leads to · ¸ · ¸ .. . E .~em ! A..~b ;

where ~b is a vector in Rm : Since the system corresponding to (1) is inconsistent, A~x = ~b for this particular vector ~b is inconsistent. In other words, ~b cannot be possibly a linear combination of ~ai : This contradicts to (3). Therefore, the assumption we made in the beginning "Suppose (3) is true but (4) is NOT true." is not a valid assumption. To demonstrate this "reverse" process, we look at a simple example · ¸ · ¸ 1 2 1 2 ~ A= R +R !R =E ¡1 ¡2 ¡¡1¡¡¡¡2¡¡¡¡!2 0 0 4

Then (with m = 2) · ¸ · ¸ · ¸ .. 1 2 0 ~ 1 2 0 E .~em = R ¡R !R : 0 0 1 ¡¡2¡¡¡¡1¡¡¡¡!2 ¡1 ¡2 1 Thus A~x = ~b is inconsistent, where ~b =

·

0 1

¸

:

(4) =) (1) : Since A has a pivot in each row, we may assume, without loss of generality, that A has the following reduced Echelon form 2 3 1 ¤ ::: ¤ ::: ¤ 6 0 1 ::: ¤ ::: ::: 7 7 A!E=6 4 ::: ::: ::: ::: ::: ¤ 5 : 0 0 ::: 1 ::: ¤ · ¸ . ~ Then, for each b; A..~b must have the reduced Echelon form ·

2

6 1 ¤ ¸ 6 ..~ 6 0 1 A.b ! 6 6 ::: ::: 4 0 0

::: ¤ ::: ¤

::: :::

::: ::: ::: ::: 1 :::

.. . .. . .. . .. .

3

¤ 7 7 ¤ 7 7: ¤ 7 5 ¤

The augmented matrix on the right is consistent since the last column is not pivot. Therefore, A~x = ~b is consistent. Example 4.2 Let 2 3 1 3 4 A = 4 ¡4 2 ¡6 5 : ¡3 ¡2 ¡7 Example 1

1. Determine if A~x = ~b has a solution for ALL ~b 2 R3 :

2. Describe Spanf~a1 ; ~a2 ; ~a3 g ; where A = [~a1 ; ~a2 ; ~a3 ] : Solution. (1) By row operations, we have 2 3 2 3 2 3 1 3 4 1 3 4 1 3 4 4 ¡4 2 ¡6 5 R2 + 4R1 ! R2 4 0 14 10 5 R3 ¡ R2 =2 ! R3 4 0 14 10 5 : R3 + 3R1 ! R3 ¡¡¡¡¡¡¡¡¡¡¡¡! 0 0 0 ¡3 ¡2 ¡7 ¡¡¡ ¡¡¡¡¡¡¡¡¡! 0 7 5

Since the last row contains no pivot, by Theorem, (4) () (1) , A~x = ~b doesn’t has a solution for all ~b: 5

(2) We need to …nd for what ~b; ~b 2 Span f~a1 ; ~a2 ; ~a3 g? In other words, we need to describe all ~b such that A~x = ~b has a solution. In other words, we need to …nd conditions on ~b under which A~x = ~b is consistent. To · this ¸ end, we perform exactly the same row operations as above for the augmented matrix ..~ A.b : 2 3 2 · ¸ 1 3 4 b1 1 3 . R + 4R1 ! R2 4 0 14 A..~b = 4 ¡4 2 ¡6 b2 5 2 R3 + 3R1 ! R3 ¡3 ¡2 ¡7 b3 ¡¡¡¡¡¡¡¡¡¡¡¡! 0 7 2 1 3 4 b1 4 R3 ¡ R2 =2 ! R3 0 14 10 b2 + 4b1 ¡¡¡¡¡¡¡¡¡¡¡¡! 0 0 0 b + 3b ¡ b =2 ¡ 2b 3 1 2 1 2 3 1 3 4 b1 4 5: = 0 14 10 4b1 + b2 0 0 0 b1 ¡ b2 =2 + b3

3 4 b1 10 b2 + 4b1 5 5 b3 + 3b1 3 5

We know that A~x = ~b has a solution i¤ the last column is not pivot, i.e., b1 ¡ b2 =2 + b3 = 0; or b3 = ¡b1 + b2 =2: We conclude that ~b 2Spanf~a1 ;~a2 ; ~a3 g i¤ b3 = ¡b1 + b2 =2; and 2 3 2 3 2 3 b1 b1 b1 + 0b2 ~b = 4 b2 5 = 4 5 = 4 0b1 + b2 5 b2 b3 ¡b1 + b2 =2 ¡b1 + b2 =2 2 3 2 3 2 3 2 3 b1 0b2 1 0 = 4 0b1 5 + 4 b2 5 = b1 4 0 5 + b2 4 1 5 ¡b1 b2 =2 ¡1 1=2

for any b1 and b2 : Later, we call it "parametric representation". Therefore, 82 3 2 39 0 < 1 = Span f~a1 ; ~a2 ; ~a3 g = Span 4 0 5 ; 4 1 5 : : ; ¡1 1=2 ² Representing Solution Sets in Parametric Vector Forms 6

Consider linear system A~x = ~b: When ~b 6= 0; we say it a non-homogeneous system. We say that (for the same A) A~x = ~0 is the corresponding homogeneous system. Any homogeneous system is always consistent. (why?) ~x = ~0 is called the trivial solution. If a homogeneous system admits one non-trivial solution, then it has at least two distinct solutions (another one is the trivial solution), and consequently it must have in…nite many non-trivial solutions. For homogeneous systems, solution sets may be expressed as subspaces spanned by several vectors. Example 4.3 Describe the solution set for the homogeneous system A~x = ~0 where 2 3 1 3 4 A = 4 ¡4 2 ¡6 5 : ¡3 ¡2 ¡7 Solution. We perform row reductions to its Echelon form: 2 3 2 1 3 4 0 1 4 ¡4 2 ¡6 0 5 R2 + 4R1 ! R2 4 0 R3 + 3R1 ! R3 ¡3 ¡2 ¡7 0 ¡¡¡ 0 ¡¡¡¡¡¡¡ 2 3¡¡!2 1 3 4 0 1 4 5 4 R3 ¡ R2 =2 ! R3 0 14 10 0 ! 0 ¡¡¡¡¡¡¡¡¡¡¡¡! 0 0 0 0 0 The system corresponding to the last matrix is

augmented matrix to obtain the reduced 3 3 4 0 14 10 0 5 7 5 0 3 2 3 3 4 0 1 0 13=7 0 1 5=7 0 5 ! 4 0 1 5=7 0 5 : 0 0 0 0 0 0 0

x1 + (13=7) x3 = 0 x2 + (5=7) x3 = 0; or x1 = ¡ (13=7) x3 ; x2 = ¡ (5=7) x3 : Therefore, 2 3 2 3 2 3 x1 ¡ (13=7) x3 ¡13=7 ~x = 4 x2 5 = 4 ¡ (5=7) x3 5 = x3 4 ¡5=7 5 x3 x3 7=7 2 3 2 3 ¡13 ¡13 x3 4 5 4 = ¡5 = t ¡5 5 = tw: ~ 7 7 7

This is called parametric vector representation of solution sets. The solution set is then Span fwg ~ : We shall provide more examples later. ² Linearity Principle for Linear System: 7

Given coe¢cient matrix A and a vector ~b, consider linear system and its homogeneous system A~x = ~0 A~x = ~b:

(2) (3)

1. If ~xh and ~yh both are solutions of homogeneous system (2), then any linear combination of ~xh and ~yh is also a solution of the same system (2). 2. If ~xh is a solution of homogeneous system (2) and if ~ynon is a solution of non-homogeneous system (3), then ~x = ~xh + ~ynon is a solution of non-homogeneous system (3). 3. If ~xnon and ~ynon both are solutions of non-homogeneous system (3), then ~x = ~xnon ¡~ynon is a solution of homogeneous system (2). Proof: (1) A (¸~xh + ±~yh ) = ¸A~xh + ±A~yh = ¸~0 + ±~0 = ~0 (2) A (~xh + ~ynon ) = A~xh + A~ynon = ~0 + ~b = ~b (3)A (~xnon ¡ ~ynon ) = A~xnon ¡ A~ynon = ~b ¡ ~b = ~0: The linearity principle leads to Theorem 4.2 Let ~xp be one particular solution of non-homogeneous system (3). Then any solution of (3) has the form ~x = ~xp + ~xh ; where ~xh is a solution of homogeneous system (2). ² Solving General Non-Homogeneous Systems To solve a system A~x = ~b; we follow the following two steps. Step 1: solve the corresponding homogeneous system A~x = ~0; and represent the solution sets of the homogeneous system using parametric vector forms. Step 2: Find a particular solution ~xp for A~x = ~b: The solution set consists of sum of ~xp and ~xh where ~xh is any solution from step 1. Remark Finding a particular solution ~xp of non-homogeneous system (3) could be very challenging. Often it involves sophisticated numerical approximations and large scale computations. We shall only discuss some simple cases. General discussion is beyond the scope of this course. We shall concentrate on Step 1: solving homogeneous systems. Example 4.4 Suppose that A has the following reduced Echelon forms. Find the parametric vector form of solution set of A~x = ~0 . 2 3 2 3 2 3 1 0 0 2 1 0 3 2 1 2 0 ¡3 2 (1) A~ 4 0 1 0 ¡1 5 : (2) A~ 4 0 1 4 ¡1 5 : (3) A~ 4 0 0 1 5 ¡1 5 0 0 1 3 0 0 0 0 0 0 0 0 0 8

Solution: (1) The corresponding equivalent system is x1 + 2x4 = 0 x2 ¡ x4 = 0 x3 + 3x4 = 0: The solution is x1 = ¡2x4 x2 = x4 x3 = ¡3x4: The parametric form

2

3 2 x1 ¡2x4 6 x2 7 6 x4 7 6 ~x = 6 4 x3 5 = 4 ¡3x4 x4: x4

3

2

3 ¡2 7 6 1 7 7=6 7 5 4 ¡3 5 t; 1

where x4 = t is the parameter, or FREE variable. Solution: (2) The linear system for 2 3 1 0 3 2 A~ 4 0 1 4 ¡1 5 0 0 0 0 x1 + 3x3 + 2x4 = 0 x2 + 4x3 ¡ x4 = 0: Hence x1 = ¡3x3 ¡ 2x4 x2 = ¡4x3 + x4 : Solutions are 2

3 2 3 2 3 2 x1 ¡3x3 ¡ 2x4 ¡3x3 ¡2x4 6 x2 7 6 ¡4x3 + x4 7 6 ¡4x3 7 6 x4 7 6 7=6 7 6 ~x = 6 4 x3 5 = 4 5 4 x3 5 + 4 0 x3 + 0 x4: 0 + x4 0 x4 2 3 2 3 2 3 2 3 ¡3 ¡2 ¡3 ¡2 6 ¡4 7 6 1 7 6 ¡4 7 6 1 7 7 6 7 6 7 6 7 = x3 6 4 1 5 + x4 4 0 5 = t 4 1 5 + s 4 0 5 ; 0 1 0 1 9

3 7 7 5

where x3 = t; x4 = s are parameters, or FREE Solution: (3) The system for 2 1 2 0 A~ 4 0 0 1 0 0 0 is

variables. 3 ¡3 2 5 ¡1 5 0 0

x1 + 2x2 ¡ 3x4 + 2x5 = 0 x3 + 5x4 ¡ x5 = 0; or x1 = ¡2x2 + 3x4 ¡ 2x5 x3 = ¡5x4 + x5 : Solution set are

2

3 2 3 x1 ¡2x2 + 3x4 ¡ 2x5 6 x2 7 6 7 x2 6 7 6 7 7=6 7 ~x = 6 x ¡5x + x 3 4 5 6 7 6 7 4 x4 5 4 5 x4 x5 x5 2 3 2 3 2 3 ¡2 3 ¡2 6 1 7 6 0 7 6 0 7 6 7 6 7 6 7 7 + v 6 ¡5 7 + w 6 1 7 ; 0 = u6 6 7 6 7 6 7 4 0 5 4 1 5 4 0 5 0 0 1

where x2 = u; x4 = v; x5 = w are parameters, or free variables. From the above examples, we see that each unknown corresponding to non-pivot column represents a parameter or free variable. Therefore, we conclude this section with the following statements: Conclusion Consider homogeneous system A~x = ~0; where A be a m £ n matrix, i.e., it has m equations and n unknowns. Then, 1. the number of free variables (or parameters) = number of non-pivot columns = number of total columns n - the number of pivots r = r (A); 2. the homogeneous system has a non-trivial solution () it has in…nite many solutions () there is at least one free variable () number of total columns n > r; 3. in particular, the homogeneous system has non-trivial solutions if n > m (i.e., number of columns > number of rows, or equivalently, number of unknowns > number of equations.) 10

Proof. All these statements have been established before. For reader’s convenience, here we provide a brief explanation again. 1. As we discussed before, in solving A~x = ~0; we …rst …nd the reduced Echelon form of matrix A: Then solve pivot variables in terms of non-pivot variables. The rank r (A) = number of pivot positions, and obviously, number of total columns n = number of pivot + number of non-pivot columns. 2. The second statement is due to the fact any consistent linear system either has one solution, or in…nite many solutions. Therefore, it has a non-trivial solutions i¤ it has in…nite many solution. This case happens i¤ there is at least one free variable. By the …rst statement, the number of free variables = n ¡ r. Consequently, n ¡ r ¸ 1: 3. Since the number of pivots r cannot exceed the number of rows m (i.e., m ¸ r), it follows from (1) that if n > m; then n > r and the number of free variables = n ¡ r > 0. In other words, there is at least one free variable, and thus there is at least one non-trivial solution. ² Homework 4 1. Which of the following products are de…ned? Compute the products if de…ned. 2 32 3 3 2 0 ¡3 2 (a) 4 0 0 1 5 5 4 ¡1 5 0 1 0 3 0 2 32 3 3 2 3 2 4 5 4 (b) 2 ¡2 1 1 5 4 1 1 ¡1 2 3 · ¸ 3 2 2 (c) 4 ¡2 ¡2 5 ¡1 4 1 2 3 2 · ¸ 7 1 2 0 ¡3 6 6 ¡1 7 (d) 4 0 1 0 3 0 5 2

2. Find general solution in parametric vector form for linear system A~x = ~b; where 2 3 2 3 1 2 0 ¡3 1 A = 4 0 2 1 5 5 ; ~b = 4 ¡1 5 0 4 0 3 2 3. Let

2

3 1 ¡3 ¡4 6 5: A = 4 ¡3 2 5 ¡1 ¡8 11

(a) Determine whether A~x = ~b has a solution for all ~b. (b) Describe Span f~a1 ; ~a2 ; ~a3 g, where ~a1 ;~a2 ;~a3 are the columns of A; i.e., A = [~a1 ;~a2 ;~a3 ] : 4. Find general solution in parametric vector form for linear system A~x = ~0 if A is row equivalent to 2 3 1 0 3 0 (a) A » 4 0 1 2 0 5 0 0 0 1 2 3 1 2 0 0 2 (b) A » 4 0 0 1 0 ¡3 5 0 0 0 1 0 5. For each of the following statements, determine whether it is true or false. If your answer is true, state your rationale. If false, provide an counter-example (the example contradicting the statement).

(a) A vector ~b is a linear combination of the columns of A if A~x = ~b is consistent. h i (b) The equation A~x = ~b is consistent if and only if the augmented matrix A ~b has a pivot in every row. (c) If the columns of an m £ n matrix A span Rm ; then the equation A~x = ~b is consistent for all ~b in Rm : (d) Any linear combination of vectors can always written in the form of A~x for a suitable matrix A and vector ~x . (e) If A is an m £ n matrix and if A~x = ~b is inconsistent for some ~b; then A cannot have a pivot in every row. (f) If ~xnon and ~ynon both are solutions of non-homogeneous system A~x = ~b, then ~x = ~xnon + ~ynon is a solution of A~x = 2~b:

12