Numerical Analysis FMN011

Transcript

1 Numerical Analysis FMN011 Carmen Arévalo Lund University Lecture 12

2 Periodic data A function g has period P if g(x + P ) = g(x) Model: Trigonometric polynomial of order M T M (x) = a 0 + M j=1 ( a j cos( 2π P jx) + b j sin( 2π ) P jx) 1

3 Example The Sturup weather station showed the following maximum temperature on the 6th of each month in : month T(C) T(C) T(C) T(C) T(C) T(C) T(C) Fit the data to an appropriate model. 2

4 Solution: P = 12, T = a 0 + a 1 cos mπ/6 + b 1 sin mπ/6, m = 1, 2,..., 84 1 cos π 6 sin π 6 1 cos 2π 6 sin 2π 6 1 cos 3π 6 sin 3π cos 84π 6 sin 84π 6 a 0 a 1 b 1 = Solve A T Ax = A T b to get a 0 = , a 1 = , b 1 = T (m) = cos mπ/ sin mπ/6 T (88) =

5 Plot of 3-term trigonometric polynomial T (m) = cos mπ/ sin mπ/ cos mπ/ sin mπ/3 30 Highest Temperatures in Malmö on the 6th of each month ( ) Temperature month 4

6 Data linearization: the exponential model The parameters of y = c 1 e c 2t are not linear. To apply linear least squares we linearize the model: ln y = ln c 1 + c 2 t Let Y = ln y and k = ln c 1, to get the straight line Y = k + c 2 t To obtain the parameters k and c 2, one needs to modify the data (t, y) to (t, Y ) = (t, ln y). After computing k, c 1 must be calculated as e k. 5

7 Data linearization: the power law The parameters of y = c 1 t c 2 are nonlinear, but we can linearize the model: ln y = ln c 1 + c 2 ln t Let Y = ln y, k = ln c 1 and T = ln t, to get the straight line Y = k + c 2 T The original data must be modified to (T, Y ) = (ln t, ln y) and to obtain c 1 we compute e k. 6

8 Orthogonal matrices vs Vandermonde We know that solving the normal equations is ill-conditioned, because A T A inherits and even worsens the large condition number of the Vandermonde matrix A. An orthogonal matrix is a matrix with real entries such that Q 1 = Q T. κ 2 (Q) = Q 2 Q 1 2 = ρ(q T Q) ρ(qq T ) = ρ(i) ρ(i) = 1 As orthogonal matrices have 2-norm condition number equal to 1, we will develop a method to solve the least squares problem using orthogonal matrices. 7

9 Gram-Schmidt Orthogonalization Compute an orthogonal basis for the space spanned by k given linearly independent vectors, {v 1, v 2,..., v k }. Define the unit vector in the direction of v 1, q 1 = v 1 v 1 2 The projection of v 2 onto q 1 is (q T 1 v 2 )q 1, so vector y 2 = v 2 (q T 1 v 2 )q 1 is perpendicular to v 1 and q 2 = y 2 y 2 2 is a unit vector that is perpendicular to v 1 and such that {q 1, q 2 } span the same subspace as {v 1, v 2 }. 8

10 Gram-Schmidt algorithm Define 1. y 1 = v 1, q 1 = v 1 v y 2 = v 2 q 1 (q T 1 v 2 ), q 2 = y 2 y 2 2. k. y k = v k q 1 (q T 1 v k ) q k 1 (q T k 1v k ), q k = y k y k 2 Note that q j q i 9

11 Example: Gram-Schmidt on the columns of a matrix A = y 1 2 = ( 4) 2 + ( 2) = 6 q 1 = y 2 = /6 2/6 4/6 and = /6 6/9 2/6 6/9 4/6 3/9 = ( v 1 v 2 ) 4/6 2/6 4/6 q 2 = 6/9 6/9 3/9 span the same plane. 10

12 Orthogonalization of a matrix Let r ii = y i 2, r ji = q T j v i. We can write v i = r ii q i + r 1i q r i 1,i q i 1, so A = ( r 11 r 12 r 1k ) q 1 q k r 22 r 2k.... r kk If v 1,..., v k are linearly independent, all r ii are nonzero. If v i is spanned by v 1,..., v i 1, then r ii = 0 and the Gram-Schmidt method terminates. 11

13 The QR-factorization Once we have the Gram-Schmidt orthogonalization of an n k matrix A, we can complete the orthonormal basis by adding vectors q k+1,..., q n, A = ( ) q 1 q k q k+1 q n r 11 r 12 r 1k r r 2k. r kk Q = (q 1 q n ) is an orthogonal n n matrix and R is an n k upper triangular matrix. 12

14 Orthogonal matrices and the QR-factorization A square matrix Q is orthogonal if Q T = Q 1. A square matrix Q is orthogonal if its columns are pairwise orthogonal unit vectors (q T i q j = 0 and q i 2 = 1). If Q is orthogonal, then Qx 2 = x 2. In A = QR, Q is an orthogonal matrix and R is upper triangular. 13

15 Example: QR-factorization by Gram-Schmidt To y 3 = , 4 7 we add vector 1 0 to complete the space R /6 2/ /9 6/9 = 1/9 2/9, q 3 = 1/3 2/ /6 3/9 2/9 2/3 r 11 = y 1 2 = 6, r 12 = q T 1 v 2 = 3, r 22 = y 2 2 = 9 The QR-factorization of A is = 4 5 2/3 2/3 1/3 1/3 2/3 2/3 2/3 1/3 2/

16 Least Squares and QR-factorization The least squares solution minimizes b Ax 2 = b QRx 2 = Q T (b QRx) 2 = Q T b Rx 2 Let d = Q T b. We can find x so that r d 11 r 12 r 1k 1. r 22 r 2k d k.... d k+1 r kk d n 0 0 x 1. x k = 0. 0 d k+1. d n 15

17 Steps for least squares solving by QR-factorization A = QR Ax = QRx = b Rx = Q T b Given the m n, m > n system Ax = b, 1. Find Q and R such that A = QR 2. Set ˆR = upper n n submatrix of R 3. Set ˆd = upper n entries of d = Q T b 4. Solve ˆRx = ˆd 16

18 Example Solve the least squares problem ( x1 x 2 ) = = 2/3 2/3 1/9 1/3 2/3 2/9 2/3 1/3 2/ ( ) ( x1 x 2 ) = ( 5 4 ) 17