Numerical Analysis FMN011 Carmen Arévalo Lund University carmen@maths.lth.se Lecture 12
Periodic data A function g has period P if g(x + P ) = g(x) Model: Trigonometric polynomial of order M T M (x) = a 0 + M j=1 ( a j cos( 2π P jx) + b j sin( 2π ) P jx) 1
Example The Sturup weather station showed the following maximum temperature on the 6th of each month in 2007-2013: month 1 2 3 4 5 6 7 8 9 10 11 12 T(C) 2007 8 1 7 8 19 23 16 24 16 15 8 9 T(C) 2008 2 8 8 8 20 23 24 20 21 13 10 5 T(C) 2009 1 4 3 12 11 13 24 24 18 14 8 6 T(C) 2010-2 -1 1 9 8 21 19 24 17 13 8 1 T(C) 2011 6 8 13 21 24 26 26 24 23 22 13 9 T(C) 2012 9 12 16 18 25 23 27 29 24 17 10 7 T(C) 2013 9 5 12 17 24 27 29 28 22 17 11 9 Fit the data to an appropriate model. 2
Solution: P = 12, T = a 0 + a 1 cos mπ/6 + b 1 sin mπ/6, m = 1, 2,..., 84 1 cos π 6 sin π 6 1 cos 2π 6 sin 2π 6 1 cos 3π 6 sin 3π 6... 1 cos 84π 6 sin 84π 6 a 0 a 1 b 1 = Solve A T Ax = A T b to get a 0 = 14.4524, a 1 = 8.0446, b 1 = 5.9254 8 1 7. 9 T (m) = 14.4524 8.0446 cos mπ/6 5.9254 sin mπ/6 T (88) = 13.3431 3
Plot of 3-term trigonometric polynomial T (m) = 11.917 7.2863 cos mπ/6 6.7863 sin mπ/6 +0.16667 cos mπ/3 + 0.43301 sin mπ/3 30 Highest Temperatures in Malmö on the 6th of each month (2007 2013) Temperature 25 20 15 10 5 0 5 0 10 20 30 40 50 60 70 80 90 month 4
Data linearization: the exponential model The parameters of y = c 1 e c 2t are not linear. To apply linear least squares we linearize the model: ln y = ln c 1 + c 2 t Let Y = ln y and k = ln c 1, to get the straight line Y = k + c 2 t To obtain the parameters k and c 2, one needs to modify the data (t, y) to (t, Y ) = (t, ln y). After computing k, c 1 must be calculated as e k. 5
Data linearization: the power law The parameters of y = c 1 t c 2 are nonlinear, but we can linearize the model: ln y = ln c 1 + c 2 ln t Let Y = ln y, k = ln c 1 and T = ln t, to get the straight line Y = k + c 2 T The original data must be modified to (T, Y ) = (ln t, ln y) and to obtain c 1 we compute e k. 6
Orthogonal matrices vs Vandermonde We know that solving the normal equations is ill-conditioned, because A T A inherits and even worsens the large condition number of the Vandermonde matrix A. An orthogonal matrix is a matrix with real entries such that Q 1 = Q T. κ 2 (Q) = Q 2 Q 1 2 = ρ(q T Q) ρ(qq T ) = ρ(i) ρ(i) = 1 As orthogonal matrices have 2-norm condition number equal to 1, we will develop a method to solve the least squares problem using orthogonal matrices. 7
Gram-Schmidt Orthogonalization Compute an orthogonal basis for the space spanned by k given linearly independent vectors, {v 1, v 2,..., v k }. Define the unit vector in the direction of v 1, q 1 = v 1 v 1 2 The projection of v 2 onto q 1 is (q T 1 v 2 )q 1, so vector y 2 = v 2 (q T 1 v 2 )q 1 is perpendicular to v 1 and q 2 = y 2 y 2 2 is a unit vector that is perpendicular to v 1 and such that {q 1, q 2 } span the same subspace as {v 1, v 2 }. 8
Gram-Schmidt algorithm Define 1. y 1 = v 1, q 1 = v 1 v 1 2 2. y 2 = v 2 q 1 (q T 1 v 2 ), q 2 = y 2 y 2 2. k. y k = v k q 1 (q T 1 v k ) q k 1 (q T k 1v k ), q k = y k y k 2 Note that q j q i 9
Example: Gram-Schmidt on the columns of a matrix A = 4 4 2 7 4 5 y 1 2 = ( 4) 2 + ( 2) 2 + 4 2 = 6 q 1 = y 2 = 4 7 5 4 4 2 7 4 5 + 3 4/6 2/6 4/6 and = 6 6 3 4/6 6/9 2/6 6/9 4/6 3/9 = ( v 1 v 2 ) 4/6 2/6 4/6 q 2 = 6/9 6/9 3/9 span the same plane. 10
Orthogonalization of a matrix Let r ii = y i 2, r ji = q T j v i. We can write v i = r ii q i + r 1i q 1 + + r i 1,i q i 1, so A = ( r 11 r 12 r 1k ) q 1 q k r 22 r 2k.... r kk If v 1,..., v k are linearly independent, all r ii are nonzero. If v i is spanned by v 1,..., v i 1, then r ii = 0 and the Gram-Schmidt method terminates. 11
The QR-factorization Once we have the Gram-Schmidt orthogonalization of an n k matrix A, we can complete the orthonormal basis by adding vectors q k+1,..., q n, A = ( ) q 1 q k q k+1 q n r 11 r 12 r 1k r 22... r 2k. r kk 0. 0. 0 0 Q = (q 1 q n ) is an orthogonal n n matrix and R is an n k upper triangular matrix. 12
Orthogonal matrices and the QR-factorization A square matrix Q is orthogonal if Q T = Q 1. A square matrix Q is orthogonal if its columns are pairwise orthogonal unit vectors (q T i q j = 0 and q i 2 = 1). If Q is orthogonal, then Qx 2 = x 2. In A = QR, Q is an orthogonal matrix and R is upper triangular. 13
Example: QR-factorization by Gram-Schmidt To y 3 = 4 2 4 1 0 0, 4 7 we add vector 1 0 to complete the space R 3. 5 0 + 4 4/6 2/6 + 6 6/9 6/9 = 1/9 2/9, q 3 = 1/3 2/3 6 9 4/6 3/9 2/9 2/3 r 11 = y 1 2 = 6, r 12 = q T 1 v 2 = 3, r 22 = y 2 2 = 9 The QR-factorization of A is 4 4 2 7 = 4 5 2/3 2/3 1/3 1/3 2/3 2/3 2/3 1/3 2/3 6 3 0 9 0 0 14
Least Squares and QR-factorization The least squares solution minimizes b Ax 2 = b QRx 2 = Q T (b QRx) 2 = Q T b Rx 2 Let d = Q T b. We can find x so that r d 11 r 12 r 1k 1. r 22 r 2k d k.... d k+1 r kk. 0 0.. d n 0 0 x 1. x k = 0. 0 d k+1. d n 15
Steps for least squares solving by QR-factorization A = QR Ax = QRx = b Rx = Q T b Given the m n, m > n system Ax = b, 1. Find Q and R such that A = QR 2. Set ˆR = upper n n submatrix of R 3. Set ˆd = upper n entries of d = Q T b 4. Solve ˆRx = ˆd 16
Example Solve the least squares problem 4 4 2 7 4 5 ( x1 x 2 ) = 3 9 0 4 4 2 7 4 5 = 2/3 2/3 1/9 1/3 2/3 2/9 2/3 1/3 2/9 6 3 0 9 0 0 ( 6 3 0 9 ) ( x1 x 2 ) = ( 5 4 ) 17