Eight Examples of Linear and Nonlinear Least Squares CEE 699.4, ME 99.4 Sstem Identification Fall, 21 c Henri P. Gavin, September 2, 21 1 Not polnomial, but linear in parameters. ŷ(t i ; a) = a 1 sin(t i ) + a 2 sin(2t i ) + a sin(t i ) + a 4 sin(4t i ) (1) SE_data =.181 a a_lls +/- da (percent) 1..9619.2184 2.27481 -. -.18274.21721 4.19941.2.2298.21828 9.791 -.1 -.71647.21676.2 2 Example 1 1. 1. -. -1 true -1. data +/- 9% c.i. -2 2 4 6 8 1 12 14 x 1
2 CEE 699.4, ME. Sstem Identification Duke Universit Fall 21 H.P. Gavin 2 Linear Fit in Multi-Dimensions. ŷ(x 1 (t i ), x 2 (t i ); a) = a 1 tanh(x 1 (t i )) + a 2 x 2 (t i ) + a x 1 (t i ) + a 4 x 1(t i ) (2) x 1 (t i ) is like a time-varing displacement x 2 (t i ) is like a time-varing velocit SE_data = 11.424 a a_lls +/- da (percent). 8.21918 2.8887.4 2. 1.7466.2124 12.16247 1..777.187 88.4244 1. 1.784 6.46211 6.1141 1 1 - true -1 data +/- c.i. -1.2.4.6.8 1 1 1 - -1-1 -1 -.. 1 Example 2 t 1 1 - -1-1 -2-1 -1-1 1 2 x 1 x 2
Eight Examples of Linear and Nonlinear Least Squares Power-Law Fit NOT linear in parameters, but transformable? ŷ(x i ; a) = a 1 x a2 i () log(ŷ(x i ; a)) = log(a 1 ) + a 2 log(x i )?? (4) Parameters from Linear Least Squares applied to transformed equations: See Fig -a SE_data =.49 a a_lls +/- da (percent) 1.e-1 1.74e-1 1.447e-1 1.772e+2 2.e+ 1.87e+ 8.8927e-2 4.92e+ Parameters from Nonlinear Least Squares: See Fig -b a a_lm +/- da (percent).1.814411.627 7.671474 2. 2.8897.6671 1.726898 1 8 6 4 2-2 2 4 6 8 1 12 1 8 6 4 2 Example - a (log-transformed linear least squares) true data +/- 9% c.i. Example - b (non-linear least squares) 2 4 6 8 1 x 1 % f i t.m nonlinear least squares with power law 2 function _ = ( x_data,a); _ = a (1) * x_data.ˆ a (2); The log-error function under-values the data with larger measurement error.
4 CEE 699.4, ME. Sstem Identification Duke Universit Fall 21 H.P. Gavin 4 Nonlinear Least Squares with Exponentials Parameters from Nonlinear Least Squares: ŷ(x i ; a) = a 1 exp[ x i /a 2 ] + a exp[ x i /a 4 ] () a a_lm +/- da (percent) 4..96166.71 1.4668 2. 2.611.664 2.94277 1..98722.1196 1.1418 1. 1.12611.816.84297 4 Example 4. 2. true data +/- 9% c.i. 2 1 1 2 1 % f i t 4.m nonlinear least squares with exponentials 2 function _ = 4 ( x_data,a) 4 _ = a (1)*exp(- x_data /a (2)) + a ()* x_data.*exp(- x_data /a (4)); x
Eight Examples of Linear and Nonlinear Least Squares Ratio of complex-valued polnomials NOT linear in parameters, but transformable? Ĥ(ω k ; a) = a 1 (iω k ) + a 2 (iω k ) 2 + a (iω k ) 1 + a 4 (iω k ) + a (iω k ) 2 + a 6 (iω k ) + a 7 (iω k ) 4 (6) e k = H(ω k ) ( 1 + a 4 (iω k ) + a (iω k ) 2 + a 6 (iω k ) + a 7 (iω k ) 4) ( a 1 (iω k ) + a 2 (iω k ) 2 + a (iω k ) )?? (7) Parameters from Linear Least Squares applied to transformed equations: See Fig -a SE_data =.26 a a_lls +/- da (percent) 1.e+1 6.4797e+ 7.6288e+ 1.177e+2-2.e+ -1.27e+ 9.477e-1 7.6669e+1 1.e+ 7.2744e-1.4972e-1 7.69e+1 1.e-1 6.116e-2 1.296e-1 1.68e+2.e-1 2.669e-1 4.8e-2 1.72e+1 1.e-2 9.287e- 8.926e- 9.611e+1 1.e-2 1.2279e-2.847e- 2.121e+1 Parameters from Nonlinear Least Squares: See Fig -b a a_lm +/- da (percent) 1.e+1 1.897e+1 6.79e-1 6.1849e+ -2.e+ -2.17e+ 1.121e-1 -.28e+ 1.e+ 1.12e+ 8.489e-2 7.2e+ 1.e-1 1.41e-1 6.71e- 6.482e+.e-1 2.971e-1 1.884e- 6.268e-1 1.e-2 1.686e-2 1.146e- 7.62e+ 1.e-2 1.474e-2 2.672e-4 1.8128e+ 1 % f i t.m nonlinear least squares with ratio of complex valued polnomials 2 function H_ = ( w_data,a); 4 iw_data = sqrt ( -1) * w_data ; 6 H_ = (a (1)* iw_data + a (2)* iw_data.ˆ2 + a ()* iw_data.ˆ )./... 7 (1. + a (4)* iw_data + a ()* iw_data.ˆ2 + a (6)* iw_data.ˆ + a (7)* iw_data.ˆ4 ); 1 % f i t b.m ratio of complex valued exponentials... 2 % for use with Levenberg Marquard f i t t i n g... lm.m % negative frequenc has the imaginar part, positive frequenc has the real part 4 function H_ = b ( w_data,a); 6 iw_data = sqrt ( -1) * w_data ; 7 8 H_ = (a (1)* iw_data + a (2)* iw_data.ˆ2 + a ()* iw_data.ˆ )./... 9 (1 + a (4)* iw_data + a ()* iw_data.ˆ2 + a (6)* iw_data.ˆ + a (7)* iw_data.ˆ4); 1 11 M = length( w_data ); 12 H_ = [imag( _ (1:( M -1)/2+1)) ; real ( _ ((M -1)/2+2: M)) ];
6 CEE 699.4, ME. Sstem Identification Duke Universit Fall 21 H.P. Gavin 2 Example (a) 1 H(f) 1 2-4 -2 2 4 Example (b) 1 H(f) 1-4 -2 2 4 f 2 parameter values 1 1-2 4 6 8 1 12 14 16 18 1 χ2 /(m-n+1) and λ 1 2 1 1 1 1-1 1-2 1-1 -4 1-1 -6 1-7 1-8 2 4 6 8 1 12 14 16 18 function calls λ
Eight Examples of Linear and Nonlinear Least Squares 7 6 Impulse Reponse Function Parameters from Nonlinear Least Squares: ŷ(t k ; a) = a 1 exp[(ia 2 + a )t k ] + a 4 exp[(ia + a 6 )t k ] (8) a a_lm +/- da (percent) 6.e+.922e+.776e-1.674e+.e+.4e+.966e- 1.96e-1-1.e-1-9.811e-2 8.87e- 9.46e+ 1.2e+1 1.229e+1 4.79e-1.8e+ 9.e+ 9.246e+ 7.16e- 7.97e-2-2.e-1-1.9498e-1 1.e-2.82e+ 1 1 Example 6 true data +/- 9% c.i. : true,, and data - -1-1 -2 1 1 2 1 % f i t 6.m impulse response function 2 function _ = 6 ( t_data,a); 4 i = sqrt ( -1); 6 _ = a (1)*exp(( i*a (2)+a())* t_data ) + a (4)*exp(( i*a ()+a(6))* t_data ); 7 8 _ = real ( _ ); % taking the real part is l i k e adding the complex conjugate t
8 CEE 699.4, ME. Sstem Identification Duke Universit Fall 21 H.P. Gavin 7 Auto Regressive - Moving Average Model NOT linear in the parameters, but transformable? ŷ(t i ; a, b) = a k ŷ(t i k ; a, b) + b k u(t i k ) (9) k=1 k= ŷ(t i ; a, b) = a k (t i k ) + b k u(t i k )?? (1) k=1 k= Parameters from Linear Least Squares (Fig 7a) and Nonlinear Least Squares: (Fig 7b) a a_lls a_nls +/- da (percent) 1.e+ 1.e+ 1.e+.99e+2.99e+4-7.e-1 -.24e-1-7.6628e-1.8e+2.997e+4.e-1 2.29e-1.11e-1 2.29e+2.9881e+4-2.e-1-1.8e-1-2.9112e-1 1.164e+2.9862e+4 b b_lls b_nls +/- db (percent).e-1 4.9687e-1 6.e-1 2.97e+2.9922e+4 1.e+ 1.112e+ 1.264e+ 4.98e+2.987e+4 1.e+ 1.67e+ 1.421e+.677e+2.998e+4 2.e+ 2.187e+ 2.1e+ 8.21e+2.986e+4 1 % f i t 7.m Auto regressive moving average (ARMA) model 2 function _ = 7 ( x_data, Cab ); 4 global u_data na nb 6 Ca_ = Cab (1: na ); 7 Cb_ = Cab ( na +1: na+nb ); 8 9 _ = f i l t e r ( Cb_, Ca_, u_data );
Eight Examples of Linear and Nonlinear Least Squares 9 Example 7(a) 1 1 - -1-1. 1 1. 2 Example 7(b) 1 1 - -1-1. 1 1. 2 t
1 CEE 699.4, ME. Sstem Identification Duke Universit Fall 21 H.P. Gavin 8 Identif Stiffness and Damping parameters directl NOT linear in the parameters. M r(t) + Cṙ(t) + Kr(t) = Mhu(t) (11) M = I C = c 1 + c 2 c 2 c 2 c 2 + c c c c K = k 1 + k 2 k 2 k 2 k 2 + k k k k h = 1 1 1 (12) [ d ṙ dt r ] = [ I M 1 K M 1 C ] [ ṙ r ] [ + h ] u(t) (1) = r (t) + u(t) (14) a_true a_lm +/- da (percent) 2.e+1 1.9617e+1 7.721e-2.78e-.e+1.291e+1 1.1698e+ 2.228e-2.e+1 4.994e+1 6.e-1 1.2677e-2 1.e+ 8.1479e-1 9.42e- 1.198e-2.e-1 2.4827e-1 9.986e-2 4.22e-1 1.e-1.9191e-1 6.9e-2 1.699e-1 1 % f i t 8.m nonlinear least squares for a linear dnamic sstem 2 function _ = 8 ( t_data,a); 4 global u_data ; 6 k1 = a (1); % s t i f f n e s s parameter 1 7 k2 = a (2); % s t i f f n e s s parameter 2 8 k = a (); % s t i f f n e s s parameter 9 c1 = a (4); % damping parameter 1 1 c2 = a (); % damping parameter 2 11 c = a (6); % damping parameter 12 1 Ks = [ k1+k2 -k2 ; % s t i f f n e s s matrix 14 -k2 k2+k -k ; 1 -k k ]; 16 17 Cs = [ c1+c2 -c2 ; % damping matrix 18 -c2 c2+c -c ; 19 -c c ]; 2 21 Ms = ee(); % mass matrix 22 2 A = [ zeros () ee() ; % dnamic matrix 24 -Ms\ Ks -Ms\ Cs ]; 2 26 B = [ ; ; ; -1; -1; -1 ]; % input i s acceleration of base 27 28 %C = [ 1 ] ; % output msmnt is displ of DOF # 29 %C = [ 1 ] ; % output msmnt is veloc of DOF # 2 C = A (6,:); % output msmnt is accel of DOF # 1 2 D = ; 4 % damp(a) % check eigenvalues of dnamics matrix 6 _ = lsim (A,B,C,D, u_data, t_data ) ;
Eight Examples of Linear and Nonlinear Least Squares 11 1 1 true data +/- 9% c.i. - -1-1 1 1 2 t 6 parameter values 4 2 1 2 4 6 7 8 1 χ2 /(m-n+1) and λ 1 2 1 1 1 1-1 1-2 1-1 -4 1-1 -6 1-7 1-8 2 4 6 7 8 function calls 1 χ2 /(m-n+1) λ