CES 735/8 Imre Pólik Outline Motivation Theoretical background Applications Optimization under uncertainty: robust optimization Imre Pólik, PhD McMaster University School of Computational Engineering and Science March 6, 2008
CES 735/8 Imre Pólik Outline Outline Motivation Theoretical background 1 Motivation The role of uncertainty Applications 2 Theoretical background Robust linear optimization Robust least squares optimization 3 Applications Truss topology design Antenna array design
Outline CES 735/8 Imre Pólik Motivation The role of uncertainty Theoretical background Applications Why uncertainty? Design on computer, measure with a ruler, cut with an axe! Uncertain data measurement errors noise unpredictability vibration Implementation errors technological limits imperfect materials
CES 735/8 Imre Pólik Robust linear optimization Outline Motivation Theoretical background Robust linear optimization Robust least squares optimization Applications
Robust linear programming the parameters in optimization problems are often uncertain, e.g., in an LP there can be uncertainty in c, a i, b i minimize c T x subject to a T i x b i, i = 1,..., m, two common approaches to handling uncertainty (in a i, for simplicity) deterministic model: constraints must hold for all a i E i minimize c T x subject to a T i x b i for all a i E i, i = 1,..., m, stochastic model: a i is random variable; constraints must hold with probability η minimize c T x subject to prob(a T i x b i) η, i = 1,..., m Convex optimization problems 4 26
deterministic approach via SOCP choose an ellipsoid as E i : E i = {ā i P i u u 2 1} (ā i R n, P i R n n ) center is ā i, semi-axes determined by singular values/vectors of P i robust LP minimize c T x subject to a T i x b i a i E i, i = 1,..., m is equivalent to the SOCP minimize c T x subject to ā T i x P i T x 2 b i, i = 1,..., m (follows from sup u 2 1(ā i P i u) T x = ā T i x P i T x 2) Convex optimization problems 4 27
stochastic approach via SOCP assume a i is Gaussian with mean ā i, covariance Σ i (a i N (ā i, Σ i )) a T i x is Gaussian r.v. with mean āt i x, variance xt Σ i x; hence ( ) prob(a T b i ā T i i x b i ) = Φ x Σ 1/2 i x 2 where Φ(x) = (1/ 2π) x e t2 /2 dt is CDF of N (0, 1) robust LP minimize c T x subject to prob(a T i x b i) η, i = 1,..., m, with η 1/2, is equivalent to the SOCP minimize c T x subject to ā T i x Φ 1 (η) Σ 1/2 i x 2 b i, i = 1,..., m Convex optimization problems 4 28
CES 735/8 Imre Pólik Robust least squares optimization Outline Motivation Theoretical background Robust linear optimization Robust least squares optimization Applications
minimize Ax b with uncertain A Robust approximation two approaches: PSfrag replacements stochastic: assume A is random, minimize E Ax b worst-case: set A of possible values of A, minimize sup A A Ax b tractable only in special cases (certain norms, distributions, sets A) example: A(u) = A 0 ua 1 x nom minimizes A 0 x b 2 2 12 10 8 x nom x stoch minimizes E A(u)x b 2 2 with u uniform on [ 1, 1] r(u) 6 4 x stoch x wc x wc minimizes sup 1 u 1 A(u)x b 2 2 2 figure shows r(u) = A(u)x b 2 0 2 1 0 1 2 u Approximation and fitting 6 17
stochastic robust LS with A = Ā U, U random, E U = 0, E U T U = P minimize E (Ā U)x b 2 2 explicit expression for objective: E Ax b 2 2 = E Āx b Ux 2 2 = Āx b 2 2 E x T U T Ux = Āx b 2 2 x T P x hence, robust LS problem is equivalent to LS problem minimize Āx b 2 2 P 1/2 x 2 2 for P = δi, get Tikhonov regularized problem minimize Āx b 2 2 δ x 2 2 Approximation and fitting 6 18
worst-case robust LS with A = {Ā u 1A 1 u p A p u 2 1} minimize sup A A Ax b 2 2 = sup u 2 1 P (x)u q(x) 2 2 where P (x) = [ A 1 x A 2 x A p x ], q(x) = Āx b from page 5 14, strong duality holds between the following problems maximize P u q 2 2 subject to u 2 2 1 minimize subject to t λ I P q P T λi 0 q T 0 t 0 hence, robust LS problem is equivalent to SDP minimize subject to t λ I P (x) q(x) P (x) T λi 0 q(x) T 0 t 0 Approximation and fitting 6 19
PSfrag replacements example: histogram of residuals r(u) = (A 0 u 1 A 1 u 2 A 2 )x b 2 with u uniformly distributed on unit disk, for three values of x 0.25 0.2 x rls frequency 0.15 0.1 0.05 x tik x ls 0 0 1 2 3 4 5 r(u) x ls minimizes A 0 x b 2 x tik minimizes A 0 x b 2 2 x 2 2 (Tikhonov solution) x wc minimizes sup u 2 1 A 0 x b 2 2 x 2 2 Approximation and fitting 6 20
CES 735/8 Imre Pólik Truss topology design Outline Motivation Theoretical background Applications Truss topology design Antenna array design
? * } ('!4 1 1 S $ 1 1 /P $ / / 7P / ² 1 1 1 Ÿ $ ² 1 QP R 1 1 1 P / P 1 P $O S S P --! " $# *) * -" *),$# $ "! ' $ ( (!4 $ $# $H ' $(! $# $$!( " % $$$! $# ' 5 (! ( $(! $# $,$$! š $ "! "! 5 ( $ $
!( )!" $ 5 "0$($! "! * " / š 0 $ '"! # ) $ 0 $ '"!)* ) % ($ $# $# ) % ( $ - $! # (!( " $* "! /)& $#!( $" " ) ' "!4 - $#! A B h: db > h y $# - $!( $" " $#!( " )! ' "- )! ' $# ($ $# * & $ š - $!( $" " $# )!! "! $ ' $ )! # $ "- $ $ ' $(! # )! 5 "! ) ( ',., 2 D " $ " $* 5 œ $# " - ( 5 -,$# ( -" )* $ * $# )!$ $# " $* $!( /!( $" " )*! $% $'! "! ' / $' (0 &!( $" " $# )!
± 5 % ($,$# - $%!( $" " %,$#( " $ " $* 5 % $# " $* $!(!( $'& " ) ) $ $$! ("! $ " "! )* -" $# $$!"! )* -" )* $ 4 (' 5 #! $( 5 4 ( -$ 1$# "- 25 ³ / $( ' $(! $$! 5 % ($,$# - $%!( $" " %,$#( " $ " $* 5 % 5 $# " $* $!(!( $'& " $($# $$! ³ ) ) )! "(! "0$($ 0 $ $# "" " š & ' $(! 0 $!,! )"' 5 $* ',-",.! $% $# $ "!)*"'& " $
% % H A G $ W N ' F % ) 5 % ($,$# - $%!( $" " %,$#( " $ " $* 5 % ) ) 5 $# " $* $!(!( $'& " $($# $$! ³ ) ) ). - 5 ' $ $ 5 -$! # ( - ' '! -$!,# 5 ( "" " & ( $#1 " Y a ( d 4 Y 8>`*f]d [a!za@b5y %l^bf]g^5fb5y a!di'z ]b5ea- ghy[z e*\a-a!\ Y[d ;e*e d 8>`a a!g`y[z]ighyo\y!^y [b5ef]z]\ ^bf]g^5fb5a-, b5a " 8>`acd]Y[gha#> \ e;eoib^5fy \]idy[gha- a!z^5dck " 2 df=;da.^ H I > e;0$iza- Y!^5ighY*' Y[\4 iddi'a7\]id 9 Y[gha- a!z^5dck= a-e K " 2 ghe*a!g^5iez Q FG mhmhm G L FG mhmhm G e; @ Y!^bigha!d,\a.- Z]iZ=1^5`aA(a!Za@bM Y!^bï 54, N Q S R W QX X Q S Q Q G S SPQ R N % QX a!iz=1^5`a1\a!di'zmo Y biy a!d Z 6 R > Y I Q S R = K 6 Q S R T X G D Q S Q R m " 2 ^ 4 Y d 4 i g 4 Y - Y i ^ R Q S R = Q S "2 a:5*^a@bzy1e[y[\ id b5a6]b5a!da!z^a!\)<7ymoa!g^e[b ^5`a1gheiY[Z]gha e; ghez]d ^bf]g^5iez Sb ^,6 id 9;:5<>=@?A B2C G `]i*a ^5`a Y!^a@biY b5a!defbghacid QX <;ighy^5i* Y %l^bf]g^5fb5y a!di'zm]b5ea- jlk idc i'oa!z 2Y [b5ef]z]\ ^bf]g^5fb5ap Y[Z7f=4 a@b ef]z]\ ez ^e ^Y1 Y!^a@biY8b5a!defbgha YeZ= ^5` ef]z]\]d ez&^5`a1b5a!defbgha!d e;ua-a- a!z^5d;y[z]\ da.^! e;e[y[\]iz=7dgha!zy bied Z]\ ghez]d ^bf]g^5iez ^5`1^5`a dv Y*a!d eddi'a">e[bd ^ghy[da# Sb ^ >6!;gheiY[Z]gha 9;:5<>=@?%$ B2C 9;:5<>=@?A R& <('*). Q S Q,- G Q S Q R. Q G FG mhmhm G L G N jlk W Q S Q R QX jlk
N A G $ W Q " 2 ^ Q S R = Q S <;ighy %l^bf]g^5fb5y a!di'zm]b5ea- id 9;:5<>=@?%$ B2C 9;:5<>=@?A R& <('*). Q S Q, - G Q S Q R. Q G FG mhmhm G L G N jlk W Q S Q R QX jlk `a@b5a " 9;:5<>=@? A Q S R B2C = D G 6 Q S R " Q S R QX X Q S Q Q "3H R \ "S Q R % FG mhmhm G L*Y b5a1\a!di'zmo Y biy a!d " I \ id>^5`a1da.^ e;ee[y[\]iz= dgha!zy bied "&.b5e Ze ez >acy[ddf4 a ^5`Y!^ 2 4 H dy!^5id -;a!d^5`a %=Y!^a@b ghez]\]i ^5ieZ7! f4 ^5ï 9 e[y[\y[z]\ e;f]d ^&%l^bf]g^5fb5y a!di'z " Z ^5`a f]dfyda.^^5iz=d e;7^5`a % ]b5ea- 6 G mhmhm G 6 idy dv Y*O- Z]i ^a da.^e;a(ce[y[\]d_e; iz^a@b5a!d ^, (C1f4 ^5ï 9 e[y[\&\a!di'z4, 4 2 Z Y ^a@bzy!^5i'oaid ^e f]da Y (C Y[ddi'Oa-,&a-*i';dei\Y da.^ e;e[y[\]dc 6 F R#! \ " 8>`a 1f4 ^5ï 9 e[y[\7y 4]b5e[Y[g`7Y[g`]ia6Oa!d,^5`a a!d ^ eddi'a b5a!df4 ^5dlY[d ;Y b>y[d ^5`a dgha!zy bie#e[y[\]d Y b5a ghez]gha@bza!\ ;f*^ ghy[zjia-\_f]z]d ^Y acghez]d ^bf]g^5iez `]ig`&ghy[zj a1ghbf]d`a!\ < (hdv Y* e*g.ghy[diezy e[y[\4, d ^ i ^ e W - Q 4 -. Q. Q. Q,4 8>`aA[b5ef]Z]\ ^bf]g^5fb5a \ela!d Ze Y[\4 bi'i\ e*\= ^5ieZ]dC Q Q Y[Z]\
/E5Ya ddf4 a!>acy b5a1\a!di'z]iz=1yy[zy ^bf]dd _Y ghy[z^5i*a6oa@b6kl^5`a,ze*\y ^bf]g^5fb5a Y[Z]\^5`a ez4' e[y[\ e;uiz^a@b5a!d ^O6 b5acy[d d`e Z&eZ_^5`a#;ig^5fb5a [b5ef]z]\d ^bf]g^5fb5acy[z]\_^5`a#e[y[\&e;uiz^a@b5a!d ^ 8>`a_e^5i* Y>diZ=a:9 e[y[\ \a!di'z ia-\]d Y Z]igha^bf]ddcY[d ;e*e dc ^5i* Y ghy[z^5i*a6oa@b diz=a:9 e[y[\_\a!di'z 4 ^5`a1gheiY[Z]ghacid ^5i* Y ghy[z^5i*a6oa@b diz=a:9 e[y[\_\a!di'z 4 ^5`a1gheiY[Z]ghacid "^c^5fbz]d ef*^1^5`y!^ `a!z ^5`a (Ce[Y[\ e; iz^a@b5a!d ^,76 id b5a6y[gha!\ i ^5` ^5`a7dV Y*16 - m - - 16 4 ( ]Y[\4' Y[gha!\4,1e*g.ghY[dieZY7e[Y[\&6 l^5`a gheiy[z]gha5f4;d ;b5e ^e 4 " Z e[b\a@b ^eci*]b5e-oa,^5`aod ^Y ;i*i ^ ce;^5`a \a!di'z a.^ f]d b5a6y[gha ^5`a&diZ=a e[y[\ e;,iz^a@b5a!d ^ 6<7^5`aa-*i';dei\ ghez^y[iz]iz= ^5`]idUe[Y[\Y[Z]\Y* e[y[\]d61e; Y Z]i ^5f]\a86 Ze ^ a:5]gha@a!\]iz= e;8^5`a Y Z]i ^5f]\ace; 6 6 6 6 6 mhmhm! 6! F G `a@b5a,6 G mhmhm G 6! Y b5a>e;^5`aze[b - m F 86" Y[Z]\16 G 6 G mhmhm G 6! idy[z&e[b^5`e[ezy ]Y[did,iZM>!
" Y[ddiZ=&;b5e ^5`aOdiZ=a:9 e[y[\ ^e ^5`a b5e;f]d ^ \a!di'z >a e*\]i'; ^5`acb5a!df4 ^,Y[d ;e*e dc ^5i* Y ghy[z^5i*a6oa@b diz=a:9 e[y[\_\a!di'z 4 ( e;f]d ^, ghy[z^5i*a6oa@b eiy[z]gha!d a!di'z %iz=a:9 e[y[\ e;f]d ^ eiy[z]ghä Sb ^,6 < gheiy[z]ghä Sb ^ e[y[\]d 64 86 - m F 86"
CES 735/8 Imre Pólik Antenna array design Outline Motivation Theoretical background Applications Truss topology design Antenna array design
I I ˆ D LP R P 1 P $ R 1-5 P P5 5 ŸP 1 6 F I -[ 6 F I -[ F F I 1. % Œ I,1F -[ F F I 1 % Œ I,1F -[!! Œ! DO / 1L: 6 F I -[ A- N : % Œ 1F -[ F I6F IA- :F IoD> 1 F I P7 žp R 3 F I 6 F I 3. «P 1 2P R P D % - P % 0. P R Ia9 * } D! * $O -ŸP F$#*I / «1 P /.š % «P "AF$#*Ia9.>?>?>?9 F$#*I % 5 QP 1 $.š 0 C J. 1 1 C ˆ) 0 C C F$#*I - $7 -²P F$#*IaR R 4 $,' Q P C F$#*I - $ 0 C Ÿ QP / QP 1P 1QP / /š P P= 5 QP P D P DO R / 5 P P 1 "' D P - 1 D. DO - AO P P QP P $ O! F I % C ˆ) 0 C C F I./ 1 D. DO 4 R
y * k! N - ² P - QP / P «.. š! * P / AO P QP5P QP P 1 QP Q$4! y F I6 ƒ < 3 F Œ [ F IF /I!! 0.25 0.2 0.15 0.1 0.05 0 0.05 P 0.1 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 S / 1 QP J 8-9 ' Œ 6 R
! 3 3 3 3 % 3 *. 5 -LP AO P 1 QP PQš 7 / 1.2 *.Q / P - - R! 1.2 1 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 0 0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 -²P P -$# QP (' ž 4 O QP! C F IQ6 % F I %& / 3 / C )-,DC C F I E J F mf IkF /I!"! C 6 8->?<ZG94GŸ6«8-9.<9.>?>?>?9k<8 «6 3 ) 0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 -²P P. P QP k P P P / $ QP P P $!!!!! "!
e } } N! DO P Q L - 1! * $O QP -žp! 1.2 1 % F I % C ),DC C F I 9 J 0.25 1.2 0.8 0.2 1 0.15 0.8 0.6 0.1 0.05 0.6 0.4 0.4 0 0.05 0.2 0 0.2 0.1 0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 S C F$#*I - F$#*I 0 ² / ² - P LP! F I % C ),DC C F I! ž P 8-9 % F I % C ),DC C F I 9 J 0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 - P. P QP k D# P P P F I 6=<B& 6T8->8 / ; $ž, C ' G 6 <9.>?>?>?9.<8 ' 1 1 QP DO R ),' D P QPT P«O R Ÿ $ 1 P 0 R QP.!,DC 6=FZ< 1 ªHC I, C 1ªHC Z % 8->88-<9m8->88-<$ C )
I } $.Q R.š žp! 1.2 1 0.8 0.6 0.4 0.2 0 0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1S QP /š 1 šp 1 P P. P"! -²P 8 6 4 2 0 2 4 6 8 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 ² -žp Z,' QP 5 Z P P 8->8 -R $# 1 P š. T 1 Ÿ L Z P R - P } H % F I % C ),DC C F I 9 J, 1 1 8 4 4.š,DC FZ< 1 ªHC I,DC (' 7.. QP «7 Q ² žp P 8-> -9.<>88-< šq! Ž až 6 6 FZ< 1 ª m9.>?>?>?9.< 1 ª ªHC 8->88-<. O N ²P $ '/ QP T. 4 4 '. R DO 'N P 1$ Ž až QP O$O š % QP $ 7 1 QP QP QP QP ' 5 Z Q.$ 5 DO š«o D $ $$ R - P 5 ž 7. -. D R
8 3 ' ' ' C ' ' 1 C } P 5 ) CA,DC QP -,DC QP $. P"! 8,DC FZ< 1 ªHC I,DC ª C / QP QP P QP O! 1ª C 6T8-B 1ª C 3 6 C 3 > 7O Qš QP $. P, C '; T L QP P QP O! 6 &/' C ) C FZ< 1 ª C I, C 1 6 &/' C ) CA,DC 1 9 F % I 3 3 6 & C ' ) C 3 C 3, C 3 > -² 1 - - 1š QP $ DO 2 Ÿ % D9 / Q $ Z / DO R 5 š 1 ' š O 8 % R R ' % ' C ) CA,DC 1 % ' C ) C 3 C 3, C3 R R R } &/' C ) C FZ< 1 ªHC I,DC 1 8 1ªHC 6«8-B 1ªa3 C 6 3 & C ' ) CA,DC 1 % & C ' ) C 3 C 3, C3 8! - 1$. P«T P. - 6 6 1 Fm9.>?>?>?9 I < (' $ / DO.š $O / 1 - O. P - O 1 6 FZFZ< 1 ª ai m9.>?>?>?9.fz< 1 ª I I P R. 7ªHC 1 T / % ª 9 ª R ;0 ² P/ ª C QP5 T šo 1 - R )m $ % - A- % 3 DO š $ / Ÿ / <8 [ R Z @ ' P 3 o D $ $.š o
& } - $= P - P = -! % F I % C ), C C F I E J 4 }.Q LS $ Qš & 6 < Z! 1.2 S QP F I %& F I % C )-,DC C F I21 C )-,DC C F I % ª & C ), C 3 C3 F I ª & C ), C3 C3 F I % E J 1 0.8 0.6 _ª 6T8->88-< 0.4 0.2 0 0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 š P TP. 1 Z P R - 0 / O š R $O ŸP $ Qš Ÿ 1 $ 5 T ' P $ $! $# 5P.š 'N Z P 7 -žo R R R
e c 3 & 3 3 3 3 } P R ' P $ Qš 1$ P R.! $# P.š ' ² Z P Ÿ ž - O R R R } F I % F I %& % F I % C ),DC C F I "E J 4 C )-,DC C F I21 ª & C ), C 3 C3 F I C ), C C F I %ª & C ), C3 C3 F I % E J P $ Qš ² žp C F$#*I T 1$ 1$P P š R.Q ' ² PQš % / 7P P 1$ / š R )QP P ' Ÿ T / 4 PQP P, C T O! - o 1$ o o H o H o o o o / š / 4 2O ' QP DO $O - D P = TP $ - $ R ). ' QP QP P $#P '.² P - - / QP= QP R R 'Q R C ü *s e c H
/ 3? P 2 ž 5 LP 1,'! * $O QP -žp!? 0.25 1.2 0.2 1 120 90 1 60 120 90 0.87966 60 0.15 0.8 0.8 0.65974 0.6 0.1 0.6 150 30 150 0.43983 30 0.4 0.05 0.4 0.2 0.21991 0 0.2 180 0 180 0 0.05 0 S C 0.2 0.2 0.4 0.6 1.2 1.4 1.6 F$#*I 0.2 0.4 0.6-1 1.2 1.4 F$#*I 0.1 0 0.8 1.6 0 0.8 1 5 QPT $2,DC ' GŸ6=<9.>?>?>?9.<8 $# / $O / F I % C ),DC C F I F I 3! 3! 210 240 270 ² P $ $O # 300 330 1P -! - $-! LP $ 210 240 270 300 330 R
ˆ I? "! š $ $O 7 NP $ 1 PL R QP. k D! P R O ' QP $# P ' 1 Z! 150 210 120 240 90 180 0 0.4 0.2 1 0.8 0.6 60 300 30 330 150 210 120 240 90 151.6557 101.1038 50.5519 180 0 60 300 30 330? N0. -Q 7 ",ž% 3 F 1 <ki, - ' 6=<8 R 1.š, ( FZ< 1 ª ( I, ( 9 ª ( J % ª 9 ª - 1$ Ÿ '"² 7. QP«O / L. - 6 6 FZ< 1 ª m9.>?>?>?9.< 1 ª ª ( ª D9 / š R 270 270 7 P $ -!N - $-! P.š R
ˆ I ˆ I? ",ž% 3 J 6 6 FZ< 1 ª m9.>?>?>?9.< 1 ª ª ( ª - P. - P 6 6 FZ< 1 ª 9.>?>?>?9.< 1 ª (*) ˆ ª 3( ª 3 - / 1; ; QP QP ' QP 5 QP. 1š D P 5 5. P QP $O.Q / ²! ",ž% 3 E J >.Q ' O P $= LP $ 'Q / / / 4! 120 90 1 60 120 90 0.84322 60 0.8 0.63241 0.6 150 30 150 0.42161 30 0.4 0.2 0.2108 180 0 180 0 210 330 210 330 240 300 240 300 270 7 P $ -!N - $-! P.š R 270 $O 4 L R R