Lecture 34: Ridge regression and LASSO
|
|
- Διώνη Ζαφειρόπουλος
- 5 χρόνια πριν
- Προβολές:
Transcript
1 Lecture 34: Ridge regression and LASSO Ridge regression Consider linear model X = Z β + ε, β R p and Var(ε) = σ 2 I n. The LSE is obtained from the minimization problem min X Z β R β 2 (1) p A type of shrinkage estimator is obtained though (1) by adding a penalty on β 2, i.e., min β R p( X Z β 2 + λ β 2 ) (2) where λ 0 is a constant controlling the penalization. β ( X Z β 2 + λ β 2 ) = 2Z τ (X Z β) + 2λβ which gives the solution to (2) as β λ = (Z τ Z + λi p ) 1 Z τ X This estimator is better than the LSE when Z τ Z is nearly singular. UW-Madison (Statistics) Stat 709 Lecture / 16
2 This gives a class of estimators called ridge regression estimators; in particular, λ = 0 gives the LSE. Bias and covariance matrix E( β λ ) = (Z τ Z + λi p ) 1 Z τ E(X) = (Z τ Z + λi p ) 1 Z τ Z β The bias of β λ is then b(β) = (Z τ Z + λi p ) 1 Z τ Z β β = λ(z τ Z + λi p ) 1 β The bias is not 0, but converges to 0 as λ 0. Var( β λ ) = (Z τ Z + λi p ) 1 Z τ Var(X)Z (Z τ Z + λi p ) 1 = σ 2 (Z τ Z + λi p ) 1 Z τ Z (Z τ Z + λi p ) 1 = σ 2 (Z τ Z + λi p ) 1 σ 2 λ(z τ Z + λi p ) 2 It can be seen that the variance converges to 0 if λ and to σ 2 (Z τ Z ) 1 if λ 0. Combining the bias and variance, we get E β λ β 2 = b(β) 2 + E β λ E (β λ ) 2 = λ 2 (Z τ Z + λi p ) 1 β 2 + σ 2 tr[z τ Z (Z τ Z + λi p ) 2 ] UW-Madison (Statistics) Stat 709 Lecture / 16
3 Theorem (Comparison between ridge regression and LSE) Let β = β 0 be the LSE. (i) If 0 < λ < 2σ 2 / β 2, then E β λ β 2 < E β β 2. (ii) Assume that the smallest eigenvalue of Z τ Z = O(n). If λ > 2σ 2 / β 2, then E β λ β 2 > E β β 2 for sufficiently large n; if λ = 2σ 2 / β 2, then E β λ β 2 = E β β 2 + O(n 3 ). Proof. Let Then Hence A = σ 2 (Z τ Z ) 1 σ 2 (Z τ Z + λi p ) 1 Z τ Z (Z τ Z + λi p ) 1 λ 2 (Z τ Z + λi p ) 1 ββ τ (Z τ Z + λi p ) 1 (Z τ Z + λi p )A(Z τ Z + λi p ) = σ 2 (Z τ Z + λi p )(Z τ Z ) 1 (Z τ Z + λi p ) σ 2 Z τ Z λ 2 ββ τ = 2λσ 2 I p + λ 2 σ 2 (Z τ Z ) 1 λ 2 ββ τ A = (Z τ Z + λi p ) 1 [2λσ 2 I p λ 2 ββ τ + λ 2 σ 2 (Z τ Z ) 1 ](Z τ Z + λi p ) 1 UW-Madison (Statistics) Stat 709 Lecture / 16
4 Assume λ > 0 and β 0. Then A > λ 2 σ 2 (Z τ Z + λi p ) 1 (Z τ Z ) 1 (Z τ Z + λi p ) 1 if and only if 2σ 2 λ 1 I p ββ τ > 0 equivalent to λ < 2σ 2 / β 2 This can be shown as follows. If 2σ 2 λ 1 I p ββ τ > 0, then 0 < β τ (2σ 2 λ 1 I p ββ τ )β = 2σ 2 λ 1 β 2 β 4, which means λ < 2σ 2 / β 2. On the other hand, if λ < 2σ 2 / β 2, then (2σ 2 λ 1 I p ββ τ )/ β 2 = (2σ 2 λ 1 β 2 1)I p + I p ββ τ / β 2 > 0, because I p ββ τ / β 2 is a projection matrix whose eigenvalues are either 0 or 1. Since Var( β) = σ 2 (Z τ Z ) 1, using the formula for Var( β λ ) we obtain E β β 2 E β λ β 2 = tr(a) Thus, (i) follows, and (ii) and (iii) follow from λ 2 σ 2 (Z τ Z + λi p ) 1 (Z τ Z ) 1 (Z τ Z + λi p ) 1 λ 2 σ 2 (Z τ Z ) 3 The ridge regression is better if the noise to signal ratio is large. UW-Madison (Statistics) Stat 709 Lecture / 16
5 High dimension problems The dimension of β in a linear model is p (Z is n p) In traditional applications: p << n; p is fixed when n. In modern applications, p is large; p = p n increases as n increases. p = O(n k ): polynomial-type divergence rate p = O(e nν ): ultra-high dimension, where ν is a constant < 1. Non-identifiability of β r = r n : rank of Z. The dimension of R(Z ) is r n. If p > n, then β is not identifiable. This means that there are β and β, β β but Z β = Z β so that the data generated under the models with β and β are the same. It is not possible to estimate all components of β consistently; we are not able to estimate something out of the data range. We can estimate consistently some useful functions of β. We can estimate the projection of β onto R(Z ). Estimation of the projection is sufficient for many problems UW-Madison (Statistics) Stat 709 Lecture / 16
6 Projection Singular value decomposition: Z = PDQ τ P: n r matrix with P τ P = I r (identity matrix) Q: p r matrix with Q τ Q = I r D: r r diagonal matrix of full rank Projection of β onto R(Z ): θ = Z τ (ZZ τ ) Z β = QQ τ β R(Z ) Z θ = PDQ τ (QQ τ β) = PDQ τ β = Z β The model Y = Z β + ε is the same as Y = Z θ + ε Ridge regression estimator of θ θ = (Z τ Z + h n I p ) 1 Z τ X h n > 0 We only need to invert an n n matrix, because (Z τ Z + h n I p ) 1 Z τ = Z τ (ZZ τ + h n I n ) 1 θ is always in R(Z ) UW-Madison (Statistics) Stat 709 Lecture / 16
7 Derivation of the bias of ridge regression estimator Let Γ = ( Q Q ), Q τ Q = 0, ΓΓ τ = Γ τ Γ = I p. Then bias( θ) = E( θ) θ = (Z τ Z + h n I p ) 1 Z τ Z θ θ = (hn 1 Z τ Z + I p ) 1 θ = Γ(hn 1 Γ τ Z τ Z Γ + I p ) 1 Γ τ QQ τ θ = ( ) ( (hn Q Q 1 D 2 + I r ) I p r = ( Q(h 1 n D 2 + I r ) 1 Q ) ( Q τ θ 0 = Q(hn 1 D 2 + I r ) 1 Q τ θ (1 + d 1n /h n ) 1 = Q... ) )( Q τ (1 + d rn /h n ) 1 Q τ ) QQ τ θ Q τ θ where d jn > 0 is the jth diagonal element of D 2 (eigenvalue of Z τ Z ). UW-Madison (Statistics) Stat 709 Lecture / 16
8 Thus, For the variance, bias( θ) 2 = θ τ Q(hn 1 D 2 + I r ) 2 Q τ θ max 1 j r (1 + d jn/h n ) 2 θ τ QQ τ θ h 2 nd 2 1n θ 2 Var( θ) = σ 2 (Z τ Z + h n I p ) 1 Z τ Z (Z τ Z + h n I p ) 1 σ 2 hn 1 I p Theorem (Consistency of θ) Assume that (C1) d 1 1n = O(n η ), η 1 and η does not depend on n. (C2) θ = O(n τ ), τ < η and τ does not depend on n. Then (i) As n, E(l τ θ l τ θ) 2 = O(hn 1 ) + O(hnn 2 2(η τ) ) uniformly over p-dimensional deterministic vector l with l = 1. (ii) n 1 E Z θ Z θ 2 = O(r n n 1 ) + O(h 2 nn (1+η 2τ) ). UW-Madison (Statistics) Stat 709 Lecture / 16
9 Remarks (C2) means that θ is sparse; without any condition, the order of θ 2 could be p. θ β so that (C2) holds if β is sparse. For any fixed l θ, l θ is consistent if hn and h n n (η τ) 0. θ is not sparse even if θ is sparse. Typically r n /n 0 so θ is not L 2 -consistent. The reason (ii) is interesting is that n 1 E Z θ Z θ 2 = n 1 E X Z θ 2 σ 2, where X is an independent copy of X and n 1 E X Z θ 2 is the average prediction mean squared error. Problem of the ridge regression estimator When p < n, θ = β has many zero components, the ridge regression estimator does not have any zero components, although it has many small components. UW-Madison (Statistics) Stat 709 Lecture / 16
10 LASSO estimator Consider linear model X = Z β + ε, β R p and Var(ε) = σ 2 I n. The ridge regression estimator of β is obtained from min β R p( X Z β 2 + λ β 2 ) If we change the L 2 penalty β 2 to the L 1 penalty β 1 = p j=1 β j, where β j is the jth component of β, then the LASSO estimator is from min β R p( X Z β 2 + λ β 1 ) Difference between LASSO and ridge regression: LASSO estimator does not have an explicit form. When a component of β is 0, its LASSO estimator may be 0, but its ridge regression estimator is never 0. The minimization for LASSO is still for a convex objective function, but the objective function is not always differentiable. Although LASSO is still defined when p > n, it is usually used in the case where p < n. If p < n, Z can be deterministic or random. UW-Madison (Statistics) Stat 709 Lecture / 16
11 Notation A = the set of indices of non-zero coefficients of β β = (β ( A,β A c), dim(β ) A ) ( = q, dim(β A c) = p q; X = (X A,X A c) C11 C C = 12 = 1 X τ A X A XA τ X ) A c C 21 C n 22 XA τ X c A XA τ = 1 X c A c n X τ X Consistency The LASSO estimator β of β is strongly sign consistent if there exists λ = λ n not depending on Y or X such that ( ) lim P sign( β) = sign(β) = 1 n which implies variable selection consistent (since sign(a) = 0 if a = 0), ( ) lim P A = A = 1 n where A is the index set of nonzero components of β. Strong Irrepresentable Condition (SIC) There exists a vector η whose components are positive such that C 21 C 1 11 sign(β A ) 1 η component-wise, where a = ( a 1, a 2,...) for a = (a 1,a 2,...) and 1 is the vector of ones. UW-Madison (Statistics) Stat 709 Lecture / 16
12 Critical Lemma Under the SIC, ( ) P sign( β) = sign(β) P(A n B n ), where { A n = B n = C 1 11 W A < n β A λ n 2 n C 1 } { C 21 C 1 11 W A W A c λ n 2 n η W A = 1 n X τ A ε W A c = 1 n X τ A cε } 11 sign(β A ) Karush-Kuhn-Tuker (KKT) condition β = ( β 1,..., β p ) is the LASSO estimator if and only if Y Xβ 2 λsign( β j ) βj 0 β = j βj = β j bounded by λ in absolute value βj = 0 UW-Madison (Statistics) Stat 709 Lecture / 16
13 Proof of the Lamma Let û = β β and V n (u) = n i=1 [(ε i X i u) 2 ε i ] 2 + λ n u + β 1 Then û = argminv n (u) It can be verified that the KKT condition is equivalent to C 11 ( nû A ) W A = λ n 2 n sign(β A ), (3) λ n 2 n 1 C 21( nû A ) W A c λ n 2 1, n (4) û A < β A (5) We now show that on A n B n, a solution û satisfying (3) and û A c = 0 must satisfy (4) and (5), and hence β = û + β is a LASSO estimator. In fact, LASSO estimator is unique. First, (3) and A n holds imply (5). Second, (3) and B n holds and the SIC imply (4). Finally, a sufficient condition for sign( β) = sign(β) is û A < β A and û A c = 0. This proves that if A n B n holds, sign( β) = sign(β). UW-Madison (Statistics) Stat 709 Lecture / 16
14 Theorem (strong sign consistency of LASSO) (i) Assume that ε i s are iid with E(εi 2k ) < for an integer k > 0, and there are positive constants c 1 < c 2 1, M 1, M 2, M 3, such that C1: n 1 Z j 2 M 1 for any j = 1,...,p, Z j is the jth column of Z ; C2: The smallest eignvalue of C 11 M 2 ; C3: q = O(n c 1); C4: n (1 c 2)/2 min j A β j M 3 ; C5: p = o(n (c 2 c 1 )k ). Under SIC, if λ is chosen with λ = o(n 1+c 2 c 1 )/2 ) and pn k /λ 2k = o(1), then ( ) P sign( β) = sign(β) 1 O(pn k /λ 2k ) (ii) Assume that ε i s are iid normal and C1-C4 hold, and C5a: p = O(e nc3 ) with a constant c 3, 0 c 3 < c 2 c 1. Under SIC, if λ is chosen with λ n (1+c4)/2, c 4 is a constant, c 3 < c 4 < c 2 c 1, then ( ) P sign( β) = sign(β) 1 O(e nc3 ) UW-Madison (Statistics) Stat 709 Lecture / 16
15 Proof. z j = the jth component of C 1 11 W A, j = 1,...,q ζ j = the jth component of C 21 C 1 11 W A W A c, j = 1,...,p q b j = the jth component of C 1 The condition E(εi 2k By the lemma, P ( sign( β) sign(β) 11 sign(β A ), j = 1,...,q ) < implies that E(z 2k j ) 1 P(A n B n ) ) < and E(ζ 2k j P ( z j n β j λb j /2 n ) j A + P ( ζ j λη j /2 n ) j A c E z j 2k E ζ j j A n k βj 2k + 2k j A (2λη c j ) 2k /n k = qo(n kc 2 ) + (p q)o(n k /λ 2k ) ) < = o(pn k /λ 2k ) + O(pn k /λ 2k ) = O(pn k /λ 2k ) This proves (i). UW-Madison (Statistics) Stat 709 Lecture / 16
16 For (ii), the normality of ε j implies that z j and ζ j are normal. Instead of using Markov inequality, using 1 Φ(t) t 1 e t2 /2 leads to the result (ii). Advantage and disadvantage of using LASSO Variable selection and parameter estimation at the same time It is very good in estimation and prediction, but it is often too conservative in variable selection. Need SIC. Population version of SIC. Σ 21 Σ 1 11 sign(β A ) 1 η, Σ kj are submatrices of Σ = Var(z j ), if z j s are iid, z j is the jth row of Z. Improvements Adaptive LASSO Group LASSO Elastic net (other penalties) LASSO plus thresholding (ridge regression plus threshodling) UW-Madison (Statistics) Stat 709 Lecture / 16
Lecture 21: Properties and robustness of LSE
Lecture 21: Properties and robustness of LSE BLUE: Robustness of LSE against normality We now study properties of l τ β and σ 2 under assumption A2, i.e., without the normality assumption on ε. From Theorem
Διαβάστε περισσότερα6.3 Forecasting ARMA processes
122 CHAPTER 6. ARMA MODELS 6.3 Forecasting ARMA processes The purpose of forecasting is to predict future values of a TS based on the data collected to the present. In this section we will discuss a linear
Διαβάστε περισσότεραST5224: Advanced Statistical Theory II
ST5224: Advanced Statistical Theory II 2014/2015: Semester II Tutorial 7 1. Let X be a sample from a population P and consider testing hypotheses H 0 : P = P 0 versus H 1 : P = P 1, where P j is a known
Διαβάστε περισσότεραPhys460.nb Solution for the t-dependent Schrodinger s equation How did we find the solution? (not required)
Phys460.nb 81 ψ n (t) is still the (same) eigenstate of H But for tdependent H. The answer is NO. 5.5.5. Solution for the tdependent Schrodinger s equation If we assume that at time t 0, the electron starts
Διαβάστε περισσότεραExample Sheet 3 Solutions
Example Sheet 3 Solutions. i Regular Sturm-Liouville. ii Singular Sturm-Liouville mixed boundary conditions. iii Not Sturm-Liouville ODE is not in Sturm-Liouville form. iv Regular Sturm-Liouville note
Διαβάστε περισσότερα2 Composition. Invertible Mappings
Arkansas Tech University MATH 4033: Elementary Modern Algebra Dr. Marcel B. Finan Composition. Invertible Mappings In this section we discuss two procedures for creating new mappings from old ones, namely,
Διαβάστε περισσότεραChapter 6: Systems of Linear Differential. be continuous functions on the interval
Chapter 6: Systems of Linear Differential Equations Let a (t), a 2 (t),..., a nn (t), b (t), b 2 (t),..., b n (t) be continuous functions on the interval I. The system of n first-order differential equations
Διαβάστε περισσότεραStatistical Inference I Locally most powerful tests
Statistical Inference I Locally most powerful tests Shirsendu Mukherjee Department of Statistics, Asutosh College, Kolkata, India. shirsendu st@yahoo.co.in So far we have treated the testing of one-sided
Διαβάστε περισσότεραEE512: Error Control Coding
EE512: Error Control Coding Solution for Assignment on Finite Fields February 16, 2007 1. (a) Addition and Multiplication tables for GF (5) and GF (7) are shown in Tables 1 and 2. + 0 1 2 3 4 0 0 1 2 3
Διαβάστε περισσότεραSCHOOL OF MATHEMATICAL SCIENCES G11LMA Linear Mathematics Examination Solutions
SCHOOL OF MATHEMATICAL SCIENCES GLMA Linear Mathematics 00- Examination Solutions. (a) i. ( + 5i)( i) = (6 + 5) + (5 )i = + i. Real part is, imaginary part is. (b) ii. + 5i i ( + 5i)( + i) = ( i)( + i)
Διαβάστε περισσότεραOther Test Constructions: Likelihood Ratio & Bayes Tests
Other Test Constructions: Likelihood Ratio & Bayes Tests Side-Note: So far we have seen a few approaches for creating tests such as Neyman-Pearson Lemma ( most powerful tests of H 0 : θ = θ 0 vs H 1 :
Διαβάστε περισσότεραC.S. 430 Assignment 6, Sample Solutions
C.S. 430 Assignment 6, Sample Solutions Paul Liu November 15, 2007 Note that these are sample solutions only; in many cases there were many acceptable answers. 1 Reynolds Problem 10.1 1.1 Normal-order
Διαβάστε περισσότεραOrdinal Arithmetic: Addition, Multiplication, Exponentiation and Limit
Ordinal Arithmetic: Addition, Multiplication, Exponentiation and Limit Ting Zhang Stanford May 11, 2001 Stanford, 5/11/2001 1 Outline Ordinal Classification Ordinal Addition Ordinal Multiplication Ordinal
Διαβάστε περισσότεραMatrices and Determinants
Matrices and Determinants SUBJECTIVE PROBLEMS: Q 1. For what value of k do the following system of equations possess a non-trivial (i.e., not all zero) solution over the set of rationals Q? x + ky + 3z
Διαβάστε περισσότεραLecture 34 Bootstrap confidence intervals
Lecture 34 Bootstrap confidence intervals Confidence Intervals θ: an unknown parameter of interest We want to find limits θ and θ such that Gt = P nˆθ θ t If G 1 1 α is known, then P θ θ = P θ θ = 1 α
Διαβάστε περισσότεραEvery set of first-order formulas is equivalent to an independent set
Every set of first-order formulas is equivalent to an independent set May 6, 2008 Abstract A set of first-order formulas, whatever the cardinality of the set of symbols, is equivalent to an independent
Διαβάστε περισσότεραUniform Convergence of Fourier Series Michael Taylor
Uniform Convergence of Fourier Series Michael Taylor Given f L 1 T 1 ), we consider the partial sums of the Fourier series of f: N 1) S N fθ) = ˆfk)e ikθ. k= N A calculation gives the Dirichlet formula
Διαβάστε περισσότεραHOMEWORK 4 = G. In order to plot the stress versus the stretch we define a normalized stretch:
HOMEWORK 4 Problem a For the fast loading case, we want to derive the relationship between P zz and λ z. We know that the nominal stress is expressed as: P zz = ψ λ z where λ z = λ λ z. Therefore, applying
Διαβάστε περισσότερα6.1. Dirac Equation. Hamiltonian. Dirac Eq.
6.1. Dirac Equation Ref: M.Kaku, Quantum Field Theory, Oxford Univ Press (1993) η μν = η μν = diag(1, -1, -1, -1) p 0 = p 0 p = p i = -p i p μ p μ = p 0 p 0 + p i p i = E c 2 - p 2 = (m c) 2 H = c p 2
Διαβάστε περισσότεραNowhere-zero flows Let be a digraph, Abelian group. A Γ-circulation in is a mapping : such that, where, and : tail in X, head in
Nowhere-zero flows Let be a digraph, Abelian group. A Γ-circulation in is a mapping : such that, where, and : tail in X, head in : tail in X, head in A nowhere-zero Γ-flow is a Γ-circulation such that
Διαβάστε περισσότεραHomework 3 Solutions
Homework 3 Solutions Igor Yanovsky (Math 151A TA) Problem 1: Compute the absolute error and relative error in approximations of p by p. (Use calculator!) a) p π, p 22/7; b) p π, p 3.141. Solution: For
Διαβάστε περισσότεραFinite Field Problems: Solutions
Finite Field Problems: Solutions 1. Let f = x 2 +1 Z 11 [x] and let F = Z 11 [x]/(f), a field. Let Solution: F =11 2 = 121, so F = 121 1 = 120. The possible orders are the divisors of 120. Solution: The
Διαβάστε περισσότεραChapter 6: Systems of Linear Differential. be continuous functions on the interval
Chapter 6: Systems of Linear Differential Equations Let a (t), a 2 (t),..., a nn (t), b (t), b 2 (t),..., b n (t) be continuous functions on the interval I. The system of n first-order differential equations
Διαβάστε περισσότεραLecture 12: Pseudo likelihood approach
Lecture 12: Pseudo likelihood approach Pseudo MLE Let X 1,...,X n be a random sample from a pdf in a family indexed by two parameters θ and π with likelihood l(θ,π). The method of pseudo MLE may be viewed
Διαβάστε περισσότεραSolution Series 9. i=1 x i and i=1 x i.
Lecturer: Prof. Dr. Mete SONER Coordinator: Yilin WANG Solution Series 9 Q1. Let α, β >, the p.d.f. of a beta distribution with parameters α and β is { Γ(α+β) Γ(α)Γ(β) f(x α, β) xα 1 (1 x) β 1 for < x
Διαβάστε περισσότερα2. Let H 1 and H 2 be Hilbert spaces and let T : H 1 H 2 be a bounded linear operator. Prove that [T (H 1 )] = N (T ). (6p)
Uppsala Universitet Matematiska Institutionen Andreas Strömbergsson Prov i matematik Funktionalanalys Kurs: F3B, F4Sy, NVP 2005-03-08 Skrivtid: 9 14 Tillåtna hjälpmedel: Manuella skrivdon, Kreyszigs bok
Διαβάστε περισσότεραReminders: linear functions
Reminders: linear functions Let U and V be vector spaces over the same field F. Definition A function f : U V is linear if for every u 1, u 2 U, f (u 1 + u 2 ) = f (u 1 ) + f (u 2 ), and for every u U
Διαβάστε περισσότεραTridiagonal matrices. Gérard MEURANT. October, 2008
Tridiagonal matrices Gérard MEURANT October, 2008 1 Similarity 2 Cholesy factorizations 3 Eigenvalues 4 Inverse Similarity Let α 1 ω 1 β 1 α 2 ω 2 T =......... β 2 α 1 ω 1 β 1 α and β i ω i, i = 1,...,
Διαβάστε περισσότεραCHAPTER 25 SOLVING EQUATIONS BY ITERATIVE METHODS
CHAPTER 5 SOLVING EQUATIONS BY ITERATIVE METHODS EXERCISE 104 Page 8 1. Find the positive root of the equation x + 3x 5 = 0, correct to 3 significant figures, using the method of bisection. Let f(x) =
Διαβάστε περισσότεραw o = R 1 p. (1) R = p =. = 1
Πανεπιστήµιο Κρήτης - Τµήµα Επιστήµης Υπολογιστών ΗΥ-570: Στατιστική Επεξεργασία Σήµατος 205 ιδάσκων : Α. Μουχτάρης Τριτη Σειρά Ασκήσεων Λύσεις Ασκηση 3. 5.2 (a) From the Wiener-Hopf equation we have:
Διαβάστε περισσότερα4.6 Autoregressive Moving Average Model ARMA(1,1)
84 CHAPTER 4. STATIONARY TS MODELS 4.6 Autoregressive Moving Average Model ARMA(,) This section is an introduction to a wide class of models ARMA(p,q) which we will consider in more detail later in this
Διαβάστε περισσότεραSOLUTIONS TO MATH38181 EXTREME VALUES AND FINANCIAL RISK EXAM
SOLUTIONS TO MATH38181 EXTREME VALUES AND FINANCIAL RISK EXAM Solutions to Question 1 a) The cumulative distribution function of T conditional on N n is Pr (T t N n) Pr (max (X 1,..., X N ) t N n) Pr (max
Διαβάστε περισσότεραInverse trigonometric functions & General Solution of Trigonometric Equations. ------------------ ----------------------------- -----------------
Inverse trigonometric functions & General Solution of Trigonometric Equations. 1. Sin ( ) = a) b) c) d) Ans b. Solution : Method 1. Ans a: 17 > 1 a) is rejected. w.k.t Sin ( sin ) = d is rejected. If sin
Διαβάστε περισσότεραMath221: HW# 1 solutions
Math: HW# solutions Andy Royston October, 5 7.5.7, 3 rd Ed. We have a n = b n = a = fxdx = xdx =, x cos nxdx = x sin nx n sin nxdx n = cos nx n = n n, x sin nxdx = x cos nx n + cos nxdx n cos n = + sin
Διαβάστε περισσότεραProblem Set 3: Solutions
CMPSCI 69GG Applied Information Theory Fall 006 Problem Set 3: Solutions. [Cover and Thomas 7.] a Define the following notation, C I p xx; Y max X; Y C I p xx; Ỹ max I X; Ỹ We would like to show that C
Διαβάστε περισσότεραLecture 15 - Root System Axiomatics
Lecture 15 - Root System Axiomatics Nov 1, 01 In this lecture we examine root systems from an axiomatic point of view. 1 Reflections If v R n, then it determines a hyperplane, denoted P v, through the
Διαβάστε περισσότεραforms This gives Remark 1. How to remember the above formulas: Substituting these into the equation we obtain with
Week 03: C lassification of S econd- Order L inear Equations In last week s lectures we have illustrated how to obtain the general solutions of first order PDEs using the method of characteristics. We
Διαβάστε περισσότεραApproximation of distance between locations on earth given by latitude and longitude
Approximation of distance between locations on earth given by latitude and longitude Jan Behrens 2012-12-31 In this paper we shall provide a method to approximate distances between two points on earth
Διαβάστε περισσότεραSecond Order Partial Differential Equations
Chapter 7 Second Order Partial Differential Equations 7.1 Introduction A second order linear PDE in two independent variables (x, y Ω can be written as A(x, y u x + B(x, y u xy + C(x, y u u u + D(x, y
Διαβάστε περισσότεραANSWERSHEET (TOPIC = DIFFERENTIAL CALCULUS) COLLECTION #2. h 0 h h 0 h h 0 ( ) g k = g 0 + g 1 + g g 2009 =?
Teko Classes IITJEE/AIEEE Maths by SUHAAG SIR, Bhopal, Ph (0755) 3 00 000 www.tekoclasses.com ANSWERSHEET (TOPIC DIFFERENTIAL CALCULUS) COLLECTION # Question Type A.Single Correct Type Q. (A) Sol least
Διαβάστε περισσότεραSOLUTIONS TO MATH38181 EXTREME VALUES AND FINANCIAL RISK EXAM
SOLUTIONS TO MATH38181 EXTREME VALUES AND FINANCIAL RISK EXAM Solutions to Question 1 a) The cumulative distribution function of T conditional on N n is Pr T t N n) Pr max X 1,..., X N ) t N n) Pr max
Διαβάστε περισσότεραEcon 2110: Fall 2008 Suggested Solutions to Problem Set 8 questions or comments to Dan Fetter 1
Eon : Fall 8 Suggested Solutions to Problem Set 8 Email questions or omments to Dan Fetter Problem. Let X be a salar with density f(x, θ) (θx + θ) [ x ] with θ. (a) Find the most powerful level α test
Διαβάστε περισσότεραArithmetical applications of lagrangian interpolation. Tanguy Rivoal. Institut Fourier CNRS and Université de Grenoble 1
Arithmetical applications of lagrangian interpolation Tanguy Rivoal Institut Fourier CNRS and Université de Grenoble Conference Diophantine and Analytic Problems in Number Theory, The 00th anniversary
Διαβάστε περισσότεραNondifferentiable Convex Functions
Nondifferentiable Convex Functions DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis http://www.cims.nyu.edu/~cfgranda/pages/obda_fall17/index.html Carlos Fernandez-Granda Applications Subgradients
Διαβάστε περισσότεραThe Simply Typed Lambda Calculus
Type Inference Instead of writing type annotations, can we use an algorithm to infer what the type annotations should be? That depends on the type system. For simple type systems the answer is yes, and
Διαβάστε περισσότεραSection 8.3 Trigonometric Equations
99 Section 8. Trigonometric Equations Objective 1: Solve Equations Involving One Trigonometric Function. In this section and the next, we will exple how to solving equations involving trigonometric functions.
Διαβάστε περισσότεραORDINAL ARITHMETIC JULIAN J. SCHLÖDER
ORDINAL ARITHMETIC JULIAN J. SCHLÖDER Abstract. We define ordinal arithmetic and show laws of Left- Monotonicity, Associativity, Distributivity, some minor related properties and the Cantor Normal Form.
Διαβάστε περισσότεραFourier Series. MATH 211, Calculus II. J. Robert Buchanan. Spring Department of Mathematics
Fourier Series MATH 211, Calculus II J. Robert Buchanan Department of Mathematics Spring 2018 Introduction Not all functions can be represented by Taylor series. f (k) (c) A Taylor series f (x) = (x c)
Διαβάστε περισσότεραEcon Spring 2004 Instructor: Prof. Kiefer Solution to Problem set # 5. γ (0)
Cornell University Department of Economics Econ 60 - Spring 004 Instructor: Prof. Kiefer Solution to Problem set # 5. Autocorrelation function is defined as ρ h = γ h γ 0 where γ h =Cov X t,x t h =E[X
Διαβάστε περισσότεραSolutions to Exercise Sheet 5
Solutions to Eercise Sheet 5 jacques@ucsd.edu. Let X and Y be random variables with joint pdf f(, y) = 3y( + y) where and y. Determine each of the following probabilities. Solutions. a. P (X ). b. P (X
Διαβάστε περισσότεραMath 446 Homework 3 Solutions. (1). (i): Reverse triangle inequality for metrics: Let (X, d) be a metric space and let x, y, z X.
Math 446 Homework 3 Solutions. (1). (i): Reverse triangle inequalit for metrics: Let (X, d) be a metric space and let x,, z X. Prove that d(x, z) d(, z) d(x, ). (ii): Reverse triangle inequalit for norms:
Διαβάστε περισσότεραBayesian statistics. DS GA 1002 Probability and Statistics for Data Science.
Bayesian statistics DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Frequentist vs Bayesian statistics In frequentist
Διαβάστε περισσότεραCongruence Classes of Invertible Matrices of Order 3 over F 2
International Journal of Algebra, Vol. 8, 24, no. 5, 239-246 HIKARI Ltd, www.m-hikari.com http://dx.doi.org/.2988/ija.24.422 Congruence Classes of Invertible Matrices of Order 3 over F 2 Ligong An and
Διαβάστε περισσότεραLecture 13 - Root Space Decomposition II
Lecture 13 - Root Space Decomposition II October 18, 2012 1 Review First let us recall the situation. Let g be a simple algebra, with maximal toral subalgebra h (which we are calling a CSA, or Cartan Subalgebra).
Διαβάστε περισσότεραderivation of the Laplacian from rectangular to spherical coordinates
derivation of the Laplacian from rectangular to spherical coordinates swapnizzle 03-03- :5:43 We begin by recognizing the familiar conversion from rectangular to spherical coordinates (note that φ is used
Διαβάστε περισσότεραConcrete Mathematics Exercises from 30 September 2016
Concrete Mathematics Exercises from 30 September 2016 Silvio Capobianco Exercise 1.7 Let H(n) = J(n + 1) J(n). Equation (1.8) tells us that H(2n) = 2, and H(2n+1) = J(2n+2) J(2n+1) = (2J(n+1) 1) (2J(n)+1)
Διαβάστε περισσότερα3.4 SUM AND DIFFERENCE FORMULAS. NOTE: cos(α+β) cos α + cos β cos(α-β) cos α -cos β
3.4 SUM AND DIFFERENCE FORMULAS Page Theorem cos(αβ cos α cos β -sin α cos(α-β cos α cos β sin α NOTE: cos(αβ cos α cos β cos(α-β cos α -cos β Proof of cos(α-β cos α cos β sin α Let s use a unit circle
Διαβάστε περισσότεραJesse Maassen and Mark Lundstrom Purdue University November 25, 2013
Notes on Average Scattering imes and Hall Factors Jesse Maassen and Mar Lundstrom Purdue University November 5, 13 I. Introduction 1 II. Solution of the BE 1 III. Exercises: Woring out average scattering
Διαβάστε περισσότεραNumerical Analysis FMN011
Numerical Analysis FMN011 Carmen Arévalo Lund University carmen@maths.lth.se Lecture 12 Periodic data A function g has period P if g(x + P ) = g(x) Model: Trigonometric polynomial of order M T M (x) =
Διαβάστε περισσότεραOptimal Parameter in Hermitian and Skew-Hermitian Splitting Method for Certain Two-by-Two Block Matrices
Optimal Parameter in Hermitian and Skew-Hermitian Splitting Method for Certain Two-by-Two Block Matrices Chi-Kwong Li Department of Mathematics The College of William and Mary Williamsburg, Virginia 23187-8795
Διαβάστε περισσότεραSection 9.2 Polar Equations and Graphs
180 Section 9. Polar Equations and Graphs In this section, we will be graphing polar equations on a polar grid. In the first few examples, we will write the polar equation in rectangular form to help identify
Διαβάστε περισσότεραk A = [k, k]( )[a 1, a 2 ] = [ka 1,ka 2 ] 4For the division of two intervals of confidence in R +
Chapter 3. Fuzzy Arithmetic 3- Fuzzy arithmetic: ~Addition(+) and subtraction (-): Let A = [a and B = [b, b in R If x [a and y [b, b than x+y [a +b +b Symbolically,we write A(+)B = [a (+)[b, b = [a +b
Διαβάστε περισσότερα= λ 1 1 e. = λ 1 =12. has the properties e 1. e 3,V(Y
Stat 50 Homework Solutions Spring 005. (a λ λ λ 44 (b trace( λ + λ + λ 0 (c V (e x e e λ e e λ e (λ e by definition, the eigenvector e has the properties e λ e and e e. (d λ e e + λ e e + λ e e 8 6 4 4
Διαβάστε περισσότεραLecture 2: Dirac notation and a review of linear algebra Read Sakurai chapter 1, Baym chatper 3
Lecture 2: Dirac notation and a review of linear algebra Read Sakurai chapter 1, Baym chatper 3 1 State vector space and the dual space Space of wavefunctions The space of wavefunctions is the set of all
Διαβάστε περισσότεραΑπόκριση σε Μοναδιαία Ωστική Δύναμη (Unit Impulse) Απόκριση σε Δυνάμεις Αυθαίρετα Μεταβαλλόμενες με το Χρόνο. Απόστολος Σ.
Απόκριση σε Δυνάμεις Αυθαίρετα Μεταβαλλόμενες με το Χρόνο The time integral of a force is referred to as impulse, is determined by and is obtained from: Newton s 2 nd Law of motion states that the action
Διαβάστε περισσότεραThe challenges of non-stable predicates
The challenges of non-stable predicates Consider a non-stable predicate Φ encoding, say, a safety property. We want to determine whether Φ holds for our program. The challenges of non-stable predicates
Διαβάστε περισσότεραSection 7.6 Double and Half Angle Formulas
09 Section 7. Double and Half Angle Fmulas To derive the double-angles fmulas, we will use the sum of two angles fmulas that we developed in the last section. We will let α θ and β θ: cos(θ) cos(θ + θ)
Διαβάστε περισσότεραHomework 8 Model Solution Section
MATH 004 Homework Solution Homework 8 Model Solution Section 14.5 14.6. 14.5. Use the Chain Rule to find dz where z cosx + 4y), x 5t 4, y 1 t. dz dx + dy y sinx + 4y)0t + 4) sinx + 4y) 1t ) 0t + 4t ) sinx
Διαβάστε περισσότεραStatistics 104: Quantitative Methods for Economics Formula and Theorem Review
Harvard College Statistics 104: Quantitative Methods for Economics Formula and Theorem Review Tommy MacWilliam, 13 tmacwilliam@college.harvard.edu March 10, 2011 Contents 1 Introduction to Data 5 1.1 Sample
Διαβάστε περισσότεραThe Probabilistic Method - Probabilistic Techniques. Lecture 7: The Janson Inequality
The Probabilistic Method - Probabilistic Techniques Lecture 7: The Janson Inequality Sotiris Nikoletseas Associate Professor Computer Engineering and Informatics Department 2014-2015 Sotiris Nikoletseas,
Διαβάστε περισσότεραExercises 10. Find a fundamental matrix of the given system of equations. Also find the fundamental matrix Φ(t) satisfying Φ(0) = I. 1.
Exercises 0 More exercises are available in Elementary Differential Equations. If you have a problem to solve any of them, feel free to come to office hour. Problem Find a fundamental matrix of the given
Διαβάστε περισσότεραΜηχανική Μάθηση Hypothesis Testing
ΕΛΛΗΝΙΚΗ ΔΗΜΟΚΡΑΤΙΑ ΠΑΝΕΠΙΣΤΗΜΙΟ ΚΡΗΤΗΣ Μηχανική Μάθηση Hypothesis Testing Γιώργος Μπορμπουδάκης Τμήμα Επιστήμης Υπολογιστών Procedure 1. Form the null (H 0 ) and alternative (H 1 ) hypothesis 2. Consider
Διαβάστε περισσότεραω ω ω ω ω ω+2 ω ω+2 + ω ω ω ω+2 + ω ω+1 ω ω+2 2 ω ω ω ω ω ω ω ω+1 ω ω2 ω ω2 + ω ω ω2 + ω ω ω ω2 + ω ω+1 ω ω2 + ω ω+1 + ω ω ω ω2 + ω
0 1 2 3 4 5 6 ω ω + 1 ω + 2 ω + 3 ω + 4 ω2 ω2 + 1 ω2 + 2 ω2 + 3 ω3 ω3 + 1 ω3 + 2 ω4 ω4 + 1 ω5 ω 2 ω 2 + 1 ω 2 + 2 ω 2 + ω ω 2 + ω + 1 ω 2 + ω2 ω 2 2 ω 2 2 + 1 ω 2 2 + ω ω 2 3 ω 3 ω 3 + 1 ω 3 + ω ω 3 +
Διαβάστε περισσότεραLecture 10 - Representation Theory III: Theory of Weights
Lecture 10 - Representation Theory III: Theory of Weights February 18, 2012 1 Terminology One assumes a base = {α i } i has been chosen. Then a weight Λ with non-negative integral Dynkin coefficients Λ
Διαβάστε περισσότεραIntroduction to the ML Estimation of ARMA processes
Introduction to the ML Estimation of ARMA processes Eduardo Rossi University of Pavia October 2013 Rossi ARMA Estimation Financial Econometrics - 2013 1 / 1 We consider the AR(p) model: Y t = c + φ 1 Y
Διαβάστε περισσότερα5. Choice under Uncertainty
5. Choice under Uncertainty Daisuke Oyama Microeconomics I May 23, 2018 Formulations von Neumann-Morgenstern (1944/1947) X: Set of prizes Π: Set of probability distributions on X : Preference relation
Διαβάστε περισσότεραHW 3 Solutions 1. a) I use the auto.arima R function to search over models using AIC and decide on an ARMA(3,1)
HW 3 Solutions a) I use the autoarima R function to search over models using AIC and decide on an ARMA3,) b) I compare the ARMA3,) to ARMA,0) ARMA3,) does better in all three criteria c) The plot of the
Διαβάστε περισσότεραMain source: "Discrete-time systems and computer control" by Α. ΣΚΟΔΡΑΣ ΨΗΦΙΑΚΟΣ ΕΛΕΓΧΟΣ ΔΙΑΛΕΞΗ 4 ΔΙΑΦΑΝΕΙΑ 1
Main source: "Discrete-time systems and computer control" by Α. ΣΚΟΔΡΑΣ ΨΗΦΙΑΚΟΣ ΕΛΕΓΧΟΣ ΔΙΑΛΕΞΗ 4 ΔΙΑΦΑΝΕΙΑ 1 A Brief History of Sampling Research 1915 - Edmund Taylor Whittaker (1873-1956) devised a
Διαβάστε περισσότεραPg The perimeter is P = 3x The area of a triangle is. where b is the base, h is the height. In our case b = x, then the area is
Pg. 9. The perimeter is P = The area of a triangle is A = bh where b is the base, h is the height 0 h= btan 60 = b = b In our case b =, then the area is A = = 0. By Pythagorean theorem a + a = d a a =
Διαβάστε περισσότεραEstimation for ARMA Processes with Stable Noise. Matt Calder & Richard A. Davis Colorado State University
Estimation for ARMA Processes with Stable Noise Matt Calder & Richard A. Davis Colorado State University rdavis@stat.colostate.edu 1 ARMA processes with stable noise Review of M-estimation Examples of
Διαβάστε περισσότεραAreas and Lengths in Polar Coordinates
Kiryl Tsishchanka Areas and Lengths in Polar Coordinates In this section we develop the formula for the area of a region whose boundary is given by a polar equation. We need to use the formula for the
Διαβάστε περισσότεραA Note on Intuitionistic Fuzzy. Equivalence Relation
International Mathematical Forum, 5, 2010, no. 67, 3301-3307 A Note on Intuitionistic Fuzzy Equivalence Relation D. K. Basnet Dept. of Mathematics, Assam University Silchar-788011, Assam, India dkbasnet@rediffmail.com
Διαβάστε περισσότεραLecture 2. Soundness and completeness of propositional logic
Lecture 2 Soundness and completeness of propositional logic February 9, 2004 1 Overview Review of natural deduction. Soundness and completeness. Semantics of propositional formulas. Soundness proof. Completeness
Διαβάστε περισσότεραNew bounds for spherical two-distance sets and equiangular lines
New bounds for spherical two-distance sets and equiangular lines Michigan State University Oct 8-31, 016 Anhui University Definition If X = {x 1, x,, x N } S n 1 (unit sphere in R n ) and x i, x j = a
Διαβάστε περισσότερα6. MAXIMUM LIKELIHOOD ESTIMATION
6 MAXIMUM LIKELIHOOD ESIMAION [1] Maximum Likelihood Estimator (1) Cases in which θ (unknown parameter) is scalar Notational Clarification: From now on, we denote the true value of θ as θ o hen, view θ
Διαβάστε περισσότεραAreas and Lengths in Polar Coordinates
Kiryl Tsishchanka Areas and Lengths in Polar Coordinates In this section we develop the formula for the area of a region whose boundary is given by a polar equation. We need to use the formula for the
Διαβάστε περισσότεραPartial Differential Equations in Biology The boundary element method. March 26, 2013
The boundary element method March 26, 203 Introduction and notation The problem: u = f in D R d u = ϕ in Γ D u n = g on Γ N, where D = Γ D Γ N, Γ D Γ N = (possibly, Γ D = [Neumann problem] or Γ N = [Dirichlet
Διαβάστε περισσότεραCHAPTER 48 APPLICATIONS OF MATRICES AND DETERMINANTS
CHAPTER 48 APPLICATIONS OF MATRICES AND DETERMINANTS EXERCISE 01 Page 545 1. Use matrices to solve: 3x + 4y x + 5y + 7 3x + 4y x + 5y 7 Hence, 3 4 x 0 5 y 7 The inverse of 3 4 5 is: 1 5 4 1 5 4 15 8 3
Διαβάστε περισσότεραECE598: Information-theoretic methods in high-dimensional statistics Spring 2016
ECE598: Information-theoretic methods in high-dimensional statistics Spring 06 Lecture 7: Information bound Lecturer: Yihong Wu Scribe: Shiyu Liang, Feb 6, 06 [Ed. Mar 9] Recall the Chi-squared divergence
Διαβάστε περισσότεραMA 342N Assignment 1 Due 24 February 2016
M 342N ssignment Due 24 February 206 Id: 342N-s206-.m4,v. 206/02/5 2:25:36 john Exp john. Suppose that q, in addition to satisfying the assumptions from lecture, is an even function. Prove that η(λ = 0,
Διαβάστε περισσότεραThe ε-pseudospectrum of a Matrix
The ε-pseudospectrum of a Matrix Feb 16, 2015 () The ε-pseudospectrum of a Matrix Feb 16, 2015 1 / 18 1 Preliminaries 2 Definitions 3 Basic Properties 4 Computation of Pseudospectrum of 2 2 5 Problems
Διαβάστε περισσότεραDynamic types, Lambda calculus machines Section and Practice Problems Apr 21 22, 2016
Harvard School of Engineering and Applied Sciences CS 152: Programming Languages Dynamic types, Lambda calculus machines Apr 21 22, 2016 1 Dynamic types and contracts (a) To make sure you understand the
Διαβάστε περισσότεραMATH1030 Linear Algebra Assignment 5 Solution
2015-16 MATH1030 Linear Algebra Assignment 5 Solution (Question No. referring to the 8 th edition of the textbook) Section 3.4 4 Note that x 2 = x 1, x 3 = 2x 1 and x 1 0, hence span {x 1, x 2, x 3 } has
Διαβάστε περισσότεραHomomorphism in Intuitionistic Fuzzy Automata
International Journal of Fuzzy Mathematics Systems. ISSN 2248-9940 Volume 3, Number 1 (2013), pp. 39-45 Research India Publications http://www.ripublication.com/ijfms.htm Homomorphism in Intuitionistic
Διαβάστε περισσότεραSurvival Analysis: One-Sample Problem /Two-Sample Problem/Regression. Lu Tian and Richard Olshen Stanford University
Survival Analysis: One-Sample Problem /Two-Sample Problem/Regression Lu Tian and Richard Olshen Stanford University 1 One sample problem T 1,, T n 1 S( ), C 1,, C n G( ) and T i C i Observations: (U i,
Διαβάστε περισσότεραBounding Nonsplitting Enumeration Degrees
Bounding Nonsplitting Enumeration Degrees Thomas F. Kent Andrea Sorbi Università degli Studi di Siena Italia July 18, 2007 Goal: Introduce a form of Σ 0 2-permitting for the enumeration degrees. Till now,
Διαβάστε περισσότερα= {{D α, D α }, D α }. = [D α, 4iσ µ α α D α µ ] = 4iσ µ α α [Dα, D α ] µ.
PHY 396 T: SUSY Solutions for problem set #1. Problem 2(a): First of all, [D α, D 2 D α D α ] = {D α, D α }D α D α {D α, D α } = {D α, D α }D α + D α {D α, D α } (S.1) = {{D α, D α }, D α }. Second, {D
Διαβάστε περισσότεραAbstract Storage Devices
Abstract Storage Devices Robert König Ueli Maurer Stefano Tessaro SOFSEM 2009 January 27, 2009 Outline 1. Motivation: Storage Devices 2. Abstract Storage Devices (ASD s) 3. Reducibility 4. Factoring ASD
Διαβάστε περισσότεραSCITECH Volume 13, Issue 2 RESEARCH ORGANISATION Published online: March 29, 2018
Journal of rogressive Research in Mathematics(JRM) ISSN: 2395-028 SCITECH Volume 3, Issue 2 RESEARCH ORGANISATION ublished online: March 29, 208 Journal of rogressive Research in Mathematics www.scitecresearch.com/journals
Διαβάστε περισσότεραSrednicki Chapter 55
Srednicki Chapter 55 QFT Problems & Solutions A. George August 3, 03 Srednicki 55.. Use equations 55.3-55.0 and A i, A j ] = Π i, Π j ] = 0 (at equal times) to verify equations 55.-55.3. This is our third
Διαβάστε περισσότερα