Tutorial on Multinomial Logistic Regression
|
|
- Τερέντιος Γιάνναρης
- 6 χρόνια πριν
- Προβολές:
Transcript
1 Tutorial on Multinomial Logistic Regression Javier R Movellan June 19,
2 1 General Model The inputs are n-dimensional vectors the outputs are c-dimensional vectors The training sample consist of m input output pairs We organize the example inputs as an m n matrix x The corresponding example outputs are organized as a m c matrix y The models under consideration make predictions ŷ = h(u) (1) u = xθ (2) θ is a n c weight matrix Note θ ij can be seen as the connection strength from input variable X i to output variable U j We evaluate the 1 c c c n c m examples n Figure 1: There are m examples, n input variables and c output variables optimality of ŷ, and thus of θ, using the following criterion L(θ) = Φ(θ) + Γ(θ) (3) m Φ(θ) = w j ρ j (4) j=1 Γ(θ) is a prior penalty function of the parameters, ρ j = f(y j, ŷ j ) (5) measures the mismatch between the j th rows of y and ŷ The terms w 1,, w m are positive weights that capture the relative importance of each of the m input-output pairs 2
3 Our goal is to find a matrix θ that minimizes L A standard approach for minimizing L is the Newton-Raphson algorithm which calls for computation of the gradient and the Hessian matrix 11 Gradient Vector Let θ i R n be the i th column of θ, ie, the set of connection strengths from the n input units to the i th output unit The overall gradient of Φ with respect to θ is as follows θ 1 Φ θ 2 Φ vec[θ] Φ = (6) θ c Φ vec[θ] is the vectorized version of θ and the gradient of a vector q with respect to a vector p is defined as follows p q = q p (7) Using the chain rule for gradients we get Note Φ = u i ρ Φ u i ρ (8) Moreover ρ u i = u i = (xθ) i = xθ i (9) u i = θ ix = x (10) 0 0 ρ 0 2 u = 2i (11)
4 and Φ ρ = w 1 w m (12) Φ = x 0 0 ρ w 1 w 2 w m (13) Equivalently θ i Φ = Φ = x wψ i (14) w = w w 2 0 (15) w m and Ψ i = vec[θ] Φ = ρ 2 x wψ 1 x wψ c (16) (17) 4
5 12 Hessian Matrix The Hessian of a scalar v with respect to a vector u is defined as follows ) 2 uv = u ( u v = ( u v) (18) u x wψ 1 2 vec[θ]φ = vec[θ] (19) x wψ c = ( ) vec[θ] x wψ 1,, x wψ c (20) vec[θ] x wψ i = vec[θ] Ψ i Ψi x wψ i ( vec[θ] Ψ i )wx (21) ( 2 vec[θ]φ = ( vec[θ] Ψ 1 )wx,, ( vec[θ] Ψ c )wx ) (22) 2 vec[θ]φ = = θ 1 Ψ 1 θ c Ψ 1 ( θ 1 Ψ 1 )wx ( θ c Ψ 1 )wx wx,, θ 1 Ψ c θ c Ψ c ( θ 1 Ψ c )wx ( θ c Ψ c )wx wx (23) (24) Note θ i Ψ j = θ i u j u j Ψ j = x Λ ij (25) Λ ij = u j Ψ j = Ψ 1j 0 0 Ψ 0 2j Ψ mj (26) 5
6 θ i Ψ j = Ψ j = Λ ij = Ψ x 1j Ψ 11 x mj m1 x 1n Ψ 1j u ni x mn Ψ mj Ψ 1j 0 0 Ψ 0 2j Ψ mj = x Λ ij (27) (28) 2 vec[θ]φ = 2 Quadratic Priors Let then x Λ 11 wx x Λ 1c wx x Λ c1 wx x Λ cc wx (29) Γ(θ) = 1 2 (vec[θ] vec[µ]) σ 1 (vec[θ] vec[µ]) (30) vec[θ] Γ(θ) = σ 1 (vec[θ] vec[µ]) (31) 2 vec[θ] = vec[θ] vec[θ] = σ 1 (32) 3 Newton-Raphson Optimization vec[θ (k+1) ] = vec[θ (k) ] ( 2 vec[θ]l(θ (k)) 1 vec[θ] L(θ (k) ) (33) θ (k) is the value of θ after k iterations of the optimization algorithm 6
7 4 Multivariate Linear Regression In this case ŷ i = u i (34) ρ i = (ŷ ik y ik ) 2 (35) k I m is the m m identity matrix Ψ i = ŷ i y i (36) Λ ij = I m (37) Φ θ j = x wx (38) 5 Multinomial Logistic Regression Let ŷ ij = f j (u i ) = e u ij c k=1 eu ik (39) and ρ i the negative log-likelihood of the output vcector y i given the input vector x i c ρi = y ik log ŷ ik (40) Note ρ i u ik = k=1 ŷ ij u ik = ŷ ij log ŷ ij u ik = ŷ ij (δ jk ŷ ik ) (41) c j=1 y ij ŷ ij ŷ ij u ij = c y ij (δ jk ŷ ik ) = ŷ ik y ik (42) j=1 we used the fact that j y ij = 1 7
8 Moreover Ψ i = ρ 2 Λ ij = y 1i ŷ 1i = y 2i ŷ 2i = ŷ i y i (43) y mi ŷ mi Ψ 1j 0 0 Ψ 0 2j Ψ mj (44) Ψ kj u ki = ŷ kj u ki = ŷ kj (δ ij ŷ kj ) (45) 51 Relationship to Linear Regression Note that the gradient in multinomial logistic regression is identical to the gradient in multivariate linear regression θ i Φ = ŷ i y i (46) The Hessians would are also very simmilar In linear regression and in logistic regression Φ θ j Φ θ j = x wx (47) = x wλ ij x (48) which can be seen as special case of linear regression the weight matrix w is substituted by the wλ ij matrix 8
9 52 Summary Training Data Inputs x R m R n Rows are examples, columns are input variables Outputs y R m R c Rows are examples, columns are labels or label probabilities All entries are non-negative and each row add up to 1 Example Weights w R m R r Diagonal matrix Each term is positive and represents the relative importance of each example Gradients vec[θ] L = θ 1 Φ θ 2 Φ θ c Φ + vec[θ] Γ (49) θ i Φ = Φ = x wψ i (50) and and Ψ i = ρ 2 = ŷ i y i (51) Hessian Matrices vec[θ] Γ = σ 1 (vec[θ] vec[µ]) (52) 2 vec[θ]l = 2 vec[θ]φ + 2 vec[θ]γ (53) 9
10 2 vec[θ]φ = θ 1 Φ θ 1 Φ θ c θ 1 θ 1 θ c Φ θ c Φ θ c (54) Φ θ j = x Λ ij wx (55) Λ ij = Ψ 1j 0 0 Ψ 0 2j Ψ mj (56) and Ψ kj u ki = ŷ kj u ki = ŷ kj (δ ij ŷ kj ) (57) and 2 vec[θ]γ = σ 1 (58) Learning Rule (Newton-Raphson) vec[θ (k+1) ] = vec[θ (k) ] ( 2 vec[θ]l(θ (k)) 1 vec[θ] L(θ (k) ) (59) θ (k) is the value of θ after k iterations of the learning rule 6 Appendix: L-p priors From a Bayesian point of view the Φ function can be seen as a log-likelihood term At times it may be useful to add a log prior term over θ, also known as a regularization term Most prior terms of interest have the following form 10
11 Γ = α n i=1 c j=1 1 ( p h(θ i,j )) (60) p α is a non-negative constant If h is the absolute value function then Γ is the L-p norm of θ Two popular options are p = 2 and p = 1 Note the absolute value function is not differentiable, and thus when p is odd, it useful to approximate it with a differentiable function Here we will use the log hyperbolic cosine function β 0 is a gain parameter and h(x) = 1 log(2cosh(βx)) (61) β Note cosh(x) = ex + e x 2 (62) lim h(x) = abs(x) (63) β h (x) = dh(x) dx = eβx e βx = tanh(x) (64) e βx + e βx and h (x) = d2 h(x) dx 2 dh(x) lim β dx = d tanh(βx) dx = sign(x) (65) = β (1 tanh 2 (βx)) (66) 61 Gradient and Hessian Γ ( ) = α h p 1 (θ ij ) h (θ ij ) θ ij ( = α ( 1 ) β log(2cosh(βx)))p 1 tanh(x) (67) (68) 11
12 The nc nc Hessian matrix is diagonal Each term in the diagonal has the following form 2 Γ θ 2 ij ( ) = α (p 1)h(θ ij ) p 2 (h (θ ij )) 2 + h(θ ij ) p 1 h (θ ij ) (69) 2 Γ θ 2 ij ( = α (p 1)( 1 β log(2cosh(βx)))p 2 (tanh(βx)) 2 (70) + ( 1 ) β log(2cosh(βx)))p 1 β (1 tanh 2 (βx) (71) Note for p = 1, we get and for p = 1, and β we get Γ = α 1 θ ij β tanh(βθ ij) (72) 2 Γ = α β (1 tanh 2 (βθ θij 2 ij )) (73) (74) Γ = α θ ij θ ij (75) 2 Γ = α θij 2 (76) 12
13 5 4 logcosh abs h(x) x 1 05 tanh sign dh(x)/dx x Figure 2: The absolute value function can be approximated by the log of a hyperbolic cosine The derivatie of the absolute value is the sign function It can be approximated by the derivative of the log cosh function, which is the hyperbolic tangent In this figure we used gain β = 4 13
Other Test Constructions: Likelihood Ratio & Bayes Tests
Other Test Constructions: Likelihood Ratio & Bayes Tests Side-Note: So far we have seen a few approaches for creating tests such as Neyman-Pearson Lemma ( most powerful tests of H 0 : θ = θ 0 vs H 1 :
Διαβάστε περισσότεραSection 8.3 Trigonometric Equations
99 Section 8. Trigonometric Equations Objective 1: Solve Equations Involving One Trigonometric Function. In this section and the next, we will exple how to solving equations involving trigonometric functions.
Διαβάστε περισσότεραPartial Differential Equations in Biology The boundary element method. March 26, 2013
The boundary element method March 26, 203 Introduction and notation The problem: u = f in D R d u = ϕ in Γ D u n = g on Γ N, where D = Γ D Γ N, Γ D Γ N = (possibly, Γ D = [Neumann problem] or Γ N = [Dirichlet
Διαβάστε περισσότερα2 Composition. Invertible Mappings
Arkansas Tech University MATH 4033: Elementary Modern Algebra Dr. Marcel B. Finan Composition. Invertible Mappings In this section we discuss two procedures for creating new mappings from old ones, namely,
Διαβάστε περισσότεραHomework 3 Solutions
Homework 3 Solutions Igor Yanovsky (Math 151A TA) Problem 1: Compute the absolute error and relative error in approximations of p by p. (Use calculator!) a) p π, p 22/7; b) p π, p 3.141. Solution: For
Διαβάστε περισσότερα3.4 SUM AND DIFFERENCE FORMULAS. NOTE: cos(α+β) cos α + cos β cos(α-β) cos α -cos β
3.4 SUM AND DIFFERENCE FORMULAS Page Theorem cos(αβ cos α cos β -sin α cos(α-β cos α cos β sin α NOTE: cos(αβ cos α cos β cos(α-β cos α -cos β Proof of cos(α-β cos α cos β sin α Let s use a unit circle
Διαβάστε περισσότεραNumerical Analysis FMN011
Numerical Analysis FMN011 Carmen Arévalo Lund University carmen@maths.lth.se Lecture 12 Periodic data A function g has period P if g(x + P ) = g(x) Model: Trigonometric polynomial of order M T M (x) =
Διαβάστε περισσότεραSCHOOL OF MATHEMATICAL SCIENCES G11LMA Linear Mathematics Examination Solutions
SCHOOL OF MATHEMATICAL SCIENCES GLMA Linear Mathematics 00- Examination Solutions. (a) i. ( + 5i)( i) = (6 + 5) + (5 )i = + i. Real part is, imaginary part is. (b) ii. + 5i i ( + 5i)( + i) = ( i)( + i)
Διαβάστε περισσότεραSolution Series 9. i=1 x i and i=1 x i.
Lecturer: Prof. Dr. Mete SONER Coordinator: Yilin WANG Solution Series 9 Q1. Let α, β >, the p.d.f. of a beta distribution with parameters α and β is { Γ(α+β) Γ(α)Γ(β) f(x α, β) xα 1 (1 x) β 1 for < x
Διαβάστε περισσότεραderivation of the Laplacian from rectangular to spherical coordinates
derivation of the Laplacian from rectangular to spherical coordinates swapnizzle 03-03- :5:43 We begin by recognizing the familiar conversion from rectangular to spherical coordinates (note that φ is used
Διαβάστε περισσότεραChapter 6: Systems of Linear Differential. be continuous functions on the interval
Chapter 6: Systems of Linear Differential Equations Let a (t), a 2 (t),..., a nn (t), b (t), b 2 (t),..., b n (t) be continuous functions on the interval I. The system of n first-order differential equations
Διαβάστε περισσότεραw o = R 1 p. (1) R = p =. = 1
Πανεπιστήµιο Κρήτης - Τµήµα Επιστήµης Υπολογιστών ΗΥ-570: Στατιστική Επεξεργασία Σήµατος 205 ιδάσκων : Α. Μουχτάρης Τριτη Σειρά Ασκήσεων Λύσεις Ασκηση 3. 5.2 (a) From the Wiener-Hopf equation we have:
Διαβάστε περισσότεραHomework 8 Model Solution Section
MATH 004 Homework Solution Homework 8 Model Solution Section 14.5 14.6. 14.5. Use the Chain Rule to find dz where z cosx + 4y), x 5t 4, y 1 t. dz dx + dy y sinx + 4y)0t + 4) sinx + 4y) 1t ) 0t + 4t ) sinx
Διαβάστε περισσότερα4.6 Autoregressive Moving Average Model ARMA(1,1)
84 CHAPTER 4. STATIONARY TS MODELS 4.6 Autoregressive Moving Average Model ARMA(,) This section is an introduction to a wide class of models ARMA(p,q) which we will consider in more detail later in this
Διαβάστε περισσότεραST5224: Advanced Statistical Theory II
ST5224: Advanced Statistical Theory II 2014/2015: Semester II Tutorial 7 1. Let X be a sample from a population P and consider testing hypotheses H 0 : P = P 0 versus H 1 : P = P 1, where P j is a known
Διαβάστε περισσότεραCHAPTER 25 SOLVING EQUATIONS BY ITERATIVE METHODS
CHAPTER 5 SOLVING EQUATIONS BY ITERATIVE METHODS EXERCISE 104 Page 8 1. Find the positive root of the equation x + 3x 5 = 0, correct to 3 significant figures, using the method of bisection. Let f(x) =
Διαβάστε περισσότεραNotes on the Open Economy
Notes on the Open Econom Ben J. Heijdra Universit of Groningen April 24 Introduction In this note we stud the two-countr model of Table.4 in more detail. restated here for convenience. The model is Table.4.
Διαβάστε περισσότεραStatistical Inference I Locally most powerful tests
Statistical Inference I Locally most powerful tests Shirsendu Mukherjee Department of Statistics, Asutosh College, Kolkata, India. shirsendu st@yahoo.co.in So far we have treated the testing of one-sided
Διαβάστε περισσότεραBayesian statistics. DS GA 1002 Probability and Statistics for Data Science.
Bayesian statistics DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Frequentist vs Bayesian statistics In frequentist
Διαβάστε περισσότεραLecture 2: Dirac notation and a review of linear algebra Read Sakurai chapter 1, Baym chatper 3
Lecture 2: Dirac notation and a review of linear algebra Read Sakurai chapter 1, Baym chatper 3 1 State vector space and the dual space Space of wavefunctions The space of wavefunctions is the set of all
Διαβάστε περισσότεραReminders: linear functions
Reminders: linear functions Let U and V be vector spaces over the same field F. Definition A function f : U V is linear if for every u 1, u 2 U, f (u 1 + u 2 ) = f (u 1 ) + f (u 2 ), and for every u U
Διαβάστε περισσότεραMath221: HW# 1 solutions
Math: HW# solutions Andy Royston October, 5 7.5.7, 3 rd Ed. We have a n = b n = a = fxdx = xdx =, x cos nxdx = x sin nx n sin nxdx n = cos nx n = n n, x sin nxdx = x cos nx n + cos nxdx n cos n = + sin
Διαβάστε περισσότεραSolutions to Exercise Sheet 5
Solutions to Eercise Sheet 5 jacques@ucsd.edu. Let X and Y be random variables with joint pdf f(, y) = 3y( + y) where and y. Determine each of the following probabilities. Solutions. a. P (X ). b. P (X
Διαβάστε περισσότεραApproximation of distance between locations on earth given by latitude and longitude
Approximation of distance between locations on earth given by latitude and longitude Jan Behrens 2012-12-31 In this paper we shall provide a method to approximate distances between two points on earth
Διαβάστε περισσότεραΑπόκριση σε Μοναδιαία Ωστική Δύναμη (Unit Impulse) Απόκριση σε Δυνάμεις Αυθαίρετα Μεταβαλλόμενες με το Χρόνο. Απόστολος Σ.
Απόκριση σε Δυνάμεις Αυθαίρετα Μεταβαλλόμενες με το Χρόνο The time integral of a force is referred to as impulse, is determined by and is obtained from: Newton s 2 nd Law of motion states that the action
Διαβάστε περισσότεραParametrized Surfaces
Parametrized Surfaces Recall from our unit on vector-valued functions at the beginning of the semester that an R 3 -valued function c(t) in one parameter is a mapping of the form c : I R 3 where I is some
Διαβάστε περισσότερα6.3 Forecasting ARMA processes
122 CHAPTER 6. ARMA MODELS 6.3 Forecasting ARMA processes The purpose of forecasting is to predict future values of a TS based on the data collected to the present. In this section we will discuss a linear
Διαβάστε περισσότερα6.1. Dirac Equation. Hamiltonian. Dirac Eq.
6.1. Dirac Equation Ref: M.Kaku, Quantum Field Theory, Oxford Univ Press (1993) η μν = η μν = diag(1, -1, -1, -1) p 0 = p 0 p = p i = -p i p μ p μ = p 0 p 0 + p i p i = E c 2 - p 2 = (m c) 2 H = c p 2
Διαβάστε περισσότεραSection 7.6 Double and Half Angle Formulas
09 Section 7. Double and Half Angle Fmulas To derive the double-angles fmulas, we will use the sum of two angles fmulas that we developed in the last section. We will let α θ and β θ: cos(θ) cos(θ + θ)
Διαβάστε περισσότεραHOMEWORK 4 = G. In order to plot the stress versus the stretch we define a normalized stretch:
HOMEWORK 4 Problem a For the fast loading case, we want to derive the relationship between P zz and λ z. We know that the nominal stress is expressed as: P zz = ψ λ z where λ z = λ λ z. Therefore, applying
Διαβάστε περισσότεραThe Simply Typed Lambda Calculus
Type Inference Instead of writing type annotations, can we use an algorithm to infer what the type annotations should be? That depends on the type system. For simple type systems the answer is yes, and
Διαβάστε περισσότεραMatrices and Determinants
Matrices and Determinants SUBJECTIVE PROBLEMS: Q 1. For what value of k do the following system of equations possess a non-trivial (i.e., not all zero) solution over the set of rationals Q? x + ky + 3z
Διαβάστε περισσότεραEE512: Error Control Coding
EE512: Error Control Coding Solution for Assignment on Finite Fields February 16, 2007 1. (a) Addition and Multiplication tables for GF (5) and GF (7) are shown in Tables 1 and 2. + 0 1 2 3 4 0 0 1 2 3
Διαβάστε περισσότεραAreas and Lengths in Polar Coordinates
Kiryl Tsishchanka Areas and Lengths in Polar Coordinates In this section we develop the formula for the area of a region whose boundary is given by a polar equation. We need to use the formula for the
Διαβάστε περισσότεραMatrices and vectors. Matrix and vector. a 11 a 12 a 1n a 21 a 22 a 2n A = b 1 b 2. b m. R m n, b = = ( a ij. a m1 a m2 a mn. def
Matrices and vectors Matrix and vector a 11 a 12 a 1n a 21 a 22 a 2n A = a m1 a m2 a mn def = ( a ij ) R m n, b = b 1 b 2 b m Rm Matrix and vectors in linear equations: example E 1 : x 1 + x 2 + 3x 4 =
Διαβάστε περισσότεραExample Sheet 3 Solutions
Example Sheet 3 Solutions. i Regular Sturm-Liouville. ii Singular Sturm-Liouville mixed boundary conditions. iii Not Sturm-Liouville ODE is not in Sturm-Liouville form. iv Regular Sturm-Liouville note
Διαβάστε περισσότεραPractice Exam 2. Conceptual Questions. 1. State a Basic identity and then verify it. (a) Identity: Solution: One identity is csc(θ) = 1
Conceptual Questions. State a Basic identity and then verify it. a) Identity: Solution: One identity is cscθ) = sinθ) Practice Exam b) Verification: Solution: Given the point of intersection x, y) of the
Διαβάστε περισσότεραJesse Maassen and Mark Lundstrom Purdue University November 25, 2013
Notes on Average Scattering imes and Hall Factors Jesse Maassen and Mar Lundstrom Purdue University November 5, 13 I. Introduction 1 II. Solution of the BE 1 III. Exercises: Woring out average scattering
Διαβάστε περισσότεραDurbin-Levinson recursive method
Durbin-Levinson recursive method A recursive method for computing ϕ n is useful because it avoids inverting large matrices; when new data are acquired, one can update predictions, instead of starting again
Διαβάστε περισσότεραg-selberg integrals MV Conjecture An A 2 Selberg integral Summary Long Live the King Ole Warnaar Department of Mathematics Long Live the King
Ole Warnaar Department of Mathematics g-selberg integrals The Selberg integral corresponds to the following k-dimensional generalisation of the beta integral: D Here and k t α 1 i (1 t i ) β 1 1 i
Διαβάστε περισσότεραInstruction Execution Times
1 C Execution Times InThisAppendix... Introduction DL330 Execution Times DL330P Execution Times DL340 Execution Times C-2 Execution Times Introduction Data Registers This appendix contains several tables
Διαβάστε περισσότεραChapter 6: Systems of Linear Differential. be continuous functions on the interval
Chapter 6: Systems of Linear Differential Equations Let a (t), a 2 (t),..., a nn (t), b (t), b 2 (t),..., b n (t) be continuous functions on the interval I. The system of n first-order differential equations
Διαβάστε περισσότεραΚΥΠΡΙΑΚΗ ΕΤΑΙΡΕΙΑ ΠΛΗΡΟΦΟΡΙΚΗΣ CYPRUS COMPUTER SOCIETY ΠΑΓΚΥΠΡΙΟΣ ΜΑΘΗΤΙΚΟΣ ΔΙΑΓΩΝΙΣΜΟΣ ΠΛΗΡΟΦΟΡΙΚΗΣ 6/5/2006
Οδηγίες: Να απαντηθούν όλες οι ερωτήσεις. Ολοι οι αριθμοί που αναφέρονται σε όλα τα ερωτήματα είναι μικρότεροι το 1000 εκτός αν ορίζεται διαφορετικά στη διατύπωση του προβλήματος. Διάρκεια: 3,5 ώρες Καλή
Διαβάστε περισσότεραCongruence Classes of Invertible Matrices of Order 3 over F 2
International Journal of Algebra, Vol. 8, 24, no. 5, 239-246 HIKARI Ltd, www.m-hikari.com http://dx.doi.org/.2988/ija.24.422 Congruence Classes of Invertible Matrices of Order 3 over F 2 Ligong An and
Διαβάστε περισσότεραAreas and Lengths in Polar Coordinates
Kiryl Tsishchanka Areas and Lengths in Polar Coordinates In this section we develop the formula for the area of a region whose boundary is given by a polar equation. We need to use the formula for the
Διαβάστε περισσότεραCHAPTER 48 APPLICATIONS OF MATRICES AND DETERMINANTS
CHAPTER 48 APPLICATIONS OF MATRICES AND DETERMINANTS EXERCISE 01 Page 545 1. Use matrices to solve: 3x + 4y x + 5y + 7 3x + 4y x + 5y 7 Hence, 3 4 x 0 5 y 7 The inverse of 3 4 5 is: 1 5 4 1 5 4 15 8 3
Διαβάστε περισσότεραFractional Colorings and Zykov Products of graphs
Fractional Colorings and Zykov Products of graphs Who? Nichole Schimanski When? July 27, 2011 Graphs A graph, G, consists of a vertex set, V (G), and an edge set, E(G). V (G) is any finite set E(G) is
Διαβάστε περισσότεραHW 3 Solutions 1. a) I use the auto.arima R function to search over models using AIC and decide on an ARMA(3,1)
HW 3 Solutions a) I use the autoarima R function to search over models using AIC and decide on an ARMA3,) b) I compare the ARMA3,) to ARMA,0) ARMA3,) does better in all three criteria c) The plot of the
Διαβάστε περισσότεραTMA4115 Matematikk 3
TMA4115 Matematikk 3 Andrew Stacey Norges Teknisk-Naturvitenskapelige Universitet Trondheim Spring 2010 Lecture 12: Mathematics Marvellous Matrices Andrew Stacey Norges Teknisk-Naturvitenskapelige Universitet
Διαβάστε περισσότεραEcon 2110: Fall 2008 Suggested Solutions to Problem Set 8 questions or comments to Dan Fetter 1
Eon : Fall 8 Suggested Solutions to Problem Set 8 Email questions or omments to Dan Fetter Problem. Let X be a salar with density f(x, θ) (θx + θ) [ x ] with θ. (a) Find the most powerful level α test
Διαβάστε περισσότεραC.S. 430 Assignment 6, Sample Solutions
C.S. 430 Assignment 6, Sample Solutions Paul Liu November 15, 2007 Note that these are sample solutions only; in many cases there were many acceptable answers. 1 Reynolds Problem 10.1 1.1 Normal-order
Διαβάστε περισσότεραConcrete Mathematics Exercises from 30 September 2016
Concrete Mathematics Exercises from 30 September 2016 Silvio Capobianco Exercise 1.7 Let H(n) = J(n + 1) J(n). Equation (1.8) tells us that H(2n) = 2, and H(2n+1) = J(2n+2) J(2n+1) = (2J(n+1) 1) (2J(n)+1)
Διαβάστε περισσότεραSpherical Coordinates
Spherical Coordinates MATH 311, Calculus III J. Robert Buchanan Department of Mathematics Fall 2011 Spherical Coordinates Another means of locating points in three-dimensional space is known as the spherical
Διαβάστε περισσότερα6. MAXIMUM LIKELIHOOD ESTIMATION
6 MAXIMUM LIKELIHOOD ESIMAION [1] Maximum Likelihood Estimator (1) Cases in which θ (unknown parameter) is scalar Notational Clarification: From now on, we denote the true value of θ as θ o hen, view θ
Διαβάστε περισσότεραHOMEWORK#1. t E(x) = 1 λ = (b) Find the median lifetime of a randomly selected light bulb. Answer:
HOMEWORK# 52258 李亞晟 Eercise 2. The lifetime of light bulbs follows an eponential distribution with a hazard rate of. failures per hour of use (a) Find the mean lifetime of a randomly selected light bulb.
Διαβάστε περισσότεραNew bounds for spherical two-distance sets and equiangular lines
New bounds for spherical two-distance sets and equiangular lines Michigan State University Oct 8-31, 016 Anhui University Definition If X = {x 1, x,, x N } S n 1 (unit sphere in R n ) and x i, x j = a
Διαβάστε περισσότεραOrdinal Arithmetic: Addition, Multiplication, Exponentiation and Limit
Ordinal Arithmetic: Addition, Multiplication, Exponentiation and Limit Ting Zhang Stanford May 11, 2001 Stanford, 5/11/2001 1 Outline Ordinal Classification Ordinal Addition Ordinal Multiplication Ordinal
Διαβάστε περισσότεραSecond Order Partial Differential Equations
Chapter 7 Second Order Partial Differential Equations 7.1 Introduction A second order linear PDE in two independent variables (x, y Ω can be written as A(x, y u x + B(x, y u xy + C(x, y u u u + D(x, y
Διαβάστε περισσότεραΚΥΠΡΙΑΚΗ ΕΤΑΙΡΕΙΑ ΠΛΗΡΟΦΟΡΙΚΗΣ CYPRUS COMPUTER SOCIETY ΠΑΓΚΥΠΡΙΟΣ ΜΑΘΗΤΙΚΟΣ ΔΙΑΓΩΝΙΣΜΟΣ ΠΛΗΡΟΦΟΡΙΚΗΣ 24/3/2007
Οδηγίες: Να απαντηθούν όλες οι ερωτήσεις. Όλοι οι αριθμοί που αναφέρονται σε όλα τα ερωτήματα μικρότεροι του 10000 εκτός αν ορίζεται διαφορετικά στη διατύπωση του προβλήματος. Αν κάπου κάνετε κάποιες υποθέσεις
Διαβάστε περισσότεραΚΥΠΡΙΑΚΗ ΕΤΑΙΡΕΙΑ ΠΛΗΡΟΦΟΡΙΚΗΣ CYPRUS COMPUTER SOCIETY ΠΑΓΚΥΠΡΙΟΣ ΜΑΘΗΤΙΚΟΣ ΔΙΑΓΩΝΙΣΜΟΣ ΠΛΗΡΟΦΟΡΙΚΗΣ 19/5/2007
Οδηγίες: Να απαντηθούν όλες οι ερωτήσεις. Αν κάπου κάνετε κάποιες υποθέσεις να αναφερθούν στη σχετική ερώτηση. Όλα τα αρχεία που αναφέρονται στα προβλήματα βρίσκονται στον ίδιο φάκελο με το εκτελέσιμο
Διαβάστε περισσότεραSOLUTIONS TO MATH38181 EXTREME VALUES AND FINANCIAL RISK EXAM
SOLUTIONS TO MATH38181 EXTREME VALUES AND FINANCIAL RISK EXAM Solutions to Question 1 a) The cumulative distribution function of T conditional on N n is Pr T t N n) Pr max X 1,..., X N ) t N n) Pr max
Διαβάστε περισσότεραANSWERSHEET (TOPIC = DIFFERENTIAL CALCULUS) COLLECTION #2. h 0 h h 0 h h 0 ( ) g k = g 0 + g 1 + g g 2009 =?
Teko Classes IITJEE/AIEEE Maths by SUHAAG SIR, Bhopal, Ph (0755) 3 00 000 www.tekoclasses.com ANSWERSHEET (TOPIC DIFFERENTIAL CALCULUS) COLLECTION # Question Type A.Single Correct Type Q. (A) Sol least
Διαβάστε περισσότεραFinite Field Problems: Solutions
Finite Field Problems: Solutions 1. Let f = x 2 +1 Z 11 [x] and let F = Z 11 [x]/(f), a field. Let Solution: F =11 2 = 121, so F = 121 1 = 120. The possible orders are the divisors of 120. Solution: The
Διαβάστε περισσότεραω ω ω ω ω ω+2 ω ω+2 + ω ω ω ω+2 + ω ω+1 ω ω+2 2 ω ω ω ω ω ω ω ω+1 ω ω2 ω ω2 + ω ω ω2 + ω ω ω ω2 + ω ω+1 ω ω2 + ω ω+1 + ω ω ω ω2 + ω
0 1 2 3 4 5 6 ω ω + 1 ω + 2 ω + 3 ω + 4 ω2 ω2 + 1 ω2 + 2 ω2 + 3 ω3 ω3 + 1 ω3 + 2 ω4 ω4 + 1 ω5 ω 2 ω 2 + 1 ω 2 + 2 ω 2 + ω ω 2 + ω + 1 ω 2 + ω2 ω 2 2 ω 2 2 + 1 ω 2 2 + ω ω 2 3 ω 3 ω 3 + 1 ω 3 + ω ω 3 +
Διαβάστε περισσότεραSCITECH Volume 13, Issue 2 RESEARCH ORGANISATION Published online: March 29, 2018
Journal of rogressive Research in Mathematics(JRM) ISSN: 2395-028 SCITECH Volume 3, Issue 2 RESEARCH ORGANISATION ublished online: March 29, 208 Journal of rogressive Research in Mathematics www.scitecresearch.com/journals
Διαβάστε περισσότεραLecture 13 - Root Space Decomposition II
Lecture 13 - Root Space Decomposition II October 18, 2012 1 Review First let us recall the situation. Let g be a simple algebra, with maximal toral subalgebra h (which we are calling a CSA, or Cartan Subalgebra).
Διαβάστε περισσότερα5.4 The Poisson Distribution.
The worst thing you can do about a situation is nothing. Sr. O Shea Jackson 5.4 The Poisson Distribution. Description of the Poisson Distribution Discrete probability distribution. The random variable
Διαβάστε περισσότεραStatistics 104: Quantitative Methods for Economics Formula and Theorem Review
Harvard College Statistics 104: Quantitative Methods for Economics Formula and Theorem Review Tommy MacWilliam, 13 tmacwilliam@college.harvard.edu March 10, 2011 Contents 1 Introduction to Data 5 1.1 Sample
Διαβάστε περισσότεραSOLUTIONS TO MATH38181 EXTREME VALUES AND FINANCIAL RISK EXAM
SOLUTIONS TO MATH38181 EXTREME VALUES AND FINANCIAL RISK EXAM Solutions to Question 1 a) The cumulative distribution function of T conditional on N n is Pr (T t N n) Pr (max (X 1,..., X N ) t N n) Pr (max
Διαβάστε περισσότεραMATH423 String Theory Solutions 4. = 0 τ = f(s). (1) dτ ds = dxµ dτ f (s) (2) dτ 2 [f (s)] 2 + dxµ. dτ f (s) (3)
1. MATH43 String Theory Solutions 4 x = 0 τ = fs). 1) = = f s) ) x = x [f s)] + f s) 3) equation of motion is x = 0 if an only if f s) = 0 i.e. fs) = As + B with A, B constants. i.e. allowe reparametrisations
Διαβάστε περισσότεραSecond Order RLC Filters
ECEN 60 Circuits/Electronics Spring 007-0-07 P. Mathys Second Order RLC Filters RLC Lowpass Filter A passive RLC lowpass filter (LPF) circuit is shown in the following schematic. R L C v O (t) Using phasor
Διαβάστε περισσότεραFourier Series. MATH 211, Calculus II. J. Robert Buchanan. Spring Department of Mathematics
Fourier Series MATH 211, Calculus II J. Robert Buchanan Department of Mathematics Spring 2018 Introduction Not all functions can be represented by Taylor series. f (k) (c) A Taylor series f (x) = (x c)
Διαβάστε περισσότεραProblem Set 3: Solutions
CMPSCI 69GG Applied Information Theory Fall 006 Problem Set 3: Solutions. [Cover and Thomas 7.] a Define the following notation, C I p xx; Y max X; Y C I p xx; Ỹ max I X; Ỹ We would like to show that C
Διαβάστε περισσότεραStrain gauge and rosettes
Strain gauge and rosettes Introduction A strain gauge is a device which is used to measure strain (deformation) on an object subjected to forces. Strain can be measured using various types of devices classified
Διαβάστε περισσότεραSpace-Time Symmetries
Chapter Space-Time Symmetries In classical fiel theory any continuous symmetry of the action generates a conserve current by Noether's proceure. If the Lagrangian is not invariant but only shifts by a
Διαβάστε περισσότεραΠρόβλημα 1: Αναζήτηση Ελάχιστης/Μέγιστης Τιμής
Πρόβλημα 1: Αναζήτηση Ελάχιστης/Μέγιστης Τιμής Να γραφεί πρόγραμμα το οποίο δέχεται ως είσοδο μια ακολουθία S από n (n 40) ακέραιους αριθμούς και επιστρέφει ως έξοδο δύο ακολουθίες από θετικούς ακέραιους
Διαβάστε περισσότεραExample of the Baum-Welch Algorithm
Example of the Baum-Welch Algorithm Larry Moss Q520, Spring 2008 1 Our corpus c We start with a very simple corpus. We take the set Y of unanalyzed words to be {ABBA, BAB}, and c to be given by c(abba)
Διαβάστε περισσότεραD Alembert s Solution to the Wave Equation
D Alembert s Solution to the Wave Equation MATH 467 Partial Differential Equations J. Robert Buchanan Department of Mathematics Fall 2018 Objectives In this lesson we will learn: a change of variable technique
Διαβάστε περισσότεραAn Inventory of Continuous Distributions
Appendi A An Inventory of Continuous Distributions A.1 Introduction The incomplete gamma function is given by Also, define Γ(α; ) = 1 with = G(α; ) = Z 0 Z 0 Z t α 1 e t dt, α > 0, >0 t α 1 e t dt, α >
Διαβάστε περισσότερα1 1 1 2 1 2 2 1 43 123 5 122 3 1 312 1 1 122 1 1 1 1 6 1 7 1 6 1 7 1 3 4 2 312 43 4 3 3 1 1 4 1 1 52 122 54 124 8 1 3 1 1 1 1 1 152 1 1 1 1 1 1 152 1 5 1 152 152 1 1 3 9 1 159 9 13 4 5 1 122 1 4 122 5
Διαβάστε περισσότεραPartial Trace and Partial Transpose
Partial Trace and Partial Transpose by José Luis Gómez-Muñoz http://homepage.cem.itesm.mx/lgomez/quantum/ jose.luis.gomez@itesm.mx This document is based on suggestions by Anirban Das Introduction This
Διαβάστε περισσότεραLecture 34 Bootstrap confidence intervals
Lecture 34 Bootstrap confidence intervals Confidence Intervals θ: an unknown parameter of interest We want to find limits θ and θ such that Gt = P nˆθ θ t If G 1 1 α is known, then P θ θ = P θ θ = 1 α
Διαβάστε περισσότεραIf we restrict the domain of y = sin x to [ π, π ], the restrict function. y = sin x, π 2 x π 2
Chapter 3. Analytic Trigonometry 3.1 The inverse sine, cosine, and tangent functions 1. Review: Inverse function (1) f 1 (f(x)) = x for every x in the domain of f and f(f 1 (x)) = x for every x in the
Διαβάστε περισσότεραECE598: Information-theoretic methods in high-dimensional statistics Spring 2016
ECE598: Information-theoretic methods in high-dimensional statistics Spring 06 Lecture 7: Information bound Lecturer: Yihong Wu Scribe: Shiyu Liang, Feb 6, 06 [Ed. Mar 9] Recall the Chi-squared divergence
Διαβάστε περισσότεραb. Use the parametrization from (a) to compute the area of S a as S a ds. Be sure to substitute for ds!
MTH U341 urface Integrals, tokes theorem, the divergence theorem To be turned in Wed., Dec. 1. 1. Let be the sphere of radius a, x 2 + y 2 + z 2 a 2. a. Use spherical coordinates (with ρ a) to parametrize.
Διαβάστε περισσότεραEstimation for ARMA Processes with Stable Noise. Matt Calder & Richard A. Davis Colorado State University
Estimation for ARMA Processes with Stable Noise Matt Calder & Richard A. Davis Colorado State University rdavis@stat.colostate.edu 1 ARMA processes with stable noise Review of M-estimation Examples of
Διαβάστε περισσότεραComputing Gradient. Hung-yi Lee 李宏毅
Computing Gradient Hung-yi Lee 李宏毅 Introduction Backpropagation: an efficient way to compute the gradient Prerequisite Backpropagation for feedforward net: http://speech.ee.ntu.edu.tw/~tkagk/courses/mlds_05_/lecture/
Διαβάστε περισσότεραCHAPTER 101 FOURIER SERIES FOR PERIODIC FUNCTIONS OF PERIOD
CHAPTER FOURIER SERIES FOR PERIODIC FUNCTIONS OF PERIOD EXERCISE 36 Page 66. Determine the Fourier series for the periodic function: f(x), when x +, when x which is periodic outside this rge of period.
Διαβάστε περισσότεραDerivation of Optical-Bloch Equations
Appendix C Derivation of Optical-Bloch Equations In this appendix the optical-bloch equations that give the populations and coherences for an idealized three-level Λ system, Fig. 3. on page 47, will be
Διαβάστε περισσότεραIf we restrict the domain of y = sin x to [ π 2, π 2
Chapter 3. Analytic Trigonometry 3.1 The inverse sine, cosine, and tangent functions 1. Review: Inverse function (1) f 1 (f(x)) = x for every x in the domain of f and f(f 1 (x)) = x for every x in the
Διαβάστε περισσότεραSrednicki Chapter 55
Srednicki Chapter 55 QFT Problems & Solutions A. George August 3, 03 Srednicki 55.. Use equations 55.3-55.0 and A i, A j ] = Π i, Π j ] = 0 (at equal times) to verify equations 55.-55.3. This is our third
Διαβάστε περισσότεραFigure A.2: MPC and MPCP Age Profiles (estimating ρ, ρ = 2, φ = 0.03)..
Supplemental Material (not for publication) Persistent vs. Permanent Income Shocks in the Buffer-Stock Model Jeppe Druedahl Thomas H. Jørgensen May, A Additional Figures and Tables Figure A.: Wealth and
Διαβάστε περισσότεραk A = [k, k]( )[a 1, a 2 ] = [ka 1,ka 2 ] 4For the division of two intervals of confidence in R +
Chapter 3. Fuzzy Arithmetic 3- Fuzzy arithmetic: ~Addition(+) and subtraction (-): Let A = [a and B = [b, b in R If x [a and y [b, b than x+y [a +b +b Symbolically,we write A(+)B = [a (+)[b, b = [a +b
Διαβάστε περισσότεραPARTIAL NOTES for 6.1 Trigonometric Identities
PARTIAL NOTES for 6.1 Trigonometric Identities tanθ = sinθ cosθ cotθ = cosθ sinθ BASIC IDENTITIES cscθ = 1 sinθ secθ = 1 cosθ cotθ = 1 tanθ PYTHAGOREAN IDENTITIES sin θ + cos θ =1 tan θ +1= sec θ 1 + cot
Διαβάστε περισσότεραThe ε-pseudospectrum of a Matrix
The ε-pseudospectrum of a Matrix Feb 16, 2015 () The ε-pseudospectrum of a Matrix Feb 16, 2015 1 / 18 1 Preliminaries 2 Definitions 3 Basic Properties 4 Computation of Pseudospectrum of 2 2 5 Problems
Διαβάστε περισσότερα1 String with massive end-points
1 String with massive end-points Πρόβλημα 5.11:Θεωρείστε μια χορδή μήκους, τάσης T, με δύο σημειακά σωματίδια στα άκρα της, το ένα μάζας m, και το άλλο μάζας m. α) Μελετώντας την κίνηση των άκρων βρείτε
Διαβάστε περισσότεραNondifferentiable Convex Functions
Nondifferentiable Convex Functions DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis http://www.cims.nyu.edu/~cfgranda/pages/obda_fall17/index.html Carlos Fernandez-Granda Applications Subgradients
Διαβάστε περισσότεραOverview. Transition Semantics. Configurations and the transition relation. Executions and computation
Overview Transition Semantics Configurations and the transition relation Executions and computation Inference rules for small-step structural operational semantics for the simple imperative language Transition
Διαβάστε περισσότερα( ) 2 and compare to M.
Problems and Solutions for Section 4.2 4.9 through 4.33) 4.9 Calculate the square root of the matrix 3!0 M!0 8 Hint: Let M / 2 a!b ; calculate M / 2!b c ) 2 and compare to M. Solution: Given: 3!0 M!0 8
Διαβάστε περισσότεραCRASH COURSE IN PRECALCULUS
CRASH COURSE IN PRECALCULUS Shiah-Sen Wang The graphs are prepared by Chien-Lun Lai Based on : Precalculus: Mathematics for Calculus by J. Stuwart, L. Redin & S. Watson, 6th edition, 01, Brooks/Cole Chapter
Διαβάστε περισσότερα