Νεςπο-Ασαυήρ Υπολογιστική Neuro-Fuzzy Computing
|
|
- Ἑρμοκράτης Κοτζιάς
- 6 χρόνια πριν
- Προβολές:
Transcript
1 Νεςπο-Ασαυήρ Υπολογιστική Neuro-Fuzzy Computing ΗΥ418 Διδάσκων Δημήτριος Τμ. ΗΜΜΥ Πανεπιστήμιο Θεσσαλίαρ Διάλεξη 10η 1
2 Numerical optimization techniques applied to Backpropagation: Conjugate Gradient (CGBP) 2
3 Conjugate Gradient Recall from your Numerical Analysis course: Steepest Descent is the simplest algorithm, but is often slow in converging Newton s method is much faster, but requires that the Hessian matrix and its inverse be calculated Conjugate Gradient it does not require the calculation of second derivatives, and yet it still has the quadratic convergence property. (It converges to the minimum of a quadratic function in a finite number of iterations We will describe how the conjugate gradient algorithm can be used to train multilayer networks We will call this algorithm Conjugate Gradient BackPropagation (CGBP)
4 Background on Conjugate Gradient 4
5 Προσέγγιση κατά Taylor μιας συνάρτησης Ανάπτυγμα της σειράς Taylor μέχρι πρώτης και δεύτερης τάξης για την πολυδιάσταση συνάρτηση f(x) γύρω από το σημείο x* (ελάχιστο) και αποτελούν γραμμική και τετραγωνική προσέγγιση της συνάρτησης, αντίστοιχα
6 Προσέγγιση κατά Taylor μιας συνάρτησης Ισοδύναμα έχουμε: όπου το ξ k βρίσκεται πάνω στο ευθύγραμμο τμήμα που ορίζεται από τα σημεία x k και x* Αυτή η συνάρτηση ονομάζεται τετραγωνική μορφή (quadratic form την ξανασυναντήσαμε στην εκπαίδευση της ADALINE) και έχει την γενική μορφή:
7 Τπενθύμιση της Steepest Descent ε κάθε βήμα, η Steepest Descent προσεγγίζει την αντικειμενική συνάρτηση κάνοντας χρήση της πρώτης παραγώγου (της πρώτης εξίσωσης της προηγούμενης διαφάνειας) όπου το λ k βρίσκεται με βελτιστοποίηση Εάν όμως η συνάρητηση είναι τετραγωνική μορφή, τότε:
8 υζυγείς διευθύνσεις ΟΡΙΜΟ. Μια μέθοδος έχει την ιδιότητα του τετραγωνικού τερματισμού όταν συγκλίνει στο βέλτιστο σημείο x* μιας τετραγωνικής αντικειμενικής συνάρτησης σε γνωστό πεπερασμένο αριθμό επαναλήψεων ΟΡΙΜΟ. Ένα σύνολο n γραμμικώς ανεξαρτήτων μη μηδενικών διανυσμάτων s 1, s 2,, s n είναι συζυγή ως προς έναν θετικά ορισμένο πίνακα H εάν s i Hs j =0 1 i j n ΟΡΙΜΟ. Μια μέθοδος βελτιστοποίησης ονομάζεται μέθοδος συζυγών διευθύνσεων εάν κατά την εφαρμογή της παράγει συζυγείς διευθύνσεις και εφαρμοζόμενη σε μια τετραγωνική μορφή με Hessian πίνακα H έχει την ιδιότητα του τετραγωνικού τερματισμού
9 υζυγείς διευθύνσεις ΘΕΩΡΗΜΑ. ε κάθε αντικειμενική συνάρτηση που είναι τετραγωνική μορφή και έχει ένα ελάχιστο, εάν ακολουθήσουμε μια μέθοδο που παράγει συζυγείς ως προς τον Hessian πίνακα διευθύνσεις, το ελάχιστο θα εντοπιστεί σε n το πολύ βήματα, ένα για κάθε συζυγή διεύθυνση ΘΕΩΡΗΜΑ. Εάν f(x) είναι τετραγωνική μορφή, και s 0, s 1,, s k σύνολο συζυγών ως προς τον Hessian πίνακα διευθύνσεων, οι οποίες παράγονται με κάποια μέθοδο, μέχρι την προσέγγιση x k+1, τότε: T f(x k+1 )s j = 0 για j=1,2, k
10 Παράδειγμα ΠΡΟΒΛΗΜΑ. Να εντοπιστεί το ελάχιστο της f(x)=x 2 1+5x ΛΤΗ. Η κλίση είναι f(x)= (2x 1, 10x 2 ) Σ Ο Hessian είναι: Ας εκκινήσουμε από το x 0 =(2 2) T Σότε, f(x 0 )= (4, 20) Σ Επιλέγοντας τυχαία μια διεύθυνση εκκίνησης, έστω την s 0 =(1/2, 3/2)T που έχει s 0 =1, παίρνω ότι λ 0 = Δεν είναι θετικό!!! γιατί η s 0 δεν είναι διεύθυνση μείωσης x 1 =x 0 +λ 0 s 0 =...=
11 Παράδειγμα Η κλίση εδώ είναι f(x 1 )= ( , ) Σ Θέλουμε τώρα να υπολογίσουμε την επόμενη διεύθυνση ελαχιστοποίησης με τέτοιον τρόπο ώστε να είναι συζυγής ως προς την s 0 σε σχέση με τον Hessian H και να είναι μοναδιαία, δηλαδή να ισχύουν: s T 1Hs 0 =0 και s T 1s 1 =1 Εάν συμβολίσουμε s 1 = (s 1 1,s 1 2) προκύπτει από τις προηγούμενες εξισώσεις ότι: Από τις δυο λύσεις του συστήματος αυτού, επιλέγουμε εκείνη που δίνει διεύθυνση μείωσης, δηλαδή να ισχύει: s T 1 f(x 1 ) < 0
12 Παράδειγμα Αυτή είναι η s 1 = ( , ) Σ Αυτή είναι η επόμενη διεύθυνση, και είναι μοναδιαία και συζυγή της προηγούμενης, κατά μήκος της οποίας θα κινηθούμε με βήμα λ 1 Από τον τύπο της Διαφάνειας-5, έχω ότι λ 1 = Παρτηρήστε ότι το λ 1 είναι θετικό όπως θα έπρεπε αφού κινούμαστε σε διεύθυνση μείωσης Η επόμενη προσέγγιση είναι: x 2 = x 1 +λ 1 s 1 = (0 0) T Αυτό είναι το βέλτιστο και βρέθηκε σε 2 βήματα (είναι τεταγωνική η αντικειμενική συνάρτηση με διάσταση 2)
13 Μέθοδοι συζυγών κλίσεων Προτάθηκε αρχικά από τους Hestenes & Stiefel (1952) και Beckman (1960) Γνωστότερη παραλλαγή αυτή των Fletcher-Reeves (1964) Παράγει στο βήμα k την διεύθυνση s k η οποία είναι γραμμικός συνδυασμός της - f(x k ) δηλαδή της διεύθυνσης της μέγιστης αλλαγής στην τρέχουσα προσέγγιση, και των προηγούμενων διευθύνσεων s 0, s 1,, s k-1 Οι συντελεστές του γραμμικού συνδυασμού επιλέγονται ώστε οι παραγόμενες διευθύνσεις από προσέγγιση σε προσέγγιση να είναι συζυγείς ως προς τον Hessian πίνακα της αντικειμενικής συνάρτησης ΣΕΛΙΚΑ προκύπτει ότι για να υπολογίσουμε αυτούς τους συντελεστές χρειαζόμαστε μόνο την τρέχουσα κλίση f(x k ) και την ακριβώς προηγούμενη f(x k-1 )
14 Fletcher-Reeves conjugate gradient
15 Fletcher-Reeves conjugate gradient
16 Παραλλαγές μεθόδων συζυγών κλίσεων Ακολουθούν τον προηγούμενο αλγόριθμο, αλλά διαφέρουν στον τύπο υπολογισμού του ω k Μέθοδος του Daniel (1967) Μέθοδος των Crowder & Wolfe (1971) Μέθοδος των Polak & Ribiere (1969), Polyak (1969)
17 Drawbacks of basic Conjugate Gradient This conjugate gradient algorithm cannot be applied directly to the neural network training task, because the performance index is not quadratic. This affects the algorithm in two ways: First, we can not minimize the function along a line, as required in Step 2 Second, the exact minimum will not normally be reached in a finite number of steps, and therefore the algorithm will need to be reset after some set number of iterations
18 Interval location Let s address the linear search first We need to have a general procedure for locating the minimum of a function in a specified direction This will involve two steps: interval location interval reduction The purpose of the interval location step is to find some initial interval that contains a local minimum The interval reduction step then reduces the size of the initial interval until the minimum is located to the desired accuracy
19 Interval location We will use a function comparison method to perform the interval location step, illustrated in the following figure We begin by evaluating the performance index at an initial point, represented by α 1. This point corresponds to the current values of the network weights and biases. In other words, we are evaluating F(x 0 ) The next step is to evaluate the function at a second point, represented by b 1, which is a distance ε from the initial point, along the first search direction p 0. In other words, we are evaluating F(x 0 +εp 0 ) This process stops when the function increases between two consecutive evaluations. The minimum is bracketed by the two points α 5 and b 5. The minimum may occur either in the interval [α 3,b 3 ] or in the interval [α 4,b 4 ].
20 Interval reduction The next step is interval reduction, which involves evaluating the function at points inside the interval [a 5,b 5 ] From the figure below we can see that we will need to evaluate the function at two internal points (at least) in order to reduce the size of the interval of uncertainty The let figure shows that one internal function evaluation does not provide us with any information on the location of the minimum However, if we evaluate the function at two points c and d, as in the right figure, we can reduce the interval of uncertainty If F(c) > F(d), then the minimum must occur in the interval [c, b] If F(c) < F(d), then the minimum must occur in the interval [a, d] We are assuming that there is a single minimum located in the initial interval.
21 Interval reduction: Golden section search We need to determine the locations of points c and d We will use Golden Section search, which is designed to reduce the number of function evaluations required At each iteration one new function evaluation is required τ is user-set tolerance
22 Final adjustments to develop the CGBP There is one more modification to the conjugate gradient algorithm that needs to be made before we apply it to neural network training For quadratic functions the algorithm will converge to the minimum in at most n iterations, where n is the number of parameters being optimized The mean squared error performance index for multilayer networks is not quadratic, therefore the algorithm would not normally converge in n iterations The development of the conjugate gradient algorithm does not indicate what search direction to use once a cycle of n iterations has been completed There have been many procedures suggested, but the simplest method is to reset the search direction to the steepest descent direction (negative of the gradient) after n iterations
23 Critique of CGBP CGBP algorithm converges in many fewer iterations than other algorithms This is a little deceiving, since each iteration of CGBP requires more computations than these other methods there are many function evaluations involved in each iteration of CGBP Even so, CGBP has been shown to be one of the fastest batch training algorithms for multilayer networks C. Charalambous, Conjugate gradient algorithm for efficient training of artificial neural networks, IEE Proceedings, vol. 139, no. 3, pp , 1992
24 Numerical optimization techniques applied to Backpropagation: Levenberg-Marquardt algorithm 24
25 Levenberg-Marquardt basic algorithm The Levenberg-Marquardt algorithm is a variation of Newton s method that was designed for minimizing functions that are sums of squares of other nonlinear functions This is very well suited to neural network training (recall that their performance index is the mean squared error) Newton s method for optimizing a performance index F(x) is as follows: where and
26 Levenberg-Marquardt basic algorithm If we assume that F(x) is a sum of squares function then the j-th element of the gradient would be The gradient can therefore be written in matrix form where the Jacobian matrix is:
27 Levenberg-Marquardt basic algorithm Next we wish to find Hessian matrix. The k,j element would be: The Hessian can be expressed in matrix form: where If we assume that S(x) is small, the we can approximate
28 Levenberg-Marquardt basic algorithm So, the Newton method evolves into the Gauss-Newton method: One problem with the Gauss-Newton method is that the matrix H=J T J may not be invertible This can be overcome by using the following modification to the approximate Hessian matrix: G=H+μI (Figure out why this matrix can be made invertible)
29 Levenberg-Marquardt basic algorithm This lead to the Levenberg-Marquardt algorithm: Or The algorithm begins with μ k set to some small value (e.g., 0.01) If a step does not yield a smaller value for F(x), then the step is repeated with μ k multiplied by some factor θ>1 (e.g., θ=10) Eventually F(x) should decrease, since we would be taking a small step in the direction of steepest descent If a step does produce a smaller value for F(x), then μ k is divided by θ for the next step, so that the algorithm will approach Gauss-Newton, which should provide faster convergence The algorithm provides a nice compromise between the speed of Newton s method and the guaranteed convergence of steepest descent
30 Training with Levenberg-Marquardt The performance index for multilayer network training is the mean squared error If each target occurs with equal probability, the mean squared error is proportional to the sum of squared errors over the Q targets in the training set: This is equivalent to the performance index for which Levenberg-Marquardt was designed It should be a straightforward matter to adapt the algorithm for network training, but it turns out that it does require some care in working out the details
31 Training with Levenberg-Marquardt The key step in the Levenberg-Marquardt algorithm is the computation of the Jacobian matrix To perform this computation we will use a variation of the backpropagation algorithm To create the Jacobian matrix we need to compute the derivatives of the errors, instead of the derivatives of the squared errors (as we did for the standard backpropagation)
32 Jacobian calculation Recall Jacobian s form from (slide 14) Error vector: Parameter vector: N=Q x S M and n=s 1 (R+1)+S 2 (S 1 + 1)+ +S M (S M-1 + 1) So:
33 Jacobian calculation The terms in this Jacobian matrix can be computed by a simple modification to the backpropagation algorithm Standard backpropagation calculates terms like: For the elements of the Jacobian matrix that are needed for the Levenberg-Marquardt algorithm we need to calculate terms like: In our derivation of standard backpropagation: where
34 Marquardt sensitivity The backpropagation process computed the sensitivities through a recurrence relationship from the last layer backward to the first layer We can use the same concept to compute the terms needed for the Jacobian matrix if we define Marquardt sensitivity: where h=(q-1)s M + k We can compute the elements of the Jacobian or if x l is a bias
35 Marquardt sensitivity Marquardt sensitivities can be computed through the same recurrence relations as the standard sensitivities with one modification at the final layer For the Marquardt sensitivities at the final layer we have: Therefore when the input p q has been applied to the network and the corresponding network a M q output has been computed, the Levenberg-Marquardt backpropagation is initialized with: where F M (n M )is define as in standard backpropagation The columns can also be backpropagated together using
36 Marquardt sensitivity The total Marquardt sensitivity matrices for each layer are then created by augmenting the matrices computed for each input: Note that for each input that is presented to the network we will backpropagate S M sensitivity vectors This is because we are computing the derivatives of each individual error, rather than the derivative of the sum of squares of the errors. For every input applied to the network there will be S M errors (one for each element of the network output) For each error there will be one row of the Jacobian matrix After the sensitivities have been backpropagated, the Jacobian matrix is computed using equation of slide 22
37 Levenberg-Marquardt backpropagation 1. Present all inputs to the network and compute the corresponding outputs and the errors e q =t q -a M q. Compute the sum of squared errors over all inputs, F(x) (slide 18) 2. Compute the Jacobian matrix. Calculate the sensitivities with the recurrence relations (equations in slide 23). Augment the individual matrices into the Marquardt sensitivities. Compute the elements of the Jacobian matrix with equations in slide Solve equation in slide 17 to obtain Δx k 4. Recompute the sum of squared errors using x k +Δx k. If this new sum of squares is smaller than that computed in Step-1, then divide μ by θ, let x k+1 =x k +Δx k and go back to Step-1. If the sum of squares is not reduced, then multiply μ by θ and go back to Step-3
38 Critique of LMBP Even given the large number of computations, however, the LMBP algorithm appears to be the fastest neural network training algorithm for moderate numbers of network parameters M. T. Hagan and M. Menhaj, Training feedforward networks with the Marquardt algorithm, IEEE Transactions on Neural Networks, vol. 5, no. 6, The key drawback of the LMBP algorithm is the storage requirement: The algorithm must store the approximate Hessian matrix J T J This is an nxn matrix, where n is the number of parameters (weights and biases) Recall that the other methods discussed need only store the gradient, which is an n-dimensional vector When the number of parameters is very large, it may be impractical to use the Levenberg-Marquardt algorithm
2 Composition. Invertible Mappings
Arkansas Tech University MATH 4033: Elementary Modern Algebra Dr. Marcel B. Finan Composition. Invertible Mappings In this section we discuss two procedures for creating new mappings from old ones, namely,
ΚΥΠΡΙΑΚΗ ΕΤΑΙΡΕΙΑ ΠΛΗΡΟΦΟΡΙΚΗΣ CYPRUS COMPUTER SOCIETY ΠΑΓΚΥΠΡΙΟΣ ΜΑΘΗΤΙΚΟΣ ΔΙΑΓΩΝΙΣΜΟΣ ΠΛΗΡΟΦΟΡΙΚΗΣ 19/5/2007
Οδηγίες: Να απαντηθούν όλες οι ερωτήσεις. Αν κάπου κάνετε κάποιες υποθέσεις να αναφερθούν στη σχετική ερώτηση. Όλα τα αρχεία που αναφέρονται στα προβλήματα βρίσκονται στον ίδιο φάκελο με το εκτελέσιμο
CHAPTER 25 SOLVING EQUATIONS BY ITERATIVE METHODS
CHAPTER 5 SOLVING EQUATIONS BY ITERATIVE METHODS EXERCISE 104 Page 8 1. Find the positive root of the equation x + 3x 5 = 0, correct to 3 significant figures, using the method of bisection. Let f(x) =
Section 8.3 Trigonometric Equations
99 Section 8. Trigonometric Equations Objective 1: Solve Equations Involving One Trigonometric Function. In this section and the next, we will exple how to solving equations involving trigonometric functions.
HOMEWORK 4 = G. In order to plot the stress versus the stretch we define a normalized stretch:
HOMEWORK 4 Problem a For the fast loading case, we want to derive the relationship between P zz and λ z. We know that the nominal stress is expressed as: P zz = ψ λ z where λ z = λ λ z. Therefore, applying
3.4 SUM AND DIFFERENCE FORMULAS. NOTE: cos(α+β) cos α + cos β cos(α-β) cos α -cos β
3.4 SUM AND DIFFERENCE FORMULAS Page Theorem cos(αβ cos α cos β -sin α cos(α-β cos α cos β sin α NOTE: cos(αβ cos α cos β cos(α-β cos α -cos β Proof of cos(α-β cos α cos β sin α Let s use a unit circle
Homework 3 Solutions
Homework 3 Solutions Igor Yanovsky (Math 151A TA) Problem 1: Compute the absolute error and relative error in approximations of p by p. (Use calculator!) a) p π, p 22/7; b) p π, p 3.141. Solution: For
Numerical Analysis FMN011
Numerical Analysis FMN011 Carmen Arévalo Lund University carmen@maths.lth.se Lecture 12 Periodic data A function g has period P if g(x + P ) = g(x) Model: Trigonometric polynomial of order M T M (x) =
6.3 Forecasting ARMA processes
122 CHAPTER 6. ARMA MODELS 6.3 Forecasting ARMA processes The purpose of forecasting is to predict future values of a TS based on the data collected to the present. In this section we will discuss a linear
Phys460.nb Solution for the t-dependent Schrodinger s equation How did we find the solution? (not required)
Phys460.nb 81 ψ n (t) is still the (same) eigenstate of H But for tdependent H. The answer is NO. 5.5.5. Solution for the tdependent Schrodinger s equation If we assume that at time t 0, the electron starts
Math221: HW# 1 solutions
Math: HW# solutions Andy Royston October, 5 7.5.7, 3 rd Ed. We have a n = b n = a = fxdx = xdx =, x cos nxdx = x sin nx n sin nxdx n = cos nx n = n n, x sin nxdx = x cos nx n + cos nxdx n cos n = + sin
Approximation of distance between locations on earth given by latitude and longitude
Approximation of distance between locations on earth given by latitude and longitude Jan Behrens 2012-12-31 In this paper we shall provide a method to approximate distances between two points on earth
TMA4115 Matematikk 3
TMA4115 Matematikk 3 Andrew Stacey Norges Teknisk-Naturvitenskapelige Universitet Trondheim Spring 2010 Lecture 12: Mathematics Marvellous Matrices Andrew Stacey Norges Teknisk-Naturvitenskapelige Universitet
Areas and Lengths in Polar Coordinates
Kiryl Tsishchanka Areas and Lengths in Polar Coordinates In this section we develop the formula for the area of a region whose boundary is given by a polar equation. We need to use the formula for the
EE512: Error Control Coding
EE512: Error Control Coding Solution for Assignment on Finite Fields February 16, 2007 1. (a) Addition and Multiplication tables for GF (5) and GF (7) are shown in Tables 1 and 2. + 0 1 2 3 4 0 0 1 2 3
Concrete Mathematics Exercises from 30 September 2016
Concrete Mathematics Exercises from 30 September 2016 Silvio Capobianco Exercise 1.7 Let H(n) = J(n + 1) J(n). Equation (1.8) tells us that H(2n) = 2, and H(2n+1) = J(2n+2) J(2n+1) = (2J(n+1) 1) (2J(n)+1)
derivation of the Laplacian from rectangular to spherical coordinates
derivation of the Laplacian from rectangular to spherical coordinates swapnizzle 03-03- :5:43 We begin by recognizing the familiar conversion from rectangular to spherical coordinates (note that φ is used
Fourier Series. MATH 211, Calculus II. J. Robert Buchanan. Spring Department of Mathematics
Fourier Series MATH 211, Calculus II J. Robert Buchanan Department of Mathematics Spring 2018 Introduction Not all functions can be represented by Taylor series. f (k) (c) A Taylor series f (x) = (x c)
The Simply Typed Lambda Calculus
Type Inference Instead of writing type annotations, can we use an algorithm to infer what the type annotations should be? That depends on the type system. For simple type systems the answer is yes, and
Section 7.6 Double and Half Angle Formulas
09 Section 7. Double and Half Angle Fmulas To derive the double-angles fmulas, we will use the sum of two angles fmulas that we developed in the last section. We will let α θ and β θ: cos(θ) cos(θ + θ)
ΚΥΠΡΙΑΚΗ ΕΤΑΙΡΕΙΑ ΠΛΗΡΟΦΟΡΙΚΗΣ CYPRUS COMPUTER SOCIETY ΠΑΓΚΥΠΡΙΟΣ ΜΑΘΗΤΙΚΟΣ ΔΙΑΓΩΝΙΣΜΟΣ ΠΛΗΡΟΦΟΡΙΚΗΣ 6/5/2006
Οδηγίες: Να απαντηθούν όλες οι ερωτήσεις. Ολοι οι αριθμοί που αναφέρονται σε όλα τα ερωτήματα είναι μικρότεροι το 1000 εκτός αν ορίζεται διαφορετικά στη διατύπωση του προβλήματος. Διάρκεια: 3,5 ώρες Καλή
Areas and Lengths in Polar Coordinates
Kiryl Tsishchanka Areas and Lengths in Polar Coordinates In this section we develop the formula for the area of a region whose boundary is given by a polar equation. We need to use the formula for the
Potential Dividers. 46 minutes. 46 marks. Page 1 of 11
Potential Dividers 46 minutes 46 marks Page 1 of 11 Q1. In the circuit shown in the figure below, the battery, of negligible internal resistance, has an emf of 30 V. The pd across the lamp is 6.0 V and
Structural and Multidisciplinary Optimization. P. Duysinx, P. Tossings*
Structural and Multidisciplinary Optimization P. Duysinx, P. Tossings* 2017-2018 * CONTACT Patricia TOSSINGS Institut de Mathematique B37), 0/57 Telephone: 04/366.93.73. Email: Patricia.Tossings@ulg.ac.be
Partial Differential Equations in Biology The boundary element method. March 26, 2013
The boundary element method March 26, 203 Introduction and notation The problem: u = f in D R d u = ϕ in Γ D u n = g on Γ N, where D = Γ D Γ N, Γ D Γ N = (possibly, Γ D = [Neumann problem] or Γ N = [Dirichlet
Econ 2110: Fall 2008 Suggested Solutions to Problem Set 8 questions or comments to Dan Fetter 1
Eon : Fall 8 Suggested Solutions to Problem Set 8 Email questions or omments to Dan Fetter Problem. Let X be a salar with density f(x, θ) (θx + θ) [ x ] with θ. (a) Find the most powerful level α test
C.S. 430 Assignment 6, Sample Solutions
C.S. 430 Assignment 6, Sample Solutions Paul Liu November 15, 2007 Note that these are sample solutions only; in many cases there were many acceptable answers. 1 Reynolds Problem 10.1 1.1 Normal-order
Example Sheet 3 Solutions
Example Sheet 3 Solutions. i Regular Sturm-Liouville. ii Singular Sturm-Liouville mixed boundary conditions. iii Not Sturm-Liouville ODE is not in Sturm-Liouville form. iv Regular Sturm-Liouville note
Srednicki Chapter 55
Srednicki Chapter 55 QFT Problems & Solutions A. George August 3, 03 Srednicki 55.. Use equations 55.3-55.0 and A i, A j ] = Π i, Π j ] = 0 (at equal times) to verify equations 55.-55.3. This is our third
Problem Set 3: Solutions
CMPSCI 69GG Applied Information Theory Fall 006 Problem Set 3: Solutions. [Cover and Thomas 7.] a Define the following notation, C I p xx; Y max X; Y C I p xx; Ỹ max I X; Ỹ We would like to show that C
Other Test Constructions: Likelihood Ratio & Bayes Tests
Other Test Constructions: Likelihood Ratio & Bayes Tests Side-Note: So far we have seen a few approaches for creating tests such as Neyman-Pearson Lemma ( most powerful tests of H 0 : θ = θ 0 vs H 1 :
Block Ciphers Modes. Ramki Thurimella
Block Ciphers Modes Ramki Thurimella Only Encryption I.e. messages could be modified Should not assume that nonsensical messages do no harm Always must be combined with authentication 2 Padding Must be
DESIGN OF MACHINERY SOLUTION MANUAL h in h 4 0.
DESIGN OF MACHINERY SOLUTION MANUAL -7-1! PROBLEM -7 Statement: Design a double-dwell cam to move a follower from to 25 6, dwell for 12, fall 25 and dwell for the remader The total cycle must take 4 sec
SCHOOL OF MATHEMATICAL SCIENCES G11LMA Linear Mathematics Examination Solutions
SCHOOL OF MATHEMATICAL SCIENCES GLMA Linear Mathematics 00- Examination Solutions. (a) i. ( + 5i)( i) = (6 + 5) + (5 )i = + i. Real part is, imaginary part is. (b) ii. + 5i i ( + 5i)( + i) = ( i)( + i)
PARTIAL NOTES for 6.1 Trigonometric Identities
PARTIAL NOTES for 6.1 Trigonometric Identities tanθ = sinθ cosθ cotθ = cosθ sinθ BASIC IDENTITIES cscθ = 1 sinθ secθ = 1 cosθ cotθ = 1 tanθ PYTHAGOREAN IDENTITIES sin θ + cos θ =1 tan θ +1= sec θ 1 + cot
Math 6 SL Probability Distributions Practice Test Mark Scheme
Math 6 SL Probability Distributions Practice Test Mark Scheme. (a) Note: Award A for vertical line to right of mean, A for shading to right of their vertical line. AA N (b) evidence of recognizing symmetry
( ) 2 and compare to M.
Problems and Solutions for Section 4.2 4.9 through 4.33) 4.9 Calculate the square root of the matrix 3!0 M!0 8 Hint: Let M / 2 a!b ; calculate M / 2!b c ) 2 and compare to M. Solution: Given: 3!0 M!0 8
Instruction Execution Times
1 C Execution Times InThisAppendix... Introduction DL330 Execution Times DL330P Execution Times DL340 Execution Times C-2 Execution Times Introduction Data Registers This appendix contains several tables
Finite Field Problems: Solutions
Finite Field Problems: Solutions 1. Let f = x 2 +1 Z 11 [x] and let F = Z 11 [x]/(f), a field. Let Solution: F =11 2 = 121, so F = 121 1 = 120. The possible orders are the divisors of 120. Solution: The
CHAPTER 48 APPLICATIONS OF MATRICES AND DETERMINANTS
CHAPTER 48 APPLICATIONS OF MATRICES AND DETERMINANTS EXERCISE 01 Page 545 1. Use matrices to solve: 3x + 4y x + 5y + 7 3x + 4y x + 5y 7 Hence, 3 4 x 0 5 y 7 The inverse of 3 4 5 is: 1 5 4 1 5 4 15 8 3
6.1. Dirac Equation. Hamiltonian. Dirac Eq.
6.1. Dirac Equation Ref: M.Kaku, Quantum Field Theory, Oxford Univ Press (1993) η μν = η μν = diag(1, -1, -1, -1) p 0 = p 0 p = p i = -p i p μ p μ = p 0 p 0 + p i p i = E c 2 - p 2 = (m c) 2 H = c p 2
Mean bond enthalpy Standard enthalpy of formation Bond N H N N N N H O O O
Q1. (a) Explain the meaning of the terms mean bond enthalpy and standard enthalpy of formation. Mean bond enthalpy... Standard enthalpy of formation... (5) (b) Some mean bond enthalpies are given below.
Matrices and Determinants
Matrices and Determinants SUBJECTIVE PROBLEMS: Q 1. For what value of k do the following system of equations possess a non-trivial (i.e., not all zero) solution over the set of rationals Q? x + ky + 3z
Physical DB Design. B-Trees Index files can become quite large for large main files Indices on index files are possible.
B-Trees Index files can become quite large for large main files Indices on index files are possible 3 rd -level index 2 nd -level index 1 st -level index Main file 1 The 1 st -level index consists of pairs
Tridiagonal matrices. Gérard MEURANT. October, 2008
Tridiagonal matrices Gérard MEURANT October, 2008 1 Similarity 2 Cholesy factorizations 3 Eigenvalues 4 Inverse Similarity Let α 1 ω 1 β 1 α 2 ω 2 T =......... β 2 α 1 ω 1 β 1 α and β i ω i, i = 1,...,
Lecture 2: Dirac notation and a review of linear algebra Read Sakurai chapter 1, Baym chatper 3
Lecture 2: Dirac notation and a review of linear algebra Read Sakurai chapter 1, Baym chatper 3 1 State vector space and the dual space Space of wavefunctions The space of wavefunctions is the set of all
Chapter 6: Systems of Linear Differential. be continuous functions on the interval
Chapter 6: Systems of Linear Differential Equations Let a (t), a 2 (t),..., a nn (t), b (t), b 2 (t),..., b n (t) be continuous functions on the interval I. The system of n first-order differential equations
Πρόβλημα 1: Αναζήτηση Ελάχιστης/Μέγιστης Τιμής
Πρόβλημα 1: Αναζήτηση Ελάχιστης/Μέγιστης Τιμής Να γραφεί πρόγραμμα το οποίο δέχεται ως είσοδο μια ακολουθία S από n (n 40) ακέραιους αριθμούς και επιστρέφει ως έξοδο δύο ακολουθίες από θετικούς ακέραιους
Exercises 10. Find a fundamental matrix of the given system of equations. Also find the fundamental matrix Φ(t) satisfying Φ(0) = I. 1.
Exercises 0 More exercises are available in Elementary Differential Equations. If you have a problem to solve any of them, feel free to come to office hour. Problem Find a fundamental matrix of the given
ΜΕΘΟΔΟΙ ΑΕΡΟΔΥΝΑΜΙΚΗΣ
ΕΘΝΙΚΟ ΜΕΤΣΟΒΙΟ ΠΟΛΥΤΕΧΝΕΙΟ Εργαστήριο Θερμικών Στροβιλομηχανών Μονάδα Παράλληλης ης Υπολογιστικής Ρευστοδυναμικής & Βελτιστοποίησης ΜΕΘΟΔΟΙ ΑΕΡΟΔΥΝΑΜΙΚΗΣ ΒΕΛΤΙΣΤΟΠΟΙΗΣΗΣ (7 ο Εξάμηνο Σχολής Μηχ.Μηχ. ΕΜΠ)
ES440/ES911: CFD. Chapter 5. Solution of Linear Equation Systems
ES440/ES911: CFD Chapter 5. Solution of Linear Equation Systems Dr Yongmann M. Chung http://www.eng.warwick.ac.uk/staff/ymc/es440.html Y.M.Chung@warwick.ac.uk School of Engineering & Centre for Scientific
forms This gives Remark 1. How to remember the above formulas: Substituting these into the equation we obtain with
Week 03: C lassification of S econd- Order L inear Equations In last week s lectures we have illustrated how to obtain the general solutions of first order PDEs using the method of characteristics. We
the total number of electrons passing through the lamp.
1. A 12 V 36 W lamp is lit to normal brightness using a 12 V car battery of negligible internal resistance. The lamp is switched on for one hour (3600 s). For the time of 1 hour, calculate (i) the energy
Solution Series 9. i=1 x i and i=1 x i.
Lecturer: Prof. Dr. Mete SONER Coordinator: Yilin WANG Solution Series 9 Q1. Let α, β >, the p.d.f. of a beta distribution with parameters α and β is { Γ(α+β) Γ(α)Γ(β) f(x α, β) xα 1 (1 x) β 1 for < x
[1] P Q. Fig. 3.1
1 (a) Define resistance....... [1] (b) The smallest conductor within a computer processing chip can be represented as a rectangular block that is one atom high, four atoms wide and twenty atoms long. One
Εργαστήριο Ανάπτυξης Εφαρμογών Βάσεων Δεδομένων. Εξάμηνο 7 ο
Εργαστήριο Ανάπτυξης Εφαρμογών Βάσεων Δεδομένων Εξάμηνο 7 ο Procedures and Functions Stored procedures and functions are named blocks of code that enable you to group and organize a series of SQL and PL/SQL
Απόκριση σε Μοναδιαία Ωστική Δύναμη (Unit Impulse) Απόκριση σε Δυνάμεις Αυθαίρετα Μεταβαλλόμενες με το Χρόνο. Απόστολος Σ.
Απόκριση σε Δυνάμεις Αυθαίρετα Μεταβαλλόμενες με το Χρόνο The time integral of a force is referred to as impulse, is determined by and is obtained from: Newton s 2 nd Law of motion states that the action
Second Order Partial Differential Equations
Chapter 7 Second Order Partial Differential Equations 7.1 Introduction A second order linear PDE in two independent variables (x, y Ω can be written as A(x, y u x + B(x, y u xy + C(x, y u u u + D(x, y
The challenges of non-stable predicates
The challenges of non-stable predicates Consider a non-stable predicate Φ encoding, say, a safety property. We want to determine whether Φ holds for our program. The challenges of non-stable predicates
ΚΥΠΡΙΑΚΗ ΕΤΑΙΡΕΙΑ ΠΛΗΡΟΦΟΡΙΚΗΣ CYPRUS COMPUTER SOCIETY ΠΑΓΚΥΠΡΙΟΣ ΜΑΘΗΤΙΚΟΣ ΔΙΑΓΩΝΙΣΜΟΣ ΠΛΗΡΟΦΟΡΙΚΗΣ 24/3/2007
Οδηγίες: Να απαντηθούν όλες οι ερωτήσεις. Όλοι οι αριθμοί που αναφέρονται σε όλα τα ερωτήματα μικρότεροι του 10000 εκτός αν ορίζεται διαφορετικά στη διατύπωση του προβλήματος. Αν κάπου κάνετε κάποιες υποθέσεις
4.6 Autoregressive Moving Average Model ARMA(1,1)
84 CHAPTER 4. STATIONARY TS MODELS 4.6 Autoregressive Moving Average Model ARMA(,) This section is an introduction to a wide class of models ARMA(p,q) which we will consider in more detail later in this
Second Order RLC Filters
ECEN 60 Circuits/Electronics Spring 007-0-07 P. Mathys Second Order RLC Filters RLC Lowpass Filter A passive RLC lowpass filter (LPF) circuit is shown in the following schematic. R L C v O (t) Using phasor
Modbus basic setup notes for IO-Link AL1xxx Master Block
n Modbus has four tables/registers where data is stored along with their associated addresses. We will be using the holding registers from address 40001 to 49999 that are R/W 16 bit/word. Two tables that
( y) Partial Differential Equations
Partial Dierential Equations Linear P.D.Es. contains no owers roducts o the deendent variables / an o its derivatives can occasionall be solved. Consider eamle ( ) a (sometimes written as a ) we can integrate
w o = R 1 p. (1) R = p =. = 1
Πανεπιστήµιο Κρήτης - Τµήµα Επιστήµης Υπολογιστών ΗΥ-570: Στατιστική Επεξεργασία Σήµατος 205 ιδάσκων : Α. Μουχτάρης Τριτη Σειρά Ασκήσεων Λύσεις Ασκηση 3. 5.2 (a) From the Wiener-Hopf equation we have:
Solutions to Exercise Sheet 5
Solutions to Eercise Sheet 5 jacques@ucsd.edu. Let X and Y be random variables with joint pdf f(, y) = 3y( + y) where and y. Determine each of the following probabilities. Solutions. a. P (X ). b. P (X
Homework 8 Model Solution Section
MATH 004 Homework Solution Homework 8 Model Solution Section 14.5 14.6. 14.5. Use the Chain Rule to find dz where z cosx + 4y), x 5t 4, y 1 t. dz dx + dy y sinx + 4y)0t + 4) sinx + 4y) 1t ) 0t + 4t ) sinx
Practice Exam 2. Conceptual Questions. 1. State a Basic identity and then verify it. (a) Identity: Solution: One identity is csc(θ) = 1
Conceptual Questions. State a Basic identity and then verify it. a) Identity: Solution: One identity is cscθ) = sinθ) Practice Exam b) Verification: Solution: Given the point of intersection x, y) of the
Chapter 6: Systems of Linear Differential. be continuous functions on the interval
Chapter 6: Systems of Linear Differential Equations Let a (t), a 2 (t),..., a nn (t), b (t), b 2 (t),..., b n (t) be continuous functions on the interval I. The system of n first-order differential equations
9.09. # 1. Area inside the oval limaçon r = cos θ. To graph, start with θ = 0 so r = 6. Compute dr
9.9 #. Area inside the oval limaçon r = + cos. To graph, start with = so r =. Compute d = sin. Interesting points are where d vanishes, or at =,,, etc. For these values of we compute r:,,, and the values
Correction Table for an Alcoholometer Calibrated at 20 o C
An alcoholometer is a device that measures the concentration of ethanol in a water-ethanol mixture (often in units of %abv percent alcohol by volume). The depth to which an alcoholometer sinks in a water-ethanol
Reminders: linear functions
Reminders: linear functions Let U and V be vector spaces over the same field F. Definition A function f : U V is linear if for every u 1, u 2 U, f (u 1 + u 2 ) = f (u 1 ) + f (u 2 ), and for every u U
2. THEORY OF EQUATIONS. PREVIOUS EAMCET Bits.
EAMCET-. THEORY OF EQUATIONS PREVIOUS EAMCET Bits. Each of the roots of the equation x 6x + 6x 5= are increased by k so that the new transformed equation does not contain term. Then k =... - 4. - Sol.
Section 9.2 Polar Equations and Graphs
180 Section 9. Polar Equations and Graphs In this section, we will be graphing polar equations on a polar grid. In the first few examples, we will write the polar equation in rectangular form to help identify
Ordinal Arithmetic: Addition, Multiplication, Exponentiation and Limit
Ordinal Arithmetic: Addition, Multiplication, Exponentiation and Limit Ting Zhang Stanford May 11, 2001 Stanford, 5/11/2001 1 Outline Ordinal Classification Ordinal Addition Ordinal Multiplication Ordinal
Jesse Maassen and Mark Lundstrom Purdue University November 25, 2013
Notes on Average Scattering imes and Hall Factors Jesse Maassen and Mar Lundstrom Purdue University November 5, 13 I. Introduction 1 II. Solution of the BE 1 III. Exercises: Woring out average scattering
ANSWERSHEET (TOPIC = DIFFERENTIAL CALCULUS) COLLECTION #2. h 0 h h 0 h h 0 ( ) g k = g 0 + g 1 + g g 2009 =?
Teko Classes IITJEE/AIEEE Maths by SUHAAG SIR, Bhopal, Ph (0755) 3 00 000 www.tekoclasses.com ANSWERSHEET (TOPIC DIFFERENTIAL CALCULUS) COLLECTION # Question Type A.Single Correct Type Q. (A) Sol least
Συστήματα Διαχείρισης Βάσεων Δεδομένων
ΕΛΛΗΝΙΚΗ ΔΗΜΟΚΡΑΤΙΑ ΠΑΝΕΠΙΣΤΗΜΙΟ ΚΡΗΤΗΣ Συστήματα Διαχείρισης Βάσεων Δεδομένων Φροντιστήριο 9: Transactions - part 1 Δημήτρης Πλεξουσάκης Τμήμα Επιστήμης Υπολογιστών Tutorial on Undo, Redo and Undo/Redo
Assalamu `alaikum wr. wb.
LUMP SUM Assalamu `alaikum wr. wb. LUMP SUM Wassalamu alaikum wr. wb. Assalamu `alaikum wr. wb. LUMP SUM Wassalamu alaikum wr. wb. LUMP SUM Lump sum lump sum lump sum. lump sum fixed price lump sum lump
Right Rear Door. Let's now finish the door hinge saga with the right rear door
Right Rear Door Let's now finish the door hinge saga with the right rear door You may have been already guessed my steps, so there is not much to describe in detail. Old upper one file:///c /Documents
Queensland University of Technology Transport Data Analysis and Modeling Methodologies
Queensland University of Technology Transport Data Analysis and Modeling Methodologies Lab Session #7 Example 5.2 (with 3SLS Extensions) Seemingly Unrelated Regression Estimation and 3SLS A survey of 206
Example of the Baum-Welch Algorithm
Example of the Baum-Welch Algorithm Larry Moss Q520, Spring 2008 1 Our corpus c We start with a very simple corpus. We take the set Y of unanalyzed words to be {ABBA, BAB}, and c to be given by c(abba)
Problem Set 9 Solutions. θ + 1. θ 2 + cotθ ( ) sinθ e iφ is an eigenfunction of the ˆ L 2 operator. / θ 2. φ 2. sin 2 θ φ 2. ( ) = e iφ. = e iφ cosθ.
Chemistry 362 Dr Jean M Standard Problem Set 9 Solutions The ˆ L 2 operator is defined as Verify that the angular wavefunction Y θ,φ) Also verify that the eigenvalue is given by 2! 2 & L ˆ 2! 2 2 θ 2 +
Statistical Inference I Locally most powerful tests
Statistical Inference I Locally most powerful tests Shirsendu Mukherjee Department of Statistics, Asutosh College, Kolkata, India. shirsendu st@yahoo.co.in So far we have treated the testing of one-sided
Every set of first-order formulas is equivalent to an independent set
Every set of first-order formulas is equivalent to an independent set May 6, 2008 Abstract A set of first-order formulas, whatever the cardinality of the set of symbols, is equivalent to an independent
k A = [k, k]( )[a 1, a 2 ] = [ka 1,ka 2 ] 4For the division of two intervals of confidence in R +
Chapter 3. Fuzzy Arithmetic 3- Fuzzy arithmetic: ~Addition(+) and subtraction (-): Let A = [a and B = [b, b in R If x [a and y [b, b than x+y [a +b +b Symbolically,we write A(+)B = [a (+)[b, b = [a +b
Notes on the Open Economy
Notes on the Open Econom Ben J. Heijdra Universit of Groningen April 24 Introduction In this note we stud the two-countr model of Table.4 in more detail. restated here for convenience. The model is Table.4.
1 String with massive end-points
1 String with massive end-points Πρόβλημα 5.11:Θεωρείστε μια χορδή μήκους, τάσης T, με δύο σημειακά σωματίδια στα άκρα της, το ένα μάζας m, και το άλλο μάζας m. α) Μελετώντας την κίνηση των άκρων βρείτε
Inverse trigonometric functions & General Solution of Trigonometric Equations. ------------------ ----------------------------- -----------------
Inverse trigonometric functions & General Solution of Trigonometric Equations. 1. Sin ( ) = a) b) c) d) Ans b. Solution : Method 1. Ans a: 17 > 1 a) is rejected. w.k.t Sin ( sin ) = d is rejected. If sin
ST5224: Advanced Statistical Theory II
ST5224: Advanced Statistical Theory II 2014/2015: Semester II Tutorial 7 1. Let X be a sample from a population P and consider testing hypotheses H 0 : P = P 0 versus H 1 : P = P 1, where P j is a known
Congruence Classes of Invertible Matrices of Order 3 over F 2
International Journal of Algebra, Vol. 8, 24, no. 5, 239-246 HIKARI Ltd, www.m-hikari.com http://dx.doi.org/.2988/ija.24.422 Congruence Classes of Invertible Matrices of Order 3 over F 2 Ligong An and
Quadratic Expressions
Quadratic Expressions. The standard form of a quadratic equation is ax + bx + c = 0 where a, b, c R and a 0. The roots of ax + bx + c = 0 are b ± b a 4ac. 3. For the equation ax +bx+c = 0, sum of the roots
Figure A.2: MPC and MPCP Age Profiles (estimating ρ, ρ = 2, φ = 0.03)..
Supplemental Material (not for publication) Persistent vs. Permanent Income Shocks in the Buffer-Stock Model Jeppe Druedahl Thomas H. Jørgensen May, A Additional Figures and Tables Figure A.: Wealth and
ΚΥΠΡΙΑΚΗ ΕΤΑΙΡΕΙΑ ΠΛΗΡΟΦΟΡΙΚΗΣ CYPRUS COMPUTER SOCIETY ΠΑΓΚΥΠΡΙΟΣ ΜΑΘΗΤΙΚΟΣ ΔΙΑΓΩΝΙΣΜΟΣ ΠΛΗΡΟΦΟΡΙΚΗΣ 11/3/2006
ΠΑΓΚΥΠΡΙΟΣ ΜΑΘΗΤΙΚΟΣ ΔΙΑΓΩΝΙΣΜΟΣ ΠΛΗΡΟΦΟΡΙΚΗΣ 11/3/26 Οδηγίες: Να απαντηθούν όλες οι ερωτήσεις. Ολοι οι αριθμοί που αναφέρονται σε όλα τα ερωτήματα μικρότεροι το 1 εκτός αν ορίζεται διαφορετικά στη διατύπωση
Lecture 34 Bootstrap confidence intervals
Lecture 34 Bootstrap confidence intervals Confidence Intervals θ: an unknown parameter of interest We want to find limits θ and θ such that Gt = P nˆθ θ t If G 1 1 α is known, then P θ θ = P θ θ = 1 α
Durbin-Levinson recursive method
Durbin-Levinson recursive method A recursive method for computing ϕ n is useful because it avoids inverting large matrices; when new data are acquired, one can update predictions, instead of starting again
Pg The perimeter is P = 3x The area of a triangle is. where b is the base, h is the height. In our case b = x, then the area is
Pg. 9. The perimeter is P = The area of a triangle is A = bh where b is the base, h is the height 0 h= btan 60 = b = b In our case b =, then the area is A = = 0. By Pythagorean theorem a + a = d a a =
CHAPTER 101 FOURIER SERIES FOR PERIODIC FUNCTIONS OF PERIOD
CHAPTER FOURIER SERIES FOR PERIODIC FUNCTIONS OF PERIOD EXERCISE 36 Page 66. Determine the Fourier series for the periodic function: f(x), when x +, when x which is periodic outside this rge of period.
Bayesian statistics. DS GA 1002 Probability and Statistics for Data Science.
Bayesian statistics DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Frequentist vs Bayesian statistics In frequentist
2. Μηχανικό Μαύρο Κουτί: κύλινδρος με μια μπάλα μέσα σε αυτόν.
Experiental Copetition: 14 July 011 Proble Page 1 of. Μηχανικό Μαύρο Κουτί: κύλινδρος με μια μπάλα μέσα σε αυτόν. Ένα μικρό σωματίδιο μάζας (μπάλα) βρίσκεται σε σταθερή απόσταση z από το πάνω μέρος ενός