Hebbian Learning. Donald Hebb ( )
|
|
- Βασιλική Λύτρας
- 6 χρόνια πριν
- Προβολές:
Transcript
1 Hebbian Learning Donald Hebb ( )
2 Hebbian Learning When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A s efficiency, as one of the cells firing B, is increased. (Donald Hebb, 1949)
3 Hebbian Learning When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A s efficiency, as one of the cells firing B, is increased. (Donald Hebb, 1949) Fire together, wire together
4 Hebbian Learning When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A s efficiency, as one of the cells firing B, is increased. (Donald Hebb, 1949) Fire together, wire together Simple model of Hebbian learning: x 1 x 2 x 3 w 1 y x N w N In the following we will drop µ for convenience. w i = η 1 M µ y µ x µ i (1) = η y µ x µ i µ (2) w = η y µ x µ µ. (3)
5 Linear Model Neuron A linear model neuron is described by: y = N w i x i = w T x. (4) i=1 This corresponds to a projection of the data onto the axis given by w scaled with w. x 2 1 w x 1-1
6 Linear Model Neuron A linear model neuron is described by: y = N w i x i = w T x. (4) i=1 This corresponds to a projection of the data onto the axis given by w scaled with w. x 2 1 w y x
7 Linear Model Neuron A linear model neuron is described by: y = N w i x i = w T x. (4) i=1 This corresponds to a projection of the data onto the axis given by w scaled with w. y
8 Linear Model Neuron A linear model neuron is described by: y = N w i x i = w T x. (4) i=1 This corresponds to a projection of the data onto the axis given by w scaled with w. y
9 Linear Model Neuron A linear model neuron is described by: y = N w i x i = w T x. (4) i=1 This corresponds to a projection of the data onto the axis given by w scaled with w. y
10 Hebbian Learning with a Linear Model Neuron w = (3) η y x (Hebbian learning) y (4) = w T x, (linear neuron)
11 Hebbian Learning with a Linear Model Neuron w = (3) η y x (Hebbian learning) y (4) = w T x, (linear neuron) w = (3) η yx (5) (4) = η (w T x)x (6) = η x(x T w) (7) = η xx T w (8)
12 Hebbian Learning with a Linear Model Neuron w = (3) η y x (Hebbian learning) y (4) = w T x, (linear neuron) w = (3) η yx (5) (4) = η (w T x)x (6) = η x(x T w) (7) = η xx T w (8) = w t+1 = w t + η xx T w t (9) = ( I + η xx T ) w t (10)
13 Hebbian Learning with a Linear Model Neuron w = (3) η y x (Hebbian learning) y (4) = w T x, (linear neuron) w = (3) η yx (5) (4) = η (w T x)x (6) = η x(x T w) (7) = η xx T w (8) = w t+1 = w t + η xx T w t (9) = ( I + η xx T ) w t (10) Hebbian learning with a linear model neuron can be interpreted as an iterated multiplication of the weight vector w with matrix (I + η xx T ).
14 Hebbian Learning with a Linear Model Neuron w = (3) η y x (Hebbian learning) y (4) = w T x, (linear neuron) w = (3) η yx (5) (4) = η (w T x)x (6) = η x(x T w) (7) = η xx T w (8) = w t+1 = w t + η xx T w t (9) = ( I + η xx T ) w t (10) Hebbian learning with a linear model neuron can be interpreted as an iterated multiplication of the weight vector w with matrix (I + η xx T ). The sign of the input vectors x is irrelevant for the learning.
15 Illustration of Hebbian Learning w = (6) η (w T x)x (Hebbian learning for a linear unit) x w x 1-1 1
16 Illustration of Hebbian Learning w = (6) η (w T x)x (Hebbian learning for a linear unit) x w x 1-1 1
17 Illustration of Hebbian Learning w = (6) η (w T x)x (Hebbian learning for a linear unit) x 2 2 w x 1-1
18 Illustration of Hebbian Learning w = (6) η (w T x)x (Hebbian learning for a linear unit) x 2 2 w x 1-1
19 Illustration of Hebbian Learning w = (6) η (w T x)x (Hebbian learning for a linear unit) w 3 x x 1-1
20 Illustration of Hebbian Learning w = (6) η (w T x)x (Hebbian learning for a linear unit) w * x x 1-1
21 Illustration of Hebbian Learning w = (6) η (w T x)x (Hebbian learning for a linear unit) w * x x 1-1 Problem: The weights grow unlimited.
22 Explicit Normalization of the Weight Vectors First apply the learning rule, w t+1 i = w t i + ηf i with, e.g.,f i = yx i. (11) Then normalize the weight vector to length one, We can verify that i ( ) w t+1 2 (12) i = w t+1 i = ( i i ( j j w t+1 w t+1 i w t+1 j w t+1 j ) 2 ) 2. (12) 2 = i ( j ( ) w t+1 2 i w t+1 j ) 2 = 1. (13) Problem: Explicit normalization is expensive and biologically implausible.
23 Implicit Normalization of the Weight Vector Explicit normalization yielded w t+1 i (η) (12) = wi t + ηf i ( ). 2 wj t + ηf j j
24 Implicit Normalization of the Weight Vector Explicit normalization yielded w t+1 i (η) (12) = wi t + ηf i ( ). 2 wj t + ηf j j To optain a simpler/more plausible normalization rule, we compute the Taylor-expansion of w t+1 i in η at η = 0 under the assumption that w t is normalized, i.e. (wj t)2 = 1, j t+1 w t+1 i (η) w t+1 i (η wi (η ) = 0) + η η η =0 (14)
25 Implicit Normalization of the Weight Vector Explicit normalization yielded w t+1 i (η) (12) = wi t + ηf i ( ). 2 wj t + ηf j j To optain a simpler/more plausible normalization rule, we compute the Taylor-expansion of w t+1 i in η at η = 0 under the assumption that w t is normalized, i.e. (wj t)2 = 1, j t+1 w t+1 i (η) w t+1 i (η wi (η ) = 0) + η η = w t+1 i (η = 0) + η f i j η =0 ( ) 2 wj t + η f j (w t i + η f i ) j ( w t j + η f j ) 2 ( ) f j wj t +η f j j ) (w j t 2 +η f j j (14) (15) η =0
26 Implicit Normalization of the Weight Vector Explicit normalization yielded w t+1 i (η) (12) = wi t + ηf i ( ). 2 wj t + ηf j j To optain a simpler/more plausible normalization rule, we compute the Taylor-expansion of w t+1 i in η at η = 0 under the assumption that w t is normalized, i.e. (wj t)2 = 1, j t+1 w t+1 i (η) w t+1 i (η wi (η ) = 0) + η η = w t+1 i (η = 0) + η f i j η =0 ( ) 2 wj t + η f j (w t i + η f i ) j ( w t j + η f j ) 2 ( ) f j wj t +η f j j ) (w j t 2 +η f j j (14) (15) η =0 = wi t + η f i wi t f j wj t. j (16)
27 Implicit Normalization of the Weight Vector Taylor-expansion of the explicit normalization rule resulted in (16) wi t + η f i wi t f j wj t. w t+1 i j
28 Implicit Normalization of the Weight Vector Taylor-expansion of the explicit normalization rule resulted in (16) wi t + η f i wi t f j wj t. w t+1 i For Hebbian learning, i.e. with f i = yx i and y = j x jw j, this becomes (16) w i = η f i w i f j w j (17) j j
29 Implicit Normalization of the Weight Vector Taylor-expansion of the explicit normalization rule resulted in (16) wi t + η f i wi t f j wj t. w t+1 i For Hebbian learning, i.e. with f i = yx i and y = j x jw j, this becomes (16) w i = η f i w i f j w j (17) = η yx i w i y x j w j j }{{} =y j j (18)
30 Implicit Normalization of the Weight Vector Taylor-expansion of the explicit normalization rule resulted in (16) wi t + η f i wi t f j wj t. w t+1 i For Hebbian learning, i.e. with f i = yx i and y = j x jw j, this becomes (16) w i = η f i w i f j w j (17) j j = η yx i w i y x j w j (18) j }{{} =y = η ( yx i y 2 ) w i i (19)
31 Implicit Normalization of the Weight Vector Taylor-expansion of the explicit normalization rule resulted in (16) wi t + η f i wi t f j wj t. w t+1 i For Hebbian learning, i.e. with f i = yx i and y = j x jw j, this becomes (16) w i = η f i w i f j w j (17) j j = η yx i w i y x j w j (18) j }{{} =y = η ( yx i y 2 ) w i i (19) w = η (yx y 2 w). (Oja s rule) (20)
32 Implicit Normalization of the Weight Vector Taylor-expansion of the explicit normalization rule resulted in (16) wi t + η f i wi t f j wj t. w t+1 i For Hebbian learning, i.e. with f i = yx i and y = j x jw j, this becomes (16) w i = η f i w i f j w j (17) j j = η yx i w i y x j w j (18) j }{{} =y = η ( yx i y 2 ) w i i (19) w = η (yx y 2 w). (Oja s rule) (20) This rule is simpler, e.g. w i does not depend on w j anymore.
33 Oja s Rule Implicit normalization yielded w (20) = η (yx y 2 w). (Oja s rule)
34 Oja s Rule Implicit normalization yielded w (20) = η (yx y 2 w). (Oja s rule) For small y we have
35 Oja s Rule Implicit normalization yielded w (20) = η (yx y 2 w). (Oja s rule) For small y we have w ηyx, (21)
36 Oja s Rule Implicit normalization yielded w (20) = η (yx y 2 w). (Oja s rule) For small y we have which corresponds to Hebb s rule. w ηyx, (21)
37 Oja s Rule Implicit normalization yielded w (20) = η (yx y 2 w). (Oja s rule) For small y we have w ηyx, (21) which corresponds to Hebb s rule. For large y we have
38 Oja s Rule Implicit normalization yielded w (20) = η (yx y 2 w). (Oja s rule) For small y we have w ηyx, (21) which corresponds to Hebb s rule. For large y we have w ηy 2 w, (22)
39 Oja s Rule Implicit normalization yielded w (20) = η (yx y 2 w). (Oja s rule) For small y we have w ηyx, (21) which corresponds to Hebb s rule. For large y we have w ηy 2 w, (22) which results in a shortening of the weight vector.
40 Oja s Rule Implicit normalization yielded w (20) = η (yx y 2 w). (Oja s rule) For small y we have which corresponds to Hebb s rule. For large y we have which results in a shortening of the weight vector. w ηyx, (21) w ηy 2 w, (22) The second term only affects the length but not the direction of the weight vector. It therefore limits the length of the weight vector without affecting the qualitative behavior of the learning rule.
41 Oja s Rule Implicit normalization yielded w (20) = η (yx y 2 w). (Oja s rule) For small y we have which corresponds to Hebb s rule. For large y we have which results in a shortening of the weight vector. w ηyx, (21) w ηy 2 w, (22) The second term only affects the length but not the direction of the weight vector. It therefore limits the length of the weight vector without affecting the qualitative behavior of the learning rule. Question: What will be the final length of the weight vector?
42 Oja s Rule Implicit normalization yielded w (20) = η (yx y 2 w). (Oja s rule) For small y we have which corresponds to Hebb s rule. For large y we have which results in a shortening of the weight vector. w ηyx, (21) w ηy 2 w, (22) The second term only affects the length but not the direction of the weight vector. It therefore limits the length of the weight vector without affecting the qualitative behavior of the learning rule. Question: What will be the final length of the weight vector? Question: How will the weight vector develop under Oja s rule in general?
43 Linear Stability Analysis y (4) = w T x (linear unit) w (20) = η(yx y 2 w) (Oja s rule)
44 Linear Stability Analysis y (4) = w T x (linear unit) w (20) = η(yx y 2 w) (Oja s rule) 1. What is the mean dynamics of the weight vectors? 1 η w µ =? 2. What are the stationary weight vectors of the dynamics? 3. Which weight vectors are stable? 0! = w µ w =? w =? stable?
45 What is the mean dynamics of the weight vectors? y (4) = w T x (linear unit) ( w) µ (20) = η(y µ x µ (y µ ) 2 w) (Oja s rule)
46 What is the mean dynamics of the weight vectors? y (4) = w T x (linear unit) ( w) µ (20) = η(y µ x µ (y µ ) 2 w) (Oja s rule) In order to proceed analytically we have to average over all training patterns µ and get ( w) µ µ = 1 M (20) = 1 M M ( w) µ (23) µ=1 M η(y µ x µ (y µ ) 2 w) (24) µ=1 w µ = η yx y 2 w µ (25) For small learning rates η this is a good approximation to the real training procedure (apart from a factor of M).
47 What is the mean dynamics of the weight vectors? y (4) = w T x, (linear unit) (25) w µ = η yx y 2 w µ. (Oja s rule, averaged)
48 What is the mean dynamics of the weight vectors? y (4) = w T x, (linear unit) (25) w µ = η yx y 2 w µ. (Oja s rule, averaged) 1 η w µ (25) = yx y 2 w µ (26) (4) = (w T x)x (w T x)(w T x)w µ (27) = x(x T w) (w T x)(x T w)w µ (28) = (xx T )w (w T (xx T )w)w µ (29) = xx T µ w (w T xx T µ w)w (30) [ C := xx T µ = C T ] (31) (31) = Cw ( w T Cw ) w (32)
49 y Interim Summary (4) = w T x (linear unit) w (20) = η(yx y 2 w) (Oja s rule) 1. What is the mean dynamics of the weight vectors? 1 η w µ (32) = Cw ( w T Cw ) w with C (31) := xx T µ 2. What are the stationary weight vectors of the dynamics? 3. Which weight vectors are stable? 0! = w µ w =? w =? stable?
50 What are the stationary weight vectors? 1 η w µ C (31) = xx T µ = C T, (symmetric matrix C) (32) = Cw ( w T Cw ) w. (weight dynamics)
51 What are the stationary weight vectors? 1 η w µ C (31) = xx T µ = C T, (symmetric matrix C) (32) = Cw ( w T Cw ) w. (weight dynamics) For stationary weight vectors we have: 0! = w µ (33) (32) Cw = ( w T Cw ) w (34) [ ( λ := w T Cw )] (35) (35) Cw = λw (36)
52 What are the stationary weight vectors? 1 η w µ C (31) = xx T µ = C T, (symmetric matrix C) (32) = Cw ( w T Cw ) w. (weight dynamics) For stationary weight vectors we have: 0! = w µ (33) (32) Cw = ( w T Cw ) w (34) [ ( λ := w T Cw )] (35) (35) Cw = λw (36) Stationary weight vectors w are eigenvectors of matrix C.
53 What are the stationary weight vectors? 1 η w µ C (31) = xx T µ = C T, (symmetric matrix C) (32) = Cw ( w T Cw ) w. (weight dynamics) For stationary weight vectors we have: 0! = w µ (33) (32) Cw = ( w T Cw ) w (34) [ ( λ := w T Cw )] (35) (35) Cw = λw (36) Stationary weight vectors w are eigenvectors of matrix C. λ (35) = w T Cw (36) = w T λw = λw T w = λ w 2 (37) 1 = w 2 (38) Stationary weight vectors w are normalized to 1.
54 y Interim Summary (4) = w T x (linear unit) w (20) = η(yx y 2 w) (Oja s rule) 1. What is the mean dynamics of the weight vectors? 1 η w µ (32) = Cw ( w T Cw ) w with C (31) := xx T µ 2. What are the stationary weight vectors of the dynamics? 0! = w µ w = c α 3. Which weight vectors are stable? with Cc α (36) = λ α c α (eigenvectors of C) 1 (38) = c α 2 (with norm 1) w =? stable?
55 Reminder: Eigenvalues and Eigenvectors Aa = λa (eigenvalue equation) (39) Solutions of the eigenvalue equation for a given quadratic N N-matrix A are called eigenvectors a and eigenvalues λ. For symmetric A, i.e. A T = A, eigenvalues λ are real, and eigenvectors to different eigenvalues are orthogonal, i.e. λ α λ β a α a β. A symmetric matrix A always has a complete set of orthonormal eigenvectors a α, α = 1,..., N (orthonormal basis), i.e. Aa α = λ αa α, (right-eigenvectors) (40) a αt A = (Aa α ) T = a αt λ α, (left-eigenvectors) (41) a α = a αt a α = 1, (with norm 1) (42) a αt a β = 0 α β. (orthogonal) (43) N v v = v αa α mit v α = a αt v (complete) (44) = v v = α=1 N a α a αt v 1 = α=1 v 2 = i v 2 i (42, 43, 44) = α v α 2 N a α a αt (45) α=1 (46)
56 Reminder: Eigenvalues and Eigenvectors A T = A, (symmetric) (47) Aa α = λ α a α, (eigenvectors) (48) a αt a β = δ αβ α, β. (orthonormal) (49) x 2 a β β λ β a Ax x a α α λ α a x 1
57 Which weight vectors are stable? 1 η w (32) µ = Cw ( w T Cw ) w (50)
58 Which weight vectors are stable? 1 η w (32) µ = Cw ( w T Cw ) w (50) Ansatz: w := c α + ɛ (with small ɛ) (51) c β w < ε>? ε c α How does ɛ develop under the dynamics?
59 Which weight vectors are stable? 1 η w µ (32) = Cw ( w T Cw ) w w := (51) c α + ɛ (with small ɛ) 1 η ɛ µ (51) = 1 w µ (52) η (32) = Cw (51) = C(c α + ɛ) ( ) w T Cw w (53) ( ) (c α + ɛ) T C(c α + ɛ) (c α + ɛ) (54) Cc α + Cɛ (c αt Cc α )c α (c αt Cɛ)c α (ɛ T Cc α )c α (c αt Cc α )ɛ (55) (36,41) = λ α c α + Cɛ (c αt λ α c α )c α (c αt λ α ɛ)c α (ɛ T λ α c α )c α (c αt λ α c α )ɛ (56) (38) = λ α c α + Cɛ λ α c α λ α (c αt ɛ)c α λ α (ɛ T c α )c α λ α ɛ (57) = Cɛ 2λ α (c αt ɛ)c α λ α ɛ. (58)
60 Which weight vectors are stable? 1 η ɛ µ w := (51) c α + ɛ, (58) Cɛ 2λ α (c αt ɛ)c α λ α ɛ.
61 Which weight vectors are stable? 1 η ɛ µ w := (51) c α + ɛ, (58) Cɛ 2λ α (c αt ɛ)c α λ α ɛ. For simplicity consider the change of the perturbation along the eigenvector c β. c β w < ε> ε c β T < ε>? c βt ε c α
62 Which weight vectors are stable? 1 η ɛ µ w := (51) c α + ɛ, (58) Cɛ 2λ α (c αt ɛ)c α λ α ɛ. For simplicity consider the change of the perturbation along the eigenvector c β. 1 η cβ T ɛ µ (58) c β T Cɛ 2λ α(c αt ɛ)(c β T c α ) λ α(c β T ɛ) (59) (36,41) = λ β (c β T ɛ) 2λ α(c αt ɛ)(c β T c α ) λ α(c β T ɛ) (60) (42,43) = λ β (c β T ɛ) 2λ α(c αt ɛ)δ βα λ α(c β T ɛ) (61) = λ β (c β T ɛ) 2λ α(c β T ɛ)δ βα λ α(c β T ɛ) (62) = β 2λ αδ βα λ α) (c β T ɛ) (λ { (63) = ( 2λ β )(c β T ɛ) if β = α (λ β λ α)(c β T ɛ) if β α (64)
63 Which weight vectors are stable? 1 η cβt ɛ µ w (51) := c α + ɛ, { (64) ( 2λ β )(c βt ɛ) if β = α (λ β λ α )(c βt ɛ) if β α.
64 Which weight vectors are stable? 1 η cβt ɛ µ w (51) := c α + ɛ, { (64) ( 2λ β )(c βt ɛ) if β = α (λ β λ α )(c βt ɛ) if β α. With c βt ɛ µ = c βt ɛ µ (65) = c βt (ɛ n+1 ɛ n ) µ (66) = (c βt ɛ) n+1 (c βt ɛ) n µ (67) = (c βt ɛ) µ (68)
65 Which weight vectors are stable? 1 η cβt ɛ µ w (51) := c α + ɛ, { (64) ( 2λ β )(c βt ɛ) if β = α (λ β λ α )(c βt ɛ) if β α. With c βt ɛ µ = c βt ɛ µ (65) = c βt (ɛ n+1 ɛ n ) µ (66) = (c βt ɛ) n+1 (c βt ɛ) n µ (67) = (c βt ɛ) µ (68) s β := c βt ɛ (perturbation along c β ) (69)
66 Which weight vectors are stable? 1 η cβt ɛ µ w (51) := c α + ɛ, { (64) ( 2λ β )(c βt ɛ) if β = α (λ β λ α )(c βt ɛ) if β α. With c βt ɛ µ = c βt ɛ µ (65) = c βt (ɛ n+1 ɛ n ) µ (66) = (c βt ɛ) n+1 (c βt ɛ) n µ (67) = (c βt ɛ) µ (68) s β := c βt ɛ (perturbation along c β ) (69) { η( 2λ κ αβ := β ) if β = α (70) η(λ β λ α ) if β α
67 Which weight vectors are stable? 1 η cβt ɛ µ w (51) := c α + ɛ, { (64) ( 2λ β )(c βt ɛ) if β = α (λ β λ α )(c βt ɛ) if β α. With c βt ɛ µ = c βt ɛ µ (65) = c βt (ɛ n+1 ɛ n ) µ (66) = (c βt ɛ) n+1 (c βt ɛ) n µ (67) = (c βt ɛ) µ (68) s β := c βt ɛ (perturbation along c β ) (69) { η( 2λ κ αβ := β ) if β = α (70) η(λ β λ α ) if β α we get (64) (68, 69, 70) s β µ κ αβ s β. (71)
68 Which weight vectors are stable? (51) w := c α + ɛ, { (71) (70) η( 2λ s β µ κ αβ s β with κ αβ := β ) if β = α η(λ β λ α ) if β α.
69 Which weight vectors are stable? (51) w := c α + ɛ, { (71) s β µ κ αβ s β with (70) κ αβ := η( 2λ β ) if β = α η(λ β λ α ) if β α. Case 1: β α, λ β < λ α β = / α λ β < λα c β -Richtung c β s β ε c α
70 Which weight vectors are stable? (51) w := c α + ɛ, { (71) s β µ κ αβ s β with (70) κ αβ := η( 2λ β ) if β = α η(λ β λ α ) if β α. Case 1: β α, λ β < λ α κ αβ = η(λ β λ α ) < 0, β = / α λ β < λα κ < 0 αβ c β -Richtung c β s β ε c α
71 Which weight vectors are stable? (51) w := c α + ɛ, { (71) s β µ κ αβ s β with (70) κ αβ := η( 2λ β ) if β = α η(λ β λ α ) if β α. Case 1: β α, λ β < λ α κ αβ = η(λ β λ α ) < 0, The perturbation along c β decays. β = / α λ β < λα κ < 0 αβ c β -Richtung c β < s > β s β ε c α
72 Which weight vectors are stable? (51) w := c α + ɛ, { (71) (70) η( 2λ s β µ κ αβ s β with κ αβ := β ) if β = α η(λ β λ α ) if β α.
73 Which weight vectors are stable? (51) w := c α + ɛ, { (71) s β µ κ αβ s β with (70) κ αβ := η( 2λ β ) if β = α η(λ β λ α ) if β α. Case 2: β α, λ β = λ α β = / α λβ = λ α c β -Richtung c β s β ε c α
74 Which weight vectors are stable? (51) w := c α + ɛ, { (71) s β µ κ αβ s β with (70) κ αβ := η( 2λ β ) if β = α η(λ β λ α ) if β α. Case 2: β α, λ β = λ α κ αβ = η(λ β λ α ) = 0, β = / α λβ = λα κ αβ = 0 c β -Richtung c β s β ε c α
75 Which weight vectors are stable? (51) w := c α + ɛ, { (71) s β µ κ αβ s β with (70) κ αβ := η( 2λ β ) if β = α η(λ β λ α ) if β α. Case 2: β α, λ β = λ α κ αβ = η(λ β λ α ) = 0, The perturbation along c β persists. c β β = / α λβ = λ κ αβ = 0 α < s > β c β -Richtung s β ε c α
76 Which weight vectors are stable? (51) w := c α + ɛ, { (71) (70) η( 2λ s β µ κ αβ s β with κ αβ := β ) if β = α η(λ β λ α ) if β α.
77 Which weight vectors are stable? (51) w := c α + ɛ, { (71) s β µ κ αβ s β with (70) κ αβ := η( 2λ β ) if β = α η(λ β λ α ) if β α. Case 3: β α, λ β > λ α β = / α λβ > λ α c β -Richtung c β s β ε c α
78 Which weight vectors are stable? (51) w := c α + ɛ, { (71) s β µ κ αβ s β with (70) κ αβ := η( 2λ β ) if β = α η(λ β λ α ) if β α. Case 3: β α, λ β > λ α κ αβ = η(λ β λ α ) > 0, β = / α λβ > λα κ > 0 αβ c β -Richtung c β s β ε c α
79 Which weight vectors are stable? (51) w := c α + ɛ, { (71) s β µ κ αβ s β with (70) κ αβ := η( 2λ β ) if β = α η(λ β λ α ) if β α. Case 3: β α, λ β > λ α κ αβ = η(λ β λ α ) > 0, The perturbation along c β grows. c β β = / α λβ > λα c β -Richtung κ αβ> 0 < s β> ε s β c α
80 Which weight vectors are stable? (51) w := c α + ɛ, { (71) (70) η( 2λ s β µ κ αβ s β with κ αβ := β ) if β = α η(λ β λ α ) if β α.
81 Which weight vectors are stable? (51) w := c α + ɛ, { (71) s β µ κ αβ s β with (70) κ αβ := η( 2λ β ) if β = α η(λ β λ α ) if β α. Case 4: β = α c β β = α ε c α -Richtung c α s α
82 Which weight vectors are stable? (51) w := c α + ɛ, { (71) s β µ κ αβ s β with (70) κ αβ := η( 2λ β ) if β = α η(λ β λ α ) if β α. Case 4: β = α κ αβ = η( 2λ β ) < 0. c β β = α κ < 0 αβ ε c α -Richtung c α s α
83 Which weight vectors are stable? (51) w := c α + ɛ, { (71) s β µ κ αβ s β with (70) κ αβ := η( 2λ β ) if β = α η(λ β λ α ) if β α. Case 4: β = α κ αβ = η( 2λ β ) < 0. The perturbation along c α always decays (if λ β > 0). c β β = α κ < 0 αβ ε c α s α c α -Richtung < > s α
84 Which weight vectors are stable? (51) w := c α + ɛ, { (71) (70) η( 2λ s β µ κ αβ s β with κ αβ := β ) if β = α η(λ β λ α ) if β α.
85 Which weight vectors are stable? (51) w := c α + ɛ, { (71) s β µ κ αβ s β with (70) κ αβ := η( 2λ β ) if β = α η(λ β λ α ) if β α. Cases 3 and 4 combined. The weight vectors turns away c α into the c β -direction. c β β = α λ > λ β α < s > c β Richtung < ε > β s β c α ε s α c α Richtung < > s α
86 Which weight vectors are stable? (51) w := c α + ɛ, { (71) (70) η( 2λ s β µ κ αβ s β with κ αβ := β ) if β = α η(λ β λ α ) if β α.
87 Which weight vectors are stable? (51) w := c α + ɛ, { (71) s β µ κ αβ s β with (70) κ αβ := η( 2λ β ) if β = α η(λ β λ α ) if β α. Case 1: β α, λ β < λ α, The perturbation along c β decays. Case 2: β α, λ β = λ α, The perturbation along c β persists. Case 3: β α, λ β > λ α, The perturbation along c β grows. Case 4: β = α, The perturbation along c α always decays.
88 Which weight vectors are stable? (51) w := c α + ɛ, { (71) s β µ κ αβ s β with (70) κ αβ := η( 2λ β ) if β = α η(λ β λ α ) if β α. Case 1: β α, λ β < λ α, The perturbation along c β decays. Case 2: β α, λ β = λ α, The perturbation along c β persists. Case 3: β α, λ β > λ α, The perturbation along c β grows. Case 4: β = α, The perturbation along c α always decays. The weight vector w = c α is stable only if perturbations in all directions decay,
89 Which weight vectors are stable? (51) w := c α + ɛ, { (71) s β µ κ αβ s β with (70) κ αβ := η( 2λ β ) if β = α η(λ β λ α ) if β α. Case 1: β α, λ β < λ α, The perturbation along c β decays. Case 2: β α, λ β = λ α, The perturbation along c β persists. Case 3: β α, λ β > λ α, The perturbation along c β grows. Case 4: β = α, The perturbation along c α always decays. The weight vector w = c α is stable only if perturbations in all directions decay, i.e. w = c α stable λ β < λ α β α. (72)
90 Which weight vectors are stable? (51) w := c α + ɛ, { (71) s β µ κ αβ s β with (70) κ αβ := η( 2λ β ) if β = α η(λ β λ α ) if β α. Case 1: β α, λ β < λ α, The perturbation along c β decays. Case 2: β α, λ β = λ α, The perturbation along c β persists. Case 3: β α, λ β > λ α, The perturbation along c β grows. Case 4: β = α, The perturbation along c α always decays. The weight vector w = c α is stable only if perturbations in all directions decay, i.e. w = c α stable λ β < λ α β α. (72) Only the eigenvector c 1 with largest eigenvalue λ 1 is a stable weight vector under Oja s rule.
91 Which weight vectors are stable? (51) w := c α + ɛ, { (71) s β µ κ αβ s β with (70) κ αβ := η( 2λ β ) if β = α η(λ β λ α ) if β α. Case 1: β α, λ β < λ α, The perturbation along c β decays. Case 2: β α, λ β = λ α, The perturbation along c β persists. Case 3: β α, λ β > λ α, The perturbation along c β grows. Case 4: β = α, The perturbation along c α always decays. The weight vector w = c α is stable only if perturbations in all directions decay, i.e. w = c α stable λ β < λ α β α. (72) Only the eigenvector c 1 with largest eigenvalue λ 1 is a stable weight vector under Oja s rule. What happens if the two largest eigenvalues are equal?
92 y Summary (4) = w T x (linear unit) w (20) = η(yx y 2 w) (Oja s rule) 1. What is the mean dynamics of the weight vectors? 1 η w µ (32) = Cw ( w T Cw ) w with C (31) := xx T µ 2. What are the stationary weight vectors of the dynamics? 0! = w µ w = c α 3. Which weight vectors are stable? with Cc α (36) = λ α c α (eigenvectors of C) 1 (38) = c α 2 (with norm 1) w = c α stable (72) λ β < λ α β α
93 Reminder: Principal Components c 2 c 1 Principal components are eigenvectors of the covariance matrix and point in the direction of maximal variance within the space orthogonal to the earlier principal components.
94 Learning Several Principal Components x 1 x 2 x 3 y y (4) = w T x (linear unit) w (20) = η(yx y 2 w) (Oja s rule) x N (Rubner & Tavan, 1989, Europhys. Letters 10:693 8; reviewed in Becker & Plumbley, 1996, J. Appl. Intelligence 6: )
95 Learning Several Principal Components x 1 x 2 x 3 y y (4) = w T x (linear unit) w (20) = η(yx y 2 w) (Oja s rule) x N x 1 x 2 x 3 x N y 1 y y 2 3 j 1 y j = wj T x + v jk y k k=1 w j = η(y j x yj 2 w j ) v jk = ɛy j y k Asymmetric inhibitory lateral connections decorrelate later from earlier output units. The units learn the principal components in order of decreasing eigenvalue. (Rubner & Tavan, 1989, Europhys. Letters 10:693 8; reviewed in Becker & Plumbley, 1996, J. Appl. Intelligence 6: )
96 Learning a Principal Subspace x 1 x 2 x 3 y y (4) = w T x (linear unit) w (20) = η(yx y 2 w) (Oja s rule) x N x 1 x 2 x 3 x N y 1 y y 2 3 y j = w T j x + N v jk y k k=1 k j w j = η(y j x yj 2 w j ) v jk = ɛy j y k Symmetric inhibitory lateral connections mutually decorrelate output units. The units learn the principal subspace but not particular principal components. (Földiák, 1989, Proc. IJCNN 89 pp ; reviewed in Becker & Plumbley, 1996, J. Appl. Intelligence 6: )
97 The Principal Components of Natural Images 15 natural images of size pixels. 20,000 random samples of size pixels. For each pixel the mean gray value over the 20,000 samples was removed. The samples were windowed with a Gaussian with std. dev. 10 pixels. Sanger s rule was applied to the samples. (Hancock, Baddeley, & Smith, 1992, Network 3(1):61 70)
98 The Principal Components of Natural Images The first principal components resemble simple-cell receptive fields in the primary visual cortex, the later ones do not. (Hancock, Baddeley, & Smith, 1992, Network 3(1):61 70)
99 The Principal Components of Natural Images The principal components look different if samples are taken from text. (Hancock, Baddeley, & Smith, 1992, Network 3(1):61 70)
Numerical Analysis FMN011
Numerical Analysis FMN011 Carmen Arévalo Lund University carmen@maths.lth.se Lecture 12 Periodic data A function g has period P if g(x + P ) = g(x) Model: Trigonometric polynomial of order M T M (x) =
Διαβάστε περισσότεραHomework 3 Solutions
Homework 3 Solutions Igor Yanovsky (Math 151A TA) Problem 1: Compute the absolute error and relative error in approximations of p by p. (Use calculator!) a) p π, p 22/7; b) p π, p 3.141. Solution: For
Διαβάστε περισσότεραHomework 8 Model Solution Section
MATH 004 Homework Solution Homework 8 Model Solution Section 14.5 14.6. 14.5. Use the Chain Rule to find dz where z cosx + 4y), x 5t 4, y 1 t. dz dx + dy y sinx + 4y)0t + 4) sinx + 4y) 1t ) 0t + 4t ) sinx
Διαβάστε περισσότεραSCHOOL OF MATHEMATICAL SCIENCES G11LMA Linear Mathematics Examination Solutions
SCHOOL OF MATHEMATICAL SCIENCES GLMA Linear Mathematics 00- Examination Solutions. (a) i. ( + 5i)( i) = (6 + 5) + (5 )i = + i. Real part is, imaginary part is. (b) ii. + 5i i ( + 5i)( + i) = ( i)( + i)
Διαβάστε περισσότεραw o = R 1 p. (1) R = p =. = 1
Πανεπιστήµιο Κρήτης - Τµήµα Επιστήµης Υπολογιστών ΗΥ-570: Στατιστική Επεξεργασία Σήµατος 205 ιδάσκων : Α. Μουχτάρης Τριτη Σειρά Ασκήσεων Λύσεις Ασκηση 3. 5.2 (a) From the Wiener-Hopf equation we have:
Διαβάστε περισσότεραChapter 6: Systems of Linear Differential. be continuous functions on the interval
Chapter 6: Systems of Linear Differential Equations Let a (t), a 2 (t),..., a nn (t), b (t), b 2 (t),..., b n (t) be continuous functions on the interval I. The system of n first-order differential equations
Διαβάστε περισσότεραEE512: Error Control Coding
EE512: Error Control Coding Solution for Assignment on Finite Fields February 16, 2007 1. (a) Addition and Multiplication tables for GF (5) and GF (7) are shown in Tables 1 and 2. + 0 1 2 3 4 0 0 1 2 3
Διαβάστε περισσότερα2 Composition. Invertible Mappings
Arkansas Tech University MATH 4033: Elementary Modern Algebra Dr. Marcel B. Finan Composition. Invertible Mappings In this section we discuss two procedures for creating new mappings from old ones, namely,
Διαβάστε περισσότεραExercises 10. Find a fundamental matrix of the given system of equations. Also find the fundamental matrix Φ(t) satisfying Φ(0) = I. 1.
Exercises 0 More exercises are available in Elementary Differential Equations. If you have a problem to solve any of them, feel free to come to office hour. Problem Find a fundamental matrix of the given
Διαβάστε περισσότεραExample Sheet 3 Solutions
Example Sheet 3 Solutions. i Regular Sturm-Liouville. ii Singular Sturm-Liouville mixed boundary conditions. iii Not Sturm-Liouville ODE is not in Sturm-Liouville form. iv Regular Sturm-Liouville note
Διαβάστε περισσότεραThe Simply Typed Lambda Calculus
Type Inference Instead of writing type annotations, can we use an algorithm to infer what the type annotations should be? That depends on the type system. For simple type systems the answer is yes, and
Διαβάστε περισσότεραStatistical Inference I Locally most powerful tests
Statistical Inference I Locally most powerful tests Shirsendu Mukherjee Department of Statistics, Asutosh College, Kolkata, India. shirsendu st@yahoo.co.in So far we have treated the testing of one-sided
Διαβάστε περισσότεραMatrices and Determinants
Matrices and Determinants SUBJECTIVE PROBLEMS: Q 1. For what value of k do the following system of equations possess a non-trivial (i.e., not all zero) solution over the set of rationals Q? x + ky + 3z
Διαβάστε περισσότεραPartial Differential Equations in Biology The boundary element method. March 26, 2013
The boundary element method March 26, 203 Introduction and notation The problem: u = f in D R d u = ϕ in Γ D u n = g on Γ N, where D = Γ D Γ N, Γ D Γ N = (possibly, Γ D = [Neumann problem] or Γ N = [Dirichlet
Διαβάστε περισσότεραPhys460.nb Solution for the t-dependent Schrodinger s equation How did we find the solution? (not required)
Phys460.nb 81 ψ n (t) is still the (same) eigenstate of H But for tdependent H. The answer is NO. 5.5.5. Solution for the tdependent Schrodinger s equation If we assume that at time t 0, the electron starts
Διαβάστε περισσότεραCHAPTER 48 APPLICATIONS OF MATRICES AND DETERMINANTS
CHAPTER 48 APPLICATIONS OF MATRICES AND DETERMINANTS EXERCISE 01 Page 545 1. Use matrices to solve: 3x + 4y x + 5y + 7 3x + 4y x + 5y 7 Hence, 3 4 x 0 5 y 7 The inverse of 3 4 5 is: 1 5 4 1 5 4 15 8 3
Διαβάστε περισσότερα( ) 2 and compare to M.
Problems and Solutions for Section 4.2 4.9 through 4.33) 4.9 Calculate the square root of the matrix 3!0 M!0 8 Hint: Let M / 2 a!b ; calculate M / 2!b c ) 2 and compare to M. Solution: Given: 3!0 M!0 8
Διαβάστε περισσότεραCHAPTER 25 SOLVING EQUATIONS BY ITERATIVE METHODS
CHAPTER 5 SOLVING EQUATIONS BY ITERATIVE METHODS EXERCISE 104 Page 8 1. Find the positive root of the equation x + 3x 5 = 0, correct to 3 significant figures, using the method of bisection. Let f(x) =
Διαβάστε περισσότεραSecond Order Partial Differential Equations
Chapter 7 Second Order Partial Differential Equations 7.1 Introduction A second order linear PDE in two independent variables (x, y Ω can be written as A(x, y u x + B(x, y u xy + C(x, y u u u + D(x, y
Διαβάστε περισσότεραX = [ 1 2 4 6 12 15 25 45 68 67 65 98 ] X X double[] X = { 1, 2, 4, 6, 12, 15, 25, 45, 68, 67, 65, 98 }; double X.Length double double[] x1 = { 0, 8, 12, 20 }; double[] x2 = { 8, 9, 11, 12 }; double mean1
Διαβάστε περισσότεραLecture 2: Dirac notation and a review of linear algebra Read Sakurai chapter 1, Baym chatper 3
Lecture 2: Dirac notation and a review of linear algebra Read Sakurai chapter 1, Baym chatper 3 1 State vector space and the dual space Space of wavefunctions The space of wavefunctions is the set of all
Διαβάστε περισσότεραConcrete Mathematics Exercises from 30 September 2016
Concrete Mathematics Exercises from 30 September 2016 Silvio Capobianco Exercise 1.7 Let H(n) = J(n + 1) J(n). Equation (1.8) tells us that H(2n) = 2, and H(2n+1) = J(2n+2) J(2n+1) = (2J(n+1) 1) (2J(n)+1)
Διαβάστε περισσότεραC.S. 430 Assignment 6, Sample Solutions
C.S. 430 Assignment 6, Sample Solutions Paul Liu November 15, 2007 Note that these are sample solutions only; in many cases there were many acceptable answers. 1 Reynolds Problem 10.1 1.1 Normal-order
Διαβάστε περισσότερα( y) Partial Differential Equations
Partial Dierential Equations Linear P.D.Es. contains no owers roducts o the deendent variables / an o its derivatives can occasionall be solved. Consider eamle ( ) a (sometimes written as a ) we can integrate
Διαβάστε περισσότερα5.4 The Poisson Distribution.
The worst thing you can do about a situation is nothing. Sr. O Shea Jackson 5.4 The Poisson Distribution. Description of the Poisson Distribution Discrete probability distribution. The random variable
Διαβάστε περισσότεραΚΥΠΡΙΑΚΗ ΕΤΑΙΡΕΙΑ ΠΛΗΡΟΦΟΡΙΚΗΣ CYPRUS COMPUTER SOCIETY ΠΑΓΚΥΠΡΙΟΣ ΜΑΘΗΤΙΚΟΣ ΔΙΑΓΩΝΙΣΜΟΣ ΠΛΗΡΟΦΟΡΙΚΗΣ 6/5/2006
Οδηγίες: Να απαντηθούν όλες οι ερωτήσεις. Ολοι οι αριθμοί που αναφέρονται σε όλα τα ερωτήματα είναι μικρότεροι το 1000 εκτός αν ορίζεται διαφορετικά στη διατύπωση του προβλήματος. Διάρκεια: 3,5 ώρες Καλή
Διαβάστε περισσότεραHOMEWORK 4 = G. In order to plot the stress versus the stretch we define a normalized stretch:
HOMEWORK 4 Problem a For the fast loading case, we want to derive the relationship between P zz and λ z. We know that the nominal stress is expressed as: P zz = ψ λ z where λ z = λ λ z. Therefore, applying
Διαβάστε περισσότεραApproximation of distance between locations on earth given by latitude and longitude
Approximation of distance between locations on earth given by latitude and longitude Jan Behrens 2012-12-31 In this paper we shall provide a method to approximate distances between two points on earth
Διαβάστε περισσότερα3.4 SUM AND DIFFERENCE FORMULAS. NOTE: cos(α+β) cos α + cos β cos(α-β) cos α -cos β
3.4 SUM AND DIFFERENCE FORMULAS Page Theorem cos(αβ cos α cos β -sin α cos(α-β cos α cos β sin α NOTE: cos(αβ cos α cos β cos(α-β cos α -cos β Proof of cos(α-β cos α cos β sin α Let s use a unit circle
Διαβάστε περισσότεραΚΥΠΡΙΑΚΗ ΕΤΑΙΡΕΙΑ ΠΛΗΡΟΦΟΡΙΚΗΣ CYPRUS COMPUTER SOCIETY ΠΑΓΚΥΠΡΙΟΣ ΜΑΘΗΤΙΚΟΣ ΔΙΑΓΩΝΙΣΜΟΣ ΠΛΗΡΟΦΟΡΙΚΗΣ 19/5/2007
Οδηγίες: Να απαντηθούν όλες οι ερωτήσεις. Αν κάπου κάνετε κάποιες υποθέσεις να αναφερθούν στη σχετική ερώτηση. Όλα τα αρχεία που αναφέρονται στα προβλήματα βρίσκονται στον ίδιο φάκελο με το εκτελέσιμο
Διαβάστε περισσότεραTMA4115 Matematikk 3
TMA4115 Matematikk 3 Andrew Stacey Norges Teknisk-Naturvitenskapelige Universitet Trondheim Spring 2010 Lecture 12: Mathematics Marvellous Matrices Andrew Stacey Norges Teknisk-Naturvitenskapelige Universitet
Διαβάστε περισσότεραSecond Order RLC Filters
ECEN 60 Circuits/Electronics Spring 007-0-07 P. Mathys Second Order RLC Filters RLC Lowpass Filter A passive RLC lowpass filter (LPF) circuit is shown in the following schematic. R L C v O (t) Using phasor
Διαβάστε περισσότεραChapter 6: Systems of Linear Differential. be continuous functions on the interval
Chapter 6: Systems of Linear Differential Equations Let a (t), a 2 (t),..., a nn (t), b (t), b 2 (t),..., b n (t) be continuous functions on the interval I. The system of n first-order differential equations
Διαβάστε περισσότεραSrednicki Chapter 55
Srednicki Chapter 55 QFT Problems & Solutions A. George August 3, 03 Srednicki 55.. Use equations 55.3-55.0 and A i, A j ] = Π i, Π j ] = 0 (at equal times) to verify equations 55.-55.3. This is our third
Διαβάστε περισσότεραJesse Maassen and Mark Lundstrom Purdue University November 25, 2013
Notes on Average Scattering imes and Hall Factors Jesse Maassen and Mar Lundstrom Purdue University November 5, 13 I. Introduction 1 II. Solution of the BE 1 III. Exercises: Woring out average scattering
Διαβάστε περισσότεραEcon 2110: Fall 2008 Suggested Solutions to Problem Set 8 questions or comments to Dan Fetter 1
Eon : Fall 8 Suggested Solutions to Problem Set 8 Email questions or omments to Dan Fetter Problem. Let X be a salar with density f(x, θ) (θx + θ) [ x ] with θ. (a) Find the most powerful level α test
Διαβάστε περισσότεραOther Test Constructions: Likelihood Ratio & Bayes Tests
Other Test Constructions: Likelihood Ratio & Bayes Tests Side-Note: So far we have seen a few approaches for creating tests such as Neyman-Pearson Lemma ( most powerful tests of H 0 : θ = θ 0 vs H 1 :
Διαβάστε περισσότεραSolutions to Exercise Sheet 5
Solutions to Eercise Sheet 5 jacques@ucsd.edu. Let X and Y be random variables with joint pdf f(, y) = 3y( + y) where and y. Determine each of the following probabilities. Solutions. a. P (X ). b. P (X
Διαβάστε περισσότεραProblem Set 9 Solutions. θ + 1. θ 2 + cotθ ( ) sinθ e iφ is an eigenfunction of the ˆ L 2 operator. / θ 2. φ 2. sin 2 θ φ 2. ( ) = e iφ. = e iφ cosθ.
Chemistry 362 Dr Jean M Standard Problem Set 9 Solutions The ˆ L 2 operator is defined as Verify that the angular wavefunction Y θ,φ) Also verify that the eigenvalue is given by 2! 2 & L ˆ 2! 2 2 θ 2 +
Διαβάστε περισσότεραNotes on the Open Economy
Notes on the Open Econom Ben J. Heijdra Universit of Groningen April 24 Introduction In this note we stud the two-countr model of Table.4 in more detail. restated here for convenience. The model is Table.4.
Διαβάστε περισσότεραThe ε-pseudospectrum of a Matrix
The ε-pseudospectrum of a Matrix Feb 16, 2015 () The ε-pseudospectrum of a Matrix Feb 16, 2015 1 / 18 1 Preliminaries 2 Definitions 3 Basic Properties 4 Computation of Pseudospectrum of 2 2 5 Problems
Διαβάστε περισσότεραMATH423 String Theory Solutions 4. = 0 τ = f(s). (1) dτ ds = dxµ dτ f (s) (2) dτ 2 [f (s)] 2 + dxµ. dτ f (s) (3)
1. MATH43 String Theory Solutions 4 x = 0 τ = fs). 1) = = f s) ) x = x [f s)] + f s) 3) equation of motion is x = 0 if an only if f s) = 0 i.e. fs) = As + B with A, B constants. i.e. allowe reparametrisations
Διαβάστε περισσότεραWritten Examination. Antennas and Propagation (AA ) April 26, 2017.
Written Examination Antennas and Propagation (AA. 6-7) April 6, 7. Problem ( points) Let us consider a wire antenna as in Fig. characterized by a z-oriented linear filamentary current I(z) = I cos(kz)ẑ
Διαβάστε περισσότεραReminders: linear functions
Reminders: linear functions Let U and V be vector spaces over the same field F. Definition A function f : U V is linear if for every u 1, u 2 U, f (u 1 + u 2 ) = f (u 1 ) + f (u 2 ), and for every u U
Διαβάστε περισσότερα상대론적고에너지중이온충돌에서 제트입자와관련된제동복사 박가영 인하대학교 윤진희교수님, 권민정교수님
상대론적고에너지중이온충돌에서 제트입자와관련된제동복사 박가영 인하대학교 윤진희교수님, 권민정교수님 Motivation Bremsstrahlung is a major rocess losing energies while jet articles get through the medium. BUT it should be quite different from low energy
Διαβάστε περισσότεραΑπόκριση σε Μοναδιαία Ωστική Δύναμη (Unit Impulse) Απόκριση σε Δυνάμεις Αυθαίρετα Μεταβαλλόμενες με το Χρόνο. Απόστολος Σ.
Απόκριση σε Δυνάμεις Αυθαίρετα Μεταβαλλόμενες με το Χρόνο The time integral of a force is referred to as impulse, is determined by and is obtained from: Newton s 2 nd Law of motion states that the action
Διαβάστε περισσότεραderivation of the Laplacian from rectangular to spherical coordinates
derivation of the Laplacian from rectangular to spherical coordinates swapnizzle 03-03- :5:43 We begin by recognizing the familiar conversion from rectangular to spherical coordinates (note that φ is used
Διαβάστε περισσότεραSection 8.3 Trigonometric Equations
99 Section 8. Trigonometric Equations Objective 1: Solve Equations Involving One Trigonometric Function. In this section and the next, we will exple how to solving equations involving trigonometric functions.
Διαβάστε περισσότερα= λ 1 1 e. = λ 1 =12. has the properties e 1. e 3,V(Y
Stat 50 Homework Solutions Spring 005. (a λ λ λ 44 (b trace( λ + λ + λ 0 (c V (e x e e λ e e λ e (λ e by definition, the eigenvector e has the properties e λ e and e e. (d λ e e + λ e e + λ e e 8 6 4 4
Διαβάστε περισσότερα6.1. Dirac Equation. Hamiltonian. Dirac Eq.
6.1. Dirac Equation Ref: M.Kaku, Quantum Field Theory, Oxford Univ Press (1993) η μν = η μν = diag(1, -1, -1, -1) p 0 = p 0 p = p i = -p i p μ p μ = p 0 p 0 + p i p i = E c 2 - p 2 = (m c) 2 H = c p 2
Διαβάστε περισσότεραSection 7.6 Double and Half Angle Formulas
09 Section 7. Double and Half Angle Fmulas To derive the double-angles fmulas, we will use the sum of two angles fmulas that we developed in the last section. We will let α θ and β θ: cos(θ) cos(θ + θ)
Διαβάστε περισσότεραω ω ω ω ω ω+2 ω ω+2 + ω ω ω ω+2 + ω ω+1 ω ω+2 2 ω ω ω ω ω ω ω ω+1 ω ω2 ω ω2 + ω ω ω2 + ω ω ω ω2 + ω ω+1 ω ω2 + ω ω+1 + ω ω ω ω2 + ω
0 1 2 3 4 5 6 ω ω + 1 ω + 2 ω + 3 ω + 4 ω2 ω2 + 1 ω2 + 2 ω2 + 3 ω3 ω3 + 1 ω3 + 2 ω4 ω4 + 1 ω5 ω 2 ω 2 + 1 ω 2 + 2 ω 2 + ω ω 2 + ω + 1 ω 2 + ω2 ω 2 2 ω 2 2 + 1 ω 2 2 + ω ω 2 3 ω 3 ω 3 + 1 ω 3 + ω ω 3 +
Διαβάστε περισσότεραInverse trigonometric functions & General Solution of Trigonometric Equations. ------------------ ----------------------------- -----------------
Inverse trigonometric functions & General Solution of Trigonometric Equations. 1. Sin ( ) = a) b) c) d) Ans b. Solution : Method 1. Ans a: 17 > 1 a) is rejected. w.k.t Sin ( sin ) = d is rejected. If sin
Διαβάστε περισσότεραEvery set of first-order formulas is equivalent to an independent set
Every set of first-order formulas is equivalent to an independent set May 6, 2008 Abstract A set of first-order formulas, whatever the cardinality of the set of symbols, is equivalent to an independent
Διαβάστε περισσότεραSolution Series 9. i=1 x i and i=1 x i.
Lecturer: Prof. Dr. Mete SONER Coordinator: Yilin WANG Solution Series 9 Q1. Let α, β >, the p.d.f. of a beta distribution with parameters α and β is { Γ(α+β) Γ(α)Γ(β) f(x α, β) xα 1 (1 x) β 1 for < x
Διαβάστε περισσότεραTridiagonal matrices. Gérard MEURANT. October, 2008
Tridiagonal matrices Gérard MEURANT October, 2008 1 Similarity 2 Cholesy factorizations 3 Eigenvalues 4 Inverse Similarity Let α 1 ω 1 β 1 α 2 ω 2 T =......... β 2 α 1 ω 1 β 1 α and β i ω i, i = 1,...,
Διαβάστε περισσότεραQuadratic Expressions
Quadratic Expressions. The standard form of a quadratic equation is ax + bx + c = 0 where a, b, c R and a 0. The roots of ax + bx + c = 0 are b ± b a 4ac. 3. For the equation ax +bx+c = 0, sum of the roots
Διαβάστε περισσότεραΕΛΛΗΝΙΚΗ ΔΗΜΟΚΡΑΤΙΑ ΠΑΝΕΠΙΣΤΗΜΙΟ ΚΡΗΤΗΣ. Ψηφιακή Οικονομία. Διάλεξη 7η: Consumer Behavior Mαρίνα Μπιτσάκη Τμήμα Επιστήμης Υπολογιστών
ΕΛΛΗΝΙΚΗ ΔΗΜΟΚΡΑΤΙΑ ΠΑΝΕΠΙΣΤΗΜΙΟ ΚΡΗΤΗΣ Ψηφιακή Οικονομία Διάλεξη 7η: Consumer Behavior Mαρίνα Μπιτσάκη Τμήμα Επιστήμης Υπολογιστών Τέλος Ενότητας Χρηματοδότηση Το παρόν εκπαιδευτικό υλικό έχει αναπτυχθεί
Διαβάστε περισσότεραThe challenges of non-stable predicates
The challenges of non-stable predicates Consider a non-stable predicate Φ encoding, say, a safety property. We want to determine whether Φ holds for our program. The challenges of non-stable predicates
Διαβάστε περισσότεραEE101: Resonance in RLC circuits
EE11: Resonance in RLC circuits M. B. Patil mbatil@ee.iitb.ac.in www.ee.iitb.ac.in/~sequel Deartment of Electrical Engineering Indian Institute of Technology Bombay I V R V L V C I = I m = R + jωl + 1/jωC
Διαβάστε περισσότερα4.6 Autoregressive Moving Average Model ARMA(1,1)
84 CHAPTER 4. STATIONARY TS MODELS 4.6 Autoregressive Moving Average Model ARMA(,) This section is an introduction to a wide class of models ARMA(p,q) which we will consider in more detail later in this
Διαβάστε περισσότεραParametrized Surfaces
Parametrized Surfaces Recall from our unit on vector-valued functions at the beginning of the semester that an R 3 -valued function c(t) in one parameter is a mapping of the form c : I R 3 where I is some
Διαβάστε περισσότεραGeneral 2 2 PT -Symmetric Matrices and Jordan Blocks 1
General 2 2 PT -Symmetric Matrices and Jordan Blocks 1 Qing-hai Wang National University of Singapore Quantum Physics with Non-Hermitian Operators Max-Planck-Institut für Physik komplexer Systeme Dresden,
Διαβάστε περισσότεραΠανεπιστήµιο Κρήτης - Τµήµα Επιστήµης Υπολογιστών. ΗΥ-570: Στατιστική Επεξεργασία Σήµατος. ιδάσκων : Α. Μουχτάρης. εύτερη Σειρά Ασκήσεων.
Πανεπιστήµιο Κρήτης - Τµήµα Επιστήµης Υπολογιστών ΗΥ-570: Στατιστική Επεξεργασία Σήµατος 2015 ιδάσκων : Α. Μουχτάρης εύτερη Σειρά Ασκήσεων Λύσεις Ασκηση 1. 1. Consder the gven expresson for R 1/2 : R 1/2
Διαβάστε περισσότεραMath221: HW# 1 solutions
Math: HW# solutions Andy Royston October, 5 7.5.7, 3 rd Ed. We have a n = b n = a = fxdx = xdx =, x cos nxdx = x sin nx n sin nxdx n = cos nx n = n n, x sin nxdx = x cos nx n + cos nxdx n cos n = + sin
Διαβάστε περισσότεραCongruence Classes of Invertible Matrices of Order 3 over F 2
International Journal of Algebra, Vol. 8, 24, no. 5, 239-246 HIKARI Ltd, www.m-hikari.com http://dx.doi.org/.2988/ija.24.422 Congruence Classes of Invertible Matrices of Order 3 over F 2 Ligong An and
Διαβάστε περισσότεραΜηχανική Μάθηση Hypothesis Testing
ΕΛΛΗΝΙΚΗ ΔΗΜΟΚΡΑΤΙΑ ΠΑΝΕΠΙΣΤΗΜΙΟ ΚΡΗΤΗΣ Μηχανική Μάθηση Hypothesis Testing Γιώργος Μπορμπουδάκης Τμήμα Επιστήμης Υπολογιστών Procedure 1. Form the null (H 0 ) and alternative (H 1 ) hypothesis 2. Consider
Διαβάστε περισσότεραProblem Set 3: Solutions
CMPSCI 69GG Applied Information Theory Fall 006 Problem Set 3: Solutions. [Cover and Thomas 7.] a Define the following notation, C I p xx; Y max X; Y C I p xx; Ỹ max I X; Ỹ We would like to show that C
Διαβάστε περισσότεραFourier Series. MATH 211, Calculus II. J. Robert Buchanan. Spring Department of Mathematics
Fourier Series MATH 211, Calculus II J. Robert Buchanan Department of Mathematics Spring 2018 Introduction Not all functions can be represented by Taylor series. f (k) (c) A Taylor series f (x) = (x c)
Διαβάστε περισσότεραThe Jordan Form of Complex Tridiagonal Matrices
The Jordan Form of Complex Tridiagonal Matrices Ilse Ipsen North Carolina State University ILAS p.1 Goal Complex tridiagonal matrix α 1 β 1. γ T = 1 α 2........ β n 1 γ n 1 α n Jordan decomposition T =
Διαβάστε περισσότεραBlock Ciphers Modes. Ramki Thurimella
Block Ciphers Modes Ramki Thurimella Only Encryption I.e. messages could be modified Should not assume that nonsensical messages do no harm Always must be combined with authentication 2 Padding Must be
Διαβάστε περισσότεραg-selberg integrals MV Conjecture An A 2 Selberg integral Summary Long Live the King Ole Warnaar Department of Mathematics Long Live the King
Ole Warnaar Department of Mathematics g-selberg integrals The Selberg integral corresponds to the following k-dimensional generalisation of the beta integral: D Here and k t α 1 i (1 t i ) β 1 1 i
Διαβάστε περισσότεραANSWERSHEET (TOPIC = DIFFERENTIAL CALCULUS) COLLECTION #2. h 0 h h 0 h h 0 ( ) g k = g 0 + g 1 + g g 2009 =?
Teko Classes IITJEE/AIEEE Maths by SUHAAG SIR, Bhopal, Ph (0755) 3 00 000 www.tekoclasses.com ANSWERSHEET (TOPIC DIFFERENTIAL CALCULUS) COLLECTION # Question Type A.Single Correct Type Q. (A) Sol least
Διαβάστε περισσότεραOn a four-dimensional hyperbolic manifold with finite volume
BULETINUL ACADEMIEI DE ŞTIINŢE A REPUBLICII MOLDOVA. MATEMATICA Numbers 2(72) 3(73), 2013, Pages 80 89 ISSN 1024 7696 On a four-dimensional hyperbolic manifold with finite volume I.S.Gutsul Abstract. In
Διαβάστε περισσότεραFinite difference method for 2-D heat equation
Finite difference method for 2-D heat equation Praveen. C praveen@math.tifrbng.res.in Tata Institute of Fundamental Research Center for Applicable Mathematics Bangalore 560065 http://math.tifrbng.res.in/~praveen
Διαβάστε περισσότεραLecture 34 Bootstrap confidence intervals
Lecture 34 Bootstrap confidence intervals Confidence Intervals θ: an unknown parameter of interest We want to find limits θ and θ such that Gt = P nˆθ θ t If G 1 1 α is known, then P θ θ = P θ θ = 1 α
Διαβάστε περισσότεραProbability and Random Processes (Part II)
Probability and Random Processes (Part II) 1. If the variance σ x of d(n) = x(n) x(n 1) is one-tenth the variance σ x of a stationary zero-mean discrete-time signal x(n), then the normalized autocorrelation
Διαβάστε περισσότεραLecture 10 - Representation Theory III: Theory of Weights
Lecture 10 - Representation Theory III: Theory of Weights February 18, 2012 1 Terminology One assumes a base = {α i } i has been chosen. Then a weight Λ with non-negative integral Dynkin coefficients Λ
Διαβάστε περισσότεραΚΥΠΡΙΑΚΟΣ ΣΥΝΔΕΣΜΟΣ ΠΛΗΡΟΦΟΡΙΚΗΣ CYPRUS COMPUTER SOCIETY 21 ος ΠΑΓΚΥΠΡΙΟΣ ΜΑΘΗΤΙΚΟΣ ΔΙΑΓΩΝΙΣΜΟΣ ΠΛΗΡΟΦΟΡΙΚΗΣ Δεύτερος Γύρος - 30 Μαρτίου 2011
Διάρκεια Διαγωνισμού: 3 ώρες Απαντήστε όλες τις ερωτήσεις Μέγιστο Βάρος (20 Μονάδες) Δίνεται ένα σύνολο από N σφαιρίδια τα οποία δεν έχουν όλα το ίδιο βάρος μεταξύ τους και ένα κουτί που αντέχει μέχρι
Διαβάστε περισσότεραHigher Derivative Gravity Theories
Higher Derivative Gravity Theories Black Holes in AdS space-times James Mashiyane Supervisor: Prof Kevin Goldstein University of the Witwatersrand Second Mandelstam, 20 January 2018 James Mashiyane WITS)
Διαβάστε περισσότεραTesting for Indeterminacy: An Application to U.S. Monetary Policy. Technical Appendix
Testing for Indeterminacy: An Application to U.S. Monetary Policy Technical Appendix Thomas A. Lubik Department of Economics Johns Hopkins University Frank Schorfheide Department of Economics University
Διαβάστε περισσότεραHISTOGRAMS AND PERCENTILES What is the 25 th percentile of a histogram? What is the 50 th percentile for the cigarette histogram?
HISTOGRAMS AND PERCENTILES What is the 25 th percentile of a histogram? The point on the horizontal axis such that of the area under the histogram lies to the left of that point (and to the right) What
Διαβάστε περισσότεραAreas and Lengths in Polar Coordinates
Kiryl Tsishchanka Areas and Lengths in Polar Coordinates In this section we develop the formula for the area of a region whose boundary is given by a polar equation. We need to use the formula for the
Διαβάστε περισσότεραk A = [k, k]( )[a 1, a 2 ] = [ka 1,ka 2 ] 4For the division of two intervals of confidence in R +
Chapter 3. Fuzzy Arithmetic 3- Fuzzy arithmetic: ~Addition(+) and subtraction (-): Let A = [a and B = [b, b in R If x [a and y [b, b than x+y [a +b +b Symbolically,we write A(+)B = [a (+)[b, b = [a +b
Διαβάστε περισσότεραLecture 13 - Root Space Decomposition II
Lecture 13 - Root Space Decomposition II October 18, 2012 1 Review First let us recall the situation. Let g be a simple algebra, with maximal toral subalgebra h (which we are calling a CSA, or Cartan Subalgebra).
Διαβάστε περισσότεραforms This gives Remark 1. How to remember the above formulas: Substituting these into the equation we obtain with
Week 03: C lassification of S econd- Order L inear Equations In last week s lectures we have illustrated how to obtain the general solutions of first order PDEs using the method of characteristics. We
Διαβάστε περισσότεραForced Pendulum Numerical approach
Numerical approach UiO April 8, 2014 Physical problem and equation We have a pendulum of length l, with mass m. The pendulum is subject to gravitation as well as both a forcing and linear resistance force.
Διαβάστε περισσότεραPg The perimeter is P = 3x The area of a triangle is. where b is the base, h is the height. In our case b = x, then the area is
Pg. 9. The perimeter is P = The area of a triangle is A = bh where b is the base, h is the height 0 h= btan 60 = b = b In our case b =, then the area is A = = 0. By Pythagorean theorem a + a = d a a =
Διαβάστε περισσότεραΜΕΤΑΠΤΥΧΙΑΚΗ ΔΙΠΛΩΜΑΤΙΚΗ ΕΡΓΑΣΙΑ «ΘΕΜΑ»
ΠΑΝΕΠΙΣΤΗΜΙΟ ΑΙΓΑΙΟΥ ΣΧΟΛΗ ΑΝΘΡΩΠΙΣΤΙΚΩΝ ΕΠΙΣΤΗΜΩΝ ΤΜΗΜΑ ΕΠΙΣΤΗΜΩΝ ΤΗΣ ΠΡΟΣΧΟΛΙΚΗΣ ΑΓΩΓΗΣ ΚΑΙ ΤΟΥ ΕΚΠΑΙΔΕΥΤΙΚΟΥ ΣΧΕΔΙΑΣΜΟΥ Π.Μ.Σ. «ΠΕΡΙΒΑΛΛΟΝΤΙΚΗ ΕΚΠΑΙΔΕΥΣΗ» ΜΕΤΑΠΤΥΧΙΑΚΗ ΔΙΠΛΩΜΑΤΙΚΗ ΕΡΓΑΣΙΑ «ΘΕΜΑ» «Εφαρμογή
Διαβάστε περισσότερα6.3 Forecasting ARMA processes
122 CHAPTER 6. ARMA MODELS 6.3 Forecasting ARMA processes The purpose of forecasting is to predict future values of a TS based on the data collected to the present. In this section we will discuss a linear
Διαβάστε περισσότεραMath 6 SL Probability Distributions Practice Test Mark Scheme
Math 6 SL Probability Distributions Practice Test Mark Scheme. (a) Note: Award A for vertical line to right of mean, A for shading to right of their vertical line. AA N (b) evidence of recognizing symmetry
Διαβάστε περισσότεραFractional Colorings and Zykov Products of graphs
Fractional Colorings and Zykov Products of graphs Who? Nichole Schimanski When? July 27, 2011 Graphs A graph, G, consists of a vertex set, V (G), and an edge set, E(G). V (G) is any finite set E(G) is
Διαβάστε περισσότεραD Alembert s Solution to the Wave Equation
D Alembert s Solution to the Wave Equation MATH 467 Partial Differential Equations J. Robert Buchanan Department of Mathematics Fall 2018 Objectives In this lesson we will learn: a change of variable technique
Διαβάστε περισσότερα2. THEORY OF EQUATIONS. PREVIOUS EAMCET Bits.
EAMCET-. THEORY OF EQUATIONS PREVIOUS EAMCET Bits. Each of the roots of the equation x 6x + 6x 5= are increased by k so that the new transformed equation does not contain term. Then k =... - 4. - Sol.
Διαβάστε περισσότεραb. Use the parametrization from (a) to compute the area of S a as S a ds. Be sure to substitute for ds!
MTH U341 urface Integrals, tokes theorem, the divergence theorem To be turned in Wed., Dec. 1. 1. Let be the sphere of radius a, x 2 + y 2 + z 2 a 2. a. Use spherical coordinates (with ρ a) to parametrize.
Διαβάστε περισσότεραΣυστήματα Διαχείρισης Βάσεων Δεδομένων
ΕΛΛΗΝΙΚΗ ΔΗΜΟΚΡΑΤΙΑ ΠΑΝΕΠΙΣΤΗΜΙΟ ΚΡΗΤΗΣ Συστήματα Διαχείρισης Βάσεων Δεδομένων Φροντιστήριο 9: Transactions - part 1 Δημήτρης Πλεξουσάκης Τμήμα Επιστήμης Υπολογιστών Tutorial on Undo, Redo and Undo/Redo
Διαβάστε περισσότεραHW 3 Solutions 1. a) I use the auto.arima R function to search over models using AIC and decide on an ARMA(3,1)
HW 3 Solutions a) I use the autoarima R function to search over models using AIC and decide on an ARMA3,) b) I compare the ARMA3,) to ARMA,0) ARMA3,) does better in all three criteria c) The plot of the
Διαβάστε περισσότεραCE 530 Molecular Simulation
C 53 olecular Siulation Lecture Histogra Reweighting ethods David. Kofke Departent of Cheical ngineering SUNY uffalo kofke@eng.buffalo.edu Histogra Reweighting ethod to cobine results taken at different
Διαβάστε περισσότεραthe total number of electrons passing through the lamp.
1. A 12 V 36 W lamp is lit to normal brightness using a 12 V car battery of negligible internal resistance. The lamp is switched on for one hour (3600 s). For the time of 1 hour, calculate (i) the energy
Διαβάστε περισσότεραMatrices and vectors. Matrix and vector. a 11 a 12 a 1n a 21 a 22 a 2n A = b 1 b 2. b m. R m n, b = = ( a ij. a m1 a m2 a mn. def
Matrices and vectors Matrix and vector a 11 a 12 a 1n a 21 a 22 a 2n A = a m1 a m2 a mn def = ( a ij ) R m n, b = b 1 b 2 b m Rm Matrix and vectors in linear equations: example E 1 : x 1 + x 2 + 3x 4 =
Διαβάστε περισσότερα