Σύγκριση είκτη Αιτιότητας κατά Granger και µεταφορικής εντροπίας και εφαρµογή σε προβλήµατα αγοράς

Transcript

1 ΑΡΙΣΤΟΤΕΛΕΙΟ ΠΑΝΕΠΙΣΤΗΜΙΟ ΘΕΣΣΑΛΟΝΙΚΗΣ ΣΧΟΛΗ ΘΕΤΙΚΩΝ ΕΠΙΣΤΗΜΩΝ ΤΜΗΜΑ ΜΑΘΗΜΑΤΙΚΩΝ ΜΕΤΑΠΤΥΧΙΑΚΟ ΠΡΟΓΡΑΜΜΑ ΣΠΟΥ ΩΝ ΣΤΑΤΙΣΤΙΚΗ ΚΑΙ ΜΟΝΤΕΛΟΠΟΙΗΣΗ Σύγκριση είκτη Αιτιότητας κατά Granger και µεταφορικής εντροπίας και εφαρµογή σε προβλήµατα αγοράς ΜΕΤΑΠΤΥΧΙΑΚΗ ΙΑΤΡΙΒΗ Κουτσούρη Ελεωνόρα, MSc Οικονοµολόγος Εγκρίθηκε από την τριµελή εξεταστική επιτροπή Κουγιουµτζής ηµήτρης Αν. Καθηγητής ΑΠΘ Μωυσιάδης Πολυχρόνης Καθηγητής ΑΠΘ Ιωαννίδης ηµήτριος Καθηγητής Παν. Μακεδονίας

2 ARISTOTLE UNIVERSITY OF THESSALONIKI FACULTY OF SCIENCES SCHOOL OF MATHEMATICS MASTER OF SCIENCES STATISTICS AND MODELLING A comparaive sudy on Granger Causaliy and Transfer Enropy Indices and an applicaion on consumer goods issues MASTER THESIS Kousouri Eleonora, MSc Economics Supervisor Kugiumzis Dimiris As. Professor AUTH February 203 Thessaloniki

3 Ελεωνόρα Κουτσούρη Οικονοµολόγος Ελεωνόρα Κουτσούρη, 203 Αριστοτέλειο Πανεπιστήµιο Θεσσαλονίκης, 203 Με επιφύλαξη παντός δικαιώµατος. All righs reserved. Απαγορεύεται η αντιγραφή, αποθήκευση και διανοµή της παρούσας εργασίας, εξ ολοκλήρου ή τµήµατος αυτής, για εµπορικό σκοπό. Επιτρέπεται η ανατύπωση, αποθήκευση και διανοµή για σκοπό µη κερδοσκοπικό, εκπαιδευτικής ή ερευνητικής φύσης, υπό την προϋπόθεση να αναφέρεται η πηγή προέλευσης και να διατηρείται το παρόν µήνυµα. Ερωτήµατα που αφορούν τη χρήση της εργασίας για κερδοσκοπικό σκοπό πρέπει να απευθύνονται προς το συγγραφέα. Οι απόψεις και τα συµπεράσµατα που περιέχονται σε αυτό το έγγραφο εκφράζουν το συγγραφέα και δεν πρέπει να ερµηνευτεί ότι εκφράζουν τις επίσηµες θέσεις του Α.Π.Θ. 2

4 Περίληψη Η αιτιότητα αποτελεί ένα ερώτημα που καταλαμβάνει κεντρική θέση στις έρευνες πολλών κλάδων τις τελευταίες δεκατετίες. Πολύ συχνά διατυπώνονται ερωτήματα που αφορούν στη σχέση μεταξύ των διερευνώμενων μεταβλητών, όπως και μια σειρά άλλων υποερωτημάτων σχετικά με την ένταση και τη διεύθυνση αυτής της σχέσης. Σημαντικό είναι να αναφερθεί πως δεν υπάρχει ένας ενιάιος και καθολικός ορισμός της έννοιας της αιτιότητας. Μέχρι και σήμερα οι ερευνητές χρησιμοποιούν την έννοια της αιτιότητας αλλά και διάφορους ορισμούς της όπως αυτοί έχουν κατά καιρούς διατυπωθεί, προκειμένουν να μελετήσουν και να κατανοήσουν την φύση και την συμπεριφορά των παρατηρήσεών τους. Οι διαφορετικοί ορισμοί, είναι εμπνευσμένοι από διαφορετικά πεδία, και διατυπώθηκαν κάθε φορά, ώστε να εξυπηρετήσουν την ακριβή φύση του ερευνητικού αντικειμένου. Στα τέλη της δεκαετίας του 960, οι οικονομολόγοι έθεσαν ερωτήματα για την αιτιότητα και για το κατά πόσο αυτή θα έπρεπε να συνυπολογίζεται κατά τη διαμόρφωση ερευνητικών μοντέλων. Ο πρώτος που ασχολήθηκε αναλυτικά με το θέμα της αιτιότητας και θεμελίωσε μεθοδολογία για την διερεύνηση της αιτιότητας μεταξύ δύο μεταβλητών ήταν ο Clive G. Granger (Granger, 969), αν και είχαν ήδη προηγηθεί μελέτες οι οποίες αναφερόντουσαν σε θεωρήτικό επίπεδο στο ζήτημα της αιτότητας (Wiener, 956). Η αιτιότητα, όπως αυτή ορίζεται από τον Granger, απαντάται όταν η διακύμανση ενός μονομεταβλητού μοντέλου για μια τυχαία μεταβλητή μειώνεται από την εισαγωγή παρελθοντικών τιμών μιας δεύτερης στο επεξηγηματικό κομμάτι του μοντελου της πρώτης. Η δουλειά του Granger έτυχε ευρείας αποδοχής από το ερευνητικό κοινό, και ονομάστηκε αιτιότητα κατά Granger. Μεταγενέστεροι ερευνητές επέκτειναν αυτή τη μεθοδολογία σε πολλαπλές χρονοσειρές, αλλά και σε πολυμεταβλητά συστήματα, αλλά και καθιέρωσαν 3

5 παραμετρικούς ελέγχους ώστε να καταστήσουν εφικτή τη στατιστική συμπερασματολογία στα αποτελέσματα του δείκτη αιτιότητας κατά Granger. Φυσικά υπήρξε και κριτική στο έργο του Granger, η οποία στηρίχθηκε κυρίως στο οτί ο ορισμός που δίνει o Granger στην αιτιότητα είναι πολύ συγκεκριμένος και φέρει αρκετούς περιορισμούς, μεταξύ των οποίων και η παραμετρική διατύπωση ενός μοντέλου, καθώς και η απαραίτητη στασιμότητα των χρονοσειρών που μελετούνται. Παρ όλ αυτά, η μεθοδολογία καθιερώθηκε και χρησιμοποιήθηκε ευρέως σε πολλά ερευνητικά πεδία χάρη στην απλή και εύχρηστή της διατύπωση. Πρόσφατα δε, το ενδιαφέρον της ερευνητικής κοινότητας για την αιτιότητα κατά Granger αναζωπυρώθηκε, καθώς το μέτρο αυτό αξιοποιήθηκε από τον κλάδο της νευροφυσιολογίας, προσφέροντας νέα ευρήματα στην ανάλυση των ηλεκροεγκεφαλογραφημάτων. Παράλληλα με τη χρήση του παραμετρικού ελέγχου, πρόσφατα έχουν προταθεί νέες μεθοδολογίες για τη διερεύνηση της αιτιότητας, οι οποίες έχουν βασιστεί σε μια εναλλακτική διατύπωση της αιτιότητας κι έχουν εμπνευστεί από τη θεωρία της πληροφορίας. Πιο συγκεκριμένα, αφετηρία αυτής της νέας θεώρησης αποτελέι το μέτρο της εντροπίας (Schreiber, 2000) στo οποίο στηρίχτηκε και διαμορφώθηκε το μέτρο της μεταφορικής εντροπίας, η οποία μελετάει την πληροφορία που περιλαμβάνει μια μεταβλητή Υ που χρησιμέυει ώστε να προβλεφτούν μελλοντικές τιμές της μεταβλητής Χ. Η ειδοποιός διαφορά, πέραν του ορισμού αυτού του νέου μέτρου είναι το πλαίσιο στο οποίο αναπτύσσεται, κάθως πρόκειται για ένα μη παραμετρικό μέτρο που δεν επιβάλλει περιορισμούς στις χρονοσειρές που μελετούνται. Ο σκοπός αυτής της εργασίας είναι παρουσιάσει τα κύρια σημεία της βιβλιογραφίας που έχει αναπτυχθεί γύρω από την αιτιότητα κατά Granger αλλά και την μεταφορική εντροπία, σε θεωρητικό πλαίσιο αλλά και σε πρακτικό επίπεδο. Κλασικά πλέον μέτρα, όπως ο δείκτης αιτιότητας κατά Granger, ο κατά συνθήκη δείκτης αιτιότητας κατά Granger, και ο μερικός δείκτης αιτιότητας κατά Granger, παρουσιάζονται μαζί με τις στατιστικές ιδιότητες τους. Σε αντιστοιχία, παρουσιάζεται το μέτρο της μεταφορικής εντροπίας, όπως αυτό έχει εισαχτεί στη 4

6 βιβλιογραφία, αλλά και το μέτρο της μερικής μεταφορικής εντροπίας, ως μια βελτίωση της πρώτης διατύπωσης. Στη συνέχεια τα μέτρα αυτά, παραμετρικά και μη, εφαρμόζονται σε συστήματα με κατασκευασμένα δεδομένα στα οποία ήδη γνωρίζουμε τις σχέσεις μεταξύ των μεταβλητών, προκειμένου να μελετηθεί η συμπεριφορά του κάθε ελέγχου και η επιτυχία διαπίστωσης των αιτιατών σχέσεων του συστήματος. Τα συμπεράσματα στα οποία οδηγηθήκαμε είναι πως και τα τέσσερα μέτρα έχουνε μεγάλο ποσοστό επιτυχίας σε δεδομένα που δημιοργούνται από ένα γραμμικό σύστημα, ή από ένα σύστημα που έχει υποστεί μια μικρή παραμόρφωση. Από την άλλη, το ποσοστό επιτυχίας μειώθηκε δραματικά σε δεδομένα που προήλθαν από μη γραμμικό σύστημα. Επίσης, αξιοσημείωτη ήταν η βελτίωση των αποτελεσμάτων με την χρήση των κατά συνθήκη δεικτών, καθώς τα λανθασμένα συμπεράσματα που προέρχονταν απο έμμεσες αιτιατές σχέσεις απαλείφτηκαν στις περιπτώσεις του γραμμικού και του παραμορφωμένου συστήματος. Τέλος, τα ίδια μέτρα αιτιότητας εφαρμόστηκαν σε πραγματικά δεδομένα που προέρχονται από την αγορά ταχυκίνητων αγαθών, και διερευνήθηκαν οι σχέσεις μεταξύ πέντε βασικών μεταβλητών που έχουν παγιωθεί στις σύγχρονες μεθόδους ανάλυσης μάρκετινγκ δεδομένων. Τα αποτελέσματα δεν είναι κοινά ανάμεσα στους τέσσερις δείκτες που χρησιμοποιήθηκαν, όμως αναδεικνύουν κάποιες βασικές σχέσεις μεταξύ των μεταβλητών που μελετήθηκαν. Λέξεις-Κλειδιά: Χρονοσειρές, Αιτιότητα κατά Granger, ΜεταφορικήΕντροπία, Ταχυκίνητα Αγαθά. 5

7 Absrac This hesis presens an overview of exising lieraure upon conceps of causaliy, boh in a parameric and an informaion heoreic conex. Proposed measures of Granger Causaliy and Transfer Enropy are inroduced and applied on simulaed daa where he causaions beween he generaed variables are a priori known o us, so as o explore he robusness and he fallacies of each measure. The same measures are applied on markeing daa, in an effor o verify cerain common hypoheses ha are commonly used in modern markeing analyics. Keywords: Time Series, Granger Causaliy, Transfer Enropy, FMCG. 6

8 Acknowledgmens I would like o hank my supervisor, Dr. Dimiris Kugiumzis, for his perseverance and valuable help during he wriing of his hesis. 7

9 Conens. Inroducion Time Series: Fundamenal Conceps Time Series Firs Mahemaical Momens and heir Sample Esimaions Populaion Measures... 4 Mean & Variance... 4 Auo Covariance & Auo Correlaion... 4 Cross Covariance & Cross Correlaion... 4 Parial Correlaion Sample Esimaors... 5 Sample Mean & Variance... 5 Sample Auo Covariance & Auo Correlaion... 6 Sample Cross Covariance & Cross Correlaion... 6 Sample Parial Correlaion Time series models... 7 Iid noise... 7 Random walk... 7 Whie noise... 7 Auoregressive process... 8 Moving average process Order Selecion for a model Saionariy The Uni roo problem & Uni Roo Tess Wold Represenaion Theorem Univariae and Mulivariae Time Series

10 Mean & Variance Auo Covariance & Auo Correlaion Cross Covariance & Cross Correlaion Parial Correlaion Sample Mean & Variance Sample Auo Covariance & Auo Correlaion Sample Cross Covariance & Cross Correlaion Sample Parial Correlaion Causaliy Granger Causaliy Condiional Granger Causaliy Parial Granger Causaliy Transfer Enropy Enropy as an informaion meric Transfer Enropy-An Indicaor Parial Transfer Enropy-An Indicaor Saisical Inference on TE and PTE Granger Causaliy and Transfer Enropy on Simulaed Daa Inroducion The esing procedure Linear Sysem Disored linear sysem Non-linear sysem Discussion Applicaion Inroducion

11 6.2. The Applicaion Se Up Uni Sales TPR- Weighed Average Price Reducion Avg Iems/Sore Selling ACV Weighed Disribuion Feaure and Display Price Formulaion of our Expecaions Resuls Overview of Resuls Granger Causaliy Condiional Granger Causaliy Transfer Enropy Parial Transfer Enropy Conclusions Summary References

12 . Inroducion Causaliy has always been a crucial opic for research. Quesions as he exisence of conneciviy beween wo observed variables, he direcion of he link beween hem and he exen o which hey relae, have been a he cener of discussions for many years; unil oday, here is sill heaed debae around his opic, and researchers coninue suggesing new ways of discovering causal links wihin a sochasic environmen. Granger - despie he fac ha his work has been preceded by ohers- has been he firs o horoughly address he issue of causaion wih formal analyical represenaions and mahemaical proof upon ime series. He herefore coined he erm of Granger causaliy, a erm ha describes a very specific form of causaliy beween wo or more variables, and ha iniially was used in economic research. Nowadays, Granger causaliy erm has become a useful and powerful ool in he hands of researchers across many disciplines. In he laes years, Granger causaliy is exensively used in he neurobiology field, in an aemp o decode EEGs (Elecroencephalograms). More recenly, Τransfer Εnropy was inroduced, accouning for causaion in an alernaive way: in a broader sense of informaion direcion. This new meric sems from informaion heory field, which is exensively used in signal analysis. The main purpose of his hesis is o presen main poins from lieraure upon Granger causaliy and Τransfer Εnropy merics, as well as o apply four causaliy ess on acual FMCG (Fas Moving Consumer Goods Daa), in an effor o ouline similariies and differences across all used ess; boh parameric and informaion-heoreic ones. The res of he hesis is organized as follows: Chaper 2 presens some inroducory conceps from Time Series Theory, necessary for he undersanding of he following chapers. Chaper 3 presens he Granger Causaliy concep, ogeher wih parameric ess ha can be used o explore causaliy relaions wihin a parameric sysem. Chaper 4 gives an overview of he lieraure upon Transfer Enropy so far. In Chaper 5, we apply four causaliy ess on simulaed daa, in order o idenify he measure capabiliies as well as

13 some of heir inefficiencies. In Chaper 6, we use he same four causaliy measures on acual markeing daa from he FMCG (Fas Moving Consumer Goods) secor, o discover causal links among five variables. Finally, in Chaper 7 we summarize our work. 2

14 2. Time Series: Fundamenal Conceps 2.. Time Series Time Series is a se of x observaions of a random variable X, each recorded a ime poin, wih aking values,.,n. A ime series can be discree or coninuous as i can consis from eiher T0 discree values or from coninuous recordings wihin a fixed ime period T0 accordingly. The sudy of ime series can prove very helpful as a paern of he underlying process can emerge, enabling he researcher o build a model ha will explain he behavior of he variable X. This is why ime series have been exensively used across many fields, such as engineering, financial forecasing, signal processing ec. A ime series model is a formal represenaion which specifies he oin disribuions (or mos commonly, only up o second order properies) of he x realizaions of a random variable X. Therefore, a valid model specificaion lies on our abiliy o define all oin disribuions of he random vecors (X,X2, Xn) for n=,2, or equivalenly of probabiliies P(X x,x2 x2, Xn xn) for - < x,,xn< and n=,2, Alernaively (and significanly easier) i is sufficien o specify he firs and second order momens (second order properies) of he oin disribuions of he random vecors (X,X2, Xn) for n=,2,, meaning ha we can formulae our expecaions regarding he mean and he variance of he underlying {X} process. 3

15 2.2. Firs Mahemaical Momens and heir Sample Esimaions The mahemaical momens of a random variable X, are measures which are used o describe he properies of is disribuion Populaion Measures Mean & Variance The firs momen is used o describe is populaion mean, E[X ] = µ Χ whereas he second momen is mos usually referred o as variance, σ = var[x ] = E[(X ) ] 2 2 X µ Χ Auo Covariance & Auo Correlaion Furher, we will also define he covariance funcion beween by a ime lag h as γ ( h) = Ε( Χ µ )( Χ µ ), X + h Χ Χ Χ and + h Χ which differ only as well as he correlaion funcion beween Χ and + h Χ as γ X ( h) Ε( Χ+ h µ Χ )( Χ µ Χ ) ρ X ( h) = =. 2 γ (0) σ X Cross Covariance & Cross Correlaion The laer wo measures can also be used o measure he covariance and correlaion beween wo random variables Χ and + h Y, and hen he measures will ake he following form : γ ( h) = Ε[( Χ µ )( Y µ )] XY + h X Y 4

16 γ XY ( h) Ε[( Χ+ h µ X )( Y µ Y )] ρxy ( h) = = γ (0) γ (0) σ σ 2 2 XX YY X Y Parial Correlaion Parial correlaion is a erm used o describe he associaion beween wo variables, while a he same ime aking ino accoun he effec of oher conrolling variables. Le us consider he case where we have hree variables, X, Y, and Z, and we wan o find ou he connecion beween X and Y, while a he same ime we conrol for he any effecs ha migh be driven from Z. ρ XY Ζ ( h) = ρ ( h) ρ ( h) ρ ( h) XY XZ ZY 2 2 ρ XZ (0) ρ ZY (0) Sample Esimaors In mos cases, i is improbable on real applicaions o have access o he whole populaion in order o calculae he firs momens. So, insead of ha, we use a subse of he populaion called sample, and esimae he momen parameers. Therefore, for n observaions in our sample and of course as before, h being he ime lag beween wo ime poins we define: Sample Mean & Variance n x n = x =, he sample variance, 5

17 n 2 sx = ( x x) n = Sample Auo Covariance & Auo Correlaion n h c ( h) = ( x x)( x x) X + h n h = And similarly, he sample auocorrelaion funcion, c rx ( h) = c X X ( h) (0) Sample Cross Covariance & Cross Correlaion We also consider he sample covariance and correlaion beween wo differen random variables X and Y n h c ( h) = ( x x)( y y) XY + h n h = r XY ( h) = c c XY ( h) (0) c (0) 2 2 XX YY Sample Parial Correlaion r XY Ζ ( h) = r ( h) r ( h) r ( h) XY XZ ZY 2 2 r XZ (0) r ZY (0) 6

18 2.3. Time series models In his secion, we presen some ime series models, which are frequenly encounered in ime-series analysis and also used in he following chapers. Iid noise This model is used o describe independen and idenically disribued realizaions of random variables wih zero mean. Since he observaions are independen, i holds ha ( ) ( ) P(X x, X x, X x ) = P(X x ) P X x P X x 2 2 n n 2 2 n n wih E[Χ ] = 0, Random walk Random walk is a sequence S X X 2.. X = of random iid variables { } X = X.. X, where =,2,,n 2 wih E[Χ ] = 0, and var[χ ] = σ Whie noise Whie noise is sequence { ε } of uncorrelaed random variables, wih E [ε ] = 0, and var[ε ] = σ 2 ε 7

19 Auoregressive process An auoregressive process AR(p), is defined is a following: X = ϕ X + ϕ X + + ϕ X + ε, p p where p represens he order of model, showing he number of lags ha explain he behavior of series X a ime, ϕ,..., ϕ pare he coefficiens of he process, ha can be grouped under he noaion ϕ, wih =, 2,..., p deermining he degree o which each X componen affecs he behavior of he process X a he presen. ε represens a whie noise procedure wih zero mean and sandard variance 2 ~ (0, ) ε WN σ ε Wih he help of a back shif operaor B X X =, we can rewrie an AR(p) process as an poluonym expression X = ϕ X + ϕ X ϕ X + ε 2 2 p p p ( ϕ B ϕ B... ϕ B ) X = ε ϕ( B) X 2 2 = ε p where ϕ( B) p = = ϕ B Therefore an auoregressive series of order p can be represened in he following compac formulaion: X p = ϕ X + ε = 8

20 Moving average process A moving average process of order q can be represened in he following way X = θε θ qε q + ε, Where θ,... θ q are real numbers represening he coefficiens deermining he exen o which each pas value of he process deermines is curren values, q is he order of he model, which in accordance o he AR(p) specificaion, represens he number of lags ha explain he behavior of series X a ime, and of course ε represens a whie noise procedure wih zero mean and sandard variance 2 ~ (0, σ ) ε WN ε 9

21 2.4. Order Selecion for a model One of he parameers of he model o be esimaed is he order of he model, meaning he p ime lags ha need o be included in he model so ha he dependen variable is adequaely explained. There exis a number of mehodologies o selec he appropriae order for a model, such as he AIC crierion, which selecs an appropriae order for our model by accouning for he radeoff beween he number of variables which add explanaory power in our model, bu ha also add variance ino i. AIC=-2ln(LLN)+2g, where g is he number of explanaory variables insered ino he model, and LLN is he maximized value of he Log Likelihood Funcion. Alernaively, one can use he Bayesian Informaion Crierion (BIC), which penalizes harder he number of independen variables insered ino he model. One can also chose he appropriae order by employing empirical mehods, and examining he auocorrelaion and parial auocorrelaion graphs, because he parial auocorrelaion funcion urns o zero for lags greaer han he order of he model. 20

22 2.5. Saionariy We can call a ime series saionary, when is properies do no change over ime. The main inuiion behind his is ha he sysem will give us back he same informaion a any given ime-poin, allowing us hus o apply several generalizing assumpions for he underlying process. There exis wo forms of saionariy, he sric and he weak saionariy, however weak saionariy is usually sufficien o allow he sudy of he respecive ime series. A formal definiion of weak saionariy: Le X be a sochasic ime series wih E[ X ] = µ, a consan, ime-invarian mean, 2 2 var[ ] [( ) ] X = E X µ = σ, a consan variance, var[ X ] = E[( X µ )( X µ ] = γ, a seady covariance, + h h Then, X can be considered covariance saionary process, for all ime poins. There are more han one ways o es wheher a series is saionary. An experienced eye can recognize a mean revering procedure only by is graphical represenaion. There are also more formal ways o es wheher a series is saionary, called uni roos ess. Broadly speaking, a sochasic process is said o be saionary if is mean and variance are consan over ime and he value of he covariance beween he wo ime periods depends only on he disance or gap or lag beween he wo ime periods and no he acual ime a which he covariance is compued. In he ime series lieraure, such a sochasic process is known as a weakly saionary, or covariance saionary, or second-order saionary, or wide sense, sochasic process, (Guarai D., 2004) 2

23 2.6. The Uni roo problem & Uni Roo Tess In lieraure, a uni roo process is a non-saionary process. In he example below, we explain how a uni roo process is non-saionary. Le us consider he following AR () process: X = ϕ X + ε, wih In he case where he coefficien φ akes he value of, hen our AR process becomes a random walk process, where all he new elemens sem from he innovaion erm, rendering us a non-predicable series. This case is broadly known in he lieraure as uni roo problem. If, however, φ < and he series has a consan mean μ for all ime poins, hen we can show ha he series fulfills weak saionariy condiions, as is mean will ake he following form: E( X ) = ϕe( X ) + E( ε ) µ = ϕµ + 0 µ = ϕ I is obvious now ha if φ = hen we canno have a mean. The same holds for he series variance provided we have a seady variance over ime. var( X ) = ϕ var( X ) + var( ε ) var( X ) = ϕ var( X ) + σ σε var( X ) = ϕ ε 22

24 There have been developed many ess o check he sabiliy of a process, esing wheher he esimaed parameer for φ is saisically differen han zero. Such a es, ha is widely used, is he Dickey Fuller es: Consider an AR(p) process ha includes an inercepα and a ime rend η, where η is he seady rae a which he series evolves hrough ime. X = α + η + ϕ X + ε The firs differences of his procedure will hen be X X = α + η + ( ϕ ) X + ε X = α + η + β X + ε, where β = ( ϕ ) And he es saisic will be ˆ ADF = β, which follows a non-sandard disribuion 2 s ( ˆ β) compued by Dickey & Fuller, wih H ˆ 0 : β = 0 and alernaive hypohesis H ˆ : β < 0, i.e. esing wheher φ is significanly differen han. An exension of his es is also he augmened Dickey Fuller es, which resores he disorion ha he model migh possibly include due o auocorrelaions in he error erm, wih he inclusion of lagged values of X in he model: Then he es on he same parameer β = ( ϕ ) is realized on he firs differences of he augmened version of he model: X = α + η + ϕ X + ϕ X + ε p i p i= X X X... p X p = α + η + β + ϕ + + ϕ + ε 23

25 A poin o consider while using he ADF es saisic is he selecion of he order p of he model. There exiss a radeoff beween he size and he power of he es, as a high order is beneficial for he es size, bu decreases our possibiliy of reecing he null hypohesis. We can use a crierion, such as he AIC or BIC crierion in order o choose he opimal order for he model. 24

26 2.7. Wold Represenaion Theorem The Wold s decomposiion heorem shows ha a weakly saionary series can be wrien in he form of an infinie moving average process. Consider he following process AR() process: X = ϕ X + ε, wih 2 ~ (0, ) ε WN σ ε Then he process can be re-wrien in an MA( ) represenaion, by replacing he righ hand side of he expression wih heir own AR represenaion: X = ϕ( ϕx 2 + ε ) + ε =... = ϕ ε = 0 Where he coefficiens ϕ are square summable ϕ < o secure sabiliy, hey are causal in he sense ha no <0 and hey are sable hrough ime. = 0 25

27 2.8. Univariae and Mulivariae Time Series So far, we have been referring o univariae ime series; however, i is more usual ha a process X is comprised by many such univariae componens which have some degree of dependence beween hem. The mulivariae case is a naural expansion of he univariae case, so we will quickly refer o basic properies, such as he firs order momens and heir esimaion, as well as he vecor represenaion (mulivariae case) of an AR(p) and of an MA(q) process. Please noe, ha from now on we will use vecor noaion o denoe he mulivariae cases (bold case leers). Le us consider a mulivariae ime series X, Χ =..., comprising of m Χ univariae ime series wih =,2, ime poins. X, m Consider also he AR represenaion of = = 0 Χ φ ε. Then, he parameers φ shoud fulfill he crieria for causaliy which is = 0 de[ φ ( z)] 0, z <. Then heir firs order momens and heir corresponding sample esimaors will be he following: 26

28 Mean & Variance µ X, µ X = E[ Χ ] =..., µ X, m σ = var[ X ] = E[( X µ )( X µ ) '] 2 Χ X X Auo Covariance & Auo Correlaion Γ ( h ) = Ε [( Χ - µ )( Χ - µ )'] X + h X X ρ Γ ( h) Ε[( Χ µ )( Χ µ ) '] X + h X X X ( h ) = = 2 Γ X (0) σ X Cross Covariance & Cross Correlaion γ( + h, ) K γm ( + h, ) ΓΧΥ ( h) = Ε[( Χ + h µ X )( Y µ Y ) '] = M O M γ m( + h, ) γ mm ( + h, ) L ρ ΧΥ ( h) ρ ( + h, ) K ρ ( + h, ) m Ε[( Χ + h µ X )( Y µ Y ) '] = = 2 2 M O M ΓX (0) ΓY (0) ρm( + h, ) L ρmm ( + h, ) Parial Correlaion ρxy ( h) ρxz ( h) ρzy ( h) ρxy Ζ ( h) = 2 2 ρ (0) ρ (0) XZ ZY In accordance o he univariae esimaors we have already defined, he sample esimaions of he above menioned heoreical momens will be he following for he mulivariae case: 27

29 Sample Mean & Variance n n = x = x, and n 2 s = ( x x ) X n = Sample Auo Covariance & Auo Correlaion n h C ( h) = ( x+ h x)( x x ), X n h C ( h) r ( h ) = X X C (0) X = Sample Cross Covariance & Cross Correlaion We also consider he sample covariance and correlaion beween wo differen random variables X and Y n h C ( h) = ( x+ h x)( y y) XY n h = r XY ( h ) = C C XY ( h) C 2 2 X Y Sample Parial Correlaion rxy ( h) rxz ( h) rzy ( h) rxy Ζ ( h) = 2 2 r (0) r (0) XZ ZY 28

30 29

31 3. Causaliy In his chaper we presen he classical definiion Granger has given o causaliy (Granger, 969). We also explore expansions of Granger-Causaliy, such as Condiional and Parial Granger-Causaliy. Noe ha in all cases, we will only visi he case of unidirecional causaliy from one variable owards anoher. Measures for accouning causaliy will be given, as well as mehods for drawing saisical inference upon hese measures. 3.. Granger Causaliy As briefly discussed in he inroducory par, Granger causaliy is a concep ha explores he exen o which one variable causes anoher. More precisely, Granger defines i (Granger 969, Seh 200) as a siuaion where he pas values of a variable Y improve he linear predicion of X. He also gives he following formulaion, σ 2 ( X U)<( X U Y) 3. which shows ha if he variance of he random variable X given he informaion se U ha we have available a ime (including pas values of Y), is smaller han he variance of X given all informaion available, excluding pas values of Y, hen we have indicaion ha Y causes X. To his end, Granger has consruced wo models, one called resriced (excluding all pas values of Y and one named unresriced (conaining informaion upon pas values of Y ), in order o compare he variances of he error erms and draw inference upon poenial causaliy from Y o X (Geweke, 984). In he following secions we will presen in deail he formal represenaion of he wo bivariae auoregressive models and a es saisic for Granger causaliy. 30

32 Le X be a saionary random vecor ha can be represened by an auoregressive model of order p, which represens he maximal lagged observaions included in he model (Seh 200). X = ϕ X + ϕ X ϕ X + ε r r r r 2 p p X where p r ϕ <, =, 2,..., = p r and he parameers of he equaion lie inside he uni circle < ϕ <, and he innovaion erm (sochasic par) belongs o a whie noise process, ε σ 2 ~ WN (0, r ) ε X We also inroduce a second variable Y ha has he same properies Y = ξ Y + ξ Y ξ Y + ε r r r r 2 p p Y wih p r ξ <, =, 2,..., = p r and < ξ <, ' 2 and ε ~ WN(0, σ ) r Y ε As we have shown in chaper 2, he wo processes can be re-wrien shorly in he following compac form X Y p r ϕ X = r = + ε p r r = ξ Y + εy = Χ 3.2 3

33 The noaion r on he parameers of he models presened so far, is an abbreviaion for he word resriced, as we will call model 3.2 as resriced model (for from now on. X and Y accordingly) Now, we will inroduce he unresriced model, where besides he auo-lagged values of each variable, he pas values of he oher variable also ake par in he predicion of and Y accordingly. Formally, X p p u u u = ϕ + ζ + ε Χ = = X X Y where p u ϕ <, = u and he parameers of he equaion lie inside he uni circle < ϕ <, =, 2,..., p u 2 and he innovaion erm (sochasic par) belongs o a whie noise process, ε ~ WN (0, σ ) X ε u X p p u u u = ξ + ϕ + εy = = Y Y X wih p u ξ <, = u and < ξ <, =, 2,..., p u 2 and ε ~ WN(0, σ ) Y u Y ε In accordance wih he resriced models, indicaor u sands for unresriced, and (3.3) will be called unresriced model from now on. 32

34 p p u u u = ϕ + ζ + ε Χ = = X X Y p p u u u = ξ + ϕ + εy = = Y Y X 3.3 The heoreical measure ha Geweke gave o measure linear dependence from Y o X was = σ ln( ) σ 3.4 F Y X 2 r ε X 2 u ε X Under he null hypohesis H : 0 0 F =, meaning ha here exiss no linear dependence Y X from Y o X, ha is, pas values of Y canno improve he predicion of X. (Inuiively, he saisic FY X would ake he value of 0 in he case where σ = σ 2 2 r u ε X ε X By performing Ordinary Leas Squares or Maximum Likelihood Esimaion on equaions (3.2 ) and (3.3 ), o rerieve he opimal linear esimaor for X we also ge he residual sum of squares RRSS, URSS of he resriced and he unresriced model respecively. Then, we can consruc an F es o es he significance of our es, under he null hypohesis ha here exiss no unidirecional causaliy from Y o X Fˆ Y X ( RRSS URSS) / q = URSS / ( n k) 3.5 H : 0 0 FY X = where n is our sample size, k he number of parameers esimaed from he unresriced model and q represens he number of resricions. 33

35 3.2. Condiional Granger Causaliy Real applicaion daa do no confine o bivariae cases. A variable migh be simulaneously affeced by wo or more variables. I is also possible and happens quie ofen ha he researcher derives false causaions beween wo ses of variables, due o a spurious correlaion wih a hird unexamined variable. As Granger firs poined ou early in his work (Granger, 969), a variable migh falsely seem o be caused by a second one, even hough boh variables are caused by a hird one, bu a differen ime lags. For example, we migh be able o derive a false causal connecion from our naional Gross Domesic Produc (GDP) and he monhly rae of car sales. Bu if one observes closely, boh variables migh be driven by a hird variable, e.g. consumpion, which is no as frequenly recorded as he monhly car sales. This example demonsraes how he naure of our daa affecs our resuls as well as ha he repeiive realizaion of bivariae analysis beween our ses of variables can someimes reurn misleading resuls. I herefore underlines he necessiy of an MVAR represenaion of he series ha are possibly relaed. Geweke se he ground for Condiional Granger Causaliy (Geweke, 984) as a means o conrol for indirec causaliy beween wo variables ha migh a firs insance appear direc, bu is in ruh mediaed by a hird se of series (Guo e al, 2008). Suppose ha we wan o explore he causaliy from he ime series Y owards X bu a he same ime, we wan o condiion on a hird se of random variables called Z. Le Z be a mulivariae ime series consised of m random saionary Zi ime series, wih i=,,m ha has he following represenaion and properies: Z = ψ Z + ψ Z ψ Z + ε Z r r r r 2 p p p where ψ = u <, =,2,.., p 34

36 and he parameers of he equaion lie inside he uni circle r < ψ <, 2 ε ~ (0, r ) and he innovaion erm (sochasic par) belongs o a whie noise process, WN Z. σ ε Then, he resriced model ha we inroduced in he previous secions will become p p r r r = + + Χ = = X φ X δ Z ε p p r r r = + + Y = = Y ξ Y ν Ζ ε 3.6 Γ = r r r var(ε ) cov(ε,ε ) Χ Χ Z r r r cov(ε,ε ) var(ε ) Z Χ Z 3.7 Whereas he unresriced will ake he following form: p p p u u u u = Χ = = = X φ X ζ Y δ Z ε p p p u u u u = Y = = = Y ξ Y φ X ν Ζ ε p p p u u u u = π + ρ + υ + Ζ = = = Ζ X Y Ζ ε 3.8 The noise covariance marix of his new sysem will urn o: u u u u u var(ε ) cov(ε,ε ) cov(ε,ε ) Χ Χ Y Χ Z u u u u u Γ 2 = cov(ε,ε ) var(ε ) cov(ε,ε ) 3.9 Y Χ Y Y Z u u u u u cov(ε,ε ) cov(ε,ε ) var(ε ) Z Χ Z Y Z 35

37 Clearly, he Granger causaliy from Y o X condiioned on Z will be: c F = ln( Γ / Γ ) Y X Inuiively, his measure will show us how he unexplained par of he X ime series has been reduced by he inclusion of he Y series as an explanaory facor, bu condiional on he Z ime series. Therefore, under a FY->X ha has a posiive value, we can conclude ha here exiss some direc influence from Y o X, no mediaed by he hird variable Z. Bu in he case where he causal influence measure akes he value of 0, we have subsanial evidence ha Y is no direcly causing X, bu he causaliy is being driven by he hird facor, Z. By linearly esimaing (OLS) models 3.6 and 3.8, we rerieve he esimaed residual marices C2 and C3 accordingly, which boh follow a chi-square disribuion, so by using he appropriae ransformaion using he degrees of freedom of he unresriced model and he number of resricion, we can consruc an F-es saisic, comparable o he Granger Causaliy Index. Then he condiional granger causaliy Index will become Fˆ H c Y X 0 ( RRSS URSS) / q = URSS / ( n k) c : F = 0 Y X 3. wih RRSS, URSS he residual sum of squares of he resriced and he unresriced model respecively, and n our sample size, k he number of parameers esimaed from he unresriced model, and q he number of resricions imposed. 36

38 3.3. Parial Granger Causaliy Parial Granger Causaliy is a furher improvemen upon he measure of condiional G- Causaliy consruced o allow he researcher o conrol for exogenous inpus and laen variables which migh no be included in he VAR model bu migh be criical for he resuls (Guo e al., 2008). The main inuiion is ha he effec of exogenous or laen facors will be refleced in he covariance marices (Seh 200). The model seup is very similar o ha of condiional G-causaliy. The sole difference lies in he error erms ha are now defined as he sum of he innovaion erm, he noise due o exogenous inpus and laen facors. Thus, he resriced model becomes: p p r r r ' r ' r E L =, where ( ) + + = + + Bx L Χ Χ Χ Χ Χ = = X φ X δ Z ε ε ε ε ε p p r r r ' r ' r E L =, where ( ) + + Y Y = B Y + + Y x L Y = = Y ξ Y ν Ζ ε ε ε ε ε 3.2 Whereas he unresriced akes he form p p p u u u u ' u ' u E L =, where ( ) = + + B L Χ Χ Χ Χ Χ Χ = = = X φ X ζ Y δ Z ε ε ε ε ε p p p u u u u' u ' u E L =, where ( ) Y = + + B Y Y Y Y L Y = = = Y ξ Y φ X ν Ζ ε ε ε ε ε p p p u u u π ρ υ = = = Ζ = X + Y + Ζ + ε ', where ' Ζ ε = ε + ε + BΖ ( L) ε u u u E L Ζ Ζ Ζ Ζ 3.3 Wih heir corresponding noise covariance marices Γ 3 = r ' r ' r ' var(ε ) cov(ε,ε ) Χ Χ Z r ' r ' r ' cov(ε,ε ) var(ε ) Z Χ Z 37

39 u ' u' u' u' u ' var(ε ) cov(ε,ε ) cov(ε,ε ) Χ Χ Y Χ Z u ' u' u ' u ' u' Γ 4 = cov(ε,ε ) var(ε ) cov(ε,ε ) Y Χ Y Y Z u' u' u' u' u' cov(ε,ε ) cov(ε,ε ) var(ε ) Z Χ Z Y Z By pariioning he noise variance marices, o manage o conrol for he effec ha he exogenous inpu have, we rerieve r r ' r ' r ' r ' r ' Γ 3 = var(ε ) cov(ε,ε )var(ε ) cov(ε,ε ) Χ Χ Z Z Z Χ u u ' u ' u ' u ' u ' Γ 4 = var(ε ) cov(ε,ε )var(ε ) cov(ε,ε ) Χ Χ Z Z Z Χ 3.4 The Parial Causaliy Index is he log raio of he wo pariioned noise covariance marices p F = ln( Γ / Γ ) 3.5 Y X 3 4 Unlike sandard and condiional Granger Causaliy indices, he saisical disribuion of he Parial Causaliy Index is unknown o us, as we are unaware of he saisical properies of he exogenous facors (Seh, 200), so we have o resor o randomizaion ess in order o be able o draw some conclusion upon he saisical significance of our indicaors. 38

40 4. Transfer Enropy 4.. Enropy as an informaion meric Transfer enropy is a concep inroduced by Schreiber o describe a measure commonly used in informaion heory ha would ake ino accoun muual informaion bu would also share he dynamics of informaion ranspor ino accoun (Schreiber, 2000). Though difficul o read, enropy in informaion heory has an one-on-one relaionship o ha of causaliy, as enropy is a erm used o describe he degree a which a bi of informaion reveals he rue message. As Schreiber described in his paper, his informaion meric quanifies he saisical coherence beween sysems evolving in ime. Shannon was he firs one o borrow he classic Bolzman enropy and inser i ino informaion heory as a measure of uncerainy. I is imporan o noe ha informaion heory analyzes sochasic processes which generae discree bis of informaion. As Shannon menions in his work (Shannon, 948), coninuous processes can be regarded as discree sources, provided we can apply some discreizaion process. The Shannon enropy is mahemaically formulaed as following: Le p i wih i=,2,,n be he probabiliy of a se of n possible evens. I is more inuiive o hink of a sae space comprising of n cells, and being a he cell i. p i represens he probabiliy of a sysem Then is enropy will be given by (Shannon, 948): n H = pi log( pi ) 4. i= 39

41 Now consider wo evens x and y, where p( i), p( ) are he probabiliy ses for x and y respecively and p( i, ) represens he oin occurrence of i for x and for y. Then he enropy for each even will form as following, H ( x) = p( i, ) log p( i, ) i, H ( y) = p( i, ) log p( i, ) i, i and he oin occurrence of he evens is given by H( x, y) = p( i, ) log p( i, ) i, whereas if he wo evens are independen, H ( x, y) = p( i, ) log( p( i) p( )) I i, Then he muual informaion beween X and Y evens will be given by he difference of he wo evens (acual relaionship minus he case hey where independen) MI = H ( x, y) H ( x, y) xy I MI = p ( i, ) log( p ( i) p ( )) + p ( i, ) log( p ( i, )) xy xy x y xy xy i, i, MI = p ( i, ) [ log( p ( i, )) log( p ( i) p ( ))] xy xy xy x y i pxy ( i, ) MI xy = pxy ( i, ) log i px ( i) py ( )

42 4.2. Transfer Enropy-An Indicaor Schreiber was he firs one o coin he erm ransfer enropy, ha would isolae he dynamics of muual informaion (due o a common hisory) revealing hus he direcion of informaion exchange beween wo sysems. Transfer Enropy can be represened in wo equivalen ways, one using he expressions of Shannon enropy, and one resembling he muual informaion crierion. Below we presen boh formulaions for consideraion. Consider wo sochasic variables X and Y. The informaion ha variable Y conains on variable X a h ime seps forward, based also on he curren saus of X is called ransfer enropy from Y o X. As already menioned, in order o be able o measure ransfer enropy, hen we have o be able o discreize he sochasic processes of X and Y, wih he binning of heir coninuous ime span ino ( ) = m τ +,..., n h ime seps, wih m being he embedding dimension used for he phase space reconsrucion, τ represening he delay dimension, and h he ime horizon ahead, ha we wish o examine. The vecors x and x = [ x, x,..., x ]' y denoe he phase sae represenaion of X and Y respecively, wih ( ) τ m τ y = [ y, y,..., y ]' ( ) τ m τ The expression of ransfer enropy from Y o X is given by T = H ( x x ) H ( x x, y ) 4.3 Y X + h + h and can be expressed hus as he difference beween x + h, condiioned on is own pas and on he pas of y (Barne, Seh, 2009). 4

43 The represenaion of TY X wih a formulaion resembling ha of muual informaion is given by he following ype (Schreiber, 2000). T = p( x, x, y ) log Y X + h p( x+ h x, y ) p( x x ) + h 42

44 4.3. Parial Transfer Enropy-An Indicaor The parial ransfer enropy, is an exension of he measure of ransfer enropy. The main feaure ha renders Parial Transfer Enropy as an imporan improvemen over he sandard Transfer Enropy measure is he fac ha i is able o conrol for he effec of a hird variable Z, hrough is inclusion of he oin probabiliy densiy funcions in he expression of he enropies (Papana e al., 202). Consider he wo vecors we inroduced in he previous chaper, wih idenical paramers (embedding dimension, ime delay). x = [ x, x,..., x ]' ( -) τ m y = [ y, y,..., y ]' ( -) τ m τ τ Le Z be a hird sochasic variable and z = [ z, z τ,..., z ( -) ]' m τ be is vecor phase sae represenaion. Then he Transfer Enropy from y o PTE = H ( x x, z ) H ( x x, y, z ) 4.4 Y X + h + h x conrolling for he effec of z will be given by The PTEY X can also be expressed in he muual informaion formulaion, by insering he condiioning on z ino he log raio (Schreiber, 2000). 43

45 4.4. Saisical Inference on TE and PTE As he saisical disribuions of he Transfer Enropy and Parial Transfer Enropy measures are no known o us, we are no able o perform any parameric ess o es heir saisical significance. In he lieraure here can be found some surrogae daa mehods, such as boosrapping and random permuaion (Seh, 200). We will also resor o a surrogae daa mehod (Kugiumzis, 2000) in order o consruc p-values on he measures and herefore be able o es he null hypohesis ha he ime series ha we examine for causaliy, do no bear any links. The inuiion behind he surrogae daa is ha afer reshifing he causal variable, hen he wo ime series bear no correlaion. Wih random resampling, we creae a disribuion of lack of correlaion, herefore we can check he null hypohesis wheher he value ha our measure has aken belongs o his disribuion of no correlaion. 44

46 5. Granger Causaliy and Transfer Enropy on Simulaed Daa 5.. Inroducion In he previous chapers we have presened ways one can accoun for causaliy beween variables. In his secion we will demonsrae four causaliy measures ha we have previously inroduced; Granger Causaliy, Condiional Granger Causaliy, Transfer Enropy and Parial Transfer Enropy. Thus we can compare he resuls ha he parameric ess deliver, in conuncion o he corresponding ones from he informaion-heoreic ess. To his end, we have applied he measures on hree variables which hide causal links. The hree variables ha we esed were consruced so as o follow cerain assumpions. Our goal was o explore he dynamics of each measure under differen models; herefore, we creaed hree ses of variables, each one generaed under differen assumpions in each sysem: a linear, a slighly disored linear one, as well as a non-linear. For each one of hese sysems, we check wheher he four causaliy merics give us back subsanial and saisically significan resuls upon he causaliy relaions of he sysem. The firs se of variables is based on a rivial linear sysem. The second sysem generaing our variables is very similar o he iniial linear sysem, wih a sligh differeniaion; by applying absolue values on our variables, we explore he behavior of our ess under a sligh deviaion from normaliy. Finally, we have also used a non-linear sysem o furher explore he possibiliies of he four mehods. 45

47 5.2. The esing procedure The seps from our esing procedure are described in he following. Firs we define he sysem which should generae he ime series we would like o es. The ime series creaed are being fied agains an AR model, and he residuals are being esed wheher hey belong o a whie noise process. This is o safeguard ha hey fulfill he crieria in order o be insered in he analysis. The four ess are applied on he hree generaed ime series, and we receive he values of he respecive causaliy measures. Because we have no sandardized way of measuring he significance of he informaionheoreic measures (ransfer enropy and parial ransfer enropy), we follow a surrogae procedure ha was shorly inroduced in previous chaper: A hundred of new ime series are generaed by randomly shifing he originally generaed series (he causal one). The purpose of his process is o creae new ses of series ha are uncorrelaed and have no causal links. Having creaed he 00 pairs, we hen calculae again he causaliy measures for each one of hese 00 ses. Thus, we rerieve a disribuion of causaliy values in cases where we have no causaliy. In he final sep, we compare he value ha he causaliy measures delivered us for he original ime series o he disribuion of values ha he non-causal surrogaes delivered. Obviously, if he original value falls ino he disribuion of he non-correlaed surrogaes, hen his would mean ha we do no have subsanial proof of causaliy for he original ime series as well. If however, he value is significanly differen from he values ha he surrogaes delivered, or lies in he ails of he non-correlaed disribuion (5% of each ail), hen we have subsanial evidence ha he wo ime series we are examining bear causal links. 46

48 Figure 5- If he value of he causaliy measure lies on he ales of he disribuion, hen we reec he null hypohesis of non- causaliy This procedure is repeaed 00 imes, o reveal he exen o which he reecions of our null hypohesis are randomly or seadily replicaed. I is imporan o noe ha we have no used any parameric ess for he Granger and Condiional Granger Causaliy. As we waned o creae a common benchmark for comparing he four measures, we have used he same procedure of surrogaes for saisical inference upon all four of hem, despie he fac ha he disribuion of he parameric ess is known, and ha confidence inervals can be consruced for hem. The specificaions of he sysems generaing he ime series, as well of he ess are shorly described below: In all hree sysems we have generaed hree ime series of lengh equal o 024 observaions. Therefore, for each measure, here were evaluaed 3x3 causaliy measures (of course he diagonal elemens, esing he causaliy of one variable owards iself resuled o zero). We chose a low order for our model, as we do no wan o decrease dramaically our es power. The embedding delay was se o 2, whereas he ime delay was se o, for compuaional reasons. We also se embedding delay and ime delay parameers for each pair of variables esed equal, as i is common use (Schreiber, 2000). Mehod of neares neighbors (Kraskov, 2004) was used o esimae he enropies, for he 47

49 calculaion of ransfer enropy; he arbirary parameer of number of neighbors for he densiy esimaion was se o 5. 48

50 5.3. Linear Sysem The linear sysem we used o creae he firs se of variables is he following: X = 0.4X 0.5X + e X,,, 2, = 0.4X 0.3X + 0.6X + e 5. 2, 2, 2, 2, 2, X = 0.5X 0.7X 0.3X + e 3, 3, 3, 2 2, 3, I is eviden in our VAR formulaion ha while he firs variable X of our sysem is no impaced by any oher variables excep for is own pas, i however influences he fuure values of X2. X2 on he oher hand direcly impacs on X3 whereas X3 does no influence any oher variable. X X2 X3 Figure 5-2 Represenaion of he rue causaliy links in he linear sysem Following, we will explore he causal links wihin he sysem hrough he four causaliy measures we have presened, and we will see how hese heoreical measures respond in praxis. The four measures we will be using are: Granger Causaliy Index, Condiional Granger causaliy Index, Transfer Enropy and Condiional Transfer Enropy. In he following diagram we see he ime series ha our sysem generaed, 49

51 Figure 5-3 The ime series ha he linear sysem has generaed, n=024 and i is obvious even from he char ha he series are saionary and able o be used in our analysis. The following able gives us an overview of he resuls we rerieved. In deail, he figures show us he percenage of successful causal links, ha is, of he imes we managed o reec he null hypohesis of no causaliy. Parameric Tess Informaion Theoreic Tess Condiional Granger Causaliy Transfer Enropy Table 5- Overview of empirical resuls on a linear sysem: Parial Transfer Enropy Var Granger Causaliy X X2 00% 00% 00% 00% X X3 00% 3% 84% 6% X2 X % 2% 6% 9% X2 X3 00% 00% 00% 88% X3 X 7% 7% 0% 9% X3 X2 6% % 2% 5% 50

52 The able presens he percenage of significan causaliies (reeced null hypoheses of non causaliy) found in 00 repeiions. As we can see in he able above, he Granger Causaliy es deeced causal links from X owards X2 and X3, as well as a link from X2 o X3 in all 00 repeiions, failing hus, o recognize he indirec flow of informaion from X o X3 hrough X2. The measure of Condiional Granger Causaliy improved our findings as he null hypohesis was reeced in only 3 ou of 00 repeiions. As for he remaining, rue causal relaions of he sysem, he measure of Condiional Granger Causaliy had a 00% success in idenifying he in all repeiions. Transfer Enropy exposed he same links wih Granger Causaliy Index, as i idenified he indirec link beween X and X3. I is herefore sill vulnerable o indirec causaliy, while parial ransfer enropy has resored his false link. The conclusions driving his comparaive sudy in a simple linear sysem are quie sraigh forward and in line wih he bibliography. All four measures have revealed he fundamenal links hroughou our sysem, while condiioning on he hird variable has refined our findings, in boh parameric and informaion-heoreic measures, by eliminaing cases of indirec causaions. The comparison has proven ha he parameric ess gave us back more robus resuls, as he same significan causaions were replicaed in all 00 replicaions, whereas he informaion heoreic measures, did no deliver he same success rae in all cases. 5

53 Figure 5-4 The figures depic he values ha he four causaliy measures ook during 00 realizaions for he linear sysem. The figure above represens graphically he values ha he causaliy indexes ook for he original ime series esed hroughou he 00 repeiions of he esing procedure. For he parameric es we can have a sraighforward inerpreaion of his graphic, because he furher away from zero he value of our causaliy index, he sronger he causaion i reflecs (provided of course, i is also found saisically significan). So we can also see graphically how he indirec link from X owards X3 is already weaker han he remaining wo causal relaions, and ha by condiioning on X2, his link vanishes. 52

54 5.4. adisored linear sysem In he previous secion, we esed causal relaions of a simple linear sysem using four differen echniques. Bu wha if we were o slighly aler he sysem we are examining? Would his significanly aler he oucome of he iniial paradigm? We will inroduce a sligh disorion o he sysem used in chaper 5.3; ha is inroducing an absolue value o he auoregressive par of he deerminisic equaion of each variable comprising our VAR sysem. X = 0.4 X 0.5X + e X,,, 2, = 0.4 X 0.3X + 0.6X + e 5.2 2, 2, 2, 2, X = 0.5 X 0.7X 0. 3X + e 3, 3, 3, 2 2, 3, The underlying connecions of our sysem are he same as he ones of he linear sysem, graphically shown in he below flow char: X X2 X3 Figure 5-3 Represenaion of he rue causaliy linkes in a disored linear sysem The series ha he sysem generaed are depiced in he following graph: 53

55 Figure 5-5 The ime series ha he disored linear sysem has generaed, n=024 Again he original se of series is underlined by a mean revering procedure, being able o be furher analyzed. Parameric Tess Informaion Theoreic Tess Parial Var Granger Causaliy Condiional Granger Causaliy Transfer Enropy Transfer Enropy X X2 00% 00% 00% 00% X X3 00% 0% 65% 20% X2 X 5% 5% 20% 0% X2 X3 00% 00% 00% 00% X3 X 5% 0% 5% 5% X3 X2 5% 0% 5% 5% Table 5-2 Overview of empirical resuls on a disored linear sysem: The able presens he percenage of significan causaliies (reeced null hypoheses of no causaliy) found in 00 repeiions. As we can see in he able above, he Granger Causaliy es has exacly he same behavior as in he linear sysem; i has deeced causal links from X owards X2 and X3, as well as a 54

56 link from X2 o X3 in all 00 repeiions. Thus, he sandard Granger Causaliy Index failed again o recognize he indirec flow of informaion from X o X3 hrough X2. The remaining hree ess were more successful in he sense ha rue links were found ou wih a 00% rae of success. The conclusions ha he disored linear sysem has led us o are very similar, almos equivalen o hose of he linear sysem. All four measures have revealed he fundamenal links hroughou our sysem, whereas condiioning on he hird variable has refined our findings, in boh parameric and informaion-heoreic measures, by eliminaing cases of indirec causaions. An ineresing fac when comparing Granger Causaliy and Transfer Enropy resuls, is ha he Transfer Enropy indicaor performed beer in comparison o is corresponding parameric es, as i failed o reec he hypohesis of non-causaliy in 35% of he cases, in conradicion o G-causaliy ha reeced he null in all insances. In oher words, ransfer enropy managed in some cases o reveal he indirec flow of informaion from X o X3. So, he examinaion of he disored linear sysem allowed us o see ha he informaion heoreic es have a sligh advanage in environmens where variables migh deviae from normaliy, or exhibi peculiar behavior, or even when he researcher is unaware of possible indirec links wihin he sysem. 55

57 Figure 5-62 The figures depic he values ha he four causaliy measures ook during 00 realizaions for he disored linear sysem. 56

58 5.5. Non-linear sysem Challenging he robusness of he four measures, we ook one sep furher and esed all four of hem on a purely non-linear sysem. We chose no o disurb he iniial assumpions and o keep he same causal links hroughou he sysem. So our sysem becomes he following: X = 0.4X 0.5X + e,,,, X = 0.4X 0.3 X (0.4 X ) + e 2, 2, 2,, 2, X = 0.5X 0.3X + e 3, 3, 2, 3, (0.7 X 3, ) 5.3 The pair of original series generaed by he sysem is depiced in he below graph. Figure 5-7 The ime series ha he non-linear sysem has generaed, n=024 57