UNIVERSITY OF PATRAS

Transcript

1 UNIVERSITY OF PATRAS SCHOOL OF ENGINEERING DEPARTMENT OF ELECTRICAL AND COMPUTER ENGINEERING Doctoral Dssertaton «MONITORING AND CONTROL OF DISTRIBUTED WEB SERVICES ON CLOUD COMPUTING INFRASTRUCTURE» DECHOUNIOTIS DIMITRIOS DIPLOMA IN ELECTRICAL AND COMPUTER ENGINEERING DISSERTATION No: 318 PATRAS

2 2

3 ΠΑΝΕΠΙΣΤΗΜΙΟ ΠΑΤΡΩΝ ΠΟΛΥΤΕΧΝΙΚΗ ΣΧΟΛΗ ΤΜΗΜΑ ΗΛΕΚΤΡΟΛΟΓΩΝ & ΤΕΧΝΟΛΟΓΙΑΣ Η/Υ Διδακτορική Διατριβή «ΠΑΡΑΚΟΛΟΥΘΗΣΗ ΚΑΙ ΕΛΕΓΧΟΣ ΚΑΤΑΝΕΜΗΜΕΝΩΝ ΔΙΚΤΥΑΚΩΝ ΥΠΗΡΕΣΙΩΝ ΣΕ ΥΠΟΛΟΓΙΣΤΙΚΗ ΑΡΧΙΤΕΚΤΟΝΙΚΗ ΝΕΦΟΥΣ» ΔΕΧΟΥΝΙΩΤΗΣ ΔΗΜΗΤΡΙΟΣ ΔΙΠΛΩΜΑΤΟΥΧΟΣ ΗΛΕΚΤΡΟΛΟΓΟΣ ΜΗΧΑΝΙΚΟΣ ΚΑΙ ΤΕΧΝΟΛΟΓΙΑΣ Η/Υ ΑΡΙΘΜΟΣ ΔΙΑΤΡΙΒΗΣ: 318 ΠΑΤΡΑ

4 4

5 Acknowledgements Foremost I would lke to thank my advsor professor Spyros Denazs for hs contnuous support, motvaton and gudance durng the research and wrtng of ths thess. Besdes my advsor, I would lke to thank the rest of my thess commttee: Prof. Antonos Tzes and Prof. Evaggelos Dermatas for ther encouragement, nsghtful comments, and ther help whenever I needed. I would lke to express my sncere grattude to Prof. George Btsors for hs contrbuton of tme, mmense knowledge and beng an excellent example of teacher. My sncere thanks also goes to Dr. Andreas Knd and Dr. Xenofontas Dmtropoulos for offerng the nternshp opportunty n the IBM Research Laboratory of Zurch and leadng me workng on exctng research topcs. I feel grateful that most tme of my research I work together wth my close frend Nkolaos Leontou, who s also PhD student n the same lab. Our collaboraton helps me to be more productve and mprove the qualty of my research. I thank my close frend Nkolaos Athanasopoulos for hs patence, advces and deas whch help me a lot to complete ths thess. I thank my close frends and fellow labmates Nkolaos Efthymopoulos, Athanasos Chrstakds and Mara Efthymopoulou for ther support, advces, productve dscussons and fun we have had all these years. I thank my frends Andreas Lambropoulos, Rana Tsoulou, Ell Andrkogannopoulou, Anastasa Zsmatou and Splos Kastans for ther frendshp and support. Last but not least I thank and dedcate ths dssertaton to my parents Panayots and Nkoletta and my brother Gorgos, who always support me n all my pursuts. Dechounots Dmtros Patras February

6 6

7 Εκτεταμένη Περίληψη στα Ελληνικά Greek Extended Abstract 1. Περιγραφή της Διατριβής. Η Υπολογιστική Αρχιτεκτονική Νέφους (ΥΑΝ cloud computng) είναι ένας πρόσφατος τρόπος ανάπτυξης και παροχής δικτυακών υπηρεσιών που αναπτύχθηκε μέσα στην τελευταία δεκαετία. Με την γρήγορη εξέλιξη των υπολογιστικών και αποθηκευτικών τεχνολογιών και την διάδοση του διαδικτύου, οι υπολογιστικοί πόροι έχουν γίνει πιο ισχυροί και προσιτοί για όλους από κάθε άλλη εποχή. Αυτή η νέα τεχνολογική τάση έχει σαν αποτέλεσμα τη δημιουργία ενός νέου μοντέλου υπολογιστικής αρχιτεκτονικής, της Υπολογιστική Αρχιτεκτονική Νέφους (ΥΑΝ), στο οποίο οι υπολογιστικοί (CPU, μνήμη και αποθηκευτικά μέσα) και δικτυακοί πόροι είναι οργανωμένοι σε υπολογιστικά κέντρα (data centers) που βρίσκονται διασκορπισμένα σε όλο τον κόσμο. Οι πόροι αυτοί παρέχονται σαν προϊόν το οποίο μπορεί να μισθώνεται κατά απαίτηση από τους χρήστες του διαδικτύου. Η ΥΑΝ προβάλλει πειστικά επιχειρήματα στους πελάτες έτσι ώστε να μεταφέρουν τις δικτυακές εφαρμογές από ιδιόκτητες υποδομές και εγκαταστάσεις σε πλατφόρμες ΥΑΝ. Τα πιο σημαντικά κίνητρα είναι το μοντέλο πληρωμής ανάλογα με τη χρήση, η δυναμική κατανομή πόρων, η εύκολη επεκτασιμότητα, η μείωση των εξόδων λειτουργίας και συντήρησης καθώς και η εύκολη πρόσβαση από ετερογενής συσκευές. Με αυτό τον τρόπο, οι πελάτες δεν είναι υποχρεωμένοι να επιβαρύνονται με την αγορά του απαραίτητου εξοπλισμού ενός ιδιωτικού υπολογιστικού κέντρου και τα έξοδα λειτουργίας και συντήρησής του, αλλά έχουν τη δυνατότητα να μισθώνουν ανάλογα με τις ανάγκες μόνο τους απαραίτητους υπολογιστικούς και δικτυακούς πόρους για την εγκατάσταση των δικτυακών τους εφαρμογών και να πληρώνουν μόνο για τη χρήση τους. Αυτό συνεπάγεται επίσης μείωση της κατανάλωσης ηλεκτρικής ενέργειας, των βλαβών του υλικού (hardware) και του αναγκαίου εξειδικευμένου τεχνικού προσωπικού. Οι πάροχοι υπηρεσιών ΥΑΝ χωρίζονται σε τρεις κατηγορίες. Όπως φαίνεται και στο Σχήμα 1, οι πάροχοι είναι διασυνδεμένοι μεταξύ τους σε ένα ιεραρχικό πολυεπίπεδο μοντέλο, όπου ο πάροχος του ενός επιπέδου είναι χρήστης του παρόχου του προηγούμενου στρώματος. Στο κατώτερο επίπεδο βρίσκονται οι πάροχοι της Υποδομής ως Υπηρεσίας (ΥωΥ Infrastructure as a Servce) που προσφέρουν όλο τον αναγκαίο υπολογιστικό και δικτυακό εξοπλισμό, όπως εξυπηρετητές, αποθηκευτικές συσκευές και δρομολογητές, με τη μορφή εικονικών μηχανών (ΕΜ Vrtual Machne). Στο δεύτερο επίπεδο οι πάροχοι της Πλατφόρμας ως Υπηρεσίας (Platform as a Servce) παρέχουν την υποστήριξη των λειτουργικών συστημάτων και τα απαραίτητα περιβάλλοντα ανάπτυξης λογισμικού. Αυτά τα δύο πρώτα στρώμα ονομάζονται μαζί ως επίπεδο πόρων (utlty computng). Στο ανώτερο επίπεδο οι 7

8 πάροχοι Λογισμικού ως Υπηρεσίας (Software as a Servce) δημιουργούν οποιαδήποτε δικτυακή εφαρμογή. Χρήστης Δικτυακή Διεπαφή ΛωΥ Επίπεδο πόρων ΠωΥ ΥωΥ Σχήμα 1 - Ιεραρχικό Μοντέλο Παρόχων ΥΑΝ Οι δικτυακές υπηρεσίες, οι οποίες αναπτύσσονται σε πλατφόρμες ΥΑΝ, έχουν διαφορετικές απαιτήσεις μεταξύ τους από άποψη λειτουργικού κόστους, αξιοπιστίας και ασφάλειας. Για το λόγο αυτό τα είδη των ΥΑΝ έχουν κατηγοριοποιηθεί ως εξής: Δημόσιες ΥΑΝ προσφέρουν τους πόρους τους σαν υπηρεσίες στο ευρύ κοινό, παρέχοντας όλα τα απαραίτητα υπολογιστικά και δικτυακά μέσα αλλά έχοντας χωρίς έλεγχο πάνω στο δίκτυο, τα δεδομένα και τις ρυθμίσεις ασφαλείας. Αυτό μπορεί να αποτελεί εμπόδιο για εταιρικές υπηρεσίες με αυστηρές προδιαγραφές. Ιδιωτικές ΥΑΝ έχουν σχεδιαστεί για αποκλειστική χρήση από ένα και μόνο οργανισμό και προσφέρουν υψηλά επίπεδα ελέγχου, αξιοπιστίας και ασφάλειας. Υβριδικές ΥΑΝ είναι ένας συνδυασμός δημόσιων και ιδιωτικών ΥΑΝ, τα οποία είναι ευπροσάρμοστα και εκμεταλλεύονται τα πλεονεκτήματα των δυο προηγούμενων ΥΑΝ με σκοπό να εκπληρώσουν τις προδιαγραφές όλων των υπηρεσιών. Εικονικές ιδιωτικές ΥΑΝ εκμεταλλεύονται την τεχνολογία των εικονικών ιδιωτικών δικτύων (Vrtual Prvate Networks) και επιτρέπουν στους παρόχους 8

9 υπηρεσιών να σχεδιάσουν τη δική τους τοπολογία και ρυθμίσεις ασφαλείας πάνω σε ένα σύνολο δημοσίων ΥΑΝ. Το παραπάνω μοντέλο των παρόχων και των υποκείμενων υποδομών προσδίδει τα ακόλουθα σημαντικά γνωρίσματα στις πλατφόρμες ΥΑΝ: Γεωγραφικά κατανεμημένες υπηρεσίες και καθολική δικτυακή πρόσβαση από ετερογενείς συσκευές. Η λειτουργία των εφαρμογών καθορίζεται από ένα Συμφωνητικό Επιπέδου Υπηρεσίας, ΣΕΥ, (Servce Level Agreement - SLA) μεταξύ του παρόχου και του πελάτη. Δυναμική παροχή πόρων ανάλογα με την ζήτηση της υπηρεσίας. Κοινή δεξαμενή πόρων μπορεί να κατανέμεται δυναμικά σε πολλαπλούς χρηστών πόρων. Μείωση του κόστους λειτουργίας και συντήρησης όπως η κατανάλωση ηλεκτρικής ενέργειας και αστοχιών του εξοπλισμού. Πληρωμή ανάλογα με τη χρήση των πόρων. Παρόλη την γρήγορη ανάπτυξη, η ΥΑΝ είναι ακόμα στα αρχικά της βήματα και υπάρχουν πολλά θέματα τα οποία πρέπει να επιλυθούν. Επιπλέον νέα θέματα προκύπτουν συνεχώς από διάφορες βιομηχανικές εφαρμογές. Τα πιο σημαντικά προβλήματα διαχείρισης των ΥΑΝ περιγράφονται σύντομα παρακάτω: Η αυτοματοποιημένη παροχή πόρων (automated resource provsonng) λύνει το πρόβλημα του διαμοιρασμού των υπολογιστικών πόρων προς τις εφαρμογές με τέτοιο τρόπο έτσι ώστε να ικανοποιούνται όλες οι απαιτήσεις ανεξαρτήτως των διακυμάνσεων του φορτίου όσο και απότομες κι αν είναι. Η μετακίνηση των εικονικών μηχανών (vrtual machne mgraton) είναι μια τεχνική για την αποφυγή εξάντλησης των πόρων ενός συνόλου εξυπηρετητών. Η συγκέντρωση των υπηρεσιών (servce consoldaton) είναι ένα σχετικό πρόβλημα που ασχολείται με τη μεγιστοποίηση της χρήσης των πόρων του συστήματος ελαχιστοποιώντας παράλληλα την ενεργειακή κατανάλωση σε περιβάλλον ΥΑΝ. Η εγκατάσταση μερών διαφορετικών εφαρμογών σε ένα συγκρότημα εξυπηρετητών επιτρέπει τη χρησιμοποίηση μόνο του αναγκαίου αριθμού εξυπηρετητών και την ενεργοποίηση ή απενεργοποίηση αυτών ανάλογα με τη ζήτηση. Η διαχείριση ενέργειας (energy management) αντιμετωπίζει το πρόβλημα της κατανάλωσης ηλεκτρικής ενέργειας το οποίο δεν είναι αμελητέο πια ως κόστος. Πολλοί πάροχοι θέτουν την εξοικονόμηση ενέργειας ως έναν από τους πρωταρχικούς τους στόχους κατά τον σχεδιασμό του καταμερισμού των πόρων του συστήματος. 9

10 Η παρακολούθηση και ανάλυση της δικτυακής κίνησης (traffc montorng and analyss) είναι ένα ενδιαφέρον ερευνητικό θέμα εξαιτίας του μεγάλου όγκου των δεδομένων προς επεξεργασία. Είναι σημαντικό να υπάρχει μια άμεση εικόνα του τι συμβαίνει σε πραγματικό χρόνο στο υπολογιστικό κέντρο για την αποφυγή αστοχιών του υλικού, διατήρηση υψηλής αξιοπιστίας και ασφάλειας και για τον απαραίτητο σχεδιασμό επέκτασης ή τροποποίησης των υπαρχόντων υπολογιστικών και δικτυακών πόρων. Η ασφάλεια δεδομένων (data securty) είναι άλλο ένα σημαντικό ερευνητικό θέμα των ΥΑΝ. Ο πάροχος πρέπει να παρέχει εμπιστευτικότητα για ασφαλή πρόσβαση και μεταφορά δεδομένων και μηχανισμούς ελέγχου (audtablty) για επιβεβαίωση αν οι ρυθμίσεις ασφαλείας εφαρμόζονται ή όχι. Λογισμικά περιβάλλοντα (software frameworks) πρέπει να αναπτυχθούν για το προγραμματισμό των διάφορων διεργασιών σε ένα ΥΑΝ περιβάλλον και για μια εύκολα επεκτάσιμη και ανεκτική σε σφάλματα (fault-tolerant) επεξεργασία δεδομένων. Τεχνολογίες αποθήκευσης και διαχείρισης δεδομένων (storage technologes and data management) για κατανεμημένη επεξεργασία εφαρμογών που παρουσιάζουν μεγάλες απαιτήσεις σε αυτό το τομέα. Πρωτότυπες αρχιτεκτονικές που συνδυάζουν διαφορετικές τεχνολογίες. Μέχρι τώρα οι ΥΑΝ χρησιμοποιούν κυρίως κεντρικοποιημένες δομές. Μπορούν όμως να συνδυαστούν όμως και με κατανεμημένες δικτυακές αρχιτεκτονικές, όπως τα διομότιμα δίκτυα (peer-to-peer networks) για το διαμερισμό βίντεο ή άλλων πραγματικού χρόνου εφαρμογών πολύ μεγάλης κλίμακας. 2. Αντικείμενο της Διατριβής. Στη παρούσα διδακτορική διατριβή δύο από τα προαναφερθέντα ερευνητικά θέματα επιλύονται. Αρχικά αναπτύσσεται μια τεχνική παρακολούθηση της δικτυακής κίνησης με σκοπό την εύρεση λειτουργικών σχέσεων μεταξύ των διάφορων μερών μιας δικτυακής εφαρμογής. Στο δεύτερο μέρος επιλύεται το πρόβλημα της αυτοματοποιημένη διανομής των πόρων σε δικτυακές εφαρμογές που μοιράζονται ένα κοινό περιβάλλον ΥΑΝ. 2.1 Παρακολούθηση δικτυακών εφαρμογών σε περιβάλλον ΥΑΝ Οι σύγχρονες δικτυακές εφαρμογές είναι πολυεπίπεδες, αφού απαρτίζονται από διαφορετικά μέρη και είναι εγκατεστημένες σε υπολογιστικά κέντρα δημόσιων ή ιδιωτικών ΥΑΝ. Επειδή η σωστή λειτουργία των δικτυακών υπηρεσιών περιγράφεται από το ΣΕΥ και η παραβίασή του έχει οικονομικό αντίκτυπο, υπάρχει ανάγκη για ολοκληρωμένα εργαλεία διαχείρισης τα οποία παρακολουθούν την λειτουργία και την απόδοση των εγκατεστημένων εφαρμογών με σκοπό να προλαμβάνουν κάθε πιθανό πρόβλημα. 10

11 Ένα τυπικό υπολογιστικό κέντρο περιέχει τα συστατικά μέρη χιλιάδων υπηρεσιών. Αυτά τα μέρη αλληλεπιδρούν με ένα ακαθόριστο και μη τυποποιημένο τρόπο με σκοπό να ολοκληρώσουν διαφορετικού τύπου συναλλαγές (transactons) μιας εφαρμογής. Οι διαχειριστές του δικτύου συνήθως δεν έχουν μια πλήρη και ξεκάθαρη εικόνα των συνθηκών λειτουργίας των εφαρμογών. Τα σημερινά συστήματα δικτυακής παρακολούθησης παρέχουν πληροφορίες και στατιστικά βασιζόμενα απλά στις δικτυακές πόρτες (network ports) χωρίς περαιτέρω ερμηνεία του πως συνεργάζονται τα λειτουργικά μέρη μιας υπηρεσίας. Στόχος αυτού του κεφαλαίου της διατριβής σε σχέση με την υπάρχουσα βιβλιογραφία είναι η δημιουργία ενός εργαλείου ανάλυσης της δικτυακής κίνησης έτσι ώστε να γίνονται κατανοητές οι λειτουργικές σχέσεις μεταξύ μερών των κατανεμημένων δικτυακών υπηρεσιών. Πιο συγκεκριμένα: Η δημιουργία ενός γράφου υπηρεσιών (servce overlay), που θα απεικονίζει τις δικτυακές και λειτουργικές σχέσεις μεταξύ των διαφόρων μερών των εγκατεστημένων εφαρμογών σε ένα υπολογιστικό κέντρο. Αξιόπιστη αναγνώριση των σχέσεων χωρίς καμία πρότερη γνώση για αυτές. Μέτρηση του σθένους μια σχέσης. Εύκολη και εκτός γραμμής (offlne) διαδικασία εκπαίδευσης του συστήματος. Γρήγορη και πραγματικού χρόνου (onlne) επεξεργασία των δεδομένων. Μικρός αριθμός παραγόμενων λανθασμένων σχέσεων. Στο Σχήμα 2 απεικονίζεται η δομή των υπηρεσιών και οι διασυνδέσεις μεταξύ των χρηστών και των εξυπηρετητών ή των βάσεων δεδομένων που είναι συνδεδεμένοι με μια συγκεκριμένη εφαρμογή. Αυτός ο γράφος είναι πρωτεύον εργαλείο για πολλές εργασίες ενός διαχειριστή που εντάσσονται στο πεδίο της ανάλυσης της απόδοσης (performance analyss) και της ανάλυσης των αρχικών αιτίων (root cause analyss). Για παράδειγμα η ανίχνευση λανθασμένων εγκαταστάσεων (msconfguraton) ή διαδικτυακών επιθέσεων και ο σχεδιασμός για την επέκταση η μετατροπή των ΥΑΝ υποδομών. Η συλλογή των ιχνών των δεδομένων (traces) γίνεται μέσω του Netflow, μιας τεχνολογίας που μπορεί να συλλέξει σε πραγματικό χρόνο σε ένα δρομολογητή διάφορες πληροφορίες για δικτυακές ροές (network flows). Στη περίπτωσή μας συλλέγουμε χρονοσφραγίδες (tmestamps) και τυπικές πληροφορίες ροών, όπως ΙΡ διευθύνσεις, δικτυακές πόρτες (ports) και πρωτόκολλα. Αρχικά γίνεται μια προεπεξεργασία των δεδομένων έτσι ώστε να βρεθούν όλα τα πιθανά ζεύγη δικτυακών ροών που μπορούν να αντιστοιχούν σε κάποια σχέση μεταξύ λειτουργικών μερών μιας υπηρεσίας. 11

12 Clents Clents 116: :7004 Clents cs relatonshp ss relatonshp 56:53 ms relatonshp Clents 116: :7002 Clents Clents 1016:80 2:111 2:755 16:88 Clents 56:659 2: :22 Clents Clents 91: :53 91:53 222:25 243:53 157:53 Clents Clents Clents 116: :53 Clents Clents 3538:80 24: :53 427:53 2: :53 Σχήμα 2 - Γράφος Υπηρεσιών Βασιζόμενοι σε συγκεκριμένα βασικά και ευρέως αποδεκτά πρότυπα κατηγοριοποιούμε τα πιθανά ζεύγη σε περιπτώσεις μεταξύ πελάτη-εξυπηρετητή (clent-server), εξυπηρετητή-εξυπηρετητή (server-server)και πολλών εξυπηρετητών σειριακά (mult server), όπως φαίνεται και στο Σχήμα 3. Στη συνέχεια ένας ασαφής μηχανισμός συμπερασμού (ΑΜΣ) αναπτύχθηκε με στόχο να αποφαίνεται αν ένα ζεύγος ροών αντιστοιχεί σε κάποια πραγματική σχέση και να μετράει το σθένος της. Η ασαφή λογική είναι κατάλληλη όχι μόνο για αυτή την ταξινόμηση, καθώς μπορεί να αποφαίνεται σωστά για την ύπαρξη ή μη κάποιας σχέσης, αλλά και κατά πόσο συχνή ή χαλαρή είναι η επικοινωνία μεταξύ των διαφόρων μερών. Ο ΑΜΣ αποτελείται από ένα μικρό σύνολο κανόνων ασαφούς λογικής (π.χ. 30 κανόνες), οι οποίοι παράγονται από έναν υβριδικό γενετικό αλγόριθμο με υψηλό ποσοστό ταξινόμησης (μεγαλύτερου του 90%). 12

13 Clents DNSServ (a) WebServ Clents WebServ DB (b) WebServ AppServ DB (c) Σχήμα 3 Ζεύγη ροών: (a) πελάτη- εξυπηρετητή, (b) εξυπηρετητή-εξυπηρετητή, c) πολλών εξυπηρετητών σειριακά Οι κύριες συνεισφορές αυτού του εργαλείου ανίχνευσης δικτυακών σχέσεων είναι οι ακόλουθες: Ανακαλύπτονται οι δικτυακές σχέσεις που αφορούν το λειτουργική δομή των εφαρμογών χωρίς να υπάρχει πρωτύτερη γνώση για αυτές. Η μέθοδος με ακρίβεια τις υφιστάμενες διασυνδέσεις και απεικονίζει σωστά το σθένος τους. Οι παραγόμενες λανθασμένες σχέσεις είναι λίγες. Ο υβριδικός γενετικός αλγόριθμος που χρησιμοποιείται για την εκπαίδευση έχει υψηλό ποσοστό ταξινόμησης και χρειάζεται μόνο ένα μικρό όγκο δεδομένων εκπαίδευσης για να παράγει τους κανόνες. Αυτή η διαδικασία μπορεί να γίνεται ξεχωριστά από την αξιολόγηση και εκτός γραμμής (offlne). Επίσης μπορεί να εφαρμοστεί σε οποιοδήποτε είδος ΥΑΝ (ιδιωτικό ή δημόσιο) και για κάθε είδους εφαρμογές. Το σύνολο των κανόνων (ΑΜΣ) που χρειάζονται για την ταξινόμηση των σχέσεων και τη μέτρηση του σθένους τους είναι σχετικά μικρό. Συνήθως τριάντα κανόνες είναι αρκετοί για να ταξινομήσουν σωστά κάθε πιθανή δικτυακή σχέση. Επιπλέον λόγω του ότι η διαδικασία της αξιολόγησης δεν χρειάζεται πολύπλοκους υπολογισμούς μπορεί να εφαρμοστεί και σε πραγματικό χρόνο. Ο σχηματιζόμενος γράφος υπηρεσιών αποτελεί για τους διαχειριστές ενός υπολογιστικού κέντρου ένα χρήσιμο εργαλείο για περαιτέρω ανάλυση της κατάστασης του δικτύου,των σφαλμάτων που συμβαίνουν και των επιπτώσεων τους, καθώς και για μελλοντικό σχεδιασμό απαραίτητων επεκτάσεων ή αλλαγών του συστήματος. 13

14 2.2 Αυτοματοποιημένη κατανομή πόρων σε περιβάλλον ΥΑΝ Το δεύτερο μέρος της παρούσας διατριβής (κεφάλαιο 3) ασχολείται με το θέμα της αυτοματοποιημένης κατανομής των υπολογιστικών πόρων ενός υπολογιστικού κέντρου ΥΑΝ σε ένα σύνολο εγκατεστημένων δικτυακών εφαρμογών. Η σύγχρονη τεχνολογία της εικονικοποίησης (vrtualzaton technology) είναι ο κύριος παράγοντας για την «συστέγαση» (consoldaton) πολλών κατανεμημένων υπηρεσιών σε υπολογιστικά κέντρα ΥΑΝ. Σε αυτά τα κέντρα οι πάροχοι προσφέρουν τις κατάλληλες υποδομές, από άποψη υλικού και εργαλείων ανάπτυξης λογισμικού, τις οποίες οι εφαρμογές χρησιμοποιούν δυναμικά και ανάλογα με τις απαιτήσεις τους. Οι δικτυακές υπηρεσίες θεωρούνται κατανεμημένες για δύο λόγους. Ο πρώτος είναι ότι αποτελούνται από αρκετά συστατικά μέρη, όπως δικτυακοί εξυπηρετητές (web servers) και βάσεις δεδομένων (databases), τα οποία εγκαθίστανται σε ξεχωριστά ΕΜ. Ο δεύτερος λόγος είναι ότι πανομοιότυπα αντίγραφα των υπηρεσιών είναι εγκατεστημένα σε ένα ή περισσότερα υπολογιστικά κέντρα ΥΑΝ. Η λειτουργία μιας υπηρεσίας περιγράφεται από ένα ΣΕΥ μεταξύ του παρόχου και του πελάτη, στο οποίο καθορίζονται συγκεκριμένες ονομαστικές τιμές - στόχοι για διάφορα δικτυακά μεγέθη που αντιστοιχούν στα επιθυμητά επίπεδα ποιότητας υπηρεσίας, ΠΥ (Qualty of Servce QoS). Τα πιο συνηθισμένα δικτυακά μεγέθη που χρησιμοποιούνται είναι ο χρόνος απόκρισης των αιτημάτων (request response tme) ο αριθμός των εξυπηρετημένων αιτήσεων (throughput) μέσα σε ένα σταθερό χρονικό διάστημα. Από τη μεριά του πελάτη, οι στόχοι είναι η επίτευξη προκαθορισμένων τιμών των δικτυακών μεγεθών και η εξασφάλιση ενός εγγυημένου επιπέδου ΠΥ με το ελάχιστο οικονομικό κόστος. Αυτοί οι στόχοι μπορεί να είναι αντίθετοι με τους στόχους του παρόχου, ο οποίος επιθυμεί το σύστημα διαχείρισης να κατανέμει βέλτιστα τους πόρους σε κάθε εγκατεστημένη υπηρεσία με τέτοιο τρόπο έτσι ώστε να επιτυγχάνονται οι στόχοι των δικτυακών μεγεθών, να διασφαλίζεται η διαθεσιμότητα των πόρων ανεξαρτήτως του εισερχόμενου φορτίου αιτήσεων και να ελαχιστοποιείται το λειτουργικό κόστος του κέντρο ΥΑΝ ( πχ. η κατανάλωση ηλεκτρικής ενέργειας). Πρέπει να αναφερθεί ότι το φορτίο αιτήσεων των δικτυακών εφαρμογών γενικά είναι απρόβλεπτο και να μεταβάλλεται αρκετές τάξης μεγέθους. Επίσης οι υφιστάμενοι υπολογιστικοί πόροι είναι πεπερασμένοι και υπόκεινται σε περιορισμούς. Όλοι οι προαναφερθέντες παράγοντες καθιστούν τη διαχείριση των υποδομών ΥΑΝ ένα ανοιχτό και ενδιαφέρον ερευνητικό θέμα. Όπως φαίνεται στο σχήμα 4 υπάρχουν δυο επίπεδα ελέγχου των δικτυακών εφαρμογών σε υποδομές ΥΑΝ. Πιο συγκεκριμένα το γενικό επίπεδο (global level) και το τοπικό επίπεδο (local level). Στο γενικό επίπεδο, οι ελεγκτές προσπαθούν είτε να μοιράσουν το φορτίο κάθε εφαρμογής μεταξύ των αντίγραφών της ή να εξισώσουν την απόδοση των υποκείμενων υπολογιστικών κέντρων ΥΑΝ σύμφωνα με ένα δείκτη απόδοσης (π.χ. χρησιμοποίηση επεξεργαστικής ισχύς). Οι ελεγκτές τοπικού επιπέδου ασχολούνται κυρίως με θέματα που αφορούν τη λειτουργία των εφαρμογών και των αντίστοιχων εξυπηρετητών στους οποίους είναι εγκατεστημένες. 14

15 Local Level AC+RA Local Level AC+RA Ste 1 Ste , 2,, n , 2,, n, 2,, 1 Global level n , 2,, n , 2,, n Local Level Local Level AC+RA AC+RA Ste 3 Ste 4 Σχήμα 4 Αρχιτεκτονική Ελέγχου Υποδομών ΥΑΝ Για παράδειγμα ο έλεγχος αποδοχής, ΕΑ, (admsson control) ο οποίος αποδέχεται ή απορρίπτει αιτήσεις ανάλογα με το εισερχόμενο φορτίο και η κατανομή πόρων, ΚΠ, (resource allocaton) που καθορίζει τους πόρους του κάθε ΕΜ (επεξεργαστική ισχύ, μνήμη και εύρος δικτύου) είτε αλλάζοντας το χρονικό ποσοστό που ανατίθεται σε μια εφαρμογή ο επεξεργαστής ή χρησιμοποιώντας το μηχανισμό δυναμικής διαβάθμισης της τάσης και της συχνότητας (Dynamc Voltage and Frequency Scalng) του επεξεργαστή έτσι ώστε να αλλάζει ο ρυθμός εξυπηρέτησης των αιτήσεων. Οι προτεινόμενες λύσεις στη βιβλιογραφία για τους ελεγκτές τοπικού επιπέδου χρησιμοποιούν θεωρία ουρών ή γραμμικά καταστατικά μοντέλα από τη θεωρία ελέγχου για τη μοντελοποίηση της δυναμικής λειτουργίας των εφαρμογών. Αυτές οι τεχνικές μοντελοποίησης συνδυάζονται με κλασσικές τεχνικές ελέγχου, όπως αναλογικοί ολοκληρωτικοί διαφορικοί ελεγκτές (Proportonal Integral Dervatve, PID), ή με μοντέρνες θεωρίες ελέγχου, όπως ο έλεγχος πρόβλεψης μοντέλου (model predctve control) ή ο προσαρμοστικός έλεγχος (adaptve control) ή 15

16 με θεωρία βελτιστοποίησης για να επιτύχουν τους στόχους του ΣΕΥ. Ωστόσο όλες οι παραπάνω τεχνικές έχουν κάποια μειονεκτήματα. Συνήθως οι περισσότερες από αυτές ασχολούνται είτε με το πρόβλημα του ΕΑ ή με το ΚΠ χωριστά. Αυτό μπορεί να οδηγήσει είτε σε υπερτροφοδότηση πόρων (overprovsonng) αν ο σχεδιασμός του καταμερισμού των πόρων έχει γίνει για το μέγιστο φορτίο ή ένα μεγάλο μέρος των εισερχόμενων αιτήσεων να απορρίπτεται επειδή η στατική ανάθεση υπολογιστικών πόρων ωα αδυνατεί να τις εξυπηρετήσει. Επίσης δεν εξετάζουν αν το επιλεγμένο σημείο λειτουργίας του συστήματος είναι εφικτό και δεν ικανοποιεί τους υφιστάμενους περιορισμούς του συστήματος. Τελικά οι περισσότερες μελέτες δεν παρέχουν καμία ανάλυση της ευστάθειας της προτεινόμενης λύσης. Στη παρούσα διδακτορική διατριβή παρουσιάζονται δυο διαφορετικοί ελεγκτές τοπικού επιπέδου, οι οποίοι επιλύουν τα παραπάνω μειονεκτήματα και παρέχουν εγγύηση της απόδοσης των δικτυακών εφαρμογών ανεξαρτήτως των μεταβολών του εισερχόμενου φορτίου και των περιορισμών του συστήματος. Πιο συγκεκριμένα: Ικανοποίηση των στόχων που καθορίζονται από το ΣΕΥ. Βέλτιστη χρήση των πόρων του συστήματος. Σχεδιασμός και υλοποίηση ελεγκτών που επιλύουν τα προβλήματα των ΕΑ και ΚΠ ταυτόχρονα. Ακριβής μοντελοποίηση της λειτουργίας των εφαρμογών. Ικανοποίηση των υπαρχόντων περιορισμών του συστήματος. Εύρεση εφικτών σημείων λειτουργίας του συστήματος βάσει του ΣΕΥ. Εγγύηση και ανάλυση της ευστάθειας του συστήματος. Εφαρμογή καινοτόμων μεθόδων από τη θεωρία ελέγχου. Το ΕΑΚΠ (έλεγχος αποδοχής και κατανομή πόρων) είναι ένα αυτόνομο πλαίσιο (framework) μοντελοποίησης και ελέγχου, το οποίο παρέχει ακριβή μοντέλα και λύνει ενοποιημένα τα προβλήματα ΕΑ και ΚΠ των δικτυακών εφαρμογών που είναι συγκεντρωμένες σε υπολογιστικά κέντρα ΥΑΝ. Στόχος του ΕΑΚΠ είναι να μεγιστοποιεί την είσοδο των αιτήσεων των χρηστών στη παρεχόμενη υπηρεσία εκπληρώνοντας παράλληλα και τις προδιαγεγραμμένες απαιτήσεις ΠΥ. Μια ομάδα γραμμικών καταστατικών μοντέλων με επιπρόσθετες αβεβαιότητες έχει χρησιμοποιηθεί με στόχο να καλύψει τις μεταβολές του εισερχόμενου φορτίου και την ποσοτικοποίηση των μη γραμμικών όρων του συστήματος. Για το σχεδιασμό του δυικού ελεγκτή, χρησιμοποιήθηκε ένας ελεγκτής από τη θεωρία συνόλων (settheoretc controller) που μας δίνει τη δυνατότητα να διασφαλίζεται η ευστάθεια του συστήματος και η ικανοποίηση όλων των υπαρχόντων περιορισμών. Επίσης το ΕΑΚΠ έχει τη δυνατότητα επιλογής του σημείου λειτουργίας του συστήματος ανάμεσα από διαφορετικά εφικτά σημεία ισορροπίας (equlbrum ponts), τα οποία 16

17 System Constrants LTI Models Equllbrum Pont Feedback Controller u Σχήμα 5 Δομή του ΕΑΚΠ πλαισίου μοντελοποίησης και ελέγχου. έχουν υπολογιστεί μέσω της επίλυσης ενός προβλήματος γραμμικού προγραμματισμού. Η δομή του ΕΑΚΠ φαίνεται στο σχήμα 5. Ο δεύτερος τοπικός ελεγκτής που παρουσιάζεται σε αυτή τη διατριβή είναι ένα αυτόνομο πλαίσιο (framework) μοντελοποίησης και ελέγχου κατανεμημένων δικτυακών εφαρμογών σε περιβάλλον ΥΑΝ, το οποίο λύνει συγχρόνως τα προβλήματα ΕΑ και ΚΠ με ενιαίο τρόπο. Πιο συγκεκριμένα, ένα γραμμικό παραμετρικά μεταβαλλόμενο (Lnear Parameter Varyng) καταστατικό μοντέλο χρησιμοποιείται για να περιγράψει τη δυναμική συμπεριφορά του συστήματος. Αυτό το είδος των γραμμικών καταστατικών μοντέλων είναι κατάλληλο για μοντελοποίηση της λειτουργίας των δικτυακών εφαρμογών επειδή αυτή εξαρτάται από διάφορες παραμέτρους όπως ο ρυθμός εξυπηρέτησης και ο ρυθμός των εισερχόμενων αιτήσεων. Αυτές οι παράμετροι μπορούν εύκολα να ενσωματωθούν στη συγκεκριμένη κατηγορία μοντέλων. Στη συγκεκριμένη περίπτωση σαν παράμετρο του μοντέλου χρησιμοποιούμε την πρόβλεψη του ρυθμού άφιξης των εισερχόμενων αιτήσεων που προκύπτει από ένα προβλεπτή Holt. Οι συνθήκες λειτουργίας των υπηρεσιών που μοιράζονται ένα σύνολο εξυπηρετητών αποφασίζονται σύμφωνα με διάφορα κριτήρια βελτιστοποίησης. Ένα σύνολο εφικτών ονομαστικών σημείων λειτουργίας υπολογίζεται με τέτοιο τρόπο έτσι ώστε να ικανοποιούνται οι προϋποθέσεις ΠΥ. Στο σχεδιασμό της στρατηγικής ελέγχου υπολογίζεται ένας καταστατικός ελεγκτής ανάδρασης (state feedback controller) και μια περιοχή ελκτικότητας (doman of attracton), τέτοια ώστε να εγγυάται η τοπική ασυμπτωτική ευστάθεια και η ικανοποίηση των υφιστάμενων περιορισμών του συστήματος για ολόκληρο το σύνολο των επιθυμητών εφικτών σημείων ισορροπίας. Η υλοποίηση του συγκεκριμένου ελεγκτή είναι εύκολη, καθώς η υπολογιστική πολυπλοκότητά του είναι μικρή και σε κάθε χρονικό διάστημα μόνο ένα πρόβλημα γραμμικού προγραμματισμού και επιλογής του σημείου λειτουργίας πρέπει να επιλύονται. Η συνολική δομή του αυτόνομου πλαισίου μοντελοποίησης και ελέγχου απεικονίζεται στο σχήμα 6. 17

18 System Constrants Predctor ~ LPV Model Equllbrum Pont Feedback Controller u Σχήμα 6 Δομή του αυτόνομου πλαισίου μοντελοποίησης και ελέγχου. Οι κύριες ερευνητικές συνεισφορές των δυο προαναφερθέντων ελεγκτών τοπικού επιπέδου σε υποδομές ΥΑΝ στην παρούσα διδακτορική διατριβή συνοψίζονται στα παρακάτω σημεία: Ακριβής μοντελοποίηση της δυναμικής λειτουργίας δικτυακών υπηρεσιών που μοιράζονται τους πόρους ενός συνόλου εξυπηρετητών. Υψηλή απόδοση και ικανοποίηση των προκαθορισμένων στόχων ενός ΣΕΥ. Ικανοποίηση των φυσικών περιορισμών του συστήματος. Καθορισμός ενός συνόλου εφικτών ονομαστικών σημείων λειτουργίας του συστήματος, τα οποία επιλέγονται ανάλογα με τις παρούσες συνθήκες λειτουργίας. Εγγύηση της ευστάθειας του συστήματος. 3. Διάρθρωση της Διατριβής. Η διατριβή είναι οργανωμένη σε τέσσερα κεφάλαια. Στο πρώτο κεφάλαιο παρουσιάζεται μια γενική περιγραφή του τι είναι ΥΑΝ, δίνοντας κάποιους βασικούς ορισμούς και κάποια χαρακτηριστικά γνωρίσματα. Περιγράφεται ποιος είναι ο ρόλος των παρόχων, η αρχιτεκτονική των επιχειρηματικών και τεχνικών μοντέλων καθώς και η κατηγοριοποίηση των υποδομών ΥΑΝ ανάλογα με τη χρήση τους. Επίσης παρουσιάζονται συνοπτικά τα πιο σημαντικά τρέχοντα ερευνητικά προβλήματα σχετικά με τις ΥΑΝ. Το δεύτερο κεφάλαιο ασχολείται με την παρακολούθηση των κατανεμημένων δικτυακών υπηρεσιών. Αρχικά δίνονται όλες οι απαραίτητες θεμελιώδεις πληροφορίες και έννοιες της ασαφούς λογικής. Έπειτα δίνονται οι ορισμοί των ροών (flows), των γεγονότων (events), των ζευγών των ροών (flow pars) και των ζευγών των γεγονότων (event pars) που είναι απαραίτητοι για την ανάλυση των Netflow δικτυακών ιχνών και την προεπεξεργασία για την μετέπειτα αξιολόγησή τους από τον 18

19 ΑΜΣ. Μετά από αυτά τα προπαρασκευαστικά βήματα περιγράφεται αναλυτικά πως παράγεται ένα σύνολο ασαφών κανόνων από έναν υβριδικό γενετικό αλγόριθμο. Τέλος η απόδοση του ασαφούς μηχανισμού ταξινόμησης δοκιμάζεται σε πραγματικά ίχνη από ένα δίκτυο εταιρείας και μιας πανεπιστημιούπολης και τα αποτελέσματα δείχνουν ότι ο μηχανισμός ταξινόμησης είναι ακριβής και μπορεί να αναγνωρίσει τις υφιστάμενες λειτουργικές σχέσεις χωρίς να έχει καμία πρωτύτερη του δικτύου. Το τρίτο κεφάλαιο παρουσιάζει τους δύο ελεγκτές τοπικού επιπέδου που επιλύουν το πρόβλημα του καταμερισμού των πόρων της ΥΑΝ. Πρωτίστως δίνονται οι βασικοί ορισμοί μοντελοποίησης και θεωρίας ελέγχου και έπειτα μια αναλυτική περιγραφή των υπαρχουσών μεθόδων μαζί με τα πλεονεκτήματα και τα μειονεκτήματα τους. Ο πρώτος ελεγκτής που παρουσιάζεται είναι το ΕΑΚΠ. Εξηγείται πως προκύπτει και αναγνωρίζεται η ομάδα των γραμμικών καταστατικών μοντέλων και διατυπώνονται μαθηματικά οι περιορισμοί του συστήματος. Στη συνέχεια παρουσιάζονται τα κριτήρια και τα βήματα για τον υπολογισμό ενός εφικτού ονομαστικού σημείου λειτουργίας που εγγυάται την ικανοποίηση των περιορισμών. Μετά παρουσιάζεται ο σχεδιασμός του δυικού ελεγκτή και οι ειδικές ιδιότητες που προσδίδονται στο σύστημα κλειστού βρόχου. Τέλος παρουσιάζεται η απόδοση του πλαισίου ΕΑΚΠ σε ένα πραγματικό σύστημα και η σύγκρισή του με δυο άλλες γνωστές τεχνικές ελέγχου. Στο δεύτερο μέρος του κεφαλαίου παρουσιάζεται ο δεύτερος ελεγκτής τοπικού επιπέδου. Αρχικά περιγράφονται αναλυτικά το γραμμικό παραμετρικά μεταβαλλόμενο μοντέλο, οι υφιστάμενοι περιορισμοί του συστήματος και ο προβλεπτής της εισερχόμενης κίνησης. Στη συνέχεια παρουσιάζονται η διαδικασία υπολογισμού ενός συνόλου ονομαστικών σημείων λειτουργίας του συστήματος και ο σχεδιασμός της στρατηγικής ελέγχου με σκοπό την εγγύηση της ευστάθειας και της ικανοποίησης των περιορισμών του συστήματος σε κάθε περίπτωση. Στο τελευταίο μέρος του κεφαλαίου περιέχεται η υλοποίηση του ελεγκτή σε ένα πραγματικό περιβάλλον ΥΑΝ και η σύγκρισή του με έναν ελεγκτή πρόβλεψης μοντέλου. Στο τέταρτο κεφάλαιο υπογραμμίζονται όλα τα συμπεράσματα της διατριβής γύρω από την παρακολούθηση και τον έλεγχο των δικτυακών εφαρμογών σε πλατφόρμες ΥΑΝ. Επίσης παρουσιάζονται κάποιες απόψεις για τη συνέχιση της έρευνας μετά τη διατριβή. Τέλος στο παράρτημα παρουσιάζεται οι δημοσιεύσεις σε διεθνή συνέδρια και περιοδικά που προέκυψαν κατά τη διάρκεια της εκπόνησης της διδακτορικής διατριβής και η σχετική βιβλιογραφία. 19

20 Abstract 1. Descrpton and Objectves of the Dssertaton Cloud computng has recently emerged as a new paradgm for hostng and delverng servces over the Internet. Wth the rapd development of processng and storage technologes and the success of the Internet, computng resources have become cheaper, more powerful and more avalable than ever before. These avalable resources (CPU, storage and network) are organzed n data centers that are spread around the glove. Ths technologcal trend has enabled the realzaton of a new computng model called cloud computng, n whch resources are provded as a product that can be leased by users through the Internet n an on-demand fashon. Cloud computng offers to customers many attractve arguments to shft ther busness and web applcatons from propretary nfrastructure to a cloud computng platforms. The most mportant ncentves are the pay-as-you-go prcng model, the dynamc resource allocaton, the hgh scalablty, the reducton of the operatng and mantenance expenses and the easy access from many dfferent devces. Nowadays, a customer s not oblged to buy all the necessary hardware equpment of a prvate data center and pay for ts operaton and mantenance, but he can lease on-demand only the necessary computng and network resources to develop hs applcatons and pay only for the usage. Ths means that he elmnates the cost of the electrcal consumpton, the hardware falures and the number of expertse n IT department. Cloud computng provders are separated n three categores. The provders are connected on a herarchcal layered model, where the provder of one layer s customer of the prevous level provder. At the frst layer of ths model, Infrastructure as a Servce (IaaS) provders offer all the necessary computng and network equpment, e.g. servers, storage devces and routers, swtches, n terms of Vrtual Machnes (VMs). On the next level, Platform as a Servce (PaaS) provder offer operatng system support and software development frameworks. These two layers are usually called the utlty computng layer. On the top level, Software as a Servce (SaaS) provders develop on demand any applcaton over the Internet. Web applcatons, whch are deployed on cloud computng platforms, have dfferent requrements n terms of operatng cost, relablty and securty. Thus some certan types of cloud have been emerged. Publc clouds offer ther resources as servces to the general publc. They provde all the essental computng and network resources but they usually lack control over data, network and securty settngs, whch s an obstacle to deploy busness servces wth strct requrements. Thus prvate or nternal clouds have been desgned for exclusve use by a sngle organzaton and t offers hgh level of control, relablty and securty. Hybrd cloud s a mxture of publc and prvate cloud models. It offers more flexblty than both publc and prvate clouds and leverages the benefts both of them n order to fulfll the requrements of all servces. 20

21 Fnally Vrtual Prvate cloud explots the vrtual prvate network (VPN) technology that allows servce provders to desgn ther own topology and securty settngs such as frewall rules on the top of publc clouds. The above layered model of the provders and the underlyng nfrastructure provdes the followng mportant features to the cloud computng platforms: Geo-dstrbuton of applcatons and ubqutous network access from many dfferent devces. Servce orented applcatons whch operaton s descrbed by a Servce Level Agreement (SLA) between the provder and the customer. Dynamc resource provsonng accordng to the current servce demand. Shared pool of resources can be adaptvely assgned to multple resource consumers. Reducton of operatng cost and rsks such as electrcal consumpton and hardware falures. Pay-per-use prcng model. Although the rapd development, cloud computng s currently at ts early steps and there are many ssues stll to be addressed. Furthermore, new challenges contnuously emerge from many ndustral applcatons. The most mportant research problems of cloud computng are the followng: Automated resource provsonng addresses the problem of allottng the computng resources to all applcatons n such a manner that ther SLA requrements are satsfed and hgh aglty s acheved towards rapd demand fluctuatons such as n flash crowd effect. Vrtual Machne mgraton s a technque, whch moves the VM of an applcaton n order to avod hotspots and bottleneck of resources n a specfc group of servers. Servce consoldaton s relatve to the above problem and t concerns the maxmzaton of the resource utlzaton whle mnmzng energy consumpton n a cloud computng envronment. Hostng many components of dfferent applcatons on a server cluster allows usng only the necessary number of servers and actvatng or deactvatng them accordng to the servce demand. Energy management addresses the problem of electrcal consumpton of data centers, whch s not a neglgble cost any more. Many provders set energy savng as one of ther prmary goal n the desgn of ther capacty plannng. Traffc montorng and analyss s a challengng problem because of the huge volume of nformaton data that need to be processed. Havng an onlne vew of what s gong on n a data center s mportant n order to 21

22 avod hardware falures, preserve hgh relablty and securty and plannng necessary expansons and changes of computng and network resources. Data securty s another mportant research topc n cloud computng. The nfrastructure provder must acheve confdentalty for secure data access and transfer and audtablty for attestng whether securty settng of applcatons has been tampered or not. Software frameworks should be developed for schedulng tasks and for scalable and fault-tolerant data processng. Storage technologes and data management for dstrbuted processng of data-ntensve tasks. Novel archtectureσ whch try to combne dfferent technologes, such as cloud computng and peer-to-peer networks n order to acheve meda delvery n a dstrbuted fashon. Ths thess addresses two of the aforementoned research topcs. Frstly a useful traffc montorng technque s developed to unvel the underlyng relatonshps between the varous components of web servces. Secondly automated resource provsonng problem s addressed usng two dfferent control theoretc solutons. Nowadays web applcatons are usually mult-ter, snce they consst of many dfferent components, and are manly nstalled n data centers of prvate or publc clouds. Due to the fnancal mpact of SLAs, ntegrated management tools, whch automatcally montor the operaton and performance of consoldated servces, are necessary n order to proactvely handle any arsng problem. A typcal data center usually consoldates the components of thousand applcatons. These components nteract n an unpredctable way n order to complete dfferent types of transactons. Network admnstrators usually do not have a clear vew of the servces operaton condtons. The current montorng systems provde data and statstcs of traffc condtons based on ports wthout further nterpretaton about how the servce components collaborate. Relatvely to the dsadvantages of the studes n the lterature, the thess focuses on the creaton of a tool for analyss of network traffc, whch mnes relatonshps between network and servce components and measures ther strength wthout havng any pror knowledge of exstng applcatons. The man contrbutons are the followng: The creaton of a servce graph overlay, whch depcts the operatonal dependences between the components of the dstrbuted web servces n a data center. Relable dentfcaton of the underlyng relatonshps wthout any a pror knowledge about them. Quantfcaton of the relatonshp s strength. 22

23 Easy and offlne tranng procedure of the nference mechansm. Fast and onlne process of the traffc data. Small number of erroneously produced data. Ths servce overlay llustrates the structure of servces and nterconnectons between network clents and servers or databases that are afflated wth a specfc servce. Ths overlay s vtal for several admnstratve tasks, such as detectng msconfguratons or DoS attacks, montorng servce performance and desgn or expanson of the cloud computng nfrastructure. NetFlow traces, collected from an enterprse network and a campus network of a unversty, nclude tme nformaton and typcal flow attrbutes,.e. IP addresses, ports and protocols. Intally a smple algorthm, based on very smple and realstc traffc patterns, s appled that reveals all possble dependences. Then a flexble fuzzy nference mechansm (FIM) s developed, whch determnes whether the canddate relatonshp s true or not and measures ts strength. The second part of our study s the desgn and tranng of FIM. FIM conssts of a small set of fuzzy rules (.e. 30 rules) produced by a hybrd genetc algorthm, whch s a tranng process wth hgh classfcaton rate. The man contrbutons are the followng: We dscover relatonshps that correspond to the applcaton structure, such as web and fle servers, wthout any pror knowledge. The generated erroneous relatonshps are few and our approach measures accurately the strength of every dependency. The hybrd genetc algorthm has a hgh classfcaton rate and needs only a small fracton of tranng data (10%). Also t s an offlne process, applcable to dfferent networks. The set of fuzzy rules s not necessarly large: 30 rules suffce to classfy correctly all canddate relatonshps. Furthermore the classfcaton by the fuzzy rules set can be done onlne. The other research topc, whch s addressed n ths thess, s automated resource provsonng of consoldated web servces on cloud computng platforms. Modern vrtualzaton technology s the key factor for the consoldaton of many dstrbuted web servces nto large data centers. Provders offer an executon platform wth all the essental means (.e hardware nfrastructure and software tools) that applcatons can use on demand. Web servces usually consst of many components such as web servers and databases, whch are deployed on separate VMs. These VMs can be hosted wthn one machne or span across many machnes. The performance of a web servce s descrbed n a SLA between the servce provder and the customer; the Servce Level Objectves (SLOs) are the network metrcs, whch prescrbe exactly the desred Qualty of Servce (QoS) levels. The most common SLOs are the request response tme and the served request rate durng a fxed tme nterval. From the customer s vew, the goals are the satsfacton of some predefned nomnal values of 23

24 the SLOs and a guaranteed level of QoS wth the mnmum fnancal cost. These objectves are contradctve to the goals of the provder, who ams for a management system that allocates the resources to each servce optmally n a manner that acheves SLOs, ensures avalablty of resources under any workload crcumstances and mnmzes the operatonal cost (e.g. the consumed power energy). In addton, the workload of these web servces s generally unpredctable and hghly varant, whle the avalable computng resources are fnte and subject to constrants. All these factors make the management of the cloud computng nfrastructure an open and challengng research feld. There are two layers of control of web servces, specfcally, the local level and the global level. On the global level, controllers try to share the volume of requests among the replcas of an applcaton or equalze the performance at each of the underlyng data centers accordng to a performance ndcator. Local level controllers manly concern applcaton specfc or server specfc problems,.d. admsson control (AC) that rejects requests under peak workload condtons and resource allocaton (RA), whch determnes for each VM resources (CPU, memory, network bandwdth) ether by changng ther capacty or usng the Dynamc Voltage and Frequency Scalng (DVFS) mechansm of servers to change ther servce rate. The exstng solutons about local level controllers n lterature use queung theory or lnear state space models from control theory to model the dynamc operaton of the servces and modern control technques, such as PI or model predctve control (MPC), or optmzaton theory to acheve the SLA objectves. However they have some certan drawbacks. They usually address the problem of admsson control and capacty plannng separately. Ths can be lead to overprovsonng whether the capacty plannng s desgn only for peak demand and a large fracton of ncomng requests can be rejected because statc resource allocaton cannot serve them. They do not examne f the selected operatng ponts are feasble and satsfy the underlyng system constrants. Fnally most of them do not provde any stablty analyss of the proposed solutons. In the thess two local level controllers are presented, whch solve the above lmtatons and guarantee the performance of the applcaton under the system constrants and any varaton of ther ncomng workload. The man contrbutons of the thess are: Satsfacton of the defned targets by the SLA. Optmal usage of the computatonal and network resources. Desgn and mplementaton of dual controllers that address admsson control and resource allocaton smultaneously. Accurate modelng of the web servce operaton. Satsfacton of the underlyng system constrants. Computaton of feasble operaton ponts accordng to SLA. Guarantee and analyss of the system stablty. 24

25 Applcaton of novel methods of control theory to recent network research problems. ACRA (Admsson Control and Resource Allocaton) s an autonomc modelng and control framework that provdes accurate modelng, jont admsson control and capacty allocaton among dfferent consoldated servces on cloud computng data centers. The objectve s to maxmze the provder s revenue (by maxmzng admttance of customers to the provded servce), whle fulfllng QoS requrements. A group of lnear state-space models wth addtve uncertantes s used n order to cover the varaton of workload and quantfy the system s nonlneartes. For the desgn of the dual controller, a set-theoretc controller s used whch provdes stablty and satsfacton of the underlyng system constrants. Fnally ACRA has the ablty to adapt between several operatng ponts, because t always assures that the system wll be drven n the neghborhood of a feasble equlbrum pont. The second local level controller of the thess s an autonomous modelng and control framework that solves admsson control and resource allocaton smultaneously n a unfed way. In specfc, a Lnear Parameter Varyng (LPV) state space model s adopted to capture the dynamc behavor of the underlyng nfrastructure. The operatng condtons are determned accordng to an optmzaton crteron. A feasble operatng pont, whch satsfes the desred QoS nomnal values, s computed. The resultng stablzng state feedback control law s an affne state-dependent control law that can be easly mplemented. The computatonal complexty of the controller mplementaton s small, snce at every tme nstant only a lnear program and a pont locaton problem are solved. Fnally, convergence to the feasble operatng pont and satsfacton of the system s constrants are guaranteed, for a number of desred operatng ponts of nterest. 2. Structure of the Dssertaton Ths dssertaton s structured n four chapters. The frst chapter s a general tutoral of what cloud computng s, gvng some basc defntons and key features. The role of provders, the archtecture of the busness model and the categorzaton of cloud computng nfrastructure are demonstrated. Also the most mportant open research problems are analyzed brefly n ths chapter. The second chapter s dedcated to the montorng of the consoldated wed applcatons. Intally the fundamental defntons and deas of fuzzy logc are gven. Then the defntons of flows, events, flow pars and event pars are gven n order to process the data from NetFlow traces and prepare them for classfcaton by FIM. After ths preparaton step, t s descrbed n full detals how the set f fuzzy rules are produced by a hybrd genetc algorthm. Fnally the performance of ths fuzzy logc classfer s tested on real traces from a prvate busness network and the network of a unversty campus and the results llustrates that t s accurate and can clearly dentfy the underlyng relatonshps wthout any prevous knowledge of the network. 25

26 Thrd chapter presents the two local level controllers that address the problem of capacty provsonng. Frstly, all the necessary defntons of modelng and control theory are gven. Then there s an analytcal descrpton of the exstng methods and ther pros and cons. The frst presented controller s ACRA. It s descrbed how the group of lnear state space model s nferred and dentfed. Then the sets of system constrant are formulated and t s explaned what are the crtera and the steps to compute a feasble operatng pont that guarantees the underlyng constrants. After ths step, t s llustrated the desgn and the specal propertes of the dual set-theoretc controller are llustrated. The evaluaton secton shows the performance of ACRA and a comparson wth two other well-known controllers. Fnally the components of the second local level controller are presented. Frstly, the LPV modelng, the underlyng constrants and the request rate predctor are analytcally descrbed. Then the computaton of a set of feasble operaton pont and the procedure of computng a control strategy, whch guarantees the stablty of the system and the satsfacton of the constrants, s explaned. The last secton of ths chapter ncludes the evaluaton of the controller on a real testbed and the comparson wth a correspondng MPC method. The fourth chapter hghlghts all the concluson of the dssertaton about montorng and control of web applcatons on cloud computng platforms. Also some aspects of possble future work are gven. Appendx presents the bblography and the dssertaton publcatons n nternatonal journal and conferences. 26

27 Contents CHAPTER 1 Cloud Computng Overvew of cloud computng Defntons Characterstcs Provders and Busness Model Types of Cloud and Cloud computng technologes Types of Cloud Archtectural desgn of data centers Dstrbuted Fle System Dstrbuted Applcaton Framework Key Features Research Challenges Automated Resource Provsonng Vrtual Machne Mgraton Servce Consoldaton Energy Management Traffc Management and Analyss Data Securty Software Frameworks Storage technologes and Data Management Novel Archtectures CHAPTER 2 Montorng of Dstrbuted Cloud Servces Fuzzy Logc Lngustc Varables Membershp Functons Set of Fuzzy Rules and Inference Mechansm Defuzzfcaton Mnng Network Relatonshps Pre-processng Data Flows and Events Flow and Event Pars Confdence Varables Relatonshp Dscovery

28 2.3 Fuzzy Inference Mechansm Evaluaton Smulatons Real Data Conclusons CHAPTER 3 Control of Dstrbuted Web Servces on Cloud Computng Infrastructure Basc Defntons of Modelng and Control Theory Lnear Tme Invarant State Space Models Lnear Parameter Varyng State Space Models Stablty ACRA: A Unfed Admsson Control and Resource Allocaton Framework for Vrtualzed Envronments Modelng and System Identfcaton State and Input Constrants Determnaton of the Equlbrum Pont ACRA Controller Desgn Evaluaton An Autonomous Admsson Control and Resource Allocaton Framework for Consoldated Web Applcatons Modelng and Identfcaton Request Rate Predctor State and Input Constrants Determnaton of the Operatng Pont Controller Desgn Evaluaton CHAPTER 4 Conclusons and Future Work Conclusons Future Work Publcatons Bblography

29 Lsts of fgures Fgure 1.1 Busness Model of Cloud Computng Fgure 1.2 Layered Archtecture of Data Center Fgure 2.1 Structure of a Fuzzy Logc System Fgure 2.2 Memebershp Functons: (a) sngleton, (b) trangular, (c) trapezodal, (d) Gaussan shape Fgure 2.3 Membershp Functons of a Lngustc Varable Fgure 2.4 Defuzzfcaton of a Fuzzy Logc System Fgure 2.5 Relatonshp Dscovery System Fgure 2.6 Flow Pars: (a) cs flow par, (b) ss flow par, (c) ms flow par Fgure 2.7 Tranng: Average Classfcaton Rate of Smulatons Fgure 2.8 Evaluaton: Average Classfcaton Rate of Smulatons Fgure 2.9 Real Data: Average Classfcaton Rate of Tranng Process Fgure Real Data: Evaluaton of the produced FIMs Fgure 2.11 Graph of Relatonshps Fgure 3.1 Control Archtecture of Cloud Computng Infrastructure Fgure 12 Structural Dagram of ACRA Framework Fgure 3.3 Group of LTI Models Fgure 3.4 State and Input Constrants set S z, S v(z), Postevely Invarant Set Δ, Target Set R Fgure Average Response Tme of ACRA, PI, MPC Fgure 3.6 Reference and Admtted Request Rate Fgure 3.7 Performance of MPC, PI controllers usng a sngle LTI Model Fgure 3.8 Average Response Tme and VMs Capacty Allocaton of ACRA Fgure 3.9 System Archtecture Fgure 3.10 Structural Dagram of Autonomous Framework Fgure DoAs R of each operatng pont x ref,i, =1,,N Fgure Tested Dagram VM,j of th applcaton on j th server Fgure Overall Performance of Consoldated Applcatons Fgure DoA of operatng ponts (red lnes and crcles), DoA of the desred operatng pont (blue lne and square) and System Trajectory (black sold lne) Fgure 3.15 Alternatve soluton of pont locaton problem Fgure Comparson wth MPC Controller

30 Glossary of Acronyms IT IaaS PaaS SaaS VM VPC VPN MDC GFS HDFS SLA CRM SLO QoS ISP TPM FIM MIMO MISO AppLst GA HGA AC RA DVFS LTI LPV MPC DoA LF ACRA RLS PI BFR Informaton Technology Infrastructure as a Servce Platform as a Servce Software as a Servce Vrtual Machne Vrtual Prvate Cloud Vrtual Prvate Network Modular Data Center Google Fle System Hadoop Dstrbuted Fle System Servce Level Agreement Customer relatonshp Management Servce Level Objectve Qualty of Servce Internet Servce Provder Trusted Platform Module Fuzzy Inference Mechansm Mult-Input Mult-Output Mult-Input Sngle-Output Applcaton Lst Genetc Algorthm Hybrd Genetc Algorthm Admsson Control Resource Allocaton Dynamc Voltage and Frequency Scalng Lnear Tme Invarant Lnear Parameter Varyng Model Predctve Control Doman of Attracton Lyapunov Functon Admsson Control and Resource Allocaton Recursve Least Square Proportonal Integral Best Ft Rate 30

31 CHAPTER 1 Cloud Computng. Cloud Computng s assocated wth a new paradgm for provsonng servces over the Internet. Cloud computng becomes attractve snce t elmnates the requrements for users to plan ahead for provsonng, and allows enterprses to start from the mnmum and ncrease resources only when there s a rse on servce demand. Avalable resources (e.g., CPU, storage and network) are provded as general utltes that can be leased and released by users through the Internet n an on-demand fashon. In a cloud computng envronment, the provder s separated n the followng two roles. The nfrastructure provders manage cloud platforms and lease resources accordng to a usage-based prcng model to servce provders, who rent resources from one or many nfrastructure provders to serve the end users. The emergence of cloud computng has made a tremendous mpact on the Informaton Technology (IT) ndustry over the past few years, where large companes such as Google, Amazon and Mcrosoft compete to provde more powerful, relable and cost-effcent cloud platforms, and busness enterprses seek to reshape ther busness models to gan beneft from ths new paradgm. Ths rest of ths chapter presents a summary of the most mportant characterstcs, models and research challenges of the exstng cloud computng platforms. The followng secton demonstrates an overvew of cloud computng, ncludng ts defnton and the most domnant features. Secton 1.2 descrbes the busness and the operatonal models of cloud computng. Secton 1.3 ncludes the exstng types of clouds and the archtectural desgn of data centers. Secton 1.4 summarzes the most mportant key features of the cloud computng nfrastructures. Fnally the last secton ncludes the research challenges that have not been fully addressed yet. 1.1 Overvew of cloud computng Defntons There are many defntons of cloud computng, but they all seem to focus on just certan aspects of the technology. We adopt the defnton of cloud computng provded by the Natonal Insttute of Standards and Technology (NIST) [1] as t covers opnon all the essental aspects of cloud computng: Cloud computng s a model for enablng convenent, on-demand network access to a shared pool of confgurable computng resources (e.g., networks, servers, storage, applcatons, and servces) that can be rapdly provsoned and released wth mnmal management effort or servce provder nteracton. The NIST defnton lsts fve essental characterstcs of cloud computng: on-demand self-servce, broad network access, resource poolng, rapd elastcty or expanson, 31

32 and measured servce. It also lsts three "servce models" (software, platform and nfrastructure), and four "deployment models" (prvate, publc, hybrd and vrtual prvate) that together categorze ways to delver cloud servces. The defnton s a mean for broad comparsons of cloud servces and deployment strateges, and provdes a baselne for dscusson from what s cloud computng to how to optmally use cloud computng. In [2], authors compared over twenty dfferent defntons from a varety of sources to confrm a standard defnton Characterstcs Cloud computng provdes several dfferent characterstcs that attract many customers to shft from tradtonal servce computng to ths new type of nfrastructure. The most mportant are summarzed below: No fxed nvestment: Cloud computng uses a pay-as-you-go prcng model. A servce provder does not need to nvest n the nfrastructure to start ganng beneft from cloud computng. He smply rents resources from the nfrastructure provder accordng to hs own needs and pays only for the usage. Easy access: Servces hosted n the cloud are generally web-based. Therefore, they are easly accessble through any devce wth Internet connectons, such as desktop and laptop computers or cell phones and PDAs. Reducng operatng cost: Resources n a cloud envronment can be rapdly allocated and de-allocated on demand. Hence, a servce provder no longer needs to allocate resources accordng to the peak load. Ths provdes huge savngs snce resources can be released to save on operatng costs when servce demand s low. Hghly scalable: Infrastructure provders offer large amount of resources from data centers and make them easly accessble. A servce provder can easly expand ts servce to large scales n order to handle rapd fluctuatons of servce demands (e.g., flash-crowd effect). 1.2 Provders and Busness Model Cloud computng adopts a layered busness model [3]. The servces of each layer are offered by a dfferent provder and every layer s consdered as a customer of the prevous layer. In practce, cloud computng servces can be grouped nto three categores: Infrastructure as a Servce (IaaS), Platform as a Servce (PaaS) and Software as a Servce (SaaS), Infrastructure as a Servce: IaaS refers to on-demand provsonng of nfrastructural resources, usually n terms of Vrtual Machnes (VMs). The cloud owner who provdes IaaS s called an IaaS provder. An example of IaaS provder s Amazon EC2 [4]. 32

33 Platform as a Servce: PaaS refers to provdng platform layer resources, ncludng operatng system support and software development frameworks. Examples of PaaS provders are Google App Engne [5], Mcrosoft Wndows Azure [6] and Force.com [7]. Software as a Servce: SaaS refers to provdng on demand applcatons over the Internet. Examples of SaaS provders are Salesforce.com [7], Rackspace [8]. Fgure 1.1 depcts the archtecture of the layered busness model. The PaaS provder runs ts cloud on top of an IaaS provder s cloud. However, usually IaaS and PaaS provders are often parts of the same organzaton. Ths s the reason that PaaS and IaaS provders are often called the nfrastructure or cloud provders. In more detals, the IaaS provders are responsble for managng the physcal resources of the cloud, ncludng physcal servers, routers, swtches, power and coolng system that are typcally located n data centers. A data center usually contans thousands of servers that are organzed n racks and nterconnected through swtches, routers or other fabrcs. Usually provdes explot the benefts of vrtualzaton at ths level. Vrtualzaton technology, such as Xen [9] and VMware [10] abstracts away the detals of physcal hardware and provdes dynamc vrtualzed End User Web Interface SaaS Utlty Computng PaaS IaaS Fgure 1.1 Busness Model of Cloud Computng 33

34 resources for hgh-level applcatons. A vrtualzed server s commonly called a vrtual machne (VM). Vrtualzaton s a fundamental element of cloud computng, as t provdes the capablty of poolng computng resources from clusters of servers and dynamcally assgnng or reassgnng vrtual resources to applcatons on-demand. Typcal ssues at ths layer nclude hardware confguraton, fault tolerance, traffc management, power and coolng resource management. The platform layer s bult on top of the nfrastructure layer and t conssts of operatng systems and applcaton frameworks. The purpose of ths layer s to mnmze the burden of deployng applcatons drectly nto VM contaners. Fnally at the hghest level of the herarchy, the SaaS provders offer the actual cloud applcatons. Dfferent from tradtonal applcatons, cloud applcatons can leverage the automatc-scalng feature to acheve better performance, avalablty and lower operatng cost. 1.3 Types of Cloud and Cloud computng technologes Types of Cloud There are many ssues to consder when movng an enterprse applcaton to the cloud envronment. Some servce provders are mostly nterested n lowerng operaton cost, whle others may prefer hgh relablty and securty. Accordngly, there are dfferent types of clouds, each wth ts own pros and cons: Publc clouds: A cloud n whch servce provders offer ther resources as servces to the general publc. They offer several key benefts to servce provders, ncludng no ntal captal nvestment on nfrastructure and shftng of rsks to nfrastructure provders. However, publc clouds lack fne-graned control over data, network and securty settngs, whch hampers ther effectveness n many busness scenaros. Prvate (nternal) clouds: are desgned for exclusve use by a sngle organzaton. A prvate cloud may be bult and managed by the organzaton or by external provders. A prvate cloud offers the hghest degree of control over performance, relablty and securty. Hybrd clouds: A hybrd cloud s a combnaton of publc and prvate cloud models that tres to address the lmtatons of each approach. In a hybrd cloud, part of the servce nfrastructure runs n prvate clouds whle the remanng part runs n publc clouds. Hybrd clouds offer more flexblty than both publc and prvate clouds. Specfcally, they provde tghter control and securty over applcaton data compared to publc clouds, whle stll facltatng on-demand servce expanson and contracton. On the contrary, desgnng a hybrd cloud requres carefully determnng the best splt between publc and prvate components. 34

35 Vrtual Prvate Cloud: An alternatve soluton to addressng the lmtatons of both publc and prvate clouds s called Vrtual Prvate Cloud (VPC). A VPC s a platform runnng on top of publc clouds. The man dfference s that a VPC leverages vrtual prvate network (VPN) technology that allows servce provders to desgn ther own topology and securty settngs such as frewall rules. VPC s essentally a more holstc desgn snce t not only vrtualzes servers and applcatons, but also the underlyng communcaton network as well. Addtonally, for most companes, VPC provdes seamless transton from a propretary servce nfrastructure to a cloud-based nfrastructure, owng to the vrtualzed network layer. Most servce provders select the rght cloud model accordng to the busness scenaro. For example, computaton-ntensve scentfc applcatons are best deployed on publc clouds for cost-effectveness. Arguably, certan types of clouds wll be more popular than others. In partcular, t was predcted that hybrd clouds wll be the domnant type for most organzatons. However, vrtual prvate clouds have started to gan more popularty Archtectural desgn of data centers A data center contans thousands of devces lke servers, swtches and routers. Proper plannng of ths network archtecture s crtcal, as t wll heavly nfluence applcatons performance and throughput n such a dstrbuted computng envronment. Furthermore, scalablty and reslency features need to be carefully consdered. Currently, a layered approach s the basc foundaton of the network archtecture desgn, whch has been tested n some of the largest deployed data centers. The basc layers of a data center consst of the core, aggregaton, and access layers, as shown n Fgure 1.2. The access layer s where the servers n racks physcally connect to the network. There are typcally 20 to 40 servers per rack, each connected to an access swtch wth a 1 Gbps lnk. Access swtches usually connect to two aggregaton swtches for redundancy wth 10 Gbps lnks. The aggregaton layer usually provdes mportant functons, such as doman servce, locaton servce, server load balancng, and more. The core layer provdes connectvty to multple aggregaton swtches and provdes a reslent routed fabrc wth no sngle pont of falure. The core routers manage traffc nto and out of the data center. A popular practce s to leverage commodty Ethernet swtches and routers to buld the network nfrastructure. In dfferent busness solutons, the layered network nfrastructure can be elaborated to meet specfc busness challenges. Bascally, the desgn of the data center network archtecture should meet the followng objectves [11], [12], [13]: 35

36 Internet Core.... Aggregaton Access Fgure 1.2 Layered Archtecture of Data Center Unform hgh capacty: The maxmum rate of a server to server traffc flow should be lmted only by the avalable capacty on the network-nterface cards of the sendng and recevng servers, and assgnng servers to a servce should be ndependent of the network topology. It should be possble for an arbtrary host n the data center to communcate wth any other host n the network at the full bandwdth of ts local network nterface. Free VM mgraton: Vrtualzaton allows the entre VM state to be transmtted across the network and mgrate a VM from one physcal machne to another. A cloud computng hostng servce may mgrate VMs for statstcal multplexng or dynamcally changng communcaton patterns to acheve hgh bandwdth for tghtly coupled hosts or to acheve varable heat dstrbuton and power avalablty n the data center. The communcaton topology should be desgned so as to support rapd vrtual machne mgraton. Reslency: Falures wll be common at scale. The network nfrastructure must be fault-tolerant aganst varous types of server falures, lnk outages, or server-rack falures. Exstng uncast and multcast communcatons should not be affected to the extent allowed by the underlyng physcal connectvty. 36

37 Scalablty: The network nfrastructure must be able to scale to a large number of servers and allow for ncremental expanson. Backward compatblty: The network nfrastructure should be backward compatble wth swtches and routers runnng Ethernet and IP. Because exstng data centers have commonly leveraged commodty Ethernet and IP based devces, they should also be used n the new archtecture wthout major modfcatons. Another area of rapd nnovaton n the ndustry s the desgn and deployment of shppng-contaner based, modular data center (MDC). In an MDC, normally up to a few thousands of servers, are nterconnected va swtches to form the network nfrastructure. Hghly nteractve applcatons, whch are senstve to response tme, are sutable for geo-dverse MDC placed close to major populaton areas. The MDC also helps wth redundancy because not all areas are lkely to lose power, experence an earthquake at the same tme Dstrbuted Fle System Google Fle System (GFS) [14] s a propretary dstrbuted fle system developed by Google and specally desgned to provde effcent, relable access to data usng large clusters of commodty servers. Fles are dvded nto chunks of 64 megabytes, and are usually appended to or read and only extremely rarely overwrtten or shrunk. Compared wth tradtonal fle systems, GFS s desgned and optmzed to run on data centers to provde extremely hgh data throughputs, low latency and survve ndvdual server falures. Inspred by GFS, the open source Hadoop Dstrbuted Fle System (HDFS) [15] stores large fles across multple machnes. It acheves relablty by replcatng the data across multple servers. Smlarly to GFS, data s stored on multple geo-dverse nodes. The fle system s bult from a cluster of data nodes, each of whch serves blocks of data over the network usng a block protocol specfc to HDFS. Data s also provded over HTTP, allowng access to all content from a web browser or other types of clents. Data nodes can talk to each other to rebalance data dstrbuton, to move copes around, and to keep the replcaton of data hgh Dstrbuted Applcaton Framework HTTP-based applcatons usually conform to some web applcaton framework such as Java EE. In modern data center envronments, clusters of servers are also used for computaton and data-ntensve jobs such as fnancal trend analyss, or flm anmaton. MapReduce [16] s a software framework ntroduced by Google to support dstrbuted computng on large data sets on clusters of computers. MapReduce conssts of one Master, to whch clent applcatons submt MapReduce jobs. The Master pushes 37

38 work out to avalable task nodes n the data center, strvng to keep the tasks as close to the data as possble. The Master knows whch node contans the data, and whch other hosts are nearby. If the task cannot be hosted on the node where the data s stored, prorty s gven to nodes n the same rack. In ths way, network traffc on the man backbone s reduced, whch also helps to mprove throughput, as the backbone s usually the bottleneck. If a task fals or tmes out, t s rescheduled. If the Master fals, all ongong tasks are lost. The Master records what t s up to n the fle system. When t starts up, t looks for any such data, so that t can restart work from where t left off. The open source Hadoop MapReduce project [17] s nspred by Google s work. Currently, many organzatons are usng Hadoop MapReduce to run large datantensve computatons. Fnally Table 1.1 demonstrates the man features of the most representatve commercal products. Cloud Product Mcrosoft Wndows Azure Amazon Elastc Compute Cloud (EC2) Google Applcaton Engne Salesforce.com Class of Provder PaaS IaaS PaaS PaaS Applcatons Compute & Storage Compute & Storage Web Applcatons Storage & CRM Vrtualzaton OS level OS level Applcaton Contaner CRM soluton User Access Web based Admnstraton & SDK Tools Amazon EC2 Command-lne Tools Web based Admnstraton Console Web Interface, App Supported Frameworks.NET Servces, SQL, Dynamc CRM, Sharepont Servces Lnux based Amazon Machne Image (AMI) Python Apex, S- controls Table 1.1 A survey of the man features of commercal cloud products 1.4 Key Features Cloud computng provdes dfferent features from the tradtonal servce computng. The most mportant features of the nfrastructure and servce provders are summarzed below, 38

39 Geo-dstrbuton and ubqutous network access: Clouds are generally accessble through the Internet and use t as a servce delvery network. Any devce wth Internet connectvty s able to access cloud servces. Furthermore, n order to acheve hgh network performance and localzaton, clouds consst of data centers located at many locatons around the world. Thus a servce provder can easly leverage geodversty to acheve maxmum servce utlty. Servce orented: In a cloud, each IaaS, PaaS and SaaS provder offers hs servce accordng to the Servce Level Agreement (SLA) negotated wth hs customers. SLA assurance s therefore a crtcal objectve of every provder. Mult-tenancy: In a cloud envronment, servces owned by multple provders are co-located n a sngle data center. The performance and management ssues of these servces are shared among servce provders and the nfrastructure provder. The owner of each layer only needs to focus on the specfc objectves assocated wth ths layer. However, multtenancy also ntroduces dffcultes n understandng and managng the nteractons among varous stakeholders. Shared resource poolng: The nfrastructure provder offers a pool of computng resources that can be dynamcally allocated to multple consumers. Such dynamc resource assgnment capablty provdes much flexblty to nfrastructure provders for managng ther own resource usage and operatng costs. For nstance, an IaaS provder can leverage VM mgraton technology to succeed n a hgh degree of server consoldaton, hence maxmzng resource utlzaton whle mnmzng cost such as power consumpton and coolng. Dynamc resource provsonng: One of the key features of cloud computng s that computng resources can be obtaned and released n an on demand fashon. Compared to the tradtonal model that adopt statc resource provsonng accordng to peak demand, dynamc resource allocaton allows servce provders to acqure resources based on the current demand, whch can sgnfcately lower the operatng cost. Self-organzng: Snce resources can be allocated on demand, servce provders are responsble to manage ther resource consumpton accordng to ther own needs. Furthermore, the automated resource management feature yelds hgh aglty that enables servce provders to respond quckly to rapd varatons of servce demand such as the flash crowd effect. Reducng busness rsks and mantenance expenses: By outsourcng the servce nfrastructure to the clouds, a servce provder shfts ts busness rsks,.e. hardware falures, to nfrastructure provders, who often have better expertse and are better equpped for managng these rsks. In addton, a servce provder can cut down the hardware mantenance and the staff tranng costs. 39

40 Utlty-based prcng: Cloud computng employs a pay-per-use prcng model. The exact prcng scheme may vary from servce to servce. For example, a SaaS provder may lease a VM from an IaaS provder on a perhour bass. On the contrary, a SaaS provder that offers on demand customer relatonshp management (CRM) may charge ts customers based on the number of clents t serves. Utlty-based prcng lowers servce operatng cost as t charges customers on a per-use bass. However, t also ntroduces complextes n controllng the operatng cost. 1.5 Research Challenges Although cloud computng s very promsng and well-establshed n ndustry, there are many challengng ssues for the research communty whch have not been fully addressed yet. Furthermore, new challenges contnuously emerge from many ndustral applcatons. Ths secton ncludes the most nterestng research topcs of cloud computng feld Automated Resource Provsonng One of the key features of cloud computng s the capablty of allottng and releasng resources on demand. The objectve of a servce provder s to allocate resources from the cloud to satsfy ts servce level objectves (SLOs), whle mnmzng ts operatonal cost. In partcular, t s not easy to determne onlne how to map SLOs such as Qualty of Servce (QoS) requrements to low-level resource requrement such as CPU and memory requrements and concurrently acheve hgh aglty and response to rapd demand fluctuatons such as n flash crowd effect. Automated servce provsonng s not a new problem. Dynamc resource provsonng for Internet applcatons has been studed extensvely n the past [18], [19]. These approaches typcally nvolve, 1. Constructng an applcaton performance model that predcts the number of applcaton nstances requred to handle demand at each partcular level, n order to satsfy QoS requrements. 2. Perodcally predctng future demand and determnng resource requrements usng the performance model. 3. Automatcally allocatng resources usng the predcted resource requrements. Applcaton performance model adopts varous technques, such as Queung theory [18], Control theory [34] and Statstcal Machne Learnng [35]. Addtonally, there s a dstncton between proactve and reactve resource control. The proactve approach uses predcted demand to perodcally allocate resources before they are needed. On the other hand, the reactve approach addresses mmedate demand fluctuatons before perodc demand predcton s avalable. Both approaches 40

41 are mportant and necessary for effectve resource control n dynamc operatng envronments Vrtual Machne Mgraton Vrtualzaton can provde mportant benefts n cloud computng by enablng vrtual machne mgraton to balance load across the data center. In addton, VM mgraton enables robust and hghly responsve provsonng n data centers. Vrtual machne mgraton has evolved from process mgraton technques [20]. More recently, Xen and VMWare have mplemented lve mgraton of VMs that nvolves extremely short downtmes rangng from tens of mllseconds to a second. It s [21] ponted out that mgratng an entre OS and all of ts applcatons as one unt allows to avod many of the dffcultes faced by process level mgraton approaches, and analyzed the benefts of lve mgraton of VMs. The major benefts of VM mgraton are to avod hotspots. However, ths s not straghtforward. Currently, detectng workload hotspots and ntatng a mgraton lacks the ablty to respond to sudden workload changes. Moreover, the n memory state should be transferred consstently and effcently, wth ntegrated consderaton of resources for applcatons and physcal servers Servce Consoldaton Servce consoldaton s an effectve technque to maxmze resource utlzaton whle mnmzng energy consumpton n a cloud computng envronment. Lve VM mgraton technology s often used to consoldate VMs hosted by multple underutlzed servers onto a sngle server, so that the remanng servers can be set to an energy-savng state. The problem of optmally consoldatng servers n a data center s often formulated as a varant of the vector bn-packng problem [22], whch s an NPhard optmzaton problem. Varous heurstcs have been proposed for ths problem [23], [24]. Addtonally, dependences among VMs, such as communcaton requrements, have also been consdered recently [25]. However, servce consoldaton actvtes should not deterorate applcaton performance. It s known that the resource usage (also known as the footprnt [26]) of ndvdual VMs may vary over tme [27]. For server resources that are shared among VMs, such as CPU, bandwdth, memory cache and dsk I/O, maxmally consoldatng a server may result n resource congeston when a VM changes ts footprnt on the server [28]. Hence, t s sometmes mportant to observe the fluctuatons of VM footprnts and use ths nformaton for effectve servce consoldaton. Fnally, the system must quckly react to resource congestons when they occur [27]. 41

42 1.5.4 Energy Management Improvng energy effcency s another major ssue n cloud computng. It has been estmated that the cost of powerng and coolng accounts for 53% of the total operatonal expendture of data centers [29]. In 2006, data centers n the US consumed more than 1.5% of the total energy generated n that year, and the percentage s projected to grow 18% annually [23]. Hence nfrastructure provders are under enormous pressure to reduce energy consumpton. The goal s not only to reduce energy cost n data centers, but also to meet government regulatons and envronmental standards. Desgnng energy-effcent data centers has recently receved consderable attenton (Green Computng). Ths problem can be approached from several drectons. For example, energy effcent hardware archtecture that enables slowng down CPU speeds and turnng off partal hardware components [30] has become common place. Energy-aware job schedulng [31] and servce consoldaton [24] are two other ways to reduce power consumpton by turnng off unused machnes. Recent research has also begun to study energy-effcent network protocols and nfrastructures [32]. A key challenge n all the above methods s to acheve a good trade-off between energy savngs and applcaton performance. In ths respect, few researchers have recently started to nvestgate coordnated solutons for performance and power management n a dynamc cloud envronment [33] Traffc Management and Analyss Analyss of data traffc s mportant for today s data centers. For example, many web applcatons rely on analyss of traffc data to optmze customer experences. Network operators also need to know how traffc flows through the network n order to take many of the management and plannng decsons. There are several challenges for exstng traffc measurement and analyss methods n Internet Servce Provders (ISPs) networks and enterprse to extend to data centers. Frstly, the densty of lnks s much hgher than that n ISPs or enterprse networks, whch deterorates the scenaro for exstng methods. Secondly, most exstng methods can compute traffc matrces between a few hundred end hosts, but even a modular data center can have several thousand servers. Fnally, exstng methods usually assume some flow patterns that are reasonable n Internet and enterprses networks, but the applcatons deployed on data centers, such as MapReduce jobs, sgnfcantly change the traffc pattern. Further, there s tghter couplng n applcaton s use of network, computng, and storage resources, than what s seen n other settngs. Currently, there s not much work on measurement and analyss of data center traffc. Greenberg et al. [12] report data center traffc characterstcs on flow szes and concurrent flows, and use these to gude network nfrastructure desgn. Benson et al. [36] perform a complementary study of traffc at the edges of a data center by examnng SNMP traces from routers. 42

43 1.5.6 Data Securty Data securty s another mportant research topc n cloud computng. Snce servce provders typcally do not have access to the physcal securty system of data centers, they must rely on the nfrastructure provder to acheve full data securty. Even for a vrtual prvate cloud, the servce provder can only specfy the securty settng remotely, wthout knowng whether t s fully mplemented. Thus the nfrastructure provder must acheve the followng objectves: 1. Confdentalty for secure data access and transfer. 2. Audtablty for attestng whether securty settng of applcatons has been tampered or not. Confdentalty s usually acheved usng cryptographc protocols, whereas audtablty can be acheved usng remote attestaton technques. Remote attestaton typcally requres a trusted platform module (TPM) to generate non-forgeable system summary (.e. system state encrypted usng TPM s prvate key) as the proof of system securty. However, n a vrtualzed envronment lke the clouds, VMs can dynamcally mgrate from one locaton to another; thus drectly usng remote attestaton s not suffcent. In ths case, t s crtcal to buld trust mechansms at every archtectural layer of the cloud. Frstly, the hardware layer must be trusted usng hardware TPM. Secondly, the vrtualzaton platform must be trusted usng secure vrtual machne montors [37]. VM mgraton should only be allowed f both source and destnaton servers are trusted. Recent work has been devoted to desgnng effcent protocols for trust establshment and management [38], [37] Software Frameworks Cloud computng provdes a platform for hostng large-scale data-ntensve applcatons. Typcally, these applcatons leverage MapReduce frameworks such as Hadoop for scalable and fault-tolerant data processng. Recent work has shown that the performance and resource consumpton of a MapReduce job s hghly dependent on the type of the applcaton [39], [40], [41]. For nstance, Hadoop tasks such as sort are I/O ntensve, whereas grep requres sgnfcant CPU resources. Furthermore, the VM allocated to each Hadoop node may have heterogeneous characterstcs. For example, the bandwdth avalable to a VM s dependent on other VMs collocated on the same server. Hence, t s possble to optmze the performance and cost of a MapReduce applcaton by carefully selectng ts confguraton parameter values [39] and desgnng more effcent schedulng algorthms [40], [41]. By mtgatng the bottleneck resources, executon tme of applcatons can be sgnfcantly mproved. The key challenges nclude performance modelng of Hadoop jobs (ether onlne or offlne), and adaptve schedulng n dynamc condtons. Another related approach argues for makng MapReduce frameworks energy-aware [41]. The essental dea of ths approach s to turn Hadoop node nto sleep mode when 43

44 t has fnshed ts job whle watng for new assgnments. To do so, both Hadoop and HDFS must be made energy-aware. Furthermore, there s often a trade-off between performance and energy-awareness. Dependng on the objectve, fndng a desrable trade-off pont s stll an unexplored research topc Storage technologes and Data Management Software frameworks such as MapReduce and ts varous mplementatons such as Hadoop and Dryad are desgned for dstrbuted processng of data-ntensve tasks. As mentoned prevously, these frameworks typcally operate on Internet-scale fle systems such as GFS and HDFS. These fle systems are dfferent from tradtonal dstrbuted fle systems n ther storage structure, access pattern and applcaton programmng nterface. In partcular, they do not mplement the standard POSIX nterface, and therefore ntroduce compatblty ssues wth legacy fle systems and applcatons. Several research efforts have studed ths problem [42], [43]. For nstance, the work n [42] proposed a method for supportng the MapReduce framework usng cluster fle systems such as IBM s GPFS. Patl et al. [43] proposed new API prmtves for scalable and concurrent data access Novel Archtectures Currently, most of the commercal clouds are mplemented n large data centers and operated n a centralzed fashon. Although ths desgn acheves economy-of-scale and hgh manageablty, t also comes wth ts lmtatons such hgh energy expense and hgh ntal nvestment for constructng data centers. Recent work [44], [45] suggests that small sze data centers can be more advantageous than bg data centers n many cases: a small data center does not consume so much power, hence t does not requre a powerful and yet expensve coolng system; small data centers are cheaper to buld and better geographcally dstrbuted than large data centers. Geodversty s often desrable for response tme-crtcal servces such as content delvery and nteractve gamng. For example, Valancus et al. [45] studed the feasblty of hostng vdeo-streamng servces usng applcaton gateways (a.k.a. nano-data centers). Another related research trend s on usng voluntary resources (.e. resources donated by end-users) for hostng cloud applcatons [46]. Clouds bult usng voluntary resources, or a mxture of voluntary and dedcated resources are much cheaper to operate and more sutable for non-proft applcatons such as scentfc computng. However, ths archtecture also mposes challenges such managng heterogeneous resources and frequent churn events. Also, devsng ncentve schemes for such archtectures s an open research problem. 44

45 45

46 CHAPTER 2 Montorng of Dstrbuted Cloud Servces. Today s networks of large organzatons, such as unverstes or enterprses, rely on many dstrbuted and non-standardzed applcatons. These applcatons are usually mult-ter, snce they consst of many dfferent components, and are manly deployed on data centers of prvate or publc clouds. Avalablty and performance of these servces are mportant for revenue-generatng busness processes, so enterprses enter servce level agreement (SLA) wth Internet servce provders (ISPs). Due to the fnancal mpact of SLAs, there s great research nterest n servce management and ntegrated management tools that automatcally montor the performance of mult-ter applcatons and that can also autonomously handle arsng problems. Montorng and management of such systems have become a crtcal and complex ssue. The purpose of a management system s to montor vtal attrbutes of network n an automatc manner and to take acton whenever needed. After the set-up of a network, network components nteract n an unpredctable way, obstructng admnstrators from havng a clear vew of the network operaton. Even a smple task, such as fetchng a web page from a web server, nvolves many network assets (DNS server, web server, databases) that have an operatonal relatonshp. Network admnstrators use manly tools lke MRTG [47] that llustrate characterstcs and statstcs of traffc volume based on ports. The major drawback of these montorng tools s that they do not provde any further nformaton about how network assets collaborate n order to complete a specfc task. Unvelng these dependences can help admnstrators to have an ntegral vew of the structure of network applcatons. A functonal and structural model of a servce or applcaton s a powerful tool for the mantenance, expanson and performance analyss of the servce. It helps admnstrators to detect whch component of a servce s responsble for a possble falure and whch other busness processes wll be affected. These two problems are known as root cause analyss and busness mpact analyss, respectvely. The frst step n buldng an operatonal model of a servce s to completely understand the nteracton of the network components that comprse an ntegrated applcaton. Because of the complexty and the heterogenety of enterprse networks, there are many types of relatonshps and dependences between the parts of mult-ter applcatons. In [48], Keller et al. present a good defnton and classfcaton of the dfferent types of dependences among servce components. Ths classfcaton s based on many characterstcs, such as localty, doman, component type and actvty as well as dependency detecton method and strength. There are many studes on the area of montorng ssues of dstrbuted and vrtualzed data centers whch are mplemented on cloud computng nfrastructure. The followng papers are the most representatve and closer to our pont of vew. In [49], a passve technque s suggested usng flow attrbutes and a fuzzy nference engne to nfer dynamc dependences from mult-ter applcatons. The dsadvantage of ths work s 46

47 that the nference engne was based on human experence and t was not flexble. Furthermore, t focused only on mult-ter applcatons wthout examnng other knds of dependences. Agarwal et al. [50] demonstrated a data-mnng method to unvel dynamc dependences n e-busness systems. Ths approach focuses only on ntrasystem dependences and does not seem to scale for large number of customers. In [51], authors used state machnes to descrbe the structure of applcatons. They llustrated the trade-off between smplcty and analyss depth, but ther study s restrcted to partcular applcatons (.e. HTTP or FTP). Aello et al. [52] modeled network hosts behavor and created communtes of nterest. Although they demonstrate nterestng parameters for relatonshp dscovery, ther man scope s to dscover clusters of nteractng hosts wthout examnng whether they are nterested n a partcular applcaton. Sherlock [53] s a system that uses a centralzed nference engne and dstrbuted agents to form a servce-level dependency graph for fault localzaton. Although Sherlock nfers dependences n depth, t cannot extract many relatonshps smultaneously but extracts dependences from one server or one servce at a tme. Kandula et al. presented expose [54], a novel technque to dscover communcaton rules between network components. They extract communcaton rules over a network usng an nformaton theoretc approach. Although expose dscovers the majorty of the underlyng dependences, t does not extract the patterns whch govern these underlyng communcaton relatonshps or ther strength. Relatvely to the dsadvantages of the studes n the lterature, the thess focuses on the creaton of a tool for analyss of network traffc, whch mnes relatonshps between network and servce components and measures ther strength wthout havng any pror knowledge of exstng applcatons. The man contrbutons are the followng: The creaton of a servce graph overlay, whch depcts the operatonal dependences between the components of the dstrbuted web servces n a data center. Relable dentfcaton of the hdden relatonshps wthout any a pror knowledge about them. Quantfcaton of the relatonshp s strength. Easy and offlne tranng procedure of the nference mechansm. Fast and onlne process of the traffc data. Small number of erroneously produced data. Ths chapter ntroduces a new nference mechansm that extracts relatonshps between network components and measures ther strength wthout havng any pror knowledge of exstng applcatons [59]. We dscover expected relatonshps, such as DNS queres before connecton to a server or database, and hdden relatonshps between servers of a dstrbuted networked fle system (ports ) or between web (port 80) and authentcaton servers (.e. Kerberos port 88). After detectng these dependences we buld a servce overlay that llustrates the structure of servces and 47

48 the structure of communcaton among network hosts and servers or databases that are afflated wth a specfc servce. Ths overlay can beneft several admnstratve tasks, such as detectng msconfguratons or DoS attacks, montorng servce performance and desgn or expanson network nfrastructure. Our work s based on NetFlow traces collected from an enterprse network and a campus network of a unversty. We do not collect data from log fles of man servers of these networks because ths work s tedous and would lose a lot of relatonshps, whch do not nclude these computers. From these traces tme nformaton and typcal flow attrbutes are used, lke IP addresses, ports and protocols. Our work s twofolded. Frst we apply a smple algorthm that reveals all possble dependences. Then we develop a flexble fuzzy nference mechansm (FIM) that determnes whether the canddate relatonshp s true or not and measures ts strength. The second part of our study s the desgn and tranng of FIM. FIM conssts of a small set of fuzzy rules (.e. 30 rules) produced by a hybrd genetc algorthm, whch s a tranng process wth hgh classfcaton rate. Our approach dscovers a wde varety of network relatonshps and measures ther strength. Snce our tranng process s an offlne procedure and produces a small set of rules n order to extract the hdden dependences, an onlne mplementaton seems feasble. The remander of the chapter s structured as follows. Intally, all the essentals prncples of fuzzy logc are descrbed. Sectons 2.2 and 2.3 contan an analytcal descrpton of our dependency dscovery algorthm and the tranng procedure, respectvely. Secton 2.4 demonstrates the mplementaton and evaluaton of the approach. Fnally Secton 2.5 summarzes the conclusons of the proposed study. 2.1 Fuzzy Logc A fuzzy logc system can be defned as the nonlnear mappng of an nput data set to a scalar output data [55]. It conssts of four man parts: fuzzfcaton, rules, nference engne, and defuzzfcaton. These components and the general archtecture of a fuzzy logc system are shown n Fgure 2.1. Inputs Fuzzfcaton Rules Set Inference Mechansm Defuzzfcaton Outputs Fgure 2.1 Structure of a Fuzzy Logc System 48

49 2.1.1 Lngustc Varables Lngustc varables are the nput or output varables of the system whose values are words or sentences from a natural language, nstead of numercal values. A lngustc varable s generally decomposed nto a set of lngustc terms. For example, let temperature (t) be the lngustc varable whch represents the temperature of a room. To qualfy the temperature, terms such as hot and cold are used n real lfe. These are the lngustc values of the temperature. Then, T(t) = { cold, cool, nomnal, warm, hot } can be the set of decompostons for the lngustc varable temperature. Each member of ths decomposton set s called a lngustc term and can cover a porton of the overall values of the temperature Membershp Functons Membershp functons are used n the fuzzfcaton and defuzzfcaton steps of a fuzzy logc system, to map the non-fuzzy nput values to fuzzy lngustc terms and vce versa. A membershp functon s used to quantfy a lngustc term. There are dfferent forms of membershp functons such as trangular, trapezodal, pecewse lnear, Gaussan, or sngleton (Fgure 2.2). The most common types of membershp functons are trangular, trapezodal, and Gaussan shapes. The type of the membershp functon can be context dependent and t s generally chosen arbtrarly accordng to the user experence [56]. Note that, an mportant characterstc of fuzzy logc s that a numercal value does not have to be fuzzfed usng only one membershp functon. In other words, a value can belong to multple sets at the same tme, as t s shown n Fgure 2.3. Fgure 2.2 Memebershp Functons: (a) sngleton, (b) trangular, (c) trapezodal, (d) Gaussan shape 49

50 Fgure 2.3 Membershp Functons of a Lngustc Varable Set of Fuzzy Rules and Inference Mechansm The mappng of the nputs to the outputs for a fuzzy system s n part characterzed by a set of condton - acton rules n If-Then form, If premse Then consequent Usually, the nputs of the fuzzy system are assocated wth the premse, and the outputs are assocated wth the consequent. Generally, these If-Then rules can be represented n mult-nput mult-output (MIMO). Although they can be mult-nput sngle-output (MISO) f there s only one out assocated wth the consequent. The set of these rules contans the knowledge of the expert about the how the system works. The evaluatons of the fuzzy rules and the combnaton of the results of the ndvdual rules are performed usng fuzzy set operatons. The operatons on fuzzy sets are dfferent than the operatons on non-fuzzy sets. Let μ Α and μ Β are the membershp functons for fuzzy sets A and B. Table 2.1 contans possble fuzzy operatons for OR and AND operators on these sets, comparatvely. The mostly used operatons for OR and AND operators are max and mn, respectvely. For complement (NOT) operaton, the followng equaton s used for fuzzy sets. 1 _ A A OR (unon) AND(ntercepton) MAX max{ ( x), ( x)} MIN mn{ ( x), ( x)} A B ASUM ( x) ( x) ( x) ( x) PROD ( x) ( x) A B A B BSUM mn{ 1, ( x) ( x)} BDIF max{ 0, ( x) ( x) 1} A B A A A B B B Table 2.1 Fuzzy Set Operaton 50

51 After evaluatng the result of each rule, these results should be combned to obtan the fnal result. Ths process s called nference. The results of ndvdual rules can be combned n dfferent ways. The maxmum algorthm s generally used for accumulaton max{ ( x), ( x)} Defuzzfcaton A B After the nference step, the overall result s a fuzzy value. Ths result should be defuzzfed to obtan a fnal numercal output. Ths s the purpose of the defuzzfer component of a fuzzy logc system. Defuzzfcaton s performed accordng to the membershp functon of the output varable. For nstance, assume that we have the result n Fgure 2.4 at the end of the nference. In ths fgure, the shaded areas all belong to the fuzzy result. The purpose s to obtan a numercal value, represented wth a dot n the fgure, from ths fuzzy result. Fgure 2.4 Defuzzfcaton of a Fuzzy Logc System There are many dfferent algorthms for defuzzfcaton. The mostly used s Center of Gravty whch determnes the fnal numercal value of output accordng to the followng equaton, U max mn max u( u) du mn ( u) du where U s the numercal result of defuzzfcaton, u s the output varable and μ(u) s the membershp functon after accumulaton. 2.2 Mnng Network Relatonshps In ths secton, the problem of detectng relatonshps between IT elements by examnng attrbutes of flow records s consdered. Collecton of the necessary nformaton s performed by Aurora [57], a flow-based network proflng system 51

52 based on NetFlow/IPFIX [58]. Flow-based traffc nformaton, such as source or destnaton IP address and ports, tmestamps, protocol and packet or octet volumes, s collected. More specfcally, by examnng the tmestamps of flows we would lke to deduce whether the coexstence of partcular flow pars n a tme wndow can be nterpreted as a relatonshp between two dfferent components that execute an ndvdual task or a part of an applcaton. By mnng all these relatonshps between network components of a servce, we can obtan a clear overvew n terms of whch elements of network nfrastructure are responsble for the provsonng of a partcular servce. Fgure 2.5 llustrates the structure of our approach. In the followng paragraphs, we explan the dfferent components of our system. AppLst HGA Flows Extracted Event Pars FIM Relatonshps Pre-processng Data Fgure 2.5 Relatonshp Dscovery System Before we formalze flow dependency, t s essental to ntroduce some domnant parameters of relatonshp dscovery. In an enterprse or a unversty campus network the traffc volume s very hgh, so our algorthm has to examne a large amount of flows, whch are tme and resource consumng. Furthermore, a large porton of these flows does not contan any meanngful nformaton for mnng dependences. For the aforementoned reasons, we make a dstncton of hosts n a network. By examng the destnaton (IP address and port) of a flow and the TCP flags we can understand whether a network asset s an actve part of an applcaton, because t receves many more requests on a specfc port (or ports n the case of co-locaton) than the other ports. We menton that we do not rely on well-known ports, because we ntend to nclude also non-standardzed applcatons and p2p relatonshps. We create a lst of these tuples (IP address and port) and we focus on flows whch nclude these tuples ether as source or as destnaton, n order to extract canddate relatonshps. We call ths lst applcaton lst (AppLst) Flows and Events We defne a flow f as a 3-tuple of the followng basc traffc attrbutes: f ( srcip, dstip, dstport ) F (2.1) 52

53 where srcip and dstip are the IP address of source and destnaton host, respectvely, and dstport s the TCP/UDP servce port of the destnaton host. The set of all flows s denoted by F. We do not nclude the servce port of the source host because t s usually random. Addtonally we defne the destnaton of a flow f as the tuple: dst( f ) ( dstip, dstport ) (2.2) An event - typcally a NetFlow/IPFIX record - s defned as, e ( f, t, t, octs, pkts) s e (2.3) where f s the flow as defned and t s and t e are the start and end tmestamps of a flow event, respectvely, and octs and pkts are the number of bytes and packets of a flow, respectvely. The set of all events s denoted as E. Fnally we denote the set of all events of a gven flow f as, Flow and Event Pars e E f (2.4) E( f ) f Nowadays, most network applcatons are dstrbuted and depend on many dfferent assets. For ths reason an event cannot reveal any assocaton between IT nfrastructure. The man dea of our approach s to mne flow pars that can show any functonal relatonshp between hosts n the applcaton lst. We defne three generc flow pars that cover every possble relatonshp. In the frst case, we examne how network clents nteract wth the hosts of AppLst. Consderng any two flows n F, we defne the clent server flow par (cs flow par) f the followng condtons are satsfed: e The frst and the second flow have an dentcal source IP address and do not belong to the AppLst. The destnaton of the frst flow s dfferent from the destnaton of the second flow and both destnatons belong to AppLst. An example of such a flow par s a par between a DNS server and a web server. When a user sends an HTTP request to a web server, a flow between the clent and the DNS server always precedes the flow between the clent and the web server. The next par between any two flows n F examnes whether a request from a clent to a host of the applcaton lst trggers a flow between the host of the applcaton lst and another host of AppLst. For nstance, ths flow par extracts relatonshps between a web server and a back-end database when a clent sends a request to a web server and consequently t communcates wth a database that contans the contents all the necessary nformaton. Formally, we defne a server server flow par (ss flow par) f the followng prerequstes are satsfed: 53

54 The source IP address of the frst flow s not ncluded n the AppLst and the destnaton of the frst flow belongs to the AppLst. The destnaton of the second flow belongs to the AppLst. The destnaton IP address of the frst flow s dentcal to the source IP address of the second flow. Fnally we defne a flow par that detects whether a request from a host to another host of the applcaton lst trggers a second flow from the second host to a thrd one of the applcaton lst. Relatonshps of mult-ter servces and updates between servers can be revealed by ths flow par. More specfcally, consderng any par of flows n F we defne a mult-server flow par (ms flow par) f the followng condtons are satsfed: Source IP addresses of both flows are ncluded n the AppLst and the destnaton of both flows belongs to the AppLst. The destnaton IP address of the frst flow s dentcal to the source IP address of the second flow. Assumng any two dfferent flows f and f j n F, we encode the prevous consderatons n the followng expressons of three flow pars functons: fp cs ( f, f j 1 ) 0 f scrip scrip and (dst(f ), dst(f )) AppLst j otherwse j (2.5) fp ss ( f, f j 1 ) 0 f dstip scrip and (dst(f ), dst(f )) AppLst j j otherwse and srcip AppLst (2.6) fp ms ( f, f j 1 ) 0 f dstip scrip and (srcip, dst(f ), dst(f )) AppLst j otherwse j (2.7) Fgure 2.6 shows a typcal example of defned flow pars. After the defnton of the three flow par functons, we defne three dfferent sets of event pars for any two dfferent flows f and f j n F, respectvely. Assumng any two dfferent flows f and f j n F (f f j ) and two events e k, e l n E, whch are events of f and f j, respectvely, three dfferent sets of event pars, one for each flow par, for these flows f and f j are defned as, cs j ( e, e ) : 0 t t t and fp ( f, f ) 1 (2.8) P ( f, f ) max k l sl sk ( e, e ) : 0 t t t and fp ( f, f ) 1 (2.9) P ( f, f ) max ss j k l sl sk cs ss j j 54

55 Clents DNSServ (a) WebServ Clents WebServ DB (b) WebServ AppServ DB (c) Fgure 2.6 Flow Pars: (a) cs flow par, (b) ss flow par, (c) ms flow par P ms ( e, e ) : 0 t t t and fp ( f, f ) 1 (2.10) ( f, f ) max j k l sl sk As shown n the above defnton, two events are consdered an event par f they satsfy one of the flow par functons and the dfference of ther start tmestamps s less than t max. The man dea of usng tme dfference as a crteron of dependency s that flow events close n tme are lkely really dependent. The value of the tme wndow t max s crtcal for the success of the algorthm. If t s too small, then t s possble not to mne real dependences. On the other sde, f t s too large, then the algorthm detects many false dependences between hosts that do not represent meanngful relatonshps. Furthermore, the tme wndow s strongly affected by the structure of an applcaton. Some applcatons,.e. f t s necessary to type a username and a password, may need to spend more tme than other smpler servces,.e. just fetch the content of a web server. Hence t s obvous that choosng a value of tme wndow whch would suffce for all varous applcatons s very dffcult. Repeatng our algorthm wth varous tme wndow values, we conclude that the number of event pars ncreases rapdly, but the number of dfferent sets of event pars does not ncrease smlarly. Ths result shows that ncreasng the tme wndow too much (more than 1 s) does not mprove the performance our algorthm Confdence Varables The tme dfference of the start tmestamps of two events s not a safe metrc of dependency by tself. In heavly loaded networks there s a hgh probablty of dscoverng many false event pars. There are three man reasons for false assocatons. Frst, some event pars are created clearly by chance. Another source of erroneous data s whether a host of AppLst s a common part for two dfferent applcatons. Fnally there are some cases of very frequent flows (.e. DNS queres) ms j 55

56 that can create multple event pars wth a flow that occurs only once. Even f there s one real event par, many false pars are dentfed and the strength of the relatonshp s dstorted. Thus only the frequency of an event par cannot reveal the correctness and strength of a relatonshp. To reduce the erroneously recognzed dependences, we defne four confdence varables based on the followng concepts: Confdence Varable 1: c ( f 1 1 ) E( f 1p E( f 1 ) ) (2.11) where E( f 1p ) s the number of events of f 1 that partcpate n an event par of set P f 1, f ) and the denomnator s the number of elements of set E( f 1 ). ( 2 Confdence Varable 2: c ( f 2 2 ) E( f E( f 2 p 2 ) ) (2.12) where E( f 2p ) s the number of events of f 2 that partcpate n an event par of set P f 1, f ) and the denomnator s the number of elements of set E( f 2 ).These two ( 2 confdence varables show whether events of flows f 1 and f 2 are related wth a specfc task and measure the strength of ths relatonshp. Confdence Varable 3: TNP( f1, f 2 ) SNP( f1, f 2 ) c3( f1, f 2 ) TNP( f, f ) 1 2 (2.13) where TNP f 1, f ) s the total number of dentcal detected event pars that flow f 1 ( 2 creates n a tme wndow t max and SNP f 1, f ) s the sequence number of the event ( 2 par (.e. for the frst detected event par SNP f, f ) 0, for the second SNP f, f ) 1). ( Confdence Varable 4: ( SNP( f1, f c4 ( f1, f 2 ) TNP( f, f ) ) (2.14) where TNP f 1, f ) s the total number of dentcal detected event pars that flow f 2 ( 2 creates n a tme wndow t max and SNP f 1, f ) s the sequence number of the event ( 2 par (.e. for the frst detected event par SNP f, f ) 0, for the second SNP ( f, f2) ( ). We use c 3 and c 4 n order to reduce the multple dentcal event

57 pars that are generated from very frequent flows. Usually whether or not many dentcal pars exst, the real one occurs earler n the tme wndow, whch means c3 1and c 4 0. The values of confdence varables are lmted between zero and one Relatonshp Dscovery In order to mne relatonshps n a network trace, we examne whether an event of a flow creates any event par, as defned prevously, n a tme wndow of 1 s. The values of all confdence varables are calculated and then a set of fuzzy rules s used n order to evaluate whether a set of event pars represents a real dependency and furthermore measure the strength of ths assocaton. In order to separate real from meanngless relatonshps we used four thresholds α and β 1, β 2, β 3. If the dependency output of the fuzzy system for P( f, f j ) s less than threshold α then ths set does not represent any real relatonshp. Thresholds α,β 1, β 2, β 3 classfy relatonshps nto four classes low, medum, hgh and very hgh accordng to ther dependency value. Default values for thresholds α and β 1, β 2, β 3 are 0.125, 0.375, and 0.875, respectvely. Algorthm 2.1 llustrates how all sets of event pars P cs are extracted. Algorthms for sets of event pars P ss and P ms are smlar and are omtted. Algorthm 2.1: Extract CS event pars 1: Fll n AppLst wth dst <dstip,dstport> (dst AppLst) {Create outgong matrx OM for each clent} 2: for all clents (clip Applst) do 3: Create a matrx of events clip dst 4: Sort matrx by start tmestamps t s 5: end for {Fnd event pars n OM} 6: for all OM of clents (clip Applst) do 7: for all event e n OM do 8: tme dfference between e and next events e j dt = t sj t s 9: whle dt 1s do 10: Update P(f, f j ) 11: end whle 12: end for 13: end for 14: Compute all confdence varables {Examne the valdty of sets of P(f, f j )and measure the strength of relatonshps} 15: for all sets P(f, f j ) do 16: for all event pars (e k, e l ) P(f, f j ) do 57

58 17: Use the set of Fuzzy Rules to calculate dependency class and ts strength dep(e k, e l ). 18: end for 19: dep(p(f, f j )) =Σ dep(e k, e l ) 20: f dep(p(f, f j )) α then 21: None real relatonshp between (dst, dst j ). 22: else f α < dep(p(f, f j )) 1 then 23: Classfy relatonshp to the approprate class accordng to the strength of t dep(p(f, f j )) 24: end f 25: end for To fully understand the underlyng relatonshps of a network and mnmze the generated event pars, we aggregate pars wth the same meanng. The cs flow par represents the nteracton between clents and IT nfrastructure for the completon of a specfc task. We can aggregate event pars that correspond to the same assocaton regardless of the clent, whch creates ths event par. Smlarly, we can aggregate event pars of an ss flow par, because t s unmportant whch clent s responsble for ths event par. We do not aggregate event pars of an ms flow par, because usually ths does not happen, and whenever t occurs the host that causes ths par may have a specal meanng. 2.3 Fuzzy Inference Mechansm It s obvous from the prevous secton that performance of our approach depends on the fuzzy nference mechansm (FIM), whch s based on a small set of fuzzy rules (30 rules). In our prevous work about mult-ter servces [49], we used an nflexble FIM generated manually by human experts, whch s not dynamc and has lmted accuracy. On the contrary, a flexble and accurate FIM s proposed n ths study whch s automatcally produced by a tranng process. FIM s a set of fuzzy rules whch determne the exstence and strength of a relatonshp. Because relatonshps and ther strength vary a lot n a network wth many dfferent applcatons, a fuzzy system s an approprate descrptve tool for the ndstnct boundares between strong, loose and no dependency. In our case we buld a fuzzy system wth fve nputs: tmestamp dfference of an event par and the four confdence varables. The output of the fuzzy system s the relatonshp strength, dep [0, 1]. If dep 0 there s no dependency; on the other hand dep 1 ndcates tght dependency between two network elements. Another mportant parameter of FIM s that each nput s descrbed by four lngustc values, such as low, hgh and medum, and a specal ( don t care ) lngustc value that helps us to aggregate smlar rules. Smlarly the output of FIM s descrbed by four lngustc values (dependency classes). The rules set of an FIM s a set of fuzzy f then rules lke the followng: 58

59 f dt s LOW and c 1 s HIGH and c 2 s HIGH and c 3 s HIGH and c 4 s LOW then dep s HIGH f dt s HIGH and c 1 s LOW and c 2 s LOW and c 3 s LOW and c 4 s HIGH then dep s LOW We produce the fuzzy rules of the flexble FIM usng the tranng procedure of a hybrd genetc algorthm (HGA). Genetc algorthms (GA) are already used as classfcaton tool and they can be appled to the dependency detecton problem. FIM produced by GA has many advantages.intally, t has a hgh classfcaton rate, reducng false generated relatonshps. It s an automatc process that can be done offlne. In a normal network stuaton, only the strength of a relatonshp changes durng certan tme perod. Relatonshps do not appear or dsappear suddenly, so the tranng process has to be done only f changes occur n IT nfrastructure. Also, GA s sutable for handlng hgh-dmensonal datasets, where the number of all possble fuzzy rules ncreases dramatcally. Fnally, because the output of a genetc-based tranng process s a small fuzzy rule set, an onlne mplementaton of our approach s possble. There are many alternants of GAs that produce a set of rules to classfy patterns from dfferent classes. All versons of GAs are based on operatons of selecton, crossover and mutaton. The populaton of GA conssts of a rules set. One approach, the Pttsburgh approach, uses as populaton many dfferent rules sets wth specfc number of fuzzy rules and has drect optmzaton ablty. The Mchgan approach uses as ntal populaton a sngle set wth fxed number of rules and has a hgh research ablty to fnd good fuzzy rules. Our tranng procedure reles on a hybrd genetc algorthm (HGA) [60] that combnes the benefts of both above approaches. We modfy the presented algorthm to acheve a better classfcaton rate. We feed HGA wth patterns of real and false mned relatonshps. Table 2.2 llustrates the parameters of HGA tranng process. Rule Sets Replaced Sets Fuzzy Rules Replaced Rules Mutaton Probablty Selecton Probablty Crossover Probablty Mchgan Probablty Table 2.2 Parameters of HGA 59

60 The steps of HGA are descrbed n Algorthm 2.2 below, Algorthm 2.2: Hybrd genetc algorthm 1: Generate N pop rule sets wth N rules fuzzy rules from tranng patterns. 2: Calculate the ftness value of each rule set n the current populaton. {Iterate the steps of the Pttsburgh approach for 150 generatons.} 3: for generaton = 1:150 do 4: Generate N repls rule sets by genetc operatons of the Pttsburgh method. {Iterate twce the steps of the Mchgan approach} 5: for =1:2 do 6: Generate N replr rules by genetc operatons of the Mchgan method, wth a prespecfed probablty. 7: Calculate the class of each new generated rule. 8: Calculate the ftness value of each new generated rule. 9: end for 10: Calculate the class of each rule n new generated rule sets. 11: Calculate the ftness value of each rule set n the current populaton. 12: end for 13: Save the rule set wth the hghest ftness value. Ftness values of the Pttsburgh and Mchgan approaches are respectvely calculated as, FV Ptt ( set ) 11.5* SFP( set ) 2* SFN ( set ) 1.5* SFR( set ) (2.15) FV Mch ( set ) RCP( rule ) 1.5* RFP( rule ) 2* RFN ( rule ) (2.16) where SFP(set ) s the number of false postves (.e. pars that are not real but they are classfed as real by rule set set ), SFN(set ) s the number of false negatves (pars that are real but no dependency was unveled by set )and SFR(set ) s the number of pars that set cannot classfy nto a class. Smlarly, RCP(rule ) s the number of correctly classfed pars by rule rule, RFP(rule ) s the number of false postves by rule and RFN(rule ) s the number of false negatves by rule. After exhaustve runnng of HGA wth dfferent parameters, the domnant factors of HGA success are the followng: Tranng patterns can be less than 10% of all patterns. We use the remanng patterns for evaluaton rules are suffcent to succeed n classfcaton rate hgher than 90%; more rules ncrease the computaton tme wth neglgble mprovement. 60

61 The larger the number of rule sets n the HGA populaton, the more lkely t s to fnd better rule sets because HGA search space grows. 150 generatons are suffcent for HGA to converge to an optmal soluton. The number of replaced rules n the Mchgan approach should be 20% of all rules n the rule set. The number of replaced rule sets n the Pttsburgh approach should be 60% of the total number of rule sets n the populaton. If t s larger, HGA has a low success rate. If t s smaller, HGA converges quckly n a soluton and the populaton does not evolve to better rule sets. Usage of the specal lngustc value don t care mproves the success rate of HGA, because t aggregates rules nto one. 2.4 Evaluaton We tested our relatonshp dscovery algorthm usng network traces from two dfferent locatons. We collected data from the IBM Zurch Research Laboratory and the Department of Electrcal Engneerng and Computer Scence from the Unversty of Patras. Both traces are anonymzed, so t s not possble to know the ground truth, although we could examne whether our results are correct, by servce ports and from nformaton that we gather from admnstrators of these networks. The frst network s an enterprse network that hosts many dfferent applcatons, so ts nfrastructure s complex and we expect to dscover all knds of dependences. Conversely, the second network s a unversty network that hosts fewer applcatons (web hostng, mal, fle servers) than an enterprse network, so we expect to mne fewer mult-ter relatonshps and stronger clent-server (cs) flow pars. Another nterestng parameter s that a campus network has more users than an enterprse network, so the traffc load s hgher n the Patras trace than n the IBM trace. Both traces show that the parameter of tme wndow strongly affects the number of dscovered event pars and relatonshps. For the remander of the paper tme wndow s set to one second Smulatons Our approach s evaluated by two dfferent methods. Frst, we create and nject event pars of artfcal relatonshps n the real data traces, and then we apply our algorthm to check whether we can dentfy these artfcal dependences. Artfcal dependences emulate typcal patterns of network traffc, such as web browsng, downloadng fles from a dstrbuted fle system and mult-ter applcatons, and are generated by normal dstrbutons wth dfferent mean and standard devaton values nsde a tme wndow of 1 s. The beneft of these smulatons s that ground truth s already known, thus we can extract safe conclusons about relatonshps over a network from the evaluaton of our approach. We extract event pars of real and false relatonshps and we feed HGA wth them to generate a set of fuzzy rules (FIM). Then we generate dfferent artfcal event pars to evaluate these rules. Fgure 2.7 llustrates the classfcaton rate of the HGA tranng process. It s obvous that HGA succeeds n producng a set of fuzzy rules that classfes correctly the 61

62 majorty of the dfferent types of the njected artfcal relatonshps. After 150 generatons of HGA the classfcaton rate s around 92% for ms relatonshps, 94% for cs relatonshps and 97% for ss relatonshps. In order to evaluate the produced sets of fuzzy rules from the tranng procedure, we create a dfferent set of artfcal event pars. Fgure 2.8 depcts the percentage of correctly classfed patterns over 30 runs. As we can see, HGA and produced fuzzy rules are accurate, because they dentfy artfcal relatonshps and precsely measure ther strength (tght or loose assocatons). The classfcaton rate s smlarly hgh as durng the tranng procedure. The proposed relatonshp dscovery algorthm succeeds n revealng 92% of the mult-server dependences, 94.5% of the clent server relatonshps and more than 95% of the mult server relatonshps. Fgure 2.7 Tranng: Average Classfcaton Rate of Smulatons 62

63 Fgure 2.8 Evaluaton: Average Classfcaton Rate of Smulatons Real Data Furthermore our method s tested usng data from real traces wthout havng any pror knowledge about relatonshps among assets of these networks. Snce nformaton s anonymzed, t s mpossble to check the correctness of our results. However t s obvous that any of the dscovered relatonshps correspond to well-known realstc dependences can be classfed as a true relatonshp. We use tme wndow of one second and FIM determnes 690 dependences, as we can see n Table 2.3. Most of them are classfed as poor relatonshps wthout any specal meanng. There are 67 assocatons wth medum dependency value that manly represent true dependences among DNS servers and other servers. There are 11 strong dependences that correspond to tasks of specfc applcatons. Furthermore, Table 2.3 shows that event pars and assocatons ncrease as the tme wndow ncreases but relatonshps wth hgh dependency value are almost constant for all values of tme wndow. As we 63

64 mentoned, we expect to dscover more ms and ss relatonshps n the IBM network and smpler ones n the unversty campus network. In the followng paragraphs we record the most typcal examples of relatonshps that we detect n both network traces. Also we present characterstc dependences of each network. HGA s fed wth patterns to produce a set of 30 fuzzy rules and dfferent data are used for tranng (about 10%) and evaluaton. Snce we do not know whch dependences are true, pars of the most well-known dscovered relatonshps are categorzed as strong or loose and the rest as meanngless (poor relatonshps). Tme wndow (sec) Event pars (10 5 ) cs relatonshps ms relatonshps ss relatonshps total relatonshps Medum Score Hgh Score Table 2.3 Effect of tme wndow n real data The classfcaton rate of HGA durng the tranng procedure s hgh. As t s demonstrated n Fgure 2.9, the average classfcaton rate of server-server (ss) and mult-server relatonshps (ms) s 90%. The classfcaton rate of the clent-server (ss) s hgher (95%). It s worth mentonng that there are some fuzzy rules are present n every set over 30 runs of the tranng procedure. Fgure 2.10 llustrates the results for the thrty dfferent FIMs whch are produced from the tranng process. The average classfcaton rate of server-server (ss) and mult-server relatonshps (ms) s 90% and the classfcaton rate of the clent-server (ss) s hgher, around 95%. It s obvous that the performance of the produced FIM s smlar over the thrty runs of HGA. Ths happens because there are many common fuzzy rules among the dfferent set of rules. Ths proves that HGA s capable of fndng good rules and sets of rules regardless of the tranng patterns. 64

65 Fgure 2.9 Real Data: Average Classfcaton Rate of Tranng Process Fgure Real Data: Evaluaton of the produced FIMs 65

66 In the rest of ths secton, the most representatve dscovered relatonshps are presented. DNS queres Most cs sets of event pars represent the relatonshp between a DNS query from a clent and a flow from the same clent and another network component. We assume that these relatonshps are real. The most often relatonshps represent an example between a DNS and a web server. We dscover smlar relatonshps between DNS servers and an SMTP server (port 25) or root server of a dstrbuted fle system (port 7000 only n IBM network). Wldcard (*) means that ths value s random. Logn and securty Clent.*: DNS.53 Clent.*:WebServ.80 Clent.*: DNS.53 Clent.*:SMTPServ.25 Usually users have to communcate wth a logn server (SSH) before accessng an applcaton server (fle server system). We consder that the followng pattern corresponds to these knds of relatonshps, whch are real and usually have hgh dependency value. Clent.*:WebServ.80 Clent.*: LognServ.22 Another example, from the trace of the Unversty of Patras, shows a relatonshp between two web servers. Users, after vstng a web server, connect to another web server over TSL/SSL sesson (port 443). AFS system Clent.*:WebServ.80 Clent.*:WebServ.443 In the IBM network we extract relatonshps between parts of a dstrbuted fle server system (AFS). Port 7000 corresponds to the fle server tself, and port 7003 to the volume locaton database. A clent talks to the root server at port 7000 to fnd whch volume server contans the data and then communcates wth the approprate server at port 7003: Clent.*: AFS Clent.*: AFS Also browsng to dfferent parts of the AFS system ntates connectons to an AFS/Kerberos authentcaton server (port 7004) and the users and groups database (port 7002): Clent.*: AFS Clent.*: AFS

67 Enterprse applcatons Another nterestng ss relatonshp, whch confrms that enterprse networks (IBM network) rely on dstrbuted applcatons, shows the assocaton between a web server and a couple of Lotus Notes applcatons (port 1352) servers. Servers updatng Clent.*:WebServ.80 WebServ.*: LotusNote Clent.*:WebServ.80 WebServ.*: LotusNote Fnally, lookng for ms relatonshps, we do not fnd any n the trace of the Unversty of Patras, as network admnstrators have already mentoned. On the contrary, n the IBM trace we fnd relatonshps that correspond to nformaton exchange between a cluster of DNS servers lke the followng. Furthermore, we deduce that a master DNS server communcates wth ths cluster of servers va multcast DNS port DNSServ1.*:MasterDNS.53 MasterDNS.5353: DNSServ2.53 Generally, after checkng all knds of relatonshps, we could say that the false postves (generated relatonshps that really do not exst) are about 12%. We conclude that co-locaton of many component of dfferent servces on a sngle physcal machne s the man reason for false postves. However, our algorthm classfes relatonshps correctly and assgns them the approprate dependency value. From our study, t s sgnfcant not only to dscover a dependency but also to measure ts strength, and fuzzy rules are approprate to descrbe the ndstnct strength of network relatonshps. We deduce that real relatonshps have low, medum or hgh dependency value. Ths happens because a flow usually s not created only for a sngle applcaton. For example, flows to DNS or web servers are used from many dfferent servces; thus relatonshps between these network assets often do not have very hgh dependency value. Fgure 5 llustrates the servce overlay of IBM busness network. The servce overlay of unversty network s omtted snce t s smpler and all the dscovered relatonshps are already depcted on the servce graph of IBM network. 2.5 Conclusons A novel general technque to nfer relatonshps between IT nfrastructure s presented n ths chapter. The produced servce overlay can be a useful tool for admnstrators to acqure a clear vew of what s happenng n ther network. The man results of ths study are the followng: We dscover relatonshps that correspond to the applcaton structure, such as web and fle servers, wthout any pror knowledge. 67

68 The generated erroneous relatonshps are few and our approach measures accurately the strength of every dependency. The hybrd genetc algorthm has a hgh classfcaton rate and needs only a small fracton of tranng data (10%). Also t s an offlne process, applcable to dfferent networks. The set of fuzzy rules s not necessarly large: 30 rules suffce to classfy correctly all canddate relatonshps. Because of the smplcty of our algorthm and the flexblty of FIM, an onlne approach seems feasble. Ths overlay can be used for several admnstratve tasks, such as detectng msconfguratons or DoS attacks, montorng servce performance and desgn or expanson network nfrastructure. Clents Clents 116: :53 50:7004 Clents cs relatonshp ss relatonshp ms relatonshp Clents 116: :7002 Clents Clents 1016:80 2:111 2:755 16:88 Clents 56:659 2: :22 Clents Clents 91: :53 91:53 222:25 243:53 157:53 Clents Clents Clents 116: :53 Clents Clents 3538:80 24: :53 427:53 2: :53 Fgure 2.11 Graph of Relatonshps 68

69 69

70 CHAPTER 3 Control of Dstrbuted Web Servces on Cloud Computng Infrastructure As t s mentoned n the frst chapter, automated resource provsonng s one of the most challengng research problems of cloud computng. Meetng performance specfcatons of consoldated web servces n a data center s crtcal ssue, snce the control of the underlyng cloud computng nfrastructure must be sophstcated enough to meet Servce Level Agreements (SLA) requrements and system constrants. Vrtualzaton technology allows the consoldaton of many servces on a server cluster. The co-hosted applcatons share the avalable resources accordng to ther requrements. Ths mples that provders optmally allot the computatonal and network resources to the competng servces and customers pay only for the consumed resources accordng to a usage-based prce model. The performance of a web servce s descrbed n the SLA between the servce provder and the customer; the Servce Level Objectves (SLOs) are the network metrcs, whch prescrbe exactly the desred Qualty of Servce (QoS) levels. The most common SLOs are the request response tme and the served request rate durng a fxed tme nterval. From the customer s vew, the goals are the satsfacton of some predefned nomnal values of the SLOs and a guaranteed level of QoS wth the mnmum fnancal cost. These objectves usually conflct wth the goals of the provder, who ams for a management system that optmally allocates the resources to each servce n a manner that acheves SLOs, ensures avalablty of resources under any workload crcumstances and mnmzes the operatonal cost (e.g. the consumed power energy and machne falures). Due to the above contradctve targets between the customer and provder, managng the performance of such complex systems s a crtcal and challengng research problem. Furthermore, there are two domnant factors that make the automated resource provsonng even more complex; the workload of these web servces s generally unpredctable and hghly varant, whle the avalable resources are subject to constrants. There are several emergng problems that an autonomc management framework must address n order to succeed all the above goals. Network controllers are mplemented n two levels, namely, the local level and the global level. Global level controllers focus on the problems of load balancng, whch shares the volume of requests among the replcas of an applcaton, or Dstrbuted Rate Lmtng (DRL) problem that ams to equalze the performance at each of the underlyng data centers accordng to a performance ndcator. Also they arrange the Vrtual Machnes (VMs) placement, whch determnes the set of applcatons executed by each server. Local level controllers manly concern applcaton specfc or server specfc problems,.d. admsson control (AC) that rejects requests under peak workload condtons and resource allocaton (RA), whch determnes the resources of each VM (CPU, memory, network bandwdth) ether by changng ther capacty or usng the Dynamc Voltage 70

71 Local Level AC+RA Local Level AC+RA Ste 1 Ste , 2,, n , 2,, n, 2,, 1 Global level n , 2,, n , 2,, n Local Level Local Level AC+RA AC+RA Ste 3 Ste 4 Fgure 3.1 Control Archtecture of Cloud Computng Infrastructure and Frequency Scalng (DVFS) mechansm of servers to change ther servce rate. Fgure 3.1 llustrates the general control archtecture for the consoldaton of web applcatons on a dstrbuted cloud computng platform. An Infrastructure as a Servce (IaaS) or Software as a Servce (SaaS) provder has a set of server stes, whch are heterogeneous n terms of resource capacty. The global level controller must share the total ncomng workload of n applcatons (, 1, n ) among ther replcas j (, 1,, n j 1, m ) takng nto consderaton the avalable resource capacty, on the local level. The local level controllers am at satsfyng the SLOs reference value, the physcal constrants and mnmzng the power consumpton. Most of the exstng solutons focus only ether on local level or the global level controllers. Studes about local controllers adopt ether admsson control or resource allocaton, whle very few tackle both problems smultaneously. These approaches have two drawbacks, namely, they do not scale well and they do not guarantee 71

72 feasblty and stablty of the desred operatng pont. On the other hand, the global level controllers solve the capacty allocaton and load balancng problems by makng strong, thus conservatve, assumptons on the underlyng nfrastructure and ts dynamc behavor. The exstng approaches whch concern the modelng and control of web servces can be categorzed accordng to the employed model and the control method. Intally, a categorzaton can be made accordng to the modelng method of the behavor of consoldated web servces on a cloud computng nfrastructure. Models based on queung theory rely on the assumpton that the system s at steady-state condton, whch appears to be contradctve to the dynamc nature of t. On the other hand, lnear tme nvarant (LTI) state-space models can effcently descrbe the transent behavor of the system. However, the accuracy of the model s lmted near an operaton pont. The most promsng category of state space models employed are lnear parameter varyng (LPV) models, because they vary accordng to the value of a modelng parameter. Ths type of modelng s sutable to descrbe web servces behavor, because t effcently captures the effect of varous parameters such as ncomng request rate or servce tme. Table 3.1 summarzes the benefts and drawbacks of each modelng method whch are used n the relevant studes. Queung Models Strengths Weaknesses Related Studes Scalablty. Vald only n steady [61] [70] state. Specfc workload dstrbutons. LTI Models Capture transent behavor. Scalablty. Accuracy. [71] [73] LPV Models Capture transent behavor. Embed system s parameters. Scalablty. Accuracy. Identfcaton Procedure. [74] [77] Table 3.1 Comparson of exstng modelng approaches. A second categorzaton of the exstng approaches concerns the control method used. On the local level, admsson control or resource allocaton s employed. Furthermore, resource allocaton approaches can be dstngushed to those that change dynamcally the CPU capacty of servce VMs and those that consder as control varable the CPU frequency of the servers wth fxed CPU capacty (DVFS technology). The above technques can be combned wth replcas addton usng a pool of dle servers or mgraton of VM. Most of the resource allocaton solutons assume that servers are 72

73 protected from overloadng. Nevertheless admsson control and resource allocaton are dual problems and must be solved jontly towards ensurng the performance of the web servce under any workload varatons. Table 3.2 summarzes the benefts and drawbacks of each control method whch are used n the followng related studes. Admsson control (AC) Strengths Weaknesses Related Studes Overloadng Fxed Capacty. [64], [69], [71], [77] Protecton. Rejecton of requests under heavy workload. Resource Allocaton (RA) Dynamc Capacty. No performance guarantee under heavy workload. [61] [63], [67], [70], [73], [75], [76], [78] AC+RA Stablty Guarantee. Overloadng Protecton. Dynamc Capacty. Complexty. [64] [66], [72], [74] Table 3.2 Comparson of exstng control approaches. Nevertheless, ndependent of the modelng and control approach, t appears that no exstng method consders the explct determnaton of a desred operatng pont. The LTI or LPV models combned wth feedback control nfer an operaton pont from measurements or the system dentfcaton procedure. However, f the state varables are far away from that nomnal operaton pont, the underlyng servers are saturated and the model becomes non-lnear. Smlarly, queung theory approaches combned wth an optmzaton method suffer from the same lmtaton. In detal, queung models assume specfc dstrbuton (eg. M/G/1) for the ncomng request and servce rate and determne the operatng pont only for steady state condtons. In the followng paragraphs, we recall some representatve studes of nterest that are close to the approaches proposed n ths chapter. Ardagna et al. [61] presented a dstrbuted algorthm for capacty allocaton and load balancng. They modeled the system s dynamc behavor wth a M/G/1 queue and solved the jont control problem as an optmzaton problem applyng a decomposton technque for non-lnear programmng. Although the resource allocaton s smplfed, snce they assume sngle ter web applcatons, VMs have predefned fxed capacty and are homogeneous n terms of resources (CPU, RAM). In [64], a jont admsson control and resource allocaton framework utlzed queung theory to capture systems dynamcs. The two controllers were separated n two dfferent optmzaton subproblems that were solved sequentally n an teratve fashon. However t was not examned whether the 73

74 sequence of solutons provdes any performance guarantee. Kusc et al. [62], [63] derved an analytcal mathematcal model from queung theory and tackled resource provsonng as an optmzaton problem solved wth a predctve control method. Some system parameters (e.g. servce rate) were emprcally computed whle the mplementaton consdered the VM placement and capacty allocaton problems separately. Also the scalablty of the approach was not demonstrated. Urgaonkar et al. [65] presented a utlty based soluton, whch maxmzes the average applcaton throughput and energy cost of the data center. Usng queung theory models and an optmzaton technque framework, they addressed the problems of admsson control, server selecton and resource allocaton separately. In [66], the authors proposed a holstc approach for applcaton placement, admsson control and resource allocaton. They used queung theory models and a greedy algorthm to solve the placement problem and consequently they splt admsson control and capacty allocaton as two decoupled subproblems. There, admsson controller s desgned separately from [79] gnorng the resource allocaton soluton. Wang et. al. [67] proposed a two level controller combned wth queung modelng. The global level determnes f the CPU share s effcent for the total ncomng workload and t actvates a certan number of servers. The local controller solves the resource allocaton problem n order to satsfy the predcted workload. However, t was assumed that the ncomng load to each applcaton replca s proportonal to the CPU share and a lnear relatonshp between CPU share and processng rate s proposed. Snce all the above solutons use queung models they are vald only for steady state condtons and they do not examne f the operatonal pont s feasble or not. In [71], a feedback admsson controller usng an LTI model was proposed, whch also takes nto account also CPU utlzaton to regulate the admtted workload. ACRA [72] s an ntegrated framework that solves jontly the admsson control and resource allocaton problems usng a group of LTI models, whch s not scalable as the number of consoldated servces ncreases. Furthermore t was shown that the determned equlbrum pont s always feasble. Yaksha [68] presented an admsson controller, whch s based on an ntal queung model and an LTI model that s derved from the lnearzaton of the queung model whle a proportonal-ntegral (PI) feedback controller s desgned. However the obtaned lnearzed resdual error model may be large. Lu et al. [69] addressed the above lmtaton by usng an adaptve controller to desgn the feedback loop workng together wth the queung model through on-lne tunng of model parameters from measurements. Ths self-tunng approach lowers the need for system characterzaton experments n desgnng the controller and makes the system more robust. However the range of admttng probablty adjustment s lmted because of the lnearzaton model near the operaton pont. In [73], authors demonstrated a resource allocaton framework that regulates CPU frequency and VM capacty to mnmze the power consumpton of server clusters. They used decoupled lnear models and PI and MPC (Model Predctve Control) controllers. Qn et. al. [75], used an LPV model to desgn a robust controller on frequency doman. However, the convergence of the dentfcaton algorthm s very senstve to some parameters. 74

75 Smlarly n [76], the authors derved an LPV model from the lnearzaton of a queung model and they desgned a robust resource allocator n frequency doman usng DVFS technology. In [74], the authors combned the admsson control and resource allocaton usng LPV system dentfcaton and Model Predctve Control, whch calculates the admsson probablty and CPU frequency usng DVFS technology. Although ther controller offers a trade-off between power consumpton and SLOs achevement, t does not guarantee the performance among dfferent operaton ponts. Gan et al. [77] presented an LPV modelng combned wth a PI admsson control, wthout addressng resource allocaton. It s one of the few studes that provded a stablty analyss of ther controller. In [70] the authors presented a two level control framework. The global level decdes how many servers are necessary and the local level optmzes the CPU share of VMs usng an MPC optmzaton algorthm. The model s derved from analytcal equatons where the operaton pont was chosen arbtrary. Fnally DynaQos [78] s a model-free self tunng fuzzy controller whch on the global level computes the necessary aggregated capacty and the local level computes the capacty share n order to succeed the SLA requrements. Apart from [72], all the aforementoned papers do not examne f ther soluton guarantees the feasblty of steady-state operaton, the stablty and the consoldated applcatons performance. Contrary to most exstng approaches, n ths chapter we propose two dfferent local level controllers that address the above lmtatons and guarantee the performance of the applcaton under the system constrants and any varaton of ther ncomng workload. These local controllers solve admsson control and resource allocaton smultaneously n a unfed framework, thus, makng the cooperaton wth exstng global level controllers easer. The man contrbutons of the thess are: Satsfacton of the defned targets by the SLA. Optmal usage of the computatonal and network resources. Desgn and mplementaton of dual controllers that address admsson control and resource allocaton smultaneously. Accurate modelng of the web servce operaton. Satsfacton of the underlyng system constrants. Computaton of feasble operaton ponts accordng to SLA. Guarantee and analyss of the system stablty. Applcaton of novel methods of control theory to recent network research problems. The frst local level controller, named ACRA (Admsson Control and Resource Allocaton) s a modelng and control framework that addresses RA and AC jontly. The objectve s to maxmze the provder s revenue (by maxmzng admttance of customers to the provded servce), whle satsfyng customer s QoS requrements. Although ACRA does not consder power management as basc target, t ncludes on ts objectve the mnmzaton of the necessary computng resources. Instead of a 75

76 sngle lnear state space model, a group of lnear state-space models wth addtve uncertantes s used n order to cover the varaton of workload and quantfy the system s nonlneartes. For the desgn of the dual controller, we use a novel settheoretc control theory whch provdes stablty and robustness guarantees. Fnally ACRA has the ablty to adapt between several operatng ponts, because t always assures that the system wll be drven to the neghborhood of a feasble equlbrum pont. The second modelng and control framework presented n ths chapter adopts a Lnear Parameter Varyng (LPV) state space models that capture the dynamc behavor of the underlyng nfrastructure. The operatng condtons are determned accordng to an optmzaton crteron. By solvng a lnear program, a set of feasble operatng ponts s calculated whch satsfes the desred QoS nomnal values. The resultng stablzng state feedback control law s affne state-dependent control law that can be easly mplemented. Indeed, the computatonal complexty of the controller mplementaton s small, snce at every tme nstant only a lnear program and a pont locaton problem are solved. Fnally, convergence to the feasble operatng pont and satsfacton of the system s constrants are guaranteed, for a number of desred operatng ponts of nterest. The remander of the chapter s structured as follows. Intally, all the essentals noton of modelng and control are descrbed n the followng secton. Sectons 3.2 descrbes ACRA framework n full detals. The second autonomous modelng and control framework s presented n secton Basc Defntons of Modelng and Control Theory Lnear Tme Invarant State Space Models A state space model of a system s the mathematcal descrpton of the relatonshp between the cause and the effect or the nputs and the outputs of the system [80]. In ths thess, only dscrete tme state space models are used. The general form of a dscrete tme state space model s, n m n x k 1 f ( x( k), u( k)), f: (3.1) n m where x( k), u( k) are the state and the nput vector respectvely and k s the tme varable whch values are nteger numbers. The most wdely used state space models are the lnear tme nvarant (LTI) state space models, where the functon f ( x( k), u( k)) of (3.1) s lnearly dependent on x ( t), u( t), 76

77 x( k 1) Ax( k) Bu( k) y( k) Cx( k) Du( k) (3.2) nn nm where A, B are constant tme nvarant matrces that descrbe the l system s dynamcs and yk ( ) s the output vector of the system Lnear Parameter Varyng State Space Models Although the LTI models have become a strongly founded modelng framework, they cannot descrbe effcently the dynamcs of complex systems. The need to operate processes wth hgher accuracy/effcency has soon resulted n the realzaton that the commonly non- lnear and tme-varyng nature of many systems must be handled by the control desgns [81]. Ths has led to the brth of lnear parameter-varyng (LPV) systems, whch are dependent on the varyng sgnal p, hence the name parametervaryng, whle the dynamc relaton between the system sgnals s stll lnear. One partcular type of model structure, whch s used n some LPV dentfcaton approaches, orgnates from the nput-output (IO) type of representaton of the data generatng system n the LTI predcton-error settng. These LPV-IO representatons are commonly defned n a flter form, y na n b a ( p) q y 1 j0 b ( p) q j j u (3.3) where n n, the coeffcents a ( p), b ( p) are functons of p wth constant a dependence and b q used n the model. j y, q u are the past values of states and nputs respectvely that are j Another type of model structure s nspred by the classcal state space (SS) representaton based LTI models. The so-called LPV-SS representatons of the data generatng system are often gven by the followng form, x( k 1) A( p) x( k) B( p) u( k) y( k) C( p) x( k) D( p) u( k) (3.4) na na na nb where A( p), B( p) are matrx functons wth statc dependence on p. Furthermore, the matrces are often consdered wth lnear dependence. In case of A, such dependence s defned as, n A A p l1 A f 0 l l (p) (3.5) 77

78 nana where have real elements and { f 1 ( p),, f ( p)} s a set of known bass A l functons. Ths type of dependence s called affne, and used as a core assumpton n many LPV control-desgn approaches. In general, the rght bass functons may not be known. Usually bass functons for whch an algorthmc mplementaton s smple are used. The most common bass functons are polynomal or perodc Stablty The noton of stablty s strongly connected wth the dynamc behavor of systems n modern control theory. Many dfferent knds of stablty are defned,.e. nput-output stablty or stablty of an equlbrum pont. In general, a stable system means that the state or output varables of the system are nsde a desred area and they reman there even f there are nsstng or momentary dsturbances. In ths chapter, we focus on the stablty of an equlbrum pont x eq. The most general type of equlbrum pont stablty s Lyapunov stablty, whch guarantees that the system trajectores wll reman close to x f they start from a neghborhood of the equlbrum pont [82]. eq Furthermore, asymptotc stablty s adopted n the followng sectons. Addtonally, set-theoretc notons of stablty analyss and control desgn problem, whch dentfy and characterze subsets of the state space contanng the desred equlbrum state wth specal propertes: postvely nvarant sets or ultmately bounded sets, are ntroduced here. In the forthcomng paragraphs wthout loss of generalty, all the necessary defntons are gven assumng that x 0. eq np Defnton 3.1.1: A sphere denoted, B wth radus s 0 and the orgn as ts center s s n B { x : x s} s where s any possble norm of vector x. Defnton 3.1.2: Assumng a dscrete tme system of the followng form x k 1 f ( x( k)) (3.6) Then the equlbrum pont x 0 s locally Lyapunov stable, f and only f eq 0 ( ) 0 then, x 0 B ( ) x( t; x0) B, t 0 Defnton 3.1.3: The zero equlbrum pont of (3.6) s contractve n a regon n D f x D then 0 78

79 The regon D s called Doman of Attracton (DoA) of the equlbrum pont. Defnton 3.1.4: The zero equlbrum pont of (3.6) s asymptotcally stable f and only f t s Lyapunov stable and contractve, where D s DoA. ( ) B D In the followng defntons, we present the essentals results of Lyapunov theory (second or drect Lyapunov method) whch connect the stablty property wth a specfc type of functons. Defnton 3.1.5: Assumng a contnuous functon V( x), V : D where D contans the orgn. Then V (x) s postve (sem)defnte n D f V( x) ( )0, x D \ {0} V(0) 0 Defnton 3.1.6: Functon V (x) s negatve (sem)defnte f V (x) s postve (sem)defnte. Defnton 3.1.7: For dscrete tme systems (3.6), the total dfference of functon n V( x), V : respectvely to system (3.6) s V( x) ( 3.6) V( f ( x)) V( x) (3.7) Then the Lyapunov theorem for dscrete tme system s formulated as, Theorem 3.1: ([82],[80]) Assumng a postve defnte functon V( x), V : D then If the total dfference V (x) (3.6 ) of (3.7) s negatve semdefnte x D 79, then the system s locally Lyapunov stable. If the total dfference V (x) (3.6 ) of (3.7) s negatve defnte x D, then the system s locally asymptotcally stable. Functon V (x), whch satsfes the above theorem, s called Lyapunov Functon (LF). From the prevous analyss, the stablty problem s equal to fndng a postve defnte functon whch s nonncreasng or decreasng along the trajectores of the system (3.6). Fndng an LF allows us to defne sets wth specal propertes respectvely to the equlbrum pont. For example, f there s an LF V (x) that guarantees the stablty or asymptotc stablty n a regon D and the sets n R( V; ) { x : V( x) } D are close and contan the zero equlbrum pont then these sets consst an estmaton of DoA. In the forthcomng lnes, all the essental defntons are gven.

80 Defnton 3.1.8: [83] The set S n s postvely nvarant to (3.6) f and only f x 0 S the system trajectory wll reman nsde S for all the future moments x ( t; x0) S, t 0. Defnton 3.1.9: [83]For a convex compact set Mnkowsk functon s, S n whch contans the orgn, the ( x ) S nf{ : 0, x S } Then, the set S s ε-contractve to (3.6), 0 1, f x S 0 t ( x( t; x )) ( x ), t t S 0 S 0 0 Now we can show the connecton between the Lyapunov functon and the postve nvarant set, the DoA and the ε-contractve set. Remark 3.1.1: Assumng the system (3.6) and a canddate LF V (x), f the set n R( V; ) { x : V( x) } s convex, compact and contans the orgn, then the followng conclusons apply: If V (x) s LF that guarantees Lyapunov stablty, then the set R( V; ) s postve nvarant to system (3.6). If V (x) s LF that guarantees asymptotc stablty, then the set R( V; ) D s postve nvarant and DoA to system (3.6). If V (x) s LF that guarantees Lyapunov stablty and t apples that V x) ( 1) V( x( )), 0 1 ( ( 3.6) t then the set R( V; ) Ds ε-contractve to system (3.6). n Defnton : [84] Let subset of the state space X be a compact set contanng the orgn as an nteror pont. System (3.6) s sad to be unformly ultmately bounded n subset for any n X f there exsts a subset Δ, n X such that t 0 T and every ntal condton x ( t 0 ) x 0 there exsts a postve nteger N( x 0 ) such that x( t; t0, x0) X for all t t 0 N( x 0 ) attracton of X.. Set Δ s set to be the doman of The most mportant benefts from Lyapunov theory s the characterzaton of the stablty of the equlbrum pont and the possblty of defnng sets wth nterestng propertes, e.g. f we can fnd a LF whch ensures the asymptotc stablty of the n equlbrum pont of a constraned system x() t S x, then any trajectory that 80

81 begns from a pont nsde wthout volatng the system constrants. R( V; ) S wll be drven to the equlbrum pont 3.2 ACRA: A Unfed Admsson Control and Resource Allocaton Framework for Vrtualzed Envronments x Ths secton presents ACRA (Admsson Control and Resource Allocaton) [72] an autonomc modelng and control framework that provdes accurate modelng, jont admsson control and capacty allocaton among dfferent consoldated cloud computng servces. The objectve s to maxmze the provder s revenue by maxmzng admttance of customers to the provded servce, whle satsfyng customers QoS requrements. Although ACRA does not consder power management as basc target, t ncludes on ts optmzaton crtera the mnmzaton of the necessarly allocated computng resources. Instead of a sngle lnear state space model, a group of lnear state-space models wth addtve uncertantes s used n order to cover the varaton of workload and quantfy the system s nonlneartes. For the desgn of the dual controller, a set-theoretc controller s used whch provdes stablty and satsfacton of the underlyng system constrants. Fnally ACRA has the ablty to adapt between several operatng ponts, because t always assures that the system wll be drven n the neghborhood of a feasble equlbrum pont. Accordng to the SLA, the average response tme (state varable) durng a specfc tme nterval s denotes as SLO and the admtted request rate (admsson control) and the CPU capacty (resource allocaton) are the control varables. The total CPU capacty of a machne s shared among VMs that are deployed on t. ACRA controller ams to preserve the average response tme under ts target value wth the mnmum CPU resources. The proposng modelng and control scheme are based on the followng features of web servces. Although the ncomng request rate of web servces s generally non-statonary and hghly varant, t usually follows specfc patterns [85], whch allow us to predct the workload condtons for a perod n a day. Secondly, nternet applcatons have many types of transactons. For example browsng on a ste needs less computng resources than buyng a product on the same ste. Also admnstratve tasks, such as savng the data of the day or executng partcular scrpts durng a specfc tme nterval, have varyng servce demands and tme constrants. Apart from the dstrbuton and the type of ncomng requests, the constrants of the applcaton s nfrastructure should be taken nto account. CPU and memory resources of the underlyng machnes are lmted, so the reserved capacty for the co-hosted VMs should never exceed the total capacty. Also there are constrants on the amount of the served ncomng requests from the VMs. Fnally there exsts an upper lmt of the response tme of the ncomng requests due to VM s servce rate and network restrctons. All the above features should be consdered n the system dentfcaton and the controller desgn. Fgure 3.2 llustrates the structure of ACRA framework. 81

82 System Constrants LTI Models Equllbrum Pont Feedback Controller u Fgure 12 Structural Dagram of ACRA Framework A sngle mathematcal model cannot descrbe the system s dynamcs under any crcumstances. Thus a set of lnear tme nvarant (LTI) state-space mathematcal models s dentfed n order to cover the whole range of workload. Furthermore, an addtonal term that models addtve bounded uncertantes and nonlneartes of the system s ntroduced. Ths modelng approach s accurate and allows determnng many dfferent feasble operatng ponts. Dependng on the current workload condtons and the desred SLO values, a set of feasble operatng ponts s computed by solvng a mult-crteron optmzaton problem whch takes nto account the system constrants. Ths mples that the system trajectory wll always be nsde a desred operatng area and t wll be adaptve towards workload fluctuatons. Fnally ACRA addresses admsson control and recourse allocaton as a dual decson problem and t guarantees the system stablty and the satsfacton of the system constrants. A settheoretc feedback controller s used whch has the ablty to ensure further propertes of stablty lke postvely nvarant sets and ultmately bounded sets Modelng and System Identfcaton In general, mult-ter servces, whch are deployed on many VMs, have complex and non-lnear dynamc behavor. Modelng the system s dynamcs wth a sngle lnear state space model s not precse. A group of lnear state space models s used to cover the total range of ncomng workload. Assumng that there are m applcatons, n servers and each applcaton has one VM on every server. For each applcaton, the ncomng request rate vares between a mnmum L ) and a maxmal value L ). We suppose that ( mn, L mn, corresponds to a small postve value and ( max, L max, to the maxmum served request rate when all VMs of the applcaton get ther maxmum CPU capacty. Next, we dvde each nterval L, L ] for every applcaton [ mn, max, 1,m n p smaller equdstant parts, e.g. spannng a range of 50 requests each. Thus, combnng all applcatons, a number of N pm dfferent regons are defned n the request rate space. For each regon, we assgn a LTI model affected by 82

83 addtve dsturbances denoted by dynamcs, when the request rates are nsde regon M q, q 1, N whch descrbes the system s M : rt( k 1) A rt( k) B u( k) ( k), q 1,, N (3.7) q q q m ( n1) m where rt s the response tme (state) vector of all m applcatons, u s the nput vector that contans the nm VM capacty nputs cap j, 1,, m, j 1,, n and the m admtted request rate varables lad, 1,, m. The effect of nonlneartes m and uncertantes s represented by tme varyng vector whch s consdered here as unknown but bounded addtve dsturbance. Wthout deteroratng the accuracy of the modelng, A q, q 1,, N are restrcted to be dagonal matrces mplyng that response tme of each applcaton depends only from ts own past values. Also we assume each applcaton does not depend on the nputs (VM capacty and admtted rate) of other co-hosted applcatons, snce ths does not mprove the model s accuracy. Thus the elements of matrces, q 1,,N, whch do not correspond to the applcaton s nputs, wll be zero. The nput vector u(k) and the matrces A, B are formulated as, q q B q T u( k) [ cap cap lad lad ], 1,, m, j 1,, n 11 j 1 B a11 0 Aq, 1,, m, q 1,, N 0 a mm b br b b rl, r 1,, m, l 1,,( n 1 m 11 1r q ) 1 We use the Recursve Least Square (RLS) algorthm [86] to dentfy the nomnal dsturbance-free LTI models, M * q : rt( k 1) A rt( k) B u( k), q q q 1, N (3.8) The bounds of vector (k) of each model (3.7) are determned by the mnmal and maxmum error of the RLS algorthm durng the dentfcaton of the correspondng nomnal model * M q. As nputs profles, we use a mx of Gaussan, constant and Posson dstrbutons for ncomng load and unform dstrbutons for VMs capacty. These typcal traffc patterns correspond to normal condtons and sudden spkes of ncomng requests. For each local model M q, q 1,, N, all the tests always satsfy the states and nput constrants, whch are analytcally descrbed n the next secton. 83

84 3.2.2 State and Input Constrants There are some constrants of the state and nput varables that must be satsfed under any crcumstances. The average response tme on a specfc tme nterval vares from zero untl the value when the system s saturated or request expred due to network constrants (TCP). Thus, states are bounded to the constrant set X, where m X { rt :0 rt rt, 1,, m} (3.9) Input constrants consder the restrctons on VM capacty and admtted request rate. Many VM can be deployed on a physcal server and they share ts CPU capacty. For each server j, j 1,, n, the CPU capacty of each VM vares from a mnmum value c mn untl a maxmum c max whch depends from the total capacty of the server. Addtonally the sum of VMs enttlement should not exceed the total capacty of the server c, j 1, n. s max, j, max c mn cap j c max, 1,, m, j 1, n (3.10) m 1 cap j c s max, j, 1,, m, j 1, n (3.11) There are also constrants regardng the admtted workload. As we mentoned n the prevous subsecton, the ncomng load of each applcaton vares from L mn, to L max,. For each local model the admtted request rate s between L mn, and L max, values, lad lad lad 1,, m mn max, (3.12) Puttng together the nput constrants, we can defne the nput constrant set t follows, U u c cap c m j n ( n1) m { : mn j max, 1,,, 1, mn m 1 cap c, 1,, m, j 1, n j s max, j lad lad lad, 1,, m } (3.13) max u U as Fnally the uncertantes of the model are bounded by the mnmal and maxmum values of modelng error of the system dentfcaton. Thus uncertantes constrants are defned as N where, m N { : n, 1,, m} (3.14) mn max 84

85 3.2.3 Determnaton of the Equlbrum Pont The target value of SLO s tme varyng accordng to workload profle and the specfc tasks of the applcaton durng a day. For each desred operatng value of average response tme and admtted request rate, the correspondng equlbrum pont has to be determned for our system, whch s descrbed by a local LTI state space model. After the defnton of the systems constrants and gven the optmzaton crtera such as maxmzaton of admtted request rate and mnmzaton of CPU resources, the feasblty of a desred operatng (equlbrum) pont has to be examned. Gven a nomnal model RTref L * M q, the desred response tme vector m X and the desred admtted request rate vector T [ lad, ], 1, m, there s a feasble equlbrum pont f there exsts an ref ref, nput RT ref vector U [ CAP L ], U U such that T ( n1) m ref ref ref ref ART BU. In order to relax the above defnton of the equlbrum pont ref ref we reformulate t as follows, mn w capj, lad, d subject to RT RT U d d ref ref ref U lad 0 c j d 1 j1 1 ARTref BU m X m L n ( n1) m ref cap d ref w m d (3.15a) (3.15b) (3.15c) (3.15d) (3.15e) (3.15f) (3.15a) can be solved as an optmzaton problem wth constrants. The constants w c and w d are the weghts of the cost functon and dependng on ther values we can have a trade-off between mnmzng the VMs capacty and succeedng the desred admtted request rate. (3.15b) corresponds to the condton of the exstence of the equlbrum pont. (3.15c) and (3.15d) correspond to the state and nput constrants respectvely. The auxlary varables d are defned n order to relax the defnton of equlbrum pont. (3.15e) and (3.15f) guarantee that f there s no soluton that contans exactly the desred L, then the soluton of the problem wll be the closest value to Lref ref whch satsfes all the constrants. The above formulaton guarantees that the computed operatng/equlbrum pont always satsfy the system constrants ACRA Controller Desgn Admsson control and resource allocaton are two decson problems that must be consdered jontly. A novel controller s presented here that computes the sutable 85

86 nput vector to lead and stablze the system near to the desred equlbrum pont usng the correspondng local system model. Most of the proposed solutons focus on stablzng the system trajectory on a specfc operatng pont, but ths s mpossble because the systems dynamcs nvolve nonlneartes n the state space descrpton as well as tme-varyng addtve terms that represent uncertantes. Thus there s no feedback control law ensurng asymptotc stablty of an equlbrum state. Instead settheoretc approaches deal wth the stablty analyss and control desgn problem, dentfyng and characterzng subsets of the state space contanng the desred equlbrum state wth specal propertes: postvely nvarant sets, dsturbance nvarant sets, or ultmately bounded sets. For more nformaton, regardng these notons, readers can refer to the excellent monograph [83], summary paper [87], and artcles [84], [88]. In our case, snce we consder nput and state constraned (3.9) and (3.13) systems affected by addtve dsturbances (3.7), and t seems straghtforward to follow the settheoretc approach. Our goal s to desgn for each model M q, q 1,, N, correspondng to a gven workload profle, an affne state feedback control law and m compute a doman of the state space such that all trajectores startng from are transferred to a target set R, whch contans the equlbrum pont, n a fnte tme and reman n t. In contrast to classcal control problem formulatons, we want to confne all trajectores n a target set R and not to drve them to the equlbrum pont snce the system s affected by addtve dsturbances. The resultng closed-loop system s sad to be ultmately bounded n R from [84], [88]. In order to apply these methods, ntally we apply the essental coordnates transformaton to obtan a transformed nomnal lnear model wth the equlbrum pont to the orgn: z( k) rt( k) RT v( k) u( k) U M q ref : z( k 1) A q ref z( k) B v( k), q q 1, N (3.16) Also state and nput constrants are transformed accordngly, m X { z : RT z rt RT } (3.17) ref max ref U v u U v u U ( n1) m { : mn ref max ref, m vcap c vcap, 1,, m, j 1, n} (3.18) j s max, j ref, j 1 1 m where vcap j cap j cap ref, j transformed VMs capacty nputs. We wll desgn a state feedback controller of the followng form, 86

87 u( k) K( rt( k) RT or v( k) Kz( k) ref ) U ref (3.19) Secondly, we must compute the target set R, whch must possess the robust nvarance property [83], meanng that f the closed-loop system trajectory s n R, t remans n t for all future nstances. There are many ways to obtan the target set, preferably small enough to guarantee that SLOs are met when the system s trajectores are nsde. Indeed, there are methods to approxmate a mnmal postvely nvarant set for ths class of systems [89], although ths s outsde the scope of the current soluton. In our case, we obtan a canddate target set S 1 by applyng the Jordan decomposton [90] of the closed-loop system A BK, where K s the gan matrx that places ts egenvalues nsde the unt rhombus [91], S { z : G z w } (3.20) m mm V where G1 : G1, w1 V A BK. 2 2m and V s the real Jordan form of matrx The state and nput constrants set X and U are formulated below usng the standard half-space representaton of polytopes by S z and S v, m S { z : G z w } (3.21) z z z S v G v w ( n 1) m v { : v v} where G, w and z 2mm 2m z G, w. v 2 n( m1) mn 2 n( m1) z The bounded uncertantes set S s, m S { : G w } (3.22) where G : G I, w 2mm mm 2m I mm. Thrdly, our goal s to compute a control law of the form (3.19) and a set Δ such that the closed-loop system s ultmate bounded n R. We choose Δ to have the form, m { z : G z w } (3.23) 1 1 Accordng to the generalzed Farkas lemma [92], we can compute a control law v Kz and set Δ, whch s the largest set, ftted n S z and S v. Ths can be descrbed by the optmzaton problem, 87

88 K, H, H, H,, subject to max * G ( A BK ) H G H H G H w H w G z w H G G K v w z v 1 w d w The above optmzaton problem s non-lnear because of the frst nequalty, thus we 1 defne b and reformulate the problem as lnear, where max V d max V K, H1, H2, H3,, b b mn (3.24a) subject to G ( A BK) H G (3.24b) H w bd w H G 2 1 H w 2 2 H G 3 1 H w 3 2 G z bw v (3.24d) * 0 (3.24h) z G K bw, 1,H2, H3 v 2 (3.24e) (3.24f) 1 (3.24c) (3.24g) H are non-negatve matrces, K s the feedback * gan matrx, and ε s postve between zero and a small value (.e. * 1) and t s a metrc of convergence speed. (3.24b), (3.24c) guarantee the postve nvarance and contractveness. Equatons (3.24d), (3.24e) guarantee that satsfes the state constrants and (3.24f), (3.24g) guarantee that S v(z ), where S z m { z v v Evaluaton : G Kz w }, satsfes the nput constrants. In order to evaluate ACRA a testbed s bult, whch emulates the operaton of an applcaton n a vrtualzed envronment. A server wth 8 CPU cores (Intel(R) Xeon(R) CPU E GHz) and 8GB RAM s used. One CPU processor s dedcated for the clent generators of each applcaton, whch produce requests of dfferent dstrbutons, such as constant, unform and Posson. Another core hosts the VMs of each applcaton. Each VM runs a scrpt whch does some complex

89 mathematcal calculatons that are CPU ntensve. On a thrd core we place a VM that hosts the controller of ACRA. We use XEN Hypervsor as a VMs hypervsor [93]. The testbed ncludes all the necessary montorng tools to measure the average response tme, ncomng workload rate, served request rate and absolute and relatve CPU utlzaton. We demonstrate an experment, whch shows ACRA s performance and compares t wth two well-known approaches. We assume two applcatons (m = 2), whch are hosted on the same CPU core n two separated VMs (n = 1). The ncomng workload for both applcatons vares between [175,325] req/s. Followng the system dentfcaton method descrbed n secton 3.2.1, we dvde the request rate space nto hypercubes (squares for two-dmensons space) wth a step of 50 requests and generate a local lnear model for each square, such as n (3.7). Fgure 3.3 llustrates the created hypercubes (squares) and the produced models Mq. The measurements and control are updated every 20sec. Fgure 3.3 Group of LTI Models 89

90 For both applcatons t s ntally assumed that the admtted request rate s hgh and ther computng resources for ths type of requests (e.g. browsng) are low. Thus the equlbrum pont corresponds to the reference value of average response tme of 1s and the desred admtted request rate s 300req/s for both servces (model M9 on Fg. 3.3). After a perod there s need to change the SLO target value of both servces, snce the domnant type of transactons needs more CPU resources and has lower ncomng request rate than the prevous. Ths determnes the target of average response tme of to be 2s and desred admtted request rate 200req/s whch corresponds to Model M1 on Fgure 3.3. The modelng uncertantes vary nsde the bounded area of [ 0.4, 0.4] for both canddate equlbrum ponts. ACRA state feedback controller s appled to regulate the system and the controllers of [73] and [74] are used n order to compare our results wth some well-establshed solutons. The frst controller s a classcal proportonal-ntegral (PI) controller, whch s used also n many other studes. The necessary egenvalues vector of the PI controller s set to T eg The second one s an MPC scheme based on predctve control theory [94]. The predctve horzon of the MPC controller s set to H = 5 and the trade-off parameter of the approach a = 0.6. The cost functon of the controller s K H 1 k K J a z( k) (1 a) v( k). (a) Equlbrum Pont 1 90

91 rt2 5 4 Intersecton of Sz and Sv(z) Ser R Ultmate Bounded by Set Postvely Invarant Target Set R rt1 (b) Equlbrum Pont 2 Fgure 3.4 State and Input Constrants set S z, S v(z), Postevely Invarant Set Δ, Target Set R. Fgures 3.4a and 3.4b are graphcal representaton of the set theoretc controller we used n our approach. On both graphs, the yellow set shows the ntersecton of the state and nput constrants set S z S v(z ), the red set depcts the postvely nvarant set Δ and the cyan set s the target set R. Set S z S v(z ) on fgure 3.4a s dentcal wth, thus t s not shown. As we observe n both fgures, R s subset of ( R ), whch means that the trajectory of the system s always n the neghborhood of the target set and remans there. Fgure 3.5 shows the average response tme that each controller succeeds for both applcatons. The three controllers regulate effcently the system near the frst equlbrum pont. They do not volate the target values. On the other hand, for the second equlbrum pont, the fgure hghlghts the dfference between ACRA and the rest approaches. PI and MPC controllers fal to drve the system near the equlbrum pont for dfferent reasons. Intally, the PI controller does not take nto account the addtve dsturbances and the constrants of the system. Furthermore, t mples asymptotc stablty and tres to drve the system on the equlbrum pont. Thus results ether to oscllatons lke the response of the frst applcaton or to drvng the system far from the second equlbrum pont (applcaton 2). 91

92 Fgure Average Response Tme of ACRA, PI, MPC Also MPC controller fals to stablze the system near the desred target values. Ths happens because ths controller does not consdered addtve dsturbances and does not provde any stablty guarantee. Addtonally ts performance s very senstve to the weghts of the cost functon. We conduct many trals to fnd a sutable value of trade-off parameter α and we conclude that even small changes of t have large mpact on the performance and the stablty of the controller. The percentage of tme ntervals where average response tme volates the target value for ACRA, PI and MPC are 4.16%, 45.83%, 50.83% respectvely. 92

93 Fgure 3.6 Reference and Admtted Request Rate Fgure 3.6 depcts the admtted and desred value of request rate. The graph shows that ACRA outperforms the two other approaches, snce t admts the hghest request rate, especally for the frst equlbrum pont. The average admtted request rate of the two equlbrum ponts for each approach are (314.38/202.67req/sec), (296.91/199.86req/sec), (301.65/202.66req/sec) respectvely. 93

94 (a) Average Response Tme (b) Admtted Request Rate Fgure 3.7 Performance of MPC, PI controllers usng a sngle LTI Model 94

95 Fgures 3.7a and 3.7b show the performance of PI and MPC controllers when we use only one LTI model for any workload condtons. On fgure 3.7a, the target values of the average response tme and admtted request rate are the same lke fgure 3.5. However for the frst equlbrum pont of Applcaton 2, both controllers fal to lead the system near the target value of admtted request rate as t s depcted on fgure 3.6b. Furthermore both controllers do not perform suffcently when the equlbrum pont changes. As t s llustrated on fgure 3.7a, on the second part of the experment (eq. pont 2) there are many volatons of the target SLO value, whch means that both controllers are nadequate. From these graphs, t s obvous that a unque LTI model cannot descrbe the system s dynamc behavor under any workload crcumstances. Although ACRA does not have as a prmary goal the mnmzaton of the system s power consumpton, t ams to mnmze the reserved VMs capacty, whch mples mnmzaton of the consumed energy. In the followng example, we show ACRA s performance when on the frst part of the experment the desred average response tme s 1s and the admtted request rate s 300req/s and on the second half the target values are 1s and 200req/s. Fgure 3.8 llustrates that the controller reduces 18% the utlzaton of VMs capacty and consequently the power consumpton of the system on the second part the resources demands are less than the frst one. Fgure 3.8 Average Response Tme and VMs Capacty Allocaton of ACRA 95

96 3.3 An Autonomous Admsson Control and Resource Allocaton Framework for Consoldated Web Applcatons Contrary to most exstng approaches n the ntroducton of ths chapter, an autonomous modelng and control framework [95] s presented n ths secton, whch addresses admsson control and resource allocaton smultaneously n a unfed manner. In specfc, a Lnear Parameter Varyng (LPV) state space model s adopted to capture the dynamc behavor of the underlyng nfrastructure. The operatng condtons are determned accordng to an optmzaton crteron. A feasble operatng pont, whch satsfes the desred QoS nomnal values, s computed. The resultng stablzng state feedback control law s an affne state-dependent control law that can be easly mplemented. The computatonal complexty of the controller mplementaton s small, snce the calculaton of the control elements s done offlne and at every tme nstant only a lnear program and a pont locaton problem are solved. Fnally, convergence to the feasble operatng pont and satsfacton of the system s constrants are guaranteed, for a set of desred operatng ponts of nterest. The local level controllers on Fgure 3.1 am at satsfyng the SLOs reference value, the physcal constrants and at mnmzng the power consumpton. Fgure 3.9 depcts the system s archtecture for the consoldaton of two three-ter applcatons. Typcally, web applcatons consst of many components such as web servers, applcaton servers and databases. The local controller must solve admsson control and resource allocaton as a common decson problem; t determnes the admttance c 1,..., c,1 2,3 Modelng and Control Framework p 1, p 2 VM 1,1 VM 2,1 VM 1,2 VM 1,3 VM2,2 VM 2, 3 Web Server Applcaton Server Database Fgure 3.9 System Archtecture 96

97 System Constrants Predctor ~ LPV Model Feedback Controller u Equllbrum Pont Fgure 3.10 Structural Dagram of Autonomous Framework. probablty ( p1, p2on Fgure 3.9) for every servce and the capacty (CPU) share for ther VMs ( c 1,1,, c 2,3 on Fgure 3.9). Fgure 3.10 llustrates the modelng and control framework presented n ths secton. Intally, an LPV model s obtaned n order to descrbe the systems dynamc behavor. Ths knd of modelng s sutable because t explctly takes nto account the systems parameters such as the predcton of the ncomng request rate and servce rate. Next, the exstng physcal system s constrants are quantfed. The CPU capacty of each VM vares between two lmt values and the sum of the VMs capactes n each server cannot exceed the total capacty of the server. In addton, response tme and ncomng request rate vary between two values that can be extracted precsely by extreme experments that lead servers near to ther saturaton pont. These constrants are taken nto account by the element of Fgure 3.10 whch determnes the equlbrum pont. The determnaton of the equlbrum pont s addressed as a decson problem whch depends on several objectves. Fnally, a stablzng state-feedback controller s desgned and mplemented whch takes nto account the SLOs target values, the system s constrants and the determned equlbrum pont from the prevous step. Consequently, all consoldated applcatons have a certan level of QoS for a wde range of workloads Modelng and Identfcaton Modelng the dynamc behavor of consoldated web servces on a cluster of servers s challengng, snce there are no analytcal equatons whch capture the system dynamcs. In prevous work [72], a group of LTI models that covers the range of the ncomng workload was used. Ths approach s more precse than a sngle LTI model 97

98 but t s not scalable wth respect to the workload range and the number of co-hosted applcatons. However there are scalablty and accuracy problems as the number of consoldated applcatons ncreases. In order to overcome ths ssue, we adopt an LPV model, whch changes accordng to a system parameter. The LPV model substtutes the group of LTI models and t s scalable towards the range of ncomng workload. Moreover t s proven to be more accurate. We consder n consoldated applcatons on m servers and each applcaton has one VM on every server. We select as state varables x, 1,, n, the 90 th percentle of response tme at the x n,.e., th k fxed tme nterval. The state vector at tme k s denoted by x x x. : 1 n We consder as nputs the CPU capactes c,, 1,, n and j 1,, m of each VM, where denotes the applcaton and j the server, and the admttance probabltes p, 1,, n for each applcaton. Consequently, the nput vector defned as follows, j u ( m1) n s u : c... c c... c... c... c p p (3.25) 1,1 1, m 2,1 2, m n,1 n, m 1... n The LPV model s of the followng form, x k 1 A( sk ) x k B( sk ) uk, (3.26) nn n( m1) n where As ( k ), Bs ( k ) are the system matrces and k s tme varable. The parameter vector s n of the LPV model contans the predcton values of the ncomng request rate of each applcaton, 1,, n,.e, s [ ]. k 1k 2k nk Table 3 summarzes the notaton used throughout the secton. The system (3.26) was dentfed usng the algorthm of [96]. The parameter varyng A s B s are chosen to have a lnear dependence on vector s, k, matrces,.e. k k n A s A s A, B s B s B (3.27) k 0 k n k 98

99 nn where A, 1,, n and n ( m 1) B n, 1,, n are lnear nvarant matrces, whch elements are dentfed below. In specfc, the followng LPV nput-output n m Number of consoldated applcatons. Number of actve servers. x k c, j k 90 th percentle of response tme n the CPU capacty of tme nterval j VM that belongs to th k tme nterval th applcaton on th j server n the th k p k Admttance probablty of th applcaton n the th k tme nterval s k LPV model parameter k Incomng request rate of th applcaton n the th k tme nterval k Predcton of Incomng request rate of th applcaton n the th k tme nterval x Target response tme vector ref u Equlbrum pont nput vector ref T SLA hghest acceptable response tme vector ref X U Set of state constrants Set of nput constrants Table Notaton of State, Input Varables and Parameters. (LPV-IO) representaton method s used to derve the unknown coeffcents from past measurements, x a ( s ) x a ( s ) x b ( s ) u a ( s ) u, k 1 k k1 na k kna 1 k k1 nb k knb where n, n are the number of past values of the state and the nput vector a b respectvely. The coeffcents a1 ( sk), 1,, na and b1 ( sk), 1,, nb nduce the elements of matrces A ( sk ), B ( s k ) of (3.27). In order to evaluate the extracted model, the standard Best Ft Rate (BFR) [81] score was used. In a typcal testbed descrbed analytcally n the followng evaluaton 99

100 subsecton, the dentfcaton experments show that takng nto account past values of states and nputs enhance the accuracy of the model. However, for n 3, n 3 the mprovement s neglgble. Also, as expected, t was found that LPV-RLS algorthm has sgnfcantly hgher performance than the classcal RLS algorthm [86] for LTI modelng. These results are summarzed n Table 4. a b n a 1, n 1 n 2, n 2 n 3, n 3 b a b a b LPV-RLS 76.72% 82.61% 79.37% 86.79% 80.29% 87.17% RLS 43.45% 57.04% 45.08% 59.23% 45.54% 59.64% Table Comparson of the BFR score of the LPV-RLS and RLS dentfcaton algorthms. In order to obtan a representaton of the model that allows the development of the subsequent result, the matrces, A s B s can be descrbed as the convex k combnaton of the extreme subsystems defned by the matrx pars ( Aˆ, Bˆ ), 1,,2 n, whch correspond to the extreme values of s k, n 2 1 n 2 A s m Aˆ (3.28 a) B s k 1 n 2 m Bˆ k 1 k (3.28b) n m 0, 1,,2 (3.28 c) m 1 (3.28 d) Ths dual representaton [83] corresponds to lnear dscrete tme systems wth polytopc uncertantes Request Rate Predctor In order to predct the ncomng request rate of each applcaton, Holt s lnear exponental smoothng (LES) flter s used [97], whch can capture the lnear trend n the tme seres. For example, durng tme step k, the estmated value k of ncomng request rate k for a one-step predcton horzon s obtaned as follows, ˆ b, k k k ˆ (1 )( ˆ b ), k k k 1 k 1 b ( ˆ ˆ ) (1 ) b, k k k 1 k 1 100

101 where are, smoothng constants, ˆk denotes the smoothed value for tme step k and bk represents the lnear trend n the measurement seres. For ntalzaton, we use a random value for ˆ 0 nsde the range of ncomng request rate and b State and Input Constrants The computatonal resources of a server cluster, whch hosts a group of applcatons, are nherently lmted. Moreover, there are network constrants that bound the range of response tme and ncomng request rate. The 90 th percentle of response tme x of the applcaton n any tme nterval vares from a small postve value (e.g 0.01ms) to the value when the system s saturated or the requests has expred due to network constrants. Thus, states are bounded to the constrant set X, where n { x : x x, 1,, n}. (3.29) max On the other hand, nput constrants consder the restrctons on VM capacty and admttance probablty. For each VM of applcaton on server j, the CPU capacty c, jvares from a mnmum value c mn to a maxmum c max. Addtonally, the sum of VMs enttlement should not exceed the total capacty of the server CT. The admttance probablty ranges from a mnmum value constrants form the nput constrant set U, :, 1,,, 1,,, ( m1) n u cmn c, j cmax n j m mn p to p max 1. All the above mn n c, j CTj, j 1,, m, (3.30) 1 p p p, 1,, n. We remnd that the nput vector s defned by (3.25) Determnaton of the Operatng Pont LPV model of (3.26) allows dentfyng a number of desred feasble operatng ponts. From a set of canddate equlbrum ponts we choose one that satsfes the SLA requrements and the state and nput constrants (3.29) and (3.30) respectvely. The determnaton of ths pont s a decson problem that ncludes several competng objectves, such as strct response tme target and mnmzaton of power consumpton. Thus dependng on the prorty of the goals set and selectng accordngly the correspondng cost functon, an optmzaton problem can be formulated whose soluton s the desred equlbrum pont. In partcular, we choose to guarantee the response tme reference value of each servce and maxmze the admttance probablty wthout volatng the system constrants. max j 101

102 To ths end, gven a desred response tme x ref soluton of the followng lnear optmzaton problem,, an nput vector u ref s computed by the n m n mn wjc, j (3.31a),,, ref wd d c j p ref d ref 1 j1 1 subject to ˆ ˆ n x A x B u, 1,,2, (3.31b) ref l ref l ref x x, 1,, n, (3.31c) ref max c c c, 1,, n, j 1,, m, (3.31d) mn, jref max n 1 c max, j CT, j 1,, m, (3.31e) p p d, 1,, n, (3.31f) ref j d 0, 1,, n. (3.31g) Equaton (3.31b) ensures that ( x, u ) s the equlbrum pont for all possble ref ref parameter varatons. Equatons (3.31c)-(3.31f) guarantee state and nput constrant satsfacton. The postve numbers w, 1,, n, j 1,, m and w, 1,, n are j weghts of the optmzaton cost functon and ther values correspond to the trade-off between the compettve objectves. The auxlary varables d n the cost functon and the last two equatons are used to maxmze the admttance probablty. In accordance to (3.25), t follows for the nput vector u ref that, d u c c c... c p p ref 1,1 ref 1, mref n,1 ref n, mref 1ref nref Controller Desgn In ths secton, we focus on the controller synthess and mplementaton, by addressng jontly the resource allocaton and admsson of requests. Due to the presence of state and nput constrants, apart from computng stablzng controllers for each desred operatng pont, t s mportant to characterze regons of attracton n the state-space for the closed-loop system. In other words, we are nterested n computng the set of ntal condtons n the state-space that can be drven (asymptotcally) to the desred operatng pont. Thus, the problem to be nvestgated can be formulated as follows: Gven the LPV system (3.26), a set of desred operatng ponts x, 1,,, N s determned by the method presented n the prevous subsecton and the state and nput constrants, Eq. (3.29) and (3.30), compute a state-feedback control strategy ref 102

103 n n g :, 1,, N and a doman of attracton (DoA) R closed-loop system s locally asymptotcally stable wth respect to x ref,. n, such that the Snce the ncomng workload s unpredctable and sudden changes occur, t s mportant to have dfferent operatng ponts n order to satsfy always the system constrants and to ensure that the response tme volatons, accordng to SLA, wll be as less as possble. Thus, a second problem to be nvestgated s to compute a control strategy such that the closed-loop system can change effcently between dfferent operatng ponts, wthout volatng the state and nput constrants. Assumng that a non-zero response tme vector x ref, s chosen to determne the feasble equlbrum pont, we apply the followng coordnates transformaton n order to obtan a transformed nomnal model that has ts equlbrum pont to the orgn. Thus for each equlbrum pont ( xref,, uref, ), 1,, N, computed n subsecton 3.3.4, we defne the translated system Also for the th z x x k k ref, v u u k k ref, accordngly to z, v, where k 1 k k k k,, (3.32) z A( s ) z B( s ) v. (3.33) system (3.33), the state and nput constrants are transformed n z : x z x x, 1,, n, (3.34) ref, max ref, :, 1,,, 1,,, ( m1) n z cmn c, j tc ref, j cmax c, j n j m ref mn n tc CT c, j 1,, m, (3.35), j j, jref 1 1 p p tp p p, 1,, n. ref max n ref where tc, j c, j c, j ref are the transformed VMs capacty nputs and tp p p are the transformed admsson probablty nputs. The above, j, j, j ref constrants can be rewrtten n the followng form of lnear nequaltes, where C n z z : C ( j) z 1, 1,, N, j 1,, p (3.36) v : ( ) 1, 1,,, j 1,, q (3.37) ( j) 1n and ( m 1) n v D j v N D( j) 1 ( m1) n are the rows of matrces, C D respectvely whch contan all the constrant nequaltes for each system (3.33). One well known 103

104 approach n control engneerng when tryng to estmate smultaneously the DoA and a stablzng control law s to apply Lyapunov based methods, see e.g. [83, chapter 2] and [98, chapter 4]. We consder a lnear state-feedback control law system, 1,, N g z K z N, (3.33), of the form, 1,, g for each M : z k 1 ( A( s ) k B( sk ) K j ) zk, 1,, N. ( 3.38) For each closed-loop system we consder a quadratc Lyapunov functon (LF) V, 1,, N of the form V ( z) z Pz (3.39) For each closed-loop system (3.38), we have to determne a matrx P 0 that satsfes the one-step decrease along the trajectores of the system (3.38) requrement of the LF, as well as the feedback gan K,.e., A B K P A B K 0 (3.40) To elmnate the non-lneartes n (3.40), we set and post multplyng eq. (3.40) wth Q we get Q and Y KQ and by pre 1 P 1 ( AQ BY ) Q ( AQ BY P (3.41 ) 0. ) Applyng the Schur complement [99] we get the followng equvalent form Q AQ BY ( AQ BY ) Q 0, (3.42) whch s a lnear matrx nequalty wth respect to Q, Y. Then we can compute the gan matrx optmzaton problems, K and Q, P for each system (3.38) M, 1,..., N, by solvng N convex 104

105 mn trace( Q ) Q, Y, subject to (3.43a) Q AQ BY ( AQ BY ) Q 0, (3.43b) 1 C( j) Q ( C ( j) Q ) Q 0, j 1,, p, (3.43c) 1 D( j) Y ( D ( j) Y ) Q 0, j 1,, q, (3.43d) Q 0, (3. 43e) 0 1. (3.43f) In the above problem, nequaltes (3.43b) and (3.43e) ensure local asympotc stablty for the closed-loop system. Inequaltes (3.43c) and (3.43d) guarantee state and nput constrant satsfacton. Parameter s a measure of the speed of convergence of the closed loop system to the equlbrum pont. The problem s convex and can be solved usng off-the-shelf software, e.g the Matlab Robust Control Toolbox [100] or the SDPT 3.0 software toolbox [101]. The nterested reader s referred to, e.g., [99], [102, Appendx A], for further detals. The Lyapunov matrx and the gan matrx for each operatng pont are fnally computed as P 1 Q and 1 K YQ respectvely. Together wth the controller, the set n S { z : z Pz 1} z s an admssble DoA for the closed loop system,.e. t s the set whch contans all ntal condtons that can be transferred asymptotcally to each equlbrum pont wthout volatng the constrants. Transformng back to the ntal state space, the doman of attracton for ths system s R { x :( x x ) P( x x ) 1}. (3.44) n T ref, ref, Fnally the control law for the ntal system (3.26) s gven by, Snce (3.39) s a LF, Ths property of the set the desred operatng pont xref, u K ( x x ) u k k ref, ref, R s a contractve for the closed-loop system (3.38) [83]. R ensures that for any x(0) R the system wll be drven to wthout volatng the nput and state constrants. Fgure 3.11 llustrates the domans of attracton of each operatng pont. For example the DoA of xref [4 2.5] s hghlghted. If any pont nsde the blue ellpsod s the ntal state then an admssble control law exsts such that the system trajectory s 105

106 transferred to the equlbrum pont xref [4 2.5]. It s worth notng that the computaton of the feedback gan and the doman of attracton R are performed offlne usng the dual representaton of (3.27). Fgure DoAs R of each operatng pont x ref,i, =1,,N. From the set of desred operatng ponts x ref,, 1,.., N, there s one operatng pont that s closest to the SLA requrements. Accordng to the SLA, for each consoldated applcaton there s a hghest acceptable response tme T *, 1,,, n. Thus, an accepted operatng area s mplctly shaped on the state space where the state varables should reman for all tme ntervals. The desred operaton pont s chosen to be the center of ths area snce then the correspondng DoA wll nclude the largest part of the accepted operatng area. We denote ths desred operaton pont as ( x, u ), ts correspondng DoA * * ref ref applcaton, we select as desred operaton value ref * R and feedback gan matrx K *. For each * * xref, Tref, / 2, 1,, n and the desred operatng pont of the system s formulated as x * [ x * x * ]. Every ref ref,1 ref, n * tme step k the response tme vector x k s measured. If ths value s not n the set R, then we have to choose the closest operatng pont that contans t accordng to the followng Eucldean dstance metrc, 106

107 mn x x (3.46a) xref, ref, subject to * ref x R, 1,.., N (3.46b) k whch s a standard pont locaton problem. There can be alternatve methods of selectng the closest operatng pont, e.g. the operatng pont that ts DoA has the largest overlap wth the DoA of the desred operatng pont ( x, u ). * * ref ref As t s shown n Fgure 3.11, there s always an overlap between the domans of attractons of each operatng pont. We can beneft from ths overlap and ntally select the closest set * R to R that contans the current response tme vector n the state space area accordng to (3.46) and after few tme steps the system trajectory we * * wll be drven to the desred operatng pont ( x, u ). The followng algorthms summarze all the necessary offlne steps for the determnaton of the set of feasble operatng ponts and the correspondng control laws and regons of attracton(algorthm 1) and the onlne mplementaton of the control strategy (Algorthm 2), Algorthm Offlne Controller Synthess 1: for all x, 1,, ref, N do 2: compute ( xref,, uref, ), 1,, N by solvng (3.31) 3: compute, n (3.34), (3.35) 4: compute R, K, P by solvng (3.43), ( xref,, uref, ), 1,, N 5: end for 6: Select ( x, u ), R, K * * * * ref ref ref ref Algorthm Onlne Controller Implementaton 1: f xk * R then 2: u K ( x x ) u k * * * k ref ref 3: else 4: Fnd xref,, u ref, by solvng (3.46) u K ( x x ) u 5: k k ref, ref, 6: end f 7: x x 1 A( s ) x B( s ) u k k k k k k 8: Go to lne 1 107

108 Remark A modfcaton to the offlne controller synthess and the onlne controller mplementaton must be made for the case where no feasble nput vector uref,, jexsts for a desred x ref,,.e., the problem (3.31) has no soluton. Frstly for each extreme system ( A ˆ, B ˆ ), j 1,,2 n, problem (3.31) s solved n order j j to compute feasble nput vectors uref,,, 1,,2 n j j for the desred equlbrum pont x ref,. Next, for all extreme realzatons ( A ˆ, B ˆ ), the translated nput constrants j, j, j 1,,2 n are formulated as (3.35). We consder as the common nput constrant set cn, the translated space to be the ntersecton of each nput constrant set,.e., n 2, c, j j1 the feedback gan matrces problem to (3.43). j. Then for each operatng pont Lyapunov matrx P, K and the DoA R are computed solvng a smlar Regardng the onlne controller mplementaton, a feasble nput vector uref, ( s k ) has to be computed at each step for the desred operatng pont xref, ( s ) xref,. In detal, the followng lnear programmng problem s solved at each nstant k. k n m n mn wjc, j wd d (3.47a),,,, c j p k d k uref, ( sk ) 1 j1 1 subject to x A( s ) x B( s ) u, (3.47b) ref, ( sk ) k ref, ( sk ) k ref, ( sk ) c c c, 1,, n, j 1,, m, (3.47c) mn, jk max n c, j CT, 1,, k j j m, (3.47d) 1 p p d, 1,, n, (3.47e) max k d 0, 1,, n, (3.47f) u, u K ( x x ) u. (3.47g) k k k ref, ( sk) ref, ( sk) The postve numbers w, 1,, n, j 1,, mand w, 1,, n, are weghts of the j optmzaton cost functon. Equaton (3.47b) ensures that ( xref, ( s ), uref, ( s )) s the equlbrum pont. Equatons (3.47c)-(3.47e) ensure state and nput constrant satsfacton. Equatons (3.47d) and (3.47e) are used to maxmze the admttance probablty. Fnally equaton (3.47g) guarantees that the control law uk satsfes the nput constrants at every tme nterval. 108 d k k

109 The above procedure allows computng a stablzaton control law and a DoA that leads the system trajectory on the operatng pont. It s worth notng that for ths case, the nput constrant satsfacton s not guaranteed by the offlne synthess as before. However, f the optmzaton problem (3.47) s feasble, the nput constrants are guaranteed to hold durng the onlne mplementaton. Fgure Tested Dagram VM,j of th applcaton on j th server Evaluaton We have bult a real testbed n order to evaluate the performance of our system modelng and control framework. It ncludes a group of servers and each servce conssts of an applcaton and a database ter. Each ter s hosted on a VM as shown n Fgure The CPU capacty of database ter s fxed and only the capacty of the applcaton ter VMs changes dynamcally. We used real, hghly varant and nonstatonary traces from [103] to create the ncomng workload of every applcaton. These traces are stll used by recent studes [104], [105] snce they are more realstc than benchmarks lke TPC-W [106] and RUBs [107] that have statonary request generators. We demonstrate an experment where the upper desred reference 90 th 109

110 percentle value of response tme, defned by SLA, s Tref 2sec, whch mples that the target value of the equlbrum pont s x * ref 1sec. We assume the predcton of the ncomng request rate ( k) as the LPV model parameter s k. Fgures 3.13a-3.13d llustrate the performance of the proposed framework. Indcatvely we present the performance of servces App1 and App2 whch VMs share the frst server n the applcaton ter of fgure Fgure 3.13a shows the 90 th percentle value of response tme for both applcatons. The percentage of tme * ntervals that the response tme s less than the target value x ref (blue lne), s 98.78%. Also there are very few tme ntervals whose response tme volates the reference value T ref (green lne). (a) Response Tme 110

111 (b) VMs Capacty (c) Admttance Probablty 111

112 (d) Incomng Request Rate Fgure Overall Performance of Consoldated Applcatons. Fgures 3.13b and 3.13c ndcate that t s essental to tackle resource allocaton and admsson control as a common decson problem. When the ncomng workload s low the VM capacty s also low and t ncreases as the ncomng load ncreases. On the other hand, admsson probablty s close to one when the workload s lght whereas t decreases f more requests arrve. Fgure 3.13d llustrates that although the ncomng request rate s dynamc and unpredctable and the exsted predctor effcently estmates the future values of workload. Fgure 3.14 llustrates a short example of the control strategy presented n ths artcle. The red dots depct dfferent operatng ponts ( xref,, u ref, ) and the red ellpsods the * * * correspondng DoAs R and the blue dot and lne correspond to the target xref, uref, R. The black sold lne corresponds to the system trajectory whch begns from x0 [ ] far away from the desred operatng pont, x * ref 1. Applyng Algorthm 3.2, the controller choose the control law and by solvng the pont locaton problem (3.46) the response tme vector enters the acceptable regon (blue lne) on state-space, x( k) T, after few steps. Ths proves that the proposed control scheme ref s adaptve and able to react f there s any sudden change of the ncomng request rate. 112

113 Fgure DoA of operatng ponts (red lnes and crcles), DoA of the desred operatng pont (blue lne and square) and System Trajectory (black sold lne). Fgure 3.15 Alternatve soluton of pont locaton problem. Fgure 3.15 shows an alternatve method of selectng the equlbrum pont dependng on the current operatng condton of the system. Ths method uses also the Eucldan dstance but t s more conservatve because t selects teratvely the same operatng pont untl t reaches ts center very closely and then t chooses a new one. We compare the proposed soluton wth study [74], whch also uses LPV modelng and MPC control wthout guaranteeng the feasblty of the operatng pont. We select ths partcular study because s closest to our control perspectve and t uses popular and a well-establshed control method. Ths study also provdes a trade-off 113

114 between response tme restrctons and energy consumpton adjustng the parameter K H 1 to the cost functon J x( k) (1 ) u( k). We used the same ncomng workload n order to compare the two approaches and fgures 3.16a 3.16c show ther performance. It can be seen that our method outperforms MPC controller. In partcular, the MPC scheme results n a large percentage of response volatons and t consumes more capacty resources and rejects a sgnfcant porton of the ncomng requests. Table 3.5 summarzes the performance of the two control solutons. MPC controller does not perform very well because of the lack of a feasble equlbrum pont that wll assure system stablty. Also t s very senstve to the values of parameter of the cost functon. Ths results n the nput oscllatons n fgure 3.16b and 3.16c. kk Accepted RT (%) Capacty (%) Probablty (%) AC+RA MPC Table Comparson of the performance of AC+RA and MPC control scheme. (a) Response Tme 114

115 (b) VMs Capacty (c) Admttance Probablty Fgure Comparson wth MPC Controller 115