Dynamic Routing Algorithms for Cloud-Ready Elastic ...

Wrocław University of Science and Technology Faculty of Electronics Department of Systems and Computer Networks

Dynamic Routing Algorithms for Cloud-Ready Elastic Optical Networks Algorytmy dynamicznego wyznaczania przepływów w elastycznych sieciach optycznych zorientowanych na rozwiązania chmurowe

PhD Thesis MICHAŁ AIBIN

Supervisor: Krzysztof Walkowiak Prof., D.Sc.

Wrocław, March 2017

Through hardships to the stars / Przez trudy do gwiazd

Wrocław University of Science and Technology Faculty of Electronics Department of Systems and Computer Networks

Copyright c – All rights reserved. Michał Aibin. This copy of the thesis has been supplied on condition that anyone who consults it is understood to recognize that its copyright rests with its author and that no information derived from the thesis may be published without quotation. Therefore, limited reproduction of this work for the purposes of private study, research, criticism, review and news reporting is likely to be in accordance with the law, particularly if cited appropriately. (Signature)

.......................... Michał Aibin

Abstract The growing popularity of streaming services, cloud computing and mobile applications, increases Internet traffic volume year by year. The inflation of data transfer is inevitable and brings significant challenges for transport networks. In addition, a significant issue is the predominant role of cloud computing and content-oriented services provided by geographically dispersed data centers. To meet the increasing requirements, new solution has to be proposed. Elastic Optical Network is a promising technology that provides flexible modulation format conversion, which allows more efficient use of spectrum resources compared to classical fix-grid in WDM networks. This dissertation focuses on dynamic routing algorithms for resource allocation in cloudready Elastic Optical Networks. The performance of the algorithms is evaluated using the concept of Software Defined Networks and Cross Stratum Optimization architecture for a provider-centric use case. The performance of single stratum algorithms using only one optical or data center stratum is compared to cross stratum techniques that simultaneously considers cost, distance, and data center utilization. The alternative algorithm that employs Monte Carlo Tree Search for traffic prediction is also incorporated. As a result of the work done in this dissertation, a framework is proposed in order to solve the problem of cloud resource allocation in optical networks. The framework is composed of three mechanisms: dynamic routing algorithms, data center allocation policies, and pricing models of cloud data center services. The performance of the proposed solution is investigated using computer simulations in two well-known network topologies, Euro28 and US26. The data center locations and the pricing models are provided by Amazon Web Services, but the solution is easily adaptable to other cloud operators. An incremental approach is used to compose the complete solution, and thus, provides a better understanding of it. First, the dynamic routing algorithms are compared. Then, the data center assignment policies are investigated. Simulation results show that the solution is able to meet the assumed requirements and successfully reduce the number of required network resources by increasing the optical spectrum utilization efficiency.

Keywords Software Defined Networks, Elastic Optical Networks, data center services, dynamic routing, traffic prediction, machine learning, wide area networks, algorithms

I

Streszczenie Rosnąca popularność usług strumieniowych, rozwiązań chmurowych oraz aplikacji mobilnych, znacząco zwiększa ilość przesyłanych danych w ruchu internetowym. Wzrost transferu danych jest nieunikniony i przynosi wyzwania dla sieci komputerowych. Ponadto, w ostatnim czasie rośnie zapotrzebowanie na usługi bazujące na rozwiązaniach chmurowych, dostarczane przez rozproszone geograficznie centra danych. Aby sprostać rosnącym wymaganiom, potrzebne jest nowe rozwiązanie. Elastyczne Sieci Optyczne to obiecująca technologia, która umożliwia bardziej efektywne wykorzystanie zasobów częstotliwości w porównaniu do klasycznych sieci WDM. Niniejsza rozprawa doktorska jest poświęcona algorytmom dynamicznego wyznaczania przepływów w elastycznych sieciach optycznych zorientowanych na rozwiązania chmurowe. Działanie algorytmów jest oceniane poprzez użycie koncepcji sieci definiowanych programowo oraz architekturę optymalizacji wielowarstwowej. Wydajność algorytmów działających w jednej warstwie, optycznej lub centrów danych, jest porównywana z algorytmami działającymi w trybie optymalizacji wielowarstwowej, które jednocześnie bierze pod uwagę koszt, odległość i utylizację zasobów w centrach danych. Dodatkowo, algorytm wykorzystujący Monte-Carlo Tree Search jest zaimplementowany w celu umożliwienia predykcji ruchu. W efekcie, zaproponowane jest rozwiązanie problemu przydziału zasobów w sieciach zorientowanych na usługi chmurowe. Składa się ono z trzech części: algorytmów trasowania dynamicznego, zasad przydziału centrów danych i modeli cenowych operatorów sieciowych. Wydajność proponowanego rozwiązania jest weryfikowana za pomocą symulacji komputerowych w dwóch znanych topologiach sieciowych - Euro28 i US26. Lokalizacje centrów danych i ceny usług dostarczane są przez Amazon Web Services, ale rozwiązania zaproponowane w tej pracy doktorskiej jest łatwo zastosować u innych operatorów. Aby skomponować kompletne rozwiązanie, a tym samym zapewnić lepsze jego zrozumienie, poniższa praca doktorska jest pisana w trybie przyrostowym. Po pierwsze, algorytmy dynamicznego trasowania są porównywane. Następnie badana jest polityka wyboru centrów danych. Wyniki symulacji pokazują że zaproponowane rozwiązanie spełnia założone wymagania i skutecznie zmniejsza liczbę wymaganych zasobów sieciowych, zwiększając efektywność wykorzystania widma optycznego.

Słowa kluczowe sieci definiowane programowo, elastyczne sieci optyczne, rozwiązania chmurowe, trasowanie dynamiczne, predykcja ruchu, uczenie maszynowe, sieci rozległe, algorytmy

III

to Ania and Julian, to my family and friends

Acknowledgements I am using this opportunity to express my gratitude to everyone who supported me throughout my PhD studies. I am thankful for your aspiring guidance, invaluably constructive criticism and friendly advices during the project work. I would like to express my sincere gratitude to my supervisor Prof. Krzysztof Walkowiak. Thank you very much for your support and understanding over these past four years. Your guidance helped me in all the time of research and writing of this thesis. I also wish to express my sincere thanks to Department of Systems and Computer Networks, Faculty of Electronics, Wrocław University of Science and Technology for providing me with all the necessary facilities for the research. In 2015, I went to Simon Fraser University to study with Prof. Ljiljana Trajković and her team. My time there has been highly productive and working with you was an extraordinary experience. Much of the analysis presented in Chapter III is owed to my time there. Most importantly, none of this could have happened without my family. I would like to thank my parents and grandparents for their unconditional love and support throughout my life. Your support enabled me to study and pursue my life goals. I would like also to thank my two wonderful angels, Sophie and Hawana, for being my mental rest sanctuary, and most importantly, my soul mates. I would like to extend my gratitude to my friends Bartosz Ornatowski, Tomasz Wolak oraz Damian Godlewski who have helped me and provided support in countless ways. Last but not least, I am especially thankful to my wife Ania and my son Julian for their patience and love. You both deserve my deepest appreciation. Ania, you were with me through my whole PhD journey, and without you, I would not have made it this far.

Wrocław, March 2017 Michał Aibin

VII

Contents

Abstract

I

Streszczenie

III

Acknowledgements

VII

Contents

IX

List of Figures

XIII

List of Tables

XVII

Abbreviations

XIX

Symbols

XXI

I

Introduction and Background

1 Introduction

3

1.1

Scope and Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

5

1.2

Publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

6

1.3

Structure of the Dissertation . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2 Trends in Cloud-Ready Networking

II

1

11

2.1

Cloud Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.2

Content-Oriented Networking . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.3

Optical Networks Evolution . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.4

Elastic Optical Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.5

Software Defined Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.6

Cross Stratum Optimization Architecture . . . . . . . . . . . . . . . . . . . 18

Optimization Problem

3 Cross Stratum Optimization Problem

21 23

3.1

Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

3.2

Network Flows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 IX

CONTENTS

3.3

3.4

III

3.2.1

Unicast Flows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.2.2

Anycast Flows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

3.2.3

Multicast Flows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Problem Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 3.3.1

Routing Modulation and Spectrum Assignment . . . . . . . . . . . . 30

3.3.2

Network Survivability . . . . . . . . . . . . . . . . . . . . . . . . . . 32

Related Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

Algorithms

37

4 Algorithms 4.1

Dynamic Routing Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . 39 4.1.1

Unicast Flows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

4.1.2

Anycast Flows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

4.1.3

Multicast Flows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

4.2

Regenerator Placement Problem

4.3

Background Traffic Generation . . . . . . . . . . . . . . . . . . . . . . . . . 55

4.4

4.5

IV

39

. . . . . . . . . . . . . . . . . . . . . . . . 53

4.3.1

Average Spectrum Width Optimization . . . . . . . . . . . . . . . . . 55

4.3.2

Vector Simulated Annealing Algorithm . . . . . . . . . . . . . . . . . 56

4.3.3

Double Window Defragmentation Algorithm . . . . . . . . . . . . . . 59

Data Center Assignment Policies . . . . . . . . . . . . . . . . . . . . . . . . 61 4.4.1

Nearest Data Center Assignment . . . . . . . . . . . . . . . . . . . . 61

4.4.2

Cheapest Data Center Assignment . . . . . . . . . . . . . . . . . . . 62

4.4.3

Least Utilized Data Center Assignment . . . . . . . . . . . . . . . . 62

4.4.4

Hybrid Data Center Assignment . . . . . . . . . . . . . . . . . . . . 63

4.4.5

Traffic Prediction with Monte Carlo Tree Search . . . . . . . . . . . 64

Path Protection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

Results and Simulation Setup

5 Performance Assessment 5.1

5.2

X

71 73

Simulation Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 5.1.1

Network Topologies

. . . . . . . . . . . . . . . . . . . . . . . . . . . 73

5.1.2

EON assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

5.1.3

Request Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

5.1.4

Data Center Locations . . . . . . . . . . . . . . . . . . . . . . . . . . 76

Simulations Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 5.2.1

Simulator Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . 77

5.2.2

Simulator External Libraries

. . . . . . . . . . . . . . . . . . . . . . 78

CONTENTS

6 Results 6.1 Dynamic RMSA Problem in EONs . . . . 6.2 Regenerator Placement Problem in EONs 6.3 Background Traffic Optimization . . . . . 6.4 Data Center Assignment . . . . . . . . . . 6.5 Path Protection . . . . . . . . . . . . . . .

V

Summary and Future Research

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

81 81 88 93 97 104

109

7 Conclusion and Summary 111 7.1 Achievements and Contributions . . . . . . . . . . . . . . . . . . . . . . . . 112 7.2 Future Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 Bibliography

133

XI

List of Figures

2.1

Comparison of a) WDM and b) EON technologies . . . . . . . . . . . . . . 14

2.2

Spectrum allocation: a) WDM networks, b) EON (single slice approach), c) EON (double-sided half slice approach) . . . . . . . . . . . . . . . . . . . 15

3.1

Unicast transmission - one-to-one . . . . . . . . . . . . . . . . . . . . . . 25

3.2

Anycast transmission - one-to-one-of-many

3.3

Multicast transmission - one-to-many . . . . . . . . . . . . . . . . . . . . 27

3.4

Cross Stratum Optimization architecture for Data Center services provisioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3.5

RMSA example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.6

Network protection/restoration schemes . . . . . . . . . . . . . . . . . . . . 33

4.1

Input graph (left) and computed graph for modulation M1 (right) . . . . . 43

4.2

Computed graphs for modulation M2 (left) and modulation M3 (right) . . 43

4.3

Minimum composite graph (left) and expanded graph (right) . . . . . . . . 43

4.4

Initial state of the network . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

4.5

Network after combining links between nodes with regenerators

4.6

Final result of the RMSA-AMRA-UNI algorithm execution . . . . . . . . . 45

4.7

Example of creating multicast tree in RMSA problem in EON . . . . . . . 51

4.8

Initial state of the lightpath, before the defragmentation

4.9

Suitable window for defragmentation (blue color) . . . . . . . . . . . . . . . 59

4.10

Lightpath state after defragmentation . . . . . . . . . . . . . . . . . . . . . 59

4.11

Example of a search tree for a network with two DCs and three candidate paths for each DC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

5.1

The Euro28 network topology . . . . . . . . . . . . . . . . . . . . . . . . . 74

5.2

The US26 network topology . . . . . . . . . . . . . . . . . . . . . . . . . . 74

5.3

All AWS DC locations

5.4

CEONS GUI – view of the network resources usage . . . . . . . . . . . . . 77

6.1

Total BBP of algorithms in dynamic RMSA problem using Euro28 network 82

6.2

Lack of spectrum in dynamic RMSA problem using Euro28 network . . . . 83

6.3

Lack of regenerators in dynamic RMSA problem using Euro28 network . . 83

6.4

Total BBP of algorithms in dynamic RMSA problem using US26 network . 84

6.5

Lack of spectrum in dynamic RMSA problem using US26 network . . . . . 84

6.6

Lack of regenerators for dynamic RMSA problem using US26 network . . . 85

. . . . . . . . . . . . . . . . 26

. . . . . . 44

. . . . . . . . . . 59

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

XIII

LIST OF FIGURES

6.7 6.8 6.9 6.10 6.11 6.12 6.13 6.14 6.15 6.16 6.17 6.18 6.19 6.20 6.21 6.22 6.23 6.24 6.25 6.26 6.27 6.28 6.29 6.30 6.31 6.32 6.33

XIV

Average usage of regenerators in Euro28 network . . . . . . . . . . . . . . . Average usage of regenerators in US26 network . . . . . . . . . . . . . . . . RMSA-AMRA BBP gain with modulation format change over the version without modulation format change in Euro28 network . . . . . . . . . . . . RMSA-AMRA BBP gain with modulation format change over the version without modulation format change in US26 network . . . . . . . . . . . . . Trade-Off Between BBP and Average Number of Regenerators using RMSAAMRA in Euro28 network . . . . . . . . . . . . . . . . . . . . . . . . . . . Trade-Off Between BBP and Average Number of Regenerators using RMSAAMRA in US26 network . . . . . . . . . . . . . . . . . . . . . . . . . . . . Improvement in BBP of RMSA-AMRA in Euro28 network using various regenerator placement algorithms in comparison to the RP-FR algorithm . Improvement in BBP of RMSA-AMRA in US26 network using various regenerator placement algorithms in comparison to the RP-FR algorithm . . The reason of BBP for low traffic loads in Euro28 network . . . . . . . . . The reason of BBP for low traffic loads in US26 network . . . . . . . . . . The reason of BBP for moderate traffic loads in Euro28 network . . . . . . The reason of BBP for moderate traffic loads in US26 network . . . . . . . The reason of BBP for high traffic loads in Euro28 network . . . . . . . . . The reason of BBP for high traffic loads in US26 network . . . . . . . . . . Difference between RMSA-STATIC-VSA and optimal results . . . . . . . . Number of cases in which RMSA-STATIC-VSA was better than FF, MSF and LSF algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . RMSA-DEF-DWD gains in BBP metric for Euro28 network . . . . . . . . RMSA-DEF-DWD gains in BBP metric for US26 network . . . . . . . . . Blocking Percentage of the DC allocation algorithms using the Euro28 network topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . DC resource utilization using different DC allocation algorithms and the Euro28 network topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . Optical resource utilization using different DC allocation algorithms and the Euro28 network topology . . . . . . . . . . . . . . . . . . . . . . . . . . Cost per hour of service with different DC allocation algorithms and the Euro28 network topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . Blocking Percentage of the DC allocation algorithms using the US26 network topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . DC resource utilization using different DC allocation algorithms and the US26 network topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Optical resource utilization using different DC allocation algorithms and the US26 network topology . . . . . . . . . . . . . . . . . . . . . . . . . . . Cost per hour of service with different DC allocation algorithms and the US26 network topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Blocking Percentage of the DC allocation algorithms using the Euro28 network topology, with 25% initial network load . . . . . . . . . . . . . . . . .

85 86 86 87 87 88 89 90 91 91 92 92 93 93 94 95 96 97 99 100 100 101 101 102 102 103 104

LIST OF FIGURES

6.34 6.35 6.36 6.37 6.38

Blocking Percentage of the DC allocation algorithms using the US26 network topology, with 25% initial network load . . . . . . . . . . . . . . . . . Blocking Percentage of the DC allocation algorithms using the Euro28 network topology and various protection methods . . . . . . . . . . . . . . . . Blocking Percentage of the DC allocation algorithms using the US26 network topology and various protection methods . . . . . . . . . . . . . . . . Cost per hour of service with different DC allocation algorithms and the Euro28 network topology, using various protection methods . . . . . . . . . Cost per hour of service with different DC allocation algorithms and the US26 network topology, using various protection methods . . . . . . . . . .

104 105 106 106 107

XV

List of Tables

3.1 3.2

Example of spectrum usage and transmission reach for the 400 Gb/s request and different modulation formats . . . . . . . . . . . . . . . . . . . . . . . . 31 Main routing choices for the example situation from Figure 3.5 . . . . . . . 32

4.1

Penalties for using different modulation formats . . . . . . . . . . . . . . . 45

5.1

Prices per hour of DC services, provided by the AWS. . . . . . . . . . . . . 75

6.1 6.2 6.3 6.4

Input parameters for dynamic routing Tuning scenarios . . . . . . . . . . . . Tuning of the number of iterations . . Input parameters for static requests .

algorithms comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

81 94 95 96

XVII

Abbreviations AWS BBP BP CAPEX CDN CEONS CON CRUD CSO DAT DC DPP EON ISP NDO OaaS OFDM OFP OPEX OSNR RSA RMSA RWA SBPP SPP SDEONS SDN SD-OTN WAN WDM

Amazon Web Services Bandwidth Blocking Percentage Blocking Percentage Capital Expenditures Content Delivery Network Complex Elastic Optical Network Simulator Content Oriented Network Create Read Update Delete Cross Stratum Optimization Distance Adaptive Transmission Data Center Dedicated Path Protection Elastic Optical Network Internet Service Provider Name Data Objects Optical as a Service Optical-orthogonal Frequency Division Multiplexing OpenFlow Protocol Operational Expenditures Optical to Signal Noise Ratio Routing and Spectrum Assignment Routing, Modulation and Spectrum Assignment Routing and Wavelength Assignment Shared Backup Path Protection Squeezed Path Protection Software Defined Elastic Optical Networks Software Defined Network Software Defined Optical Transport Nodes Wide Area Network Wavelength Division Multiplexing

XIX

Symbols

a A⇣ b barr B c(d) c(ddown ) c(dup ) d D dany dcpu dmulti dram dstorage duni dist(m) e E EX G = (V , E) i imax j k l(p) l(e) L LP C m M n(c(d), m) narr nregs nregsv

a search action (Monte Carlo Tree Search) a set of available actions at a given state ⇣ (Monte Carlo Tree Search) an index of slice a set of slice indices from each path segment pseg a set of slices a capacity of a request d (bit-rate) a downstream capacity of an anycast request d (bit-rate) an upstream capacity of an anycast request d (bit-rate) an index of a request a set of requests an anycast request a cpu request (Data Center Assignment) a multicast request a RAM memory request (Data Center Assignment) a storage request (Data Center Assignment) an unicast request a maximum distance that modulation m can support an index of an optical network link a set of optical network links an exploration constant (Monte Carlo Tree Search) a directed graph, representing the network an iteration number (Simulated Annealing) a maximum iteration number (Simulated Annealing) a cooling rate (Simulated Annealing) a number of candidate paths a length of a candidate path p a length of an fiber link e a set of link lengths a large positive constant (Monte Carlo Tree Search) an index of modulation format a set of modulation formats a number of slices required to serve a request of capacity c(d) using m a set of number of used slices on each path segment pseg a number of available regenerators in the network a number of available regenerators in the network node v XXI

Symbols

p(d) P (d) pseg r rcpu rram rstorage R reg s(d) t(d) T v V Vr Vreg z

◆  ⇡

✓ ⇣

XXII

an index of candidate routing path for request d a set of candidate routing paths for request d a segment of an lightpath between two nodes with regenerators an index of data center node a cpu utilization metric (Data Center Assignment) a RAM memory utilization metric (Data Center Assignment) a storage utilization metric (Data Center Assignment) a set of data center nodes a number of regenerators a source node of request d a destination node of request d a starting temperature (Simulated Annealing) an index of a network node a set of network nodes a set of network nodes with data center a set of network nodes with regenerators a multiply factor (Simulated Annealing) a computational budget (Monte Carlo Tree Search) the best decision in search (Monte Carlo Tree Search) the frequency of link failures (Path Protection) a reward (Monte Carlo Tree Search) a decision value (Monte Carlo Tree Search) an action policy (Monte Carlo Tree Search) a search node index (Monte Carlo Tree Search) a set of all search nodes (Monte Carlo Tree Search) a visit count (Monte Carlo Tree Search) a visit count of a current node in a search tree (Monte Carlo Tree Search) a search state (Monte Carlo Tree Search)

Part I Introduction and Background

1

Chapter

1

Introduction

O

ver the past few years, the Internet consolidated itself as a very powerful platform. It has changed our means of communication and has given a "globalized" dimension to the world. We can observe the impact of the progression of social media in our life. Web services such as Facebook, Google+, etc., break records in the number of users. For example, on August 27 2015, a billion users of Facebook were online, which represents 1/7 of the world’s population; shortly after, What’s App announced that it has 900 million active users. The first widely available measurement of Internet users from December 1995, revealed 16 million registered people (0,4% of world population). Since then, the number of people connected to a global network has multiplied more than 200 times. In June 2016, there were over 3400 million Internet users [1]. In addition, the growing popularity of streaming services, Content Delivery Networks (CDNs), cloud computing and mobile applications, increases Internet traffic volume per year by approximately 34% [2], [3]. These services generate data flows of 10 Gb/s up to terabit level. Currently, the inflation of data transfer is inevitable and brings significant challenges for transport networks. To meet the increasing requirements, new technology has to be deployed in backbone networks [4]. Elastic Optical Networks (EONs), based on optical-orthogonal frequency division multiplexing (OFDM), has been able to accommodate such high bandwidth demand applications. The purpose for developing EONs in current networks is mainly due to the growing popularity of Web services such as cloud computing, CDN etc. These systems can process a large amount of data, therefore requiring new types of transmissions available in EONs. The basic concept of EON is to fragment the whole available spectral resources into tight, width-constant spectral slices (optical channels represented in frequency domain) that correspond to different optical wavelengths. The slices are allocated over a whole optical path, by source, destination and volume of a requests [5]. In EONs, it is possible to use different, distance-adaptive modulation formats, for instance BPSK, QPSK, and m-QAM, where m belongs to 8, 16, 32, 64. The selection of a particular modulation format is made according to a lightpath characteristic (e.g., path OSNR, path length, bitrate). Moreover, the modulation formats can use different spectral efficiency (expressed in [bit/s/Hz]) [6]. It should be noted that with a larger value of the spectral efficiency, less resources of spectrum are required to provision a particular lightpath. Nevertheless, a higher value of an optical signal to noise ratio (OSNR) is required to detect the transmission. This significantly reduces the transmission reach of the modulation format that 3

triggers the need to use regenerators and increases the capital and operational expenditure (CAPEX/OPEX) network cost, as well as power consumption. By breaking the fixed-grid spectrum allocation limit of conventional Wavelength Division Multiplexing (WDM) networks, EONs increase the flexibility in the connection provisioning. As a matter of fact, the available Routing and Wavelength Assignment (RWA) proposals for WDM networks are no longer directly applicable in EONs. In EONs, routing concerns establishment of lightpaths for individual clients requests. The process is accompanied by solving the problem of Routing and Spectrum Allocation (RSA), which concerns finding a routing path and a contiguous segment of spectrum subject to the constraint of no frequency overlapping in network links. If a modulation format is to be determined, the problem is called Routing, Modulation and Spectrum Allocation (RMSA) [7]. In this dissertation, I consider dynamic routing. In dynamic routing the route from the source to the destination is chosen dynamically, depending on the network state. A lightpath is set up for each request as it arrives, and then it is released after some finite amount of time. The network state is determined by all requests that are currently in progress, thus, the network has to accommodate incoming traffic in real time [8]. The main difference between dynamic and static routing is the time of decision - the dynamic routing tables are not known for the entire network but are complemented on a regular basis, dependent on current resources available; a necessity required when a decision is to be made in a matter of milliseconds. Upon tear down of connections, their allocated spectral resources are released, and could be assigned to new requests. In a dynamic traffic scenario, the arrival and departure process of the requests in the network leads to the fragmentation of the spectral resources. Therefore, in medium and high loaded network scenarios, the possibility of finding sufficient contiguous spectrum to establish a connection over such a fragmented spectrum can be very low, leading to request blocking despite having enough, but non-contiguous, spectrum resources. In light of this, there is an increasing demand to optimize algorithms used in RMSA problems, especially using methods, that could adapt their decisions to the current network state. Moreover, new transmission methods could improve network performance. Traditional unicast is being replaced by more efficient flows such as anycast and multicast. Anycasting – defined as one-to-one-of-many transmission – is a very effective way to serve network services provisioned in data centers (DCs) including popular cloud computing services. The anycast request is defined by a client node, the set of admissible DC nodes and upstream and downstream bandwidth requirement [9]. A multicast request requires a point-to-multipoint connection from a source node to multiple destination nodes [10]. In this dissertation I will consider above-mentioned transmission methods and compare their performance in cloud-ready networks. The infrastructure of those consists of DCs interconnected via optical networks. Data centers are associated with selected network nodes. Through the use of EON architecture and its elastic and on-demand bandwidth connectivity, we are able to build an effective transport network, which is essential for new services. 4

1.1

Scope and Thesis

This dissertation proposes algorithms for the problem of RMSA in dynamic routing in cloud-ready EONs. The two main objectives of this thesis are: to minimize Bandwidth Blocking Percentage (BBP) in optical layer, defined as the volume of rejected traffic divided by the volume of all traffic offered to the network; and to minimize the total request Blocking Percentage (BP) in data center layer, which is calculated as the percentage of rejected requests for data center resources. Furthermore, I introduce a new adaptive and regenerator-aware solution for dynamic RMSA that facilitates the possibility of modulation change along a lightpath. Numerical experiments clearly show advantage of my approach compared to previous methods in order to achieve the best trade-off between the CAPEX/OPEX network cost and to obtain low BP. In addition, a significant issue is the predominant role of cloud computing and content-oriented services provided by geographically dispersed data centers. Anycast and multicast transmissions gain much attention due to the need for cost-effective and scalable data delivery in inter-data center networks. To meet requirement of the growing traffic needs, I propose algorithms for unicast, anycast and multicast network flows. Many data center services require lower delay and higher availability with end-to-end guaranteed QoS, which involves both application and transport network resources. However, in the current mode of operation, the control of EONs and DCs is separately deployed. Based on the benefits of Software Defined Networks (SDNs), Cross Stratum Optimization (CSO) allows global optimization and control across EONs and data center application stratum heterogeneous resources to meet the QoS. This architecture is intended to enable optical as a service (OaaS), realizing both spectrum and application elasticity. OaaS is proposed and designed to provide the optical network resources directly for the data centers with considering both data center and network resources, which can be implemented with SDN. In this thesis I propose the algorithms to meet the requirements of that architecture to optimize the network resources usage under dynamic traffic scenarios. Therefore, the following thesis of this dissertation is: Cloud-ready and cross-stratum provisioning strategies enable optical as a service and increase the optical spectrum utilization efficiency, thereby reducing the number of required network resources. The problem of cloud computing and content-oriented services provisioning throughout optical networks is an emerging topic for network operators and cloud providers. The proposed solution is carefully placed in the cloud architecture assuming the coexistence of the network and DC locations. Additionally, the simplicity and the quick processing times of this solution permits the use of the algorithms in real-life situations. In order to prove the research hypothesis, the following goals are formulated: • design and implement algorithms for the dynamic routing with joint unicast, anycast and multicast flows; 5

• implement the CSO architecture in data center networks to check the efficiency of cross stratum algorithms in comparison to single stratum ones; • develop adaptive service provisioning strategies using machine learning techniques that enable traffic prediction.

1.2

Publications

Some of the achievements presented in this dissertation, as well as general considerations related to the topic, have been published in journal articles and conference papers. All the papers underwent a thorough review process before their final publication. The list of relevant publications is (in chronological order): • M. Aibin, K. Walkowiak, “Dynamic Routing of Anycast and Unicast Traffic in Elastic Optical Networks with Various Modulation Formats - Trade-Off Between Blocking Probability and Network Cost” in IEEE High Performance Routing and Switching. Vancouver, Canada, June 2014 [11], Web Of Science; • M. Aibin, K. Walkowiak, “Simulated Annealing Algorithm for Optimization of Elastic Optical Networks with Unicast and Anycast Traffic” in International Conference on Transparent Optical Networks, pp. 1-4, Graz, Austria, July 2014 [12], Web Of Science; • M. Aibin, K. Walkowiak, „Adaptive Modulation and Regenerator-Aware Dynamic Routing Algorithm in Elastic Optical Networks” in IEEE International Conference on Communications (ICC) (pp. 5138 – 5143), London, UK, 2015 [7], Web Of Science; • M. Aibin, K. Walkowiak, “Regenerator Placement Algorithms for Cloud-Ready Elastic Optical Networks” in International Conference on Transparent Optical Networks, pp. 5-8, Budapest, Hungary, July 2015 [13], Web Of Science; • M. Aibin, M. Błażejewski, “Complex Elastic Optical Network Simulator (CEONS)” in International Conference on Transparent Optical Networks, pp. 1 - 4, Budapest, Hungary, July 2015 [14], Web Of Science; • M. Aibin, K. Walkowiak, „Adaptive Survivability Algorithm for Path Protection with Various Traffic Classes in Elastic Optical Networks” in 7th International Workshop on Resilient Networks Design and Modeling (RNDM), pp. 56 - 62, Munich, Germany, October 2015 [15], Web Of Science; • M. Aibin, K. Walkowiak, „Defragmentation Algorithm for Joint Dynamic and Static Routing Problems in Elastic Optical Networks with Unicast and Anycast Traffic” in International Conference on Computing, Networking and Communications (ICNC), Workshop on Computing, Networking and Communications (CNC), pp. 350 - 354, Kauai, Hawaii, USA, February 2016 [16], Web Of Science; 6

• M. Aibin, K. Walkowiak, „Resource Requirements in Fixed-Grid and Flex-Grid Networks for Dynamic Provisioning of Data Center Traffic” in IEEE Canadian Conference on Electrical and Computer Engineering (CCECE), pp. 1 – 4, Vancouver, Canada, May 2016 [17], Web Of Science; • M. Aibin, R. Goścień, K. Walkowiak, “Multicasting versus Anycasting: How to Efficiently Deliver Content in Elastic Optical Networks” in International Conference on Transparent Optical Networks, pp. 1 - 4, Trento, Italy, July 2016 [18], Web Of Science; • M. Furdek, L. Wosinska, R. Goścień, K. Manousakis, M. Aibin, K. Walkowiak, S. Ristov, M. Gushev, J. L. Marzo, „An Overview of Security Challenges in Communication Networks” in 8th International Workshop on Resilient Networks Design and Modeling (RNDM), pp. 43 - 50, Halmstad, Sweden, September 2016 [19], Web Of Science; • M. Aibin, K. Walkowiak, „Different Strategies for Dynamic Multicast Traffic Protection in Elastic Optical Networks” in 8th International Workshop on Resilient Networks Design and Modeling (RNDM), pp. 174 - 180, Halmstad, Sweden, September 2016 [20], Web Of Science; • M. Aibin, K. Walkowiak, A. Sen, „Software-Defined Adaptive Survivability for Elastic Optical Networks” in Optical Switching and Networking 23, pp. 85 – 96, Elsevier Science, January 2017 [21], Impact Factor: 1.071; • M. Przewoźniczek, K. Walkowiak, M. Aibin, „The Evolutionary Cost of Baldwin Effect in the Routing and Spectrum Allocation Problem in Elastic Optical Networks” in Applied Soft Computing, Volume 52, pp. 843 – 862, Elsevier Science, March 2017 [22], Impact Factor: 2.81; • M. Przewoźniczek, K. Walkowiak, M. Aibin, „The Effectiveness of the Simplicity in Evolutionary Computation” in 9th Asian Conference on Intelligent Information and Database Systems, pp. 1 – 4, Kanazawa, Japan, April 2017 [23]; • M. Aibin, S. Haeri, K. Walkowiak, L. Trajkovic, „Cross-Stratum Optimization for Provisioning of Data Center Services”, in a review [24]. The main contribution of [11] is to find a trade-off between the Bandwidth Blocking Probability and using extra regenerators in the context of dynamic routing of anycast and unicast traffic in EONs. A higher modulation format consumes less spectrum, which can lead to a smaller BBP in dynamic routing. However, at the same time, a higher modulation format provides a smaller transmission distance, which triggers the need to use regenerators and increases CAPEX/OPEX network costs as well as power consumption. The publication [12] focuses on joint optimization of unicast and anycast traffic in EONs including both Routing and Spectrum Allocation (RSA). A novel heuristic algorithm, based on the Simulated Annealing, is proposed. To evaluate proposed approach, a wide range of experiments on real network topologies is conducted. 7

The article [7] focuses on a Routing, Modulation and Spectrum Allocation (RMSA) problem in the context of dynamic routing for anycast and unicast traffic in EONs. A new adaptive and regenerator-aware algorithm for dynamic RMSA that facilitates the possibility of modulation change along the lightpath was introduced. Numerical experiments clearly show advantage of this approach comparing to previous methods in terms of blocking percentage. Moreover, results show an interesting trade-off between blocking percentage and the usage of regenerators. In the work [13] problem of regenerator placement in cloud-ready Elastic Optical Networks (EONs) is discussed. The proposed algorithms include information on DCs location and ratio of anycast traffic in the network to address the fact that cloud computing services are provisioned by DCs with the use of anycasting. Evaluation of proposed algorithms is made in the context of dynamic traffic scenario with the objective to minimize blocking percentage experienced in a network with regenerator location yielded by a particular placement algorithm. Results of experiments show that the cloud-aware location of regenerators can significantly reduce the blocking percentage as well as reduce the number of used regenerators comparing to classical approaches. The publication [14] presents the features of an advanced software simulator for the EONs – Complex Elastic Optical Networks Simulator (CEONS). The CEONS is as a test environment for solving main issues addressed in EONs: Routing, Modulation and Spectrum Assignment (RMSA) problems and addressing regenerator placement problem. With a universal API, users can design their own algorithms in any programming language and implement them in the simulator as plugins to solve the problems mentioned above. The ease and versatility of implementing the algorithms allows usage of the CEONS in advanced problems and for the introduction of EONs technology. The main contribution of [15] is a new adaptive algorithm for dynamic routing in survivable EON. The key novelty of this approach is the possibility to handle various traffic classes in a different way in terms of the required protection level. This new method was extensively tested using representative network topologies with realistic assumptions concerning the network traffic and operation of the EON. The obtained results clearly indicate that the this algorithm significantly outperforms classical protection methods including Dedicated Path Protection and Shared Backup Path Protection in terms of two performance metrics, namely, bandwidth blocking probability and ratio of traffic lost (not restored) due to network failures. In the work [16] algorithm that improves the network performance under dynamic traffic scenarios in terms of blocking probability and usage of regenerators is proposed. An important novelty of this approach is that the defragmentation algorithm is applied for both dynamic and static traffic allocated in the network before the arrival of new dynamic requests. The article [17] presents the minimum requirements for Wavelength Division Multiplexing and Elastic Optical Networks resources in order to achieve the best trade-off between the network cost and obtaining the various thresholds of Service Level Agreement. Forecasted traffic volumes for the next few years are considered to perform an improved estimation of the required network resources. 8

The publication [18] focuses on a problem related to joint optimization of unicast, anycast, and multicast traffic. Several heuristic algorithms including greedy methods and metaheuristics such as Tabu Search and Simulated Annealing are proposed. Moreover, experiments on representative topologies with realistic EON assumptions are performed. In the work [19] deliberate human-made attacks aimed at gaining unauthorized access to the network or disrupting the service are discussed. In order to develop methods for increasing network security in the presence of attacks, it is crucial to first identify the security vulnerabilities and attack methods that exploit them, as well as the capabilities and shortcomings of existing security schemes. In this article comprehensive overview of attack methods and security approaches from the literature is presented. This overview covers the security vulnerabilities inherent to the underlying physical layer, the implications of software-defined networking to security, and security challenges in cloud networks spanning geographically distributed data centers. The article [20] concentrates on dynamic multicast routing in survivable EONs. Various protection methods that may be applied to protect multicast requests in EONs are compared, using two dynamic routing algorithms with additional survivability constraints and the possibility to change the modulation format at the regeneration nodes. The results clearly show that various QoS levels for multicast protection allow to save optical network resources, thus, accept more incoming traffic in the network. The publication [21] focuses on survivability problems in dynamic routing in EONs. An Adaptive Survivability (AS) approach is proposed to achieve the best trade-off between the efficiency of path protection and cost of routing. The main conclusion is that using AS approach results in significant improvement of network performance. The main objective of the work [22] is to investigate the pros and cons of hybridization on the base of a hard practical and up-to-date problem, namely the Routing and Spectrum Allocation of Multicast Flows (RSA/M) in Elastic Optical Networks (EONs). The second objective of this paper is to propose an effective optimization method for solving the RSA/M problem in EONs. The article [23] concentrates on a situation in which the simplification of the method leads to the improvement of its effectiveness. The presented results show situations when primitive methods, like Random Search (RS), can compete with highly sophisticated and highly effective methods. They were obtained for an up-to-date, practical, NP-complete problem, namely the Routing and Spectrum Allocation of Multicast and Unicast Flows (RSA/MU) in Elastic Optical Networks (EONs). In the work [24] the evaluation of resource allocation algorithms for data center networks is performed using the concept of Software Defined Networks and Cross-Stratum Optimization architecture for a provider-centric use case. Results indicate that algorithms that jointly optimize two strata improve network performance. The proposed algorithms enable network and data center operators to more efficiently utilize resources. 9

1.3

Structure of the Dissertation

This dissertation is divided into five parts. The first part provides a theoretical background for the research conducted in this thesis. Chapter 1 is a general introduction which states the thesis. Chapter 2 describes current trends and technologies in the networks. The second part begins with Chapter 3, which defines Cross Stratum Optimization problem for dynamic routing in EONs, for all types of flows. It introduces the theoretical knowledge about optimization of network flows as well as it presents work related to dynamic routing in cloud-ready data center networks. The third part focuses on algorithms for the dynamic Cross Stratum Optimization in EONs. This approach lowers the delay and increases availability of the DCs resources with the end-to-end guaranteed Quality of Service (QoS). In this part, the proposed algorithms are also compared to algorithms available in the literature, then is followed by the results of the extensive simulations, presented in part four and summary of the research presented in part five. Please note that I use "I" whenever the text speaks about my own decisions and choices I made and is the default voice. I use "we", whenever the discourse is explanatory, such as an exposition of a proof. Therein, "we" stands for "me and the reader". Some of the achievements presented in this dissertation were separately funded by the Polish National Science Centre (NCN) under Grant 2016/21/N/ST7/02147 and Grant DEC-2012/07/B/ST7/01215, statutory funds of the Department of Systems and Computer Networks, Wroclaw University of Science and Technology and the European Commission under the 7th Framework Programme, Coordination and Support Action, Grant Agreement Number 316097, ENGINE - European research centre of Network intelliGence for INnovation Enhancement and by the COST Action CA15127 supported by European Cooperation in Science and Technology.

10

Chapter

2

Trends in Cloud-Ready Networking

T

he increasing demand for highly scalable and efficient distribution of Internet content has motivated the development of future Internet architectures. This chapter introduces cloud and content oriented services, presents the evolution of optical networks, and describes the concept of Software Defined Networking. Moreover, the most important research in the topics of cloud services and dynamic routing is discussed.

2.1

Cloud Services

Cloud services is a recent trend that moves IT services and data away from desktop and portable PCs into large data centers. Cloud services, defined by [25] are: A large-scale distributed computing paradigm that is driven by economies of scale, in which a pool of abstracted, virtualized, dynamically-scalable, managed computing power, storage, platforms, and services are delivered on demand to external customers over the Internet. To achieve the cloud paradigm, we need to ensure: • Scalability: The most fundamental feature of scalable systems is the ability of a computer application or product to continue to function well when it is changed in size or volume in order to meet a user need [26]. • Encapsulation: The system has the ability to serve as an abstract entity that delivers different levels of services to customers outside the cloud [27]. • On-demand configuration: Cloud services need to be easy to configure (via virtualization or other approaches) and delivered on demand [28]. The most popular enterprise cloud solutions are: Amazon Web Services (AWS) [29], which provides a broad set of infrastructure available in seconds, with pay-as-you-go pricing; Google App Engine [30], which is a platform that acts as an application container; Microsoft One Drive [31], which is a storage infrastructure. Moreover, we observe a rapid increase in cloud end user solutions. The commonly used applications are mainly responsible for cloud storage, for example Dropbox, Google Drive, Western Digital Cloud. In this dissertation, I discuss the approach of Cross Stratum Optimization, which is based on cloud-ready EONs and cloud data centers infrastructure, with pricing provided 11

by AWS. The algorithms shown in this dissertation are universal, and might be used for other cloud data center solutions.

2.2

Content-Oriented Networking

Nowadays, network architectures are based on named data objects (NDOs), for example: web pages, videos, documents, or other pieces of information. The approach of these architectures is commonly called content oriented networking (CON) [32]–[35]. The benefits of the content-oriented communication model include: the possibility to trade storage space for lower average latency by caching content locally; support for multicast and anycast transmission that removes the sender-side bottleneck for popular data thus scaling flash crowds. Also, it is cost-effective for Internet Service Providers (ISPs) to cache a data object instead of using global transit to download it if the item is requested frequently enough. People value the Internet for its content, not where it is from. To enable CON, the following constraints must be met: • Availability: Fast, reliable content access requires application-specific mechanisms. Data and services should have high availability, in terms of both reliability and lowlatency. Availability is usually provided by replication at endpoints, and the network’s role is to allow user requests to find nearby copies. The first large-scale solution to this was deployed by Akamai, using intelligent DNS servers and URL rewriting. More recently, services like Netflix have become more and more popular [36]. • Security: A vital factor is trust in online content. To obtain high security level, we need to authenticate the connections. Today, authenticity is typically achieved by securing the channel to the source, rather than explicitly authenticating the data [37]. • Location-independence: There should be no equivalent of “broken links” when data is moved to another site. Therefore, HTTP redirect and dynamic DNS are used to minimize this problem, but are not sufficient. To solve the issue, we can map the content to data center locations, which enables high persistence, as long as the underlying data or service is available [38]. The concept of CON has not yet been settled in all details. Many technical aspects including the general service model, the architecture, and protocols that will be standardized, are still discussed. A popular solution used in the Internet as a step towards the full realization of the CON is named Content Delivery Networks (CDNs). According to [39], CDN is defined in the following way: A content delivery network represents a group of geographically dispersed servers deployed to facilitate the distribution of information generated by Web publishers in a timely and efficient manner. In concrete, CDN replicates a very selective set of content to the replica servers and only sends those requests for the replicated content to a replica server. The key questions in designing an effective CDN are: 12

• How to place replica servers and distribute content copies to replica servers? • How to route the requests to the proper replica server having the desired content are the key challenges in designing an effective CDN? The set of content stored in particular CDN servers should be selected carefully to approach the hit ratio of 100%, what this means is that almost all requests to replicated servers are fulfilled [40].

2.3

Optical Networks Evolution

Over the last decade, optical networks have gone through a rapid evolution, starting with 16 wavelengths of 2.5 Gb/s in late 1990s to 80 wavelengths of 100 Gb/s in 2012 [41], [42]. Today, the term optical networks denote high-capacity telecommunications networks based on optical technologies and components that can provide capacity, provisioning, routing, grooming, and/or restoration at the wavelength level. With estimated exponential traffic growths, future networks have to boost their capacity. The channel capacity will need to be increased beyond 100 Gb/s per channel or higher, with an increase of spectral efficiency. Currently used optical technology networks are Wavelength Division Multiplexing (WDM). The main idea underlying the concept of WDM networks is to communicate end-users in optical layer via all-optical WDM channels, which are referred to as lightpaths [43]. A connection in a wavelength-routed WDM network is supported by a lightpath which may span multiple fiber links. In addition, when there are no wavelength converters, a lightpath must occupy the same wavelength on all the fiber links through which it traverses due to the wavelength-continuity constraint. Despite all benefits of conventional WDM networks, their biggest problem is a low bandwidth efficiency due to a fixed granularity [44]. Full wavelength capacity has to be dedicated for establishing a connection between end-nodes, even when the traffic between the nodes does not need to fill the full spectrum. Fixed-grid causes the poor utilization of the residual bandwidth of wavelength. Flexible-grid technologies can alleviate these limitations for demand provisioning. These technologies represent promising candidates for future optical networks supporting transmissions of 100 Gb/s and beyond [45]. In this case, several wavelengths are aggregated and allocated according to the request. On the other hand, it is easy to perform segmentation of large requests in flexible-grid networks (see Figure 2.1). Aiming to break the fixed-grid spectrum allocation limit of conventional WDM networks, a novel spectrum efficient and scalable optical transport network architecture, called Elastic Optical Networks (EONs) is introduced. Depending on the traffic volume, an appropriate sized optical spectrum is allocated to connections in EONs. In this way, incoming requests can be served in a spectrum-efficient manner. In view of this, next section deeply outlines the general characteristics of EONs as a promising solution for future high-speed and content oriented optical transport.

13

Figure 2.1: Comparison of a) WDM and b) EON technologies [5]

2.4

Elastic Optical Networks

The idea underlying the concept of Elastic Optical Networks is to allocate appropriatesized, optical bandwidth to an end-to-end optical path. It is different than in fixed-sized optical bandwidth allocation in WDM. The allocation is performed according to the traffic volume in a highly spectrum-efficient and scalable manner. Unlike the rigid bandwidth in WDM, an optical path in EONs expands according to the traffic volume [46]–[48]. The current ITU-T frequency grid specified in G.694.1 [49] equals to 193.1 THz, and supports various channel spacing of 12.5 GHz, 25 GHz, 50 GHz, and 100 GHz as shown in 2.2. The most popular channel spacing used by the network providers in current WDM networks is 50 GHz which supports transmission of 100 Gb/s. The use of smaller frequency windows (25 GHz, 12.5 GHz) would decrease the transmission speed, due to the impossibility of aggregating requests (e.g. a request with a capacity of 100 Gb/s would not fit on the spectrum width of the 12.5 GHz, so it will not be allocated). On the other hand, the use of larger frequency windows, for example 100 GHz, would result in even greater losses due to inefficient use of a spectrum (for example, the demand of 20 Gb/s capacity would occupy the entire 100 GHz spectrum). New services like Content Delivery Networks and cloud computing, trigger the need to provision connections of 400 Gb/s or even 1 Tb/s [50], which cannot be directly implemented in WDM systems. A novel spectral resource specification scheme will efficiently accommodate mixed rate traffic volumes. It should be independent of the signal format and future-proof for various new possible data rates. Finally, it is intuitive for network designers and operators. The concept of a frequency slice instead of the current frequency grid meets all these criteria. In this approach the spectral resource of an optical path can be allocated by assigning the 14

Figure 2.2: Spectrum allocation: a) WDM networks, b) EON (single slice approach), c) EON (double-sided half slice approach) [6] necessary number of frequency slices. In 2.2 we can see two possible candidates for the flexible spectrum resource designation scheme. First of them is called single slice on the grid [6]. In this approach the designated frequency f on the ITU-T frequency grid using a channel spacing fcs and a frequency number fn (corresponding to the slice number) should be interpreted as a spectral slice having a frequency segment between 193.1 + (fn ± 12 )fcs THz. This single slice on the grid approach is intuitive, but it fails in certain scenarios. The center frequency of the frequency segment of this approach deviates from wider channel spacing frequency grids by 12 fcs which disables the possibility to express the frequency segment designated by wider channel spacing frequency grids. A second approach, called double-sided half slice [6], specifies a frequency f on the ITUT frequency grid, using a channel spacing fcs and a frequency number fn , interpreted as two spectral slices at both sides of the frequency f . In this approach the slice width of each frequency is 12 fcs . The frequency segment with various widths can be represented in the same way as the single slice on the grid approach. The half-slice represents the frequency segment designated by wider channel spacing frequency grids better than the segment compatible with the ITU-T G.694.1, due to the half width of the slot [6]. Moreover, the concept of superchannels [51], which have a bandwidth of more than one 50 GHz channels, e.g., 100 GHz is a big step towards the beyond 100 Gb/s bit-rate. In order to implement slice technology, the following conditions must be met: spectrum contiguity, spectrum continuity and slice opacity. To fulfill the spectrum contiguity constraint only one consistent set of slices can be allocated per demand (for example if a 15

demand requires four slices – a set of slices with numbers 1-4 can be used but not two sets of slices with numbers 1-2 and 5-6). To meet spectrum continuity, a chosen set of slices have to be used over an entire end-to-end optical path. Finally, slice opacity constraint can be met when each slice is allocated only once in a single link (in other words it is used only to one demand) [52]. Another key benefit of EONs is the ability to change the modulation format in the nodes with regenerators [42]. Note that when changing the modulation format, we keep the principle of the central spectrum frequency. For example, if the modulation occupies eight slices (1-8) and we change it to the one that will occupy only four slices, the slices with indices 3-6 will be chosen. The possibility to change modulation format allows to treat each segment of the lightpath independently, utilizing the optical resources much more efficiently, than in the situation that one modulation format is fixed for the entire lightpath. When we have a lot of available spectrum in particular lightpath segments, we promote low modulation formats. On the other hand, when we possess a small amount of free spectrum, we promote high modulation formats to allocate requests more effectively in smaller slots, exposing a lack of available regenerators (high modulations cause a shorter transmission distance). Introducing the slices technology in optical networks allows for an adaptive allocation of spectrum resources which besides improving spectral efficiency, gives an opportunity to increase reliability by low-effort implementation of backup optical paths and detours. In case of a failure, even in a situation when there are no available resources on backup paths, allocated demands can be shrink and be reallocated in order to transfer high priority data. The EONs technology introduces a new degree of freedom in optical networking. This emerging technology is still in its infancy. At first sight, the existing WDM installations may still look sufficient enough to cover traffic needs in the nearby future. However, with the expected traffic growth, the operators will end up either losing money, since they are not able to route all traffic, or they will be migrating into a next-generation optical network (presumably EON) [53]. In light of this, it would be a wise decision to follow up the gradual migration of technology towards Elastic Optical Networks. The main advantages of using EONs are: • a highly-efficient architecture for transparent optical transmission [54]; • improvements of spectral efficiency up to 170% comparing to current solutions [6]; • a possibility to increase the disaster resistance of a network by introducing new path protection methods [55]–[59]; • Time-Dependent Bandwidth Sharing – possibility to prioritize the demands according to business hours [60]; • an option to increase electrical efficiency [61]–[63]. More benefits will be presented in the next sections as a result of using novel techniques of data transmission and EONs technology. 16

2.5

Software Defined Networks

The idea of “programmable networks” has been proposed as a way to facilitate network evolution. In particular, Software Defined Networking (SDN) is a new networking paradigm in which the control plane and data plane are centralized, decoupled from forwarding hardware. SDN not only reduces the complexity seen in today’s networks, but also helps cloud service providers host millions of virtual networks without the need for common separation isolation methods such as VLAN. SDN also enables network administrators to manage network services from a central management tool by virtualizing physical network connectivity into logical network connectivity. It allows to simplify network management and enable innovation and evolution [64]–[68]. SDN architecture consists of three separate strata. These are: management, control and data planes. An SDN infrastructure, similarly to a traditional network, is composed of a set of networking equipment such as switches, routers etc. The main difference resides in the fact that those traditional physical devices are now simple forwarding elements without embedded control or software to make autonomous decisions. In SDN, network intelligence is logically centralized into the control plane, which serves as a "brain" for the whole network [69]. In order to achieve the centralized logic, companies are using software-based controllers, which are responsible for routing and management decisions. The data plane becomes a simple packet forwarding that may be programmed via an open interface, for example OpenFlow [70]. Traditional switches use Spanning Tree or newer standards to determine how to forward packets. OpenFlow moves the forwarding decision from individual switches to a controller, typically a server or workstation. Forwarding instructions are based on a flow, which consists of all packets sharing a common set of characteristics. Upon a packet arrival at an OpenFlow switch, packet header fields are extracted and matched against the matching fields portion of the flow table entries. A large variety of parameters can be specified to define a flow. Possible criteria include the switch port where the packet arrived, the source Ethernet port, source IP port, VLAN tag, destination Ethernet or IP port, and a number of other packet characteristics [71]. The controller specifies to the switch the set of parameters that define each flow and how packets that match the flow should be processed. Using all the features of SDN architecture benefits network operators in many ways. The biggest advantage of adopting SDN is that it provides the ability to model its physical networking environment into software, which, in turn, helps in reducing the overall CAPEX and OPEX. Here are some specific benefits SDN may offer: • centralized network provisioning which accelerate service delivery and provide more agility in provisioning both virtual and physical network devices from a central location [72]; • network flexibility and holistic management by allowing companies to set up new applications and virtual machines on demand to accommodate new processing requests such as those for big data [73]; 17

• more granular security and specific trust policies [74]; • lower OPEX costs [75]; • lower CAPEX costs due to hardware savings and virtual services [75]; • a cloud ready environment which allows to managed massive data center platforms from a single SDN controller [76]; • guaranteed content delivery, possibility to define multiple QoS rules and the ability to shape and control data traffic is one of the primary advantages of software defined networking [66].

2.6

Cross Stratum Optimization Architecture

In this chapter architecture that is used to implement and deliver cloud computing services is presented. The main purpose of cloud computing is to offer a large variety of different services in an on-demand fashion [77]. The traffic is unpredictable, that is why cloud architecture must be investigated under a dynamic traffic scenario. Decisions of basic CRUD operations (create, read, update, delete) are to be made in a matter of milliseconds. In this dissertation I want to achieve Cross Stratum Optimization (CSO). I focus on two network strata – optical, where EON technology is applied to provide optical connections and DC stratum. Both strata are connected to SDN controller, based on OpenFlow protocol [71]. As a result, I achieve Software Defined Elastic Optical Networks (SDEONs) architecture. This architecture enables optical as a service (OaaS) realizing both spectrum and application elasticity [78]. OaaS is proposed and designed to provide the optical network resources directly for the data centers with considering both data center and network resources, which can be implemented with SDN. By an application elasticity I mean the scalability and flexibility requirements of provisioning the DC requests, generated by the clients. Those requirements have been described in [79]. The distributed data center networks are interconnected with EON, that contain deployed DC (CPU, memory and storage) and network stratum resources. In CSO, DC requests are provisioned. DC requests are client requests of particular DCs, carried out by anycast flows. Their objective is to provide access to cloud services, such as requests made for computing units, memory units or storage space. Each stratum resource is controlled by OpenFlow protocol (OFP), that is deployed in SDN controller, in a unified manner [80]. To control the multi stratum networks, OpenFlow enabled switch nodes with OFP agent software are required, which are referred to as Software Defined Optical Transport Nodes (SD-OTN) [81]. There are two major benefits of such architecture: firstly, SDEON can emphasize the cooperation between network strata to realize CSO with joint and global interworking of cross stratum resources [82]; secondly, using Distance Adaptive Transmission (DAT) [6], EONs can adjust the physical layer parameters (e.g., bandwidth allocation and modulation format) to optimize resource utilization on the routing path calculated by SDN controller [83]. 18

The success of DCs has revolutionized the way various network services are delivered to end-users while bringing new optimization challenges. In particular, the key goal is to achieve cost-effective and highly scalable implementations and deliver DC services with QoS guarantees using backbone optical networks. The strong competition among telecommunication companies in the global market has further influenced the need to develop appropriate and powerful approaches for these new services. To answer the increase of traffic demands, the concept of CSO has recently gained much attention [78]. The key approach is to improve performance of various network systems using a tighter coordination among network strata. CSO may allow global optimization and control across the optical transport network and the DC resources in order to meet the expected QoS requirements and to reduce the CAPEX and OPEX associated with provisioning of DC services. Current Wide Area Networks (WANs) based on optical transport networks are often not optimized for DC services that are highly dynamic and characterized by large bit-rate requirements. The network control plane is focused only on network resources and does not account for availability of DC resources. Moreover, in order to provision a new lightpath for a DC request, manual operations are required, which precludes the establishment of dynamic connection requests. These drawbacks may be mitigated by employing the SDN concept. SDN solutions allow to decouple the control and data planes, centralize network intelligence, and abstract underlying network infrastructure from the cloud applications [71]. Furthermore, SDN enables deployment of a centralized and programmable network control and management functions thus enabling efficient orchestration of network and DC resources as well as the application of various optimization techniques to control the network [84]. Centralized network control becomes more important in hybrid and complex architectures such as CSO. The SDN controller logically centralizes the network intelligence and enables joint optimization of resources in two separate domains: the optical transport network (optical spectrum, network devices) and the DC (CPUs, memory, storage). Hence, enabling the network operators to coordinate DC service provisioning with the available optical network resources such as establishing lightpaths required to deliver the data between clients and DCs. As a consequence, SDN provides a great flexibility for operators and significantly improves the overall system performance expressed by CAPEX/OPEX, throughput, responsiveness, and scalability [71]. Introducing the SDN concept in wide area networks is an emerging approach, as SDN was first proposed in local networks. The main idea of proposed network model is an evaluation of a specific provider-centric use case for control and provisioning of cloud services. I assume that the SDN controller jointly orchestrates the optical and DC resources. This enables application of advanced optimization techniques for cloud services and provisioning of dynamic lightpaths.

19

Part II Optimization Problem

21

Chapter

3

Cross Stratum Optimization Problem

O

ver the past few years, a trend has emerged among many organizations, both large and small, to use cloud computing facilities. One of the main reasons of using those facilities is the CAPEX of creating or upgrading operating sites. The use of cloud resources introduces the following issues: identification of DC base locations, carrying out network routing, and identifying locations for backups. In this chapter I describe the problem of Cross Stratum Optimization and I introduce notation and request specification.

3.1

Notation

The following notation is used. The physical network is modeled as graph G(V , E, B, L) where V denotes a set of nodes (vertices), E is a set of fiber links (directed edges), each fiber link can accommodate |B| frequency slices at most, and L = [l(1), l(2), ..., l(|E|)] represents link lengths for each e 2 E. An assumption is made that there are |R| data centers already located at some nodes of the network. For the sake of simplicity, I assume that the network node to which the DC is connected to is equivalent with this node, what means that I do not consider the physical connection between the server and the backbone network node. A set of |D| traffic requests is created dynamically, during the simulations, accordingly to the distribution, presented in Chapter 5.1.3. The set contains multicast, unicast, and anycast requests. Furthermore, I assume that all DCs can provide the same requested service/content. Each request d between the client nodes may be unicast or multicast type, whereas requests to and from DCs are anycast type, described in Chapter 3.2. Finally, each anycast request may be assigned to any of the DCs. It is assumed that various modulation formats may be used in the EONs. Let M denote a set of available modulation formats. For each modulation format m 2 M we are given constant dist(m) that denotes the maximum distance that a particular modulation may support. Moreover, let n(c(d), m) denote the number of slices required to serve request of bit rate c(d) using modulation m. Without loss of generality I assume that with the higher modulation format, the higher spectral efficiency is achieved, leading to lower spectrum demand, at the cost of shorter transmission distance (limited by dist(m)). Since regenerators in optical networks are costly, I assume that the selection of a modulation format is made in order to minimize the number of regenerators placed in 23

the network. The spectrum requirement for a particular request is determined according to a distance-adaptive transmission (DAT) rule [6]. In a nutshell, EON supports multiple modulation formats, e.g., BPSK, QPSK, and x-QAM, where x belongs to 8, 16, 32, 64. A modulation format for a particular request is selected using the transmission distance and the bit-rate of the request, where the path length is defined as the maximum distance from the source node to any receiver. For this purpose, all available modulation formats are examined. The most spectrally efficient modulation format is chosen from all modulations. In addition the transmission distance cannot exceed the chosen modulation transmission range. Consequently, additional regenerators (which are very expensive) are not required in the network, while the spectrum consumption is kept on a low level. The number of all regenerators in the network is defined by variable nregs, whereas number of required regenerators for the request d allocated on path p, using n slices, starting from the index b is defined by reg. The path distance l(p) and maximum range dist(m) of each modulation m 2 M is compared to find the assignment that minimizes the number of regenerators and maximizes the efficiency of the modulation format, accordingly to the transmission model presented in [85].

3.2

Network Flows

The flow modeling and optimization in communication networks is a well-known problem. Many groups of researchers are developing efficient optimization models and methods to meet the increasing needs of end-users [86], [87]. The main goal of networks is to send information, therefore, data flows between network nodes have to be optimized. A multicommodity flow is commonly used to model different types of network flows [86]. A commodity, in other words request or demand, is simply defined by a source and destination node, and a bit-rate. In order to establish flows, one need to find routes to allocate requests in the network. Therefore, along with the problem of flow optimization, there is a problem of selection of candidate routing paths. Two key questions that arise in candidate path selection are how many paths are needed and how to find these paths. This problem refers to the graph theory [88]–[90]. In a simple graph, the candidate path denotes a graph or sub-graph whose edges connect source and destination vertices. In a computer network the edges are links, whereas the vertices are nodes. The number and the quality of the candidate paths dictate the performance of routing algorithms. In the ideal scenario, we should be able to find a good candidate paths in a reasonable time, so it is desirable to minimize the number of paths used for routing. It will significantly reduce the overhead associated with establishing, maintaining and tearing down of connections in the network as well as reduce the complexity of the scheme that distributes traffic among multiple paths. In general, a path can be categorized as good if its inclusion in the candidate path set decreases the overall blocking percentage considerably [91]. Therefore it is desirable to select the trade-off between minimizing the number of candidate paths and the minimization of the network traffic congestion. The most popular algorithms of path finding are: Dijkstra [92], Ford-Fulkerson [93] and A* [94]. The comprehensive study on path finding algorithms is also available at [95]. 24

Optical data transmission is usually indicated as the most appropriate for providing various services due to its high throughput. On one hand, the data center operators need much efficient solutions to sustain the increase of network traffic due to cloud computing. Optical interconnection networks can provide higher throughput while consuming less power than other solutions [96]–[98]. Moreover, the evolution of optical networks leads to the concept of optical clouds (optical networks realizing next-generation cloud solutions) [99], [100]. Taking into account the needs of emerging network services and recent advances in optical networking, it is expected that optical transport networks will evolve toward more flexible and functional architectures. Therefore, in this section, various optical transmission methods are introduced and analyzed. Three types of network flows, namely, unicast, anycast and multicast are investigated. The simplest and at the same time the most popular type of a network transmission is an unicast transmission. It is the basic approach to send data in computer and communication networks widely applied in many services and applications. It is not an efficient tool for new, highly data-consuming services. This may be significant, since revenues from simple data transmission are decreasing. In contrast, anycast and multicast flows are relatively new approaches that have been recently gaining a lot of attention.

3.2.1

Unicast Flows

Figure 3.1: Unicast transmission - one-to-one Unicast is the simplest form of communication, and at the same time the most popular one. This is a one to one communication, in which there is one sender and one receiver. Nowadays, where content distribution is dispersed, it is a very ineffective form of transmission, for example, when we need to send the same video content to various users. The key thing in terms of cost and scalability is that there are few economies of scale. Each client of a unicast transmission takes up additional bandwidth. For example, if eight customers wants to download exactly the same video streams, then eight disjoint 25

unicast connections are created. This can be avoided by using anycast flows. Increasing the number of data centers served by unicast flows greatly increases the network congestion. In this dissertation unicast request d is described by a source node s(d), a destination node t(d) and a bit-rate c(d). Notice that in the case of a unicast request, the set of candidate paths P (d) for routing between s(d) and t(d) include exactly k paths, where k is defined in a simulation setup.

3.2.2

Anycast Flows

Figure 3.2: Anycast transmission - one-to-one-of-many Anycasting – defined as one-to-one-of-many transmission – is a very effective way to serve network services provisioned in data centers including popular cloud computing and content-oriented services. Based on the location of the client request, the routers send it to the data center in the network that are chosen by various criteria. The simplest one is distance. This is beneficial since, among other things, it reduces latency and increases redundancy. An anycast request d is defined by a source node s(d), where the aggregation of anycast traffic is performed; a bit-rate cdown (d) that travels from the data center (DC) node to the source node (downstream capacity); a bit-rate cup (d) from the source to the DC (upstream capacity); and set of admissible DC nodes R. We assume that each DC provides the same service and in consequence each DC can serve every anycast request. For dynamic routing of both anycast and unicast requests we must find a routing path and allocate spectrum. However, in the case of anycast requests the DC node must be selected for each request. For more details on modeling of anycast traffic refer to [9]. The use of anycast routing brings significant savings in terms of spectrum utilization, and network cost. This mostly follows from the fact that when using anycasting the service is placed closer to the end user and therefore shorter routing paths may be used. Moreover, anycast improves redundancy by using multiple DCs in the network. This allows for traffic to be rerouted to the next nearest available server in the case that one server fails or goes offline. 26

Finally, load balancing may be utilized. This takes some of the resource requirements off of a singular DC and disperses them across multiple DCs. When two requests identify the same anycast request (upstream and downstream), they are referred to as associated. We assume that associated requests must be connected to the same DC. The set of candidate paths P (d) contains k ⇥ |R| paths, for each associated requests, since each DC node r 2 R is considered.

3.2.3

Multicast Flows

Figure 3.3: Multicast transmission - one-to-many Multicast is known as a one-to-many transmission. It allows routing a request from a sender to various selected endpoints simultaneously. It is also the most efficient way to deliver the same content to a group of users. The role of multicast transmission has been increasing in recent years due to the growing popularity of various network services related to the concept of content-oriented networks, such as video streaming systems, Content Delivery Networks (CDNs) or cloud computing [35]. Moreover, it should be emphasized that all kinds of streaming video and cloud services are the fastest growing segment of the Internet. Multicast communication arises in a wide variety of applications such as video distribution and teleconferencing. It is beneficial when a user uses multiple DCs at the same time, for example streaming audio or video on live broadcasting platforms. The multicast request d is described by a source (root) node s(d), set of destination nodes T (d) and a bitrate c(d). The most popular approaches to model multicast flows proposed in the literature are: canonical formulation, where Steiner tree is used to model the traffic [101] and flow formulation, where each path from the source s(d) to the receiver in multicast request is modeled as a traditional unicast path [87]. The main weakness of the first approach is that the computation time increases exponentially with the number of nodes. Therefore, it is hard to apply the canonical formulation in modeling multicast flows in large optical networks. The second approach is also not efficient, because it creates separate unicast 27

requests in multicast request for all pairs of sender – receiver nodes. The approach that I propose to multicast modeling in EONs is based on the candidate tree formulation studied in [102], in the context of p-Cycle based multicast protection. More specifically, for each multicast request (source and set of receivers) I calculate a set of candidate trees P (d) that originate at the source node and include all receiver nodes. For a request to be established, one of the candidate trees is selected. The main advantage of that approach is scalability. We may easily control the simulations by selecting the number of candidate trees. For example, for small multicast requests, the number of candidate trees may be increased and the simulations will be performed in a reasonable time. With larger problem sizes, we may decide to use a smaller number of candidate trees to increase the speed of simulations. The process of calculation of candidate trees is defined in the next sections.

3.3

Problem Description

The research in this dissertation is provided from the perspective of the network operator. Big companies (e.g. Netflix, Flickr) are outsourcing their computing demands towards cloud services. The reason is that they are stressing for explicit bandwidth and low latency connectivity services. On the other hand, network operators look forward to deliver new type of services to those companies in a cost-efficient and scalable way. The motivation is to create and implement innovative framework for the optimization of cloud services provisioning. The solution proposed in this dissertation should help to choose the best approach for provisioning cloud computing requests of the network operators’ clients. The CSO architecture consists of the optical network where the EON technology is deployed, SDN controller that facilitates setting up and tearing down traffic flows, DC infrastructure, and incoming network requests that are send by the clients. The CSO architecture is shown in Figure 3.4. The SDN controller implements optimization and system management modules, that are easy to configure [71]. Inside the optimization module, we may define the policies of DC assignment, including Path Computation Element and our own submodules, e.g. Traffic Prediction module. In addition, system management module is mainly need to present the real-time SDN operation state, topology and the flow tables’ content of SDN switches [103]. The SDN controller obtains information about current utilization of optical links from the network devices installed in an optical layer. It may, therefore, optimize flow management by sending lightpath requests to the optical layer, to provision DC requests and support service/user requirements for scalability and flexibility. Each DC has three types of resources: computational cores (CPU units), RAM (GBs), and storage (TBs). Spectrum (THz) and optical regenerators (units) are the main considered resources in optical layer [46]. Lightpath requests are considered as requests for optical strata resources, whereas DC requests are established to allocate DC resources. The SDN controller has complete information about the current state of available resources to perform routing. Various approaches presented in this dissertation are implemented in the SDN controller in order to make the best decision for serving the incoming DC requests. 28

Figure 3.4: Cross Stratum Optimization architecture for Data Center services provisioning The sequence of tasks is: • New client requests arrive to the network. The SDN controller selects the best optical path and DC to serve the requests based on the implemented algorithms [82]. I assume that the DCs are allocated in particular network nodes, therefore, the physical connection between DCs and backbone network is not considered. • The SDN controller sends information about the selected path, spectrum, modulation, and selected DC to the optical layer. EON transmitter (which can change flexibly the modulation format and/or bit rate of the optical signal) then establishes the connection [104], [105]. During lightpaths provisioning, three Routing Modulation and Spectrum Allocation constraints have to be satisfied: spectrum contiguity - each channel has to consists of an adjacent set of frequency slices; spectrum continuity — a channel allocated on a light-structure has to be the same on all of the included links (has to consist of the same frequency slices on all the included links); spectrum non-overlapping — a slice on a particular link can be allocated to, at most, one light-structure [106]. A light-structure is a connection of a routing structure and 29

frequency channel allocated on each structure link [107]. In the case of unicast and anycast flows, a light-structure refers to the routing path that connects source and destination nodes. For the case of multicast flows, it indicates the tree connecting the root node and all its receivers. • After setting up connections, the SDN controller updates all network metrics. It includes utilization of DC resources (CPU/RAM/Storage usage) and optical strata resources (regenerators and spectrum usage). This process is repeated for every incoming request. • All requests have a defined lifetime, after which are torn down. When the request ends, the network resources are released and the utilization metrics are updated. This process is repeated for every request that has ended. The CSO architecture enables dynamic topology control. In other words, it means that we can perform reconfiguration of network resources on the fly to cope with incoming traffic demands [108]. It also enables various implementations of the SDN controller such as single controller, multiple controllers connected in a mesh, or hierarchical settings [109]. The CSO also enhances responsiveness to end-to-end DC requests and performs service resilience after a disaster via cooperative recovery techniques between application and network. This architecture is intended to enable OaaS simultaneous stratum optimization [78]. Moreover, all these implementations enable the usage of a traffic prediction. The network programmability enabled by SDN permits seamless communication among all network strata. This significantly enhances utilization of network resources and enables defining cost plans that are devised based on requested services. With the usage of SDEONs we are able to increase the efficiency of dynamic routing in optical layer by enabling network orchestration, which is extremely important these days, when many companies (e.g. Amazon Web Services) treat their network resources as a service in the cloud. SDEON technology enables Cross Stratum Optimization of network stratum resources and provide the elastic physical layer parameter adjustment, e.g., modulation format and bandwidth [109].

3.3.1

Routing Modulation and Spectrum Assignment

In order to handle traffic in an optical layer, we need to implement algorithms for Routing, Modulation and Spectrum Assignment problem [107], [110], [111]. Considering the physical topology discussed above, our goal is to find a route, assign a modulation format and allocate spectrum for each request such that the average bandwidth blocking percentage is minimized. The solution to the problem is subject to the following constraints: • Spectrum contiguity, spectrum continuity and slice opacity: In EONs, continuous spectrum resources to the specific request must be assigned. In other words, a set of slices have to be used over an entire end- to-end optical path. Moreover, if one frequency slice has been allocated to an existing request, this slice cannot be assigned to another request [106]. 30

• Modulation format constraint: In EONs, different modulation formats may be used. However, there exists a trade-off between reachability and spectrum minimization. In brief, if we use a high level modulation format to reduce the spectrum cost, the reachability of the signal will diminish [6].

Figure 3.5: RMSA example Let’s consider the following example, as shown in Figure 3.5. We want to establish a connection between node A and node G with bit-rate 400 Gb/s. In the Table 3.1, I present an example that estimates the transmission reach of an optical signal in a function of the selected modulation format and transported bit-rate for 400 Gb/s request according to the physical model from [85]. It is easy to observe that higher modulation formats allows to save spectrum. On the other hand their transmission reach is lowered. Table 3.1: Example of spectrum usage and transmission reach for the 400 Gb/s request and different modulation formats Modulation Format

SE [b/s/Hz]

Number of slices

Range [km]

BPSK QPSK 8-QAM 16-QAM 32-QAM 64-QAM

1 2 3 4 5 6

34 18 14 10 10 8

1912 1200 989 779 569 359

The procedures of dynamic RMSA are as follows. First, SDN controller tries to find the best route to establish a lightpath. Searching for the best routing path, we have to remember that different modulations have different transmission ranges and spectral efficiency. Therefore, the designed RMSA algorithm must select a path and modulation format together, in order to reduce bandwidth blocking percentage. In our example, the path A-D-E-G is chosen. The total distance between the source and the destination node is 1850 km. For the case without regenerators, we could only use a BPSK modulation 31

format, because of transmission reach limitations. On the other hand, there are available regenerators in nodes C, D and E, which might be used to regenerate the signal. Therefore, the path from A to G is divided into the following segments: A-D, D-E and E-G, with distances of 450 km, 600 km and 800 km, respectively. A regenerator’s usage allows to use modulation formats with higher spectral efficiency, for example 8-QAM or 16-QAM. Main options for routing in discussed example are described in Table 3.2. Table 3.2: Main routing choices for the example situation from Figure 3.5 Path [km]

Modulation formats

Total number of slices

A-B-C-G [1850] A-B-F-C-G [2050] A-F-G [2000] A-D-F-E-G [1950] A-D-E-G [1850] A-F-C-G [2100]

QPSK (A-C), 8-QAM (C-G) BPSK (A-C), 8-QAM (C-G) Not possible to allocate 32-QAM (A-D), 16-QAM (D-E), 8-QAM (E-G) 32-QAM (A-D), 16-QAM (D-E), 8-QAM (E-G) BPSK (A-C), 8-QAM (C-G)

50 116 44 34 82

The RMSA problem may be considered for either static or dynamic scenarios. The static scenarios concerns network planning, where a set of requests is given in advance. The time to allocate all static requests is indefinite, therefore the usage of more complex algorithms and exact optimization methods is available. The goal of the static problem version is to perform allocation in a criterion-efficient manner. The most popular criteria are network cost, power consumption and spectrum usage. Concurrently, in the dynamic scenarios the traffic is unknown before the simulation start. The goal is to provide the best connectivity for all incoming requests, reducing the bandwidth blocking percentage and network OPEX costs. In this thesis, the dynamic version of the RMSA problem is considered.

3.3.2

Network Survivability

Critical services in a telecommunication network should be continuously provided even when undesirable events like sabotage, natural disasters, or network failures happen. It is essential to provide virtual connections between peering nodes with certain performance guarantees such as minimum throughput, maximum delay or loss. It is especially important for transport networks, where the tremendous amount of data aggregated from a number of users is transmitted over a long distance (between different cities, countries or even continents). The design, construction and management of virtual connections, network infrastructures and service platforms aim at meeting such requirements. The definition in [112] states:

Survivability is the ability of a network computing system to provide essential services in the presence of attacks and failures, and recover full services in a timely manner. Focusing on the survivability in EONs, methods described in [113], [114] and shown in 3.6 should be mentioned. 32

Figure 3.6: Network protection/restoration schemes In general, there are two main approaches for providing network survivability: protection and restoration [114], [115]. The first approach is based on actions undertaken in the network planning phase and require reserving redundant backup resources for working paths. These might include backup bandwidth, network devices, etc. Application of protection methods increases the CAPEX and the OPEX cost of the network [116]. On the other hand, the restoration process is applied to runtime phase. During network failures, operators or systems tries to restore the required connectivity in the network, by updating the rules/routes for routing. The latter, restoration, does not reserve any redundant resources in advance, and therefore is more cost-efficient than protection. However, the main disadvantage of the restoration mechanisms is a long recovery time when compared to protection mechanisms [117]–[119]. Thus, in this dissertation, I am comparing various protection methods, as the processing time is a crucial issue for optical networks. The first technique that I am using is called the Dedicated Path Protection (DPP) [116], [120]–[122]. It implies that each request is served by a working (primary) path and a backup path, which is link-disjoint with the working path. The DPP method requires a large amount of extra capacity for protection purposes, keeping protection resources idle when there is no failure. The second way of obtaining survivability in optical networks that I use is Shared Backup Path Protection (SBPP) [57], [116], [120], [123], [124]. It allows for different backup paths to share spectrum resources on the overlapping portion if the corresponding working paths are link-disjoint. Regardless of which link fails on the working lightpath, the same protection route will always be used to recover from the failure. SBPP utilizes capacity more efficiently than DPP, but in some cases may not provide 100% protection. For example, there can be multiple-link failures in the network, which concurrently affect several requests that share the same resources on the backup paths. 33

Another technique of ensuring network survivability in EONs that is implemented in this dissertation is Squeezed Path Protection (SPP) [55]. The key idea of SPP is the use of bandwidth squeezing after a link failure, i.e., only a part (e.g. 25%) of the traffic on the working path is to be protected (or restored). Therefore, the backup path requires much less spectrum resources compared to the working path. Moreover, similar to the SBPP technique, the SPP method allows different backup paths to share spectrum on the overlapping portion if the corresponding working paths are link-disjoint. Considering the fact that network resources are limited, the SPP technique enhances network connectivity and availability by partially recovering the bandwidth of an affected optical channel. The concept of SPP guarantees a Service Level Agreement (SLA) while simultaneously the network does not need to reserve so much bandwidth as in conventional dedicated protection. The backup path bandwidth is reduced to the required minimum amount, which enables cost-effective restoration in terms of spectral resource utilization. An important criteria for measuring the advantages and disadvantages of different protection techniques, we should consider spare capacity efficiency and network control complexity. Spare capacity redundancy is defined as a ratio of the total reserved protection capacity in the whole network to the total working capacity in the whole network. The second, network control complexity is reflected by how much the control plane needs to pay in order to realize a network protection technique. In general, simple network control and management is preferred for a network protection technique. So far, there have been several solutions proposed for survivable EONs in the literature. Survivable RSA algorithms under single-link failure for DDP have been studied in [58], [125], [126]. Concurrently, offline problems for SBPP have been proposed in [127], [128]. Eventually, dynamic survivable EON scenarios have been studied in [123]. All these solutions assure network survivability under single link failures. An algorithm for multi-link failure has been proposed in [129]. For more details on network survivability in optical networks refer to [86], [87]. In this dissertation I assumed that overall the data center network exhibits high reliability. I do not consider protection of DC services. By protection, I mean optical strata protection, as the research is provided from the perspective of the network operator. DC services can be provided by big cloud services operators, such as Amazon Web Services [29] or Microsoft Azure [130] and they have their own mechanism to provide high reliability. Moreover, algorithms and DC assignment policies presented in this thesis are universal, therefore majority of cloud data center solutions are feasible.

3.4

Related Works

In this dissertation I focus on Elastic Optical Networks and Software Defined Networks technologies. Elastic Optical Network technology was introduced in [46]. It was further investigated in many papers, i.e. [6], [48], [52], [113], [131]. While describing the EONs, the RSA and RMSA problems should be mentioned. Those problems were widely discussed for both static and dynamic scenarios. In static problems, the main optimization criterion was spectrum usage. In [12], [132]–[136] the static heuristics and metaheuristics were presented to solve offline RSA problems with anycast flows. Moreover, in [22], [137]–[141] 34

the static RSA problem with multicast flows was presented. The most common approach is to use genetic algorithms [55], [137], [142]–[144], simulated annealing [12], [145] or other heuristics [107], [133], [146]–[152] to solve static optimization problems. The dynamic problems include demand or bandwidth blocking percentage as optimization criterion. The dynamic routing algorithms proposed in [153]–[163] need to be fast and reliable due to the frequent changes of a network state. ILP models and complex methods are not suitable for such problems, as they do not scale well for larger network instances. In [153] authors implement distance adaptive dynamic routing algorithms for mesh and ring type of networks. They achieve promising results, but the networks proposed in this paper are not comparable with a real-life optical topologies. Moreover, the authors of [154] propose several dynamic routing algorithms for RMSA problem with a hybrid single-/multi-path routing scheme for mesh networks. The authors of this paper assume four working modulation formats, whereas in this dissertation six available modulation formats is presented. In [156], [157] authors discuss the RSA problem, whereas in [155], [161] the Shortest Path First algorithm is investigated under various usage scenarios. In [158] authors implement dynamic routing algorithm with holding time awareness. It attempts to reduce resource fragmentation by scheduling requests based on their holding time. Dynamic routing algorithms proposed in this dissertation achieve better results than [153], [154], [158], [159] in terms of BP. Moreover, authors of [160] propose on-demand spectrum resource allocation for elastic optical networks with multi-core fibers. In this dissertation single core fibers are considered. Finally, the authors of [162] focus on RMSA problem for small mesh networks, and the authors of [163] discuss the challenges of defragmentation during dynamic routing scenarios. In addition to traditional RSA and RMSA problems, survivability constraints were introduced. The following path protection methods were described: Dedicated Path Protection in [122], [125], [164], Shared Backup Path Protection in [56], [123], [124], [165], [166], Squeezed Backup Path Protection in [55], [113], [137] and the adaptive survivability method in [15]. In this dissertation survivability aspects are shown only to prove the trend that the Cross Stratum Optimization algorithms achieve better results for both, not protected and protected networks, in comparison to single stratum approach. The more sophisticated comparison of protection methods is shown for example in [167]. The Software Defined Networks concept was introduced in [168], and officially approved in [169]. While the concept of SDN is still evolving, there are many papers discussing the challenges, architecture and benefits of using it - [64], [66], [170]–[175]. Moreover, in a most recent papers, joint Elastic Optical Networks and Software Defined Networks technologies correlation is taken into account. The [79], [138], [176], [177] present the design of experimental testbeds enabling Software Defined Elastic Optical Networks concept. Those papers focuses more on hardware solutions, rather than on dynamic routing algorithms that might be used in DC resources allocation. In [178]–[183] the challenges and optimization problems for SDEONs are introduced. In [178] authors discuss the possibilities and challenges of developing mechanisms for supporting flexible provisioning of resources in EONs. These challenges include issues related to the cost-effective design of network architectures and ways to perform reconfiguration of resources to support dynamic re35

source allocation. Furthermore, the authors of [179] explain the relation between network functions virtualization and the SDEONs concept. Authors of [180] focus on physical and hardware aspects of SDEONs, whereas in [182] the abstract, virtual optical network concept is being researched. The routing problems have begun to be investigated in [83], [184], [185]. The authors of [184] focuses on survivability issues in optical layer, that serves as interconnection between DCs, with the OpenFlow [71] protocol implemented for multipath routing. This work also confirms assumption made in this dissertation, that in order to protect DC services, one need to focus on providing optical layer protection as cloud DC providers assure 100% availability of their services. The concept of SDEONs is still a new proposition, alongside with related Cross Stratum Optimization. The CSO problem was proposed in [78], [83], [186]. To the best of my knowledge, this work is the first one that addresses the problems of Cross Stratum Optimization using jointly anycast, multicast and unicast flows. Moreover, in this dissertation a novel approach of using machine learning techniques to predict traffic patterns in dynamic DC resource allocation problem is investigated. The machine learning techniques in the networks are mostly used for classification of IP traffic [187]–[189] or network intrusion/anomaly detection [190]–[198]. In this dissertation, the Monte Carlo Tree Search algorithm [199] is adapted to the DC resource allocation problem. The first tutorial on using the game theory in communication networks was presented in [200]. Furthermore, the [201], [202] focuses on implementation of Monte Carlo Tree Search algorithm for deflection routing in complex networks. However, the topic of a probabilistic routing/prediction is still not widely discussed in the literature, and there is a need for further study of this topic.

36

Part III Algorithms

37

Chapter

4

Algorithms

I

n this chapter I introduce algorithms, including novel approaches for the considered problem formulation as well as methods that were proposed in the related literature. This chapter is divided into five sections. In the first section, I present dynamic routing algorithms used in RMSA problem in optical layer. Moreover, I discuss spectrum assignment techniques used to allocate optical resources. The following two sections cover the topic of regenerators’ placement problem and background traffic generation that is optimized with the ILP model. Next, I introduce DC assignment policies that are used to choose the best DC and a lightpath to process data center requests in the network. Finally, the path protection approach is introduced.

4.1

Dynamic Routing Algorithms

The main difference between dynamic and static routing is the time of decision - the dynamic routing tables are not known for the entire network but are complemented on a regular basis, dependent on current resources available; a necessity required when a decision is to be made in a matter of milliseconds. Static routing is used in small networks with a simple construction, whereas the use of dynamic routing in such networks is completely pointless. However, in large networks of hierarchical structure, in which there are many routers and networks, redundancy is used, therefore static routing configuration and maintenance becomes troublesome due to low scalability. In EONs, dynamic routing concerns an establishment of lightpaths for individual connection requests and is accompanied by RMSA problem, introduced in Part II. In this dissertation I consider three types of flows: unicast, anycast and multicast. In order to achieve better transparency I introduced three subsections, related to abovementioned flows. In each of them, three algorithms are introduced. First algorithm, Shortest Path First (SPF), has been used to generate reference results. Its implementation has been widely discussed in the literature, including [153], [192], [203]–[210]. The second algorithm that is introduced is named Auxiliary, Composite and Expanded Graph Construction (RMSA-ACEGC-UNI/ANY) from [21] is developed as the result of my collaboration with my supervisor and Prof. Arunabha Sen from the School of Computing, Informatics and Decision Systems Engineering, Arizona State University in the United States. The third algorithm presented in following subsections is named Adaptive Mod39

ulation and Regenerator Aware (RMSA-AMRA-UNI/ANY/MULTI), presented at [15]. I fully designed and implemented this algorithm.

4.1.1

Unicast Flows

Unicast is a very quick way to establish a connection between two nodes in the network. It has a low complexity, but it also has its drawbacks, as shown in Chapter 3.2. The first algorithm, which will be presented for unicast flows, is the simplest SPF algorithm as shown in Algorithm 1 pseudo-code. The goal is to find n contiguous slices starting from b along path p. If this operation is successful, then the algorithm returns the best selected path pbest , the first allocated slice index bbest and the number of allocated slices on each link nbest . In line 2 the spectrum usage is calculated according to the current allocation of requests (following from all accepted requests at a given time). Moreover, in line 3, the slice index bbest and the best path pbest are initialized with null, whereas the number of used slices nbest and the number of required regenerators regbest are initialized with maximum values that are possible to achieve. In particular, the nbest is equal to |B|, and the regbest is equal to the number of all regenerators in the network nregs. Next, in the main loop (lines 4-15), all candidate paths that are sorted in ascending order of the path length are analyzed. For each path, we check every modulation format (line 5), starting from the most spectral efficient one. It provides the lowest requirement of slices but also the smallest transmission reach that the modulation supports. In line 6, the number of required slices nm is calculated according to the bit-rate of demand d and the current modulation format m. Next, in lines 7-10, for subsequent values of the slice index b, we check availability to allocate the request with the current modulation format on the current path. In line 7, we try to allocate the request requiring nm slices on path p using the First Fit (FF) method, starting from the lowest index. In the next step, we check if the following assumptions are met (line 8): the request using current modulation is allocated (bm 6= null) and the number of used slices for the modulation m is smaller or equal to the best solution (nm  nbest ); the number of optical regenerators used for the current modulation (regm ) is smaller than the number of regenerators used in the best solution (regbest ) or is equal to zero. If both of the above conditions are true, we save the current solution as the best one (line 9). Moreover, if solution on particular candidate path is found, we only analyze all modulations for that path, and then the solution is returned (line 13). Otherwise, we proceed to the next candidate path in the set. If all candidate paths and modulations are checked and there is no a feasible path, the request is blocked (rejected). The execution times for RMSA-SPFUNI are less than 10 milliseconds per request. It is a very fast approach, but preliminary results show that it might not provide requested accuracy. The time complexity of this algorithm is O(|P | ⇥ |M | ⇥ |E| ⇥ |B|), as in the worst case scenario for each path p 2 P and each modulation m 2 M , we need to allocate a maximum of |B| slices on |E| links. The second algorithm is named RMSA-ACEGC-UNI [21] (see Algorithm 2). Since the pseudo-code of the algorithm is complex, let me first explain the idea behind the algorithm. The vertex set V of the input graph G = (V , E, B, L), may be split into two groups Vreg 40

Algorithm 1: Shortest Path First (RMSA-SPF-UNI) Data: Request d, characterized by source s(d) and destination t(d) nodes, capacity c(d), type of DC resource (CPU/RAM/Storage), and set of candidate paths P (s(d), t(d), k), where k denotes the number of available candidate paths. Result: Path pbest , slice index bbest and the number of slices nbest , if lightpath is established, null if not. 1 begin 2 Update available spectrum on each link; 3 Set pbest ( null, bbest ( null, nbest ( |B|, regbest ( nregs; 4 for p 2 P(s(d),t(d),k) do 5 for m 2 M do 6 nm ( n(c(d), p, m); 7 (bm , regm ) ( F F (p, nm ); 8 if (bm 6= null) ^ (nm  nbest ) ^ ((regm < regbest ) _ (regm == 0)) then 9 regbest ( regm , nbest ( nm , pbest ( p, bbest ( bm ; 10 end 11 end 12 if bbest 6= null then 13 return pbest , bbest , nbest ; 14 end 15 end 16 return null; 17 end

and V? , indicating the nodes, where regenerators are located and the nodes, where there are no regenerators, respectively. In first loop of the algorithm (line 4), we compute the shortest path between every pair of nodes in the node set s(d) [ t(d) [ Vreg and store it with a distance between the nodes in Distance Table (DisT ab). We then compute auxiliary graphs, Hm , ..., H|M | , corresponding to each modulation technique m in the following way (lines 5-11). Each auxiliary graph Hm where m 2 M has |Vreg |+2 nodes where |Vreg | is the number of regenerator nodes and two additional nodes correspond to the source and destination nodes s(d) and t(d). In the auxiliary graph Hm where, there is an edge between the node vi and node vj , if the transmission reach for m and path p is met. The weight of this edge is x ⇥ n/c(d), where x is the number of edges of the graph G that makes up the shortest path from vi to vj , c(d) is the capacity of the modulation technique m and n is bandwidth requested by the connection from s(d) and t(d). In the lines 12-13, we construct a composite graph Gc from the set of auxiliary graphs Hm , ..., H|M | in the following way. The composite graph Gc has exactly the same set of nodes as H|M | graphs. The weight of the edge between nodes vi and vj in Gc is equal to the smallest weight between the nodes vi and vj in H|M | taken over all m. In the following step, we create an expanded graph, Ge by replacing each node of Gc (except s(d) and t(d)) by a clique of size equal to the degree of the node. The weight of the edges in the clique is the value of the parameter ↵ (set at the beginning of the simulations). The weight of the remaining edges of Ge is equal to 1–↵ times the weight of the corresponding edge in Gc . Finally, we compute the shortest path using Dijkstra algorithm and allocate request on it, using FF method. If 41

request is allocated, we return pbest , bbest and nbest (line 18), otherwise it is null. The time complexity of this algorithm is O(|V |2 ⇥ |E| log |V | ⇥ |M | + |E| ⇥ |B|), where |E| log |V | is a complexity of Dijkstra algorithm and |E| ⇥ |B| is a processing time of request allocation on the lightpath. Algorithm 2: Auxiliary, Composite and Expanded Graph Construction (RMSAACEGC-UNI) Data: Request d, characterized by source s(d) and destination t(d) nodes, capacity c(d), type of DC resource (CPU/RAM/Storage), and set of candidate paths P (s(d), t(d), k), where k denotes the number of available candidate paths. Result: Path pbest , slice index bbest and the number of slices nbest , if lightpath is established, null if not. 1 begin 2 Update available spectrum on each link; 3 Set pbest ( null, bbest ( null, nbest ( |B|, regbest ( nregs; 4 foreach vi , vj 2 s(d) [ t(d) [ Vreg do compute shortest path between vi , vj and store it in DisT ab(vi , vj ); 5 for m 2 M do 6 Construct auxiliary graph Hm with a node set s(d) [ t(d) [ Vreg ; 7 if Om DisT ab(i, j) then 8 create an edge between vi , vj 2 Hm for each pair of nodes vi , vj 2 s(d) [ t(d) [ Vreg ; 9 end 10 Assign a weight of x ⇥ n/c(d) on this edge; 11 end 12 Construct composite graph Gc with node set s(d) [ t(d) [ Vreg ; 13 Assign a weight wc (vi , vj ) on the edge (vi , vj ) 2 Gc , in the following way. Set wc (vi , vj ) = min of wm (vi , vj ), over all m, where wm (vi , vj ), represent the weight of the edge (vi , vj ) 2 Hm ; 14 Construct expanded graph Ge by replacing each node of Gc (except s(d) and t(d)), by a clique of size equal to the degree of the node; 15 Assign a weight of ↵ to the edges in the clique. To the remaining edges of Ge , assign a weight of (1–↵) times the weight of the corresponding edge in Gc ; 16 Compute the shortest path pbest from s(d) to the t(d) in the graph Ge using Dijkstra; 17 Allocate a request on the path pbest using F F and save the nbest and the bbest ; 18 if bbest 6= null then 19 return pbest , bbest , nbest ; 20 end 21 return null; 22 end

For the sake of elaboration, I explain the execution of the algorithm with the help of an example. We would like to compute the shortest path between the source node v1 to the destination node v17. The distance between the nodes are as shown in Figure 4.1(a). In this example, we have 3 available modulation formats (with optical reach 500, 400, 300 and capacity 10, 20, 30, respectively) requested bandwidth equals to 100 Gb/s and ↵ parameter is 0.7. Since the optical reach for example M1 is 500, there will be an edge 42

(a)

(b)

Figure 4.1: Input graph (left) and computed graph for modulation M1 (right) [21] between v3 and v7 in the auxiliary graph H1 , as shown in Figure 4.1(b).

(a)

(b)

Figure 4.2: Computed graphs for modulation M2 (left) and modulation M3 (right) [21]

(a)

(b)

Figure 4.3: Minimum composite graph (left) and expanded graph (right) [21] Since the requested bandwidth for the request is 100 and capacity of M1 is 10, 100/10=10 43

slices will be needed to be carried over two edges e4 and e9. As a result, the length of this edge will be 2*100/10=20. The constructed auxiliary graphs corresponding to modulation techniques M1 , M2 and M3 , which are shown in Figures 4.1(b), 4.2(a) and 4.2(b), respectively. Moreover, the distances between nodes v8 and v9 in H2 , H2 and H3 are 10, 5 and 3.33, respectively. We then take the smallest (minimum) of these set of values (3.33) and assign it to be the weight of edge between the nodes v8 and v9 in the composite graph Gc as shown in Figure 4.3(a). The degrees of nodes v8 and v9 in the composite graph Gc are 3 and 4, respectively. Accordingly, in the expanded graph Ge , v8 is replaced by a clique of size 3 and v9 is replaced with a clique of size 4, as shown in Figure 4.3(b). The weight of the edges in the clique is the value of the parameter ↵, which in this example is set to 0.7. Finally, the shortest path between the v3 and v7 is computed. The third algorithm for unicast flows is named Adaptive Modulation and RegeneratorAware and it is marked with RMSA-AMRA-UNI [7] acronym (the pseudo-code number 3). To illustrate the main idea of this algorithm, I use a simple example. Figure 4.4 shows the first state of the network. We have six nodes, from A to F , and three nodes with regenerators (B, D, E). We want to establish connection between nodes A and F .

Figure 4.4: Initial state of the network In the first step of the algorithm, we update available spectrum on each link, followed by variables initialization, and in the next step (line 4), we calculate k candidate paths using Link Utilization Metric (LU M ) introduced in [15]. This is a modified version of the criteria from [110], [154]. We assume that new requests should be allocated on the links with low utilization. LU M is then calculated as the ratio of occupied slices to all slices in a given link. We introduce 11 ranges of LU M = 0-9% Utilization of Link ) LUM = 1, 10-19% ) 2, 20-29% ) 3,...,90-99% ) 10, 100% ) 1. For the k candidate path with the best metrics we perform a first loop (lines 5-14). First, in line 6, we sum the lengths of the links between nodes that have unassigned regenerators, as seen in Figure 4.5.

Figure 4.5: Network after combining links between nodes with regenerators In our example, we sum the lengths of the links only between nodes B and D, as there are no available regenerators in node C. This step is necessary to continue the algorithm, when we have to decide which modulation format to choose, by looking at the modulation maximum transmission distances. We cannot regenerate a signal in node C, because there are no regenerators. Instead, we choose a modulation format that allows us to transmit data from node B to node D without regeneration (in this case BPSK). In line 7, for each new path segment (pseg ), which is a path between two nodes with regenerators, we 44

choose a modulation format that supports the distance of the new segments and has the lowest metric of penalties. The penalties are introduced in the Table 4.1. If we cannot Table 4.1: Penalties for using different modulation formats Link Usage

BPSK

QPSK

8-QAM

16-QAM

32-QAM

64-QAM

0%-20% 21%-40% 41%-60% 61%-80% 81%-90% 90%-100%

0 1 2 3 4 5

1 0 2 3 4 5

2 1 0 3 4 5

3 2 1 0 4 5

4 3 2 1 0 5

5 4 3 2 1 0

find a modulation to allocate any of these links, a request is rejected due to a lack of regenerators (line 9). Otherwise, we combine segments with the same modulation format, assuming that a transmission distance for these modulation formats allows us to complete this action. In the example situation (see Figure 4.5), we calculate that 16-QAM has the best metric between nodes A B and QPSK between nodes D E and E F . For the path between nodes B D, we choose BPSK. The distance between nodes D and F is 800 km, so we can combine those nodes and treat them as one link D F using QPSK (refer to Figure 4.6). After this operation, we add penalties for using regenerators, 5 points for every used regenerator on the lightpath (line 13). Next, we sort candidate paths in an ascending order with newly calculated metrics (line 15). Steps 16-33 are similar as in RMSA-SPFUNI. For a new order of k candidate paths, we try to allocate a request on each of the path segments (lines 17-20). We also introduce two arrays, barr and narr , to store the information of slice index b and number of required slices n on every calculated path segment. If on each segment of the lightpath there is a possibility to allocate a request (b0arr 6= null) and the 0 0 result is better than the current best one (n0best  nbest ^ (regbest < regbest _ regbest == 0)), we store the information about the new best result in variables regbest , pbest , barr , narr , as shown in lines 25-27. We return null, if allocation fails on any of the path segments, and there is no single candidate path that allows to successfully allocate the request d. The time complexity of this algorithm is O(|P | ⇥ |E| log |V | + |P | log |P | + |P | ⇥ |E| ⇥ |B|), where |P | log |P | is a complexity of a merge sort used to sort candidate paths.

Figure 4.6: Final result of the RMSA-AMRA-UNI algorithm execution

45

Algorithm 3: Adaptive Modulation and Regenerator-Aware (RMSA-AMRA-UNI) Data: Request d, characterized by source s(d) and destination t(d) nodes, capacity c(d), type of DC resource (CPU/RAM/Storage), and set of candidate paths P (s(d), t(d), k), where k denotes the number of available candidate paths. Result: Path pbest and arrays barr and narr , if lightpath is established, null if not. 1 begin 2 Update available spectrum on each link; 3 Set pbest ( null, barr ( null, narr ( null, nbest ( |B|, regbest ( nregs; 4 Calculate k candidate paths with LU M ; 5 for p 2 P (s(d), t(d), k) do 6 Sum links lengths between nodes with regenerators on whole path; 7 Choose m available to allocate with best metric between nodes with regenerators; 8 if m 2 ? then 9 return null; 10 else 11 Combine link fragments with the same modulation; 12 end 13 Include penalties for using regenerators; 14 end 15 Sort k candidate paths ascending using new metrics; 16 for p 2 P (s(d), t(d), k) do 17 for pseg 2 p do 18 nseg ( n(c(d), pseg , mseg ); 19 (bseg , regseg ) ( F F (pseg , nseg ); 20 end 21 Save all bseq into the array b0arr ; 22 Save allP nseq into the array n0arr ; 23 n0best = pseg 2p nseg ; P 0 24 regbest = pseg 2p regseg ; 0 0 25 if (b0arr 6= null) ^ (n0best  nbest ) ^ ((regbest < regbest ) _ (regbest == 0)) then 0 0 0 26 regbest ( regbest , narr ( narr , nbest ( nbest , pbest ( p, barr ( b0arr ; 27 end 28 end 29 if barr 6= null then 30 return pbest , barr , narr ; 31 else 32 return null; 33 end 34 end

46

4.1.2

Anycast Flows

Let us start with the simplest algorithm - RMSA-SPF-ANY (see Algorithm 4). The procedure for establishing an anycast request is similar to the unicast one. In the case of an anycast, we check which of the two associated anycast requests (downstream or upstream) has a larger bit-rate to determine which one is processed first (line 4). Algorithm 4 shows the case when the downstream request is processed first. Set of shortest paths to available DCs for each request d and all modulation formats m are considered in the ascending order of the path length (lines 5-19). If none of the paths for the downstream connection is feasible, the anycast request is rejected. Otherwise, the upstream request is examined as a unicast request (line 16), since the server node is already decided for the downstream request (line 15). The complexity of RMSA-SPF-ANY is O(|P | ⇥ |M | ⇥ |E| ⇥ |B| ⇥ |R|). Algorithm 4: Shortest Path First (RMSA-SPF-ANY) Data: Associated requests ddown and dup , characterized by source s(d) and destination t(d) nodes, capacity c(ddown ) and c(dup ), type of DC resource (CPU/RAM/Storage), and set of candidate paths P (s(d), t(d), k), where k denotes the number of available candidate paths. Result: Paths pbest/up , pbest/down , slice indices bbest/up , bbest/down and the number of slices nbest/up , nbest/down , if lightpath is established, null if not. 1 begin 2 Update available spectrum on each link; 3 Set pbest ( null, bbest ( null, nbest ( |B|, regbest ( nregs; 4 Compare c(ddown ) and c(dup ) to determine which request to process first; 5 for p 2 Pdown (s(d), r, k) // it is assumed that ddown is processed first 6 do 7 for m 2 M do 8 nm ( n(c(ddown ), p, m); 9 (bm , regm ) ( F F (p, nm ); 10 if (bm 6= null) ^ (nm  nbest ) ^ ((regm < regbest ) _ (regm == 0)) then 11 regbest ( regm , nbest ( nm , pbest ( p, bbest ( bm ; 12 end 13 end 14 if bbest 6= null then 15 pbest/down ( pbest , bbest/down ( bbest , nbest/down ( nbest , rdown ( r; 16 pbest/up , bbest/up , nbest/up ( RMSA-SPF-UNI(s(d), rdown , k); 17 break; 18 end 19 end 20 if bbest/up 6= null ^ bbest/down 6= null then 21 return pbest/up , pbest/down , bbest/up , bbest/down , nbest/up , nbest/down ; 22 else 23 return null; 24 end 25 end The second algorithm for anycast flows, that I present is named RMSA-ACEGC-ANY, pseudo-code nr 5. Anycasting is specified by the quadruple: a client node, the set of 47

admissible DC nodes and upstream and downstream bandwidth requirement. Since the upstream and downstream bandwidth requirement may be different, if the path length is measured using the metric from line 5, the upstream path length from the client node to a DC node may be different from the downstream path length between the same pair of nodes. For this reason, after computing the DC node (lines 5 and 6), we process the request as an unicast transmission (line 11). The result objects are the same as in Algorithm 4. The time complexity of the RMSA-ACEGC-ANY algorithm is O((|V |2 ⇥|E| log |V |⇥|M |+ |E| ⇥ |B|) ⇥ |R|). Algorithm 5: Auxiliary, Composite and Expanded Graph Construction (RMSAACEGC-ANY) Data: Associated requests ddown and dup , characterized by source s(d) and destination t(d) nodes, capacity c(ddown ) and c(dup ), type of DC resource (CPU/RAM/Storage), and set of candidate paths P (s(d), t(d), k), where k denotes the number of available candidate paths. Result: Paths pbest/up , pbest/down , slice indices bbest/up , bbest/down and the number of slices nbest/up , nbest/down , if lightpath is established, null if not. 1 begin 2 Update available spectrum on each link; 3 Set pbest ( null, bbest ( null, nbest ( |B|, regbest ( nregs; 4 Compare c(ddown ) and c(dup ) to determine which request to process first; 5 foreach DC node rdown do compute the shortest path from client s to data center rdown using metric ↵Rp + (1 ↵)SP , and store the shortest path length in SP (s(d), rdown ); 6 rdown = min(SP (s(d), rdown )) 2 R; 7 if rdown == null then 8 return null; 9 else 10 pbest/down ( pbest , bbest/down ( bbest , nbest/down ( nbest , rdown ( r; 11 pbest/up , bbest/up , nbest/up ( RMSA-ACEGC-UNI(s(d), rdown , k); 12 end 13 if bbest/up 6= null ^ bbest/down 6= null then 14 return pbest/up , pbest/down , bbest/up , bbest/down , nbest/up , nbest/down ; 15 else 16 return null; 17 end 18 end The third algorithm that we want to present is anycast version of [15]. Anycast version, shown in Algorithm 6 pseudo-code, is in general similar to unicast. The main difference is that the two associated anycast requests (downstream and upstream) must be processed together to assure that both associated anycast request use the same DC node. First, the larger request of the associated requests is allocated (lines 5-29), considering all possible DCs, then the corresponding smaller associated request is served as a unicast, as the DC node is already selected (lines 30-40). If none of the paths are feasible, the anycast request is rejected (line 34 or line 39). The time complexity of the RMSA-AMRA-ANY is O((|P | ⇥ |E| log |V | + |P | log |P | + |P | ⇥ |E| ⇥ |B|) ⇥ |R|). 48

Algorithm 6: Adaptive Modulation and Regenerator-Aware (RMSA-AMRA-ANY) Data: Associated requests ddown and dup , characterized by source s(d) and destination t(d) nodes, capacity c(ddown ) and c(dup ), type of DC resource (CPU/RAM/Storage), and set of candidate paths P (s(d), t(d), k), where k denotes the number of available candidate paths. Result: Paths pbest/up , pbest/down and arrays barr/up , barr/down and narr/up , narr/down , if lightpath is established, null if not. 1 begin 2 Update available spectrum on each link; 3 Set pbest ( null, barr ( null, narr ( null, nbest ( |B|, regbest ( nregs; 4 Compare c(ddown ) and c(dup ) to determine which request to process first; 5 Calculate k candidate paths to DCs with LU M ; 6 for p 2 Pdown (s(d), r, k) do 7 Sum links lengths between nodes with regenerators on whole path; 8 Choose m available to allocate with best metric between nodes with regenerators; 9 if m 2 ? then 10 return null; 11 else 12 Combine link fragments with the same modulation; 13 end 14 Include penalties for using regenerators; 15 end 16 Sort k candidate paths ascending using new metrics; 17 for p 2 Pdown (s(d), r, k) do 18 for pseg 2 p do 19 nseg ( n(c(ddown ), pseg , mseg ); 20 (bseg , regseg ) ( F F (pseg , nseg ); 21 end 22 Save all bseq into an array b0arr ; 23 Save allP nseq into an array n0arr ; 0 24 nbest = pseg 2p nseg ; P 0 25 regbest = pseg 2p regseg ; 0 0 26 if (b0arr 6= null) ^ (n0best  nbest ) ^ ((regbest < regbest ) _ (regbest == 0)) then 0 0 0 27 regbest ( regbest , narr ( narr , pbest ( p, barr ( barr ; 28 end 29 end 30 if barr 6= null then 31 pbest/down ( pbest , barr/down ( barr , narr/down ( narr , rdown ( r; 32 pbest/up , barr/up , narr/up ( RMSA-AMRA-UNI(s(d), rdown , c( dup ), k); 33 else 34 return null; 35 end 36 if barr/up 6= null ^ barr/down 6= null then 37 return pbest/up , pbest/down , barr/up , barr/down , narr/up , narr/down ; 38 else 39 return null; 40 end 41 end 49

4.1.3

Multicast Flows

There is an increasing number of applications such as software distribution, audio/video conferences, and audio/video broadcasts where data is destined to multiple receivers. During the last decade, multicast routing and multicast delivery have evolved. The need for efficient multicast routing algorithms is more and more important in current networks. The upcoming requests may be realized using candidate lighttrees (CL) or sets of unicast lightpaths (from the source to each destination). The definition of CL was firstly introduced in [10]. In this dissertation, I decided to use CL approach, as it is more efficient than establishing sets of unicast lightpaths [211]. Moreover, I present two algorithms RMSA-SPF-MULTI and RMSA-AMRA-MULTI. The complexity of ACEGC algorithm, presented in [21] is very high and require a long computation time. I decided not to implement it for multicast version, as dynamic routing algorithms are characterized by a very quick response time. Let’s start with describing the steps required to build a multicast tree. For the sake of elaboration, I prepared Figure 4.7. The input for the procedure of creating the multicast tree is a multicast request, consisting of a source node and receivers nodes, as well as an optical network with link utilization and regenerators’ usage. In this example, the source of transmission is in node a and the receivers are in nodes e, h, i. Using the shortest path policy or adaptive penalties from AMRA algorithm [7], we calculate the processing order of connections between the root node s(d) and receivers T (d) in a multicast tree. After this calculation, we choose the first node pair to establish a connection as shown in Figure 4.7(b). The connection is established using one technique described specifically in RMSA algorithms: Adaptive Modulation and Regenerator-Aware algorithm (RMSAAMRA-MULTI), which was introduced in [15], with a pseudo-code in Algorithm 8 or Shortest Path First (RMSA-SPF-MULTI), which was used as a baseline solution in many papers, i.e [154], [158], [212], with the pseudo-code described in Algorithm 7. For the remaining receivers, we calculate the best connection between the t(d) and the node, that is already in a multicast tree. An example is shown in Figure 4.7(c). The shortest links to connect to the existing multicast tree are between e — d and i — g. Moreover, in the next step, we need to determine the modulation format and regeneration scheme in lightpaths. When there are free regenerators, we promote spectrum-efficient modulation format (i.e., 32-QAM or 64-QAM), regenerating the signal in the nodes with regenerators. On the other hand, when we lack regenerators, a less efficient modulation format is used to achieve the connection between nodes in the network. Note that modulation format between the nodes may be changed [42], and it is implemented in the RMSA-AMRA-MULTI algorithm. If we decide to regenerate a signal/change modulation format, the modulation format needs to be the same for all links on the output of the node until next node with a regenerator, as shown in Figure 4.7(d). The whole procedure is named buildAMulticastTree() and is repeated for each incoming request, by the RMSA algorithms. Let’s proceed with a pseudo-code of RMSA-SPF-MULTI (see Algorithm 7). The steps of this algorithm are simple. In line 2 we update an available spectrum on each link and set starting parameters in line 3. In the next step, we set strategy for building multi50

(a)

(b)

(c)

(d)

Figure 4.7: Example of creating multicast tree in RMSA problem in EON

cast trees, used in the buildAMulticastTree() method. In RMSA-SPF-MULTI we consider only link lengths, when searching for candidate lightpaths for multicast tree. It is a very fast approach, with low complexity, but has some limitations, as shown in Chapter 6. Furthermore, for each multicast request, we invoke the above-mentioned buildAMulticastTree() method. If multicast tree with allocated slices is found, we return pbest , bbest and nbest (line 9), otherwise null. On one hand this is a very simple algorithm, but on the other hand the execution times for RMSA-SPF-MULTI are less than 25 milliseconds per multicast request, which is a very good result. The time complexity of the RMSA-SPF-MULTI algorithm is O(|V | ⇥ |E| log |V | ⇥ |E| ⇥ |B|). More advanced metrics are taken into consideration in RMSA-AMRA-MULTI (see Algorithm 8). It’s very similar to RMSA-SPF-MULTI, except the strategy for choosing CL in the process of building the multicast tree is different. In RMSA-AMRA-MULTI, we 51

Algorithm 7: Shortest Path First (RMSA-SPF-MULTI) Data: Request d, characterized by source s(d) and destination t(d) nodes, capacity c(d), type of DC resource (CPU/RAM/Storage), and set of candidate paths P (s(d), t(d), k), where k denotes the number of available candidate paths. Result: Path pbest , slice index bbest and the number of slices nbest , if lightpath is established, null if not. 1 begin 2 Update available spectrum on each link; 3 Set pbest ( null, bbest ( null, nbest ( |B|, regbest ( nregs; 4 Set strategy for building multicast tree as sorting CL by path length; 5 for upcoming multicast request d do 6 buildAMulticastTree(); 7 end 8 if bbest 6= null then 9 return pbest , bbest , nbest ; 10 else 11 return null; 12 end 13 end

take into consideration penalties for using regenerators and various modulation formats, as it was described in Chapter 4.1. It allows to build less loaded trees for multicast requests, thus, increasing the efficiency of the whole network. The time complexity of this algorithm is also O(|V | ⇥ |E| log |V | ⇥ |E| ⇥ |B| + |P | log |P |). Algorithm 8: Adaptive Modulation and Regenerator-Aware (RMSA-AMRAMULTI) Data: Request d, characterized by source s(d) and destination t(d) nodes, capacity c(d), type of DC resource (CPU/RAM/Storage), and set of candidate paths P (s(d), t(d), k), where k denotes the number of available candidate paths. Result: Path pbest and arrays barr and narr , if lightpath is established, null if not. 1 begin 2 Update available spectrum on each link; 3 Set pbest ( null, barr ( null, narr ( |B|, regbest ( nregs; 4 Set strategy for building multicast tree using penalties for using regenerators and various modulation formats; 5 for upcoming multicast request d do 6 buildAMulticastTree(); 7 end 8 if barr 6= null then 9 return pbest , barr , narr ; 10 else 11 return null; 12 end 13 end

52

4.2

Regenerator Placement Problem

The placement of regenerators in EONs requires careful decision making. The location of regenerators in the network is crucial to reduce the load on the most frequently used links and in consequence affects the key performance metric defined in the context of dynamic routing, namely, BBP. In EONs, the use of regenerators and adequate modulation formats brings significant savings in spectrum utilization. There may be several equally valid options for the placement of regenerators, each resulting in different spectrum requirements of the same lightpath. In this dissertation, I introduced four regenerator placement algorithms, namely: Fixed Regenerator (RP-FR), Distance Adaptive (DA-FR), Increase Data Center Priority (RP-IDCP), and Decrease Data Center Priority (RP-DDCP) [13]. The complexity of all RP algorithms is linear - O(|V |). The first algorithm, RP-FR, uses a fixed regenerator placement approach that is a very common method in the literature. To be more precise, a fixed number of regenerators are placed at each node in the network. This approach results in equal treatment of all nodes in the network, but does not consider additional information following from anycast traffic features or the network topology information in terms of link lengths. To recall, the number of all regenerators available to allocate is defined by nregs, whereas nregsv defines the number of regenerators assigned to particular network node. This algorithm is used as a reference method in our experiments. The pseudo-code is presented in Algorithm 9. Algorithm 9: Fixed Regenerator Placement Algorithm (RP-FR) Data: Number of available regenerators in the network nregs. Result: Regenerators located in the network. 1 begin 2 for v 2 V do 3 nregsv = bnregs/|V |c; 4 end 5 end The second algorithm, RP-DA calculates the location of regenerators, based on the link lengths. Pseudo-code is presented in Algorithm 10. We save x as the sum of the lengths of all links in the network and xv as the sum of link lengths connected to a given node v. The E indicates a set of all edges in the network, whereas Ev provides information about the set of edges connected to particular network node v. For each network node (lines 3-6), we count the number of node regenerators as a floor of the total number of available regenerators nregs multiplied by the division of xv and x. The final two algorithms, RP-IDCP and RP-DDCP, implement an innovative approach, which facilitates additional information on network traffic. The pseudo-codes are presented in Algorithm 11 and Algorithm 12, respectively. In more detail, I assume that the anycast ratio (AR), defined as the ratio of anycast traffic to the whole traffic, is known. Parameter AR for a particular network can be obtained by analysis of the network traffic, in the period preceding the moment when the regenerator placement problem is solved. As in the RP-DA algorithm, we take into account link lengths. A new prioritization mechanism 53

Algorithm 10: Distance Adaptive Placement Algorithm (RP-DA) Data: Number of available regenerators in the network nregs. Result: Regenerators located in the network. 1 begin P 2 Set x = e2E l(e); 3 for v 2 VPdo 4 xv = e2Ev l(e); 5 nregsv = bnregs ⇥ (xv /x)c; 6 end 7 end for nodes hosting DC is used, when calculating the amount of regenerators for each node in the network. We use an AR parameter to calculate nregsv (lines 5 and 9). First, we calculate the number of regenerators in a set of nodes with DC (Vr ), as shown in lines 3-6, then for the remaining nodes in the network (lines 7-10). Algorithm 11: Increase Data Center Priority Placement Algorithm (RP-IDCP) Data: Number of available regenerators in the network nregs. Result: Regenerators located in the network. 1 begin P 2 Set x = e2E l(e); 3 for v 2 VP r do 4 xv = e2Ev l(e); 5 nregsv = b[nregs ⇥ (xv /x)] ⇥ (100% + AR)c; 6 end 7 for v 2 VP , where V [ Vr 2 ? do 8 xv = e2Ev l(e); 9 nregsv = b[nregs ⇥ (xv /x)] ⇥ (100% AR)c; 10 end 11 end

Algorithm 12: Decrease Data Center Priority Placement Algorithm (RP-DDCP) Data: Number of available regenerators in the network nregs. Result: Regenerators located in the network. 1 begin P 2 Set x = e2E l(e); 3 for v 2 VP r do 4 xv = e2Ev l(e); 5 nregsv = b[nregs ⇥ (xv /x)] ⇥ (100% AR)c; 6 end 7 for v 2 VP , where V [ Vr 2 ? do 8 xv = e2Ev l(e); 9 nregsv = b[nregs ⇥ (xv /x)] ⇥ (100% + AR)c; 10 end 11 end

54

4.3

Background Traffic Generation

In optical networks, we can have customers, who are using a part of available spectrum in links for a long time. The remainder of spectrum is used for on-demand, dynamic requests needs. To properly allocate new incoming requests, we should consider the optimization of static requests allocation as well as spectrum defragmentation. By static requests, I mean long time, background traffic, existing in the network. In this section, I present the ILP model, introduced in [213], for the purpose of static traffic generation. I also discuss Simulated Annealing algorithm, published in [12], for the allocation optimization of static requests in a network. Moreover, I introduce the algorithm for the defragmentation procedure, presented in [16].

4.3.1

Average Spectrum Width Optimization

Let’s start with an ILP model for an offline problem of Routing and Spectrum Allocation with Joint Anycast and Unicast routing (RSA/JAU) in EONs with static traffic demands in order to minimize average optical spectrum usage. In particular, we address one objective function: the average spectrum usage calculated as the average required spectrum (number of slices) in network links. Average Sets B D Dany Dany(DS) Duni C(d, p) P (d)

Spectrum Width ILP model

slices demands anycast demands (upstream and downstream) anycast downstream demand unicast demands candidate channels for demand d 2 D candidate paths for demand d 2 D if d 2 Duni the candidate path connects end nodes of the demand if d 2 Dany (upstream), path p connects the client and the DC nodes if d 2 Dany (downstream), path p connects the DC and the client nodes E network links Constants 1, if link e belongs to path p realizing demand d; 0, otherwise edp ndp requested number of slices for demand d on path p 1, if channel c associated with demand d on path p uses slice b; 0, otherwise dpcb ⌧ (d) index of a demand associated with demand d if d is a downstream demand, then ⌧ (d) have to be an upstream connection if d is an upstream demand, then ⌧ (d) have to be a downstream connection s(p) source node of path p t(p) destination node of path p

55

Variables xdpc 1, if channel c on candidate path p is used to realize demand d; 0, otherwise yeb 1, if slice b is occupied on link e; 0, otherwise ye the largest index of allocated slice in link e Objective min

1 X ye |E|

(4.1)

e2E

Subject to X

X

p2P (d) c2C(d)

X X

X

xdpc = 1, d 2 D

dpcb edp xdpc

d2D p2P (d) c2C(d,p)

X

X

p2P (d) c2C(d,p)

xdpc s(p) =

X

 yeb , e 2 E, b 2 B

X

p2P (⌧ (d)) c2C(⌧ (d),p)

x⌧ (d)pc t(p), d 2 Dany(DS)

byeb  ye , b 2 B

(4.2) (4.3) (4.4) (4.5)

I take a similar approach as in [213] for formulating the ILP problem. The objective (4.1) is to minimize average spectrum usage by taking sum of the largest index of allocated slices divided by number of links in each demand. Equation (4.2) assures that for each demand d exactly one candidate path and exactly one candidate channel are selected. Next, equation (4.3) guarantees that a slice on a particular link can be allocated to at most one lightpath. Moreover, the constraint (4.4) assures that both associated anycast demands use candidate paths connected to the same DC node. Finally, the (4.5) defines byeb denoting the largest index of allocated slice in link e.

4.3.2

Vector Simulated Annealing Algorithm

In this section, I present the Vector Simulated Annealing (RMSA-STATIC-VSA) algorithm proposed to solve the ILP model presented in the previous subsection [12] (see Algorithm 13 pseudo-code). The VSA method is based on the Simulated Annealing approach, which is a generic probabilistic heuristic for the global optimization of a given function in a large search space. Inspiration and the name came from annealing in metallurgy technique, which involves heating and controlled cooling of a material. The first use of slow cooling was implemented and independently described in the Simulated Annealing algorithm by [214] as a slow decrease in the probability of accepting worse solutions as it explores the solution space. From that time, Simulated annealing is a stochastic optimization procedure which is widely applicable and has been effective in solving large solution space problems. Solution of the optimization problem in RMSA-STATIC-VSA algorithm is represented as a sequence (order) of demands. To calculate the objective function of a particular 56

solution, we allocate the demands in the network one by one according to the particular sequence of demands. To allocate the demand, we analyze all candidate paths and by using the First Fit approach, we calculate the value of the objective function assuming that the demand is assigned to a particular path. Finally, we select a candidate path that guarantees the best value of the objective function. Note that in the case of anycast demands, the candidate paths to all possible DC nodes are analyzed. Moreover, I assume that static traffic exists in the network only in the form of anycast and unicast flows. In the first step, we calculate an initial solution (sequence of demands and value of the objective function) denoted as a current using the Most Spectrum First (MSF) algorithm introduced in [215]. It means that the most spectrum consuming demands are allocated at first. This solution is saved as the best solution (best) and current solution (current) in line 3. We calculate also the initial temperature T using fully automated approach. We take the result (number of slices) generated by MSF in Step 1 and multiply it by z parameter, which is a tuned input parameter of RMSA-STATIC-VSA. The simulation is being processed until we fulfill conditions from line 4. In the first step of the while loop, we randomly select two demands, swap them in the sequence of demands and calculate a new result. We save the value of the current solution (number of allocated slices) as a tmp, using the countAllUsedSlices() method. Next, we process it using the Simulated Annealing algorithm. If a condition from line 8 is fulfilled, we assign a temporary result to the most best result and calculate , as a difference between current and tmp. Furthermore, we check if the current solution is better than the best available at this moment, by checking the value of . Accordingly to the result, we assign current as tmp or we randomly pick value of x to check if it is smaller than Boltzmann function of probability distribution (line 15). If condition is fulfilled, we do the same operation as in line 12, even if the new solution is worse than the previous one. It allows us to avoid local optima, in searching for global optimum. In another case (when x is larger or equal then Boltzmann function), we swap the demands to restore the previous state without saving the result. Finally, we reduce the temperature T with the j parameter, which is a cooling rate parameter. After the while loop, we return best as a result of an algorithm. For better understanding of the algorithm refer to [214].

57

Algorithm 13: Vector Simulated Annealing (RMSA-STATIC-VSA) Data: Set of static demands D, starting temperature T , maximum number of iterations imax , cooling rate parameter j, multiply factor z. Result: Best solution for static demands allocation. 1 begin 2 Calculate current0 , using ordering demands by Most Spectrum First (MSF); 3 Set current ( current0 , best ( current, T ( best ⇥ z; 4 while (i < imax ^ T > 0.01) do 5 d1 ( rand(D), d2 ( rand(D); 6 swap(d1 , d2 ); calculate current; 7 tmp = countAllUsedSlices(); 8 if tmp < best then 9 best = tmp; 10 = temp current; 11 if < 0 then 12 current = tmp; 13 else 14 x = rand(); 15 if x < exp /T then 16 current = tmp; 17 else 18 swap(d1 , d2 ); 19 end 20 end 21 end 22 T = T ⇥ j; 23 end 24 return best; 25 end

58

4.3.3

Double Window Defragmentation Algorithm

In the context of dynamic routing with static traffic in the background, demands arrive one by one without priori information about future demands. The main objective is to minimize the rejection of incoming network demands denoted as a metric called Bandwidth Blocking Percentage (BBP). Minimization of BBP in the dynamic routing is a difficult issue, because demands are transmitted in real time and should be established as soon as possible to avoid rejection. As a result, we have to use simpler, less time-consuming algorithms. One of the possible solutions to improve the performance of dynamic routing algorithms is a defragmentation algorithm. More specifically, the defragmentation algorithm is launched in the background and optimizes resources (both static demands existing in the network, and upcoming dynamic requests). The key goal of the defragmentation algorithm is to reconfigure the established demands in terms of routing and spectrum usage to assure better utilization of the network resources. In this section, I describe the Double Window Defragmentation (RMSA-DEF-DWD) algorithm, which is used for spectrum defragmentation of allocated demands in the network. The main assumptions are: • the RMSA-DEF-DWD algorithm is processing demands in a background thread, hence, it does not affect upcoming connections; • one single demand is being defragmented, one unit at a time; • it works simultaneously for both, static and dynamic routing; • the time complexity is O(|D| ⇥ |E| ⇥ |B|), where |D| denotes a number of demands in the network, and |E| ⇥ |B| is a time to reallocate requests. For the sake of elaboration, I introduce the defragmentation process, using simplified example in Figures 4.8-4.10.

Figure 4.8: Initial state of the lightpath, before the defragmentation

Figure 4.9: Suitable window for defragmentation (blue color)

Figure 4.10: Lightpath state after defragmentation As an input (see Figure 4.8), RMSA-DEF-DWD algorithm searches for fragmented links, in order to improve the demand allocation. When the lightpath and demand are 59

allocated, RMSA-DEF-DWD looks for a first free spectrum window, with an identical size to the currently occupied by the chosen to defragment demand, with slice indices in the range from 1 to |B|/2 (see Figure 4.9), on the entire path from s(d) to t(d). We look for available windows only on the lower part of the link (up to slice no. |B|/2) because the aim is to achieve a better resource fragmentation with demands accumulated on the lower slices indices. As a result, we achieve well-defragmented link (see Figure 4.10). Let’s take a look at RMSA-DEF-DWD pseudo-code (see Algorithm 14). In the first step, we initialize the tmp array. It is used to store the information about the first unused spectral frequency window (range of unused slices, e.g. slices 3-6), that we can use to reallocate a particular demand, during the defragmentation process. For example, if demand d1 uses four slices with indices from 82 to 85, and we find a potential spectral window of four slices to reallocate, information about those four unused slices is saved into tmp array. At the start of the simulation this variable is empty. In the set of all currently allocated demands, we first search for a single demand with a biggest lifetime. Then, for this demand, we look for a lowest available spectrum window, with an identical size to the one this demand is using, on the same lightpath. The slices indices need to be between 1 and |B|/2, continuous on the entire path from s(d) to t(d). If we find such a window, we set it as tmp (see line 4). To avoid allocation of the newly upcoming demands on the same slice ranges as tmp, we lock this window (see line 6). After performing this action, demand d (see line 8) is reallocated from an old frequency window to a new, and then, the previously used slices are released. Algorithm 14: Double Window Defragmentation (RMSA-DEF-DWD) Data: Set of allocated, static and dynamic demands D and temporary allocation window tmp. Result: Reallocated demand. 1 begin 2 Set tmp ( null; 3 Find d with a longest lifetime; 4 Find lowest unassigned spectrum window, where b(d) 2 h1; |B|/2i, and set it as tmp; 5 if tmp 6= null then 6 lock tmp; 7 Reallocate d on tmp; 8 release previously used slices; 9 end 10 return d; 11 end The process of defragmenting demands runs continuously in a separate thread. Therefore, there is a situation, where two spectrum windows are occupied (tmp and B(d)). It guarantees that the defragmentation process will not interrupt the connection. Furthermore, RMSA-DEF-DWD algorithm does not violate continuity and contiguity of spectrum resources. Only after a successful reallocation process, we release previously locked spectrum window (see line 8), hence the name of the algorithm is Double Window Defragmen60

tation. A loop executing the process of defragmentation is repeated indefinitely, optimizing allocation of one demand at a time unit - RMSA-DEF-DWD algorithm does not require complicated calculations, thus it is not computation resource-intensive.

4.4

Data Center Assignment Policies

Resource allocation and management in cloud and content oriented computing is a very complex task. This is mainly due to the scale of the problem and the number of services deployed in it. Since cloud users and service providers are given access to high level resources, their effect over the cloud’s overall performance is greater than ever. The last section of this chapter describes DC assignment policies. The three simple approaches that may be employed to identify a DC for providing services to a request are: the nearest (to the request source), the cheapest, or the least utilized DC. While implementing these approaches is rather simple, preliminary simulation results show that they do not scale well and, hence, result in high ( 1%) blocking percentage of requests even in networks with low to moderate traffic loads. Moreover, I propose two approaches: a Hybrid DC assignment approach that combines the three simple approaches to improve network performance, and a Traffic Prediction algorithm that employs the Monte Carlo Tree Search [199].

4.4.1

Nearest Data Center Assignment

The simplest and the most common approach to assign DCs is choosing the nearest DC to the request origin (DC-NEAR algorithm). It takes into account only one strata optical layer. Let us take a look at the pseudo-code shown in Algorithm 15. Algorithm 15: Nearest Data Center Assignment (DC-NEAR) Data: Source node s(d) with CPU/RAM/Storage request (dcpu , dram , dstorage ), set of available DCs R with CPU/RAM/Storage utilization metrics (rcpu , rram , rstorage ). Result: The nearest DC r 2 R and the path p to it. 1 begin 2 Update CPU, RAM and Storage utilization metrics; 3 Calculate k shortest paths p to the nearest DCs, in ascending order; 4 for p 2 P (s(d), r, k) do 5 if dcpu < rcpu ^ dram < rram ^ dstorage < rstorage then 6 return p, r; 7 end 8 end 9 return null; 10 end There are |R| DCs in the network. We consider k shortest candidate paths between the request source and the nearest DCs. Moreover, we have the full information about the request CPU/RAM/Storage units need. In the first step of the DC-NEAR algorithm, it updates utilization metrics on DCs. Next, we calculate k candidate paths in ascending 61

order, using path length as a metric. For each calculated candidate path, we check if it is possible to assign request to particular DC, by checking the updated utilization metrics. If DC is found, we return it and the DC-NEAR algorithm runs RMSA algorithm to allocate request in optical layer using returned path. Otherwise, the request is marked as rejected. The time complexity of this algorithm is linear, depending on the number of candidate paths (O(|P | ⇥ |E| log |V |)).

4.4.2

Cheapest Data Center Assignment

Another approach of DC assignment is to choose the cheapest DC. When choosing the cheapest, I consider the price for service per hour in AWS DC instances. This approach is named DC-CHEAP, and it also takes into account only one strata - DC strata. Let us take a look at the pseudo-code shown in Algorithm 16. Algorithm 16: Cheapest Data Center Assignment (DC-CHEAP) Data: Source node s(d) with CPU/RAM/Storage request (dcpu , dram , dstorage ), set of available DCs R with CPU/RAM/Storage utilization metrics (rcpu , rram , rstorage ). Result: The cheapest DC r 2 R and the path p to it. 1 begin 2 Update CPU, RAM and Storage utilization metrics; 3 Sort DCs ascending, using the price per hour of service; 4 for r 2 R do 5 Calculate k shortest paths p in ascending order; 6 for p 2 P (s(d), r, k) do 7 if dcpu < rcpu ^ dram < rram ^ dstorage < rstorage then 8 return p, r; 9 end 10 end 11 return null; 12 end 13 end The difference between DC-NEAR and DC-CHEAP is that we first choose DC, then we calculate k shortest candidate paths and try to allocate the request. This operation is repeated for all DCs in the network. In none is feasible, the request is marked as rejected. The time complexity of this algorithm is O(|R| ⇥ |P | ⇥ |E| log |V |).

4.4.3

Least Utilized Data Center Assignment

The last single strata approach of the DC assignment is to choose the least utilized DC. In order to calculate the least utilized DC, we take an average out of all utilization metrics. Furthermore, we sort them in ascending order and check if we are able to assign a request to a particular DC, using a specified candidate path. The pseudo-code of DC-LU is shown in Algorithm 17. The time complexity of this algorithm is also O(|R| ⇥ |P | ⇥ |E| log |V |). 62

Algorithm 17: Least Utilized Data Center Assignment (DC-LU) Data: Source node s(d) with CPU/RAM/Storage request (dcpu , dram , dstorage ), set of available DCs R with CPU/RAM/Storage utilization metrics (rcpu , rram , rstorage ). Result: The least utilized DC r 2 R and the path p to it. 1 begin 2 Update CPU, RAM and Storage utilization metrics; 3 Sort DCs ascending, using the average resource utilization metric; 4 for r 2 R do 5 Calculate k shortest paths p in ascending order; 6 for p 2 P (s(d), r, k) do 7 if dcpu < rcpu ^ dram < rram ^ dstorage < rstorage then 8 return p, r; 9 end 10 end 11 return null; 12 end 13 end

4.4.4

Hybrid Data Center Assignment

In Hybrid (DC-HYBRID) approach, I consider DC utilization, cost, and the optical layer utilization of candidate paths from the request source to DCs, to calculate a preference score for every DC and candidate path pair. The DC and the path with the lowest score are then selected for service provisioning. The pseude-code is shown in Algorithm 18. Algorithm 18: Hybrid Data Center Assignment (DC-HYBRID) Data: Source node s(d) with CPU/RAM/Storage request (dcpu , dram , dstorage ), set of available DCs R with CPU/RAM/Storage utilization metrics (rcpu , rram , rstorage ). Result: The best DC r 2 R and the path p to it. 1 begin 2 Update CPU, RAM and Storage utilization metrics; 3 Assign utilization and price metrics for each DC; 4 Sort DCs descending, using metrics from previous step; 5 for r 2 R do 6 Calculate k least utilized paths p in descending order; 7 Assign utilization scores for each path; 8 for p 2 P (s(d), r, k) do 9 if dcpu < rcpu ^ dram < rram ^ dstorage < rstorage then 10 return p, r; 11 end 12 end 13 end 14 return null; 15 end Let us assume that there are |R| DCs in the network. We consider k candidate paths 63

between a DC and the request source. Utilization and price scores are first assigned to each of the DCs, as shown in line 3. Utilization score of a DC is calculated as the ratio of the lowest utilization (CPU, RAM, storage) among all DCs in the network to the utilization of the evaluated DC. Similarly, the price score of a DC is the ratio of the cost of the cheapest DC among all DCs in the network to the cost of the evaluated DC. Hence, the least utilized and the cheapest DCs have the highest price scores. In the next step, we sort DCs descending, using the above-mentioned metrics. Next, in lines 5-13, for the |R| ⇥ k candidate paths from the request source to DCs, utilization scores are calculated as the ratio of the utilization of the least utilized path in the network to the utilization of the evaluated path. The preference score of a (DC, candidate path) pair is finally calculated as the sum of the DC utilization, price, and the candidate path utilization scores. In the DC-HYBRID approach, we then select for service provisioning the (DC, candidate path) with the highest preference score. If none is feasible, the request is marked as a rejected. The time complexity of this algorithm is O(|R| ⇥ |P | ⇥ |E| log |V |).

4.4.5

Traffic Prediction with Monte Carlo Tree Search

The most complex approach, that I used in the DC assignment strategies is Traffic Prediction, using Monte Carlo Tree Search (MCTS) algorithm. In this approach, the DC requests are processed in batches. Monte Carlo Tree Search is then used to identify the best combination of DCs and candidate path pairs for provisioning service to the current batch of requests. MCTS [199] is a promising approach for searching game trees for rewarding actions. If the decision-making agent has access to a generative model of the system that is capable of generating samples of successor states ⇣ 0 and rewards ◆ given a state ⇣ and an action a, it may be used to perform a sampling-based look-ahead search for rewarding actions [216]. Monte Carlo Tree Search builds a sparse search tree and selects actions using Monte Carlo sampling. These actions are used to deepen the tree in the most promising direction [217]–[219]. Figure 4.11 shows the example prediction steps for a network with two DCs and three candidate paths for each of them. The nodes and edges of the search tree correspond to states and actions, respectively. The root of the tree corresponds to the initial state ⇣0 . Let |A⇣ | be the number of available actions at a given state ⇣. The search tree node that corresponds to this state has |A⇣ | child nodes, each corresponding to a possible next state ⇣ 0 that is a result of selecting an action a 2 A⇣ . Each tree node stores a value  and a visit count . A path from the root to a leaf node defines an action policy ⇡. The higher value of  the better the quality of the decision. The  is calculated using the trade-off between the cost of service and request blocking percentage. The MCTS balances path and DC utilization. For example it would be better to use paths and DCs that are moderately utilized than the non-utilized paths and highly utilized DCs. The indicates how often the particular pair of path and DC was chosen in the prediction. Choosing often the same DC and path pair, leads to higher utilization of it. The parameter balances the choices, by lowering the reward of DC and path pairs that are overused. The total reward is a sum of scores, calculated from the root to the termination leaf node, using the (4.7) 64

⇣0  = 92 =1

th1 ) 1, P a

(D C

th1 )

th1 ) 1, P a

(D C

h 3) , P at

h 3)

 = 93 =3

...

 = 79 =4

...

 = 85 =4

...

 = 88 = 10

...

 = 87 =2

⇣di

...

⇣dj

...

⇣dk

...

⇣dl

...

⇣dm

⇣16

(D C 2

h 3) , P at

1, P a

 = 90 =4

...

(D C 2

, P at

(D C

) h3 at

 = 90 =5

...

(D C 2

⇣21

,P C2

 = 87 =3

(D

⇣11

(DC2 , P ath1 )

(D

,P C1

) h1 at

...

...

 = 94 =5

(6d

⇣d

1

⇣236

)

Figure 4.11: Example of a search tree for a network with two DCs and three candidate paths for each DC. equation. The goal is to choose the decision that leads to the higher value of reward. In our example, it is better to choose DC2 and P ath3 in the first step, as prediction showed that it can increase the reward of DC1 and P ath1 for next request allocation. MCTS begins with a tree that only consists of the root node. It then executes until a predefined computational budget is exhausted, as shown in Algorithm 4.11. In simple words indicates the number of search tree levels that are going to be created. The following four phases of the MCTS algorithm can be distinguished: 1. Selection: At this stage, the tree is traversed from the root until a non-terminal leaf node is reached. At each level of the tree, a child node is selected based on a selection strategy, which may be exploratory or exploitative. An exploratory strategy probes the undiscovered sections of the search tree in order to find better solutions. On the other hand, the exploitative strategy focuses on the promising subtrees that have already been discovered. The exploration vs. exploitation trade-off [220] must be considered when employing a selection strategy [221]. 65

In this dissertation, The Upper Confidence Bounds for Trees (UCT) [199] is used as a selection strategy. It is one of the most common selection strategy for MCTS algorithm. Let ✓ denote the visit count of current node of the search tree and the set of all its children. Furthermore, let  and denote the value and visit count of a node with an index . UCT selects a child from: = arg max



+ EX

2

s

ln ✓

(4.6)

,

where EX is an exploration constant that determines the balance between exploration and exploitation. If EX = 0, the selection strategy is strictly exploitative. The MCTS algorithm was originally introduced in Go game [221]. It is a twoplayer board game with three possible outcomes: win, draw, or loss. The solutions may be encoded as 1, 0, or -1, respectively. The UCT selection strategy (4.6) does not consider possible deviations in the values of the children nodes. This deviation does not play an important role in two-players games. The deviation becomes more important in single-player games. It is similar to problem in this dissertation. SinglePlayer MCTS (SP-MCTS) [222] is a variant of MCTS that has been proposed for solving single-player puzzles. It introduces a deviation term to UCT (4.6), hence: = arg max 2



+ EX

s

ln ✓

+

sP

2

◆2

 + LP C

(4.7)

is used as the selection strategy, where ◆2 is the sum of the squared rewards that the th child node has received so far and LP C is a large positive constant. 2. Expansion: After a non-terminal leaf node is selected, one or more of its successors are added to the tree, to expand it. In this dissertation, the most common expansion strategy is used, which adds one node for every execution of the four MCTS phases. The new node corresponds to the next state of the prediction [221]. 3. Simulation: From the given state of the non-terminal node that has been selected, a sequence of actions is performed until a terminal state is reached. Even though MCTS converges with randomly selected actions [199], utilizing domain knowledge may improve the convergence time [217]. 4. Backpropagation: After reaching the simulation time limit or the terminal state, a reward is calculated. This reward is then propagated from the terminal node to the root in order to calculate the solution quality. The computational budget may be defined as the number of evaluated action samples per selection cycle or by selecting the time limit. In this dissertation, the 5 selection cycles for each requests set was established. After repeating the four phases of MCTS times, the child of the root with the highest average value is selected as an optimal action. The final result is a trade-off between the cost and the blocking percentage of the request. The 66

MCTS then enters its next state and the selected child is chosen to be the new root of the search tree. A search tree is first constructed where the root corresponds to the current DC and the optical resource utilization in the network. The root has |R| ⇥ k children for each (DC, candidate path) pair available for serving the current DC request. Monte Carlo simulations are executed using the current distribution of the DC requests to deepen the search tree up to levels. When a leaf node at depth is reached, its value is calculated as the sum of all utilization scores of DCs and optical links in the network. The (DC, candidate path) pair that corresponds to the root’s child with the highest value is then selected for serving the current request. The runtime of the algorithm can be simply be computed as O(|A⇣ | ⇥ ), where |A⇣ | is the number of random children to consider per search and is the computational budget. The advantage of this approach is that, time permitting, it may search for better optical path and DC assignments thus more efficiently utilizing the network resources. Network operators may adjust the execution time of this approach based on the traffic load in order to improve network performance.

Algorithm 19: Traffic Prediction using Monte Carlo Tree Search (DC-TP) Data: Set of source nodes S(D) with CPU/RAM/Storage requests (dcpu , dram , dstorage ), set of available DCs R with CPU/RAM/Storage utilization metrics (rcpu , rram , rstorage ). Result: The best DC r 2 R and path p for each d, and processing order of S(D). 1 begin 2 Update CPU, RAM and Storage utilization metrics; 3 while > 0 do 4 d0 ( root; 5 while d 2 S(D) // tree traversal phase 6 do 7 lastN ode = di ; 8 Select(di+1 ); 9 end 10 Expand(lastN ode) // a node is added to the search tree 11 reward ( Simulate(lastN ode)// a simulated game is played 12 currentN ode = lastN ode; 13 while d 2 S(D) // the reward is backpropagated 14 do 15 Backpropagation(currentN ode, Reward); 16 currentN ode ( parent of currentN ode; 17 end 18 = 1; 19 end 20 end 21 return the action that leads to the root’s child with highest reward;

67

4.5

Path Protection

In this dissertation I consider only single link failures. Then, depending on the protection type method, algorithms assign appropriate backup paths. Let’s take a look for an approach used to establish backup path, shown in Algorithm 20. After calculating of the primary path and allocation of primary spectrum, we process requests in order to establish a backup path. The set of possible backup paths P 0 is calculated on the fly, after each primary path is established. For the case of DPP, each request is served by a primary path and a backup path, which is link-disjoint with the working path (lines 9-13). The second way of obtaining survivability in optical networks that I use is SBPP (lines 14-18). It allows for different backup paths to share spectrum resources on the overlapping portion if the corresponding working paths are link-disjoint. Finally, the last technique of ensuring network survivability in EONs that is implemented in this dissertation is SPP (lines 19-23). The key idea of SPP is the use of bandwidth squeezing after a link failure, i.e., only a part (e.g. 25%) of the traffic on the working path is to be protected (or restored). Moreover, similar to the SBPP technique, the SPP method allows different backup paths to share spectrum on the overlapping portion if the corresponding working paths are link-disjoint. The time complexity of path protection methods is based on the chosen dynamic routing algorithm. In more detail, the selected dynamic RMSA algorithm is executed at most twice, to get the primary and the backup paths. For example, it results in the complexity of O(2 ⇥ |P | ⇥ |M | ⇥ |E| ⇥ |B|) for the RMSA-SPF-UNI algorithm. As the number two becomes insignificant, it ends up to be the same time complexity as for the RMSA-SPFUNI algorithm without path protection. The same rule applies to all dynamic routing algorithms with path protection methods.

68

Algorithm 20: Adaptive survivability for path protection (DC-BACKUP) Data: Request d, characterized by source s(d) and destination t(d) nodes, capacity c(d), type of DC resource (CPU/RAM/Storage), and set of candidate paths P (s(d), t(d), k), where k denotes the number of available candidate paths. Result: Paths p, p0 and slice indices b, b0 , if primary and backup lightpath are established and null if not. 1 begin 2 Update available spectrum on each link; 3 Calculate primary path p 2 P (s(d), t(d), k) and allocate request d, using one of the RMSA algorithms mentioned in this thesis; 4 if p == null then 5 return null; 6 else 7 Calculate set of possible backup paths P 0 , using one of the RMSA algorithms mentioned in this thesis, where P 0 [ P 2 ?; 8 switch d do 9 case DPP do 10 for p0 2 P 0 (s(d), t(d), k) do 11 Allocate backup path and spectrum for request d, using RMSA algorithm, respecting the rule that backup spectrum cannot be shared with other backup requests; 12 end 13 end 14 case SBPP do 15 for p0 2 P 0 (s(d), t(d), k) do 16 Allocate backup path and spectrum for request d, using RMSA algorithm, respecting the rule that corresponding primary paths are link-disjoint; 17 end 18 end 19 case SPP do 20 for p 2 P (s(d), t(d), k) do 21 Allocate backup path and spectrum for request d, using RMSA algorithm, respecting the rule that backup spectrum window is 4 times smaller and that corresponding primary paths are link-disjoint; 22 end 23 end 24 end 25 end 26 end

69

Part IV Results and Simulation Setup

71

Chapter

5

Performance Assessment

C

omputer simulations were performed in order to evaluate the algorithms proposed in this dissertation. Steady-state discrete-event simulations were conducted in order to obtain mean values of parameters indicating the effectiveness of the proposed algorithms. In order to prove the rigor of the results, in this chapter, I introduce the details of the conducted simulations and the simulator environment. The simulator environment, which implements CSO architecture and simulation cases, was deployed based on the simulator introduced in [14]. The simulator was written in Java language with the ability to extend of additional algorithms and network architectures. Moreover, tools, pseudo-random number generators, flow designs and operational rules are described. Finally, simulation input parameters and assumptions about network architectures are introduced in this chapter in order to provide a clear and sufficiently detailed technical description of the environment.

5.1

Simulation Details

In order to gather samples containing independent observations an independent replications method [223] was considered. It requires running a simulation several times, each time using differently seeded pseudo-random number generators. I assume that the requests arrive in batches, based on a Poisson distribution with the mean arrival rate of requests per unit time while their lifetime is exponentially distributed with the mean 1/ . Hence, the traffic load is / Erlangs. The number of requests in the simulation scenarios in both Euro28 and US26 networks is 457,000. I do not consider the first 7,000 requests because the network loads did not reach a steady-state. Each simulation was repeated 10 times with different seeds for random generation. I present the average results of those simulations.

5.1.1

Network Topologies

To assess the proposed algorithms, simulations were performed in two well-known reference networks: the Euro28 network (28 nodes, 82 unidirectional links, and 7 DCs) and the US26 network (26 nodes, 84 unidirectional links, and 10 DCs) shown in Figure 5.1 and Figure 5.2, respectively, available at [224]. The number of DCs in both networks was chosen based on various tuning simulations, to provide the most reliable comparison on performance of the algorithms. Furthermore, 73

Figure 5.1: The Euro28 network topology

Figure 5.2: The US26 network topology the above-mentioned network topologies were chosen to show two separate cases of the allocation of DCs - in Euro28 DCs are more centralized, while in the US26, DCs are scattered on both sides of the continent. The location of DCs, interconnection points, and submarine cable landing stations are obtained from the Data Center Map website [225]. In each DC location, I assumed that ten m3.2xlarge Amazon Web Services (AWS) EC2 74

general purpose instances are available. Each instance has 8 CPU units, 30 GB RAM and 160 GB Storage [29]. The pricing model for DC services is based on the AWS, and its shown in the Table 5.1. Table 5.1: Prices per hour of DC services, provided by the AWS. Region

# of DCs locations

US-WEST US-CENTRAL US-EAST EURO-WEST EURO-CENTRAL EURO-EAST

5 2 3 3 3 1

Price per hour 0.431 0.474 0.442 0.585 0.499 0.632

$ $ $ $ $ $

The EON technology is used for optical layer. In all simulation scenarios, the entire band of 4 THz spectrum, which is divided into 6.25 GHz frequency slices, is available thus resulting into 640 slices. Each network has three interconnection points to other networks that carry international traffic. We take into consideration the physical impairment of links (fiber attenuation, component insertion loss) and use regenerators for signals that require higher modulation formats.

5.1.2

EON assumptions

I use the physical model of EONs as in paper [97]. In more detail, I assume using of EON with BV-Ts implementing the PDM-OFDM technology with multiple modulation formats selected adaptively between BPSK, QPSK, and x-QAM, where x belongs to 8, 16, 32, 64. Note that PDM stands for Polarization Division Multiplexing, which allows doubling the spectral efficiency. The spectral efficiency is set to 1,2,...,6 [b/s/Hz], respectively, for these modulation formats and EON operates within a flexible ITU-T grid of 6.25 GHz granularity. I use a transmission model proposed in [85], which estimates the transmission distance in a function of the selected modulation level and transported bit-rate. Moreover, a 12.5 GHz guard band between neighboring connections is introduced. For all considered modulation formats, the transmission reach is extended by using regenerators, which are applied whenever necessary.

5.1.3

Request Generation

For simulations, three independent pseudo-random number generators were used. The first one was used to generate inter-arrival times between requests, following the Poisson distribution [226]. The second one was used to generate lifetime of the requests, with exponential distribution [227]. The last generator was used to obtain the requests parameters, such as bandwidth usage requirement and DC services usage (CPU/RAM/Storage), following Irwin-Hall distribution [228], [229]. The correctness of the generator implementation is assured by the built-in validation mechanisms. Therefore, this is a suitable approach of developing pseudo-random number generators. 75

5.1.4

Data Center Locations

To handle client requests in the network, in addition to solving the RMSA problem in the optical network, SDN controller has to decide on choosing the best DC. In order to implement network architecture presented in this chapter, the Amazon Web Services DC has been used as a default configuration [29]. All available AWS DCs locations are shown in 5.3. The AWS Cloud provides a broad set of infrastructure services, such as computing

Figure 5.3: All AWS DC locations power, storage options, delivered as a utility: on-demand, available in seconds, with payas-you-go pricing. To effectively explore the possibilities for the network operator, in this dissertation I prepared a range of scenarios how to choose DC, introduced in the section 4.4. DC are characterized by five main parameters: the amount of computational units (CPU units), the amount of memory (RAM units), the amount of disk space (storage units), the expense of use (expressed in USD / hour) and the location (which is used to calculate the distance between client and DC). Depending on the network, data centers are centralized (Euro28 network) or scattered around the two shores of the continent (US26 network). SDN provides the flexibility to implement load balancing algorithms in the network and avoids the need for operators to manually install traffic splitting rules or use custom load balancing solutions.

5.2

Simulations Environment

The availability of new technologies increases the number of researchers focuses on the optimization and development of these technologies. To meet the expectations of industry enterprises it is required not only to prepare the theoretical concept, but also to create a simulation environment. For the purpose of this dissertation I prepared a comprehensive simulator for Elastic Optical Networks, in order to facilitate the commercialization of the 76

results achieved by the various groups of researchers. This is one of the first solution that works on any operating system (software written in Java) and does not require high computing power [14].

5.2.1

Simulator Architecture

First, it is important to discuss features of the simulator. At the beginning we need a storage format for the topology of networks and other miscellaneous data. The choice is to store a matrix of links between nodes. The simulator is able to calculate the distances between nodes on the run in just a few seconds, before the start of the simulation, without a need to load pre-calculated paths. Other data concerning additional information about network components is stored in the form of lists where values are addressed by corresponding identification numbers. An important feature of the loading module is its complete separation from the rest of the program. It facilitates a storage format modification as well as the possibility of using multiple storage formats. Currently, the simulator uses an improved version of storing the network to eliminate the need of calculating the position of the nodes, every time the network is loaded.

Figure 5.4: CEONS GUI – view of the network resources usage The next aspect concerns data structures that are used during the simulation. Generic 77

graph collections were created to provide efficiency and flexibility of usage. As the idea of EONs is still under development and it is possible that some new features may come in the future this software was designed to facilitate any modifications and additions with ease. Next step was to design interfaces for request creation. It is based on statistical data and probability distributions that were calculated from existing data samples. A random generator uses a seed, which allows the user to make exactly the same simulation multiple times. It is also helpful during analysis of the influence of different input parameters in the results. In addition to the back end architecture, the graphic interface is designed as it is shown in Figure 5.4. The goal is to make a program that can be easily picked up by other researchers. It also adds another layer of analysis on the visual level during the simulation, which makes observation of the resources usage in the network easier. This allows not only obtaining numerical results from the simulation but also gives information about where there are possible bottlenecks in the topology. The filling of nodes in the graph indicates the usage of regenerators on particular node and the filling on the links is proportional to the spectrum usage. To display a network of which topology was stored in the form of a link matrix it is crucial to solve the distance geometry problem which revolves around calculating the positions of nodes in a graph given complete or incomplete matrix of distances between them. A simple force-based algorithm was implemented to allow loading and displaying such networks. Finally, a very important feature is the RMSA algorithm interface. The application was designed to allow quick and easy implementation of own RMSA algorithms with a flexible API. Each implementation code can be loaded as a plugin and used in the program.

5.2.2

Simulator External Libraries

To build a code, I am using Maven framework [230]. Based on the concept of a project object model (POM), Maven can manage a project’s build, reporting and documentation from a central piece of information. Moreover, in order to implements graphics managements, I decided to use JavaFX technology [231]. The look and feel of JavaFX applications can be customized. Cascading Style Sheets (CSS) separate appearance and style from implementation. The main features of JavaFX [231] are: • Java APIs: JavaFX is a Java library that consists of classes and interfaces that are written in native Java code. The APIs are designed to be a friendly alternative to Java Virtual Machine (Java VM) languages, such as JRuby and Scala [231]. • Swing interoperability: Existing Swing applications can be updated with new JavaFX features, such as rich graphics media playback and embedded Web content. • Built-in UI controls and CSS: JavaFX provides all the major UI controls required to develop a full-featured application. Components can be skinned with standard Web technologies such as CSS. • Multi-touch support: JavaFX provides support for multi touch operations, based on the capabilities of the underlying platform. 78

• Self-contained application deployment model: Self-contained application packages have all of the application resources and a private copy of the Java and JavaFX runtimes. They are distributed as native installable packages and provide the same installation and launch experience as native applications for that operating system.

79

Chapter

6

Results

T

his chapter focuses on presenting the results of extensive computer simulations performed in order to prove the thesis of this dissertation. There are three main parts of this chapter. In first, the algorithms for the dynamic RMSA problem in EONs are compared. The best is then chosen for the Cross Stratum Optimization problem. Next, various approaches for regenerator placement problem and background traffic optimization are investigated. Again, the best approach is used then in the CSO problem to calculate the locations of the regenerators. Finally, various data center assignment policies are shown to prove the main thesis that cross stratum provisioning strategies are more efficient than single stratum approaches. All introduced algorithms were implemented in the Java simulator and tested extensively to confirm the correctness of the results. The simulations were carried out on the UNIX system with 8 GB of RAM memory and Intel i7 2.7 GHz processor with two available CPU cores.

6.1

Dynamic RMSA Problem in EONs

The main goal of the experiments in this section is to compare the performance of algorithms in the context of dynamic routing of anycast, unicast and multicast traffic in EONs. In Figure 6.1 and Figure 6.4, I present the comparison of the algorithms, namely RMSA-SPF algorithm, RMSA-AMRA, RMSA-ACEGC and RMSA-MNC [158] for the Euro28 network and the US26 network, respectively. All algorithms are widely described in section 4.1. The main performance metric is Bandwidth Blocking Percentage (BBP) defined as the volume of rejected traffic divided by the volume of all traffic offered to the network. We can get two types of BBP – lack of spectrum and lack of regenerators. The input parameters for the above-mentioned simulations are presented in Table 6.1. Table 6.1: Input parameters for dynamic routing algorithms comparison Type of request City to City City to Data Center Data Center to Data Center International Traffic

Bit-rate 10-100 10-200 40-400 10-100

Gbps Gbps Gbps Gbps

Percentage of all

Type of flows

7.1% 51.8% 21.1% 20%

Unicast and Multicast Unicast and Anycast Unicast and Anycast All flows 81

Moreover, the number of k-shortest paths available for each node pair is 10, the number of available regenerators is fixed to 1000 for both networks. Finally, the maximum number of receivers in a multicast request is set to 10. The volume of multicast traffic is calculated as the traffic obtained by all receivers. In particular, if the multicast request bit-rate is 100 Gb/s and it includes eight receivers, it sums up to 800 Gb/s of traffic. The distribution of requests is: 15% multicast, 35% anycast, and 50% unicast requests. The model of traffic is created under the forecasts [2], [3].

Average Bandwidth Blocking Percentage (%)

As we can see in both Figures, the RMSA-AMRA algorithm outperforms other algorithms. First occurrence of blocking is observed at a traffic ratio of 500 ER, while other methods experience the blocking even for traffic loads equal 400 ER. With the same number of regenerators RMSA-AMRA achieves BBP of 1% up to 700 ER, what follows mainly from the fact that it provides a better use of regenerators and has an adaptive ability to choose the modulation formats. The second best algorithm is RMSA-ACEGC. The reference method, RMSA-MNC [158], performed better than the simplest method RMSA-SPF. It is also worth noticing, that the network links started to fill-up more around 600 Erlangs, resulting in more rapid increase in BBP.

4

3

2

1

0 300

350

RMSA-SPF

400

450 500 550 Traffic load (Erlang)

RMSA-MNC

RMSA-AMRA

600

650

700

RMSA-ACEGC

Figure 6.1: Total BBP of algorithms in dynamic RMSA problem using Euro28 network Looking into more details, we can differentiate the results based on the type of BBP lack of spectrum or lack of regenerators, as shown in Figure 6.2 and Figure 6.3. Interesting trends can be observed. The main problem of RMSA-SPF algorithm is that it overuses spectrum. It can be justified that RMSA-SPF looks always for the shortest path first, not taking into an account the available regenerators. On the other hand the RMSAMNC algorithm utilizes spectrum better than the available regenerators. As authors of RMSA-MNC [158] state, the RMSA-MNC algorithm utilizes different modulation formats to decrease bandwidth requirement. However, the number of available regenerators is not directly taken into an account. Both, RMSA-AMRA and RMSA-ACEGC algorithms provide more balanced utilization of available network resources (spectrum and regenerators). 82

Average Bandwidth Blocking Percentage (%)

2

1.5

1

0.5

0 300

350

RMSA-SPF

400

450 500 550 Traffic load (Erlang)

RMSA-MNC

600

RMSA-AMRA

650

700

RMSA-ACEGC

Average Bandwidth Blocking Percentage (%)

Figure 6.2: Lack of spectrum in dynamic RMSA problem using Euro28 network

2

1.5

1

0.5

0 300

350

RMSA-SPF

400

450 500 550 Traffic load (Erlang)

RMSA-MNC

RMSA-AMRA

600

650

700

RMSA-ACEGC

Figure 6.3: Lack of regenerators in dynamic RMSA problem using Euro28 network Following on the Figure 6.4, the results for the network US26 confirm the trends shown in simulations performed using Euro28 network. The BBP for all algorithms is slightly higher, but it is associated with larger distances between nodes in the network. Subsequently, the RMSA-AMRA method reveals similar trends, outperforming other algorithms. In addition, the RMSA-SPF performance is very poor in comparison to its results from the Euro28 network. This follows from the fact that the nodes are concentrated in the East and the West Coasts while nodes in the Euro28 network are more centralized. It results in larger distances between nodes, thus, more rapid worsening effect on wrong decisions. The detailed results of BBP metric, shown in Figure 6.5 and Figure 6.6, reveal similar trends to the one achieved for the Euro28 network. The only difference is that the lack 83

Average Bandwidth Blocking Percentage (%)

4

2

0 300

350

RMSA-SPF

400

450 500 550 Traffic load (Erlang)

RMSA-MNC

600

RMSA-AMRA

650

700

RMSA-ACEGC

Figure 6.4: Total BBP of algorithms in dynamic RMSA problem using US26 network

Average Bandwidth Blocking Percentage (%)

of regenerators is more visible for higher traffic loads. It also follows from the fact that US26 network has larger distances between nodes, therefore it needs to utilize regenerators more frequently. At the time when we lack regenerators, we cannot establish lightpaths for new connections, thus, the spectrum becomes less utilized, and there is less rejections of requests due to the lack of it.

4

3

2

1

0 300

350

RMSA-SPF

400

450 500 550 Traffic load (Erlang)

RMSA-MNC

RMSA-AMRA

600

650

700

RMSA-ACEGC

Figure 6.5: Lack of spectrum in dynamic RMSA problem using US26 network Furthermore, I investigate the average regenerators usage under different dynamic routing algorithms in Figure 6.7 and Figure 6.8. The two main optical network resources are regenerators and optical spectrum available in the links. In order to minimize the BBP one needs to utilize both optical resources equally. Too high utilization of regenerators 84

Average Bandwidth Blocking Percentage (%)

4

3

2

1

0 300

350

RMSA-SPF

400

450 500 550 Traffic load (Erlang)

RMSA-MNC

600

RMSA-AMRA

650

700

RMSA-ACEGC

Figure 6.6: Lack of regenerators for dynamic RMSA problem using US26 network

Average Usage of Regenerators (%)

may cause too low spectrum utilization and vice verse. 100 80 60 40 20 0 300

RMSA-SPF

400

RMSA-MNC

500 Traffic load (Erlang)

RMSA-AMRA

600

700

RMSA-ACEGC

Figure 6.7: Average usage of regenerators in Euro28 network We can notice that RMSA-AMRA and RMSA-ACEGC methods provide the best utilization of regenerators due to their load-balancing methods. Both algorithms for high traffic loads uses over 80% of the available regenerators in the network. The results reported in Figures 6.7-6.8 provide further explanation of the results shown in Figures 6.1 and 6.4. The worse performance of RMSA-SPF and RMSA-MNC, mostly follows from the fact that these methods cannot efficiently utilize the available regenerators in order to adaptively select the modulation formats. Moreover, the regenerators usage for for the US26 network is higher than for the Euro28 network, due to the different topology design. 85

Average Usage of Regenerators (%)

100 80 60 40 20 0 300

400

RMSA-SPF

500 Traffic load (Erlang)

RMSA-MNC

600

700

RMSA-AMRA

RMSA-ACEGC

Figure 6.8: Average usage of regenerators in US26 network

Gains in Blocking Percentage (%)

Furthermore, I investigate the benefits of RMSA-AMRA following from the possibility to change a modulation format in each regenerator in Figures 6.9 and 6.10.

30

20

10

0 300

400

500

600 700 Traffic load (Erlang)

800

900

1,000

Figure 6.9: RMSA-AMRA BBP gain with modulation format change over the version without modulation format change in Euro28 network The growing requirement of achieving higher bit-rates and improving the spectral efficiency, has opened the door to new modulation formats. As we can see in Figure 6.9 and Figure 6.10, the option to change the modulation format provides an efficient use of bandwidth and results in a lower BBP. This is essential for higher traffic in the network, when we observe an improvement of about 20-30% in terms of the BBP. The gains are higher for the US26 network (see 6.10) than for the Euro28 network (see 6.9). The possibility to change a modulation format increase networks complexity, but also provides big benefits for the network operators. By lowering the BBP, one can establish more lightpaths, thus, 86

Gains in Blocking Percentage (%)

30

20

10

0 300

400

500

600 700 Traffic load (Erlang)

800

900

1,000

Figure 6.10: RMSA-AMRA BBP gain with modulation format change over the version without modulation format change in US26 network serve more clients in the network.

Number of required regenerators

Finally, I present the increase in the number of regenerators required to achieve different values of a Service Level Agreement (SLA) in subsequent years in Figures 6.11 and 6.12. The input data is similar to the one presented in the Table 6.1. The only difference is that the generated traffic reflects the expected traffic increase through the aforementioned "Cisco Visual Networking Index" and "Cisco Global Cloud Index" reports for the respective years [2], [3]. The Figure 6.11 corresponds to the results achieved for Euro28 network, whereas the Figure 6.12 presents results for US26 network.

2,000

1,000

0 2016

0-1% BP

2017

1-2% BP

2018 Year

2-5% BP

2019

2020

>5% BP

Figure 6.11: Trade-Off Between BBP and Average Number of Regenerators using RMSAAMRA in Euro28 network For example, to obtain the SLA between 0% and 1% in 2018, approximately 700 regenerators are required in the network. This calculation can help to estimate the needed 87

Number of required regenerators

2,000

1,000

0 2016

0-1% BP

2017

1-2% BP

2018 Year

2-5% BP

2019

2020

>5% BP

Figure 6.12: Trade-Off Between BBP and Average Number of Regenerators using RMSAAMRA in US26 network investments in coming years in terms of the network regenerators. Based on the results, several major trends can be discussed. The difference between obtaining different levels of SLA and the number of available regenerators will increase with the passage of years. For example, in 2016 the difference between getting the best SLA threshold and the threshold above the 5% was about 50 regenerators in the network. It gives roughly about 2 regenerators per network node. On the other hand, in 2020, the difference increased to about 400 regenerators, which gives about 15 regenerators for each network node. The increase in network traffic, especially to and from the data center, will increase the competitiveness of operators over the years. Operators wishing to maintain the highest quality of services will have to invest a lot more money than the competitors, compared to the years 2015-2016. This may affect the prices of network services. Additionally, for the years 2019-2020 the cumulative growth in network traffic can be seen, mainly due to the development of services in the cloud, and thus, much greater demand for optical networks resources. Moreover, to ensure the similar effectiveness of the RMSA algorithms for both networks, one need to install more regenerators in the US26 topology. This trend follows again from the topology design, where nodes in the US26 network are spread over East and West shores and nodes in the Euro28 network are more centralized.

6.2

Regenerator Placement Problem in EONs

Another novelty of this dissertation, as previously mentioned, is the development of various regenerator placement algorithms. At the beginning, I compared the performance of dynamic routing algorithms in terms of BBP under different regenerators’ placement strategies. As a baseline solution I have chosen Fixed Regenerators (RP-FR) algorithm used in majority of research papers [61], [107], [232]. The algorithms that I proposed are: Distance Adaptive (RP-DA), which takes into an account link lengths in the network; 88

Increase Data Center Priority (RP-IDDP) and Decrease Data Center Priority (RP-DDCP), both calculates placement using anycast traffic ratio. All algorithms are widely described in section 4.2. Figures 6.13, 6.15, 6.17 and 6.19 present various results for Euro28 network, whereas Figures 6.14, 6.16, 6.18 and 6.20 present results achieved using US26 network. The simulations were conducted in the following way. First, selected regenerator placement algorithm calculates the locations of regenerators in the network and assign them to nodes. Next, the dynamic routing simulation with incoming requests begins using the RMSAAMRA algorithm. The input parameters for the RMSA problem were similar to the ones presented in the Table 6.1. The results show the gains in terms of BBP for both networks. The number of available regenerators to allocate was equal to 1000 for both networks.

Gains in Blocking Percentage (%)

First, the impact on BBP for RMSA-AMRA algorithm is shown in Figures 6.13 and 6.14. The first noticeable fact is that the best algorithm in this scenario is RP-DA, for both networks. The RMSA-AMRA algorithm has already implemented adaptive techniques to balance the usage of spectrum and regenerators, so the anycast metric inside the RP-IDCP and RP-DDCP is not beneficial as for other RMSA algorithms. As we can observe, the second best regenerator placement algorithm is RP-DDCP. In the Euro28 network, the distances between client nodes and DC nodes are relatively short. The DDCP algorithm allocates more regenerators to nodes without DC. It takes into consideration the fact that the nodes with DC are end nodes of anycast requests, and they do not need frequent regeneration of the signal due to shorter path lengths. In addition, the observation that allocating more regenerators to nodes without DCs is more effective is confirmed by performance of the algorithm RP-IDCP, which uses an approach opposite to RP-DDCP. As we can see in Figures 6.13 and 6.14, increasing the number of regenerators in the nodes with DCs causes an increase in BBP, worsening the results in comparison to the RP-FR algorithm. As shown by the observations of related works, deterioration for higher values of RP-FR is caused by a lack of effective use of regenerators in the network. 15

10

5

0

5 300

400

RP-DA

500

600 700 Traffic load (Erlang)

RP-IDCP

800

900

1,000

RP-DDCP

Figure 6.13: Improvement in BBP of RMSA-AMRA in Euro28 network using various regenerator placement algorithms in comparison to the RP-FR algorithm 89

Gains in Blocking Percentage (%)

Additionally, using the US26 network (see 6.14), regenerator placement algorithms are starting to improve the results of dynamic routing algorithms much faster, even for the lower volumes of network traffic. It follows from the fact that with a larger distance between nodes, the corresponding placement of the regenerators in the network has a greater influence on the final results. This trend is deteriorating for higher network traffic because the network US26 undergoes a faster overflow, causing the network links to be over-utilized. 15

10

5

0

5 300

400

RP-DA

500

600 700 Traffic load (Erlang)

RP-IDCP

800

900

1,000

RP-DDCP

Figure 6.14: Improvement in BBP of RMSA-AMRA in US26 network using various regenerator placement algorithms in comparison to the RP-FR algorithm In the next Figures, the reason for BBP was furthermore investigated. I performed simulations for dynamic RMSA problem, using RMSA-AMRA algorithm. In each simulation scenario, I was increasing the number of available regenerators in networks, for low (see Figures 6.15 and 6.16), moderate (see Figures 6.17 and 6.18) and high (see Figures 6.19 and 6.20) traffic loads, to reach the point, when rejections where caused only by lack of spectrum. In more details, when the percentage for lack of regenerators is zero, it means that there is no single instance of BBP caused by insufficient number of regenerators. Low traffic loads were equal to 300 ER, moderate to 700 ER and high to 1000 ER. The first interesting observation is the fact that the number of required regenerators for low traffic loads for the US26 network is two times bigger than for the Euro28 network. Moreover, the RP-IDCP was not able to reach the value of 0% BBP due to the lack of regenerators for the same parameters as other algorithms. The best method was RP-DA, which was able to reduce the BBP to 0% for the lowest number of regenerators (around 200). Following the results from Figures 6.15 and 6.16, we can conclude that the number of required regenerators for low traffic loads is around 7 per node in the Euro28 network, and 19 per node in the US26 network. The trend for the moderate traffic loads becomes more similar for both networks. The number of required regenerators for the Euro28 network (see Figure 6.17) is about 100 90

Lack of regenerators (%)

100 80 60 40 20 0

75

50

100

RP-FR

125 150 175 Number of regenerators

RP-DA

RP-IDCP

200

225

250

RP-DDCP

Figure 6.15: The reason of BBP for low traffic loads in Euro28 network

Lack of regenerators (%)

100 80 60 40 20 0 100

200

RP-FR

300 Number of regenerators

RP-DA

RP-IDCP

400

500

RP-DDCP

Figure 6.16: The reason of BBP for low traffic loads in US26 network units lower than for the US26 network (see Figure 6.18). It follows from the fact, that with moderate traffic loads, available spectrum is more important than the available regenerators in the network. The number of required regenerators for the best algorithm (RP-DA) is around 21 for the Euro28 network and around 27 for the US26 network. Finally, the trends for the high traffic loads reveal similar trends as for the moderate traffic loads. The number of required regenerators for the US26 network is higher than for the Euro28 network. Moreover, only the RP-DA algorithm managed to minimize the rejections due to the lack of regenerators to zero, with less than 800 regenerators for the Euro28 network (see 6.19) and less than 1000 regenerators for the US26 network (see 6.20). To conclude, adaptive methods of regenerators’ placement, such as RP-DDCP or RPDA, provide higher gains in regenerators usage than other methods. For further simulations, I decided to use the distribution of the regenerators in the network calculated by the 91

Lack of regenerators (%)

100 80 60 40 20 0 200

300

RP-FR

400 500 Number of regenerators

RP-DA

RP-IDCP

600

700

RP-DDCP

Figure 6.17: The reason of BBP for moderate traffic loads in Euro28 network

Lack of regenerators (%)

100 80 60 40 20 0 200

300

RP-FR

400 500 Number of regenerators

RP-DA

RP-IDCP

600

700

RP-DDCP

Figure 6.18: The reason of BBP for moderate traffic loads in US26 network RP-DA algorithm, as it works better with chosen RMSA-AMRA algorithm. The number of regenerators in DC assignment problem, for both networks, was set as 1000.

92

Lack of regenerators (%)

100 80 60 40 20 0 400

500

RP-FR

600 Number of regenerators

RP-DA

RP-IDCP

700

800

RP-DDCP

Figure 6.19: The reason of BBP for high traffic loads in Euro28 network

Lack of regenerators (%)

100 80 60 40 20 0 500

600

RP-FR

700 800 Number of regenerators

RP-DA

RP-IDCP

900

1,000

RP-DDCP

Figure 6.20: The reason of BBP for high traffic loads in US26 network

6.3

Background Traffic Optimization

In this section, I present the results regarding optimization of static, background traffic in the network. In order to analyze scenarios with background traffic, problem of static requests allocation in the network should be investigated first. The total traffic equal to 2.5 Tbps was generated and allocated in the network using the RMSA-STATIC-VSA algorithm. Then, the simulation of RMSA problem using the RMSA-AMRA algorithm was performed, with the RMSA-DEF-DWD algorithm working in the background thread. Both algorithms are widely described in section 4.3. Before the main simulations, the RMSA-STATIC-VSA algorithm was tuned to provide the best results. To tune RMSA-STATIC-VSA, I examined the relatively small Euro16 topology, available from [224]. I analyze several scenarios referring to different parameters 93

used in the RMSA-STATIC-VSA. I choose 15 cases with different numbers of DCs (from 2-4) and different number of candidate paths (from 2-30). The simulations were conducted to tune the following three parameters of RMSA-STATIC-VSA algorithm: • number of iterations - i; • initial temperature - T , associated with the z parameter; • cooling rate parameter - j. Firstly, the cooling rate (j) and initial temperature (T ), defined using z parameter were tuned as shown in Table 6.2. Table 6.2: Tuning scenarios Scenario No.

j

m

Scenario No.

j

m

Scenario No.

j

m

S1 S2 S3

0.99 0.9995 0.99999

10% 10% 10%

S4 S5 S6

0.999 0.9995 0.99999

5% 5% 5%

S7 S8 S9

0.999 0.9995 0.99999

2.5% 2.5% 2.5%

In Figure 6.21, the average optimality gap is shown for each analyzed scenario. Each individual result is averaged over 15 cases and 10 repetitions of the RMSA-STATIC-VSA algorithm for each case.

Optimality Gap (%)

4

3

2 S1

S2

S3

S4

S5 Scenario

S6

S7

S8

S9

Figure 6.21: Difference between RMSA-STATIC-VSA and optimal results Number of iteration of algorithm was limited to 15,000. The best combination of tuning parameters was obtained in scenario S5. The average difference between RMSA-STATICVSA and optimal results obtained by the CPLEX solver for this scenario was a 2.12% worse than optimal. The results were not improving after passing the average number of 10,000 iterations. Additional simulations were carried out to determine the optimal number of iterations. The T and the z parameters were chosen as in the scenario S5 (see 6.2). The maximum 94

limit of iterations was set to 15,000. As we can see in Table 6.3, the results obtained for 10,000 iterations were almost the same as for the one with a limit of 15,000. The best trade-off between quality of results and execution time is to set i to 10,000. Table 6.3: Tuning of the number of iterations Number of iterations

Average distance from the optimal result

2,500 5,000 7,500 10,000 15,000

21.8% 12.9% 7.52% 2.12% 2.03%

To summarize the tuning process, the following values of the parameters were selected for main simulations: i – 10,000, z – 5%, j – 0.99. Moreover, the performance of RMSA-STATIC-VSA was compared with other heuristics available in the literature, namely First Fit (FF) [6], Most Spectrum First (MSF) [215] and Least Spectrum First (LSF) [215]. It is presented in Figure 6.22.

Percentage (%)

100

98

96

k=2

k=3

FF

k=5 k=10 Number of candidate paths

LSF

k=30

MSF

Figure 6.22: Number of cases in which RMSA-STATIC-VSA was better than FF, MSF and LSF algorithms As it is shown in Figure 6.22, RMSA-STATIC-VSA performed better for larger number of candidate paths. The RMSA-STATIC-VSA uses different solution construction algorithm than all other competing methods, reducing the number of potential solutions. Therefore, the RMSA-STATIC-VSA converges faster to valuable solutions areas when the solution space is large. In the context of dynamic routing, when requests arrive one by one in real time, the main objective is to minimize the rejection of incoming network requests denoted by BBP. Minimization of BBP in the dynamic routing is a difficult issue, because requests are transmitted in real time and should be established as soon as possible to avoid rejection, thus 95

we have to use simpler, less time-consuming algorithms. One of the possible solutions to improve performance of dynamic routing algorithms is a defragmentation algorithm. More specifically, the defragmentation algorithm is launched in the background and optimizes allocation of requests that are already established in the network. The key goal of the defragmentation algorithm is to reconfigure the allocation in terms of routing and spectrum usage to assure better utilization of the network resources. The algorithm that I propose for the defragmentation for both static and dynamic requests is RMSA-DEF-DWD algorithm. In order to test the efficiency of it, the network is filled with the static demands. The input parameters for static demands are in Table 6.4. Table 6.4: Input parameters for static requests Type of flow

Bit-rate

Percentage of all

Additional parameters

Unicast Anycast

10-100 Gbps 10-200 Gbps

60% 40%

2-4 Data Centers

Gains in Blocking Percentage (%)

I choose the percentage of links that have to be assigned and then perform generation of demands (e.g. with network utilization set to 25%, there will be 25% allocated slices in network links). Then, the RMSA-STATIC-VSA algorithm performs optimization of slice usage in links using the Average Spectrum Width ILP model, presented in Section 4.3. After that, the optimized sets of demands are loaded into a network based on particular network utilization values (e.g. 0%, 25%, 50%) with an infinitive lifetime parameter. The requests generated for dynamic traffic allocation are identical to the one shown in Table 6.1.

30

20

10

500

0% initial network load

550

600 Traffic load (Erlang)

25% initial network load

650

700

50% initial network load

Figure 6.23: RMSA-DEF-DWD gains in BBP metric for Euro28 network In the results presented in Figure 6.23, we can see three scenarios for the Euro28 network. In the first one, I test the impact of the RMSA-DEF-DWD algorithm on RMSAAMRA algorithm for dynamic traffic only. This means that the network at the start of the simulation is empty. The primary criterion for assessing the improvement in BBP is comparison to RMSA-AMRA without RMSA-DEF-DWD working in the background. In 96

Gains in Blocking Percentage (%)

the second and the third scenario I used a pre-generated static traffic matrices which uses 25% and 50% of the free slices in the network, respectively. Then, the simulation is started with incoming dynamic requests, with the execution of the RMSA-DEF-DWD algorithm in the background. The first observation is that the gain increases with the heavy network traffic, for the scenario without static requests. More incoming dynamic requests trigger more efficient operation of the defragmentation algorithm. Furthermore, when we look at the difference between non-defragmented use of RMSA-AMRA and defragmented RMSA-AMRA for higher traffic loads, we can observe a 20% gain in the BBP metric. It is because we get more possible requests to reallocate and RMSA-DEF-DWD produces better results. An interesting trend can be observed when we increase the initial network load - improvement becomes more rapid at lower traffic loads and then begins to decrease slowly, especially for higher initial loads. This is connected with the fact that for 50% of the initial network load, there is already very small number of free slices, and thus, the overall efficiency of the RMSA-AMRA and RMSA-DEF-DWD drops. It is also worth mentioning that by using the defragmentation methods for low traffic loads, we obtain the relatively low BBP, in contrast to the cases where those methods are not used. The results for the US26 network, shown in Figure 6.24 confirm previous findings from Figure 6.23. The only difference is that the RMSA-DEF-DWD provides higher gains for the first scenario, and lower gains for scenarios with initial traffic loads, than in Euro28 network due to the different network architecture.

30

20

10

500

0% initial network load

550

600 Traffic load (Erlang)

25% initial network load

650

700

50% initial network load

Figure 6.24: RMSA-DEF-DWD gains in BBP metric for US26 network

6.4

Data Center Assignment

In this Section, I present experimental results and compare performance of the proposed DC requests provisioning algorithms, described in section 4.4. For each simulation the most important indicators were collected: 97

• DC resource utilization; • Optical resource utilization; • Request blocking percentage; • Cost. DC resource utilization is calculated as a ratio of used DC units (CPU, RAM, and storage) to all available units in particular DC. Optical resource utilization is calculated based on two metrics. First, available spectrum in EONs, and the second, optical regenerators installed in the network. The optical utilization is calculated similarly to DC resource utilization, namely as a ratio of used resources to the all available resources. The total request blocking percentage (BP) is calculated as the ratio of rejected requests to all requests. Additionally, the BP can be measured also for selected requests types, e.g. for CPU/RAM/Storage related requests. In each case, the indicator is measured analogously as the ratio of rejected requests of specific type to all requests of that type. The last metric that I am collecting is cost of the simulations. It is a sum of network and DC services cost per hour of service. The reason of this approach is that it is easier compare to other providers for example [29], where prices are provided per hour of service. The traffic model is based on the projection of [2] forecasts. The types of requests presented in these reports are: • Processing as a Cloud (PaaC): Cloud providers deliver a computing platform, typically including an operating system, programming-language execution environment, database, and web server. PaaC requests require CPU (maximum 16 units) and RAM (maximum 64 GB) from DCs. I consider node to DC and international traffic to DC PaaC requests, served by anycast and unicast flows. It represents 17.8% of all traffic, 10-200 Gbps of requested bit-rate. • Storage as a Cloud (SaaC): These type of requests require storage resources from DCs. Cloud storage may be used for copying virtual machine images from the cloud to on-premises locations, to import a virtual machine image from an on-premises location to the cloud image library, or to move virtual machine images between user accounts or between data centers. I consider node to DC, DC to DC, and international traffic to DC SaaC requests, served by all types of flows. It represents 5.6% of all traffic, 100-400 Gbps of requested bit-rate and maximum of 1 TB storage per request. • Software as a Service (SaaS): This is one of the most popular type of cloud services sometimes referred to as “on-demand software”. It is often priced on a pay-per-use or subscription basis. SaaS requires all three types of DC resources. In more detail it uses maximum of 8 CPU units, 32 GB of RAM per request and 100 GB storage per request. I consider node to DC and international traffic to DC SaaS requests, served by unicast and multicast flows. It represents 58% of all traffic, 10-100 Gbps of requested bit-rate. 98

• Optical as a Service (OaaS): It requires storage resources from DCs and optical network resources in network to transfer large amounts of data between nodes and DCs or between DC and DC, served only by anycast flows. It represents 18.6% of all traffic, 100-400 Gbps of requested bit-rate, maximum of 4 CPU units, 4 GB of RAM per request and 4 TB storage per request.

Average Request Blocking Percentage (%)

The Euro28 network is simulated at first. DC-HYBRID and DC-TP algorithms outperform the remaining three algorithms in terms of average request blocking percentage (Figure 6.25). For lower traffic loads (less than 700 Erlangs), DC-TP, DC-HYBRID, and DC-CHEAP algorithms offer satisfactory performance ( 1% blocking percentage). 20

15

10

5

0 500

DC-TP

600

DC-HYBRID

700 800 Traffic load (Erlang)

DC-LU

900

DC-CHEAP

1,000

DC-NEAR

Figure 6.25: Blocking Percentage of the DC allocation algorithms using the Euro28 network topology The DC-LU algorithm rejects approximately 5% requests for traffic load of 600 Erlangs. This algorithm takes into account only one stratum (DC resources). This prevents efficient use of optical resources (see Figure 6.27) because requests are always sent to the least loaded DC, which may result in sending requests over long distances thus over-utilizing optical fiber resources. As a consequence, the DC resource utilization is low while the cost is almost twice as high than other algorithms, as shown in Figure 6.26. The DC-NEAR algorithm also performs poorly. It offers consistent results albeit at unacceptable levels ( 1% blocking percentage). It also takes into account only one stratum (optical resources). By selecting the nearest DC, paths between the source and destination nodes are shorter by ⇡ 20% compared to other algorithms. However, DC resources are used inefficiently. The DC-CHEAP algorithm offers better performance then DC-NEAR and DC-LU. It maintains acceptable blocking percentage (see Figure 6.25) and relatively low cost for traffic loads below 700 Erlangs (see Figure 6.28). For traffic load above 850 Erlangs, its cost is higher than the DC-NEAR algorithm. This is due to longer distances between source and destination nodes. These distances 99

Average DC Resource Utilization (%)

100

80

60

40

20 500

DC-TP

600

DC-HYBRID

700 800 Traffic load (Erlang)

DC-LU

900

DC-CHEAP

1,000

DC-NEAR

Average Optical Resource Utilization (%)

Figure 6.26: DC resource utilization using different DC allocation algorithms and the Euro28 network topology 100

80

60

500

DC-TP

600

DC-HYBRID

700 800 Traffic load (Erlang)

DC-LU

900

DC-CHEAP

1,000

DC-NEAR

Figure 6.27: Optical resource utilization using different DC allocation algorithms and the Euro28 network topology are artificially longer because this algorithm always tries to establish a connection to the cheapest available DC. When the traffic load is high, it results in costly decisions because of overloading the cheapest DC at the beginning of simulations. The two best algorithms are DC-HYBRID and DC-TP. While DC-HYBRID is slightly cheaper than the DC-TP algorithm, at higher traffic loads DC-TP outperforms DC-HYBRID algorithm in terms of blocking percentage. DC-TP utilizes DC resources more efficiently than DC-HYBRID and uses fewer optical resources. Due to occasional “flash crowds”, network traffic may be unpredictable. In such cases, the DC-TP algorithm ensures the best performance and provides a guarantee proper handling of that traffic load peaks. Moreover, the DC-TP algorithm preserves the highest residual network capacity and, hence, 100

Average Cost Per Hour Service (USD)

15

10

5

500

600

DC-TP

700 800 Traffic load (Erlang)

DC-HYBRID

DC-LU

900

DC-CHEAP

1,000

DC-NEAR

Figure 6.28: Cost per hour of service with different DC allocation algorithms and the Euro28 network topology

Average Request Blocking Percentage (%)

larger number of network requests may be served without imposing additional cost. 20

15

10

5

0 300

DC-TP

350

400

DC-HYBRID

450 500 550 Traffic load (Erlang)

DC-LU

600

DC-CHEAP

650

700

DC-NEAR

Figure 6.29: Blocking Percentage of the DC allocation algorithms using the US26 network topology Moreover, the simulation were performed for the US26 network. The DC-HYBRID and DC-TP algorithms significantly outperform the other algorithms that result in over 5% blocking percentage for low and above 10% for moderate traffic loads. These three algorithms in US26 network show inferior performance than in Euro28 network because the paths between nodes are much longer. Furthermore, DCs are concentrated in the East and the West Coasts in US26 network while DCs in the Euro28 network are more centralized. Hence, poor decisions more significantly affect the network performance. They also result in faster depletion of optical resources (see Figure 6.30), leaving the DCs underutilized (see 101

Average DC Resource Utilization (%)

Figure 6.31). We only consider the DC-HYBRID and DC-TP algorithms for traffic loads above 500 Erlangs as other approaches performed poorly over this value.

80

60

40 300

350

DC-TP

400

DC-HYBRID

450 500 550 Traffic load (Erlang)

DC-LU

600

DC-CHEAP

650

700

DC-NEAR

Figure 6.30: DC resource utilization using different DC allocation algorithms and the US26 network topology

Average Optical Resource Utilization (%)

The DC-TP algorithm has the ability to predict traffic and, hence, more efficiently allocate DC and optical resources than the DC-HYBRID algorithm. The blocking probabilities are within acceptable limits ( 1%) for traffic loads below 600 Erlangs and remain low for higher traffic loads. In the case of 700 Erlangs, DC-TP is almost five times better than DC-HYBRID. 100

80

60

40 300

DC-TP

350

400

DC-HYBRID

450 500 550 Traffic load (Erlang)

DC-LU

600

DC-CHEAP

650

700

DC-NEAR

Figure 6.31: Optical resource utilization using different DC allocation algorithms and the US26 network topology The cost of using DC-TP algorithm is slightly higher for low and moderate traffic loads and lower for high traffic loads compared to DC-HYBRID. Similar trends were observed 102

Average Cost Per Hour Service (USD)

in the Euro28 network since both DC-HYBRID and DC-TP algorithms simultaneously optimize two network strata. 10

8

6

4

2 300

DC-TP

350

400

DC-HYBRID

450 500 550 Traffic load (Erlang)

DC-LU

600

DC-CHEAP

650

700

DC-NEAR

Figure 6.32: Cost per hour of service with different DC allocation algorithms and the US26 network topology Finally, the DC assignment algorithms are examined under the scenario with the networks load with static demands (background traffic) in 25%. Those loads are generated as an infinitive lifetime demands and optimized by RMSA-STATIC-VSA algorithm. The offered traffic is composed of two traffic types: background and DC traffic. DC request types are: PaaC, SaaC, SaaS and OaaS. The Figures 6.33 and 6.34 show the simulations results for Euro28 network and US26 network, respectively. Furthermore, the RMSA-DEF-DWD algorithm works in background of the DC algorithms, to provide better fragmentation of resources. Again, the cross stratum algorithms (DC-HYBRID and DC-TP) performed much better than simple, single stratum ones. Even with 25% initial network load, DC-TP managed to provide satisfactory SLA ( 1%) for moderate traffic loads in Euro28 network and for low traffic loads in US26 network. All these results show that CSO can reduce the BP and optimize the network resources utilization much more efficiently. Furthermore, using traffic prediction and knowing the details of available resources of each DC and the current load in the network, one can make better decisions about DCs services provisioning. Those results confirm the thesis of this dissertation. After evaluating various approaches to optimize resource allocation in EONs and DCs using the CSO architecture, I can confirm that simple algorithms, considering only one stratum (optical network or data center), were unable to deliver required performance. More sophisticated algorithms, considering both strata, were able to better coordinate resource allocation, thus improve spectrum utilization efficiency and reduce required network resources.

103

Average Request Blocking Percentage (%)

20

10

0 500

DC-TP

550

DC-HYBRID

600 Traffic load (Erlang)

DC-LU

650

DC-CHEAP

700

DC-NEAR

Average Request Blocking Probability (%)

Figure 6.33: Blocking Percentage of the DC allocation algorithms using the Euro28 network topology, with 25% initial network load

20

10

0 300

DC-TP

350

DC-HYBRID

400 Traffic load (Erlang)

DC-LU

450

DC-CHEAP

500

DC-NEAR

Figure 6.34: Blocking Percentage of the DC allocation algorithms using the US26 network topology, with 25% initial network load

6.5

Path Protection

In addition to unprotected provisioning of cloud services, I performed several simulations to prove the trends from the previous section, for the scenarios when we need to protect cloud services. In order to protect network connections, backup connectivity in the optical network is provided. For network operators, protecting optical links is most important. We assume that provider (e.g. Amazon) is responsible for protection of DC infrastructure. The AWS infrastructure is available 100% of the time. During the simulation, we randomly choose single links and mark them as broken. 104

The single link failure frequency is based on the model presented in [21]. To examine the influence of network failures on performance of the analyzed protection methods in more detail, parameter is introduced. In particular, parameter denotes the frequency of link failures in the network. For instance, = 0 shows that there are no failures, while = 1 means that the failure frequency is equal to the average lifetime of the request. For experiments in this thesis, I have chosen the = 0.1. It was showed in [15] that this value of provide the most reliable results, thus, the differences between various protection are more visible. Moreover, it is assumed that after a failure, the broken link needs to be repaired and the repair time is ten times of the average lifetime of the request. Depending on the protection type, algorithms then assign appropriate backup paths. The type of requests is similar to the one presented in section 6.4.

Average Request Blocking Percentage (%)

First, the BP metric for the Euro28 and US26 networks is presented in Figure 6.35 and Figure 6.36, respectively. Next, I compare the cost of using various protection methods in Figure 6.37 and Figure 6.38. 10 8 6 4 2 0 500

550

600 650 Traffic load (Erlang)

DC-HYBRID/NP DC-HYBRID/SBPP DC-TP/DPP

700

750

DC-HYBRID/DPP DC-TP/NP DC-TP/SBPP

Figure 6.35: Blocking Percentage of the DC allocation algorithms using the Euro28 network topology and various protection methods The DC-Hybrid and DC-TP algorithms are simulated with: no protection (NP), dedicated path protection (DPP), and shared backup path protection (SBPP). The DC-TP method provides lower request blocking percentage at the cost of higher price per hour of service than DC-HYBRID algorithm. The DC-TP calculations and decisions required to lower BP are more expensive, but guarantee better final results. Moreover, DPP provides the best results for low traffic loads while SBPP performs better with moderate and higher traffic loads. This is due to a fact that DPP allocates more resources for backup connectivity than SBPP, but provides 1+1 protection. It means that we are always protected in case of single-link failure, and we are also protected in some multi-link failures. The only case of a non-protected multi-link failure is when the primary 105

Average Request Blocking Percentage (%)

20

10

0 500

550

600 Traffic load (Erlang)

DC-HYBRID/NP DC-HYBRID/SBPP DC-TP/DPP

650

700

DC-HYBRID/DPP DC-TP/NP DC-TP/SBPP

Average Cost Per Hour Service (USD)

Figure 6.36: Blocking Percentage of the DC allocation algorithms using the US26 network topology and various protection methods 9

8

7

6

5 500

550

600 650 Traffic load (Erlang)

DC-HYBRID/NP DC-HYBRID/SBPP DC-TP/DPP

700

750

DC-HYBRID/DPP DC-TP/NP DC-TP/SBPP

Figure 6.37: Cost per hour of service with different DC allocation algorithms and the Euro28 network topology, using various protection methods and backup path/tree of one request are broken at the same time. It causes congestion around particular nodes in the network, with high traffic loads, and thus increase the BBP. With lower traffic loads, network links are less utilized and additional spectrum can be dedicated for backups. However, the SBPP is much cheaper than DPP. While the cheapest algorithm is to not provide path protection, it results in higher blocking percentage due to a higher number of lost requests after network failures. 106

Average Cost Per Hour Service (USD)

25

20

15

10

5 500

550

600 Traffic load (Erlang)

DC-HYBRID/NP DC-HYBRID/SBPP DC-TP/DPP

650

700

DC-HYBRID/DPP DC-TP/NP DC-TP/SBPP

Figure 6.38: Cost per hour of service with different DC allocation algorithms and the US26 network topology, using various protection methods The trends are similar for both networks. The only difference is that the performance of all protection methods in US26 network is inferior to their performance in the Euro28 network. The nodes are concentrated in the East and the West Coasts in the US26 network while in the Euro28 network are more centralized. Hence, because of larger distances in US26 network, different decisions more significantly affect the network performance.

107

Part V Summary and Future Research

109

Chapter

7

Conclusion and Summary

I

n this dissertation the possibility of increasing the optical spectrum utilization in cloudready Elastic Optical Networks while reducing the number of required network resources was investigated. More specifically, the cross stratum and single stratum provisioning strategies for data center resource allocation were proposed. Elastic Optical Networks appear as a promising solution to the optical transport networks. By breaking the fixed-grid spectrum allocation limit of conventional WDM networks, EONs increase the flexibility in requests provisioning. In addition, the Software Defined Networks concept allows better coordination among the optical and data center strata. As a result, the Software Defined Elastic Optical Networks framework is achieved that uses Cross Stratum Optimization architecture. This enables Optical as a Service realizing both spectrum and application elasticity to meet the increasing requirements for transport networks. The three main parts of the solution ensure that requirements constrained by the cloud architecture and paradigms are respected. First, the dynamic routing algorithms for joint unicast, anycast and multicast flows are discussed. The best algorithm is then chosen for data center allocation strategies. Moreover, the regenerator placement problem is investigated. It is possible to increase the efficiency of optical layer, by proper setup of regenerators in the optical networks. Again, the best strategy is then used to generate regenerators locations for data center resource allocation problem. Finally, the abovementioned data center resource assignment algorithms are proposed, beginning with single stratum approaches, followed by cross stratum ones. The performance of the proposed solution was examined using a custom-built simulation tool. The simulator was described in detail and the credibility of the results was discussed. The simulations have been performed using two-well known network topologies: Euro28 and US26. Moreover, different values of input parameters were investigated in various simulation scenarios. The proposed solution should also be considered with regard to the real-life, deployment possibilities. The discussed architecture is easily adapted to majority of modern cloud networks solutions, such as Amazon Web Services or Microsoft Azure. The locations of data centers and pricing for their cloud services is provided by Amazon Web Services, however changing the cloud services operator should not raise any concerns. Moreover, the centralized control of data centers using Software Defined Networks controller is a very common approach for IT resources provisioning. 111

7.1

Achievements and Contributions

The main achievements and contributions of this dissertation are: 1. Introduction of the modern cloud architecture from the perspective of the network operator. It is assumed that network topology and SDN controller is maintained by the operator that wants to serve as many clients’ cloud requests as possible. The cloud data centers locations and pricing are provided from external operator and are assumed to be available at all times. Furthermore, the cloud request flow reflects the on-demand character of the architecture. 2. Proposition of a novel RMSA dynamic routing algorithm that includes a possibility to change the modulation format in the network nodes on the lightpath. The best tradeoff between blocking percentage and network costs following from the usage of extra regenerators is obtained when we implement adaptive methods to choose modulation formats and routing paths. This approach provides relatively good results in both main performance metrics: bandwidth blocking percentage and usage of regenerators. In addition, adaptive penalties for using modulation formats and constant penalties for using regenerators provide the best performance in comparison to simple method, for example finding the shortest path between request origin and destination. 3. Presentation of the results for minimum network resources requirements to provide a required level of SLA in the context of dynamic routing with unicast, anycast and multicast requests. It is showed that with growing data center traffic, transition from WDM to EONs is only a matter of time. 4. Investigation of the regenerator placement problem in cloud-ready Elastic Optical Networks. Various regenerators’ placement algorithms are proposed and evaluated in extensive simulations. The best regenerators’ placement algorithm includes information on DC location, network link lengths and ratio of anycast traffic in the network, when calculates location of the regenerators. This approach provides relatively good impact on routing algorithms in both main performance metrics: blocking percentage and usage of regenerators. It is also shown that increasing the number of regenerators can bring savings in the spectrum usage, especially for higher traffic loads. 5. Proposition of an efficient spectral defragmentation algorithm named for unicast, anycast and multicast flows. The main advantage of it is ability to work with static and dynamic flows simultaneously, maintaining a quick processing times. This algorithm optimizes resources, while launched in the background. 6. Evaluation of the Average Spectrum Width ILP model, under static traffic scenarios for background traffic generation. 7. Comparison of the various approaches to optimize resource allocation in Elastic Optical Networks and data centers using the Cross Stratum Optimization architecture. 112

Simple algorithms (cost, distance, or utilization) that consider only one stratum (optical network or data center) were unable to deliver required performance. Therefore, the cross stratum algorithms are proposed, which enable a better coordination of resource allocation in both strata, thus resulting in lower blocking percentage. The cost for using cross stratum algorithms is slightly higher than the cost of using single stratum ones. Albeit delivering a more costly solution, cross stratum techniques results in more efficient utilization of optical and data center resources. 8. Proposition of the traffic prediction mechanism based on the Monte Carlo Tree Search algorithm. The results show that using such an approach allows to more efficiently allocate DC and optical resources compared to other approaches. 9. An implementation of a custom simulation tool for data center resource allocation. 10. Extensive simulation experiments on all the proposed mechanisms, performed using computer simulations, to verify research thesis. The complete solution was composed using an incremental approach, in order to improve understanding. Following the research conducted in this dissertation, it can be stated that the thesis from the section 1.1: Cloud-ready and cross-stratum provisioning strategies enable optical as a service and increase the optical spectrum utilization efficiency, thereby reducing the number of required network resources, has been confirmed.

7.2

Future Works

For future work, the following research directions are proposed: • further analysis of the current and future optical and data center network resource requirements; • design and implementation of algorithms for resource allocation in a cloud data centers using various heuristics and machine learning techniques; • design and implementation of algorithms for other physical models that can be implemented in Elastic Optical Networks; • design and development of traffic prediction mechanisms based on the different algorithms than Monte Carlo Tree Search, for example using the Las Vegas algorithm; • evaluation of the new cloud data center types and new network topologies (Asia); • further research on path protection, more specifically on multi link failures caused by natural disasters; • investigation of an in-data center allocation policies.

113

Bibliography [1] ITU, “ICT Facts and figures 2016,” Tech. Rep., 2016. [2] Cisco Systems, “Cisco Visual Networking Index: Forecast and Methodology,” pp. 1–41, 2016. [Online]. Available: http://www.cisco.com/c/en/us/solutions/collateral/service-provider/ip-ngnip-next-generation-network/white{_}paper{_}c11-481360.pdf [3] ——, “Cisco Global Cloud Index: Forecast and Methodology,” pp. 1 – 41, 2016. [Online]. Available: http://www.cisco.com/en/US/solutions/collateral/ns341/ns525/ ns537/ns705/ns1175/Cloud{_}Index{_}White{_}Paper.html{#}wp9000816 [4] L. Zong, G. N. Liu, A. Lord, Y. R. Zhou, and T. Ma, “40/100/400 Gb/s Mixed Line Rate Transmission Performance in Flexgrid Optical Networks,” in Optical Fiber Communication Conference, Anaheim, USA, 2013. [5] M. Jinno, H. Takara, B. Kozicki, Y. Tsukishima, Y. Sone, and S. Matsuoka, “Spectrum-efficient and scalable elastic optical path network: Architecture, benefits, and enabling technologies,” IEEE Communications Magazine, vol. 47, no. 11, pp. 66–73, 2009. [6] M. Jinno, B. Kozicki, H. Takara, A. Watanabe, Y. Sone, T. Tanaka, and A. Hirano, “Distance-adaptive spectrum resource allocation in spectrum-sliced elastic optical path network,” IEEE Communications Magazine, vol. 48, no. 8, pp. 138–145, 2010. [7] M. Aibin and K. Walkowiak, “Adaptive modulation and regenerator-aware dynamic routing algorithm in elastic optical networks,” in IEEE International Conference on Communications (ICC), London, UK, 2015, pp. 5138–5143. [8] H. U. I. Zang, J. P. Jue, and B. Mukherjee, “A Review of Routing and Wavelength Assignment Approaches for Wavelength- Routed Optical WDM Networks,” Optical Networks Magazine, vol. 1, pp. 47–60, 2000. [9] K. Walkowiak, “QoS Dynamic Routing in Content Delivery Networks,” Networking, vol. 346, no. Lectures in Computer Science, pp. 1120–1132, 2005. [10] L. Sahasrabuddhe and B. Mukherjee, “Light trees: optical multicasting for improved performance in wavelength routed networks,” IEEE Communications Magazine, vol. 37, no. 2, pp. 67–73, 1999. 115

[11] M. Aibin and K. Walkowiak, “Dynamic routing of anycast and unicast traffic in elastic optical networks with various modulation formats - Trade-off between blocking probability and network cost,” in 15th International Conference on High Performance Switching and Routing (HPSR), Vancouver, Canada, 2014, pp. 64–69. [12] ——, “Simulated Annealing algorithm for optimization of elastic optical networks with unicast and anycast traffic,” in International Conference on Transparent Optical Networks, Graz, Austria, 2014, pp. 2–5. [13] ——, “Regenerator Placement Algorithms for Cloud-Ready Elastic Optical Networks,” in 17th International Conference on Transparent Optical Networks (ICTON), Budapest, Hungary, 2015, pp. 2–5. [14] M. Aibin and M. Blazejewski, “Complex Elastic Optical Network Simulator (CEONS),” in 17th International Conference on Transparent Optical Networks (ICTON), Budapest, Hungary, 2015, pp. 1–4. [15] M. Aibin and K. Walkowiak, “Adaptive survivability algorithm for path protection with various traffic classes in elastic optical networks,” in 7th International Workshop on Reliable Networks Design and Modeling, Munich, Germany, 2015, pp. 56–62. [16] ——, “Defragmentation algorithm for joint dynamic and static routing problems in elastic optical networks with unicast and anycast traffic,” in International Conference on Computing, Networking and Communications (ICNC), Kauai, Hawaii, USA, 2016, pp. 1–5. [17] ——, “Resource requirements in fixed-grid and flex-grid networks for dynamic provisioning of data center traffic,” in IEEE Canadian Conference on Electrical and Computer Engineering (CCECE), Vancouver, Canada, 2016, pp. 1–4. [18] M. Aibin, R. Goscien, and K. Walkowiak, “Multicasting versus anycasting: How to efficiently deliver content in elastic optical networks,” in 18th International Conference on Transparent Optical Networks (ICTON), Trento, Italy, 2016, pp. 1–4. [19] M. Furdek, L. Wosinska, R. Goscien, K. Manousakis, M. Aibin, K. Walkowiak, S. Ristov, M. Gushev, and J. Marzo, “An overview of security challenges in communication networks,” in 8th International Workshop on Resilient Networks Design and Modeling, Halmstad, Sweden, 2016. [20] M. Aibin and K. Walkowiak, “Different strategies for dynamic multicast traffic protection in elastic optical networks,” in 8th International Workshop on Resilient Networks Design and Modeling, Halmstad, Sweden, 2016, pp. 1–4. [21] M. Aibin, K. Walkowiak, and A. Sen, “Software-defined adaptive survivability for elastic optical networks,” Optical Switching and Networking, vol. 23, pp. 85–96, 1 2017. 116

[22] M. W. Przewozniczek, K. Walkowiak, and M. Aibin, “The evolutionary cost of Baldwin effect in the routing and spectrum allocation problem in elastic optical networks,” Applied Soft Computing, vol. 52, pp. 843–862, 2017. [23] ——, “The Effectiveness of the Simplicity in Evolutionary Computation,” in 9th Asian Conference on Intelligent Information and Database Systems, Kanazawa, Japan, 2017, pp. 1–4. [24] M. Aibin, S. Haeri, K. Walkowiak, and L. Trajkovic, “Cross-Stratum Optimization for Provisioning of Data Center Services,” Journal of Optical Communications and Networking (under review), 2017. [25] J. Geelan, “Twenty-One Experts Define Cloud Computing,” Cloud Computing Journal, p. 5, 2009. [26] S. P. Mirashe and N. V. Kalyankar, “Cloud Computing,” Communications of the ACM, vol. 51, no. 7, p. 9, 2010. [27] I. Foster, Y. Zhao, I. Raicu, and S. Lu, “Cloud computing and grid computing 360degree compared,” in Grid Computing Environments Workshop, 2008, pp. 1–10. [28] R. Buyya, C. S. Yeo, and S. Venugopal, “Market-oriented cloud computing: Vision, hype, and reality for delivering IT services as computing utilities,” in IEEE International Conference on High Performance Computing and Communications, 2008, pp. 5–13. [29] Amazon Web Services, “Elastic Compute Cloud (EC2) Cloud Server & Hosting – AWS,” 2016. [Online]. Available: https://aws.amazon.com/ec2/ [30] Google, “Google App Engine,” 2016. [Online]. Available: http://appengine.google. com [31] Microsoft, “One Drive,” 2016. [Online]. Available: https://onedrive.live.com/ [32] K. Visala, D. Lagutin, and S. Tarkoma, “LANES: an inter-domain data-oriented routing architecture,” in Proc. of ReArch, 2009, pp. 55–60. [33] V. Jacobson, D. K. Smetters, J. D. Thornton, M. Plass, N. Briggs, and R. Braynard, “Networking named content,” Communications of the ACM, vol. 55, no. 1, p. 117, 2012. [34] T. Koponen, M. Chawla, B.-G. Chun, A. Ermolinskiy, K. H. Kim, S. Shenker, and I. Stoica, “A data-oriented (and beyond) network architecture,” in ACM SIGCOMM Computer Communication Review, vol. 37, no. 4, 2007, p. 181. [35] J. Choi, J. Han, E. Cho, T. T. Kwon, and Y. Choi, “A survey on content-oriented networking for efficient content delivery,” IEEE Communications Magazine, vol. 49, no. 3, pp. 121–127, 2011. 117

[36] B. Ahlgren, C. Dannewitz, C. Imbrenda, D. Kutscher, and B. Ohlman, “A survey of information-centric networking,” IEEE Communications Magazine, vol. 50, no. 7, pp. 26–36, 7 2012. [37] G. Carofiglio, G. Morabito, L. Muscariello, I. Solis, and M. Varvello, “From content delivery today to information centric networking,” Computer Networks, vol. 57, no. 16, pp. 3116–3127, 2013. [38] S.-Z. Yu, W.-T. Wu, Y. Gao, and W.-X. Liu, “Caching efficiency of informationcentric networking,” IET Networks, vol. 2, no. 2, pp. 53–62, 2013. [39] G. Held, A Practical Guide to Content Delivery Networks. Raton, FL, USA, 2011.

CRC Press, Inc. Boca

[40] G. Peng, “CDN: Content Distribution Network,” Tech. Rep., 2004. [41] I. Tomkos, B. Mukherjee, S. K. Korotky, R. Tucker, and L. Lunardi, “The Evolution of Optical Networking,” Proceedings of the IEEE, vol. 100, no. 5, pp. 1017–9219, 2012. [42] J. M. Simmons, Optical Network Design and Planning, ser. Optical Networks. Springer International Publishing, 2014, no. 2nd Edition. [43] I. Chlamtac, A. Ganz, and G. Karmi, “Lightpath communications: An approach to high bandwidth optical WAN’s,” IEEE Transactions on Communications, vol. 40, no. 7, pp. 1171–1182, 1992. [44] O. Gerstel, “On The Future of Wavelength Routing Networks,” IEEE Network, vol. 96, no. 11, pp. 14–20, 1996. [45] P. Monteiro, A. Sousa, and C. B. Lopes, “Design Cost and Spectrum Efficiency Comparison of Fixed-Grid and Flex-Grid Optical Networks with Grooming,” in International Telecommunications Network Strategy and Planning Symposium, 2014, pp. 1–4. [46] M. Jinno, H. Takara, and B. Kozicki, “Concept and Enabling Technologies of Spectrum-Sliced Elastic Optical Path Network (SLICE),” in Asia Communications and Photonics Conference and Exhibition, Shanghai, China, 2009. [47] ——, “Dynamic optical mesh networks: Drivers, challenges and solutions for the future,” in 35th European Conference on Optical Communication, Vienna, Austria, 2009, pp. 2–5. [48] I. Tomkos, S. Azodolmolky, J. Sole-Pareta, D. Careglio, and E. Palkopoulou, “A tutorial on the flexible optical networking paradigm: State of the art, trends, and research challenges,” in Proceedings of the IEEE, vol. 102, no. 9, 2014, pp. 1317–1337. [49] ITU-T, “Spectral grids for WDM applications: DWDM frequency grid,” 2012. [Online]. Available: http://www.itu.int/rec/T-REC-G.694.1-201202-I/en 118

[50] A. Vakali and G. Pallis, “Content Delivery Networks: Status and Trends,” IEEE Internet Computing, vol. 7, no. 6, pp. 68–74, 2003. [51] S. Chandrasekhar, X. L. X. Liu, B. Zhu, and D. W. Peckham, “Transmission of a 1.2-Tb/s 24-carrier no-guard-interval coherent OFDM superchannel over 7200-km of ultra-large-area fiber,” in European Conference on Optical Communication, 2009, pp. 7–8. [52] O. Gerstel, M. Jinno, A. Lord, and S. J. B. Yoo, “Elastic optical networking: A new dawn for the optical layer?” IEEE Communications Magazine, vol. 50, no. 2, pp. 12–20, 2012. [53] M. Tahon, S. Verbrugge, D. Colle, M. Pickavet, P. Demeester, P. Wright, and A. Lord, “Valuing Flexibility in the Migration to Flexible-Grid Networks,” Journal of Optical Communications and Networking, vol. 5, no. 10, 2013. [54] H. Waldman, R. C. A. Jr, K. D. R. Assis, and R. C. Bortoletto, “Spectrum-Sliced Elastic Optical Networking,” in International Conference on Transparent Optical Networks, Cartagena, Spain, 2013, pp. 3–6. [55] Y. Sone, a. Watanabe, W. Imajuku, Y. Tsukishima, B. Kozicki, H. Takara, and M. Jinno, “Bandwidth Squeezed Restoration in Spectrum-Sliced Elastic Optical Path Networks (SLICE),” IEEE/OSA Journal of Optical Communications and Networking, vol. 3, no. 3, pp. 223–233, 2011. [56] A. Tarhan and C. Cavdar, “Shared Path Protection for Distance Adaptive Elastic Optical Networks under Dynamic Traffic,” in International Congress on Ultra Modern Telecommunications and Control Systems and Workshops, Almaty, Kazahstan, 2013, pp. 62–67. [57] K. Walkowiak and M. Klinkowski, “Shared Backup Path Protection in Elastic Optical Networks: Modeling and Optimization,” in International Conference on Design of Reliable Communication Networks, Budapest, Hungary, 2013, pp. 187–194. [58] M. Klinkowski and K. Walkowiak, “Offline RSA algorithms for elastic optical networks with dedicated path protection consideration,” in International Congress on Ultra Modern Telecommunications and Control Systems and Workshops, St. Petersburg, Russia, 2012, pp. 670–676. [59] J. López Vizcaíno, Y. Ye, V. López, F. Jiménez, F. Musumeci, M. Tornatore, A. Pattavina, and P. M. Krummrich, “Protection in optical transport networks with fixed and flexible grid: Cost and energy efficiency evaluation,” Optical Switching and Networking, vol. 11, pp. 55–71, 1 2014. [60] L. Velasco, M. Klinkowski, M. Ruiz, V. López, and G. Junyent, “Elastic Spectrum Allocation for Variable Traffic in Flexible-Grid Optical Networks,” in Optical Fiber Communication Conference and Exposition and the National Fiber Optic Engineers Conference (OFC/NFOEC), Los Angeles, USA, 2012. 119

[61] A. Fallahpour, H. Beyranvand, S. A. Nezamalhosseini, and J. a. Salehi, “Energy Efficient Routing and Spectrum Assignment With Regenerator Placement in Elastic Optical Networks,” Journal of Lightwave Technology, vol. 32, no. 10, pp. 2019–2027, 2014. [62] A. Fallahpour, H. Beyranvand, and J. A. Salehi, “Energy-Efficient Manycast Routing and Spectrum Assignment in Elastic Optical Networks for Cloud Computing Environment,” Journal of Lightwave Technology, vol. 33, no. 19, pp. 4008–4018, 2015. [63] S. Zhang and B. Mukherjee, “Energy-Efficient Dynamic Provisioning for Spectrum Elastic Optical Networks,” in International Conference on Communications ICC, Ottawa, Canada, 2012, pp. 3031–3035. [64] B. A. A. Nunes, M. Mendonca, X. N. Nguyen, K. Obraczka, and T. Turletti, “A survey of software-defined networking: Past, present, and future of programmable networks,” IEEE Communications Surveys and Tutorials, vol. 16, no. 3, pp. 1617– 1634, 2014. [65] R. Jain and P. Subharthi, “Network Virtualization and Software Defined Networking for Cloud Computing: A Survey,” Communications Magazine, IEEE, vol. 51, no. November, pp. 24–31, 2013. [66] C. E. Rothenberg, S. Azodolmolky, and S. Uhlig, “Software-Defined Networking : A Comprehensive Survey,” in Proceedings of the IEEE, vol. 103, no. 1, 2015, pp. 14 – 76. [67] B. Hu, K. L. Yeung, and C. He, “On Iterative Scheduling for Input-queued Switches with a Speedup of 2-1 / N,” in IEEE 15th International Conference on High Performance Switching and Routing (HPSR), Vancouver, Canada, 2014, pp. 26–31. [68] Y. Jarraya, T. Madi, and M. Debbabi, “A survey and a layered taxonomy of softwaredefined networking,” IEEE Communications Surveys and Tutorials, vol. 16, no. 4, pp. 1955–1980, 2014. [69] M. Yang, Y. Li, D. Jin, L. Zeng, X. Wu, and A. V. Vasilakos, “Software-Defined and Virtualized Future Mobile and Wireless Networks: A Survey,” Mobile Networks and Applications, vol. 20, no. 1, pp. 4–18, 2015. [70] Z. Zhu, C. Chen, X. Chen, S. Ma, L. Liu, X. Feng, and S. B. Yoo, “Demonstration of Cooperative Resource Allocation in an OpenFlow-Controlled Multi-Domain and Multinational SD-EON Testbed,” Journal of Lightwave Technology, vol. 33, no. 8, pp. 1508 – 1514, 2015. [71] R. Jain, “OpenFlow, Software Defined Networking (SDN) and Network Function Virtualization (NFV),” in IEEE International Conference on Communications (ICC), 2014, pp. 1–102. 120

[72] B. Fraser, D. Lake, C. Systems, J. Finnegan, N. Viljoen, and S. O. E. N. Etworking, “Are We Ready for SDN ? Implementation Challenges for Software-Defined Networks,” IEEE Communications Magazine, vol. 51, no. 7, pp. 36–43, 2013. [73] X. Yi, F. Liu, J. Liu, and H. Jin, “Building a network highway for big data: Architecture and challenges,” IEEE Network, vol. 28, no. 4, pp. 5–13, 2014. [74] K. Giotis, G. Androulidakis, and V. Maglaris, “Leveraging SDN for efficient anomaly detection and mitigation on legacy networks,” in 3rd European Workshop on Software-Defined Networks, EWSDN, 2014, pp. 85–90. [75] A. Akhunzada, E. Ahmed, A. Gani, M. K. Khan, M. Imran, and S. Guizani, “Securing software defined networks: taxonomy, requirements, and open issues,” IEEE Communications Magazine, vol. 53, no. 4, pp. 36–44, 4 2015. [76] M. Banikazemi, D. Olshefski, A. Shaikh, J. Tracey, and G. Wang, “Meridian: An SDN platform for cloud network services,” IEEE Communications Magazine, vol. 51, no. 2, pp. 120–127, 2013. [77] R. Buyya, J. Broberg, and A. Goscinski, Cloud Computing. Inc., 2011.

John Wiley & Sons,

[78] H. HuiYang, Y. Zhao, J. Zhang, Y. Lee, Y. Lin, and F. Zhang, “Cross Stratum Optimization Architecture for Optical as a Service,” IEEE Communications Magazine, vol. 15, pp. 130–140, 2015. [79] J. Zhang, H. Yang, Y. Zhao, Y. Ji, H. Li, Y. Lin, G. Li, J. Han, Y. Lee, and T. Ma, “Experimental demonstration of elastic optical networks based on enhanced software defined networking (eSDN) for data center application,” Optics Express, vol. 21, no. 22, pp. 26 990–27 002, 2013. [80] J. Santos, “On the Impact of Deploying Federated SDN Controllers in Optical Transport Networks,” in 18th International Conference on Transparent Optical Networks (ICTON), Trento, Italy, 2016, pp. 5–7. [81] H. Yang, J. Zhang, Y. Zhao, Y. Ji, H. Li, Y. Lin, G. Li, J. Han, Y. Lee, and T. Ma, “Performance evaluation of time-aware enhanced software defined networking ( TeSDN ) for elastic data center optical interconnection,” Opt. Express, vol. 22, no. 15, pp. 17 630–17 643, 2014. [82] J. R. d. A. Amazonas, G. Santos-Boada, S. Ricciardi, and J. Sole-Pareta, “Technical challenges and deployment perspectives of SDN based elastic optical networks,” in International Conference on Transparent Optical Networks (ICTON), Trento, Italy, 2016, pp. 1–5. [83] H. Yang, Y. Zhao, J. Zhang, J. Wu, J. Han, Y. Lin, Y. Lee, and Y. Ji, “Multi-stratum resilience with resources integration for software defined data center interconnection based on IP over elastic optical networks,” in European Conference on Optical Communication, ECOC, vol. 18, no. 10, 2014, pp. 1735–1738. 121

[84] F. Hu, Q. Hao, and K. Bao, “A Survey on Software Defined Networking (SDN) and OpenFlow: From Concept to Implementation,” IEEE Communications Surveys & Tutorials, vol. 16, 2014. [85] C. T. Politi, V. Anagnostopoulos, C. Matrakidis, A. Stavdas, A. Park, M. Heath, and U. Kingdom, “Dynamic Operation of Flexi-Grid OFDM-based Networks,” Optical Fiber Communication Conference and Exposition and the National Fiber Optic Engineers Conference (OFC/NFOEC), vol. 1, 2012. [86] K. Walkowiak, Modeling and Optimization of Cloud-Ready and Content-Oriented Networks. Springer International Publishing, 2016, vol. 56, no. Decision and Control. [87] M. Pióro and D. Medhi, Routing, Flow, and Capacity Design in Communication and Computer Networks. Elsevier, 2004. [88] R. Wilson, “Introduction to graph theory,” Discrete Mathematics, vol. 37, no. 1, p. 133, 1981. [89] B. Bollobas, “Modern graph theory,” Computers & Mathematics with Applications, vol. 37, no. 3, p. 136, 2 1998. [90] J. A. Bondy and U. S. R. Murty, Graph theory with applications. vol. 290.

Springer, 1976,

[91] S. Nelakuditi, Z. L. Zhang, and D. H. C. Du, “On selection of candidate paths for proportional routing,” Computer Networks, vol. 44, no. 1, pp. 79–102, 2004. [92] E. Dijkstra, “A note on two problems in connexion with graphs,” Numerische Mathematik, vol. 1, no. 1, pp. 269–271, 1959. [93] L. R. Ford and D. R. Fulkerson, “Maximal flow through a network,” Journal canadien de mathématiques, vol. 8, pp. 399–404, 1956. [94] P. E. Hart and J. Nils, “A Formal Basis for the Heuristic Determination of Minimum Cost Paths,” IEEE Transactions on Systems Science and Cybernetics, no. 2, pp. 100–107, 1968. [95] V. J. Rayward-Smith, T. H. Cormen, C. E. Leiserson, and R. L. Rivest, “Introduction to Algorithms,” The Journal of the Operational Research Society, vol. 42, no. 9, p. 816, 1991. [96] C. Lam, H. Liu, B. Koley, X. Zhao, V. Kamalov, and V. Gill, “Fiber optic communication technologies: What’s needed for datacenter network operations,” IEEE Communications Magazine, vol. 48, no. 7, pp. 32–39, 2010. [97] M. M. Klinkowski and K. Walkowiak, “On the advantages of elastic optical networks for provisioning of cloud computing traffic,” Network, IEEE, vol. 27, no. 6, pp. 44–51, 2013. 122

[98] C. Kachris, K. Kanonakis, and I. Tomkos, “Optical interconnection networks in data centers: Recent trends and future challenges,” IEEE Communications Magazine, vol. 51, no. 9, pp. 39–45, 2013. [99] S. Azodolmolky, P. Wieder, and R. Yahyapour, “Cloud computing networking: Challenges and opportunities for innovations,” IEEE Communications Magazine, vol. 51, no. 7, pp. 54–62, 2013. [100] T. Wang, Z. Su, Y. Xia, and H. Mounir, “Rethinking the Data Center Networking : Architecture , Network Protocols , and Resource Sharing,” IEEE Access, vol. 2, pp. 1481–1496, 2014. [101] T. Koch and A. Martin, “Solving Steiner tree problems in graphs to optimality,” Networks, vol. 32, no. 3, pp. 207–232, 1998. [102] A. Smutnicki and K. Walkowiak, “A new approach to optimization of p-cycle protected multicast optical networks,” in International Congress on Ultra Modern Telecommunications and Control Systems and Workshops, 2013, pp. 74–81. [103] Y. Zhang, X. Gong, Y. Hu, W. Wang, and X. Que, “SDNMP : Enabling SDN Management Using Traditional NMS,” in IEEE ICC 2015 - Workshop on Advances in Software Defined and Context Aware Cognitive Networks 2015 (IEEE SCAN-2015) SDNMP:, 2015, pp. 357–362. [104] G.-w. Lu, T. Sakamoto, and T. Kawanishi, “Flexible High-Order QAM Transmitters for Elastic Optical Networks,” Photonic Network Communications, vol. 31, no. 1, 2016. [105] H. Y. Choi, T. Tsuritani, and I. Morita, “Multi-format and multi-rate transmitter for flexible and elastic optical networks,” in Opto-Electronics and Communications Conference, OECC, 2012, pp. 773–774. [106] M. Klinkowski and K. Walkowiak, “Routing and spectrum assignment in spectrum sliced elastic optical path network,” IEEE Communications Letters, vol. 15, no. 8, pp. 884–886, 2011. [107] B. C. Chatterjee, N. Sarma, and E. Oki, “Routing and Spectrum Allocation in Elastic Optical Networks: A Tutorial,” IEEE Communications Surveys & Tutorials, vol. 17, no. 3, pp. 1776–1800, 2015. [108] K. Shiomoto, E. Oki, and W. Imajuku, “Distributed Virtual Network Topology Control Mechanism in GMPLS-Based Multiregion Networks,” IEEE Journal on Selected Areas in Communications, vol. 21, no. 8, pp. 1254–1262, 2003. [109] L. Lei and Fujitsu Laboratories of America, “SDN Orchestration for Dynamic Endto-End Control of Data Center Multi-Domain Optical Networking,” IEEE Communications, vol. 12, no. 8, pp. 10–21, 2013. 123

[110] L. Zhang, W. Lu, X. Zhou, and Z. Zhu, “Dynamic RMSA in spectrum-sliced elastic optical networks for high-throughput service provisioning,” in International Conference on Computing, Networking and Communications (ICNC), San Diego, USA, 2013, pp. 380–384. [111] X. Zhou, W. Lu, L. Gong, and Z. Zhu, “Dynamic RMSA in elastic optical networks with an adaptive genetic algorithm,” in IEEE Global Communications Conference (GLOBECOM), Anaheim, USA, 2012, pp. 2912–2917. [112] B. Ellison, D. Fisher, R. Linger, H. Lipson, T. Longstaff, and N. Mead, “Survivable Network Systems : An Emerging Discipline,” Technical Report CMU/SEI-97-TR013, Tech. Rep. May, 1999. [113] G. Shen, H. Guo, and S. K. Bose, “Survivable elastic optical networks: survey and perspective (invited),” Photonic Network Communications, vol. 31, no. 1, pp. 71–87, 2016. [114] J. Rak, Resilient Routing in Communication Networks, ser. Computer Communications and Networks. Springer International Publishing, 2015. [115] R. Ramaswami, K. Sivarajan, and G. Sasaki, Optical Networks A Practical Perspective. Morgan Kaufmann, 2010. [116] S. Ramamurthy and B. Mukherjee, “Survivable WDM mesh networks, Part I - protection,” in Proceedings - IEEE INFOCOM, vol. 2, 1999, pp. 744–751. [117] M. Wang, M. Furdek, P. Monti, and L. Wosinska, “Restoration with Service Degradation and Relocation in Optical Cloud Networks,” in Asia Communications and Photonics Conference, 2015. [118] Q. Zhang, X. Wang, P. Palacharla, M. Sekiya, and Q. She, “On efficient utilization of transceivers for shared restoration in flexible grid optical networks,” in European Conference on Optical Communication, ECOC, 2014. [119] G. Mohan and C. Siva Ram Murthy, “Lightpath Restoration in WDM Optical Networks,” IEEE Network, vol. 14, no. 6, 2000. [120] S. Chalasani and V. Rajaravivarma, “Survivability in optical networks,” in Proceedings of the 35th Southeastern Symposium on System Theory, 2003, pp. 6–10. [121] M. Furdek, N. Skorin-Kapov, and L. Wosinska, “Attack-aware dedicated path protection in optical networks,” Journal of Lightwave Technology, vol. 34, no. 4, pp. 1050–1061, 2016. [122] K. Walkowiak, M. Klinkowski, B. Rabiega, and R. Goścień, “Routing and spectrum allocation algorithms for elastic optical networks with dedicated path protection,” Optical Switching and Networking, vol. 13, pp. 63–75, 2014. 124

[123] X. Shao, Y.-K. Yeo, Z. Xu, X. Cheng, and L. Zhou, “Shared-Path Protection in OFDM-based Optical Networks with Elastic Bandwidth Allocation - OSA Technical Digest,” Optical Fiber Communication Conference, 2012. [124] S. Kosaka, H. Hasegawa, K.-i. Sato, T. Tanaka, A. Hirano, and M. Jinno, “Shared Protected Elastic Optical Path Network Design that Applies Iterative Reoptimization based on Resource Utilization Efficiency Measures,” in European Conference and Exhibition on Optical Communication, 2012. [125] M. Klinkowski, “A Genetic Algorithm for Solving RSA Problem in Elastic Optical Networks with Dedicated Path Protection,” in International Joint Conference CISIS, 2013, pp. 167–176. [126] A. N. Patel, P. N. Ji, J. P. Jue, and T. Wang, “Survivable Transparent Flexible Optical WDM (FWDM) Networks,” in Optical Fiber Communication Conference/National Fiber Optic Engineers Conference, 2011. [127] A. Eira, J. Pedro, and J. Pires, “Optimized Design of Shared Restoration in FlexibleGrid Transparent Optical Networks,” in National Fiber Optic Engineers Conference, 2012. [128] A. Castro, L. Velasco, J. Comellas, and G. Junyent, “On the benefits of multi-path recovery in flexgrid optical networks,” Photonic Network Communications, vol. 28, no. 3, pp. 251–263, 2014. [129] B. Chen, J. Zhang, Y. Zhao, C. Lv, W. Zhang, Y. Gu, S. Huang, and W. Gu, “A novel recovery algorithm for multi-link failures in spectrum-elastic optical path networks,” in IEEE Asia Communications and Photonics, 2011. [130] Microsoft, “Azure Cloud Servers,” 2016. [Online]. Available: https://azure.microsoft. com/ [131] T. Morioka, M. Jinno, and H. Takara, “Innovative Future Optical Transport Network Technologies,” Tech. Rep., 2011. [132] K. Walkowiak and M. Klinkowski, “Joint anycast and unicast routing for elastic optical networks: Modeling and optimization,” in IEEE International Conference on Communications, 2013, pp. 3909–3914. [133] L. Zhang and Z. Zhu, “Spectrum-efficient anycast in elastic optical inter-datacenter networks,” Optical Switching and Networking, vol. 14, no. PART 3, pp. 250–259, 2014. [134] K. Walkowiak, M. Kucharzak, P. Kopec, and A. Kasprzak, “ILP model and algorithms for restoration of anycast flows in Elastic Optical Networks,” in 6th International Workshop on Reliable Networks Design and Modeling, 2015, pp. 102–108. 125

[135] R. Goścień, K. Walkowiak, and M. Klinkowski, “Tabu search algorithm for routing, modulation and spectrum allocation in elastic optical network with anycast and unicast traffic,” Computer Networks, vol. 79, pp. 148–165, 2015. [136] S. Talebi, I. Katib, and G. N. Rouskas, “Offline distance-adaptive routing and spectrum assignment (DA-RSA) in rings,” in IEEE Global Communications Conference, GLOBECOM, 2016. [137] Q. Wang and L. K. Chen, “Performance Analysis of Multicast Traffic over Spectrum Elastic Optical Networks,” in Optical Fiber Communication Conference, vol. 2, Washington, D.C., 2012. [138] X. Liu, L. Gong, and Z. Zhu, “Design integrated RSA for multicast in elastic optical networks with a layered approach,” in GLOBECOM - IEEE Global Telecommunications Conference, 2013, pp. 2346–2351. [139] L. Li, R. Gu, Y. Ji, L. Bai, and Z. Huang, “A Flexible All-Optical OFDM Network Coding Architecture for multicast in Elastic Optical Networks,” in Asia Communications and Photonics Conference, Washington, D.C., 2013. [140] X. Song, D. Zhang, H. Guo, W. Du, Y. Li, and J. Wu, “A Novel RSA Scheme for Multicast in Elastic Optical Networks with Modulation Format Conversion,” in Asia Communications and Photonics Conference, 2013. [141] K. Walkowiak, R. Goscien, M. Wozniak, and M. Klinkowski, “Joint optimization of multicast and unicast flows in elastic optical networks,” in IEEE International Conference on Communications, 2015, pp. 5186–5191. [142] J. H. L. Capucho and L. C. Resendo, “ILP model and effective genetic algorithm for routing and spectrum allocation in elastic optical networks,” in 2013 SBMO/IEEE MTT-S International Microwave & Optoelectronics Conference (IMOC). IEEE, 8 2013, pp. 1–5. [143] R. Morais, C. Pavan, A. Pinto, and C. Requejo, “Genetic Algorithm for the Topological Design of Survivable Optical Transport Networks,” IEEE/OSA Journal of Optical Communications and Networking, vol. 3, no. 1, pp. 17–26, 2011. [144] R. C. Almeida, R. a. Delgado, C. J. a. Bastos-Filho, D. a. R. Chaves, H. a. Pereira, and J. F. Martins-Filho, “An evolutionary spectrum assignment algorithm for Elastic Optical Networks,” in 15th International Conference on Transparent Optical Networks (ICTON), Cartagena, Spain, 2013, pp. 1–3. [145] Y. Zeng, N. Hua, X. Zheng, H. Zhang, B. Zhou, and Y. Cao, “Defragmentation of flexible optical networks based on simulated annealing,” in Asia Communications and Photonics Conference, ACP, 2012. [146] J. Zhao, H. Wymeersch, and E. Agrell, “Nonlinear impairment aware resource allocation in elastic optical networks,” Journal of Lightwave Technology, vol. 33, no. 22, pp. 4554–4564, 2015. 126

[147] S. Varma and J. P. Jue, “Spectrum and waveband assignment in elastic optical waveband networks,” in GLOBECOM - IEEE Global Telecommunications Conference, 2012, pp. 2901–2906. [148] H. Takara, K. Yonenaga, and M. Jinno, “Spectrally-efficient elastic optical path networks toward 1 Tbps era,” in Optical Fiber Communication Conference, Washington, D.C., 2012. [149] J. Wu, J. Zhao, and S. Subramaniam, “Co-scheduling computational and networking resources in elastic optical networks,” in IEEE International Conference on Communications, ICC, 2014, pp. 3307–3312. [150] O. Rival and A. Morea, “Resource requirements in mixed-line rate and elastic dynamic optical networks,” in Optical Fiber Communication Conference and Exposition and the National Fiber Optic Engineers Conference (OFC/NFOEC), 2012. [151] G. Zhang, M. De Leenheer, and B. Mukherjee, “Optical Traffic Grooming in OFDMBased Elastic Optical Networks,” Journal of Optical Communications and Networking, vol. 4, no. 11, pp. B17–B25, 2012. [152] Y. Zhang, X. Zheng, Q. Li, N. Hua, Y. Li, and H. Zhang, “Traffic grooming in Spectrum-Elastic Optical Path Networks,” in 2011 Optical Fiber Communication Conference and Exposition and the National Fiber Optic Engineers Conference, 2011, pp. 1–3. [153] T. Takagi, H. Hasegawa, K. Sato, T. Tanaka, B. Kozicki, Y. Sone, and M. Jinno, “Algorithms for maximizing spectrum efficiency in elastic optical path networks that adopt distance adaptive modulation,” in 36th European Conference and Exhibition on Optical Communication, vol. 1-2, 2010, pp. 1–3. [154] Z. Zhu, W. Lu, L. Zhang, N. Ansari, S. Member, W. Lu, L. Zhang, and N. Ansari, “Dynamic Service Provisioning in Elastic Optical Networks With Hybrid Single- / Multi-Path Routing,” Journal of Lightwave Technology, vol. 31, no. 1, pp. 15–22, 2013. [155] K. Walkowiak, A. Kasprzak, and M. Klinkowski, “Dynamic routing of anycast and unicast traffic in Elastic Optical Networks,” in IEEE International Conference on Communications (ICC), 2014, pp. 3313–3318. [156] X. Wan, L. Wang, N. Hua, H. Zhang, and X. Zheng, “Dynamic Routing and Spectrum Assignment in Flexible Optical Path Networks,” in Optical Fiber Communication Conference/National Fiber Optic Engineers Conference, Washington, D.C., 2011. [157] H. Beyranvand and J. a. Salehi, “A quality-of-transmission aware dynamic routing and spectrum assignment scheme for future elastic optical networks,” Journal of Lightwave Technology, vol. 31, no. 18, pp. 3043–3054, 2013. 127

[158] N. Wang and J. P. Jue, “Holding-time-aware routing, modulation, and spectrum assignment for elastic optical networks,” in IEEE Global Communications Conference, 2014, pp. 2180–2185. [159] S. Zhang, C. Martel, and B. Mukherjee, “Dynamic traffic grooming in elastic optical networks,” IEEE Journal on Selected Areas in Communications, vol. 31, no. 1, pp. 4–12, 2013. [160] S. Fujii, Y. Hirota, H. Tode, and K. Murakami, “On-demand spectrum and core allocation for reducing crosstalk in multicore fibers in elastic optical networks,” Journal of Optical Communications and Networking, vol. 6, no. 12, pp. 1059–1071, 2014. [161] T. Takagi, H. Hasegawa, K.-i. Sato, Y. Sone, B. Kozicki, A. Hirano, and M. Jinno, “Dynamic Routing and Frequency Slot Assignment for Elastic Optical Path Networks that Adopt Distance Adaptive Modulation,” in Optical Fiber Communication Conference/National Fiber Optic Engineers Conference 2011, Washington, D.C., 2011. [162] H. Ding, M. Zhang, B. Ramamurthy, Z. Liu, S. Huang, and X. Chen, “Routing, modulation level and spectrum allocation with dynamic modulation level conversion in elastic optical networks,” Photonic Network Communications, vol. 28, no. 3, pp. 295–305, 2014. [163] Y. Yin, K. Wen, D. J. Geisler, R. Liu, and S. J. B. Yoo, “Dynamic on-demand defragmentation in flexible bandwidth elastic optical networks,” Opt. Express, vol. 20, no. 2, pp. 1798–1804, 2012. [164] K. Walkowiak, R. Goscien, W. Kmiecik, and M. M. Klinkowski, “Content Distribution in Elastic Optical Networks with Dedicated Path Protection,” in International Workshop on Reliable Networks Design and Modeling (RNDM), Barcelona, Spain, 2014, pp. 116–122. [165] M. Liu, M. Tornatore, and B. Mukherjee, “Survivable Traffic Grooming in Elastic Optical Networks — Shared Protection,” IEEE/OSA Journal of Lightwave Technology, vol. 31, no. 6, pp. 903–909, 2013. [166] H. Yang, X. Zhu, W. Bai, Y. Zhao, J. Zhang, Z. Liu, Z. Zhou, and Q. Ou, “Survivable VON mapping with ambiguity similitude for differentiable maximum shared capacity in elastic optical networks,” Optical Fiber Technology, vol. 31, pp. 138–146, 2016. [167] R. Goscien, K. Walkowiak, M. Klinkowski, and J. Rak, “Protection in Elastic Optical Networks,” IEEE Network, vol. 29, no. 6, pp. 88–96, 2015. [168] N. McKeown, “Software Defined Networking,” in International Conference on Computer Communications (INFOCOM), Keynote Talk, 2009, pp. 1–21. [169] Open Networking Foundation, “Software-Defined Networking: The New Norm for Networks [white paper],” Tech. Rep., 2012. 128

[170] K. Kirkpatrick, “Software-defined networking,” Communications of the ACM, vol. 56, no. 9, p. 16, 2013. [171] S. H. Yeganeh, A. Tootoonchian, and Y. Ganjali, “On scalability of software-defined networking,” IEEE Communications Magazine, vol. 51, no. 2, pp. 136–141, 2 2013. [172] H. Farhady, H. Lee, and A. Nakao, “Software-Defined Networking: A survey,” Computer Networks, vol. 81, pp. 79–95, 2015. [173] P. Hui and T. Koponen, “Software Defined Networking,” Dagstuhl Seminar 12363, vol. 2, no. 9, pp. 95–108, 2012. [174] S. Ortiz, “Software-Defined Networking: On the Verge of a Breakthrough?” Computer, vol. 46, no. 7, pp. 10–12, 7 2013. [175] J. Chen, X. Zheng, and C. Rong, “Survey on software-defined networking,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9106, 2015, pp. 115–124. [176] M. Channegowda, R. Nejabati, and D. Simeonidou, “Software-defined optical networks technology and infrastructure: Enabling software-defined optical network operations,” IEEE/OSA Journal of Optical Communications and Networking, vol. 5, no. 10, 2013. [177] C. Chen, X. Chen, M. Zhang, S. Ma, Y. Shao, S. Li, M. S. Suleiman, and Z. Zhu, “Demonstrations of efficient online spectrum defragmentation in software-defined elastic optical networks,” Journal of Lightwave Technology, vol. 32, no. 24, pp. 4099– 4109, 2014. [178] J. Jue, V. Eramo, V. López, and Z. Zhu, “Software-Defined elastic optical networks,” Photonic Network Communications, vol. 28, no. 1, pp. 1–3, 2014. [179] P. Bhaumik, S. Zhang, P. Chowdhury, S. S. Lee, J. H. Lee, and B. Mukherjee, “Software-defined optical networks (SDONs): A survey,” Photonic Network Communications, vol. 28, no. 1, pp. 4–18, 2014. [180] B. Collings, “New devices enabling software-defined optical networks,” IEEE Communications Magazine, vol. 51, no. 3, pp. 66–71, 2013. [181] J.-P. Elbers and A. Autenrieth, “From static to software-defined optical networks,” in 16th International Conference on Optical Network Design and Modelling (ONDM), 2012, pp. 1–4. [182] J. Oliveira, J. Oliveira, M. Siqueira, R. Scaraficci, M. Salvador, L. Mariote, N. Guerrero, L. Carvalho, F. Van’T Hooft, G. Santos, E. Magalhaes, and J. Januario, “Towards software defined autonomic terabit optical networks,” in Conference on Optical Fiber Communication, Technical Digest Series, 2014. 129

[183] F. Cugini, F. Paolucci, F. Fresi, G. Meloni, N. Sambo, L. Potí, A. D’Errico, and P. Castoldi, “Toward Plug-and-Play Software-Defined Elastic Optical Networks,” Journal of Lightwave Technology, vol. 34, no. 6, pp. 1494–1500, 2016. [184] H. Yang, L. Cheng, J. Yuan, J. Zhang, Y. Zhao, and Y. Lee, “Multipath protection for data center services in OpenFlow-based software defined elastic optical networks,” Optical Fiber Technology, vol. 23, pp. 108–115, 2015. [185] A. Dupas, E. Dutisseuil, P. Layec, P. Jennevé, S. Frigerio, Y. Yan, E. Hugues-Salas, G. Zervas, D. E. Simeonidou, and S. Bigo, “Real-Time Demonstration of SoftwareDefined Elastic Interface for Flexgrid Networks,” in Optical Fiber Communication Conference, Washington, D.C., 2015. [186] V. D. Philip and Y. Gourhant, “Cross-Control : A Scalable Multi-Topology Fault Restoration Mechanism Using Logically Centralized Controllers,” in IEEE 15th International Conference on High Performance Switching and Routing (HPSR), 2014, pp. 57–63. [187] S. Zander, T. Nguyen, and G. Armitage, “Automated traffic classification and application identification using machine learning,” in IEEE Conference on Local Computer Networks, 2005, pp. 250–257. [188] N. Williams, S. Zander, and G. Armitage, “A preliminary performance comparison of five machine learning algorithms for practical IP traffic flow classification,” ACM SIGCOMM Computer Communication Review, vol. 36, no. 5, p. 5, 2006. [189] M. Mirza, J. Sommers, P. Barford, and X. Zhu, “A machine learning approach to TCP throughput prediction,” IEEE/ACM Transactions on Networking, vol. 18, no. 4, pp. 1026–1039, 2010. [190] S. Suthaharan, “Big Data Classification: Problems and Challenges in Network Intrusion Prediction with Machine Learning,” ACM SIGMETRICS Performance Evaluation Review, vol. 41, no. 4, pp. 70–73, 2014. [191] W. Huang, G. Song, H. Hong, and K. Xie, “Deep architecture for traffic flow prediction: Deep belief networks with multitask learning,” IEEE Transactions on Intelligent Transportation Systems, vol. 15, no. 5, pp. 2191–2201, 2014. [192] C. Sommer, “Shortest-path queries in static networks,” ACM Computing Surveys, vol. 46, no. 4, pp. 1–31, 2014. [193] T. Subbulakshmi and S. M. Shalinie, “Detection and Classification of DDoS Attacks Using Machine Learning Algorithms,” European Journal of Scientific Research, vol. 47, no. 3, pp. 334–346, 2010. [194] J. L. Berral, N. Poggi, J. Alonso, R. Gavaldà, J. Torres, and M. Parashar, “Adaptive distributed mechanism against flooding network attacks based on machine learning,” in Proceedings of the 1st ACM workshop on Workshop on AISec - AISec ’08, 2008, p. 7. 130

[195] M. M. Najafabadi, T. M. Khoshgoftaar, C. Kemp, N. Seliya, and R. Zuech, “Machine learning for detecting brute force attacks at the network level,” in Proceedings - IEEE 14th International Conference on Bioinformatics and Bioengineering, BIBE 2014, 2014, pp. 379–385. [196] T. Ahmed, B. Oreshkin, and M. Coates, “Machine learning approaches to network anomaly detection,” Proceedings of the 2nd USENIX workshop on Tackling computer systems problems with machine learning techniques, pp. 7:1–7:6, 2007. [197] M. V. Mahoney, “A Machine Learning Approach to Detecting Attacks by Identifying Anomalies in Network Traffic,” Ph.D. dissertation, 2003. [198] C. F. Tsai, Y. F. Hsu, C. Y. Lin, and W. Y. Lin, “Intrusion detection by machine learning: A review,” Expert Systems with Applications, vol. 36, no. 10, pp. 11 994– 12 000, 2009. [199] L. Kocsis and C. Szepesvári, “Bandit based monte-carlo planning,” in Proceedings of ECML, 2006, pp. 282–203. [200] W. Saad, Z. Han, M. Debbah, A. Hjorungnes, and T. Basar, “Coalitional Game Theory for Communication Networks: A Tutorial,” IEEE Signal Processing Magazine, vol. 26, pp. 77–97, 2009. [201] S. Haeri, W. W. K. Thong, G. Chen, and L. Trajkovic, “A reinforcement learningbased algorithm for deflection routing in optical burst-switched networks,” in 14th International Conference on Information Reuse and Integration, IEEE IRI, 2013, pp. 474–481. [202] S. Haeri and L. Trajkovic, “Deflection routing in complex networks,” in Proceedings - IEEE International Symposium on Circuits and Systems, 2014, pp. 2217–2220. [203] T. Hashimoto, K. I. Baba, and S. Simojo, “A study on routing, modulation level, and spectrum allocation algorithms for elastic optical path networks,” in 3rd International Conference on Photonics, 2012, pp. 395–399. [204] T. Akiba, Y. Iwata, and Y. Yoshida, “Fast exact shortest-path distance queries on large networks by pruned landmark labeling,” in SIGMOD ’13, New York, USA, 2013, p. 349. [205] F. Wei, “TEDI,” in Graph Data Management. IGI Global, 2010, pp. 214–238. [206] D. Villeneuve and G. Desaulniers, “The shortest path problem with forbidden paths,” European Journal of Operational Research, vol. 165, no. 1, pp. 97–107, 2005. [207] G. Rétvári, J. J. Bíró, and T. Cinkler, “On shortest path representation,” IEEE/ACM Transactions on Networking, vol. 15, no. 6, pp. 1293–1306, 2007. [208] J. Gao, Q. Zhao, W. Ren, A. Swami, R. Ramanathan, and A. Bar-Noy, “Dynamic Shortest Path Algorithms for Hypergraphs,” IEEE/ACM Transactions on Networking, vol. 23, no. 6, pp. 1805–1817, 2015. 131

[209] R. W. Floyd, “Algorithm 97: Shortest path,” Communications of the ACM, vol. 5, no. 6, p. 345, 1962. [210] R. K. Ahuja, K. Mehlhorn, J. Orlin, and R. E. Tarjan, “Faster algorithms for the shortest path problem,” Journal of the ACM, vol. 37, no. 2, pp. 213–223, 1990. [211] S. Sankaranarayanan and S. Subramaniam, “Comprehensive Performance Modeling and Analysis of Multicasting in Optical Networks,” IEEE Journal on Selected Areas in Communications, vol. 21, no. 9, pp. 1399–1413, 2003. [212] Y. Wang, X. Cao, and Y. Pan, “A study of the routing and spectrum allocation in spectrum-sliced Elastic Optical Path networks,” in Proceedings - IEEE INFOCOM, 2011, pp. 1503–1511. [213] K. Walkowiak, R. Goscien, and M. Klinkowski, “On Minimization of the Spectrum Usage in Elastic Optical Networks with Joint Unicast and Anycast Traffic,” in Asia Communications and Photonics Conference (ACP), 2013, pp. 4–6. [214] S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi, “Optimization by simulated annealing.” Science (New York, N.Y.), vol. 220, no. 4598, pp. 671–80, 1983. [215] K. Christodoulopoulos, I. Tomkos, and E. a. Varvarigos, “Elastic Bandwidth Allocation in Flexible OFDM- based Optical Networks,” Journal of Lightwave Technology, vol. 29, no. 9, pp. 1354 – 1366, 2011. [216] M. Kearns, Y. Mansour, and A. Y. Ng, “A sparse sampling algorithm for nearoptimal planning in large Markov decision processes,” in IJCAI International Joint Conference on Artificial Intelligence, vol. 2, 1999, pp. 1324–1331. [217] T. Cazenave, “Nested Monte-Carlo search,” in IJCAI International Joint Conference on Artificial Intelligence, 2009, pp. 456–461. [218] M. Betancourt, “Nested sampling with constrained Hamiltonian Monte Carlo,” in AIP Conference Proceedings, vol. 1305, 2010, pp. 165–172. [219] C. D. Rosin, “Nested rollout policy adaptation for Monte Carlo tree search,” in IJCAI International Joint Conference on Artificial Intelligence, 2011, pp. 649–654. [220] J. G. March, “Exploration and Exploitation in Organizational Learning,” Organization Science, vol. 2, no. 1, pp. 71–88, 1991. [221] R. Coulom, “Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search,” in Computers and games, 2007, vol. 4630, pp. 72–83. [222] M. P. D. Schadd, M. H. M. Winands, M. J. W. Tak, and J. W. H. M. Uiterwijk, “Single-player Monte-Carlo tree search for SameGame,” Knowledge-Based Systems, vol. 34, pp. 3–11, 2012. [223] J. Tyszer, Object-Oriented Computer Simulation of Discrete-Event Systems, the kluwer ed. Springer, 1999. 132

[224] S. Orłowski, M. Pióro, A. Tomaszewski, and R. Wessaly, “SNDlib 1.0 - Survivable Network Design Library,” in IEEE International Network Optimization Conference, 2007. [225] Data Center Map, “DataCenterMap.com,” //www.datacentermap.com

2017. [Online]. Available:

http:

[226] S. Mneimneh, “Computer Networks: Modeling arrivals and service with Poisson process,” Tech. Rep. [227] W. Barreto-Souza, A. H. S. Santos, and G. M. Cordeiro, “The Beta Generalized Exponential Distribution,” Journal of Statistical Computation and Simulation, vol. 80, no. 11-12, pp. 1005–1014, 2008. [228] J. Irwin, “On the Frequency Distribution of the Means of Samples from a Population Having any Law of Frequency with Finite Moments, with Special Reference to Pearson’ s Type II,” Biometrica Trust, vol. 19, no. 3/4, pp. 225–239, 1927. [229] P. Hall, “The Distribution of Means for Samples of Size N Drawn From a Population in which the Variate Takes Values Between 0 and 1, All Such Values Being Equally Probable,” Biometrika, vol. 19, no. 3/4, pp. 240–245, 1927. [230] F. P. Miller, A. F. Vandome, and J. McBrewster, Apache Maven. 2010.

Alpha Press,

[231] C. Dea, M. Heckler, G. Grunwald, J. Pereda, and S. Phillips, JavaFX 8: Introduction by Example, 2nd ed. Berkely, CA, USA: Apress, 2014. [232] M. Klinkowski, “On the effect of regenerator placement on spectrum usage in translucent Elastic Optical Networks,” in International Conference on Transparent Optical Networks, Coventry, England, 2012, pp. 1–6.

133