By Supervised By

Balancing energy consumption to maximize network lifetime in underwater acoustic sensor networks

By Obaida Abdul Karim 766-FBAS/MSCS/F14

Supervised By Dr. Nadeem Javaid

Department of Computer Science and Software Engineering Faculty of Basic and Applied Sciences International Islamic University, Islamabad 2018

Balancing energy consumption to maximize network lifetime in underwater acoustic sensor networks

By Obaida Abdul Karim 766-FBAS/MSCS/F14

Supervised by:

Dr. Nadeem Javaid, Associate Professor, Department of Computer Science, COMSATS University Islamabad, Islamabad Campus Co-Supervised by:

Dr. Husnain Abbas Naqvi, Assistant Professor, Department of Computer Science and Software Engineering, International Islamic University, Islamabad

Department of Computer Science and Software Engineering, Faculty of Basic and Applied Sciences International Islamic University, Islamabad 2018

Department of Computer Science and Software Engineering, International Islamic University, Islamabad Date: 7 August 2018 Final Approval

This is to certify that we have read the thesis submitted by Obaida Abdul Karim, 766-FBAS/MSCS/F14. It is our judgment that this thesis is of sufficient standard to warrant its acceptance by International Islamic University, Islamabad for the degree of MS Computer Science. Committee:

External Examiner: Dr. Basit Raza, Assisstant Professor, Department of Computer Science, COMSATS University Islamabad.

Internal Examiner: Dr. Tehmina Amjad, HoD, Department of Computer Science and Software Engineering, International Islamic University, Islamabad.

Supervisor: Dr. Nadeem Javaid, Associate Professor, Department of Computer Science, COMSATS University, Islamabad.

Co-Supervisor: Dr. Husnain Abbas Naqvi, Assistant Professor, Department of Computer Science and Software Engineering, International Islamic University, Islamabad.

ii

DEDICATION

𝒟edicated

to my father (May! Allah bless him. ameen), encouraging mentor Dr. Nadeem Javaid, supporting family and my caring friends.

iii

A dissertation Submitted To Department of Computer Science and Software Engineering, Faculty of Basic and Applied Sciences, International Islamic University, Islamabad. As a Partial Fulfillment of the Requirement for the Award of the Degree of MS Computer Science.

iv

Declaration

I Obaida Abdul Karim (Registration No. 766/FBAS/MSCS/F14) hereby declare that I have produced the work presented in this thesis, during the scheduled period of study. I also declare that I have not taken any material from any source except referred to wherever due that amount of plagiarism is within acceptable range.

If a

violation of HEC rules on research has occurred in this thesis, I shall be liable to punishable action under the plagiarism rules of the HEC.

Date: 7 August 2018 Obaida Abdul Karim 766/FBAS/MSCS/F14

v

ACKNOWLEDGEMENT

Alhumdulillah, Thanks to Allah Almighty who blessed me with this day when I am giving acknowledgment for my thesis work.

First of all, I am thankful to such an

intelligent, supportive and responsible mentor Dr. Nadeem Javaid . He has not only given me his guidance in research work but also shared his wisdom about different prospects of life. I feel so fortunate to be a part of Comsens Lab , COMSATS University, Islamabad that has provided me with a healthy working environment and made my project possible. I might have been a difficult student for you, I might have been a troublesome person but I am so proud to be your student.

Thank you for

providing me with an opportunity to work as your research assistant and for keeping patience with me. Your wisdom and guidance made me able to understand life in a total perspective. Now, my only aim is to make you proud. You are a true supervisor since its sound says “super-wiser” and thats what I believe you are. I also pay my heartiest gratitude to my Co-supervisor Dr.Husnain Abbas Naqvi for his kind efforts, guidance and support which is a reason to my success. I am also thankful to my family who has been a constant support system for me throughout my project. Special thanks to those friends like Zuno, Atifa and Anila who supported me and gave me a push and your saying that “you can do it Obaida”, “slow and steady wins the race”

and “Obaida! you have to do it.”

really motivated me.

Thank you Almighty Allah for creating them and bringing them all in my life. O Dear Allah! bless them all. (Ameen)

vi

ABSTRACT Energy is a valuable resource for underwater sensor nodes which plays an important role to prolong the lifespan of underwater acoustic sensor networks (UASNs). Existing protocol is known for balancing energy consumption in order to maximize network lifetime in data-gathering sensor networks (EBDG). This routing protocol performs hybrid data transmission at different radii of network. Nodes of large radii die earlier which affect network life.

It restricts EDBG to small networks.

While,

in UASNs, direct transmission from farthest place and high burden on closer vicinity nodes reduce the lifetime of nodes.

As a result, EBDG is limited to a small

scale network in terms of radius. However, our proposed routing protocol enhanced EBDG (EEBDG), identifies this limitation and solves it by optimum transmission range

(𝑅𝑜𝑝𝑡 ).

within the

According to our proposed routing protocol, mixed transmission is used

𝑅𝑜𝑝𝑡

and then multihop transmission is used. Moreover, a mobile sink is

also introduced in both routing protocols known as (EBDG-MS) and (EEBDG-MS) to enhance the performance of them. As, EBDG and EEBDG with mobile sink (EBDGMS and EEBDG-MS) are conventional routing protocols. Therefore, these protocols are highly dependent on the architecture of a network. Due to this dependency, they use high energy consumption.

Hence, UASNs need self-configuring, adaptive and

energy-efficient routing protocols.

Thus, a Q-learning based efficient and balanced

energy consumption data gathering routing protocol (QL-EEBDG) and with moblie sink (QL-EEBDG-MS) are presented. These routing protocols set an optimal next hop forwarder for each node to transmit its sensed data. However, QL-EEBDG and QL-EEBDG-MS deplete more energy than EEBDG and EEBDG-MS. Because it uses a static reward throughout the network to find the neighbor nodes on the basis of minimum distance to the destination. This static reward leads to the shortest paths. These paths fail to achieve minimum energy consumption and increase the network stability period. Consequently, we incorporate this Q-learning in EBDG (Q-EBDG) and EEBDG (Q-EEBDG) with the dynamic factors to enhance the performance. In Q-EBDG and Q-EEBDG, for each node, we select an appropriate forwarder node on the bases of residual energy of source node and the cumulative energy of neighbor

vii

nodes. Thus, an energy efficient path is formed between source node and sink which helps in balancing energy consumption among the network nodes. Furthermore, due to topological changes, the void hole may occur in network which affects the network lifetime and network stability period. In order to cope with this limitation, we have proposed Q-EBDG and Q-EEBDG with adjacent nodes (Q-EEBDG-ADN) and (QEBDG-ADN). Simulations are carried out to validate the proposed work in real time scenario.

Results for the legitimacy of our work are evaluated in consideration to

the following parameters: energy tax, network lifetime, network stability period and throughput.

viii

Publications

Proceeding Conferences 1.

Obaida Abdul Karim,

Nadeem Javaid, Arshad Sher, Zahid Wadud, and

Sheeraz Ahmed, "A balanced energy consumption based routing protocol for efficient data gathering in underwater ASNs", 2nd EAI International Conference on Future Intelligent Vehicular Technologies (Future5V), 2017, vol. 18, no. 17.

2.

Obaida Abdul Karim,

Nadeem Javaid, Arshad Sher, Zahid Wadud, and

Sheeraz Ahmed, "QL-EEBDG: QLearning based energy balanced routing in underwater sensor networks", 2nd EAI International Conference on Future Intelligent Vehicular Technologies (Future5V), 2017, vol.18, no. 17.

3. Nadeem Javaid,

Obaida Abdul Karim,

Arshad Sher, Muhammad Imran,

Ansar Ul Haque Yasar, and Mohsen Guizani, "Q-Learning for energy balancing and avoiding the void hole routing protocol in underwater sensor networks", in 14th IEEE International Wireless Communications and Mobile Computing Conference (IWCMC-2018).

4. Aasma Khan, Nadeem Javaid, Faisal Hayat, Ghazanfar Latif,

Obaida Abdul Karim,

and Zahoor Ali Khan, "Void Hole and Collision Avoidance in Geographic and Opportunistic Routing in UWSNs", in 6th International Conference on Emerging Internetworking, Data and Web Technologies (EIDWT-2018).

ix

Contents

List of Figures

xiii

List of Tables

xv

List of Algorithms

xvi

1 Introduction 1.1

1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2

1.1.1

Problem statement

. . . . . . . . . . . . . . . . . . . . . . . .

5

1.1.2

Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . .

6

2 Literature Review 2.1

10

Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

11

2.1.1

Routing protocols incorporating Q-learning technique . . . . .

11

2.1.2

Routing protocols addressing energy balancing . . . . . . . . .

12

2.1.3

Routing protocols addressing energy depletion . . . . . . . . .

14

2.1.4

Routing protocols avoiding the void hole

. . . . . . . . . . . .

19

2.1.5

Routing protocols adding mobile sink . . . . . . . . . . . . . .

22

3 Proposed Schemes: EEBDG, EEBDG-MS, EBDG-MS, QL-EEBDG, QL-EEBDG-MS, Q-EBDG, Q-EEBDG, Q-EBDG-ADN, Q-EEBDGADN 27 3.1

Proposed routing protocol: EEBDG . . . . . . . . . . . . . . . . . . .

28

3.1.1

28

Network configuration

. . . . . . . . . . . . . . . . . . . . . .

x

3.2

3.1.2

Balanced Energy consumption:

. . . . . . . . . . . . . . . . .

3.1.3

Inter Corona Energy Consumption

. . . . . . . . . . . . . . .

30

3.1.4

Mobile sink in EBDG and EEBDG

. . . . . . . . . . . . . . .

30

3.1.5

Data Transmission

. . . . . . . . . . . . . . . . . . . . . . . .

31

Automate routing protocol with static reward: QL-EEBDG . . . . . .

32

3.2.1

An overview of Q-learning technique

. . . . . . . . . . . . . .

32

3.2.2

The routing protocol

. . . . . . . . . . . . . . . . . . . . . . .

33

3.2.3

An example for the convergence in Q-learning

. . . . . . . . .

34

3.2.4

Q-learning algorithm . . . . . . . . . . . . . . . . . . . . . . .

46

3.2.5

Format of a packet

. . . . . . . . . . . . . . . . . . . . . . . .

49

3.2.6

Data transmission . . . . . . . . . . . . . . . . . . . . . . . . .

49

3.2.7

Automate routing protocol with static reward and mobile sink: QL-EEBDG-MS . . . . . . . . . . . . . . . . . . . . . . . . . .

3.3

4.2

50

Automate routing protocols with dynamic reward and solution of void hole problem: Q-EBDG, Q-EEBDG, Q-EBDG-ADN, Q-EEBDG-ADN

50

3.3.1

Q-learning based strategy

. . . . . . . . . . . . . . . . . . . .

50

3.3.2

Analysis of reward function

. . . . . . . . . . . . . . . . . . .

51

3.3.3

Scenario for selection of adjacent neighbor node

. . . . . . . .

53

3.3.4

Transmission phase . . . . . . . . . . . . . . . . . . . . . . . .

55

4 Results and Discussions 4.1

30

57

Performance Evaluation: EEBDG and EEBDG-MS

. . . . . . . . . .

58

. . . . . . . . . . . . . . . . . . . . . .

58

. . . . . . . . . . . . . . . . . . . . . . .

58

4.1.1

Simulation parameters

4.1.2

Performance Metrics

4.1.3

Simulation results of both schemes without mobile sink

4.1.4

Simulation results of both schemes with a mobile sink

. . .

59

. . . .

62

Performance Evaluation: QL-EEBDG and QL-EEBDG-MS . . . . . .

65

4.2.1

Simulation analysis of proposed protocol with static sink

. . .

65

4.2.2

Simulation analysis of our proposed protocol a MS

. . . . . .

67

4.2.3

Energy tax

. . . . . . . . . . . . . . . . . . . . . . . . . . . .

67

xi

4.3

Performance evaluation: Q-EEBDG and Q-EEBDG-ADN . . . . . . .

69

4.3.1

Impact of changing radii on energy tax . . . . . . . . . . . . .

69

4.3.2

Impact of changing radii on network lifetime . . . . . . . . . .

70

4.3.3

Impact of changing radii on network stability period . . . . . .

71

4.3.4

Impact of changing radii on throughput . . . . . . . . . . . . .

72

5 Conclusion

74

xii

List of Figures

3-1

Configuration of a network . . . . . . . . . . . . . . . . . . . . . . . .

29

3-2

Mobile sink in the network

. . . . . . . . . . . . . . . . . . . . . . .

31

3-3

Network topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

35

3-4

Convergence of Q-values

. . . . . . . . . . . . . . . . . . . . . . . . .

47

3-5

Selection of ADN. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

54

3-6

Workflow of Q-EEBDG-ADN

. . . . . . . . . . . . . . . . . . . . . .

56

4-1

Comparison of energy tax at different network radii. . . . . . . . . . .

59

4-2

Comparison of a network lifetime at different network radii.

. . . . .

60

4-3

Effects of changing different radii on the network stability.

. . . . . .

61

4-4

Effects of different radii on the throughput. . . . . . . . . . . . . . . .

62

4-5

Comparison of energy tax at changed network radii with a mobile sink.

62

4-6

Comparison of a network lifetime at changing different radii with a mobile sink. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

63

4-7

Effects of different radii on the network stability with mobile sink. . .

64

4-8

Effects of changing different radii on throughput with a mobile sink. .

65

4-9

Pattern of energy tax versus network radii. . . . . . . . . . . . . . . .

65

4-10 Network stability period versus network radii.

. . . . . . . . . . . . .

66

4-11 Impact of network radii on energy tax with a MS. . . . . . . . . . . .

67

4-12 Impact of network radii on a network stability with a MS.

. . . . . .

68

. . . . . . . . . . . . . . . .

69

4-13 Energy tax compared with various radii.

4-14 Network lifetime compared with various radii.

. . . . . . . . . . . . .

70

4-15 Network stability period compared with various radii. . . . . . . . . .

71

xiii

4-16 Throughput compared with various radii. . . . . . . . . . . . . . . . .

xiv

72

List of Tables

1.1

Comparison of the communication mediums

. . . . . . . . . . . . . .

3

2.1

Summary of state of the art work

. . . . . . . . . . . . . . . . . . . .

25

4.1

Control parameters in a network . . . . . . . . . . . . . . . . . . . . .

58

xv

List of Algorithms

1

QL algorithm

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xvi

48

Chapter 1

Introduction

1

Chapter 1 1.1

Introduction

Underwater acoustic sensor networks (UASNs) consist of small number of sensor nodes; collectively deployed in a specific area to perform monitoring tasks. For example, scientific study of ocean, marine life exploration, oil and gas leakage investigation, etc. These nodes in underwater environment are named as node or mote which sense the required information according to the demand of application. Then they transmit the sensed information to the base station through direct or multihop routing. When nodes are in line of sight to sink, then they directly send data towards sink otherwise multihop transmission method is used [1]. These nodes are the backbone of UASNs. However, they have limited computational capacity, limited sensing power, limited battery (difficult to change once deployed in the network) and limited memory. While underwater is a crucial environment as link quality is affected due to noise, long propagation delay, higher bit error rate, etc. The transmission and reception power in UASNs is also high.

All these factors of

underwater environment and the uneven energy consumption are the basic reasons for depletion of nodes’ battery [1]. Because in UASNs, network lifetime depends upon the number of alive nodes. Nodes are expensive due to hardware protection measures in harsh environment of underwater. Therefore, UASNs demand for the energy aware routing protocols in which nodes act efficiently with the changes in the topology [2]. More specifically as mentioned in [3], UASNs have some special characteristics i.e., lower frequency rate, high propagation delay, higher path loss, maximum transmission path and optical waves are also easily scattered in underwater environment. In such a way, communication medium in underwater is acoustic communication. Its speed is

1.5 × 103 m/s

which is lower than the radio waves in terrestrial sensor networks

(TSNs). Moreover, the deployment of node in UASNs is sparse while it is dense in TSNs because TSNs nodes are less expensive. Similarly, underwater environment is harsh because of some challenges like high attenuation, noise and water current, etc. Thus, the topology in this environment is highly dynamic whereas the topology of TSNs is static or low dynamic. Due to the underwater current in UASNs, the nodes

2

1.1. Introduction

Chapter 1

move 1.3 m/s while in TSNs, nodes have no such type of current, therefore, its nodes are static [3]. Generally, bandwidth depends upon distance thus, UASNs have low bandwidth as compared to TSNs and TSNs are fixed for some areas because of its high bandwidth [3]. All over the world, researchers are getting more and more interest in developing applications for underwater acoustic environment and feasible topological deployment for UASNs. According to optimum transmission range

(𝑅𝑜𝑝𝑡 )

(eq.

3.2) and frequency (10

kHz) [1] different values of absorption, spreading and attenuation in acoustic, radio and optical fiber communication are given in the table 1.1.

Table 1.1: Comparison of the communication mediums

Medium Absorption Spreading Attenuation acoustic 4.76 × 102 dB 1.10 × 102 1.10 × 1035 [5] [4] dB [4] dB/Km 06 radio Same as Att. 2.38 × 10 0.10 dB/m (free space) dB/m [6] 15 optical −1.02 × 10 [7] - 5.54 dB/km [8] At the beginning, acoustic, optical and radio waves were used in UASNs [9]. However, due to their characteristics, only acoustic waves are more suitable for UASNs as compared to optical and radio communication mode. as given in the table 1.1. This is because radio waves have highly absorbing, spreading and attenuating rates and need maximum transmission power than acoustic communication. Whereas, optical communication has very low absorption rates as compared to acoustic. Its waves neither scatter nor even attenuate in underwater. However, it has very low transmission range and it is more expensive than acoustic. Therefore, it is not feasible for large scale routing protocol. As a result, acoustic wave is used in UASNs because of being reliable, cost-free, omnidirectional in nature, and dispersed network access with tolerable signal attenuation. Acoustic signals have certain weak points that attract researchers to design routing protocols that can mitigate these weaknesses. These lim-

3

Chapter 1 itations include large propagation delay, finite battery, transmission and path losses, limited bandwidth and noise. In addition, mostly routing protocols in UASNs are the conventional routing protocols. The UASNs routing protocols need to be adaptive, robust and energy efficient, which demand a priori information about the network and restrictions on network architecture [10]. When reinforcement learning is incorporated in underwater routing protocol, it makes the protocol independent of the network architecture. In this technique, an agent performs an action in unknown environment and gets the reward for its action. The action is performed to achieve the maximum cumulative reward. The reason behind Q-learning implementation in UASNs is its behavior, because it optimizes the behavior of an agent and each agent acts efficiently and intelligently in UASNs in terms of energy saving. This technique has many features which will be explained in Chapter 3, however, one feature is very effective in optimizing the behavior of an agent which is the reward parameter. In many other protocols, this parameter is achieved in different conditions like, initial energy, residual energy, energy among the neighbor, distance and waiting time of a mobile sink (MS), node density [10], [11], [12], [13], [14], [15], [16], [17] and [18]. It is noticed that the more refined reward parameter, the more chance it has to optimize the behavior of agents which ultimately improves the network lifetime. Moreover, Q-learning has some excellent properties due to which we have employed in our protocols: generality in framework, lowering the overhead of a network, adaptive nature and load balancing nature. Therefore, Q-learning has minimized overhead of our proposed routing protocols because it adapts the hybrid behavior (reactive and proactive). In case of proactive strategy in UASNs, a control packet is broad-casted throughout the network to find out the routes to the destination and then stores the routes in the table. This table is regularly maintained and it expires when topology changes. This process drains out a large amount of energy in a network. On the other hand, the reactive strategy has larger overhead due to finding complete route (on demand) by broadcasting in the whole network and then stores this route information in the table [1]. When topology changes, this table will no longer be helpful, network

4

1.1.1. Problem statement

Chapter 1

again demands for the new routes information, thus, new broadcasting is initiated to find the new routes and again maintained the routing table [13].

Hence, in our

proposed routing protocols, broadcasting of control packets is limited at one hop. All nodes keep the information of their own neighboring nodes instead of keeping routes of the whole network. Thus, Q-learning overcomes the disadvantages of both (reactive and proactive), while easily acquiring the advantages of reactive and proactive. Furthermore, this Q-learning has load balancing property, thus, it is implemented in our proposed protocols to balance the load of forwarder nodes transmission. The newly proposed protocols automatically switch the forwarder nodes during selection. This switching balances the load of transmission, which consumes a balanced amount of energy throughout the network.

1.1.1 Problem statement Efficient and balanced energy depletion for the maximizing the network lifetime, making the underwater routing protocols adaptive and self-configured and avoidance of void hole problem to prolong the network lifespan are the major challenges of UASNs [19] and [20]. A hybrid technique is used for transmission of data packet for balancing energy consumption to maximize network lifetime in data gathering sensor networks (EBDG) [19]. This technique achieves balanced energy consumption for all nodes in the network.

However, this protocol is limited to small scale network (in terms of

network radius) due to the hybrid technique. When nodes in the network use their energy inefficiently that is, far away nodes transmit the data packet directly to the sink which is out of their range. Thus, their energy is used on packets which are not successfully received by the base station. It means that a large part of their energy is wasted on the direct transmission in larger radii. This excessive energy consumption soon end their limited energies and they die soon. The far away nodes are dead soon while the energy of nearer nodes still remaining in the network.

This imbalanced

energy consumption affects the network lifetime and network stability period. Therefore, avoidance of direct transmission on larger radii helps in maximization of network lifetime and network stability period in underwater routing protocols [21].

5

Chapter 1 In addition, conventional routing protocols are dependent on the architecture of a network and need a prior information about the network. It makes the underwater routing protocols less energy-efficient and degrades all the performance parameters. To make the traditional routing protocols more adaptive, robust and energy-efficient, Q-learning is incorporated in underwater routing protocol which is machine learning technique [11] and [12]. On the other-hand the void hole is an empty space which stops the transmission during the lifetime of a network. Reasons behind its existence in the network are: the node-death [22], untrustworthy link, randomly deployment of nodes, etc. While the node which has no destination in its transmission range is called a void node. The void node receives the data packet from the predecessor nodes (nodes that exist in the region behind the void node). When this node is not able to transmit its own generated information to the base station, it also drops the information of its predecessor nodes. Consequently, it increases the packet drop, eventually the network gets disconnected and partitioned. In addition, a large amount of energy depletes due to void node, as a result the routing protocol fails to achieve its objectives.

Conven-

tional routing protocols solve this limitation with different approaches as discussed in Chapter 2, however, when topology changes due to dynamic nature of underwater or due to other factors, this issue still affects the routing protocols [23] and [24].

1.1.2 Contributions In EBDG, let supposed when nodes are at 250 meters range, direct transmission is used because sink is in range of all nodes, packet loss ratio is closed to zero. However, when distance of nodes increases from 250 meters and nodes use direct transmission mechanism then, packet loss ratio increases. Because acoustic signal has limited range, nodes deployed far from the sink cannot directly send data to the sink. For this, to lower the packet dropped ratio, a specific transmission range is used which restrict the direct transmission mode on larger radii. Therefore, in our proposed protocol, enhanced and efficient balanced energy consumption in data gathering routing (EEBDG) calculates the

𝑅𝑜𝑝𝑡

with the help of predefined variables. Corresponding

6

1.1.2. Contributions to the

𝑅𝑜𝑝𝑡 , it confines the hybrid transmission throughout the network.

are within the

𝑅𝑜𝑝𝑡 ,

Chapter 1

𝑅𝑜𝑝𝑡

When nodes

hybrid transmission is used and when their range is greater than

multihop transmission is performed by avoiding the direct transmission mode.

In this way, all nodes in the network consume balanced energy and network is extended to the large scale network and maximum packet delivery ratio can be achieved with increased network stability. To enhance the performance parameters of the proposed protocol and existing protocol, a MS is used in the network. When MS is introduced in the aforementioned protocols the resultant protocols are named as:

(EBDG-MS), (EEBDG-MS). The

movement of MS in these protocols is clockwise. The decision parameter for MS is the minimum transmission range. If the MS is in node’s range then nodes send the data packet to MS, otherwise it follows base procedure. However, EBDG and EEBDG with mobile sink are the conventional routing protocols. In these protocols, once a forwarder node is selected among neighbor nodes, then source node continuously communicate with this forwarder node, without selecting another neighbor node. For example, during multihop transmission, the source node selects a forwarder node based on shortest distance to the base station. That forwarder node is continuously selected till its death.

Due to frequent selection of

this node, energy of that node is quickly depleted than other nodes in the network. Due to quick depletion of energy of this node, it dies soon creating a void hole in the network. It is noticed that there is no mechanism for the selection of forwarder nodes which overcomes the burden of transmission load among the nodes. As a consequence of this imbalance energy consumption, nodes die soon which become the reasons for void hole in the network. However, energy is depleted in an unbalanced manner throughout the network and the network gets partitioned. For this purpose, a machine learning technique is incorporated in our routing protocol. Q-learning based routing protocol is named as Q-learning based efficient and balanced energy consumption data gathering routing protocol (QL-EEBDG) for UASNs and with mobile sink (QL-EEBDG-MS). This automation reduces the energy consumption in the network at great extent and also introduces a balanced and optimal policy for the selection of

7

Chapter 1 forwarder nodes. This technique takes decision about eligible neighbor node on the basis of reward parameter. Thus, in QL-EEBDG the reward is set to the minimum distance to the static sink. Besides the automation in QL-EEBDG and QL-EEBDG-MS, the energy consumption is more than EEBDG and EEBDG-MS. This Q-learning algorithm selects an eligible forwarder node among the neighbor nodes on the basis of minimum distance. However, there is no check for the nodes when they run out of their energies. Therefore, QL-EEBDG and QL-EEBDG-MS have more energy consumption than EEBDG and EEBDG-MS. For more adaptivity in EBDG and EEBDG, we set the reward dynamic on the basis of energy of source and forwarder nodes instead of the minimum distance [13].

Its means that by applying Q-learning with real time scenario in EBDG and

EEBDG, then it becomes (Q-EBDG) and (Q-EEBDG). When reward is introduced on the basis of energy factor. The performance parameters are better enhanced than the reward on the basis of distance. Further in Q-EBDG and Q-EEBDG, when a source node wants to send the data packet to the sink through forwarder node. This node has no forwarder node in its transmission range (𝑇𝑟𝑎𝑛𝑔𝑒 ). Thus, the data packet of a source node is dropped and the energy of this node is wasted.

This void area does not only affects the energy

tax as well as the other performance parameters: network lifetime, throughput and network stability period of the base protocols. In short, both protocols are facing this issue and have no mechanism to solve it. For the avoidance of void hole, the adjacent node idea is used. While implementing this idea in Q-EBDG and Q-EEBDG, we form (Q-EBDG-ADN) and (Q-EEBDG-ADN). When a source node has no forwarder node leading to the base station, it sends its data packet towards the adjacent nodes of its own sub concentric corona. From the adjacent nodes, one helper node is selected which has forwarder node in desired sub concentric corona.

The packet of source

node instead of getting dropped due to no forwarder node is sent to the helper node. Thus, the data packet is not dropped due to a void hole, instead an alternative path is used. By this way, Q-EBDG-ADN and Q-EEBDG-ADN achieve the best results in terms of energy tax, network lifetime, network stability period and throughput (as

8

1.1.2. Contributions

Chapter 1

we will see in Chapter 4).

9

Chapter 2

Literature Review

10

2.1. Literature Review 2.1

Chapter 2

Literature Review

In this section, we have reviewed some existing routing protocols proposed to prolong network lifetime by balancing energy consumption, by avoiding void hole phenomenon, by incorporating the mobile sink and Q-learning technique in the field. These related works are categorized into: routing protocols incorporating Q-learning technique, routing protocols addressing energy balancing, routing protocols addressing energy depletion, routing protocols avoiding the void hole and routing protocols incorporating mobile sink. Brief discussion is given below:

2.1.1 Routing protocols incorporating Q-learning technique In this section, we have discussed some of the routing protocols based on Q-learning. A Q-learning based kinematics and sweeping (QKS) is proposed in [11]. It is a proactive protocol and it also helps to handle the energy hole problem. QKS maintains a table for each node which stores node velocity and its neighbor node location. Authors in [12] propose an adaptive and efficient energy using Q-learning based delay tolerant network routing protocol (QDTR). This protocol employs a Q-learning scheme to perform online learning and handles node mobility using contact history of successor node.

Correspondingly, a Q-learning based adaptive routing protocol (QELAR) is

presented in [13]. It selects the forwarder nodes on the basis of reward function which considers the residual energy of an individual node as well as of groups.

However,

QELAR also uses the mechanism of retransmission of the packet that causes more end to end delay in the network. Forster et al. in [15], propose feedback routing for optimizing multiple sinks in wireless sensor networks with reinforcement (FROMS). This protocol avoids the overhead of neighbor nodes with the help of multiple MSs. FROMS also provides the recovery mechanism for node failure due to node mobility. As a result FROMS achieves low network cost as compared to its counterpart protocols. For efficiently using the limited battery of nodes, authors in [16], propose a Q-learning based delay aware routing protocol (QDAR). It finds the global optimal neighbor node rather than local best neighbor node. For this purpose, utility func-

11

Chapter 2 tion is calculated on the basis of residual energy and low propagation delay.

Both

parameters help in decision making of the best neighbor node. Thus, this protocol performs energy balancing in the whole networks and it achieves 20-25% network lifetime prolongation. In [17], a Q-learning based routing algorithm intelligently configures the route from source to destination with minimum energy consumption. An adaptive algorithm based on reinforcement technique for packet forwarding is presented in [18]. This algorithm adapts the shortest paths towards destination while forwarding a packet. The advantage of this strategy is that it makes the Q-routing independent from switching of a network and minimum time is spent on a delivery of a packet even when a network faces the heavy load. However, it takes the local optimum decision in overall routing.

2.1.2 Routing protocols addressing energy balancing In this section, we have reviewed the existing protocols that improve the network lifespan of UASNs by balanced energy depletion. In [21], two protocols have been introduced to achieve prolong network lifetime and maximized throughput by balancing energy consumption. Name of the both protocols is efficient and balanced energy consumption technique (EBET) and enhanced EBET (EEBET), respectively. In EBET, relay node selection, is on the basis of optimal distance and in EEBET number of hops minimization is on the basis of depth threshold. Both schemes avoid direct transmission mode for balancing energy consumption in rings whereas EEBET balances energy in whole network. In a same way, in [23], it is identified that the imbalanced energy consumption in deep underwater and solve this limitation by introducing the automobiles underwater vehicles (AUVs). AUVs collect the data from all the burdened deep underwater nodes.

Moreover, this protocol swaps the layers when there is an important data.

The delay is reduced by receiving the data quickly from the nearer nodes with the help of AUVs. Thus, both the far away nodes and nearer nodes remove this in the network have balanced energy consumption which leads to maximum network lifetime and receives the maximum amount of data at the end of the network lifetime.

12

2.1.2. Routing protocols addressing energy balancing

Chapter 2

In [24], routing is designed to avoid energy hole in UASNs. Thus, authors use distributed mechanism to balance the overall load of a network. Therefore, all nodes are restricted to transmit continuously generated data within their

𝑇𝑟𝑎𝑛𝑔𝑒 .

In addition,

forwarding policy determines the load weight of forwarder nodes and selects the one that leads to lower energy utilization. By applying this balancing technique, routing protocol leads to balanced energy consumption.

Hence, it handles the energy hole

and improves the network lifetime. The authors in [25], present a balanced energy transmission mechanism in which each node changes its transmission mode according to their energy level. In this way, energy consumption in the network is balanced. In [26], authors proposed a balanced adaptive routing protocol (BEAR) that is a location based protocol. It is a location based routing protocol. In multihop transmission, the BEAR identifies the forwarder nodes on the basis of the location information. Then, it uses the cost function to further determine the successor node and the facilitator node among forwarder nodes.

For balancing the energy consumption, one

of the node between successor and facilitator node is selected for transmission which has maximum relative residual energy. This balanced procedure helps in successfully transmitting the data packets to the base station and also accomplishes the maximum network lifetime up to 55% than the base protocols on the cost of the minimum residual energy. In [27], authors propose a technique to extend the network lifetime in UASNs by handling the limitations i.e., energy hole and unbalanced energy consumption. Therefore, authors propose two protocols that are a balanced energy consuming and hole alleviating (EBCHA) and energy-aware EBCHA (EA-EBCHA). In first protocol, the unbalanced distribution of load is balanced in the whole network, while the other protocol works on minimum energy depletion to enhance throughput. Both the protocols were successful in achieving higher packet delivery ratio, prolonged network lifetime and the balanced energy consumption. However, the cost paid for this achievement is the increased delay. Similarly, the achieved parameter in [28], is balanced energy consumption per node

13

Chapter 2 which is the primary reason for an extended network lifetime. In a same way, authors in [29], work for the optimization of network life by incorporating multiple sinks in the network filed. Further for balanced energy utilization throughout the network, more energy is assigned to nearer nodes of base station as compared to the farthest nodes in [30]. Furthermore, an optimal multimodal routing protocol (OMR) is presented in [31]. In this protocol, radio waves combined with acoustic waves to provide equal facility to all nodes.

Therefore, nodes in the network do not die because of

overloading and burden caused by the relay nodes. However, due to the radio waves OMR is limited to a small scale network.

2.1.3 Routing protocols addressing energy depletion In this section, some of routing protocols are discussed which efficiently utilize the limited energy of underwater nodes. In [32], authors propose the describe the geographic multipath routing based on geospatial division in duty cycle for UASNs.

According to this routing protocol,

whole network is divided into three dimensional small cubes. In data transmission phase, relative small cubes of the source nodes are efficiently selected with the help of greedy geographic forwarding based on geospatial division (GGFGD) algorithm. Then forwarder node in that particular cubes (target cubes) is selected with the help of geographic forwarding based on geospatial division (GFGD). Finally, the target cube collectively transmits the data packet towards the sink. Both of the algorithms help to find out the optimal routes for packet forwarding. Hence, this routing protocol achieves the lower end to end delay with minimum energy consumption. In addition, a duty cycled procedure is applied in which group of nodes collaborated by switching sleep mode among themselves to conserve energy. On the other hand, authors in [33], present the geographical routing protocol known as the relative distance based forwarding (RDBF) for UASNs. RDBF uses the fitness function that calculates the degree of a node. In this way, this function finds the most reliable successor nodes in the network as well as restricts the number of nodes that are involved in the routing. This restriction has reduced the energy consumption in

14

2.1.3. Routing protocols addressing energy depletion

Chapter 2

the network. Authors in [34], proposed an energy-efficient and interference-aware routing protocol (EEIAR) in UASNs. The main objectives of this work is to maximize the throughput and overcome the packet loss by avoiding the interference route.

In UASNs,

forwarder node is responsible for successfully delivering the data packet towards the sink. Therefore, this paper proposes a technique for the selection of forwarder node. An appropriate forwarder node is selected on the bases of the minimum depth and least number of the neighbor nodes.

Therefore, this strategy has small number of

nodes involved in routing which helps in reducing interference while forwarding packets towards sink, hence, this technique eliminates packet collision and packet loss. Authors in [35], propose AUV-based data delivery protocol (ADDP) for Ad hoc UASNs. Distinct feature of ADDP that controls the movement of the AUVs. Results show that ADDP has higher throughput, minimize the overhead and energy consumption of the network. Similarly, another routing protocol for reliable data delivery is the AUVs aided effect data gathering (AEDG) in [36]. In AEDG, AUVs worked together with gateways for extention of the network lifetime. Likewise in [37], authors present an efficient data gathering in the field by using sink mobility. In this routing protocol, authors incorporate AUVs, courier nodes (CNs) and MSs in the network to prolong the network lifetime by minimum energy depletion. Correspondingly, authors in [38], incorporate the AUVs in the network to minimize the packet loss in the network and propagation delay. Thus, a hierarchal concept is use for large scale network and upgrades the performance parameter to some extent. Identically, in [39], authors propose AUV aided routing protocol (AURP). In this protocol multiple AUVs are deployed in the network. These AUVs gathered the data from all the sensor nodes and then transmit towards the sink. Due to efficient energy consumption by AURP, maximum throughput is achieved in the network. Similarly, in [40], authors propose an energy efficient routing protocol considering the distance-varied collision probability and residual energy of each node (DRP) in UASNs. This protocol identifies the transmission collision that occurs due to the different transmission distances because this collision has negative impact on the lifetime of a network.

15

Therefore, authors

Chapter 2 suggest the route that has high transmission rate and maximum residual energy. In this way, DRP achieves prolonged network lifetime which is confirmed by the theoretical analysis. The achievements of DRP are lower latency, maximum throughput and network lifespan. The Authors in [41], present energy-efficient grid routing based on 3D cubes (EGRCs). ERGCs is an energy-efficient and reliable data transmission scheme for complex environmental monitoring in UASNs. In this protocol, optimal cluster heads are selected on the basis of maximum residual energy and localization (minimum distance to sink) of sensor nodes. In [42], the goal is to minimize energy consumption, increase reliability and achieve low communication cost.

The technique used is a novel layered multi-path power

control (LMPC) considering the noise attenuation in deep water, to organize the transmission power and control the data rate across the whole network. In [43], present selection of transmission range (OSTR) for UASNs. In this network, nodes are deployed randomly. To achieve goal of efficient energy utilization or minimum energy consumption, network uses an adaptive transmission power strategy of all nodes. In [44], optimizing number of hops and retransmissions for efficient energy consumption in multihop communication. It saves energy in some other conditions and also in small trials of retransmission.

In this paper authors have analyzed and studied

that how much energy is required for multi-hop transmissions in order to successfully deliver data.

Optimum number of hops, retransmissions, code rate and signal-to-

noise ratio are also considered. This scheme achieves minimum energy consumption suffering from higher end-to-end delay. Complex network approach to topology control problem in UASNs is presented in [45]. Objectives of this work is coverage, connectivity, optimized energy consumption and propagation delay as much as possible. A scale free model is applied in 3D environment. Topology control strategy based on complex network theory (TCSCN) is used to design double clustering structure selecting cluster heads. Scheme does not work on self-adaptive solution like transmission rate.

16

2.1.3. Routing protocols addressing energy depletion

Chapter 2

A routing protocol on the basis of depth for efficient energy consumption in UASNs is proposed by Yan et al. in [46]. In this protocol, a node is selected as a forwarder node, if its depth is less than the depth of preceding relay node. The authors in [47] introduced the multi path free error correction (M-FEC) scheme that depends on Hamming Code which increases the reliability of a network and minimizes the energy consumption.

For minimization of packet error rate and minimum number of hop

counts the Markovian model is used. However, encoding at source and decoding at destination in Hamming code maximized the delay in the network. Wu et al. in [48], propose the time synchronize routing protocol, for easily construction of network and reduction of the node conflict therefore, the time slot based routing (TSR) algorithm is designed. This algorithm avoids the bit error rate and minimize the repetition in the network. At the end it gives the result in the form of minimized energy depletion and prolonged network lifetime. Javaid et al. in [49], proposed improved adaptive mobility of courier nodes in threshold optimized depth based routing (iAMCTD) protocol. It is a reactive, localization free and flooding based routing protocol for time sensitive applications.

iAMCTD

calculates the forwarding functions (FFs) for maximization of network lifetime and transmission loss. This protocol also uses an optimal courier nodes in the networks to maximize the number of packets received and degrade the delay in the network. Further, selection of eligible forwarders nodes occur on the basis of depth based threshold. Yang et al. in [50] present multi-path routing protocol for underwater ad hoc networks with directional antenna (UMDR). In this protocol, the whole network area is divided in different segments. Each node from its own segments directly communicate with one another through direct antenna. The advantage of this technique in a form of lower energy consumption and overcomes the interference. The data delivered to the destination is directly forwarded without involving broadcasting. Direct delivery of data helps in lower end to end delay throughout the network. However, each time direct antenna calculation for each forwarder node.

This estimation increased the

overhead in the network. Vector based forwarding (VBF) in [51] form a vector for forwarder nodes to transmit

17

Chapter 2 the data packet. Source node from its own distance to the destination formed a vector. Then source node select those nodes as forwarder which are exist in this vector. By this way, data packets are send towards the destination. Similarly, in [52] modify the VBF by topology control VBF (TC-VBF) for dense network. Likewise a routing protocol based on received signal strength (RRSS) in [53]. RRSS selects forwarder node inside the vector on basis of received signal strength. Thus, it achieve the minimum energy consumption however, the nodes inside the vector soon consumes their energies because of the heavy load on them. Li et al. in [54], design a depth-based routing aware MAC protocol for data collection (DBR-MAC). In this protocol, a forwarder node is selected on the base of three metrics: low depth, angle and overhead. This selection criteria frequently selects the nodes nearer to the sink which creates a hotspots nodes in the network. This hotspot nodes form void hole in the network and also increase the packet dropped ratio. Correspondingly, another routing protocol is a message dissemination approach in storage-limited (MDA-SL) is suitable for maximization of throughput in [55]. In MDA-SL, a forwarder node is selected on the basis of high mobility and maximum residual energy. Due to continuous selection of that particular forwarder nodes, that nodes die soon as a result the network face the void hole problem.

A localization free and greedy routing protocol is proposed by

Zhou et al. in [56]. An energy efficient routing protocol for UASNs in the internet of underwater things.

In this protocols, the forwarder nodes is selected in hop by

hop manner and when network is also stable. Due to waiting for stable network, the routing protocol ends with higher delay. For balanced energy consumption, authors in [57], [58] and [59] introduce two MSs in a network. These MSs regularly get the data from all nodes by moving on a random and predefined circular trajectory. Besides these protocol, Walayat et al. in [60], incorporate two MSs for the farthest nodes. These sinks move in a predefined linear path to get data. Hence, the achievement in the form of less number of packet dropped throughout the network.

18

2.1.4. Routing protocols avoiding the void hole

Chapter 2

2.1.4 Routing protocols avoiding the void hole Routing protocols are based on the avoidance of a void hole in UASNs are presented in this section. Authors in [9] present the routing protocol for the avoidance of void hole named as energy and depth variance based opportunistic void avoidance (EDOVE) for UASNs. EDOVE also helps in balancing energy consumption throughout the network. For the avoidance of void hole, it checks upto the two hops neighbor nodes from the source node. While for the balanced energy depletion it adopts the strategy in which one hop neighbor nodes are selected on the base of normalized variance of residual energy, while the second hop nodes on the basis of normalized variance of depth. At the end, EDOVE is successful in avoiding the void hole and also consuming balanced energy throughout the network. As a result, EDOVE achieves prolonged network lifespan with the minimum energy consumption and succeeds 15% maximum throughput than base line protocols. Energy hole minimization with field division for energy-efficient routing in WSNs is presented in [61]. In this paper, authors investigated factors affecting network lifetime which contribute to energy hole creation due to unbalanced energy consumption. The performance evaluation shows that network is stable for longer time, therefore, packet delivery ratio is improved as compared to existing schemes.

Although, in order to

achieve extended network lifetime, some packets are dropped. Geographic and opportunistic depth based routing protocol (GEDAR) for UASNs, in which neighbor nodes are selected on the basis of their location [62]. GEDAR confirms the path from source to destination and then sends the data packet on the particular path. This routing protocol also has a recovery mechanism for the avoidance of the void holes. When node faces the void area, that node broadcasts itself as void node and adjusts its depth along with the new neighbor node position. GEDAR has an achievement in terms of maximizing the network lifespan on the basis of maximum latency. A channel aware routing protocol with the depth adjustment (CARP-DA) has been

19

Chapter 2 proposed for UASNs in [63], for the avoidance of a void hole and maximum throughput.

CARP-DA uses the virtual route strategy for the data transmission which is

dependent on the acoustic speed and noise along with the depth of a node.

The

successor node is selected on the basis of one hop neighbor node. Furthermore, for the avoidance of a void hole, a backward transmission is performed in the network in order to increase the number of received packet. This backward transmission causes more energy consumption and a higher end to end delay. In [64], an adaptive procedure is used to avoid the void hole problem in UASNs. According to this protocol, a node has information of its one hop neighbor, a source node and sink location. The main feature of this technique is the transmission level adjustment. If node density is low around the sender node, it selects another node by increasing its transmission level. In [65], propose two location based routing protocol to overcome the void hole probability in UASNs. In first protocol, the two-hop adaptive hop-by-hop vector based forwarding (2hop-AHH-VBF), the authors avoid the void hole on the basis of two hop neighbor nodes. While in second, quality forwarding adaptive hop-by-hop vector based forwarding (QF-AHH-VFB), the authors compute the composite priority function for suitable successor nodes. This function depends on the two metrics: distance and energy of potential forwarder in the virtual vector. Thus, this function helps in improving the network lifetime of QF-AHH-VBF. Moreover, in order to avoid duplicate data packets in the virtual pipeline proximity, these protocols use the holding time parameter. Improved network lifetime of both protocols means that they consume minimum energy per packet and receive more packets as compared to the base protocols, however, the cost paid by them is the maximized end to end delay. Authors in [66] use the balanced energy utilization strategy to overcome the void hole problem and efficiently utilizes the limited battery of nodes.

Thus, their protocol

called joint routing and energy management (JREN) for UASNs. In this protocol, the authors evenly distribute the transmission load in the whole network with the adjustable transmission range during transmission of data packets. JREN figures out the load weight at each possible forwarder node for balancing energy consumption

20

2.1.4. Routing protocols avoiding the void hole among nodes.

Chapter 2

As a result of this effort, the energy holes are overcome, that ulti-

mately prolong the network lifetime On the avoidance of coverage hole area and energy hole in underwater networks [67] gave solution by coverage repair algorithm and balanced energy consumption techniques.

This scheme works on successful packet delivery ratio to achieve prolong

network lifetime, energy consumption and throughput. End to end delay is increased because of time spent on removing coverage hole in the network. A well-known protocol, weighting depth and forwarding area division depth based routing (WDFAD-DBR) for UASNs is used to avoid the void hole problem. In this protocol, if during transmission a node does not find any forwarded node to send data packet, then the selected data packet is sent to available candidate node to further forward it to the base station [68]. A new strategy for void hole detection and avoidance through optimal selection forwarder in WDFAD-DBR (SHSF-WDFAD-DBR) is presented in [69].

In order to

overcome the void hole problem in network, authors use a technique which give prior information about the void holes. Besides this procedure, authors also get the optimal number of potential forwarder nodes at each node for the backup data loss due to void hole problem. In addition, the network is logically designed into sub forwarding regions in order to overcome the duplication of transmission in the network. In [70], propose a hydraulic based routing protocol (HRP). That HRP addresses the low bandwidth, high energy consumption and mobility of the nodes in UASNs. Like DBR, HRP has adopted the energy hole prevention mechanism. Thus, its performance improves in terms of maximum packet delivery, minimum delay in the network. It also helps to minimize the energy utilization. In [71], present a localization-free interference and energy holes minimization (LF-IEHM). For the avoidance of interference, authors use a unique packet holding time technique. LF-IEHM also eliminates the void hole problem during transmission phase where source node has no forwarding nodes. Thus, LF-IEHM receives a reliable data at the end of network. Authors in [72], present two protocols for the avoidance of interference and void hole problem. First is the geo-spatial division based geo-opportunistic routing scheme for interference avoid-

21

Chapter 2 ance (GDGOR-IA) while second is geographic routing for maximum coverage with sink mobility (GRMC-SM). The network field is divided into small cubes by making efficient local decision for low energy depletion. In geo spatial region, interference is avoided by limitation on the number of nodes. The mobile sink is used to overcome the loads on the intermediate nodes and also recover the data from the void hole. Moreover, optimal holding time technique is also estimated for successful transmission acknowledgment. In [73], adaptive forwarding layer multipath power control routing protocol is proposed. In this protocol, authors work to avoid retransmission of data packet through the network for the reliable data delivery.

This protocol also avoids the void hole

problem by considering the two hop and three hop neighboring nodes. However, every node checks the number of hops before transmission which increases the end delay in the network. Likewise, other preventing void hole routing protocols, in [74], also avoiding the void hole and balancing the load in the network. This protocol is named as balance load distribution (BLOAD) routing protocol. In this protocol, data is divided into fraction.

For every sensor node an adjustable transmission range is used for equally

delivering of this fraction data to each next nodes.

Thus, BLOAD is successful in

prolonging network lifetime and maximizing the network stability period. A channel aware routing protocol (CARP) presented by Basagni et al. in [75] for the avoidance of void hole and shadow zones.

CARP uses the power control strategy along with

successfully packet transmission history of nodes. The protocol presented in [76] used beacon signals and information of residual energy of sensor nodes to eliminate void hole problem.

2.1.5 Routing protocols adding mobile sink Since, in underwater environment nodes change their position due to the water current, hence, in this section, we discussed some of the existing routing protocols that adopt the mobile sink, which are given below: handled void hole limitation in [77].

Mobicast routing protocol (MR),

In MR, a MS is used that moves around the

22

2.1.5. Routing protocols adding mobile sink

Chapter 2

predefined routes. This MS collects the data from all the sensor nodes and in this way it covers the whole network field. In [78], authors presented four protocols for efficient energy consumption in TSNs. Balanced energy efficient network integrated super heterogeneous (BEENISH) routing protocol. In BEENISH, different energy levels of nodes are considered and cluster head is selected on the basis of average energy levels. To prolong the network lifetime improved-BEENISH (iBEENISH) is designed in which cluster head is dynamically nominated. To maximize the number of packets received throughout the network and higher the network stability period, a mobile sink is introduced in both aforementioned routing protocols, named as MBEENISH and iMBEENISH. In addition, in [79] and [80] authors also introduce cluster and a mobile sink technique for minimum energy consumption and for maximization of throughput. Another routing protocol derived for this technique of mobile sink is the balanced transmissions based trajectories of mobile sink in homogenous TSNs. In [81] routing protocol, authors focus on maximization of network lifetime and minimization of path loss and end to end delay.

Thus, authors use a MS that receives the data

from random location as well as the static location. In [82], present an energy scaled and expanded vector-based forwarding scheme with sink mobility (ESEVBF) that is suitable for minimum energy consumption and delay sensitive network. ESEVBF expands the holding time by the help of residual energy of node and vector base pipeline distance ratio. The expansion of holding time has great advantage on the selection of minimum and eligible forwarder nodes for transmission. Limited number of forwarder nodes help in lowering the energy depletion and balanced energy consumption throughout the network. As, we have discussed earlier that balanced energy utilization in routing protocols, i.e., [21], [23], [24], [25], [26], [27], [28], [29], [30] and [31] is the primary reason for maximization of network lifetime. Therefore, in our routing protocols, we use a balanced strategy for neighbor nodes and for adjacent nodes, in order to balance energy utilization. To avoid the void hole, different strategies are used in [9], [61],..., in [75]. Thus, for avoidance of this limitation, our routing protocol proposes the idea of adjacent node.

Moreover, Q-learning make routing protocols

more robust, energy efficient and lifetime-aware as in [11], [12], [13], [15], [16], [17]

23

Chapter 2 and [18], therefore, we incorporate this strategy in protocols. For sake of simplicity, related works are summarized in table 2.1.5.

24

2.1.5. Routing protocols adding mobile sink

Table 2.1: Summary of state of the art work

Protocol

Features

EEBET [21]

Enhance and remove the deficiencies of BTM and EBET and avoids the direct transmission mode

Procedure Relay node on the basis of distance and depth threshold. Use energy levels for balancing energy consumption

Achievements

Limitations

An extended lifetime and scalability period of a network, high throughput and low transmission loss

High end to end delay

BTM [25]

Effective and balance energy consumption, mixed routing

By balancing transmission distance perform direct or hop by hop communication.

Extend the network lifetime and balance energy consumption

DIB, EBH [28]

Balance energy utilization per node, mixed transmission on the basis of residual energy, different levels of battery power for each node.

Underwater moored monitoring network and the balance energy between the shallow and deep water.

Balancing energy utilization per node, extend network lifetime

According to water characteristics, an adaptive, effective energy protocol based on Q-learning

Low packet delivery ratio, more throughput, low network overhead, latency, low communication overhead

QELAR [13]

Reinforcement learning algorithm and reward function used for selecting the relay nodes.

25

Chapter 2

Direct transmission over long distance causes transmission loss, data load, energy hole and not suitable in large scale network Network configure only for sparse, network stable for more times because of long delay and with a path loss. Energy hole creation near the sink or base station. Low network lifetime 20% over existing protocol and using more energy on operation

Chapter 2

Protocol

RDBF [33]

UFCA [88]

Features Small numbers of relay nodes are selected to forward the packet to the sink. No extra burden for validation of relay node.

Energy effective routing, node with low residual energy and far away node go to sleep

Minimize extra transmission by forwarder node ER𝑃 2 R which is selected on the basis of [89] residual energy and local distance.

EEDBR [90]

Check on the selecting the relay nodes, holding time technique and multiple sink

Procedure

Achievements

Limitations

For minimum energy consumption the forwarders are judged by fitness factor

Packet delivery ratio, effective energy, end delay and effective transmissions

Suitable only for small scale, not work on large scale when number of nodes or an area increased and more overhead of the communication.

An ultrasonic frog calling based algorithm

Higher packet delivery ratio, minimum energy utilization, high throughput, low end delay

Efficient routing algorithm based on physical and residual energy

Low energy utilization, extended network lifetime and lowered end delay

Effective energy based protocol

Low energy consumption, lower end delay, extend network lifetime

26

Imbalanced energy utilization, consider the minimum energy in the whole network not considered the energy consumption among nodes, so when one node dies, the whole network become disconnect. More overhead, imbalance energy consumption, in a case of dense network, more nodes are selected as a forwarder therefore, more energy consumption in the network For dense network it creates overhead by again and again broadcasting messages and no mechanism for void holds in the network.

Chapter 3

Proposed Schemes: EEBDG, EEBDG-MS, EBDG-MS, QL-EEBDG, QL-EEBDG-MS, Q-EBDG, Q-EEBDG, Q-EBDG-ADN, Q-EEBDG-ADN

27

Chapter 3 3.1

Proposed routing protocol: EEBDG

In this section, the detail of our proposed routing scheme is discussed.

3.1.1 Network configuration ˆ

In our configuration, we assume a circular monitoring area A, with a radius of R starting from 100 meters and then increasing with an increment of 100 up to 1000 meters respectively as illustrate in Fig. 3-1(a).

ˆ

Whole area is divided into concentric coronas,

𝐶1 , 𝐶2 , 𝐶3 ,......., 𝐶𝑚 ;

where m is

the total number of concentric coronas.

ˆ

Each concentric corona has the same width of

𝑅𝑎𝑑𝑖𝑢𝑠 𝑅 𝑟 = ( 𝑇 𝑜𝑡𝑎𝑙𝑁 ) = (𝑚 ) 𝑜.𝑜𝑓 𝑐𝑜𝑟𝑜𝑛𝑠

as

shown in Fig. 3-1(b). [19].

ˆ

Single static sink (dark hexa pentagon) is deployed at the center of concentric coronas for data gathering process.

ˆ

All nodes are homogenous in nature as shown in Fig.

3-1(c), randomly and

uniformly distributed over a circular area of radius R and node distribution density

𝜌

is same and all nodes have the same initial energy

𝑇𝑡𝑟𝑎𝑛𝑠 ≤ 250

𝐸0

[19].

ˆ

All nodes have the same

as shown in Fig. 3-1(d).

ˆ

All nodes using hybrid transmission i.e, direct transmission mode (DT) or multihop transmission mode (MT).

ˆ

Each individual concentric corona is further divided into equal number of subconcentric coronas. For example, let j is an arbitrary value, thus, the concentric corona

𝐶𝑗 ,

is further divided into two sub-concentric corona as shown in Fig.

3-1(c). For clarity, Corona 2 contains both light-grey and dark-grey portions, light-grey portion represents concentric corona 2-1 and dark-grey portion represents 2-2, respectively. Likewise, all the concentric coronas are divided into sub-concentric coronas correspondingly [19].

28

3.1.1. Network configuration

Chapter 3

r

2r

…..

5r

1

C r

R

2 C

R

..

…

r

C 5

…..

r

Base station R 100m ~ 1000m (a) Network field

Z4 1-a

a

Z3 1-

1c

Z2 1c

1-

d

1-

Z4

Z4

Z4 1

c

-c

Z3

z42-i

Z2 1a

1a

Z2

z22-i i

z22-v

1c

Z3 1c

Z2

i z42-v

1a

i z2 2-i z22-v

i

z32-i

Z3

z42-i i

z32-v

z42-iv

pt

Ro

d

1-

z22-v

z32-iv

a

z22-iv

z42-v

i

i

z22-ii

z32-ii

i z42-ii

b 1-

ission

ansm ihoptr Mult

d

`

b 1-

Direc

Z3

z42-iv

z32-iv

z22-iv

SC2

e

Z2

i

z42-ii

i

z32-ii

i z22-ii

SC2

1

aa

on smissi

ttran

g T ran

z22-ii

Z3

Z4

1-a

b 1Z4

b 1-

Z3

1

1

SC

b 1Z2

SC

SC

`

b 1Z4

z32-ii

z22-ii

. . . .. . . . . .. . . .. . . . . .. . .. . . . . . . . .. . . . . . . . . . . . . . .. . . . . . . . . . . . .. . . . z32-ii

1Z2

. . . .. . . . . .. .. . . . . . . . . . . . . . .. . ... . . . . . . . . . . . .. . .. . . . . . . . . . . . . . . . . .

z32-i

.

z42-ii

.

.

z42-ii

i z32-v

.

C= Concentric corona (C1 ~ C5) r= Radius of concentric corona (b) Division of network

SC2

z32-v

z42-v

SC 1: Lower sub concentric corona SC 2: Upper sub concentric corona

Multihop transmission Direct transmission

Division of zones and deploy- (d) Transmission range of a network ment of nodes and node (c)

Figure 3-1: Configuration of a network

29

Chapter 3

3.1.2 Balanced Energy consumption: According to [19] energy consumption among nodes within each corona can be balanced only if the amount of data received in each corona is balanced. Energy consumption is balanced for the

𝐸(𝑎) = 𝐸(𝑏)

∀𝑎𝜖𝐶𝑗

𝐶𝑚

∀𝑏𝜖𝐶𝑗

corona if and only if,

1