IoV distributed architecture for real-time traffic data

Available online at www.sciencedirect.com Available online at www.sciencedirect.com

ScienceDirect ScienceDirect

Available online at www.sciencedirect.com Procedia Computer Science 00 (2018) 000–000 Procedia Computer Science 00 (2018) 000–000

ScienceDirect

Procedia Computer Science 130 (2018) 480–487

www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia

The 9th International AmbientNetworks Systems,and Networks and Technologies 9th International ConferenceConference on Ambienton Systems, Technologies, ANT-2018 and The 9th International Conference on(ANT Ambient Systems, Networks and Technologies 2018) the 8th International Conference on Sustainable Energy Information Technology, (ANT 2018) SEIT 2018, 8-11 May, 2018, Porto, Portugal

IoV distributed architecture for real-time traffic data analytics IoV distributed architecture for real-time traffic data analytics Mohamed Nahriaa, Azedine Boulmakoulaa*, Lamia Karima,b , Ahmed Lbathc Mohamed Nahri , Azedine Boulmakoul *, Lamia Karima,b, Ahmed Lbathc a a

LIM/IOS, FSTM, Hassan II University of Casablanca, B.P. 146 Mohammedia, Morocco b Higher School of Technology EST of Berrechid, Hassan University, Morrocco LIM/IOS, FSTM, Hassan II University Casablanca, B.P.1st 146 Mohammedia, Morocco b c Higher School of Technology ESTAlpes, Berrechid, 1st University, University Grenoble CNRS,Hassan LIG/MRIM, France Morrocco c University Grenoble Alpes, CNRS, LIG/MRIM, France

Abstract Abstract In this paper, we present necessary premises for the deployment of the Internet of Vehicles (IoV) integrating Big Data analytics In this we traffic presentmeasurements necessary premises for the deployment of Morocco. the InternetThus, of Vehicles (IoV) an integrating Big based Data analytics of roadpaper, network of the city of Mohammedia, we introduce architecture on three of road network of the of Mohammedia, Morocco. Thus, we introduce an architecture based on layer three main layers such traffic as IoV,measurements Fog Computing andcity Cloud Computing Layer. We specifically put more focus on Fog Computing main layers as IoV, Fog Computing and Cloudcollecting Computing We specifically put more on Fog Computing layer in which wesuch develop a framework for a real-time andLayer. processing events generated by focus intelligent vehicles as well as in which wetraffic develop framework forsection. a real-time collectingwe and processing events generated bythe intelligent as well as visualizing statea on each road Furthermore, consider deployment and test of proposedvehicles framework using visualizing trafficfrom statea on each road micro section. Furthermore, we consider deployment andthe testfirst of the proposed framework using events retrieved Vanets-type simulation. Finally, we present and discuss obtained results as well as the events retrieved from a Vanets-type microarchitecture. simulation. Finally, we present and discuss the first obtained results as well as the advantages and limitations of the proposed advantages and limitations of the proposed architecture. © 2018 The Authors. Published by Elsevier B.V. © 2018 The Authors. Published by Elsevier B.V. © 2018 The Authors. Published by B.V. Program Chairs. Peer-review responsibility of Elsevier the Conference Conference Peer-review under under responsibility of the Program Chairs. Peer-review under responsibility of the Conference Program Chairs. Keywords: IoV, Big Data analytics, Fog computing, Real-time data analytics, Traffic control. Keywords: IoV, Big Data analytics, Fog computing, Real-time data analytics, Traffic control.

1. Introduction and motivation 1. Introduction and motivation Intelligent vehicle concept has grown considerably over the past decade. Many vehicles have on-board systems Intelligent vehicle concept has grown considerably over the conditions. past decade.The Many vehiclesofhave incorporating components monitoring different environmental emergence the on-board internet ofsystems things incorporating components monitoring different environmental conditions. The of emergence of thefrom internet of things (IoT) and communication technologies allows collecting different types information sensors and (IoT) and communication technologies allows collecting different types of information from sensors and

* Corresponding author. Tel.: +212 5 23 31 47 08 ; fax: +212 5 23 31 53 53. * Corresponding author. Tel.: +212 5 23 31 47 08 ; fax: +212 5 23 31 53 53. E-mail address: [email protected] E-mail address: [email protected]

1877-0509 © 2018 The Authors. Published by Elsevier B.V. Peer-review©under the Conference Program 1877-0509 2018responsibility The Authors. of Published by Elsevier B.V. Chairs. Peer-review under responsibility of the Conference Program Chairs.

1877-0509 © 2018 The Authors. Published by Elsevier B.V. Peer-review under responsibility of the Conference Program Chairs. 10.1016/j.procs.2018.04.055

2

Mohamed Nahri et al. / Procedia Computer Science 130 (2018) 480–487 Nahri et al./ Procedia Computer Science 00 (2018) 000–000

481

surrounding systems. The convergence and the opening of these technologies lead to the emergence of a new concept called IoV. Exploiting data generated by IoV environment in a centralized manner will certainly have great benefits for traffic control. In this work, we propose the implementation of IoV integrating Big Data analytics. Thus, we introduce an architecture for collecting and analyzing big data traffic respecting IoV environment specificities and allowing efficiency, scalability and high performance. Furthermore, we are developing a traffic data collecting and processing framework. The first tests of this framework are performed thanks to events retrieved from Vanettype simulation. The rest of this paper is organized as follows. In section 2, we present a state of art of IoV environment interacting with Big Data analytics. Section 3 develops the proposed architecture and its different parts including the developed framework. Deployment, tests and results of the framework are given in section 4. Section 5 highlights advantages and limitations of the proposed architecture. Finally, a conclusion of this work is developed in section 6. 2. State of art Internet of things is mainly based on smart objects working in a collaborative manner and interacting instantly with surrounding environment. The emergence of IoT has opened new perspectives for intelligent transportation systems. IoV represents a particular case of IoT accepting information exchange between vehicles themselves as well as between vehicles and any kind of surrounding objects (see Fig. 1) such as traffic lights, infrastructure, pedestrians and Cloud1. This concept is more general than vehicular ad-hoc networks (Vanets), which is limited to vehicle to vehicle (V2V) and vehicle to infrastructure (V2I) communication2. IoV technology is more developed than classical intelligent transportation systems based on sensors and cameras throughout road network and monitoring center3. The rapid development of wireless communication technologies has favored the emergence of IoV architecture. These technologies involve particularly RFID4, WSN5, Zig Bee6, NFC, Bluetooth, 6LoWPAN7 and other practices which have shaped M2M communication as well as dedicated communication technologies to vehicular environment8 such as 802.11p,WAVE and CALM. IoV architecture is also favored by the development of smart infrastructure for smart-cities including MAN networks. Cellular network technologies have recently considerably grown to the extent of testing 5G technology, which will begin deployed by 2020. Ensuring a speed 100 times larger than existing 4G9, 5G technology will certainly open great opportunities in several areas, especially for IoV10. All these communication technologies and other practices support IoV. However, this latter faces many challenges among with exchange standardization and interoperability between different technology kinds as well as routing, storage and analysis of large quantities of data generated by heterogeneous sources. Interoperability challenge can be overcome by adopting exchange protocols such as MQTT, AMQP, STOMP, HTTP and others. The work cited in11 propose an architecture based on these protocols ensuring interoperability and transparency between connected objects. Effectively, these protocols require a tiers system such as RabbitMQ and ActiveMQ functioning on the level of a layer called middleware. This latter is often confused with so-called Fog layer, which is positioned between Cloud layer and connected objects layer. Fog computing layer mainly ensures interoperability and security exchanges between connected objects as well as massive data management and data preprocessing. The work of Bonomi12 introduce the role of this layer integrating IoT and Big Data analytics qualifying it as an ideal layer for the real time data processing showing a low latency. In13, authors have proposed the creation of several instances of Fog layer according to geographical distribution of connected objects. The Fog layer will certainly have a decisive role in our work tending to integrate IoV with the real-time analysis of events produced by connected vehicles. The design and technological choices for this layer will be done with great care regarding to huge amount of shared data, which we will develop in section 3. 3. Modeling and proposed architecture We propose an IoV-BigData architecture based on three layers (see Fig. 1). The IoV layer includes connected vehicles, intelligent road infrastructure and communication technologies. The Fog Computing layer is in itself responsible for the collect, real-time processing and storage of events generated by the connected vehicles and

482

Mohamed Nahri et al. / Procedia Computer Science 130 (2018) 480–487 Nahri et al./ Procedia Computer Science 00 (2018) 000–000

3

infrastructure. Finally, the Cloud Computing layer performs data storage, real-time data visualization and the Fog Computing results analysis.

Fig. 1. IoV-Big Data architecture.

The main objective is the analysis of events generated by connected vehicles in real time. Namely, we aim to extract real time traffic situation in each road network section. To attain such a result, architecture layers, explained before, work as follows:  Connected vehicles, at the level of IoV layer, periodically generate events including position and individual Travel Time Index (TTI)14.  In Fog computing layer, elements are responsible for collecting and processing massive flows of generated events as well as calculating TTI for each road section, along with making these results available to the CloudComputing layer. Moreover, to keep traceability, all collected events are stored in a BigData database.  Cloud layer components allow collecting these results, storing them in a Big Data database for a high-level analysis and provide real-time visualization of the traffic situation. 3.1. Intelligent vehicle and infrastructure IoT has revolutionized the automotive industry in recent years. Automakers invest heavily to improve smart vehicle performance. This trend has opened huge opportunities for the intelligent transportation systems. IoV is a very difficult concept referring to the diversity of the proposed architectures, communication technologies and hardware and software solutions. IoV layer development concerns other works performed by our team. For instance, patent given in 19 shows an interesting way favoring the deployment of an IoT architecture for real-time communication between vehicles and infrastructure. In the rest of this work, we are interested on data generated by vehicles regardless IoV communication. Thus, a vanets-type simulator has replaced IoV layer, generating events for each vehicle describing individual state. Namely, generated events inform on position, individual travel-time index (TTI) and other information such as timestamp and vehicle id. In the following, we focus more on data collection and analytics Framework.

4

Mohamed Nahri et al. / Procedia Computer Science 130 (2018) 480–487 Nahri et al./ Procedia Computer Science 00 (2018) 000–000

483

3.2. Real-time data collection and analytics framework In this part, we describe Fog Computing and Cloud Computing layers commented above. Precisely, we focus on the real-time data flows collection and processing. Then, we introduce three components for our data collection and processing framework. The first is responsible for events collection from IoV layer. The second focuses on event flows processing. Recall that the principal mission of this last component is to calculate TTI on each road network section from space-time events generated by intelligent vehicles. The third component concerns the transmission and/or provision of processing results to the Cloud layer for performing real time visualization and high-level analysis. Before developing each component features, let us introduce the characteristics of technological choices. These latters must justify great fault tolerance, support of large data streams and low latency to stay in real-time zone. Data flows collection is usually performed by message broker systems. Kafka greatly exceeds other message brokers such as RabbitMQ and ActiveMQ, supporting larger amount of messages and presenting less latency15. Therefore, this makes it the most suitable message broker for large data flows. Kafka is a distributed platform working under Zookeeper16 which is specialized in the orchestration of distributed and high-performance systems. In our framework, we distinguish two types of interlocutors interacting with Kafka. The first includes connected vehicles behaving like message producers. The second includes Spark and storage microservice behaving like message consumers (see Fig. 2). Kafka organizes messages in a series of topics in which each topic contains sequences of messages working in FIFO. Moreover, Kafka adds an important aspect manifesting on the group partitioning system that we qualify in our case to be very important. The principle of this partitioning system is presented as follows: two consumers enrolled in two distinct groups can independently consume messages of a single subject. Conversely, the FIFO mechanism is applied to consumers enrolled in the same group. This principle allows the system to process message flows in parallel with storage. Moreover, this principle favors the parallel data flows processing. Choosing a system for event flows processing seems to be difficult regarding the variety of the proposed items and the space-time nature of events. The work cited in17 compare three continuous flow-processing systems, which are Spark Streaming, Storm and Flink, concluding that Storm and Flink present low latency than Spark Streaming. However, there is a major difference in flows processing between these three systems. Namely, spark Streaming processes data by batches, while Storm and Flink process data by records. As previously mentioned and taking in mind that we aim to calculate TTI displayed in each road section from space-time events for a given time interval, the batch processing is very essential in this case. Therefore, we use Spark as flows processing system. Indeed, this system integrates Spark Streaming, Spark Core and Spark SQL libraries. Event batches processing is performed as follows:  Spark Streaming receives event batches from Kafka with frequency of five seconds, while connected vehicles send the events with frequency of one second. The Dstream is an RDD (resilient distributed dataset) that represents a sequence of events.  Dstream is reduced by an events aggregation by vehicle and by section averaging their individual TTI.  TTI for each section is calculated using SparkSQL components such as Dataframes and Dataset.  The results are finally sent to another Kafka on the Cloud (see Fig. 2).

Mohamed Nahri et al. / Procedia Computer Science 130 (2018) 480–487 Nahri et al./ Procedia Computer Science 00 (2018) 000–000

484

5

Fig. 2. Component diagram of the real-time event processing architecture.

Figure 2 illustrates the Component diagram of the real time event flows processing system. The developed framework is based on Kafka, Spark, MongoDB and Microservices. The latters are developed with Spring Boot and deployed in Docker container. Indeed, they are used primarily for the storage and results visualization. Data storage is occurring on two times. The first concerns the storage of vehicle events and the second concerns the storage of events describing road section situations. All events pursue the following process:  First, vehicular events are produced as JSON text format and then they are published in “Vehicle” topic of the first Kafka server. For instance, an example of transmitted events is presented as follows: {"idvhl": "vhl19", "time": "2017-12-12T14:25:04+00:00", "tti": 1.84883, "lat": 33.6930, "long": -7.3925, "lane": "hassan2_21"}.  These vehicular events are consumed by storing microservice simultaneously with Spark Streaming calculating TTI for each road section. These situations are in the JSON text format and they are presented as follows: {"idlane": "hassan2_21", "time": "2017-12-12T16:12:35+00:00", "tti": 5.46418}. Then, these resulting events are published in “LaneSituation” topic of the second Kafka server.  Similarly, resulting situation events are consumed by storing microservice simultaneously with Spark Mlib as well as dashboard microservice for real-time visualization. We report that the fog layer described above starts with the first Kafka server and terminates with Spark streaming as mentioned in figure 2. In the following, we describe the deployment of this architecture and we present the first results. 4. Deployment and first results To perform the first tests on the architecture, we deployed each component on a separate machine (see Fig. 3). Kafka, MongoDB and Docker platforms were deployed on i5 machines with 4 GB of RAM. While Spark ecosystem was deployed on an i5 machine with 16 GB of RAM. Moreover, all machines are running under Debian8.8 operating system.

6

Mohamed Nahri et al. / Procedia Computer Science 130 (2018) 480–487 Nahri et al./ Procedia Computer Science 00 (2018) 000–000

485

Fig. 3. Deployment diagram.

In order to accelerate tests, we used a Vanets-type simulator to generate vehicular events. In addition, we performed the tests choosing a limited area of the Mohammedia network as shown in figure 2. The following results were displayed in real time. Figure 4 shows a synopsis of the traffic situation around Hassan 2 Avenue. In this visualization, three levels of congestion are distinguished. The green indicates a fluid traffic on the section, brown color shows an average congestion for the section and red color means that the section is congested.

Fig. 4. Synoptic of the traffic status.

Figure 5 shows the real time variation of the inverse of TTI over time displayed on the three sections shown on the map below. The correlation between the three sections is obvious.

486

Nahri et al./ Procedia Science 00Science (2018) 000–000 Mohamed Nahri et al. /Computer Procedia Computer 130 (2018) 480–487

7

Fig. 5. TTI Variation on the indicated 3 links.

5. Benefits and limitations of proposed architecture 5.1. Security issue for IoV Security presents a big challenge for IoT environments regarding the very large number of information exchanges performed and the diversity and openness of used communication technologies. Particularly, security is very complex for IoV due to its particular context characterized by high mobility of vehicles as well as high dynamism of telecommunication topologies. In such a context, attacks can affect different parts of the ecosystem starting by intelligent vehicles and infrastructure up to data-collection and analysis servers. The security issue for IoV has been matter of several research studies. The work of Sakiz and Sen18 presents different types of attacks that may constitute big danger for vehicular networks in Vanets and IoV. We focus more on the functional aspect of IoV taking the necessary countermeasures to secure our system. 5.2. Advantages of proposed solution Architecture presented in this paper presents several advantages obtained from the separation of real-time processing layer and delayed-time processing layer. Namely, this separation allows detecting road traffic anomalies in a lower latency by carrying out light processing at the fog layer level. Furthermore, it allows the possibility of instantiating fog layer in the way want and as much as we need regarding according to geographic areas. This last possibility allows efficiency and scalability for the developed framework. Effectively, we can having multiple instances of fog layer displaying real-time results and single instance of Cloud Computing layer for advanced and comprehensive data analysis. In addition, another advantage coming from components used in developed framework which provide the ability to operate in cluster ensuring high performance, scalability, fault tolerance and support of large amount of data. 5.3. Future works and limits to anticipate Architecture presented in this paper shows some limitations described as follows:  Developed framework has not been tested with large quantities of data. Therefore, deploying the solution in an intensive mode is necessary. Hence, identifying its handicaps and taking necessary measures results in mitigating their intensity.  Interoperability issue in IoV has not been treated. However, near version requires integration of all components including IoV components and performing system integration tests.

8

Mohamed Nahri et al. / Procedia Computer Science 130 (2018) 480–487 Nahri et al./ Procedia Computer Science 00 (2018) 000–000

487

 Security issue discussed above has not been considered in the architecture. Then, we will align our activities with some practices in item. 6. Conclusion This work tends to integrate IoV and real-time Big Data analytics. For this purpose, a collection and analytics framework has been developed. The developed framework is based on Big Data technologies that have proven their performance such as Kafka, Spark, MongoDB and Microservices ensuring scalability and high performance. The obtained results represent an interesting step towards a deep analysis of road traffic state. Moreover, other works that concern both IoV and Big-Data Analytics are planned to complete overall architecture prototype development. Furthermore, to achieve more improvement regarding robustness, performance and efficiency of our analytics framework, we have envisaged in the short term concerning the following points:  Testing the architecture over a wide geographical area and evaluating the real-time aspect.  Finalizing the implementation of the Cloud Computing layer. In particular, achieving development of the part of Big Data analytics deriving an intelligible knowledge for the road traffic management. References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19.

Maglaras L, Al-Bayatti A, He Y, Wagner I, Janicke H. Social Internet of Vehicles for Smart Cities. J Sens Actuator Networks. 2016;5:3. Available from: http://www.mdpi.com/2224-2708/5/1/3 Wang Q, Fan P, Letaief K Ben. On the joint V2I and V2V scheduling for cooperative VANETs with network coding. IEEE Trans Veh Technol. 2012;61:62–73. Lamboley C, James B. Apport des nouvelles technologies dans la gestion du trafic urbain, Les Cahiers du numérique, 2001/1 (Vol. 2), Pages : 256, Éditeur Lavoisier 2001. Chen, M, Chen, S. RFID Technologies for Internet of Things, ISBN 978-3-319-47355-0, https://doi.org/10.1007/978-3-319-47355-0, Springer Cham. Sheng Z, Mahapatra C, Zhu C, Leung VCM. Recent advances in industrial wireless sensor networks toward efficient management in IoT. IEEE Access. 2015;3:622–37. Baronti P, Pillai P, Chook VWC, Chessa S, Gotta A, Hu YF. Wireless sensor networks: A survey on the state of the art and the 802.15.4 and ZigBee standards. Computer Communications 30 (2007) 1655–1695. Mulligan G. The 6LoWPAN architecture. In: Proceedings of the 4th workshop on Embedded networked sensors - EmNets ’07. New York, USA: ACM Press; 2007. p. 78. Available from: http://portal.acm.org/citation.cfm?doid=1278972.1278992 Paul A, Chilamkurti N, Daniel A, Rho S. Chapter 1 – Introduction: intelligent vehicular communications. Intell Veh Networks Commun. 2017;1–20. Lauridsen M, Giménez LC, Rodriguez I, Sørensen TB, Mogensen P. From LTE to 5G for Connected Mobility. IEEE Commun Mag. 2017;55:156–62. Soldani D, Manzalini A. Horizon 2020 and beyond: On the 5G operating system for a true digital society. IEEE Veh Technol Mag. 2015;10:32–42. Ngu AH, Gutierrez M, Metsis V, Nepal S, Sheng QZ. IoT Middleware: A Survey on Issues and Enabling Technologies. IEEE Internet Things J. 2017;4:1–20. Bonomi F, Milito R, Natarajan P, Zhu J. Fog Computing: A Platform for Internet of Things and Analytics. In 2014. p. 169–86. Available from: http://link.springer.com/10.1007/978-3-319-05029-4_7 Negash B, Rahmani AM, Liljeberg P, Jantsch A. Fog Computing in the Internet of Things. 2018;3–13. Available from: http://link.springer.com/10.1007/978-3-319-57639-8 Falcocchio J, Levinson H. Road Traffic Congestion: A Concise Guide. Vol. 7, Springer Tracts on Transportation and Traffic. 2015. p. 110113. John V, Liu X. A Survey of Distributed Message Broker Queues. arXiv:1704.00411 [cs.DC] 2017; Junqueira F, Reed B. ZooKeeper: Wait-free Coordination for Internet-scale Systems, USENIX annual technical conference 8 (9), 2010/6/23. Chintapalli S, Dagit D, Evans B, Farivar R, Graves T, Holderbaugh M, et al. Benchmarking streaming computation engines: Storm, flink and spark streaming. Proc - 2016 IEEE 30th Int Parallel Distrib Process Symp IPDPS 2016. 2016;1789–92. Sakiz F, Sen S. A survey of attacks and detection mechanisms on intelligent transportation systems: VANETs and IoV. Ad Hoc Networks. 2017;61:33–50. Available from: http://dx.doi.org/10.1016/j.adhoc.2017.03.006 Boulmakoul, A., Karim, L. and Daissaoui A. ιΤmS_2 C: A real-time monitoring intelligent transportation system and symbolic control of urban traffic congestion based on the diffusion of e-fuzzy gaussian pheromone and deployed on a cloud-based distributed ecosystem integrating virtual detection sensors. 2016. In International Patent: PCT-MA2016-000007PCT-MA2016-000000. March 2016.