Development Of Flexible Autonomous Car System Using Machine Learning and Blockchain S.Shreyas Ramachandran1, A.K.Veeraraghavan2 ,UvaisKarni3, K.Sivaraman4 1,2,4 UG Student, Department of Electrical and Electronics Engineering, Sri Sairam Engineering College, West Tambaram,Chennai 3 UG Student, Department of Computer Science and Engineering, Meenakshi College of Engineering, Chennai Email:1
[email protected],
[email protected], 3
[email protected],4
[email protected] Abstract- Autonomous Driving car is an upcoming technology. In our project, we are taking a step towards this vision by developing a system using Raspberry Pi, image processing and machine learning and connect the system to any electric car. The proposed system Provides an autonomous car feature to any existing electric car on road that doesn’t has autonomous driving feature inbuilt within it. Most existing electric cars that are on roads don’t have this technology and this is mostly found in new and expensive cars. An alarming fact about the autonomous cars is that, many of them are being frequently hacked, indicating a problem related to security. The application of blockchain network, which seems to provide security and transparency in the usage of network is employed to transfer data. Using the proposed system, such autonomous car feature can be installed separately at a cheaper expense in all existing electric cars. We aim to achieve the above by using image processing which is trained by using neural networks to create a model through which autonomous cars are achieved. With the usage of blockchain network, security and transparency of data transfer can be achieved. The hardware components used in this project are Raspberry PI 3 B microcomputer and camera module. This Raspberry Pi and camera unit forms a separate system which, when connected to the electronic control unit, helps the car to drive automatically. Keywords: image processing; Blockchain ; Raspberry Pi; machine learning; Convolutional Neural Network ; autonomous system; Electronic Control Unit(ECU); IoT I.
INTRODUCTION
In our upcoming world, the number of accidents occurring has increased drastically during the recent years leading to increase in fatal deaths. This is mostly caused by the distractions a driver encounters , for example, texting and driving,less attention span of driver,etc. The worldwide ADAS and autonomous driving segments showcase produced $2.76 billion in the year 2016, as demand rose for driver assistance features such as adaptive cruise control, parking assistance etc [1]. Research has been going around the globe for improving vehicular communication. Various industries are coming up with innovations, providing autonomous self driving feature for the automobile industry. Self driving autonomous driving car technology is a booming field in which, lots of companies are investing to develop, cars based on this technology. Companies have started manufacturing cars that are able to drive automatically and driver assistance. Only 8% of the cars in the world have the connected car technology. To increase the count of the cars that have this feature would require the people across the globe to buy a new upcoming car in the future, forsaking the old cars. The objective of the project is to achieve autonomousity in an existing electric car so as to eliminate the existing disadvantages that comes with a human driver. This is achieved by using image processing which is trained by using neural networks and machine learning to create a model through which autonomous cars are achieved. The application of blockchain in the system is to provide security for data transmission. Block chain was introduced in the year of 2014 by DLT and has been growing vastly and has gained a lot of support. Its mainly been used in IOT structure based device. Its been predicted that by 2025 all IOT deployments will have the basic level of block chain service enabled. Block chain provides a lot of features that help in achieving peer to peer level based connectivity. Its success has been already proven in field of cryptocurrency.The key concept of
block chain is shared database that is distributed across multiple computers, and over time, research have been taking place for the application of blockchain for security purposes [2]. The block diagram of this system is represented in figure 1.
Fig.1 Block diagram
II.
RELATED WORKS
Each huge automaker is seeking after the tech, anxious to rebrand and remake itself as a "portability supplier" before the possibility of vehicle proprietorship goes done for. Waymo, the organization that rose up out of Google's self-driving vehicle venture, has been busy the longest, yet its syndication has dissolved generally. Ride-hailing organizations like Lyft and Uber are hustling to reject the benefit eating human drivers who currently carry their clients about. There are numerous other dynamic research programs concerning selfsufficient vehicles, huge numbers of them highlighting joint efforts among colleges and carmakers. Oxford University, for instance, showed a self-driving Nissan LEAF in 2012. Volkswagen and an examination group from Stanford University have made a driverless Audi sports auto, which has been flashing around US race tracks. In another examination venture financed by the European Union, Volvo effectively drove a caravan of five vehicles that just had a human driver ahead of the pack auto.An Ad-hoc based Block-VN model have been researched upon for a new distributed blockchain architecture based on the vehicle network [3], and with the integration of blockchain and IOT, the scope of research has widened [4]. III.
HARDWARE ARCHITECTURE
All the work done in the field is meant exclusively for cars that are new and expensive ones. The model proposed in our system provides a system that is adaptable to any vehicles and could be interfaced with any vehicle’s Electronic control Unit. In our model, we interface an Raspberry Pi to the ECU of a car. A combination of Raspberry Pi and a camera is used over here for image processing of data is required. The entire system consists of 2 parts.
a) Main Controller System The central system consists of a camera, interfaced with Raspberry Pi forms a system and is used to keep our area under constant surveillance. The images that are taken by this camera is processed frame by frame by the Raspberry Pi and OpenCV using python programming language. We utilize Raspberry Pi 3 Model B has a Broadcom BCM2837 64bit ARMv7 Quad Core Processor Single-board computer with remote LAN and Bluetooth connectivity. We use this model, particularly because it is good in processing images and videos at a faster rate. PI camera module captures 2592 * 1944 pixel static images and also supports 1080p at 30fps @ 60fps and 540 * 480p 60/90 video recording. Camera module is interfaced with the central microcontroller Raspberry Pi 3. Secure Digital (SD) cards are used to store the operating system and program memory in either SDHC or Micro SDHC sizes. It will require micro SD cards.
b) Central Server System The central server mainly consists of web servers and central storage devices. The server enables connection between the main controller system and the central storage. For the communication with main controller system, the system uses a GPS for communication based on the location. The location of the image detected is provided to the farmer using GPS This module named Neo 6M-0-00-1 U-Blox operating at 5V DC. The Global Positioning System is used for tracking the location of each Device placed in various parts. The Global Positioning System is connected up with satellites, ground stations, and receivers. Once the receiver calculates its distance from four or more satellites, it knows our exact location . IV.
SYSTEM OPERATION
The operation of the separate attachable system is to provide easy automatic control of electric vehicle. For this purpose, the operation of the vehicle is split into 2 parts
a) Image processing of the surrounding environment First, is the main controller system which is trained using image processing. The images for training are and the vehicle is controlled utilizing directional arrows and every one of the pictures are recorded in a similar organizer alongside the relating bearing of turn. This training is done using Neural NetworksOne benefit of utilizing neural system is that once the system is trained, it just needs to stack trained parameters a while later, along these lines forecast can be quick. The neural network used is convolution neural network - CNN because we are working on classification orientated output.CNN also has a very high rate of accuracy which is around 95 %. CNN is similar to any other NN, the only difference being that it processes on chuck sized data, that is it can analyze detailed patterns. CNN influences utilization of filters to identify what highlights, for example, edges, are available all through a picture. The filter moves over each piece of the picture to check if the component intended to identify is available. To give an esteem speaking to how certain it is that a particular component is available, the filter completes a convolution activity[5]. Using cost function we can find the most effective learn rate suitable for the model. As its with all neural network the images used to train network need to be converted to n-array and labeled, CNN is supervised so all the data that is used to train the model must be labeled. There are 38,400 nodes in the info layer and 32 nodes in the shrouded layer. The quantity of nodes in the concealed layer is picked genuinely subjective [6]. Now that the model is trained , all that needs to be done is classify the output generated when prediction is run which is quite simple and can be achieved using sigmoid function. Sigmoid is a gradient curve which classifies the output as forward ,backward , right extra.The images captured during training are partially loaded into the main controller system, which has SD card and the remaining images are stored into the storage of the central server system. This image splitting is mainly done to ease the operation of main controller system. The images stored in the main controller system help the vehicle to monitor the immediate surrounding of the vehicles such as vehicles in front, irregularities on the roads etc. the images stored in the central server, help the vehicle to navigate by feeding the surrounding map of the images though internet, based on the GPS location of the vehicle. This helps the portable system to be more efficient and reduces training of multiple controller systems for every vehicle. The data and images obtained by training the vehicle is split into 2. The data and images required for image processing of immediate surroundings are stored within the SD card of the main controller system. The images and data required for navigation and mapping are stored in the storage units in the central server system. The images and data of the surrounding map are transmitted to the vehicle using the GPS location and internet. Based on the images of the surroundings such as vehicles, irregularities and navigation route, the main controller system helps in controlling the motion of the vehicle by controlling the motor.
b) Communication between server and cars For communication between the central server and vehicles, we use IOT and blockchain. Block chain allows the dependency of central control towers to be neglected as all the data are stored, distributed and it works by connection within trusted devices , so there will be no misunderstanding. Most importantly all data are stored redundantly across the devices , there will be no issues when a node i.e the IOT device gets compromised. The main reasonwhy block chain is been used is due to data integrity and data security which is achieved by using encryption standards. Block chain and IOT have several use cases that it can be used when combined , both these technologies work together perfectly as shown in figure 2. IOT consist of modules that has sensors , they are used to collect large set of data that can be used for developing users experience subsequently much more and block chain provides environment where all these IOT modules can connect and securely save all these data.
Fig 2. Block diagram of Blockchain Communication Architechture for Autonomours Car system
Fig 3. Representation of Block Chain Implemented
Autonomous cars are developed by processing heaps of data. these data are very important because they are used to train the AI models that run these cars so its important that we transmit it securely. That is the cars are being driven over various locations and the immense data generated by the car’s sensors needs to transmitted and processed to improve that accuracy and behaviour of the cars. This is where block chain comes , all the benefits of block chain can eliminate the issues faced by the cars in the present scenario. There have been several reports that autonomous cars such as Tesla are being hacked. There are countless disturbing actions that can be carried out by the hacker ,such as complete control of the car. By the integration of block chain,all the
data will be encrypted and safe. The above mentioned idea is just one of the advantage. The Block chain data structure is a back-linked list of blocks of transactions. Each block is identifiable by a hash .These hash are being generated by algorithm on the header of each block. The SHA 256 algorithm generated 32 bytes hash is impossible to reverse to the output. It is also impossible to find two block with the same hash as it is collision free.Effectiveness of Block chain has been already proven by Bit-Coin and crypto-currency [7]. The data stored on the car is gold, as it can be used to upgrade the AI model but their is problem that is the data is too big to store it all.Hence we transmit to the Server. Transmitting the data is a good idea although it can be stolen by hacker and lead to loose of privacy.So we implement block chain structure to transmit data between the demanded points.Block Chain provides all the advantages mentioned above and mitigates all the issues faced. The implementation of SHA 256 , block structure and hash function makes it impossible to steal or modify data. The Data is then stored by server which is used to update model and this model can also be transmitted back to the car safely. Application of the system, we provide periodical updates, i.e OTA updates to the cars securely using Block Chain, that helps for car to run on latest map updates, firmware and also provide immediate bug fixes when needed. Region Based Software Version allows the car to run on model which are generated based on region. To be more specific the prototype car can be run on a particular city and based on information collected the model can be generated ,its because the different cities have their own set of rules and regulation. By using this method those problems are resolved. A Path Prediction by default as city level data mapping is used to train the model. So it can be used to add precautionary one step ahead information, so that the car can see what is coming up ahead.
c) Control of Car Each vehicle has a motor driver which provides motor with the pulses for operation. The pulses are given according to the acceleration given by the user. Generally, the pulses are given using PWM modulation to the motor to operate with better accuracy. Here, instead of the driver giving the acceleration, the pulses are given based on the images and data processed by the controller. Based on the images fed to the controller system, the controller will provide pulses to the motor driver which will control the motion of vehicle. In the case of a road dwelling car the components needs to scaled in order to control it efficiently.It’s basically the same set of hardware modules used.We are using UMC Drive 3.0 Universal Motor Controller, as shown in figure 4,which as the name denotes allows supports a lot of electric road cars like Toyota Prius ,Tesla Model S, Tesla Model X, Nissan Leaf ,Chevrolet Volt and Smart EV. The benefit of this Motor Controller is that it adapts to respective cars drive cycle power stages , inverter drive modes, sensors does not to be changed which greatly reduces the initial conversion cost and its supports sensor-less drive mode too [8]. The UMC provides a CAN bus,3 phase full bridge control signals as well as a Resolver and inputs encoder. For bus and phase measurements are 4 isolated High Voltage inputs. The Hardware is over-current trip protected and the Digital input, output channels are isolated.
Fig 4. UMC 3.0 ADVANTICS
Emulation of legacy instrumentation clusters such as speedo, RPM, temp and fuel are easily done. The process of controlling the motor remains the same using pulse modulation involving a encoder with more senor based inputs from the motor.In terms of hardware a high voltage capacity relay must interfaced with raspberry to support the currents levels needed to drive the motor [9]. V.
RESULT AND TESTING
To prove the viability of the proposed concept, it was implemented in a generic RC cars shown on Figure 5. The raspberry Pi and the camera as the main controller system, were able to control the cars motion by suitably providing pulses to the driver of the RC car. The performance of the RC Autonomous Down scaled Model Car mentioned in the paper has been evaluated by testing it on a map.
Fig 5 The RC Autonomous Down scaled Model Car
Here, first the car is controlled by user when the Pi Cam takes pictures of its environment, and so a map is built. This neural network model is built using openCV on a laptop and was fed to the car, makes the car ride correctly o the map without user input as it knows what direction has to be changed how to control the motors on the map based on the neural network model from the pictures taken by Pi cam as shown in Figure 6.
Fig 6. RC car on Map riding autonomously
Also identifying stop sign works via Haar Cascade and the Car stops when it comes by a stop sign as shown in figure 7. As see from above, the concept has been proven correct on an RC toy car and so can be applied on electric cars on a large scale.
Fig 7.Haar Cascade Stop Sign Working
VI.
FUTURE SCOPE
As the system the integrated with internet, it can used for developing connected cars and V2X system. The application of internet of vehicles will bring in wider range of features into the system. 3D laser can be used for effective mapping environment. The model can be trained over several laps to increase accuracy and adapt to traffic and other obstructions. VII.
CONCLUSION
The proposed system in this paperis based on the flexible autonomous car whosedownscaled system was developed based on the mentioned algorithms. A flexible autonomous system of this kind can be a path changer which can be installed in any existing electric vehicles and thus the dream of realizing wide spread autonomous cars in a much safer way can be achieved.Incorporation of blockchain concept will be a perk in reducing the cost, enhancing security, better traceability, improved efficiency and operation speed. As a whole this techonology takes the vehicular industry to next step of development.
Acknowledgement The authors would like to acknowledge that downscaled model developed for testing, explained in the paper above has been made in to a prototype and was presented at IEEE SS12 Maker Fair 2018 Pilot at Jeppiaar Institute of Technology, Chennai, India and was recognized as being a noteworthy project and concept. VIII.
REFERENCE
[1] Q. Memon, M. Ahmed, S. Ali, A. R. Memon and W. Shah, "Self-driving and driver relaxing vehicle,"2016 2nd International Conference on Robotics and Artificial Intelligence (ICRAI), Rawalpindi, 2016, pp. 170-174. [2] Li, Xiaoqi& Jiang, Peng & Chen, Ting & Luo, Xiapu& Wen, Qiaoyan. (2017). A Survey on the Security of Blockchain Systems. Future Generation Computer Systems. 10.1016/j.future.2017.08.020. [3] Sharma, Pradip. (2017). Block-VN: A Distributed Blockchain Based Vehicular Network Architecture in Smart City. Journal of Information Processing Systems. 13. 184-195. 10.3745/JIPS.03.0065. [4] M. Samaniego and R. Deters, "Blockchain as a Service for IoT,"2016 IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData), Chengdu, 2016, pp. 433-436. [5] T. Guo, J. Dong, H. Li and Y. Gao, "Simple convolutional neural network on image classification,"2017 IEEE 2nd International Conference on Big Data Analysis (ICBDA)(, Beijing, 2017, pp. 721-724.
[6] Yamashita, Rikiya,Nishio, Mizuho Do, Richard KinhGian, Togashi, Kaori, Convolutional neural networks: an overview and application in radiology [7] M. Sivanesan, A. Chattopadhyay and R. Bajaj, "Accelerating Hash Computations Through Efficient Instruction-Set Customisation,"2018 31st International Conference on VLSI Design and 2018 17th International Conference on Embedded Systems (VLSID), Pune, 2018, pp. 362-367. [8] https://evannex.com/blogs/news/35045701-reverse-engineering-a-tesla-drivetrain [9] o Mueller, Peter &Ukil, Abhisek&Andenna, Andrea. (2010). Intelligent Motor Control. ABB Review. 27-31.
