Journal of Advanced Research in Dynamical and Control Systems
Vol. 9. Sp– 6 / 2017
MOBILE ROBOT TELE OPERATION USING ARC (AUGMENTED REALITY BASED CONTROLLER) 1 1
Shubham Pandey, 2Manik Jain, 3S. Nivash
B.Tech., Department of Electronics and Communication, SRM University, Chennai, Email:
[email protected]
2
B. Tech., Department of Electronics and Communication, SRM University, Chennai, Email:
[email protected]
3
Assistant Professor, Department of Electronics and Communication, SRM University,Chennai, Email:
[email protected]
ABSTRACT The scientific objective of this paper is to control a mobile robot via internet. The aim is to move beyond standard forms of interaction such as the keyboard and mouse which most people work with on a daily basis. This is seen as an unnatural way of working which forces people to adapt to the demands of the technology rather than the other way around. But a virtual environment does the opposite. It allows someone to fully immerse themselves in a visual world which they explore by means of their senses. This natural form of interaction within this world often results in new forms of communication and understanding. This work also demonstrates the use of an ARC (Augmented Reality Controller) for mobile robot teleoperation. The main objective of this paper is to establish the use of ARC in various day-to-day life applications. This system uses wireless serial communication to interact with the mobile robot. Moreover, the mobile robot is capable of doing live video streaming to monitor the environment. User needs to have a smartphone or a tablet or a PC with an internet browser to monitor the video streaming. Keywords: Augmented Reality, Live Video streaming,mobile robot, teleoperation.
1. INTRODUCTION Telerobotics is a new field of robotics, which has gathered attention of many researchers over the years [1, 2, 3]. The term ‘telerobotic’- Internet refers to a remotely controlled robot JARDCS
Special Issue On Trends and Future in Engineering
1153
Journal of Advanced Research in Dynamical and Control Systems
Vol. 9. Sp– 6 / 2017
system surveyed through the Internet. The growing interest in this area is stimulated by promoting the Internet, which provides access to various computing resources virtually anywhere in the world. The development of Internet provides the opportunity for the teleoperation of robots using standard communication protocol. The internet does not provide a guaranteed quality of service (QoS); it gives a number of limitations and difficulties, such as bandwidth constraints, transmission delay, packet loss, connection loss, etc. the above situation influences the performance of telerobotic systems based on internet [4,5,6]. The concepts behind augmented reality are based upon theories about a long held human desire to escape the boundaries of the ‘real world’ by embracing cyberspace. Once there we can interact with this virtual environment in a more naturalistic manner which will generate new forms of human-machine interaction (HMI). The main problems of the tele-robotics are the time delay, bandwidth and the quality of the video signal.
Our teleoperation system consists of a remote host PC and a mobile robot
connected with each other via the network over the Internet communication protocol TCP / IP (Figure 1). The operator via a virtual human-robot communication interface remotely controls the robot movements with the help of an intermediary client PC, and the operator has the opportunity to observe the environment through the camera, attached to the Raspberry Pi over the robot car, via the transmission channel (internet). Moreover, the Raspberry pi can be operated fully using an android application on a smartphone connected on the network.
Figure 1: Detailed architecture of teleoperation of robot car
JARDCS
Special Issue On Trends and Future in Engineering
1154
Journal of Advanced Research in Dynamical and Control Systems
Vol. 9. Sp– 6 / 2017
Augmented Reality controller designed to meet new forms of human-machine interaction (HMI). It offers an advantage that enable the embedded system designer to easily, quickly and seamlessly interface ARC with the microcontroller by UART for their various applications. The proposed framework can be utilized as a substitute for controllers across various platforms. It serves as a new method of human machine interaction. Home automation, Hotel, Movie theatres, Schools and Space stations are few areas, where ARC can be installed and operated.
2. COMPONENT DESCRIPTION 2.1 RASPERRY PI 2 Model B The Raspberry Pi 2 uses a Broadcom BCM2836 SoC with a 900 MHz 32-bit quad-core ARM Cortex-A7 processor (as do many current smartphones), with 256 KB shared L2 cache. Raspberry Pi 2 includes a quad-core Cortex-A7 CPU running at 900 MHz and 1 GB RAM. In Model B-the Ethernet port is provided by a built-in USB Ethernet adapter using the SMSC LAN9514 chip. The Raspberry Pi may be operated with any generic USB computer keyboard and mouse. It may also be used with USB storage, USB to MIDI converters, and virtuallythrough connection to any other device/component with USB port facilities. Other peripherals can be attached through the various pins and connectors on the surface of the Raspberry Pi. The Raspberry Pi primarily uses Raspbian, a Debian-based Linux operating system. It has 40 GPIO (General Purpose Input Output) pins and 4 USB ports. Moreover, it has additional connection interface for camera and display. 2.2 MOBILE ROBOT VEHICLE The robot vehicle consists of chassis and wheels attached to both the sides at its rear end. A castor wheel for the rotational movement of front side of robot vehicle. Two 9V DC motors are connected to drive the vehicle. As the output at the GPIO pins of the Raspberry Pi gives 3.3 V output, which is not enough to drive the motors. Therefore, we use a Pi development board which has L293D (H-bridge motor driver) and several additional functions too.
JARDCS
Special Issue On Trends and Future in Engineering
1155
Journal of Advanced Research in Dynamical and Control Systems
Vol. 9. Sp– 6 / 2017
Figure 2: Pi Development Board
2.3. WIRELESS CONNECTIVITY The Raspberry Pi is made wireless with the use of an adaptor (Edimax EW-7811Un). EW7811Un is a nano USB wireless adapter that supports maximum range and speed. Despite the size, this tiny USB adapter supports higher data rate of up to 150Mbps when connecting with wireless 802.11n device which is 3 times faster than your normally 11g connection. The configuration files for wireless connectivity on the Raspberry Pi OS are manipulated as per the requirement for the communication network.
Figure 3: Edimax Nano usb adapter
2.4 AUGMENTED REALITY CONTROLLER The ARC (Fig 4) projects the image for the control of robot vehicle, and the camera on the top of the controller, captures the response in front of it at particular interval of time. The images are sent to the system and is interpreted by the application developed for the image tracking and processing. On the basis of user and product response data is transmitted from the ZigBee connected to the system. According to image touched, a single character will be sent via UART. JARDCS
Special Issue On Trends and Future in Engineering
1156
Journal of Advanced Research in Dynamical and Control Systems
Vol. 9. Sp– 6 / 2017
Figure 4: Augmented Reality Controller
2.5 PI CAMERA We are using pi camera v1.3 5MP (Fig. 5) which is compatible to Raspberry Pi 2. It’s able to deliver a crystal clear 5MP resolution image, or 1080p HD video recording at 30fps.
Figure 5: Pi camera
2.6 ZIGBEE ZigBee devices are required to conform to the IEEE 802.15.4-2003 Low- Rate Wireless Personal Area Network (LR-WPAN) standard. The standard specifies the lower protocol layers are the physical layer (PHY), and the Media Access Control portion of the data link layer (DLL). The technology defined by the ZigBee specification is intended to be simpler and less expensive than other wireless personal area networks (WPANs), such as Bluetooth or Wi-Fi. Its low power consumption limits transmission distances to 10–100 meters line-of- sight, depending on power output and environmental characteristics.
JARDCS
Special Issue On Trends and Future in Engineering
1157
Journal of Advanced Research in Dynamical and Control Systems
Vol. 9. Sp– 6 / 2017
ZigBee devices can transmit data over long distances by passing data through a mesh network of intermediate devices to reach more distant ones. ZigBee is typically used in low data rate applications that require long battery life and secure networking. ZigBee has a defined rate of 250 Kbit/s, best suited for intermittent data transmissions from a sensor or input device.
Figure 6: ZigBee 2.4GHZ TI CC2500
2.7 POWER BANK We used a 16,000mAh power bank from ModMyPi- which is the most powerful portable solution for keeping your Raspberry Pi powered anywhere. After testing a single Raspberry Pi 2 Model B running 1080p Stargate-SG1 via OpenELEC, and it lasted for over 28 hours on the power bank. This power bank also features dual USB ports (one 5V @ 2.1A, and one 5V @ 2.4A), so you can power two Raspberry Pi's at the same time.
Figure 7: Power Bank by ModmyPi
JARDCS
Special Issue On Trends and Future in Engineering
1158
Journal of Advanced Research in Dynamical and Control Systems
Vol. 9. Sp– 6 / 2017
3. EXPERIMENTAL WORK We developed a windows compatible application (Fig 8) which serves as a GUI for the image tracking and processing. The image processing is done in the background with the help of codes embedded into the application. When the response is captured by the ARC, the application processes it and sends a serial data through the ZigBee (TX end) connected to remote user pc. The ZigBee (Rx end) connected to the Raspberry pi on the mobile robot vehicle receives the data from the transmitter end and then it responds to the commands issued by the ARC.
Figure 8: Image tracking & processing application
Figure 9: Control panel projected by ARC
The remote user can control the mobile robot on the video streaming feed provided by the pi camera. The user can keep his hands on the desired control block and the ARC transmits that data to mobile robot. There are many strategies to issue commands to the internet based robot such as ‘Move and Wait’ strategy [4] and GEMMA-Q [7, 8], these are used to control the operation of robot JARDCS
Special Issue On Trends and Future in Engineering
1159
Journal of Advanced Research in Dynamical and Control Systems
Vol. 9. Sp– 6 / 2017
based on the quality of service so that robot reaches the destination. In our experimental setup, we will be issuing commands to the robot on the basis of the latency in the video feed and reducing the transmission delay in control command issued by ARC. 3.1 ATTRIBUTES OF THE SYSTEM DESIGN a. Video Streaming For video streaming from the robot vehicle to the user system, we have opted for the most optimized and effective method from all the available methods (Fig 10). On the basis of the experiment with the pi camera module, we have achieved a delay of (
