A Person-following Robotic Cart Controlled via a Smartphone Application: Design and Evaluation Nathir A. Rawashdeh, Ramez M. Haddad, Omar A. Jadallah, Abdelhadi E. To’ma Department of Mechatronics Engineering German Jordanian University Amman, Jordan
[email protected] Abstract—Person following ground robots can have applications in military, civil, and domestic settings. Various sensors may be used to detect a person including cameras, infrared, laser, and ultrasonic sensors. This paper describes the implementation and testing of a robotic cart that is controlled via a smart phone application and can be tele-operated, as well as set into a follower mode where it uses ultrasonic sensors to locate the person walking in front of it, and keeps a constant distance. The cart’s usable loading area is 80x60 cm. The maximum carry capacity is 80 kilograms, but at an impractical velocity of 0.15 meters per second. Its maximum velocity is 1.4 meters per second at no load, and 1 meter per second at a load of 40 kilograms. The maximum operation time is 2 hours at full speed, but at a moderate weight and velocity, it reaches 4 hours. Keywords—smart phone; app; cart; mobile robot; follower
This paper describes the development and testing of an electric robotic cart that follows a person and avoids collisions using ultrasonic sensors. It is suited for carrying loads of up to 40 kg at walking speeds, in indoor environments, where the ground is flat and clean. For example, in warehouses, at airports, or shopping malls. The cart’s frame construction and an example user following application (i.e. supermarket isle) are depicted Figure 1. The user controls the cart via am smartphone application. It allows the user to make the cart stand still, while loading and unloading, or follow the walking user, in addition to allowing the user to tele-operate the cart via a Bluetooth link. The cart uses ultrasonic sensors to center itself relative to a person in front of it and to keep a constant distance while the person walks.
I. INTRODUCTION Indoor mobile robots are usually battery operated and use a differential drive mechanism with coaster wheels and a variety of sensors [1, 2]. There are many applications for human following robots, where such robots must localize the walking person and avoid obstacles. Some examples are: a hospital nurse following cart with a 20kg carrying capacity and utilizing ultrasonic sensors [3]; a luggage carrying cart that follows a wheelchair using a special color pattern and an optical vision sensor [4]; an automated shopping cart that uses ultrasonic and infrared sensors [5]; a line following robot that is installed under the shopping cart to move it intelligently, and is controlled by a smartphone application that guides the cart to the locations of the desired items to be purchased [6]; a robotic suitcase that follows the traveler based on wireless communication with a handheld device and embedded shoe sensors [7]; a ground robot that follows a marathoner while training - it is capable of obstacle avoidance and human runner detection using a laser range finder [8]. Some example technologies employed in follower robots are: a human following robot based on an RGB camera with a depth sensor [9]; a person following robot based on image processing using a Field Programmable Gate Array (FPGA) and an Internet Protocol (IP) camera [10]; a following robot that anticipates human behavior and can follow the person from behind, the side or the front. It also uses an RGB camera with a depth sensor [11]; a following robot that detects leg movement using a laser range finder. It also uses a camera for face detection [12].
978-1-5386-1883-7/17/$31.00 ©2017 IEEE
Fig 1. Overview of the robotic cart
II. DESIGN Design requirements resulted from defining the desired functionality of the robotic cart. It should be able to sustain a walking speed of about 1 meter per second while carrying a 40kilogram load. Thus, the cart was constructed from steel tubing with a cargo area of 80 by 60 centimeters. It uses a differential drive using two DC motors at the front, and has two coaster wheels that are mounted on a spring suspension to ensure stability. It uses a microcontroller that reads six ultrasonic
sensors and actuates the drive motors via Pulse Width Modulation (PWM). A summary of specifications is given in Table 1. In addition, a depiction if the robotic cart components is given in Figure 2. The two drive motors are shown on the left side, and the suspension is visible on the right. The center box contains the battery, microcontroller, PWM drivers, and Bluetooth module for communication with the smart phone application. TABLE I. Component Cart Size Wheels
SMART CART SPECIFICATIONS Specifications L x W x H = 80 x 60 x 25 cm 2 front drive, 2 rear coasters on springs
Suspension Srpings
K = 16350 N/m
Maximum Velocity
1.4 m/s
Maximum Load Battery Average Operation Time Microontroller Sensors Actuators
80 kg 12 V, 6.4 Ah, Lead Acid, 2 kg 3 hours Arduino MEGA 2560 3 top and 3 bottom ultrasonic 2 front 12 V DC motors, PWM control
Smartphone App
Android Studio IDE, Google
Communication
Bluetooth
The cart employs six ultrasonic sensors: three at the top and three at the bottom as shown in Figure 3. The bottom sensors consist of a downward pointing sensor to detect the presence of the ground. If the user walks off an edge, like a flight of stairs for example, the robot will not follow. Also at the bottom, there are two sideways pointing sensors to avoid collisions while turning. The three sensors at the top are used to detect the presence of the person and follow them at a constant distance. The middle top sensor is used to keep this distance, while the top left and right sensors are angled toward the center. The cart will turn until both of the angled sensors detect the person. In this case, the cart assumes that the person is in front of the robot, enabling the distance keeping using the center sensor. The connectivity of these components is shown in Figure 4.
Fig 3. Locations of the six ultrasonic sensors
The employed microcontroller is an Arduino Mega 2560 running at 5 Volts and a 16 MHz clock rate, and 8 kilobytes of RAM memory. The ultrasonic sensor used is an HC-SR04 sensor which is compatible for use with an Arduino microcontroller. It provides a 2cm to 400cm non-contact distance measurement range with an accuracy of 3mm. Its operation is not affected by sunlight or black material like other sensors such as laser range finders. The module includes ultrasonic transmitters, receivers and a control circuit. The basic principle of its operation depends on a trigger signal and echo signal with an input voltage (VCC) and a connection to ground. The Trigger port receives a pulse of 5V for at least 10 µs, this will initiate the sensor making it transmit 8 cycles of ultrasonic bursts at 40 kHz and wait for the reflected energy. When the sensors detects a signal on the receiver, it sets the Echo pin to 5V. The distance can then be calculated from the transmit-receive delay time. The Bluetooth module allows the smartphone application to communicate with the Arduino via as simple connection. This design uses CSR’s Bluetooth chip and V2.0 protocol standards. The size of this module is compact and it is fitted on-board with an integrated antenna and it can be used in a master or a slave setting.
Fig 2. Component integration showing motors, PWM drivers, and battery
Left Tracking Sensor
Center Tracking Sensor
Right Tracking Sensor
Center Ground Sensor
Left Obstacle Sensor
Right Obstacle Sensor
Smartphone Application
Bluetooth Module
The smart phone application was coded using Android Studio, the official Integrated Development Environment (IDE) for developing applications on the Android platform. It is freely available under the Apache License. A screenshot of the developed smartphone application is given in Figure 6. It communicated with the microcontroller via Bluetooth at 9600 bits per second. The control functions available to the user are: • a cart speed control slider that changes the PWM duty cycle
Microcontroller
• a “Follow User” mode, which makes the microcontroller honor its sensors and implement the following functionality • a “Stand by” mode, where the cart stands still for loading/unloading
Motor Driver
Motor Driver
Left Motor
Right Motor
• a “Remote Control” function which allows the user to tele-operate the cart using four arrows The last function is useful when the cart must be maneuvered in tight spaces.
Fig 4. Robot system architecture
Fig 5. Electrical wiering diagram
which slipping occurs. The battery runs out in 2 hours when operating at full speed, however if the load is minimal and the speed isn’t set high it can run up to 4 hours, while the motors draw less than 1 Ampere each. The battery can be fully charged from empty in 3 hours.
Fig 7. Motor voltage by PWM duty cycle
Fig 6. Smartphone application in remote control mode Fig 8. Cart velocity at different loard and drive voltages.
III. PERFORMANCE The two drive motors are controlled via a pulse width modulation signal from the microcontroller at 490 Hz. Here the duty cycle of a square wave is varied to control the on-time of the motors, thus the velocity of the cart. The PWM signal produces an average voltage that powers the motors. This voltage has a linear relationship to the PWM duty cycle as shown in Figure 7. To assess the performance, the cart velocity was measured at varying drive voltages i.e. duty cycles. The results are shown in Figure 8. At no load, there is a dead-zone where the cart remains stationary until the drive voltage exceeds 1.5 V. Similarly, the dead-zone threshold increases to 2.8 V at a load of 40 kg. As expected, the no-load maximum velocity is larger (at 1.4 m/s) than the maximum velocity with a 40 kg load (at 1 m/s). The robotic cart was able to carry 80 kg, but at an impractical slow velocity of 0.15 m/s. Thus, the recommended maximum load is around 60 Kg. The cart was tested on inclined tarmac surfaces, and the results show that the operation is acceptable at inclinations up to 16 degrees, after
IV. CONCLUSIONS A robotic cart that follows a person was developed and tested. Constructed of steel tubing and wooden panels, this differential drive cart can carry 40 kg at a maximum velocity of 1 meter per second – walking speed. Suitable applications of this robot are in warehouses, shopping malls and airports where the ground is clean and flat and the operation time is not very long. If equipped with a location and mapping feature, the cart could autonomously find its way back to the charging base once the user no longer requires it. The smartphone communication link also enables the development of a business model where the time of use can be logged and monetary charges can be made. ACKNOWLEDGMENT The presentation of this work was supported by a mobility grant from the Deanship of Scientific Research at the German Jordanian University.
REFERENCES [1]
N. A. Rawashdeh, L. M. Alkurdi, and H. T. Jasim, “Development of a low cost differential drive Intelligent Ground Vehicle,” in 2012 8th International Symposium on Mechatronics and its Applications, 2012, pp. 1–5.
[7]
Y. DiGiacomcantonio, M. and Gebreyes, “Self-propelled luggage,” Pat. Appl. 14/054,758, 2013.
[8]
E. Jung, J. Lee, B. Yi, J. Park, and S. Yuta, “Development of a laser-range-finder-based human tracking and control algorithm for a marathoner service robot,” ASME Trans., 2014.
[9]
M.-Q. Do and C.-H. Lin, “Embedded human-following mobilerobot with an RGB-D camera,” in 2015 14th IAPR International Conference on Machine Vision Applications (MVA), 2015, pp. 555–558.
[2]
N. A. Rawashdeh and H. T. Jasim, “Multi-sensor input path planning for an autonomous ground vehicle,” in 2013 9th International Symposium on Mechatronics and its Applications (ISMA), 2013, pp. 1–6.
[10]
[3]
B. Ilias, R. Nagarajan, M. Murugappan, K. Helmy, A. S. A. Omar, and M. A. A. Rahman, “Hospital nurse following robot: hardware development and sensor integration,” Int. J. Med. Eng. Inform., vol. 6, no. 1, p. 1, 2014.
D. Sanjay, T. S. Savithri, and P. R. Kumar, “Person follower robotic system,” in 2014 International Conference on Control, Instrumentation, Communication and Computational Technologies (ICCICCT), 2014, pp. 1324–1327.
[11]
[4]
M. F. Ahmad, S. S. N. Alhady, S. Kaharuddin, and W. A. F. W. Othman, “Visual based sensor cart follower for wheelchair by using microcontroller,” in 2015 IEEE International Conference on Control System, Computing and Engineering (ICCSCE), 2015, pp. 123–128.
J.-S. Hu, J.-J. Wang, and D. M. Ho, “Design of Sensing System and Anticipative Behavior for Human Following of Mobile Robots,” IEEE Trans. Ind. Electron., vol. 61, no. 4, pp. 1916–1927, Apr. 2014.
[12]
[5]
S. Rastogi, P. Singh, and V. Agarwal, “An Intelligent Sensing Follower Cart (AUTOMATIC SHOPPING TROLLEY),” Journal of Controller and Converters 2.1 (2017): 21-25.
N. Bellotto and H. Huosheng Hu, “Multisensor-Based Human Detection and Tracking for Mobile Service Robots,” IEEE Trans. Syst. Man, Cybern. Part B, vol. 39, no. 1, pp. 167–181, Feb. 2009.
[6]
Y. Leng Ng, C. Siong Lim, K. A. Danapalasingam, M. Loong Peng Tan, and C. Wei Tan, “Automatic Human Guided Shopping Trolley with Smart Shopping System,” J. Teknol., vol. 73, no. 3, Mar. 2015.
