Design and Development of a Mobile Application to Control a Chess Playing Robot using NI-EVS and LabVIEW Authors Michel Owayjan, M.E.; Elie Abdo, M.S.; Jad Ballout; Mark Bou Malhab; and Rabih Hassoun Department of Computer and Communications Engineering Faculty of Engineering American University of Science & Technology (AUST) 16-6452, Beirut - Lebanon Products Used NI Embedded Vision System (EVS) NI LabVIEW™ 2012 Edition (National Instruments, Inc., Austin, TX, USA). Category Research and Academic The Challenge The challenge is to control a robotic arm that moves chess pieces on a chess board against a human opponent. The robot has to be manipulated wirelessly from a distance by either human operators or an Artificial Intelligence (AI) program. The Solution Designing and developing an Android® (Google Inc., CA, USA) mobile application that users can use to control a chess playing robot. The mobile app can either send the user’s chess move or a move calculated by the AI wirelessly through WiFi to the National Instrument Embedded Vision System (NI-EVS) that is connected to a microcontroller that drives the motors of the robotic arm to play the move on the chessboard. A camera connected to the EVS sends the opponent’s move to an embedded LabVIEW™ program that sends the opponent’s move to the mobile app. Abstract Chess is a game that develops mental abilities such as concentration, abstract reasoning, critical thinking, problem solving, strategic planning, and creativity. Playing against a computer screen is not fulfilling, but playing on a real chess board is entertaining. A chess playing robot that can be controlled by an android mobile application was built aiming in establishing new routes in connecting Smartphone users to whatever electronic device they desire to control. This project is based on an Android application, a mobile supporting Android OS, NI-EVS (Embedded Vision System), IP cam, PIC microcontroller and NI LabVIEW™ software. A wireless WiFi connection is made between the mobile that will order the robot to move a specified chess piece, and the NI-EVS that forwards this order to the PIC microcontroller in order for the robot to perform the desired request. NI-LabVIEW™ software programming codes allowed the NI-EVS to interact with the robot and the mobile application. Feedback from the camera connected to the NI-EVS with the image processing NI LabVIEW™ program communicates to the mobile application the opponent’s move.
Introduction Nowadays, mobiles occupy the major parts of individual’s life and concern people of different ages and careers. Meanwhile, electronic devices have prospered and became one of the most necessary aspects for people. Therefore, electronic devices can be modified in a way that allows remote access via Smartphone. By using a microcontroller connected to the electronic device and with an application built on Java for Android, the connection is ensured between the mobile and the device through a wireless connection along with the NI-EVS. A chess playing robot was built to automatically play against another player. Accordingly it was decided to modify the robot and build an android application that will allow a wireless control to be established. Furthermore this idea can be applied to use Smartphone as a remote control for many other devices. The Chess game is considered to be a mind and mental developer game. But playing chess against a screen can be incapacitating. Playing against an opponent on a real, physical chessboard is more interesting. The problem rises if the opponent is not in the same location or cannot move the chess pieces because of physical disabilities. Thus came along the idea of designing and constructing a mobile application that can control a chess playing robot using the NI-EVS and the NI LabVIEW™ graphical programming language. The System Design Figure 1 and Figure 2 detail the overall system design. The user uses the mobile application on his/her Android mobile phone. The mobile app has a two-way communication using WiFi with the NI-EVS as the core part of the project. The NI-EVS sends a command corresponding to the user’s move to the controller of the robotic arm. The PIC microcontroller translates the desired move into motor movements to move the robotic arm and execute the desired chess move on the physical chess board. On the other hand, the camera captures the move of the opponent and a LabVIEW™ program running on the NI-EVS analyzes the captured images in order to determine the opponent’s move which is sent back to the mobile app. Chess Board
Robotic Arm
Controller Camera
Embedded Vision System
Figure 1. System Block Diagram.
Mobile with Android Application
Figure 2. System Design.
Controlling the robot arm will be done by the mobile application connected through EVS. The control is performed by EVS that translate chess moves from the orders given by the application into the PIC 16F877A microcontroller commands. The microcontroller drives the four DC motors of the robotic arm. Using NI LabVIEW™, graphical programming of the software is needed to connect the mobile to the microcontroller can be done and injected into the EVS. The camera used does image processing for the movements that the opponent performed, and sends the information about the movement being done to the application. The Android application The Android application represents a chess game, written on Java for Android and designed based on XML (Extendable Markup Language). It is a fully functional chess game with alert dialogs if a movement cannot be done, as well as an artificial assistance. Moreover, this application has the ability to save a game and can also be modified to save high scores. The basic function of this app is to send data encapsulated in TCP/IP packets to the LabVIEW™ program on the NI-EVS, which in turn releases the packets via a block of code that is a Gcode (Graphical Code), and sends an order to the control of the robot to execute the same move. Figure 3 shows the Android application interface.
Figure 3. Android Application.
The NI-EVS The NI-EVS receives wirelessly the command from the android application and runs a LabVIEW™ program in order to de-encapsulate the TCP/IP packets and sends the data to the PIC-microcontroller via Serial connection to activate the motors. The NI-EVS has an IP camera connected to it, sending snapshots at specific intervals of time so that the NI-EVS acquires the opponent’s moves. After analyzing the images obtained and differentiating among several snapshots, LabVIEW™ forwards the information in TCP/IP packets to the smartphone handling the application so it can update its interface. The Controller A PIC 16F877A microcontroller has 40 pins and only 35 RISC instructions. It is easy to program. The microcontroller has the ability to store non-volatile data internal EEPROM and has a sufficient performance for ordinary controller applications in addition to internal and external interrupt support such as timers and port change. A microcontroller is a single integrated circuit (IC) used to control electronic devices. It can be considered as a computer on a chip because it has the same main features of a computer that are central processing unit, input/output interfaces (ports) and memories. It also includes timers, analog to digital signal converters and a clock generator that can be an oscillator, RC circuit or a resonator. It is designed to perform a single specific application and therefore its cost is low. Microcontrollers are usually part of an embedded system and are then called controllers. The job of the PIC microcontroller is to convert the move that it receives from the NI-EVS into voltages to control the four motors of the robotic arm.
The Robotic Arm and the Chess Board A robotic arm is a type of mechanical arm programmable with similar functions to a human arm. The robotic hand used in this project (Figure 4) is based on PIC 16F877A microcontroller commands, has six degrees of freedom, a vertical axis to allow rotations in the horizontal plane, and three axes for the robotic arm itself. In order to perform all the chess moves, the robot arm has four 12 V DC motors, each consuming around 0.4 mA except for the gripper‘s motor that consume about 0.17 mA. The maximum lifting capacity of the gripper is of 0.5 kg. The distance of the horizontal sliding base is of 70 cm. A chess board is a 64 squared board of eight rows and eight columns and has two different colors that are referred to as light squares and dark squares. Each color has 16 pieces divided into six classes: one king, one queen, two rooks, two Knights, two bishops, and eight pawns. Each of the classes has its own style of moving.
Figure 4. The Robotic Arm with the Chess Board.
The NI LabVIEW™ Program Using LabVIEW, and injecting the program into the NI-EVS, allowed the connection between the robot and the mobile application. The program initiates the WiFi connection. It consists of several blocks and each has its own functionality. TCP listen block ensures a TCP/IP connection for EVS specifying the IP address and the port at which the connection would be established. TCP write block acquires the data received from vision system that is able to acknowledge the movements done on the chess board and forwards the information to the application in order to update the chess board interface. TCP read block gathers data from the mobile application and sends this data to the VISA write block in order to move the robot. VISA configure serial port block configures the port connected to it with specifying the baud rate and the number of bits to be transferred. VISA close and TCP close connection blocks terminates the connections established in case of errors. The whole program is enclosed in a while loop that ensures continuous flow of data with a delay of three seconds. This is shown in Figure 5. The other blocks are used for image processing to recognize the opponent’s moves.
Figure 5. Part of the VI Block Diagram.
Results and Conclusions The system was implemented and tested on the robotic arm. The reason why NI products were chosen is mainly the time savings and the simplicity they bring in. The NI-EVS operates real time LabVIEW OS which makes it very fast and reliable. The NI LabVIEW graphical programming language makes faster and more intuitive programming. In addition, the builtin blocks for the TCP/IP connection and the IMAQ vision system made the work on the project much easier without the hassle of other programming languages. The programming time was radically reduced and the authors needed the spared time to focus on the hardware implementation. The NI engineering support team in Beirut was also very helpful. The main goal behind this project was to maintain the ability to control a robot via a Smartphone. According to the achievements, it was clear that a mobile application with the help of some external hardware and a special program such as LabVIEW is a way to remotely access any electronic device and control it. Furthermore, the project is innovative as it brings new concept to the chess world, making people of different ages interested in enrolling into such a game. Contact Information Michel Owayjan, M.E. Department of Mechatronics Engineering (MTE), Faculty of Engineering American University of Science & Technology (AUST) Ashrafieh, Alfred Naccache Street, P.O.Box: 16-6452 Beirut – Lebanon Telephone: +961-1-218716/7 extension 246 Mobile: +961-3-523418 E-mail:
[email protected]
