Robot Navigation using Camera by Identifying Arrow Signs Jongan Park Dept. of Information Communication, Chosun University, Kwangju, South Korea
[email protected]
Abstract Robot navigation is an active area of research. In this paper, we emphasize on the image processing algorithms for the navigation of robots. A track with left and right arrow signs is provided for navigating the robot from one end of the track to the other end. The navigation is done by identifying the arrows signs and making the decisions to either turn left or right based on the direction of the arrow signs. We selected four different features to isolate the arrow region in the image from rest of the regions. Further, we exploited the shape of the arrow to decide on the direction of the arrow. We found that the robot accurately navigates through the track with left and right arrow signs and it reaches the destination successfully.
can navigate their way autonomously. Such robots are required in various applications including automatic vehicle driving (both terrestrial and in space), helping the elderly in their homes, guiding the traffic on the roads etc. Robotics has also entered at the micro and nanoscale. We can define the field of micro-robotics to include the robotic manipulation of objects with characteristic dimensions in the millimeter to micrometer range (micromanipulation) as well as the design and fabrication of robotic agents in a similar size range (micro-robots). Micro-robots tasks require novel tools and sensors that operate at the micro-scale. It has many applications which include, but are not limited to, assembly of micro-sized parts, manipulation of biological cells, micro-autonomous vehicle etc.
2. Related Work Index Terms—Robot navigation, wireless camera, sign board identification
1. Introduction Humans perform various physical as well as mental tasks with great accuracy. Furthermore, humans do not require any measurements to perform those tasks. For example, parking of a car, playing tennis, giving lecture in specified time, etc. The computational methods on the other hand are based on mathematical modeling and analysis. Hence, we try to model different scenarios and incorporate it using artificial intelligence. Therefore, to date, robots have been very successful at manipulation in simulation and controlled environments such as a factory. Outside of controlled environments, robots have only performed sophisticated manipulation tasks when operated by a human [1]. Similarly, navigation is quite easy for a human but on the other hand, all the scenarios need to be detailed for a robot. The world is moving towards robots that
In this section, we present some of the work done in the field of robot navigation. Seraji and Howard [2] present strategy for behavior-based navigation of field mobile robots on terrain, using a fuzzy logic approach. Three terrain characteristics are selected. These three characteristics affect its traversability and they are roughness, slope, and discontinuity. These three are extracted from video images obtained by on-board cameras. This traversability data is utilized the terrain Fuzzy Rule-Based Traversability Index. This index succinctly quantifies the ease of traversal of the regional terrain by the mobile robot. Behavior based navigation [3] has been suggested based on the quality of the terrain. They classify terrain in four different categories namely flat, uneven, obstacle and unknown. Different terrain characteristics are exploited for labeling of the segmented cells of the terrain. The segmented cells of the terrain are based on three–dimensional image data. Different terrain characteristics exploited include point density, altitude, and mean vector etc. An estimate of the probability for each label is determined from prior learning based on
Bayesian classification methodology. The classification itself depends on prototypes classified by humans. For untraversable terrain region, Langer et al. [4] focus on the development of a navigation system that generates recommendations for vehicle steering. Cells are classified as objects by determining the traversability. Traversability itself is determined by combining elevation and slope information. A threshold is selected such that the cells having slope values above the given threshold are classified as untraversable. Cells without enough range information are not classified, allowing the rover to approach these areas and retrieve additional information if necessary. A steering command is generated by combining votes from multiple behaviors and deciding on the best command based on the location of obstacle cells. One very important application is discussed by Dellon and Matsuoka [5]. The requirement for assistive robotic devices will increase significantly in near future. This fact can be observed from the statistics that the world population over 65 was 4.9% in 1950s while its over 20% today and is expected to be over 35% bu 2050. Toon et. al. [6] present a system for autonomous mobile robot navigation. They use an omnidirectional camera as a sensor. This system builds topologically organized environment maps of a complex, natural environment. They use fast wide baseline feature matching, which yields an efficient description of the scene, with a focus on man-made environments.
Image Processing Module performs the following actions on the image: Converts to binary image Removes small objects and noise Do region segmentation Labels the segmented image Gives different color to each segment for better visualization and also numbers them • Identifies the arrow sign board from the different segmented regions in the image and separates it from the rest of the image • Finds out whether the arrow is a left direction arrow or a right direction arrow Once the Image Processing module identifies whether the sign board indicates left or right turn, it passes this information back to the Control Module. The Control Module then sends a signal to the robot via the serial port and instructs the robot to either turn left or turn right. The robot on receiving the signal from the serial port of the computer, initiates its algorithm to either turn left or turn right. It uses the IR sensors to avoid collision with the blocks on the road. After completing the turn, the vehicle resets the control signal received from the computer and starts moving in straight direction. And again keeps on checking repeatedly if any control instruction is received from the computer. This way the robot is navigated to reach its destination using traffic sign board. This whole operation is shown in the block diagram in figure 1. • • • • •
3. Details of navigation Computer A track is made with various turns for the robot to move around. On every turn, a right or left arrow is provided for robot to recognize the arrow symbol by capturing images using a wireless camera. There is an initial starting point and the final destination point. The robot is supposed to navigate through this track to reach the final destination point with the help of arrow signals. The robot starts moving in straight direction. While moving, images are captured frame by frame in real time by a digital wireless camera and the captured frames are sent back to the computer through wireless connection. Single frame is captured at a time. That single frame is a colored image of 320 x 240 resolution and it is sent to the computer. The Control Module is running on the computer and is waiting to receive the image from the wireless camera via USB port. Once it gets the image, it passes that image to the image processing module.
Image Acquisitio
IP Module
Serial Port
Buffer for control signal
IR SENSOR
Wireless Camera
Processor
Robot Turning Module
Figure 1. Block diagram of basic architecture
4. Pre-Processing The image acquisition module receives the images from the wireless camera through the USB port. The images are passed on to the image processing module where they are first pre-processed. The image is preprocessed to remove any kind of noise present in the image as well as to prepare it for further processing. The following actions are taken in the pre-processing stage:
Convert to binary image: Image is converted to binary image. First it is converted to grayscale and then global thresholding is used to convert it to binary image. Otsu's method for thresholding is used, which chooses the threshold to minimize the intraclass variance of the black and white pixels. Final result is the normalized intensity values that lie in the range [0, 1].
Remove small objects and noise: First, all the connected components are determined and the area of each of the connected component in the image is calculated by using 4-connectivity. Once we know the area of each of the connected component, it is easier to remove isolated noise pixels and small objects. Therefore, finally we remove from the binary image all the connected components (objects, isolated noise pixels) that have fewer than P pixels, producing another binary image. For our case, P is selected as 150 pixels. 4-Connectivity is used for processing so as to reduce the processing time.
5. Region Segmentation After pre-processing, the image is segmented using the regional segmentation techniques. One such technique is “Split & Merge”. In this technique, the image is kept on splitting into 4 halves and each splitted region is further continued on splitting into 4 halves till each splitted region has uniform or same pixel values. Then once the splitting is complete, the next step is merging. And those splitted regions are merged which have same pixel values and at the same time they satisfy the condition for connected region. Hence, the final result is an image with segmented regions. Once the image is segmented into regions, each of the regions is labeled as region 1, region 2 etc. There is a possibility that the regions might have holes in it. In that case, we get parents as well as holes. However, we already know that the arrow sign does not have a hole in it. So keeping in mind this characteristic of the arrow
sign, we ignore all the regions that have holes. In this way, we reduce the number of regions and hence also reduce processing time for later stages of the algorithm. After labeling the image, the images are displayed with each region having unique color and a unique number.
6. Identification Stage Our purpose is to accurately extract the arrow from the image and then find out the direction of the arrow, i.e., left arrow or right arrow. Till this stage, we have identified various regions in the image and labeled them. So now we select certain features that will help us in the identification of the arrow region from rest of the regions and then the direction of the arrow itself. The selection of the features is an important step because the whole identification stage depends this selection. We select four features that will help us in distinguishing the region containing arrow. The following four features are calculated for each of the labeled regions: Convex Area: It is number of pixels in Convex Image and hence is distance dependent. Eccentricity: It is the ratio of distance between foci of ellipse and its major axis and hence it does not change with distance as it’s a ratio. Extent: It is the area of the region divided by the area of bounding box surrounding the region. Hence, it’s a ratio and it does not change with distance too. Solidity: Solidity is the area of the region divided by the Convex Area. Again it’s a ratio and is independent of the distance. The four features are carefully selected so that the information provided by those features can help is isolating the arrow region. The first feature, Convex Area, is distance dependent and its value changes as the robot moves closer to the arrow sign. The other three features are ratios and hence distance independent and they provide good estimate for identification of various regions. Hence, we calculate these four features for every labeled region to distinguish them from each other. For the arrow sign, we have already computed various values for the above mentioned four features and stored them in our knowledge database. After extracting the above mentioned four features for all the regions, these features are matched with the pre-stored features of the arrow sign. We simply use the Euclidean (L1) distance measure for comparison. And hence the
region for the arrow sign is identified accurately and separated from the rest of the image. Once we isolate the arrow region then we can easily extract it from the image. So, now we are left with only the segmented region of arrow sign. The final task is to identify whether it’s a right arrow or a left arrow. To identify whether it’s a left turn or right turn, the shape of the arrow is exploited. For making a decision to turn right or left, we exploit the shape of the arrow. The arrow is very narrow on the front side with the width of the arrow increasing steadily in a linear fashion. In addition, it has a almost vertical line at the back end too. Hence, this information is utilized and number of pixels is counted in the first few columns of the segmented arrow sign board. If the number of pixels is small, i.e. less than four pixels in initial few columns then it means that arrow is on the left side since we initially process the columns at the beginning of the image. So the sign board is identified as the Left arrow sign. Otherwise it’s a Right arrow sign. Fig 2 shows the actual image acquisition, region segmentation, labeling and final arrow extraction.
(c) Image labeling
(d) Arrow Extraction
Figure 2. Image acquisition, segmentation, labeling and extraction
Once the decision is made about the right or left turning based on the direction of arrow sign, this information is passed on to the Control Module of the robot. The information is sent through the serial port. In future, we plan to replace the serial port with wireless USB port. The control module on receiving the instruction initiates the left or right turning sequence. (a) Image frame acquired using wireless camera
7. Discussion and Conclusion In this paper we have presented an image processing algorithm for navigation of robot. The navigation is done by identifying the left or right arrow sign board. The images are captured through the wireless camera. We found that the robot accurately navigates through the track with left and right arrow signs and it reaches the destination successfully. In future, we plan to use the wireless USB port replacing the serial port.
(b) Image segmentation
8. Acknowledgement This part of the paper will be populated if and when the paper is uploaded as camera ready.
10. References [1] Charles C. Kemp, Aaron Edsinger, E. Torres-Jara, “Challenges for robot manipulation in human environment”, IEEE Robotics and Automation Magazine, pp. 20-29, March, 2007. [2] H. Seraji and A. Howard, “Behavior-based robot navigation on challenging terrain: A fuzzylogic approach”, IEEE Transactions on Robotics and Automation, vol. 18, No. 3, pp. 308-321, 2002. [3] S. Lacroix, R. Chatila, S. Fleury, M. Herrb, and T. Simeon, “Autonomous navigation in outdoor environment: Adaptive approach and experiment”, in Proc. IEEE International Conference on Robotics and Automation, San Diego, CA, pp. 426–432, 1994. [4] D. B. Gennery, “Traversability analysis and path planning for a planetary rover”, Journal of Autonomous Robots, vol. 6, pp. 131–146, 1999. [5] B. Dellon and Y. Matsuoka, “Prosthetics, Exoskeletons, and Rehabilitation”, IEEE Robotics and Automation Magazine, pp. 30-34, March, 2007. [6] Toon Goedemé, Marnix Nuttin, Tinne Tuytelaars and Luc Van Gool, “Omnidirectional Vision Based Topological Navigation”, International Journal of Computer Vision, vol. 74, No. 3, pp. 219-236, September 2007.
