Chapter 109
A Handheld Controller with Embedded Real-Time Video Transmission Based on TCP/IP Protocol Mingjie Dong, Wusheng Chou, and Yihan Liu
Abstract Cross-platform video transmission is of vital importance in industrial applications. In this paper, we introduce a method for transmitting video from the computer with Windows system to the ARM11 board with embedded Linux system using the Ethernet based on the TCP/IP protocol. The ARM11 board is used as the server to receive video information using its Qt GUI, while the computer on the bank is used as the client that receives video information from the remote-operated underwater vehicle showing with its MFC (Microsoft Foundation Classes) interface and then sends the video information to the handheld controller. The image gained from the computer MFC is JPG format, and after coding, the images are transmitted to the server on the handheld controller continuously. Then the Qt GUI receives the data and decodes the JPG images before displaying them on the screen. The transmission is based on TCP/IP protocol and an image parsing protocol made by us. After testing, the video image can successfully conduct real-time transmission and can meet the industry application.
109.1
Background Information
The project comes from the national 863 project—miniature underwater submarines. Because of the complex underwater environment of the nuclear power plant, the ROV (remote operated vehicle) must be controlled on the bank, and the information of the reactor pool gained by the camera attached to the ROV will be transmitted to the control box made of an IPC (Industrial Personal Computer) with Windows operation system on the bank. Considering the big size of the control box, we need to develop a portable handheld controller which can replace the control box to a certain extent. The video image information will be transmitted to the M. Dong (*) • W. Chou • Y. Liu Intelligent Technology and Robotics Research Center, School of Mechanical Engineering and Automation, Beijing University of Aeronautics and Astronautics, Beijing 100191, China e-mail:
[email protected] W.E. Wong and T. Zhu (eds.), Computer Engineering and Networking, Lecture Notes in Electrical Engineering 277, DOI 10.1007/978-3-319-01766-2_109, © Springer International Publishing Switzerland 2014
955
956
M. Dong et al.
system box from the camera on the ROV, and then the control box will transmit the video image information to the handheld controller directly. The latter is just what we will talk on this paper. The video transmission is based on the TCP/IP protocol [1]. The control box will be the client, while the handheld controller will be the server during the video transmission. The transmission is through Ethernet using network cable, so the transmission is very fast and can be real time after a certain optimization.
109.2
Introduction of the Hardware Platform
The handheld controller is made of an ARM11 development board with an S3C6410 platform. The S3C6410, whose CPU can be up to 667 MHz, is a 32-bit RISC microprocessor which is designed by Samsung to provide a cost-effective, low-power capabilities and high-performance processor solution for mobile devices [2]. During the research, a Linux operation system with kernel number 2.6.38 is transplanted to it. After cross-compiling using arm-Linux-gcc cross-compiler with version number 4.5.1, the program written on the host machine can run successfully on the development board [3]. Given its excellent property, the development board is right for video transmission (Fig. 109.1).
109.3
Software Application Platform Building
The software application platform is composed of two different parts, the software on the Windows system and the software on the embedded Linux system.
(WKHUQHW LQWHUIDFH 3RZHU PDQDJHPHQW EORFN
'0
1DQG)ODVK
Fig. 109.1 The hardware platform of the handheld controller
''55$0 6& SURFHVVRU
86% LQWHUIDFH
&U\VWDOV
109 A Handheld Controller with Embedded Real-Time Video Transmission. . . Fig. 109.2 Qt/E mechanism structure diagram
957
4W($SSOLFDWLRQ 4W(PEHGGHG )UDPHEXIIHU
'HYLFHGULYHU
(PEHGGHG/LQX[
109.3.1
Software on the Windows System
The software on the Windows system refers to the platform using MFC built on the control box with the Windows system. This part shows the video image transmitted from the camera attached on the ROV. The MFC program uses the SDK functions of the image capture card to display the images. Meanwhile, it is used as the client of the network video transmission through socket based on TCP/IP protocol.
109.3.2
Software on the Embedded Linux System
In the ARM11 development board, a Linux system is transplanted with the kernel version number 2.6.38, and then we successfully setup the cross-compiler arm-linux-gcc with the version number 4.5.1 on it. With all above settled, we set up Qt/E (QT embedded) on the development board in order to use its convenient GUI to decode the JPG image and display the video information on the screen. Qt/E is an open-source software development kit provided by the software developer Trolltech, with C++ program language as its development tools. Qt/Embedded has a lot of simple class libraries and interface modification tools. Especially, it has unique signals and slots mechanism [4]. It is across platforms and can be easily transplanted. The simple Qt/E mechanism is showed in Fig. 109.2.
109.4
Application of TCP/IP Protocol
Based on the TCP/IP protocol, we use socket as the application programming interface. The socket programming is based on the system call of the socket. At first, we use the function socket () to create a new socket, and then we connect the socket address with the socket we just created using bind (). In order to build the socket connection between the client and the server, we should use connect () and accept () functions. The function connect () is used to build connection, while the function accept () is used by the server, waiting for the connection request from the client. Meanwhile, the server uses the function listen () to monitor whether there is a
958
M. Dong et al.
VRFNHW
ELQG
VHUYHU
OLVWHQ
FUHDWH
DFFHSW
%XLOG FRQQHFWLRQ
VRFNHW
FRQQHFW
VHQG FRPPXQLFDWLRQ VHQG UHFHLYH UHFHLYH
FOLHQW
Fig. 109.3 The socket’s working procedure
client request. Once a connection between the client side and the sever side is built successfully, we can exchange data from the client and the server [5]. The socket’s working procedure is in Fig. 109.3 [6].
109.4.1
Socket on the Server Side
The server is built on the development board with embedded Linux operation system using the Qt/E GUI. We use QTcpServer, a Qt class, to build a sever class. Then, we use the function listen () to monitor the port we set at the beginning [7]. Because of the high encapsulation of Qt, we can save a lot of trouble. Once the server side gets the connection request from the client, the server will accept the request and the Qt function readyread() will be called, and then in our program, the function dataReceived() written by us will run according to our settled signals and slots mechanism. The data from the client will be gained successfully.
109.4.2
Socket on the Client Side
On the client side, as we use the MFC in the visual C++6.0 environment, we should use the Windows socket API to build the socket and build the connection with the server in the Linux operation system. At first, we use socket (AF_INET, SOCK_STREAM, 0) [8] to create a new socket based on the TCP/IP protocol. Then we give the address variables a certain value to make sure the socket address is connected with the socket we created. Lastly, the client will send a connection request to the server using the function connect (). When the server accepts the request, the connection is finished.
109 A Handheld Controller with Embedded Real-Time Video Transmission. . .
109.5
959
Protocol Development of the Video Transmission
For the data to be transmitted in the form of binary, so after capturing the images information from the camera attached to the ROV, we will change the image format from JPG to binary streams. In order to get the images streams coded in the client side and decoded in the server side securely and to accept the whole image streams in the development board smoothly, we should set up our own video transmission protocol.
109.5.1
Image Acquisition and Coding on the Client Side
Every time we capture an image, the image will be saved in the form of binary with a temporary name given by us. Then we make a protocol for the whole image transmission, using the format”AA:file name*length:EE” in order to identify that we get a whole image all the times. The file name is the temporary name we give after an image is captured, while the length is its length of binary form. Using a while structure loop, the images will be sent to the server continuously.
109.5.2
Image Receiving and Decoding on the Server Side
In the server written by us using Qt GUI, a lot of data will be received. At first, we estimate whether a set of data the server received is a whole image through the image transmission protocol we made. If the data meet the format”AA:file name*length: EE”, the video transmission protocol made by us, we can make sure that the data the server received is a whole image transmitted from the client, and then the image will be displayed on the screen. Just as before, using a while structure loop, we decode and display the images continuously. Because of the fast flow of the continuous images using network cable and the excellent property of the development board, we can watch the video information smoothly (Fig. 109.4).
109.6
Experimental Verification
After building the environment on both the Windows system and the embedded Linux system, we programmed in the client side and the server side, respectively, based on the video transmission protocol made by us and the TCP/IP protocol. Then we test the program, and the result is very satisfying. The video information captured from the client side can be transmitted to the server side on the ARM11 development board smoothly.
960 Fig. 109.4 The video transmission procedure
M. Dong et al.
:LQGRZV 0)&
7&3,3 SURWRFRO
/LQX[ 4W(
FDSWXUH LPDJHV
VRFNHW FRPPXQLFDWLRQ
UHFHLYH LPDJHV
LPDJH FRGLQJ
YLGHR WUDQVPLVVLRQ SURWRFRO
LPDJH GHFRGLQJ
VHQG LPDJHV
GLVSOD\ LPDJHV
Fig. 109.5 The video information running in the client side on the control box
In order to test whether the video transmission is stable and to gain the transmission speed, we set up the function frame_test() in the Qt GUI. By calling the Qt class libraries with the function, we find that the transmission is very stable and the speed can be up to 24 frames per second. It is real time and can meet the industry application perfectly. The effect picture of the video transmission is shown in Figs. 109.5 and 109.6.
109 A Handheld Controller with Embedded Real-Time Video Transmission. . .
961
Fig. 109.6 The video information running in the server side on the handheld controller
109.7
Conclusion
Summarily, this paper describes an effective method to transmit video image information across the different operation system based on the TCP/IP protocol. The images transmission protocol we made is simple and useful, while the cost of the whole handheld controller is very low and the video transmission is real time; it can easily meet the needs of the project.
References 1. Wilder, F. (1993). A guide to the TCP/IP protocol suite (pp. 3–21). Boston, MA: Artech House. 2. Friendly ARM. (2011). Tiny6410 hardware manual. Retrieved August 5, 2011, from http:// www.arm9.net 3. Liu, Z. (2012). Application of embedded database in room environment supervisory system. Modern Electronics Technique, 35(10), 2–3. 4. Bangwei, Y., & Deng, H. (2011). Control interface of embedded digital monitoring system based on Qt/embedded. Part and Application, 35(24), 1–2. 5. Anand Kumar, M., & Karthikeyan, S. (2011). Security model for TCP/IP protocol suite. Journal of Advance in Information Technology, 2(2), 1–5. 6. Zhang, Y.-G., Liu, C.-C., Liu, W., & He, F.-Z. (2006). Remote monitoring and control system based on socket and multithread. Control Engineering of China, 13(2), 2–3.
962
M. Dong et al.
7. Blanchette, J., & Summerfield, M. (2008). C++ GUI programming with Qt4 (2nd ed., pp. 295–303). Beijing, China: Publishing House of Electronics Industry. 8. Tian, L. (2012). Network communication technology based on Winsock. Hubei: Software Guide, 11(1), 1–4.
