Conference REV2006
29/30 June 2006 Maribor, Slovenia
Virtual Instruments for Remote Experiments Visualization Bojan Gergič, Darko Hercog University of Maribor, Faculty of Electrical Engineering and Computer Science
Key words: remote experiments, visualization, virtual instruments, LabVIEW Abstract: Virtual instruments for remote experiments visualization implemented with LabVIEW graphical software tool are presented. First solution uses low cost web cameras with universal serial bus (USB) interface and NI-IMAQ for USB Cameras driver. Images from any USB imaging device with DirectShow support can be acquired with these free software driver. Images acquired within server application are lossy compressed with Joint Photographic Experts Group (JPEG) standard and sent over the Internet by using the National Instruments (NI) DataSocket technology. Client application displays received and decompressed image to the remote experiment user. Second solution uses digital cameras complying with IIDC specification and NI-IMAQ for IEEE 1394 Cameras driver. Compressed images are sent over the Internet by using LabVIEW shared variable.
1 Introduction Rapid progress in the development of information technology has an influence on teaching and learning. Conventional classes are more and more often supplemented by distance learning remote laboratories to enhance classroom lectures with remote experiments which are available to the students all the time [1], [2]. Remote experiments are frequently implemented with virtual instrumentation that could be monitored and controlled over the internet [3]. The concept of virtual instrumentation is to combine different hardware and software components to create a customized system for testing, measurement, and industrial automation. Software is the most important component of a virtual instrumentation and LabVIEW has become the most popular development tool because of its easy-to-use graphical programming environment. It has the powerful display, analysis and communication capabilities required for today's rapid development of virtual instrumentation [4]. The teacher can quickly publish the front panel of a LabVIEW application for use in a Web browser by using Remote Panels feature of LabVIEW. Once published, the students can monitor and control the experiment which executes locally on the remote laboratory server. The student simply points the Web browser to the Web page associated with the application. Then, the user interface for the application shows up in the Web browser and is fully accessible to the student. If another student is controlling the remote laboratory, the other students are able to monitor the actions of the controlling student. Once time-out occurs or the controlling student has released control, the remote laboratory is available for the next student. A very important part of the live experiments is visual feedback [5]. Visual feedback can be added to a remote laboratory in many different ways. A simple solution would be to integrate the live images into the front panel of remote experiment and than using Remote Panels. Unfortunately this solution will slow down the program execution significantly because the 1(6)
Conference REV2006
29/30 June 2006 Maribor, Slovenia
images are sent uncompressed over the internet. A better solution would be to embedding the live webcam images in the Web page created by LabVIEW Web publishing tool. This solution requires separate software package for implementing remote experiment computer as the image host and creating the code necessary to add the live images to the existing LabVIEW created HTML code [6]. Another solution uses a separate application for visualization of remote experiments. One such application is Microsoft Windows Media Encoder 9 Series which could be downloaded free of charge from Microsoft's home page [7]. Microsoft Windows Media Encoder 9 Series is a powerful tool for converting live video into compressed Windows Media streams that are then viewed by remote users with Windows Media Player. The weakness of this solution is long delay between image acquisition and image display. The two solutions proposed in this paper send compressed images over Internet with short delay. They are implemented as separate LabVIEW applications and require a free LabVIEW run-time engine installed on the client computer. LabVIEW run-time engine is also necessary in order to operate remote experiments using Remote Panels. Software toolkits and drivers necessary for development of proposed applications are included in National Instruments Academic Site License that is used by many academic institutions worldwide [8].
2 Virtual instruments for remote experiments visualization with USB web cameras This solution uses low cost web cameras with universal serial bus (USB) interface and NIIMAQ for USB Cameras driver [9]. NI-IMAQ for USB Cameras driver provides the ability to use any USB camera that have Microsoft DirectShow support with LabVIEW and IMAQ Vision development tools. NI-IMAQ for USB Cameras driver can acquire and set acquisition properties using the camera manufacturer driver and Direct Show functions. IMAQ Vision for LabVIEW is required to use the NI-IMAQ for USB Cameras virtual instruments (VI) library. Figure 1 shows continuous image acquisition using IMAQ USB Grab. A grab is a continuous, high-speed acquisition of data to a single buffer in host memory. This function performs an acquisition that loops continually on one buffer. A copy of the acquisition buffer is obtained by grabbing a copy to a LabVIEW image buffer. NI-IMAQ for USB Cameras does not support simultaneous acquisitions from more than one camera.
Figure 1. Acquiring images using IMAQ USB Grab. Images acquired within server application are lossy compressed with Joint Photographic Experts Group (JPEG) standard. Compressed images are sent over the Internet by using the National Instruments DataSocket technology to the client application which displays received and decompressed image to the remote experiment user. Figure 2 shows encoder algorithm which is implemented within server application. The main idea of this algorithm is to save acquired image to JPEG compressed file, then read JPEG file back as binary byte stream and send it to the DataSocket Server by using the DataSocket Transport Protocol (dstp). 2(6)
Conference REV2006
29/30 June 2006 Maribor, Slovenia
Figure 2. Encoder algorithm. DataSocket is an Internet programming technology that simplifies data exchange between networked computers and applications [10]. It provides one common Application Programming Interface (API) to a number of different communication protocols, including DataSocket Transport Protocol (dstp) and NI Publish-Subscribe Protocol (NI-PSP). DataSocket efficiently pass data over the Internet and responds to multiple users without the complexity of low-level Transmission Control Protocol (TCP) programming. When dstp protocol is used the VI communicates with the DataSocket Server. A named tag for the data, which is appended to the Uniform Resource Locator (URL), must be provided. The data connection uses the named tag to address a particular data item on a DataSocket Server. Client application displays received and decompressed images to the remote experiment user. Figure 3 shows decoder algorithm which is implemented within client application. When the DataSocket Read function receives the compressed image from the DataSocket Server it is first saved to binary file. This file is actually JPEG compressed image, which is then decompressed with Read JPEG File function and displayed to the remote experiment user in Picture indicator. Picture indicator is used instead of Image Display indicator to avoid usage of Vision Development Module run-time engine in client application.
Figure 3. Decoder algorithm.
3 Virtual instruments for remote experiments visualization with IEEE 1394 cameras This solution uses digital cameras complying with the 1394 Trade Association’s Industrial and Instrumentation specification for Digital Cameras (IIDC) [11] and NI-IMAQ for IEEE 1394 Cameras driver [12]. The NI-IMAQ for IEEE 1394 Cameras driver is a group of virtual instruments that enable use of LabVIEW with an IEEE 1394 camera. NI-IMAQ for IEEE 1394 Cameras complies with the 1394 Trade Association’s Industrial and Instrumentation specification for Digital Cameras (IIDC) and controls all available modes of the digital camera. Setting of camera attributes, such as gain is directly from the NI-IMAQ for IEEE 1394 Cameras software or through the Camera Attributes tab in Measurements and Automation Explorer (MAX). NI-IMAQ for IEEE 1394 Cameras driver supports simultaneous acquisitions from more than one camera and Unibrain's Fire-i™ Color Digital Cameras complying with IIDC were chosen for remote experiments visualization because of good picture quality and reasonable price [13]. Server application (figure 4) sends the table of 3(6)
Conference REV2006
29/30 June 2006 Maribor, Slovenia
active cameras with corresponding URLs and experiment titles to the client application where the student can chose the camera from the list of friendly experiment titles (figure 5).
Figure 4. Front panel of server application for remote experiments visualization with IEEE 1394 cameras.
Figure 5. Front panel of client application. Camera list and JPEG compressed image data are sent over the Internet by using LabVIEW shared variable. Shared variables are configured software items that can send data between VIs [10]. Shared variables can share data on the local computer or across a network. Networkpublished shared variables communicate between VIs, remote computers, and hardware through the Shared Variable Engine. The Shared Variable Engine uses the NI PublishSubscribe Protocol (NI-PSP) data transfer protocol to write and allow users to read live data. NI-PSP is a proprietary technology that provides fast and reliable data transmission and is installed as a service on the computer when the LabVIEW is installed. LabVIEW identifies network-published shared variables through a network path that includes the computer name, project library name, and shared variable name. The block diagram of encoder and decoder algorithm is practically the same as for the remote experiments visualization with USB web cameras shown on figures 2 and 3. The difference is that the encoder now uses Shared Variable node instead of DataSocket Write function and that URL of decoder's DataSocket Read function addresses NI Publish-Subscribe Protocol (NI-PSP) instead of DataSocket Transport Protocol (dstp). 4(6)
Conference REV2006
29/30 June 2006 Maribor, Slovenia
4 Conclusion Virtual instruments for remote experiments visualization based on client-server architecture, implemented with LabVIEW graphical software tools included in National Instruments Academic Site License, were presented. Server application sends JPEG compressed images to the client application which decompress and displays received images to the remote experiment user. Two different solutions were presented. First uses low cost web cameras with USB interface and NI-IMAQ for USB Cameras driver. This driver does not support simultaneous acquisitions from more than one camera and settings of camera attributes are note directly but are done by using the camera manufacturer driver. Second solution uses digital cameras complying with IIDC specification and NI-IMAQ for IEEE 1394 Cameras driver which enables simultaneous acquisitions from more than one camera and directly settings of camera attributes. The client application uses separate window which is always on top of other windows and can be freely moved around the desktop or minimized. It can not be integrated in Remote Panels based experiment, but it can be integrated in remote experiment which is based on the same client-server architecture. The client application requires free LabVIEW run-time engine which is also required for the LabVIEW based remote experiments. It is important to minimize the processing power for remote experiments visualization, particularly when the same server computer is also used for remote laboratory experiments. The processing power requirements increase rapidly with image size and frame rate. For most remote laboratory experiments it is enough to have image size with 320 × 240 pixels or less and frame rate with less than 5 fps (frame per second). In this case the server application for remote experiments visualization uses less than 15 % of processor power on today's average personal computer. The proposed codec is based on JPEG compression and therefore can not achieve such compression rates and quality as Microsoft Windows Media Video (WMV) 9 codec which uses efficient moving estimation, but it has very short delay when compared to the combination of Microsoft Windows Media Encoder 9 Series with Windows Media Player.
References: [1] Gergič, B.; Hercog, D.; Mikola, L.; Matko, V.: Using the Internet and Virtual Instrumentation to enhance the learning of Electrical Measurements, Proceedings of the REV2004, Villach (Austria), September 2004 [2] Hercog, D.; Gergič, B.; Matko, V.: Remote lab for electric drives, Proceedings of the ISIE 2005, Dubrovnik (Croatia), pp. 1685-1690, June 2005 [3] Ertugrul, N.: Towards Virtual Laboratories: A Survey of LabView Based Teaching Learning Tools and Future Trends, Int. J. Eng. Educ., 16(3), 2000, pp. 171-180 [4] Bishop, R. H.: Learning with LabVIEW 7 Express, Prentice Hall, 2004 [5] Salzmann, C; Gillet, D.; Huguenin, P.: Introduction to Real-time Control using LabVIEWTM with an Application to Distance Learning, Int. J. Eng. Educ., 16(3), 2000, pp. 255-272 [6] Webcam Corp., Webcam 1-2-3, (http://www.webcamsoft.com) [7] Microsoft, Windows Media technologies, (http://www.microsoft.com/windows/windowsmedia) [8] National Instruments, Academic Products and Resources, (http://www.ni.com/academic) [9] National Instruments, NI-IMAQ for USB Cameras User Guide, Austin (Texas), 2005 [10] National Instruments, LabVIEW Help, Austin (Texas), 2005 [11] 1394 Trade Association, IIDC 1394-based Digital Camera Specification, (http://www.1394ta.org) [12] National Instruments, NI-IMAQ for IEEE 1394 Cameras User Manual, Austin (Texas), 2005 [13] Unibrain, Fire-i™ Firewire Digital Camera, (http://www.unibrain.com)
5(6)
Conference REV2006
29/30 June 2006 Maribor, Slovenia
Author(s): Bojan Gergič, Ph.D. University of Maribor, Faculty of Electrical Engineering and Computer Science Smetanova ulica 17, 2000 Maribor, Slovenia Email:
[email protected] Darko Hercog, B.Sc. University of Maribor, Faculty of Electrical Engineering and Computer Science Smetanova ulica 17, 2000 Maribor, Slovenia Email:
[email protected]
6(6)
