Interactive Projector: Application of MATLAB ...

Current Trends in Signal Processing Volume 3, Issue 1, ISSN: 2277-6176 __________________________________________________________________________________________

Interactive Projector: Application of MATLAB Embedded JAVA Yashpal Javia*, Dhaval Gondaliya, Kenil Desai, Darshan Pandya, Dhruvik Monpara, Prof. A. M. Kothari ATMIYA Institute of Technology and Science, Rajkot, India

Abstract In this paper, the authors have worked on a technique to convert a simple projector into an interactive projector by using a LASER pointer. For this, the authors specially concentrated on transformation, namely projective transformation, for calibration purpose, feature extraction of a color image for locating a pointer and java commands to perform mouse operations. The authors manipulated an interested part of a captured image and matched its coordinates with screen resolution. To locate a pointer, the authors decided different threshold values for intensity of color, which can be obtained by subtracting gray scale of original color image from interested color image. After finding pointer location, mouse operations are performed accordingly.

Keywords: Projective transform, pointer detection, centroid, digital image processing, embedded Java commands in MATLAB. *Author for Correspondence E-mail: [email protected]

INTRODUCTION Interactive projector is a projector with additional features, which allows the user to interact with the projector without use of a mouse. These projectors allow one to interact with one’s projected lesson plan from practically anywhere in the seminar hall. By using a LASER pointer, one can do mouse operations like click, double click events from many feet away from the screen. The need for an interactive projector arises because one has to stick with the mouse and the keyboard during presentations in a classroom or an auditorium. There are many interactive projectors available in the market; here the authors introduce one of the cost effective way to transform a simple projector into an interactive projector. This paper focuses on calibration process, color detection and embedded java commands. Firstly, the authors are performing a transformation process on captured image namely projective transformation. The second step is to detect the color pointer on the projector screen, which is performed by the digital image processing. In the last step, mouse operations take place with the help of

java commands embedded into MATLAB. This paper is organized in six main sections. Projective Transformation explains the calibration process. Elaborate color detection and mouse operations are explained under the heads Color Detection and Embedded Java Command respectively. The next section explains the proposed method and the head Conclusions gives the results of the proposed method.

PROJECTIVE TRANSFORMATION In mathematics, a projective transformation is a function that maps one vector space into another and it is often implemented by a matrix. A mapping is considered to be a projective transformation if it preserves vector addition and scalar multiplication. To apply a projective transformation on a vector (i.e., coordinates of one point, in the pesent case, x and y values of a pixel), it is necessary to multiply this vector by a matrix which represents the projective transform. As an output, one will get a vector with transformed coordinates [1]. Use of projective transformation is required when screen appears tilted. Straight lines remain straight, but parallel lines converge toward vanishing

CTSP (2013) 1-11 © STM Journals 2013. All Rights Reserved

Page 1

Interactive Projector Javia et al. __________________________________________________________________________________________

points that might or might not fall within the image [2]. The projective transform between two images is required for normalizing the image variation due to different angles of cameras [3]. Figure 1 gives an idea about this transformation.

Fig. 1: Projective Transform.

HSV color model represents image in better prospect than RGB color model because it separates luma, or the image intensity. One of the detection techniques is shown in Figure 2.

EMBEDED JAVA COMMAND The java class “java.awt.Robot” has ability to control mouse pointer position, motion and click events. Java class is used to create applications where mouse and keyboard control is needed. Java commands are also interfaced with MATLAB. These commands can control various internal operations of both mouse and keyboard. To operate any event related to mouse operation one needs to give a pixel value as input argument of these commands. Similarly, for events related to keyboard operation the user has to assign the particular key name. In “java.awt.Robot” command, one can control mouse operation by assigning the value of robot as mouse, similarly as one has to assign the value of robot as keyboard to use key operation [4, 5].

COLOR DETECTION Color detection process detects the required color from the captured image. In image processing color detection is the most important part. The interested color can be detected by many techniques like threshold boundary, HSV color space converting color image in gray scale, etc. Need for color detection is generated because in an image one has to work on particular colors. The color detection algorithm scans every pixel of the image and gives required color output according to the threshold value.

THE PROPOSED METHOD There are certain steps in the proposed method which are described as below. Calibration Process Calibration process is a kind of process, which compares and gives output with correctness. It is obvious that while one should acquire an image, it contains some portion other than projective screen, and it also gives tilted image of the screen. Therefore, every image capturing process this problem occurred. To convert a simple projector into an interactive projector the projecting screen has to be coordinated with the device screen. To perform calibration in this method, one first generates an image of the same size as screen resolution. That image is projected on a projector screen and captured by a camera and taken into the computer. In that captured image, an algorithm identifies that image and calculates the interested part from the acquired image. This procedure helps us to detect projector screen.

Fig. 2: Red Color Recognition.

The starting pixel of the projective screen is located on the captured image. This detection helps us to generate a transformation matrix. As discussed earlier, this transformation

JoPC (2013) 1-11 © STM Journals 2013. All Rights Reserved

Page 2

Current Trends in Signal Processing Volume 3, Issue 1, ISSN: 2277-6176 __________________________________________________________________________________________

matrix could be generated by projective transformation. To manipulate a complete calibrated image, one needs to multiply this transformation matrix with the detected pixel coordinates of the captured image. As a result of all these processes, one receives a perfect rectangular image of the interested part and from that one can correlate the screen resolution with the interested portion. The whole calibration process is described in Figure 3 [6].

for better detection. For that, one scan each image for both color detections of laser pointers. Both color detections contain the same procedure to be followed; only the threshold value differs. This must be accurate because on this value various mouse operations have to be performed. For detection of the interested color, many procedures are available. Nevertheless, for accurate recognition, the authors developed a new technique for color detection. For that, first of all one converts the color image, i.e., the RGB image into a gray scale image and stores that matrix-formed image. Now to detect red pointer on projective screen, take a gray scale of red plane, store it, and then subtract previous gray scale image from it. This way, one can detect red color in an efficient way (Figure 4). Now for second color laser pointer detection, subtract that gray scale image of color image from the interested color’s gray scale image.

Fig. 3: Calibration Process. Calibration process needs to be performed only one time when the camera is set up at one position; whenever the camera is moved, calibration needs to be performed again. Pointer Detection Algorithm This part of the proposed method is actually the heart of the whole paper as the authors’ main aim is to control all computer operations through a laser pointer, located on a projector screen. Thus, to control all mouse operations according to laser movement one needs to detect the laser pointer accurately. This process relates to this task. To do all mouse operations like left click and double click, two color laser pointers are taken

Fig. 4: Red Pointer Detection. In this way, one can obtain pointer’s accurate location, which can help us to perform mouse operations. Whenever the color of laser pointer changes for a while and aging set as original color, one detects as a click and similarly all other events taken place [7–11]. As mentioned in the above paragraph, according to various color changes in laser

CTSP (2013) 1-11 © STM Journals 2013. All Rights Reserved

Page 3

Interactive Projector Javia et al. __________________________________________________________________________________________

pointer various mouse operations are performed but there is a problem that is faced. The user’s hand is continuously fluctuating so it is not possible to relocate the color changes of laser pointer at the same pixel location. As a solution, pixel location is stored at the end of each frame and whenever color change is detected, it performs the related mouse events on that last stored location. Controlling Mouse Operation After the identification of laser pointer by the above-mentioned algorithm, the next process to be followed is mouse operations. Now, in pointer detection one cannot obtain only one pixel value though the algorithm was good in accuracy because laser pointer is so much powerful, so one gets group of pixels in this detection. However, to perform mouse operations one needs to give a particular pixel’s coordinates. To find out only one pixel value, one finds centroid of that group of pixels and according to that one is able to perform various mouse operations easily. After finding out centroid, multiply it with already calculated transformation matrix to correlate this pixel location with screen resolution. According to that location, the mouse has to be moved in every frame of the video. Whenever color of the laser pointer changes for several frames, it is denoted as a click event, and on stored location one performs according to events of the mouse. Now if that color change occurs for more number of frames that one is deciding then one takes that event as a double click on that location. In a similar way, one can act upon different mouse operations. All these mouse operations can be handled by the above described command “java.awt.Robot,” which is embedded inside the software. For various operations, the command differs accordingly.

RESULT OF THE PROPOSED ALGORITHM From this proposed methodology, one is able to perform every mouse operation due to better recognition of laser pointer. As one performs calibration process, it allows one to place a camera anywhere in the projector room that

can cover the projector screen. In addition, the surrounding area can affect detection process. However, accuracy of this algorithm can be very high when laser is pointed in front of the projector screen with a viewing angle of 45º both sides (Table 1).. Table 1: Efficiency with Viewing Angle. Interested region angle

> 90 %

80– 90%

70– 80%

50– 70%

0º 15º 30º 45º

96% 94% 91% 88%

93% 90% 88% 84%

91% 88% 83% 79%

87% 86% 78% 75%

Resolution of the camera can also affect the process. With the increase in the resolution of the camera the result is better. However, the only drawback of increasing resolution is that it takes higher time for calibration process and color detection.

CONCLUSIONS From the work carried out, the authors conclude that the proposed algorithm was very affective and accurate for turning a simple projector into an interactive one. The nearby environment also affects this algorithm because in much brighter environment, detection was not that much accurate as in a dark area. In this method, the authors perform calibration process, which can provide mobility to a camera. One can put the camera at any place at any viewing angle that can cover the projector screen. Resolution of the camera also affects the speed of the algorithm as resolution becomes high speech going to decrease and vice-versa.

REFERENCES 1. Cotting D, Naef M, Gross M, et al. Embedding Imperceptible Patterns into Projected Images for Simultaneous Acquisition and Display. Third IEEE and ACM International Symposium on Mixed and Augmented Reality. 02-05 November 2004; 100-109p. 2. Johnny Chung Lee. Projector-Based Location Discovery and Tracking. Human Computer Interaction Institute School of Computer Science, Carnegie Mellon University, Pittsburgh: May 2008; PA 15213.

JoPC (2013) 1-11 © STM Journals 2013. All Rights Reserved

Page 4

Current Trends in Signal Processing Volume 3, Issue 1, ISSN: 2277-6176 __________________________________________________________________________________________

3. http://www.umiacs.umd.edu/~ramani/cmsc8 28d/ProjectiveGeometry.pdf 4. docs.oracle.com/javase/6/docs/api/java/awt/ Robot.html 5. docs.oracle.com/javase/7/docs/api/java/awt/ event/KeyEvent.html 6. Affine and Projective Transformations: http://www.aurigma.com/docs/gm/Transfor mations.htm 7. Web pagehttp://en.wikipedia.org/wiki/YCbC r 8. http://homepages.inf.ed.ac.uk/rbf/HIPR2/ affine.htm 9. Rafael C. Gonzalez, Richard E. Woods, Steven L. Eddins. Digital Image Processing Using MATLAB. Pearson Education, Inc. 10. Paul Beardsley, Ramesh Raskar, Clifton Forlines, et al. Interactive Projection. Mitsubishi Electric Research Laboratories. TR2004-107. January 2005. 11. http://homepages.inf.ed.ac.uk/rbf/CVonline/ LOCAL_COPIES/BEARDSLEY/node3.html AUTHORS Mr. Yashpal V. Javia is studying in final year of bachlor degree in Electronics & Communication Engineering at ATMIYA institute of technology and science, rajkot, gujrat, india under Gujarat Technological University(GTU), gandhinagar, gujarat, india. His area of interest is embedded systems along with signal processing. Mr. Kenil J. Desai is studying in final year of bachlor degree in Electronics & Communication Engineering at ATMIYA institute of technology and science, rajkot, gujrat, india under Gujarat Technological University(GTU), gandhinagar, gujarat, india. His area of interest is communication networking and signal processing. Mr. Dhaval C. Gondaliya is studying in final year of bachlor degree in Electronics & Communication Engineering at ATMIYA institute of technology and science, rajkot, gujrat, india under Gujarat Technological University(GTU), gandhinagar, gujarat, india. His area of interest is embedded systems and VLSI along with signal processing.

Mr. Dhruvik K. Monpara is studying in final year of bachlor degree in Electronics & Communication Engineering at ATMIYA institute of technology andscience, rajkot, gujrat, india under Gujarat Technological University(GTU), gandhinagar, gujarat, india. His area of interest is embedded systems and VLSI along with signal processing. Mr. Darshan N. Pandya is pursuing in final year of bachlor degree in Electronics & Communication Engineering at ATMIYA institute of technology and science, rajkot, gujrat, india under Gujarat Technological University(GTU), gandhinagar, gujarat, india. His area of interest is to develop signal processing applications, microcontrollers and embedded systems. Mr. Ashish M. Kothari has completed his bachlor and masters degree in the Electronics & Communication Engineering from saurashtra university, rajkot, gujarat, india and he is currenly pursuing his Ph. D. in the same discipline from shree jagdishprasad jhambarmal tibrewal university, jhunjhunu, rajsthan, india. His Area of reaserch is to design a robust method for the video watermarking. He is currently working as an assistant professor in the department of electronics & communication engineering at Atmiya Institute of technology & science, rajkot, gujarat, india. He is also appointed as a director of the UDISHA club by Gujarat Technological University for the rajkot-veraval sankul 1. He is also appointed as PG coordinator for the masters program in EC in the department of Electronics & Communication at Atmiya Institute of technology & science, rajkot, gujarat, india. He has guided more than 20 UG students in their project work and more than 7 students in their dissertation work at PG level. He has presented 12 national and international papers and published 4 international papers. He has published a book titled real time anlysis of digital image watermarking with Lambart Publishing house, Germany.His area of interest is Image and video processing, microcontrollers and embedded systems.

CTSP (2013) 1-11 © STM Journals 2013. All Rights Reserved

Page 5