Integrating constraint-based technology with multi-touch table Caroline Sabty and Maged Zamzam Faculty of Media Engineering and Technology, German University in Cairo
[email protected],
[email protected] http://met.guc.edu.eg Abstract. Multi-touch systems are evolving technology which aims at multi-user usage of some applications. In this paper, the integration of multi-touch systems with the power of constraint-based platforms are discussed. Two prototypes are developed to illustrate the approach. This integration is inter-operable which can work on any constraint-based algorithm.
1
Introduction
Touch screen technology is an arising visual display technology that detects the presence and location of a touch within the displaying area. Multi-touch systems are hardware devices which enhance touch screen technology, by providing the user the ability to apply multiple finger gestures to send commands to the device [2]. The strength of multi-touch applications lie in the visualization of a problem along with collaborative user interaction. Most of the existing applications offer a possibility for multiple users to work together in an interactive manner. The problem with most of these applications does not react on the triggers of the user. In this work, we will discuss the integration of multi-touch environment with constraint based applications by providing the user a first proposal of a solution to a problem. The user has the possibility to interact with the interface to adapt this solution according to his needs while satisfying the underlying constraints of the problem. We will introduce two constraint layout prototypes developed on multi-touch environment and solved using constraint technology. Layout problems aim at placing some items in specifc space. The first application is concerned with the placement of furniture (sofa, armchair,...etc) in a specific room. The second application is concerned with placing journal items (ads, news, articles, columnar articles and headlines) into a page of a newspaper. The two prototypes compute a layout according to some constraints specified by the user like the wasted space or ordering of items according to their weight.
2
Problem Description
The aim of the work is to integrate constraint based application implemented using ECLi PSe platform with multi-touch systems, because constraint based
application does not support graphical user interface on the multi-touch table on their own. Moreover, embedding the power of constraint technology in solving problems with the power of multi-touch in visualization. The problem arising is that the applications are implemented in different programming languages, thus they do not understand the operation of each other because they do not have a direct communication channel. Thus, we need to reflect the solution generated from ECLi PSe as an input to the multi-touch. To generate a solution in layout problems, the items are placed according to some requirements specifed by the customer. These requirements are some restrictions to the position of these items in order to propose a layout. The two prototypes show the different rules of layout problems as follows. Intelligent home design: The furniture layout problem can be applied to different types of rooms (e.g. living room, bedroom, etc...). In this prototype we focus on the living room and a set of objects that could be placed in it. The proposed layout must be optimal in term of minimizing the waste of space. This is done by minimizing the distance between the walls and some of the objects in the room. We also have to give the user the ability to edit the proposed layout by moving any object. Newspaper lay-outing problem: The newspaper layout problem is placing of i items (e.g. ads, news, articles, columnar articles and headlines) in a m×n container in order to maximize the number of items and placing them according to a cost variable (priority of item). Ordering the items and maximizing them in each page is done by minimizing the cost of the solution proposed. The cost is determined by the positions of the items and the priority of each of them.
3
Implementation
In order to integrate the two applications, the integrated system is composed of three components as shown in figure 1, the first component according to the enduser prospective is the multi-touch application which is implemented in PyMT 1 , this component is responsible for selecting the items for generating and manipulating the solution. The second component is the connection between PyMT and the constraint solver and it is implemented in Java programming language. The third component is the constraint logic programming algorithm implemented using ECLi PSe programming language. This architecture is interoperable, meaning that it is valid for any constraint logic programming algorithm. Python and ECLi PSe uses different protocols in sending commands to each other (Python uses XML and ECLi PSe uses EXDR message interchanging protocols), and can not directly send the list of selected items to ECLi PSe in order to render the generated solution. An intermediate language should be implemented which understands both protocols to overcome the problem of different protocols, java was used as it has Apache XML-RPC libraries for sending/receiving data with python, also it 1
PyMT is a python open-source framework for developing multi-touch applications
Fig. 1. Connection between multi-touch and constraints-based platform
has ”parctechnologies” libraries for querying ECLi PSe and getting the solution back. Remote procedure call (RPC) is the mechanism (or the communication channel) which is used for interchanging data between applications on same machine or on different ones [4]. The RPC classiffes the interface component and the constraint-based technology component into two sides; the server side and the client side. The client side (PyMT) specifes the procedure name and parameters in the XML request to send it to the server. The server side (ECLi PSe ) executes the called procedure and returns either the XML response that contains the result or a fault to the client. [6] To execute a query in ECLi PSe we invoke the rpc(String query) method, where we specify the name of the predicate and its parameters. To represent ECLi PSe data types we construct a CompoundTerm. A CompoundTerm is the java data type where the result of the query must be saved in. The solution returned is composed of a collection of CompoundTerm objects each representing a result. These fields of the result are accessed using arg method. This method takes an integer representing the position of the output argument of the query and returns an Object which is the Java representation of the ECLi PSe argument at that position. It is then represented as a string, in order to be sent in the response message to the client side. The main component of PyMT is Widget, it is anything that can handle input events or can be drawn. The developer can also build his own widget. Multi-touch events can be added to the widgets, there are many events available (e.g. on touch down, on touch move) [7] The interfaces of the two prototypes intelligent home design and newspaper lay-outing problem on a multi-touch table are as shown in the following figures
Fig. 2. Java code of the RPC server
Fig. 3. Prototypes on the multi-touch system
References 1. PC Magazine Encyclopedia, (June 2010). From website: http:/www.pcmag.com/ 2. Shneiderman, B. Touch screens now offer compelling uses. IEEE Software, 8, 2, (March 1991) 93-94, 107. 3. Rina, D. (2003). Constraint Processing, P.481. 4. Phillip, M; et al. (2005). XML Remote Procedure call, P.32. 5. Krzysztof, R. A; Mark, W (2007). Constraint Logic Programming using ECLiPSe, P.349. 6. Web services, (February 2010). From website: http:/www.w3schools.com/webservices/default.asp . 7. Alex, T; et al. (2009). Multitouch technologies, P.100.
This article was processed using the LATEX macro package with LLNCS style
