Transforming MFPs into Ubiquitous Computing Devices Diego López-de-Ipiña, Juan Ignacio Vazquez and JosuKa Díaz Faculty of Engineering University of Deusto Avda. de las Universidades, 24 48007 Bilbao, SPAIN {dipina, ivazquez, josuka}@eside.deusto.es
Abstract Last generation multi-purpose printers or MFPs are very capable computational devices, often equipped with a built-in programming framework which eases its extensibility. However, they include so much functionality that often it is too hard to operate them. This paper describes two services which turn MFPs into Ubiquitous Computing devices, i.e. more intuitive, personalised, user-friendly, upgradeable and context-aware devices. For that, two novel interaction models between users and MFPs are proposed, namely the “show ringcode and click” and the “install once, upgrade and run forever” interaction models.
1. Introduction One of the main goals of Ubiquitous Computing/AmI is to add some degree of intelligence to everyday physical objects [1][4]. Thus, our everyday objects will be able to react and anticipate to our needs by executing or suggesting services proactively. Examples of such future intelligent objects are: • A teacup that changes colour depending on its contents temperature or sugar concentration in order to alert people with diabetes. • A television that switches on with the program most likely to be watched by its users, taking into consideration contextual attributes such as the day of week, who is actually sitting in front of the TV and their preferences and program watching history patterns. Over the last few years, a type of electronic device which has experienced an interesting progression is the enterprise multi-functional printer or MFP. These devices are now not only capable of printing, scanning and faxing, but also they do
these things at a cheaper price than ever, they offer excellent interaction tactile colour screens, and more importantly, they even include a programming framework that enables extending their default functionality. Currently, some printer manufactures try to differentiate from competition by not only selling the printing hardware but also adding some software services running on the frameworks provided which are aimed to ease their customers’ main business processes. Unfortunately, configuring complex jobs on an MFP printer is usually a tedious task, since first the functionality sought has to be found and then several button clicks are required in order to complete the job configuration and finally instruct the printer to start. In order to alleviate such situation the RingCode2Job service is proposed. This service turns an MFP into a more intuitive and user-friendly device capable of undertaking user provided complex jobs by a single click on the printer’s start button. For that, RingCode2Job adds to an MFP the capability of recognising circular barcodes or ringcodes (from now on) which represent jobs previously configured by users. On the other hand, MFPs present ever improving input capabilities (scanning, tactile colour screens), even better output mechanisms (colour printing) and they are extendable (include a JVM where organisations’ business processes can be implemented). Moreover, such devices are often located in strategic locations within an office. Thus, MFPs seem ideal candidates to act as control points for the devices and services deployed within an organisation. The EnterpriseController service proposed aims to address this. The rest of this paper is divided as follows. Section 2 overviews some related work on attempts to add intelligence to everyday objects and printers in particular. Section 3 explains the RingCode2Job service. Section 4 analyses the
Enterprise Controller service. Finally, section 5 offers some conclusions and discusses further work.
2. Related Work One of the major current technological trends on Ubiquitous and Pervasive Computing research is to embed sensing, communication, computation and actuation in physical artefacts leading to the creation of smart objects [1][4][11]. Current industrial and academic research indicates that smart objects might soon become an integral part of our daily lives with applications in economically important areas [10] such as retail, supply chain management, asset management, and safety critical situations in work places and healthcare. Smart objects will be an important building block to bridge the gap between the physical and digital world [6] by providing information about aspects of their physical environment. While systems of smart objects clearly will need to build on emerging technologies such as RFID [9] and wireless sensor networks [13], the envisioned ubiquity of smart objects raises important questions still to be resolved. Some of those unresolved research issues are: the digital representation of physical artefacts, their cooperation paradigms, integration into backend infrastructures or the applications that will benefit and influence their design and development. Lately, the leading MFP manufacturers have released platforms that enable the core default functionality of a printer to be extended. Examples of such platforms are Canon’s Multifunctional Embedded Application Platform (MEAP) [2], Ricoh’s Embedded Software Architecture (ESA) [8] or Xerox’s Extensible Interface Platform (EIP) [14]. Such application development frameworks enable the creation of customized embedded applications that run on MFPs. Most of them are based on the Java ME technology and provide packages to control most aspects of a device operation, including the control panel GUI, network communication with servers and core capabilities such as printing, scanning, faxing and saving files. The applications developed in these embedded platforms, termed Xlets [12] and presenting a similar execution model to MIDlets in Java ME, usually connect to an organisation’s back-end
system in order to exchange data. Examples of possible applications are: customised workflow applications for vertical markets such as retailing, direct printing of company’s news or financial reports from the printer, or MFP integration with back-end systems such as CRMs, ERPs or databases. The OSGi [7] specification is currently being used in many architectures for pervasive computing, since it tackles the discovery, control and management of ubiquitous services [3]. In fact, even some printer embedded platforms include an OSGi server, such as the Ricoh Aficio series. The main reasons that suggest OSGI as an ideal infrastructure for Ubiquitous Conputing are: a) it implements a complete and dynamic SOAbased component model within a JVM, b) it allows components (bundles) remote lifecycle management without reboot, c) it provides a service registry to detect new and disappearing bundles and it d) fosters cooperation among bundles to accomplish functionality, i.e. it encourages code reuse. All those reasons justify our selection of OSGi for implementing the ubiquitous services proposed. The work described in this paper contributes towards adding intelligence to MFPs, in contrast with previous work, in the following aspects: • The interaction between users and MFPs should be simplified making these devices job- and user-aware. Job-aware so that the user interactions with the printer in order to initiate a job will be minimised. User-aware so that the functionality offered to a user will be different depending on their job profile and personal preferences. • MFPs can be configured as suitable control hubs for the dynamic devices and services provided within an Intelligent Environment, in contrast with past work where printers were used only to interact with business processes and other devices (mobile devices and PCs) were considered more suitable for service discovery and control.
3. The RingCode2Job Service Multi-functional printers (e.g. Ricoh MFP Aficio 3025) are very capable: they print, copy, send fax, include an application server, they are network connected including HTTP and SMTP servers and so on. However, interacting with them and
accessing this powerful functionality is cumbersome. It is difficult to find the functionality sought and it does require many clicks. RingCode2Job aims to solve this by configuring complex jobs on MFPs in a simpler manner. The core idea is to simplify user interaction with an MFP by means of a “show ringcode and click” paradigm. This is, whenever a ringcode identifying a job is recognised by a MFP, start the execution of such job. 3.1. TRIP Ringcodes RingCode2Job makes use of the TRIP [5] ringcode-based identification system. A TRIPcode, see Figure 1, is a circular ringcode representing a ternary number, encoded within the 16 sectors of its two encoding rings. The most remarkable feature of TRIP is that it makes use of conventional and low-cost computing-wise image processing algorithms (adaptive thresholding, binary image edge detection, edge following or ellipse fitting) to obtain the ID encoded within a TRIPcode.
organisation, whose reutilisation could bring about interesting time savings. 3.3. Service Architecture The RingCode2Job system follows a client/server architecture, as shown in Figure 2, composed of two main components: the RingCode2Job Xlet and the RingCode2Job Server. The RingCode2Job Xlet, developed through Ricoh’s ESA Xlet SDK and installed on a Ricoh MFP Aficio 3025 fulfils a two-fold purpose: a) provide a user interface for user interaction and b) communicate with the RingCode2Job server. The RingCode2Job Server is a Java EE application developed with the Struts framework, the postscript generator and translator Ghostview and the Hibernate serialization framework over a MySQL database. It runs on a Tomcat servlet container, located in the MFPs owner’s LAN, being its main duties: • Translate images containing TRIP ringcodes into JobXML files which are returned to the printer. • Allow the configuration of complex jobs through a simple Web 2.0 interface. • Maintain and manage a pre-configured job repository.
Figure 2. RingCode2Job Architecture Figure 1. TRIP ringcode
3.2. An MFP Job An MFP job is a macro including the combination of several common printer tasks: scanning, photocopying, sending email, stapling and so on, with their corresponding attributes. These macros are tasks repeated constantly within an
3.4. Implementation Details The main technical contribution of RingCode2Job is to make a printer job-aware, i.e. to translate scanned TRIP ringcodes images sent to the server into job descriptions in the form of JobXML files. Figure 3 shows an example of such XML files returned to the RingCode2Job’s Xlet. The Xlet uses the contents of those files to apply the tasks composing a job to the paper pages
that follow the one including the TRIPcode. For example, the XML of Figure 3 indicates that the first two pages will be scanned, pages 2 to 4 will be printed out 3 times each and pages 5 and 6 will be scanned and sent as attachment to dipina@ deusto.es 130.206.137.153 admin 0000 Ricoh 1 2 3 2 4
[email protected] 5 6 Figure 3. JobXML example
3.5. Show RingCode and Click Paradigm The following five stages are required to set up and run a job with the help of RingCode2Job: 1. Job definition. A job is defined through the web interface shown in Figure 4, which enables assigning N tasks (scanning, printing, sending email and so on) to a job. 2. Generation and printout of a RingCode. A ringcode, i.e. a unique ID, is assigned to the newly defined job. The ringcode ID serves as a key in the DBMS hosting the Jobs table. 3. Preparation of paper stack with interleaved ringcodes. A ringcode must precede the pile of papers on the printer’s tray over which a job will be applied. Jobs may be concatenated by interleaving ringcodes.
4.
5.
Xlet execution. Once the Xlet has been selected on the printer, the user can start its execution by simply clicking on the start button, see Figure 5. When the Xlet is run for first time the server IP address must be configured. End of job notification. The job creator is notified by email when the job execution has concluded.
Figure 4. RingCode2Job configuration website
The first two stages are for job configuration whilst the last three ones which will be repeated continuously conforming the “show ringcode & click” user to printer interaction model proposed.
Figure 5. RingCode2Job Xlet.
3.6. RingCode2Job applicability RingCode2Job is ideal for companies which undertake complex and repetitive MFP-related tasks. Its main advantages are: • Provides a simple web interface to configure jobs • Defines a new “show ringCode and click” paradigm between users and MFPs • Improves usability/accessibility of an MFP • Allows for “job macro” reutilisation and en of job alerting which leads to better document management. The ideal target users of RingCodeJob are: public and academic institutions, service companies such as law firms, banks or printing centres to provide a la carte printing services. In conclusion, adding sentient features to an MFP through RingCode2Job, i.e. making it jobaware, turns a printer into a much more powerful, user-friendly and intuitive device.
4. The EnterpriseController Service EnterpriseController is a service which aims to transform an MFP into a personalised and extensible Universal Remote Controller of Ubiquitous Services and Devices. It personalises an MFP since users are offered a menu with only their most commonly used services and which are accessible according to their job profile and personal preferences. It makes an MFP extensible adapting it to the dynamic nature of an organisation, coping with the emergence of new services and enabling the secure authenticated control and consumption of “any” service by “any” user. Figure 6 shows the EnterpriseController concept, where a set of ubiquitous and enterprise services can be consumed and managed from a printer. Its main objectives are: • Provide user/profile-based service personalisation in an MFP. • Improve accessibility to enterprise services (business and environment ones). • Ensure future-proof extensibility features. • Consolidate MFP as a multi-purpose core asset in an enterprise
Figure 6. EnterpriseController concept.
4.1. EnterpriseController Applets and Player An EnterpriseController Applet is a two-fold component split into front-end and back-end. The front-end is the GUI proxy of an enterprise service hosted on the server and downloadable by the EnterpriseController Xlet Players installed on MFPs. The EnterpriseController Xlet Player is a component capable of downloading, verifying, installing and running applet front-ends on demand. The most remarkable feature of the EnterpriseController service is the adoption of a player/applet model where new functionality can dynamically be downloaded or old functionality upgraded by the player Xlet. Thus, EnterpriseController enhanced MFPs are turned into extensible future-proof systems accommodating to any enterprise dynamics. 4.2. EnterpriseController Architecture Figure 7 shows the architecture of the EnterpriseController system composed of the following three elements: • EnterpriseController Player Xlet. This is the user front-end to the EnterpriseController service ecosystem, providing the following functionality: o Login control by username/password input or using a fingerprint reader attached to the printer. Fingerprint recognition is carried out by means of a
•
•
Nitgen Fingkey Hamster and its C SDK, linked to the Player Xlet though JNI. o User job profile and preference-based service front-end menu retrieval and rendering. Every user has a preconfigured service menu automatically assigned according to their job profile, but still customizable through a menu configuration web front-end. The user menu downloaded in the form of an XML file enables the Player Xlet to figure out whether new functionality or upgrades to old functionality need to be downloaded and installed in the printer from the EnterpriseController server. o Handling of user interaction with active enterprise front-ends which communicate with the services hosted at the EnterpriseController server. EnterpriseController Server. It is an OSGibased server hosting and managing enterprise services, which acts as a bridge between the MFP and the distributed heterogeneous enterprise services and devices within an organisation. It serves as a repository for applet front-ends downloadable to MFPs and as an execution environment for applet backends. EnterpriseController Web Front-end. It offers a user friendly web interface for: a) user registration with the system, b) users’ main menu panel generation taking into consideration a users’ job profile access restrictions and enabling personalization according to user’s service access rights, and c) management of applets uploaded into the server, e.g. configuration of access rights assigned to them, job profiles which may used them and so on.
Figure 7. EnterpriseController Architecture
dipina http:///ECServer houseA.bmp 200 160 150 80 ... Figure 8. EC XML configuration for dipina
4.3. Implementation details Similarly to the case of RingCode2Job, this service also uses an XML for configuration purposes, see Figure 8. In this case, an XML file is returned from the EnterpriseController server to the EnterpriseController Xlet once a user logs in. Such XML file contains the personalized menu for the logged in user, including configuration details for the services he has access to according to his job profile. Notably, this file informs about the current set of applet front-ends which should be available for a logged in user in the MFP. The Player Xlet analyses the list of services in the XML file to find out whether new applet front-end downloads have to take place. In order to enable the verification and execution of downloaded applets, the EnterpriseController Player Xlet makes extensive use of Java Reflection. For that, every EnterpriseController applet must implement the es.deusto.enterprisecontroller.Fr ontEnd interface which provides a run()
method. This method is invoked by the Player Xlet to display the applet GUI front-end on the printer and thus allow the user interaction with the remotely available enterprise or device services, represented by the chosen front-end. The EC player has been implemented on an Aficio MP C2500 model. The EC server makes use of the Knopflerfish OSGi implementation.
•
GoogleMaps, assembles them into a PDF and prints it out. User contacts applet. It allows an MFP user to search for contact details though keyboard input or retrieve a contact’s details by recognising a barcode printed on a business card. Once a contact’s details have been retrieved, a scanned document can be sent to that contact or print out his business card.
4.4. EnterpriseController Applets As has been previously mentioned an applet is divided in two parts, making their development a two stage process: • Development of an applet back-end. This implies the creation of an OSGi bundle which knows how to communicate with the enterprise (e.g. payroll system) or environment (e.g. lighting system) services it represents. • Development of an applet front-end. This requires creating a class which implements the FrontEnd interface, requirement of every applet downloaded, verified and run by the EC player. Four applets, two environmental and two enterprise ones have been implemented, which illustrate the potential of this service. The two environmental services are: • Enterprise Lighting Control Applet. It enables a user to control the lighting system in an office through an applet OSGi back-end that uses X10 protocol to send commands over power line to switch on/off and dim lights. Figure 9 illustrates the operation of this applet. • Surveillance Control Applet. It enables a user to control from the printer a set of D-Link DCS 5300G IP cameras distributed through our research lab, see Figure 10. The back-end bundle issues HTTP GET requests to the cameras to move them and capture images from them. On the other hand, the enterprise services provided are: • GoogleMap Retrieval and Printout Applet. It calculates the latitude and longitude corresponding to a scanned post code in a business card, retrieves the map images corresponding to such coordinates from
Figure 9. X10 light controlling applet.
Figure 10. IP cam controlling applet.
5. Conclusion and Further Work This paper has described our efforts to transform MFPs into more intelligent devices by following two new user to printer interaction models: a) “show ringcode and click” through the RingCode2Job service and b) “install once, run and upgrade continuously” through the EnterpriseController service. The RingCode2Job service enhances a conventional MFP by making it job-aware, i.e. more user-friendly and intuitive, only requiring a job identifying TRIPcode to be placed in front of a document on the printer’s tray and then click the start button to apply the job to the document. The EnterpriseController service enhances an MFP by making it user-aware and extensible. A
fingerprint recognizer is added to an MFP which allows user identification and provision of personalised services to them. Moreover, the functionality offered to a given user can be continuously modified thanks to the applet/player design model adopted. Thus, EnterpriseController allows an MFP to accommodate to new emerging services or updates to current available services within an organisation. Besides, through EnterpriseController an MFP can be turned into a prime controlling element within an Intelligent Environment, as shown by the lighting and surveillance EnterpriseController applets described. As an extension of this work we are currently working on a BundleDiscoverer service to be integrated on the EnterpriseController Player Xlet so that users can discover newly available services from the printer without their explicit and manual configuration of a new services menu aided by the EnterpriseController Web front-end.
Acknowledgments We thank the excellent implementation work carried out by the students Mikel Fernández de Matauko and Irantzu Marquínez for RingCode2Job and to Álvaro Pérez for the EnterpriseController. We also thank Ricoh Spain for the provision of the Ricoh MFP Aficio 3025 and Ricoh Aficio MP C2500 models.
References [1] Beigl M., Gellersen H.-W., and Schmidt A., Mediacups: Experience with design and use of computer-augmented everyday artifacts, Computer Networks, vol. 35, no. 4: pp. 401– 409, 2001 [2] Canon Develper Support, MEAP Information Page, http://www.developersupport.canon.com/mea p.htm, May 2007 [3] Choonhwa Lee; Nordstedt, D.; Helal, S., Enabling smart spaces with OSGi, Pervasive Computing, IEEEm vol. 2, no. 3, pp. 89 – 94, July-Sept. 2003 [4] Kranz, M., Schmidt A. Prototyping Smart Objects for Ubiquitous Computing, International Workshop on Smart Object Systems, 7th International Conference on
Ubiquitous Computing (UbiComp 2005), September 2005 [5] López de Ipiña, D., Mendonça P. and Hopper A. TRIP: a Low-cost Vision-based Location System for Ubiquitous Computing, Personal and Ubiquitous Computing, vol. 6, no. 3, pp. 206-219, May 2002. [6] López-de-Ipiña D., Vázquez J.I., et al., EMI2lets: a Reflective Framework for EnablingAmI, Journal of Universal Computer Science (J.UCS), Vol. 12, No. 3, (2006), 297314 [7] OSGi Alliance, OSGi – The Dynamic Module System for Java, http://www.osgi.org/, May 2007. [8] Ricoh, Ricoch Embedded Software Architecture, http://www.ricoh.com/esa/overview.html, May 2007 [9] Römer K., Schoch T., Mattern F., Dübendorfer T. Smart Identification Frameworks for Ubiquitous Computing Applications. Wireless Networks, Vol. 10, No. 6, pp. 689-700, December 2004 [10] Strassner M., Schoch T., Today's Impact of Ubiquitous Computing on Business Processes, Proceedings International Conference on Pervasive Computing. pp. 62-74, Pervasive2002, April 2002 [11] Vanhala, J., Mäyrä F. and Koskinen, I. (2005), Living in Metamorphosis: Proactive Computing in the Home Environment. ACM Interactions Vol XII.4, July-August 2005, pp. 28-31. [12] Wang X.,Xlet: A Different Kind of Applet for J2ME, Java Developer Journal, SYS-CON Media, http://jdj.sys-con.com/read/37615.htm, July 2003 [13] Want R., Sensor-Driven Computing Comes of Age, IEEE Pervasive Computing, vol. 6, no. 2, pp. 4-6, April-June 2007 [14] Xerox, Xerox Extensible Interface Platform, http://www.office.xerox.com/eip/esar.html, May 2007
